This idiom is generally used to refer to manual labour, but can also express any type of difficult or exhausting work. For example, if a job or task requires a lot of running around, or is simply mentally draining, it will still have a physical effect on a person, even if it is just to feel extremely tired. For that reason, this expression can be used in a context that does not necessarily involve lifting or moving heavy things. Phrases that share a similar meaning include 'work your fingers to the bone,' and 'do the donkey work.'