They don’t go to the gym, they don’t have the energy; the job shapes you. More or less the same for the farmers in the family.
Perhaps this was less so in the industrial era because of poor nutrition (source: Bill Bryson, hopefully well researched). Hunter gatherer cultures that we still study today have tremendous fitness (Daniel Lieberman).