A medieval peasant most likely had more leisure time than you. Having a local technological maxima doesn't mean "constant progress".
It's morning to evening back-breaking physical work, undernourishment as a matter of fact, and very high probability of death - some of the families we talked to had been 2/3 decimated by the age of 35, from traffic accidents, a host of diseases like cholera, snakes, you name it... There is no question when I'd rather live.
There was always work to done. If wasn't something that was pressing right now (fieldwork, baling hay), then it was needed to be done soon (fixing fences, equipment maintenance), or improving your infrastructure (building better winter pens for animals, better storage for hay/crops). Add to that without IC engines, all work has to be done using human or animal labor. A Speaking of animals, they don't take days off. If you have milk cows, that's 20 minutes a day (for each cow) of milking by hand. Think your commute sucks? Try walking everywhere you go.
Just go to an Amish community and you will quickly see that they spend more than 40 hours a week working, and most communities still take advantage of some technology (small tractors, tools and other items that are manufactured by the outside world).
There seems to be this idea among office workers that the M-F, 9-5 "grind" of cube-life are the most terrible conditions humans have ever lived through, and it shows a laughably sheltered viewpoint.