I'm not arguing that the Enclosure Acts reduced the prosperity of society in aggregate, merely that one of their direct results was that "previously starving peasants moved into the cities" wasn't a free
choice on the part of the peasants in many cases. They may have even ended up better off in doing so, but that's different from whether it was voluntary. I don't think you need a Marxist analysis to come to the conclusion that the Enclosure Acts didn't have a neutral effect on the pace of urbanization; it's what you find in standard mainstream histories from folks like Spielvogel as well.
Mostly I don't think it's at all clear that "being poor in the city working in a factory" would actually be preferred over "being a peasant farmer" given conditions of the time, if it were truly a free choice. In the United States at around the same time, you find many people choosing to leave cities to become subsistence farmers on the frontiers, because the Homestead Acts made it a viable option. If the UK had taken the enclosed land and distributed it via a Homestead-Act style process, would British peasants have chosen that option?