I'm not even going to go down the route that most people in the world didn't have grandparents with a huge garden - let's accept as a baseline that maybe there was a time in human history when every human had access to something like this.
Before the modern world, a single frost, blight, or death in the family, could have wiped out an entire harvest and everyone in a community would starve.
That's less likely today.
Result: Probably a net benefit for humanity overall. Worth it? Highly subjective but as a humnist, i think I have to say yes.