Nah it doesn't take anywhere near that amount of time to raise some produce. Maybe a half hour total per week, if that.
Some stuff is easier to grow, some is harder, but for example I barely did anything on my strawberry patch this year other than picking them when they got ripe. I'd say I spent maybe 15 minutes a month on the strawberries.
Now if you're trying to grow a lot of stuff then the time investment might increase but most backyard gardens don't take up that much time once you've planted everything.