This is in no way a "if you're homeless just buy a house" argument, it's a "you can't have it both ways, pick a lane and stick to it" argument.
You want to unilaterally decide that you don't want to pay much tax on income, billionaires decide that they don't want to pay much tax on capital gains, yet both of you want to continue living in a society where you can buy cheap bread baked from flour milled from wheat grown on subsidized farms, heavily reliant on public infrastructure, and you want to drink clean water and drive on public roads, all of which is paid for through taxes that you want to opt out of, and somehow you don't see a problem with that?
You can't pick and choose parts of society that benefit you and opt out of your duties, that's not how society works. All of those parts that you don't see value in are essential to someone else.