You're ignoring the reductions/eliminations of other taxes. For example, replacing property taxes with LVT is the usual "first step", and would reduce costs for most homeowners (whose land is typically less valuable than the house sitting on it). Same with sales tax, or income tax, or any other non-LVT tax; non-LVT taxes are less efficient than LVT, so replacing them with LVT would maximize tax revenue while reducing individual tax burdens for all but the largest landowners (namely: landlords, be they residential or commercial/industrial).
> taxation is a taking of assets from its owners
The private ownership of land is itself theft from the commons, and is itself the product of state power and authority. A tax on the value of that land is therefore compensation for that theft, and is more than fair.
Modern society has gone on long enough letting land owners have their cake and eat it too. If land owners expect society to respect their exclusive claims over land, then it's on them to compensate society accordingly.