If tax dollars were spent exactly how each individual tax payers wanted them to be, there would be little point in taxation. And I highly doubt that someone living in public housing who is able to live close to work would see those tax dollars being "wasted".
The right to own private property is more or less bought through taxes. If the deal becomes too unfair, you get Trump or Bernie. If the deal becomes untenable, you get guillotines.