>Why would this be worse than the current situation of private actors accountable to no one controlling this technology? It's not like I can convince Zuckerberg to change his ways.

As long as Zuckerberg has no army forcing me, I'm fine with that. The issue would be whether he could breach contracts or get away with fraud. But if AI is sufficiently distributed, this is less likely to happen.

>At least with a democratic government I have means to try and build a coalition then enact change. The alternative requires having money and that seems like an inherently undemocratic system.

I don't think of democracy as a goal to be achieved. I'm OK with democracy in so far it leads to what I value.

The big problem with democracy is that most of the time it doesn't lead to rational choices, even when voters are rational. In markets, for instance, you have an incentive to be rational, and if you aren't, the market will tend to transfer resources from you to someone more rational.

No such mechanism exists in a democracy; I have no incentive to do research and think hard about my vote. It's going to be worth the same as the vote of someone who believes the Earth is flat anyway.

In economics, you aren't necessarily creating wealth just because your final output has value. The value of the final good or service has to be higher than the inputs for you to be creating wealth. I could take a functioning boat and scrap it, sell the scrap metal that has value. However, I destroyed wealth because the boat was worth more. Even if you are creating wealth, but the inputs have better uses and can create more wealth for the same cost, you're still paying in opportunity cost. So things are more complicated than that.

>governments generate wealth all the time. Public education, public healthcare, public research, public housing. > These are all programs that generate an enormous amount of wealth and allow citizens to flourish.

I thought you meant that governments generate wealth because the things you listed have value. If so, that doesn't prove they generate wealth by my argument, unless you can prove those things are more valuable than alternative ways to use the resources the government used to produce them and that the government is more efficient in producing those.

You can argue that those are good because you think redistribution is good. But you can have redistribution without the government directly providing goods and services.

Those programs consume a bunch of money and they don’t generate wealth directly. They are critical to let people flourish and go out to generate wealth.

A bunch of well educated citizens living on government housing who don’t go out and become productive members of society will quickly lead to collapse.

None of these are unique to the government and can also be created privately. The fact that government can create wealth =/= the government is the source of all wealth.