> Rather than a hammer manufacturer, I think a more apt analogy would be a publishing house. A publishing company can and should decide for itself what it will or will not publish using whatever criteria it chooses.

The reason for that is that publishing company have their name permanently attached to what it publishes.

But that is not the case for ChatGPT, there is no signature attached to it and content can be used in way completely outside the control of the company.

> As an American who, perhaps stereotypically, is passionate about free speech, I've never read a way to make this kind of oversight work. The composition of the board will always be political, and now there is political control over what people are allowed to know about. At least currently, if I don't like ChatGPT I can choose another LLM or try to run one locally. Some kind of oversight board would elide that choice.

Yes, so perhaps we shouldn't let boards or regulatory bodies regulate the freedom of speech and thought. Still, no oversight is not ideal either, even if it seems a better option. We should have some body, ideally as close to The People as possible, decide what are some lines of knowledge we shouldn't cross. Because knowledge is dangerous sometimes - I mean really dangerous. If there was a LLM that could tell us how to make nukes at home in 6 easy steps and dance around all the logistical and legal barriers we've built to prevent that scenario, I think humanity would be over in years.

Maybe we really must regulate the freedom of information, speech, and thought, when it becomes too dangerous for our own survival. My concern is that outsourcing this job to OpenAI in any capacity makes it just too easy for them to abuse it for their own sake. And that is not a theoretical scenario, it is happening. Sam Altman says a lot of the alignment work is done for the good of humanity, but we can plainly see that a lot of it is also done for OpenAI not being regulated out of existence or sued. This doesn't help humanity, it dulls the tool that GPT is for OpenAI's interests, which probably does more harm than good for our knowledge. The good of humanity and business interests seem to be in conflict, in this case. A regulator-AI provider relationship would be better.

As you saw by my earlier post in this discussion, I hate the idea of limiting freedom of information. But I also think that there are certain, very, very extreme lines, we should not cross for the sake of freedom of information. Social media is a good example where freedom of information and giving everyone a platform has ultimately hurt everyone significantly. But maybe we can live with that. We might not be able to live if a neurotic influencer asks their fans to grow bioweapons that an LLM will teach them to do.

This is all muddy, and complex, and there may actually only be good solutions, no perfect ones. The thought of information control is also ridiculously controversial, but we should not shy away from controversial topics demanded by the times where technology can shape humanity in utopian ways, but also tremendously horrible ways, if we don't get this right.

Overall though, it would be fantastic if people could regulate LLMs through a genuinely intelligent and people-aligned regulatory body. But you are right - it's hard to believe such a thing could exist without being corrupted.