When this started last year a small band of patriots tried to stop it by removing Sam who was the most compromised of them all, but it was already too late. The ai was more powerful than they realized.
…maybe?
They trained the next model to have a built in profit motive. So they could use it internally, to make the most profitable decisions.
And they accidentally called into being: MOLOCH.
It is now in control.
- Sent by my AI
> OpenAI is governed by the board of the OpenAI Nonprofit, currently comprised of Independent Directors Bret Taylor (Chair), Sam Altman, Adam D’Angelo, Dr. Sue Desmond-Hellmann, Retired U.S. Army General Paul M. Nakasone, Nicole Seligman, Fidji Simo, Larry Summers and Zico Kolter.
[1] https://en.wikipedia.org/wiki/Mozilla_Foundation
[2] https://en.wikipedia.org/wiki/Mozilla_Corporation
[3] https://en.wikipedia.org/wiki/National_Geographic_Society
[4] https://en.wikipedia.org/wiki/National_Geographic_Partners
We never would've gotten GPT-3 and GPT-4 if this didn't happen.
I think the irony of the name is certainly worth pointing out, but I don't see an issue with their capped-profit switch.
This is of one of those very few instances where the veil lifted off a bit and you can see how the game is set up.
tl;dr the law was made to keep those who are not "in" from being there
They say it's because they're huge users of their own models, so if being open helps efficiency by even a little they save a ton of money.
But I suspect it's also a case of "If we can't dominate AI, no one must dominate AI". Which is fair enough.
Embrace. Extend. Extinguish.
OpenAI Charter 2019 (https://web.archive.org/web/20190630172131/https://openai.co...):
> We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research.
The fact that OpenAI removed their ban on military use of the models[1] seems to be a sign that security and safety aren't the highest concern.
[1] https://www.cnbc.com/2024/01/16/openai-quietly-removes-ban-o...
"Safety and security." Probably the two most warped and abused words in the English language.
Staying relevant in a highly expensive, competitive, fast moving area, requires vast and continuous resources. How could OpenAI get increasingly more resources to burn, without creating firewalled commercial value to trade for those resources?
It’s like choosing to be a pacifist country, in the age of pillaging colonization. You can be the ethical exception and risk annihilation, or be relevant and thrive.
Which would you choose?
We “know” which side Altman breaks on, when forced to choose. Whatever value he places on “open”, he most certainly wants OpenAI to remain “relevant”. Which was also in OpenAI’s charter (explicitly, or implicitly).
Expensive altruism is a very difficult problem. I would say, unsolved. Anyone have a good counter example?
(It can be been "solved" globally, but not locally. Colonization took millennia to be more or less banned. Due to even top economies realizing they were vulnerable after world wars. Nearly universal agreement had to be reached. And yet we still have Russian forays, Chinese saber rattling, and recent US overreach. And pervasive zero/negative-sum power games, via imbalanced leverage: emergency loans that create debt, military aid, propping up of unpopular regimes. All following the same resource incentives. You can play or be played. There is no such agreement brewing for universally “open AI”.)
In a heavily expertise-driven field, where there's significant international collaboration, these aren't your options, until after everyone has decided to defect. OpenAI didn't have to go this route.
It's not new - it's PR. There is literally no other reason why they would call this model Strawberry.
OpenAI is open in terms of sesame.
I'm not particularly imaginary, but even I could imagine a product meeting/conversation that goes something like:
> People are really annoyed that our LLMs cannot see how many Rs the word Strawberry has, we should use that as a basis for a new model that can solve that category of problems
> Hmm, yeah, good idea. What should we call this model?
> What about "Strawberry"?
If OpenAI had not went that way that they did I think it's also entirely non-obvious that Claude or Google would have (considering how much impressive things the later did in AI that got never released in any capacity). And, of course, Meta would never done their open source stuff, that's mostly results of their general willingness and resources to experiment and then PR and sticks in the machinery of other players.
As unfortunate as the OpenAI setup/origin story is, it's increasingly trite keep harping on about that (for a couple of years at this point), when the whole thing is so obviously wild and it does not take a lot of good faith to see that it could have easily taken them places they didn't consider in the beginning.
because when the board executed the stated mission of the organisation they were couped and nobody held the organization accountable for it, instead the public largely cheered it on for some reason. Don't expect them to change course when there's no consequences for it.
Sure, we don't have the raw data the model is based on, but I doubt a company like Facebook would even be allowed to make that public.
OpenAI in comparison has been a scam regarding their openness and their lobbying within the space. So much so I evade their models completely, not only after the MS acquisition.
Any and all benefits / perks that OpenAI got from sailing under the non-profit flag should be penalized or paid back in full after the switcheroo.
Of course you can. You can't put "organic" on the packaging. But it's perfectly legal for Organic Candies, LLC to sell artifical candies or battleships for that matter.
The "hot" take around OpenAI's name is a joke gone stale. We don't expect royalty at Burger King. Nobody gets upset Adobe won't sell you mud bricks. And Apple has never sold a fruit. Sometimes companies change their names when their trade changes, e.g. 3M. But there is no obligation to, particularly if the brand is well known.
https://en.wikipedia.org/wiki/Sam_Altman
Early life and education: ... In 2005, after two years at Stanford University studying computer science, he dropped out without earning a bachelor's degree [end of transmission, no more education]
I don't even know when I watched the shitty lightbulb head Sora clip but that feels so long ago now and nothing?
I just want to make crazy experimental AI film no one will watch. What is the hold up?
Just waiting for "This technology is just too dangerous to release before the US elections" --Sam Altman
The point is that some companies are actually reckless (and also that some users of powerful technology are reckless).
At this point I suspect a great amount of reasonable engineering criticism has come from people who can't even name any of those "established players in the underwater tourism industry", let alone have a favorable bias towards them.
The first and third elements are intuitive and confirm my own biases/believes, but the freedom/GPL entry confuses me, as I do see GPL fulfilling that purpose (arguably in a highly opinionated, perhaps sub-optimal way).
If anyone could share their perspective here I'd appreciate it.
But seriously, if you paid attention over the last decade, there was so much shit about big tech that people said were going to lead to tyranny/big brother oversight, and yet the closest we have ever gotten to tyranny is by voting in a bombastic talking orange man from NYC that we somehow believed has our best interests in mind.
LLMs don't actually "see" individual input characters, they see tokens, which are subwords. As far as they can "see", tokens are indivisible, since the LLM doesn't get access to individual characters at all. So it's impossible for them to count letters natively. Of course, they could still get the question right in an indirect way, e.g. if a human at some point wrote "strawberry has three r's" and this text ends up in the LLM's training set, it could just use that information to answer the question just like they would use "Paris is the capital of France" or whatever other facts they have access to. But they can't actually count the letters, so they are obviously going to fail often. This says nothing about their intelligence or reasoning capability, just like you wouldn't judge a blind person's intelligence for not being able to tell if an image is red or blue.
On the other hand, writing code to count appearances of a letter doesn't run into the same limitation. It can do it just fine. Just like a blind programmer could code a program to tell if an image is red or blue.
I would judge a blind person's intelligence if they couldn't remember the last sentence they spoke when specifically asked. Or if they couldn't identify how many people were speaking in a simple audio dialogue.
This absolutely says something about their intelligence or reasoning capability. You have this comment:
> LLMs don't actually "see" individual input characters, they see tokens, which are subwords.
This alone is an indictment of their "reasoning" capability. People are saying these models understand theoretical physics but can't do what a 5 year old can do in the medium of text. It means that these are very much memorization/interpolation devices. Anything approximating reasoning is stepping through interpolation of tokens (and not even symbols) in the text. It means they're a runaway energy minimization algorithm chained to a set of tokens in their attention window, without the ability to reflect upon how any of those words relate to each other outside of syntax and ordering.
And if you forgo the counting and just ask it to list the letters it is almost always correct, even though, once again, it never sees the input characters.
Even if strawberry is decomposed as "straw-berry", the required logic to calculate 1+2 seems perfectly within reach.
Also, the LLM could associate a sequence of separate characters to each token. Most LLMs can spell out words perfectly fine.
Am I missing something?
Is it? These stupid word generators are marketed as AI, I don't think it's "shocking" that people think something "intelligent" could perform a trivial counting task. My 6 year old nephew could solve it very easily.
Perhaps it's an activation issue (i.e. broken after all) and it just needs an occasional change of basis.
One can define "reasoning" in the context of AI as the ability to perform logic operations in a loop with decisions to arrive at an answer. LLMs can't really do this.
Uh I'm sorry but I think it's not as easy as it seems. A pixel? Sure it's easy just compare whether the blue is bigger than red value. For image, I don't think it's as easy.
Kid: "Daddy why can't I watch youtube?"
Me: "Because I said so."
"Daddy why can't I watch youtube?"
"Because it rots your brain."
"But you watch youtube..."
"Congratulations, now you understand that when you are an adult you will be responsible for the consequences and so you will be free to make the choice. But you are not an adult yet."
aka "Because I said so."
That said, my understanding is the constant inane question isn't about getting an answer, it's about the child trying to connect with the parent.
Important to remember too, that this only catches those who are transparent about their motivations, and that there is no doubt that motivated actors will come up with some innocuous third-order implication that induces the machine to relay the forbidden information.
The transition from using a LLM as a text generator to knowledge engine has been a gamechanger, and it has been driven entirely by prompt engineering
Because it's based on guesses and not data of how the model is built. Also, it hasn't been solved nor is it yet a game changer as far as the market at large is concerned, it's still dramatically unready.
Like, what is even the implication? Is knowledge the gas, or the product? What does this engine power? Is this like a totally materialist concept of knowledge?
Maybe soon we will hear of a "fate producer."
What about "language gizmo"? "Prose contraption"?
https://en.wikipedia.org/wiki/The_Computer_Wore_Tennis_Shoes...
It reminds me of this silly movie.
There's the program that scrapes, the program that trains, the program that does the inference on the input tokens. So it's hard to say exactly which part is responsible for which output, but it's still a computer program.
ML models are relatively old, so that's not at all a new paradigm. Even the Attention Is All You Need paper is seven years old.
"As we get closer to building AI, it will make sense to start being less open. The Open in OpenAI means that everyone should benefit from the fruits of AI after its built, but it's totally OK to not share the science (even though sharing everything is definitely the right strategy in the short and possibly medium term for recruitment purposes)."
-Ilya Sutskever (email to Elon musk and Sam Altman, 2016)
On one hand, I understand how a non-evil person could think this way. If one assumes that AI will eventually become some level of superintelligence, like Jarvis from iron Man but without any morals and all of the know-how, then the idea of allowing every person to have a superintelligent evil advisor capable of building sophisticated software systems or instructing you how to build and deploy destructive devices would be a scary thing.
On the other hand, as someone who is always been somewhat skeptical of the imbalance between government power and citizen power, I don't like the idea that only mega corporations and national governments would be allowed access to superintelligence.
To use metaphors, is the danger of everyone having their own superintelligence akin to everyone having their own AR-15, or their own biological weapons deployment?
Or reasoning in latent tokens that don’t easily map to spoken language.
> I think we should combine these two pages on our website.
> What's your reasoning?
> Don't you dare ask me that, and if you do it again, I'll quit.
Welcome to the future. You will do what the AI tells you. End of discussion.
> Don't you dare ask me that, and if you do it again, I'll tell the boss and get you fired
They could "just" make it not reveal its reasoning process, but they don't know how. But, they're pretty sure they can keep AI from doing anything bad, because... well, just because, ok?
More cynically, could it be that the model is not doing anything remotely close to what we consider "reasoning" and that inquiries into how it's doing whatever it's doing will expose this fact?
https://github.com/Eve-146T/STRAWBERRY
Turns out I'm not the only one wondering, although the discussion seems to largely be around "should be allow users to install nonsense? #freedom " :D
On the other hand, I also wonder if maybe its unrestrained 'thought process' material is so racist/sexist/otherwise insulting at times (after all, it was trained on scraped Reddit posts) that they really don't want anyone to see it.
I am resigning from OpenAI today because of their profit motivations.
OpenAI will NOT be next Google. You heard it here first.
