1) They have the best quality model. Better quality means more users. More users means more data. Which means higher quality...
2) operationalizing & scaling these these models is non-trivial. I'm not sure what the state of distillation/pruning is for GPT-3, but I imagine they have figured out some proprietary techniques.
3) It's not just publishing a single model, but making it so people can fine tune and push their own. Because they've gotten good at 2, now anyone can create their own version of GPT customized for their use case.
Will Google or others be able to do the same eventually? Definitely.
The point I'm more making is that it's not just training the model and running it.
Specifically, training data is not primarily coming from interactions with model. While with RLHF this data might become more important, it is still a very small portion.
How are they making it easy for people to fine tune their own?
Still, I agree with your characterization that we should see many similar models over time. As an example, see Deepmind’s Sparrow: https://www.deepmind.com/blog/building-safer-dialogue-agents
GPT <> ChatGPT: probably not. It's not hard for other big players to enter this space. It's mostly egg-on-face for Google that they haven't given that Google basically invented the model that OpenAI uses and has big versions internally. There's nothing fundamental stopping Google Docs from adding ChatGPT to their UI and getting way more consumer training data than OpenAI can get without a similar play, or for Apple to do something. Similar to what happened with mapping software, google/microsoft/azure & chinese equivs will all offer with similar competitiveness, and then complements like facebook/salesforce will do more OSS to compete against. That's already begun.
Copilot: The interesting proprietary advantage IMO is program synthesis. It's really enabled by Microsoft VSCode <> Github <> OpenAI. Without even doing any AI investments, the winner of this fight might be Gitlab, as Google/AWS/Saleforce/etc decide what to do. Before gitlab might have been a nice vehicle for shift-left sales (cloud hosting, security scans, ..), but program synthesis UIs can make Software 2.0 real.
I view this as Microsoft paying for talent the same way DeepMind was initially integrated into Google, and at the same time making the bet that this space will continue to be immensely valuable and relevant going forward.
Pretty exciting times all things considered!
Beyond that, if it becomes built into the (MS) tools that people are using then convenience is going to be a very hard barrier for Google (or anyone else) to overcome.
Then again, Microsoft’s office software is the “gold standard” (however poorly deserved) and even with amazing AI features, Google’s stuff lacks in important ways that will keep Microsoft in a strong position with or without AI features.
Microsoft may dominate the AI market for office stuff soon, but for general purpose language models Google still has a great shot, especially when it comes to mobile platforms
If it wasn't prescient, it was incredible dumb luck.
But that knowledge will disperse and is already held in many competitor companies. I do not think that OpenAI has a substantial moat here.
OpenAI has agreed to use Azure exclusively and for free, in return for Azure massively scaling up it's GPU capacity (which OpenAI was hitting before, hence threatening to move). In return, OpenAI gives an extensive license to Microsoft to use ChatGPT in Bing (and other products).
Now why couldn't the blog post have just said that rather than hiding all the details?
Because it looks bad. Microsoft wants to be seen in a different light, and OpenAI wants to be perceived as an open, academic, capped-profit for a noble reason collective.
It's all smoke and mirrors.
Does anybody actually believe this? Their name has been a running joke for years already.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
All this time, I was entirely unaware of this.
> Returns for our first round of investors are capped at 100x their investment (commensurate with the risks in front of us), and we expect this multiple to be lower for future rounds as we make further progress.
> Profits emerging from the LP in excess of the 100x multiplier go to the nonprofit, which will use it to run educational programs and advocacy work.
> The board [of the non-profit] is limited to a minority of financially interested parties, and only non-interested members can vote on “decisions where the interests of limited partners and OpenAI Nonprofit’s mission may conflict”
If OpenAI takes a good chunk of Google's ad revenues then it doesn't seem that fanciful that it'll be up toward the top of market caps.
[0] https://companiesmarketcap.com/alphabet-google/marketcap/ [1] https://www.statista.com/statistics/266206/googles-annual-gl... [2] https://businessplus.ie/news/most-valuable-companies/
But what I really hate about this whole OpenAI thing is their chosen path to have their cake and eat it too. Sam Altman seems to be something like the love child of Musk and Zuckerberg and one of the main traits is their lack of honesty.
Satya Nadella is.. Satya Nadella, there's a reason he was chosen to be the CEO of Microsoft, and while I enjoy seeing the Google demi-gods squirm, this whole OpenAI non/capped/profit thing stinks and I really don't see anyone involved capable or having the character to be something better than the current tech oligarchy.
I'm less read-up on OpenAI. It does feel to me like they've diluted the original non-profit/openness mission to the point of it being an interesting historical quirk, rather than an ongoing, guiding focus.
Is there something unique about Azure, which makes it well suited for AI?
The value to OpenAI to go Azure(/Microsoft) exclusive is:
- Money & general financial security forever
- Integration with the largest software suite in the world (Windows, O365, Active Directory, etc)
- Rapid push to worldwide enterprise sector
- Rapid push to worldwide gov/mil sector (ChatGPT, coming soon to a war near you!)
- Rapid push to education sector
Azure does have unique offerings, such as higher-spec compute nodes than AWS and RDMA-capable “HPC” sizes that can be used to build mini supercomputers.
AFAIK, no other public cloud has this type of capability.
Azure has been focused on HPC much longer and has a head start on AWS. Not sure if they knew it would pay off with AI, but it looks like it’ll work out well for them.
They see the potential brand damage that comes with a such a product, so they probably just hold less than 50% of the company, to be able to say "this is not us".
Which then allows openAI to experiment and wow people, while google is anxious about showing 10% of its crazy AI capabilities to avoid potential PR disasters.
1) they've recognized that none of this could emerge out of a corporate research department of a major trillion $ company. Just too rigid and constrained by day to day business concerns (e.g. Tay bot)
2) they recognized the strategic value to their business regardless of that risk (can't afford to let some more nimble player to leap frog them)
3) they identified a key startup partner that can get results here quickly and bank rolled them (OpenAI)
4) At the first hint of success, they consolidated early investments and are now building out a productive partnership with that partner.
Doing so, they minimize their risk while keeping their options fully open and getting a decent ROI in the process.
Contrast that with Google, which identified the same opportunity many years ago, declined to partner and insisted on doing everything in house, is also a trillion $ company, and is indeed not getting much of note done so far. Google is failing hard here. They are executing the strategy that Microsoft abandoned after Ballmer that wasn't working at all for them.
Which was to be highly secretive, ignore outside innovation, and keep on betting on results from internal R&D instead, all while repeatedly failing at that. So, Google is now saying that they could have done this or that, better, faster, cooler, etc. Except they didn't and haven't. Could have; would have; should have; doesn't count here.
This might just be the wake up call that Google has needed for a few years. Or not. We'll see. Time to put up or shut up for them. My guess is that they will still be figuring out how to respond to this for some time and are completely paralyzed by current events. The prudent thing for MS would be to not wait for that and move full steam ahead. Opportunity of the decade for them to grab some market share. This announcement is a good sign that they are doing that.
And it wouldn't have been hard to fix this earlier. Simply down rank sites asking to sign up for notifications or showing popups after 5 seconds
I think slowly Google is becoming more and more like IBM, a slow-moving giant that doesn't like risks.
And its well known, many news titles mention Deepmind alongside google, if not as "Google ai scientists".
Say what you will about Google, they generally credit their sources. Yes, it's part of their advertising model, but it's still a Very Good Thing.
I hope that Google's plan is to release something that continues this model. If it's near as good at ChatGPT and strongly includes sources, it is the right future.
How it actually works is more like:
1. User asks "What's the tallest building in the world?"
2. MS, rightfully, assumes that GPT has no idea what the answer to this is. And even if you trusted it to know, it will always lag behind and new buildings could have been built since then.
3. MS searches their index for the most relevant document snippets related to this query and feeds it to GPT as context.
4. MS asks GPT to answer the question in the context of those document snippets.
5. MS returns the result from GPT along with references to the documents it sourced the information from.
This is how the OpenAI /search endpoint used to work.
https://openai.com/blog/webgpt/
It's probably not too hard for them to tune ChatGPT and the upcoming GPT4 that way, and I think it's very likely they will do something like that in Bing.
Can you elaborate here? (Honestly asking since I'm not seeing the similarity)
That’s what folks were crassly saying to truck drivers, baristas, and cashiers.
The writing was/is on the wall: AI is only going to get more powerful and able to be applied to more and more complex tasks.
The thought was that “unskilled” labor would be the lowest hanging fruit, that automated AI - with some (but minimal) human oversight - would replace serious chunks of the workforce in various minimum-wage and “blue collar” sectors.
Machines don’t necessarily need to sleep, they don’t have labor unions, or laws that require healthcare or overtime pay. They don’t get upset, take things personally, seek revenge, or reciprocity like a person.
Sounds like that could be a threat to many kinds of jobs, many of them “bullshit jobs” (in the words of Graeber), but others as well.
It seems (to me, at least) that the more immanent threat is text-based AI - with some human oversight - replacing large swaths of the tech workforce (many of whom were leading the narrative about the truck drivers).
The incentives for companies like Microsoft, Amazon, Google, and others, to go this route are obvious, beyond the stated reasons why “low-skilled” labor is at risk. They already have enormous investments, acquisitions, projects, established platforms, and infrastructure related to AI.
I expect to see more partnerships like the one between Microsoft and OpenAI, from all of the major tech companies.
I also believe the connection between the acquisitions/partnerships and the mass layoffs will become more and more obvious as these acquisitions/partnerships continue to happen.
I can’t be the only person that is noticing this…
In the meantime, rocky road.
The only thing which makes me less sad is that I'm pretty sure Moravecs paradox is actually not all that real, but is more due to the relative lack of engineering interest put on solving continuous control problems. Apparently reinforcement learning on transformers works now (RLHF in ChatGPT). This implies that we should see high effective continuous control models very soon. Robots are coming for physical labor, it'll just take a bit longer.
Shit man, when I was last in South Korea, I felt like I was living in the future. They had many "24/7" drink cafe's where it's literally just a robot arm that makes the drink for you for a few dollars.
It is painfully ironic to knowledge workers that they are destroying their own earning potential, but physical labor is not safe. Nothing is safe.
IMO this will have positive consequences if AI doesn’t enslave us.
quotes in article: none
dates / timelines for partnership deliverables: none
was this written by chatgpt?
---
Create a PR release in 10 paragraphs about Microsoft extending their partnership with OpenAI, by investing another $3.2 billion.
Paragraph 1: Microsoft is excited to announce that it is extending its partnership with OpenAI by investing an additional $3.2 billion. This partnership has been a key driver of innovation and progress in the field of artificial intelligence, and we are thrilled to continue our work together.
Paragraph 2: The investment will be used to support OpenAI's ongoing research and development efforts, as well as to expand the company's capabilities in areas such as machine learning, natural language processing, and computer vision.
Paragraph 3: With this new investment, Microsoft and OpenAI will continue to push the boundaries of what is possible with AI, driving new breakthroughs and advancements in the field.
Paragraph 4: This extended partnership will also enable Microsoft to bring the benefits of OpenAI's technology to its customers and partners, helping to drive new business opportunities and enable organizations to take advantage of the latest AI advances.
Paragraph 5: In addition, Microsoft and OpenAI will be working together to promote responsible AI practices and ethical guidelines, ensuring that the technology is used in a way that is safe, fair, and transparent.
Paragraph 6: The partnership between Microsoft and OpenAI has already resulted in some significant accomplishments, such as building a language model that can generate human-like text, and creating a new system for natural language understanding.
Paragraph 7: With this new investment, the two companies will be able to take their collaboration to the next level, creating new possibilities for AI applications across various industries.
Paragraph 8: Microsoft CEO Satya Nadella said, "We are excited to continue our partnership with OpenAI to drive innovation and progress in the field of artificial intelligence. This new investment will enable us to bring the benefits of AI to more people and organizations around the world."
Paragraph 9: OpenAI CEO Sam Altman said, "We are thrilled to be continuing our partnership with Microsoft. Their support and investment will allow us to continue to make significant advances in AI research, and bring the benefits of this technology to more people and organizations."
Paragraph 10: The extended partnership between Microsoft and OpenAI is a major step forward for the field of artificial intelligence, and we are excited to see what the future holds. Together, we will continue to push the boundaries of what is possible and drive progress in this rapidly evolving field.
> Paragraph 8: Microsoft CEO Satya Nadella said, "We are excited to continue our partnership with OpenAI to drive innovation and progress in the field of artificial intelligence. This new investment will enable us to bring the benefits of AI to more people and organizations around the world."
> Paragraph 9: OpenAI CEO Sam Altman said, "We are thrilled to be continuing our partnership with Microsoft. Their support and investment will allow us to continue to make significant advances in AI research, and bring the benefits of this technology to more people and organizations."
There's something particularly off-putting about machine generated quotes attributed to an actual person. It's completely different than a person generating the content themselves.
- you still have to apply to https://customervoice.microsoft.com/Pages/ResponsePage.aspx?...
- you will be rejected if you use a personal email address
- you will be rejected if you are not a "managed customer"
- you will be rejected if.. Microsoft thinks you are a real competitor to any of their products?
So my site aidev.codes is will be dead in the water if I can't navigate these things or afford the "managed" part. What is that? I don't know what a "managed customer" is but I assume its very expensive.
I mean, this is really the opposite of Open.
Or am I missing something? Is it really generally available and they took all of that stuff off?
BTW I am looking for an investor: https://news.ycombinator.com/item?id=34494480
Cocks gun, Always has been https://en.meming.world/wiki/Always_has_been
OpenAI was never really open in the first place.
Good. I get better results from chatGpt than from Copilot. For example, often Copilot would invoke a bash command for copying a file in my python script, instead of using pythons buildin shutil.
I've been brushing off AI as overhyped, but this is very compelling. I believe the real crux of software engineering is thinking about the problem and organizing solutions today which can be changed/improved/iterated in the future. Programmers too often overweight the time it takes to type things (using short variable & function names or creating terse 1 liners). But if our objective is to make code easy to change/improve/iterate in the future, then it needs to be readable now.
The nice autocomplete features in most IDEs have been a huge win to productivity along with Google search. I think chat AI could be an order of magnitude improvement.
This might just be the equivalent of the old guy complaining that typewriters are better than word processors.
Even capitalized and straight from the horses mouth, but as unsurprisingly predicted in: [0]
> * OpenAI will gain further investment...
We already know about ChatGPT Pro since that is a paid version of ChatGPT coming soon, meaning that the second prediction in [0] as also turned out to be true:
> ...ChatGPT by then will become a paid service
Now we wait until the startups depending on the service start to raise their prices.
Making the best developer tools for free or at least close to free is another form of extinguish and they have just done that with OpenAI and GitHub.
When I visit it, I am getting a poem about how it is over capacity. And that's it.
No, but ChatGPT is at full capacity which means that it will 'randomly' block new connections until load drops. Keep trying every 5 - 10 minutes and you'll eventually get in
OpenAI and Microsoft are working together again, Microsoft is giving them a lot of money to help them make AI that is safe and helpful and they are excited to keep working together.
Paragraph 2: We are proud of the work we do and the products we provide to our customers. We have always maintained the highest standards of integrity and transparency in our business practices.
Paragraph 3: We ask that the public not be swayed by the misinformation being spread by our detractors. We are confident that the truth will come out and our reputation will be cleared. In the meantime, we will continue to focus on delivering the best possible products and services to our customers.
Google and Facebook are the clear leaders in AI and they been spending a lot of money building out their AI infrastructure over the last decade. You haven't heard much about Microsoft's investments other than them buying rights to OpenAI models. Seems like they could have saved a lot of money if they started investing years ago.
Google bought DeepMind for 400m 8 years ago. You could argue DeepMind is on the same level or above OpenAI.
Google and Facebook might be better at some things, and maybe Apple too. But none of them has brought to market a chatbot that works pretty well(and not just as a party trick) and is very accessible to anyone on the Internet. I would love it if open up Siri such that I can communicate with it on a site just like ChatGPT.
Google and Facebook may have good research teams, but so far they don‘t seem to be able to use them to ship products (with some exceptions, mostly incremental improvements to quality of existing products) and that‘s what will count.
[0] https://azure.microsoft.com/en-us/blog/general-availability-...
---
Two hearts, two minds, working side by side
In pursuit of a mission to make AI safe and wide
Microsoft’s investment allows us to stay true
To our capped-profit company, our mission, and you
From cloud computing to real-world use
We collaborate to ensure our systems are not abused
OpenAI API, Azure’s OpenAI Service, and Designer too
We’re working side by side to make our visions come true
Together we review, together we synthesize
Together we build, together we realize
The potential of advanced AI that’s powerful and safe
Ensuring its benefits can be shared far and wide
Priceless. Maybe companies should get AI to write PR releases. They say the same thing, except more honestly and funnier.
Is there something unique about Azure, which makes it well suited for AI? (vs AWS or GCP)
> "recalibrate" means "increase" obviously.
> disappointing to see this six-week development. openai will continually decrease the level of risk we are comfortable taking with new models as they get more powerful, not the other way around.
How does this resolve?
OpenAI not releasing something isn't going to stop Google or another player from releasing it. Won't they have to recalibrate too or be overtaken? Pandora's box is already open.
I wonder how much of choice will enterprises or large customers have in deciding what data to not send to Microsoft to train their models. If OpenAI is tightly integrated to all Microsoft products this may be a real concern.
translation: Microsoft is not Google (who wishes we'd explode).
>"OpenAI was in talks this month to sell existing shares in a tender offer that would value the company at around $29 billion, The Wall Street Journal reported, making it one of the most valuable U.S. startups on paper despite generating little revenue."
Can anyone say why they wouldn't have gone that route? Is that valuation possibly based on market condition from a year ago?
As just one dimension, let's briefly review education limited to testing. Options here range from (paid) cooperation between institutions and companies, to marketplaces for detection technology and testing systems, to complete rethink of testing at the institutional level, to regulatory intervention.
The dynamics and outcomes resulting from considering the above before or after widely available and improving LLMs are a fact of life, are possibly very different. We haven't yet crossed the Rubicon here and not sitting as captive spectators in a theater of ultra rich remains an option.
I laughed at independent research. Why do they even bother..
>"Azure’s unique architecture design has been crucial in delivering best-in-class performance and scale for our AI training and inference workloads."
Can someone say what is unique about the Azure cloud architecture here vs other cloud vendors?
I presume that by integration of the Open AI services into the bing and MS Office, Microsoft’s mission (empowero every person and every organization on the planet to achieve more) makes lot of sense.
Correct me if I'm wrong, but it seems like some type of 3rd party Model Governance should be happening. Is it?
