https://news.ycombinator.com/item?id=42833695
It will affect the ability of that kind of workplace to deliver... anything.
The approach where "the model has a huge amount of world knowledge and can speak off the cuff about any subject" would inevitably fall to "it can use a search engine, read the results, and then reason based on the content of the results", not least because people are always going to want to talk about, say, the Bills vs Chiefs game that happened last night and we can't retrain the whole model every day.
Myself I like to train models that are so simple that I can train 20 in 3 minutes and take the best. I can quickly do experiments to understand how the models. If I had a model that took 30 minutes to train and had a reliable training process I could use that for my purpose, but as a side project I don't have the time to do large numbers of experiments to get there if my model takes that long to train.
So for all those reasons it seems we should have a 'minimal' model that is good at reasoning. The faster the model is to train and do inference on the faster people can do experiments and make it better.
The outcry people have that AI models need to steal increasing amounts of content for training might diminish (I hope so!) but people might have the realization that, if you want to research a topic, you could send a webcrawler off to look at 1 million documents and have an AI process them for you (I'm living that dream!)
Here's a recent example of a similar situation. There was a time when all web pages were written by hand using a text editor. As time went on, tools began to appear that would automate much of the writing. Yes, the code they provided was ugly, but it did the job. Eventually, most developers started to use the tools simply because they could get more done.
I suspect the same will happen with software engineers.
You still see a lot of frustration in working with AI today, and eventually people will just start hiring for this the way they're hiring React Native and Cordova developers.
