This tagline is representative of every part of the hype around GenAI. It makes it sound like security has fundamentally changed and we all need to re-learn what we know. Everything to do with GenAI is treated like this: we need new security plans, we need AI Engineers as a new job title, we need to completely reevaluate our corporate strategies.
Security in the world of generative AI is not substantially different than infosec has been for a while now: User prompts are untrusted input. Model outputs are untrusted input. Treat untrusted input appropriately, and you'll be fine.
The same goes for "AI engineers", who are in the business of wiring up APIs to each other like any other backend engineer. We take data from one black box and transfer it to another black box. Sometimes a black box takes a very long time to respond. It's what we've always done with many different kinds of black boxes, and the engineering challenges are mostly solved problems. The only thing that's really new is that the API of these new black boxes is a prompt instead of a deterministic interface.
Don't get me wrong, there will be things that will be different in the post-LLM world. But my goodness do the current crop of companies overestimate how large that difference will be.
Another big area of hype is "prompt engineering." That one seems to have calmed down slightly, but for a while, there were large swaths of the Internet who were amazed that the set intersection of "talk like a decent human being" and "be precise in your communication" could generally lead to good results.
In many ways, "AI" right now is magic marketing sprinkles that you can put on anything to make it more delicious. (Or, if you're inside a big company, it's magic prioritization sprinkles.)
I think your comment conveys your obliviousness of the problem domain.
The main driving need for prompt engineering is not an inability to "talk like a decent human being". That's just your personal need to insult and demean people who are interested in a problem domain you know nothing about.
The main driving need for prompt engineering is aspects like not being able to control how context is formed and persisted in a particular model, and how to form the necessary and sufficient context to get a model to output anything interesting. Some applications require computationally expensive and time-consuming runs, and knowing what inputs to provide to a system which by it's very mature is open-ended is a critical skill to adequately use the system in professional settings.
Let's put it like this: GitHub copilot is a LLM service which is extremely narrow in what are their applications and use cases. Yet, you can't even get it to add unit tests to a function following a specific style without putting the effort to build up the context it needs to output what you expect.
I think the problem is they're trying to introduce nuance and a narrow path to allow this. They want an acceptable level of risk to using untrusted model output for the efficiency/productivity gains it will bring, notwithstanding hallucinations.
Generative AI would not have flown in the security theater of Yesteryear, but CTOs see productivity multipliers.
The CTOs are hallucinating as much as the LLMs are.
I was trying to understand what prompt engineering was, because I thought there is no way this is a discipline for how to ask ChatGPT questions... And then I realized it was...
Sure, I get that there is much to learn regarding formulating effective prompts, but a new career path?
until the next one, of course. ;)
for example, in rough order, some past hype trends: 3GLs, structured programming, initial AI (then AI winter), expert systems, CASE tools, 4GLs, OOP/OOAD, UML and round trip engineering, design patterns, dot com boom (and bust), agile, functional programming, Web 2.0, SaaS, crypto, Web 3.0, big data, data science, ML/AI.
most of them had or have some actual benefits, but nothing like the hype parrotted, by those with and without vested interests.
been there, seen them, from the third or fourth one onwards.
I keep going back to the basics: Serverless is servers. Machine learning is servers. GenAI is servers. And, from what I've heard, most of AWS revenue is servers and storage.
(For the record: I am also an AWS Hero, and an AWS customer since 2006.)
I don't agree. I think AWS has always been extremely customer-focused, and they scramble to offer whatever service might have any traction at all from customers. It's just that they are already providing the low-level baseline services, and now they are progressing to offer increasingly higher-level ones.
I'm talking about machine learning-driven firewalls, backend for mobile applications, video streaming, edge computing, even Blockchain and now LLM services.
As much as it might surprise you, there is plenty of real-world demand for these services. You might accuse it of being "bandwagonny", but if you take an objective look at it you'll find that they are playing the role of supply store owners during the gold rush. It comes at no surprise that AWS is the one part of Amazon whose revenue is growing massively year-on-year, with the last report pointing to a 17% growth year over year.
https://www.cnbc.com/2024/04/30/aws-q1-earnings-report-2024....
Is that what you would call bandwagonny?
The way that cloud businesses work, you sell the servers for about as cheap as you possibly can do. Instance prices are all a race to the bottom among the providers, because servers are largely commodity hardware that's easy to get from any number of providers, and it's one of the first prices customers see, and often plays a big role in their choice of provider.
So that's not where you make your profits. So you're right, lots of revenue, but crucially, there is no real profit. There never will be. That makes it a boring product, not worth focusing a lot on from a marketing perspective etc. Same tends to go for all of what you might think of as the basic building blocks of the cloud. e.g. object storage prices are often really close to what it actually costs to provide the service.
You make your profits on what you sell that runs on the cloud. All those additional things like databases, streaming services, kubernetes bits, functions etc. Those are where you make your actual profits. GenAI is a big potential profit driver for AWS, so that's where they're pushing. A couple of years ago it was "$foo, but on Kubernetes". Before that it was "$foo, but Serverless". They're just pushing where the profit and interest is, and pretty much always have done.
sort of side-note: Gartner's evaluation of cloud providers got really absurd around kubernetes stuff. Because one cloud would do it, you'd miss out on points if you didn't also add it, even if being on kubernetes literally added zero benefit, or arguably was worse. Same for "Serverless". It didn't matter if customers were actually using it, or wanted it, if AWS/Azure/GCP launched it, you'd better have it too.
I don't think that's it. Those who migrated their EC2 apps to ECS/EKS/Fargate/App Runner have already migrated, so there is diminished returns in pushing those technologies.
The same goes for serverless. The whole world already adopted this to it's full extent. Those who want/can use these services, are already running these services, and AWS is already getting better utilization rates from their idle computational resources from this.
These are not fads. They are already infrastructure.
What we are seeing is additional high-level services being released to meet customer demand. There's now a massive need for training and running your custom private LLMs. There is absolutely no justification to skip the revenue you can generate by serving these markets.
If nobody wants you to use them, is that because everyone already has as much conventional architecture as they need? Perhaps the new opportunities are all in AI because we've pushed conventional stuff as far as it could go, and we were just rearranging deck chairs.
I'll be honest that, if we've run out of ideas, I dunno if AI really solves any problems I want solved. But even if not I don't see how appealing to AWS fixes anything.
There's a ton of low hanging fruit in all the cloud vendor products. Look outside AWS at tailscale, vercel, and fly.io for some obvious examples.
Why can you use a public HTTP gateway, but not in a VPC?
All this stuff would make my life so much better than any form of genAI.
> The same goes for feature releases. If the vast bulk of all new feature releases are geared towards GenAI, it implicitly means AWS is rerouting investments from classic infrastructure to shiny GenAI. It means that the products I love get smaller budgets. It means that the products I use will not get the next feature I want, or only at a slower pace.
I think the article does hyperbolize a bit, but this seems like a hard truth. Unless AWS has hired an entirely new swathe of AI-focused engineering talent, or if their public face at events is significantly disconnected from where they're spending their real money.
So, I do not belief that clouds are at the end of their innovation.
AWS should be spending significant time explaining how their giant portfolio of conventional tools are improving. If they have stopped, they've lost focus. But hopefully it's just the marketing team focused on magic beans.
The problem being that nobody listens of course. You still have to build heaps of genAI crud because leadership is so excited about it.
Then you end up building a free text box to automatically determine one of 4 different problems the customer can have, and scratch your head wondering why we don’t just provide 4 options instead of letting the customer write a whole story...
See but that won't work because leaders are dumb. The central tension of civilization is that the smart ones are needed down in the boiler/operating room to solve complex low-level problems, and whoever's left must steer the ship.
However, what happened is that it became apparent that not everything needs to be big data. Business needs will shine through as they always have and dictate what is truly important.
I'm not afraid of the wave of gen AI. Think of it as the new power tool that just came out that everyone's currently talking about. You'll add it to your toolbox because you don't want to be obsolete. It'll blend into everything else once the hype wave is over.
Enhancements expected by the community will be delayed, engineers need to come up with temporary solutions which needs to be rewritten next year, new ways of solving problems will not be shared, new content will be created only for a couple of products.
The problem is, for all it's talk over the last few years, AWS remains a complete non-player in the GenAI space, much less so than Azure. In my opinion the problem is exactly the same as for every other high-level service they've tried to launch. QuickSight, Lex, Polly, Cognito, CodeGuru, SageMaker, etc: they're not good. Nobody ever said "I really like QuickSight, I sure wish it had GenAI capabilities". So when the hastily-expanded QuickSight team(s) then goes on to release 42 different Q enabled SKUs, nobody cares. For various reasons, AWS is organizationally incapable of launching a non-infrastructure product that is simply great, as doing so would take attention to detail and deeply caring about things like UX which are anathema to Amazon.
On the positive side, GenAI model access will be commoditized and part of the basic undifferentiated cloud infra, and AWS will do fine there.
I’d argue that nothing AWS launches is great at launch. The good products we actually use have all been around for 10+ years.
It was an awesome (and awesomely overwhelming) experience, but I completely agree with the author. GenAI EVERYWHERE.
The other topics that the author brought up from re:Invent 2022 were still present, but not without heavy mentions of how AI contributes to them.
That said, I have some predictions that might make OP happier.
DevOps and Platform Engineering is still a hot topic, especially in a world where companies are repatriating back to the data center (or are at least going hybrid). All of the 2010s bare metal tech (Foreman, Ansible, etc) are going to come back in big ways, and Kubernetes consumption will only increase. eBPF and systems engineering is still hot and will really help here for high-performance observability.
Companies that won't repatriate or want to use the cloud for prototyping will want to focus on cost optimization. This requires serious cloud engineering skills (using spot instances and S3 lifecycle policies is table stakes; much more can be done, especially on reporting and automation).
GenAI will help here (super helpful for analyzing time series data and surfacing patterns), but having the fundamentals will always be useful here.
c.f. Google IO keynote this year. I couldn't tell you a single thing Google is launching this year, beyond limited, rushed features where Gemini chat is in a side pane.
And that's not on me: it's because Google literally didn't talk about a single other thing.
And as usual, Google is out of touch and doesn't get the joke, c.f. at the end, Sundar presenting their own count of how many times they said AI.
I sorely miss tech industry of the 00s, I simply cannot imagine ex. 2000 Apple/Steve Jobs falling for this. There's this weird vapid MBA brain drain in charge everywhere. But hey, stonk goes up.
Most tech companies don't have a whole lot to show right now, so AI sucks up all the oxygen in the room. This becomes a feedback loop with the stock market, too.
> There's this weird vapid MBA brain drain in charge everywhere.
Yep. And Apple's playing along as well. Their latest WWDC presentation has the most weird tone I've ever seen in their presentations ever: "we added some AI features and they're pretty cool I guess...also it's super private! Here's all the ways it's kept private."
So much hedging going on. So little excitement. Because they're just playing to someone else's tune, and they're not good at doing that.
2024 Apple with AI: this is our best year ever. Look at our CTOs hair. We fixed Siri being eons behind, now if it thinks ChatGPT can help you can tap a confirmation dialogue.
I wouldnt be so quick to assume that. Let’s not forget that Jobs bought Siri and then integrated it into every platform they sold.
AI has been a buzzword for literally decades. It’s just exploding in popularity right now because the capabilities of GenAI have recently exploded.
It’s a little like how VR and AR has been around for decades. It’s just taken this long for the technology to make mixed reality a possibility for the masses.
Haha be kind to your ex-employer. This was supposed to be a joke. I agree they don't get it but they at least tried this time. I ranked this high on the Sundar-joke ladder (which is a low bar I know).
No he would not entertain any of this nonsense. IIRC he had a hard time with the Siri demo too. It appears Apple is happy to take itself and shareholders on a side quest with all their cash…good time to be earning interest on it too.
For example, the team / leadership / foundation behind Home Assistant has been pushing AI features hard in the past 18 months or so. This coincides with my feeling that there hasn't been any relevant improvement in Home Assistant's core features and usability — it's in stagnation for over a year now.
This is of course my own opinion, but it makes sense: if a significant share of resources is spent on AI stuff, that share is not available anymore for other needs.
> Your first leadership principle is customer obsession: “Leaders start with the customer and work backwards”. > I’m your customer, and I’m begging you: please let me be a cloud engineer again.
However, as with many enterprise products, the author is not the customer; it’s the user.
The customers are the companies that buy AWS because it’s an essential technology for their strategy. When the whole tech world is talking about generative AI, they want to be there, and Azure seems to be ahead because of the MS deal with OpenAI. (even if they are not ahead, customers' perception matters most).
So basically, what Amazon is trying to do by making all of these conferences and announcements about GenAI is to send a message to their customers: we are ahead on the wave of GenAI and you can still trust that our products are going to help you be on the hype.
> I’m your customer, and I’m begging you: please let me be a cloud engineer again.
Only AWS knows how many H100 GPUs they have, how busy they are. How many people are paying for them, how many people want them and can't get them, and how many people just don't care at all.
It's possible that the focus on GenAI for Re:Invent 2023 wasn't based on any hard data like that, and is really just up to the whims of Adam Selipsky since Jassy moved over, but maybe someone who better knows their planning process can comment.
This opinion is based on admittedly anecdotal experience, but I’ve worked in a large range of domains on AWS over the years and by far the biggest AWS bills were for startups specialising in GenAI.
Startups doing machine learning on EC2 might have large bills, but that isn't what the AWS focus on generative AI is about!
Ironically, trying to compete like that, and then focusing on that, causes the problem they're trying to avoid IMHO. They're always going to be third as long as MS gets to serve OpenAI and Google has in-house AI talent building models that are top-tier competitive. And if you set this one farking thing aside, they're #1.
* ex. first time I've seen "FM" as an acronym is on the page you link. they mean "foundation model", which itself is a term I'm likely to see in the Economist, but never on HN. Colloquially, it means "big AI like ChatGPT"*
In private they are truly thirsty for AI applications they can write uses cases on that they even offer upwards of 100K credits for Gen AI purposes only.
I think the technical specialty that will be most at threat from automation by AI would be the exact job that he authored has -- solutions engineers that build commodity cloud infra on AWS, Azure, G cloud, etc.
Look at progressions and range of abstraction between standard sys admin IT work to serverless deployments, especially with IaaC tools.
You can describe your architecture to chatGPT and it can spit out a CloudFormation YAML. It will be rudimentary and poor, but I could see a Gen AI tool offered by cloud providers where al you do is describe your app and then deployed infra on your behalf, and optimize form there.
Not trying to talk down on folks who do this type of work, but sharing my opinion on where I think the author is ultimately coming from.
Can you point to any actual AI product in this space that functions? Everything I've seen is like, if you squint then it kinda looks like it's doing something, but it's actually producing something embarrassingly wrong, unsafe, or otherwise unusable. And no, having a SME repeatedly prompt until it does the right thing doesn't really make sense.
If we're just talking about hypothetical tools that someone could make, but haven't, we're talking about magic.
You might know the difference as an SME, but if you're not, and it passes terraform apply, it's getting used.
Today? No. But I think we will get here sooner than automating on any other type of engineering role.
So that’s not nothing.
