undefined | Better HN

0 pointschang12y ago0 comments

I had a similar moment of existential career crisis as OP.

I'm at the 20 year mark as well, in terms of developing software professionally. I've always felt like with new technology, I could grok at a high level how things worked. But LLMs like GPT seem like magic and I went through stages of initial astonishment -> despair realizing the potential impact it would have on the industry -> acceptance.

While I still feel uncertainty and fear about the future, as others have echoed, I'm realizing it's a tool for developers to use. We can either choose to accept it and understand how to work with it, or reject it. The things GPT can generate amazes me, but I'm finding that it's a good starting point or reference to build on... not a final solution. It will generate things that are sometimes completely wrong, and it's your own experience and judgement that has to be used to determine that. GPT cannot do that... at least not yet.

I think back 20 years ago and remember reading through a lot of physical books with occasional web searches landing on experts-exchange or random forums. Then came Stack Overflow and that became in invaluable tool, along with the ubiquity of free tutorials on YouTube and elsewhere. And now we have GPT which I'll ask if I really get stuck on something and it gives me new ideas to try. Perhaps in the near future, GPT is the tool that I'll use first.

I found this podcast episode helpful for me to process what I’ve felt: [Lex Fridman Podcast #376 – Stephen Wolfram: ChatGPT and the Nature of Truth, Reality & Computation][1].

It's an unsettling feeling (in general) to feel like a foundation you've built and live on could potentially be made quickly irrelevant. I'd like to say I have words of wisdom to get rid of that feeling but I don't. What has helped me is to acknowledge these feelings as valid, and then try to get clarity in what direction to move. It's not the foundation itself that's important per se, but it's the skills you've acquired in building the foundation that's more important.

[1]: https://lexfridman.com/stephen-wolfram-4/

0 comments

thaway_thaway342y ago

>> I've always felt like with new technology, I could grok at a high level how things worked. But LLMs like GPT seem like magic

This is exactly how I feel. I felt so out of my depth looking at the ML architectures and I could not make any sense of it. I thought perhaps, they get inspired by neuroscience for the layers etc.

But a friend who works on LLMs mentioned, the architecture of large ML models, are mostly experimentally discovered, not designed. If that's the case, that's even worse... it means an entire field which perhaps could replace me in future, doesn't even have a knowledge foundation for its breakthroughs, but just goes by experiment... I thought it was only the weights inside the model that evolves, not the architecture itself.

Which body of knowledge do I study then, and is it even engineering anymore? That's something else, which I am not sure if my programming experience applies.

The amount of GPU/Capital it takes to evolve such architectures, run such experiments has to be prohibitively expensive.

ryandrake2y ago

Checking in with the same feeling. If I had to do an interview and they asked me to sketch out on a whiteboard a high level diagram of how anything from the last 20 years of computing worked, I could probably muddle my way through it. A 3D engine, a database, a word processor, a web site with a REST API, you name it. It might not be 100% right in the details, but I could at least describe it in the general sense and talk about the constraints of such a system.

If you held a gun to my head and asked me to tell you even at a sky-high architectural level (let alone in any detail) how ChatGPT worked, well... tell my family I love them. This is the first time in my 20+ year career I have felt like some computing thing is total unexplainable black magic.

DougN72y ago

It’s OK. From what I’ve read, nobody actually knows how it works. I mean we know we have layers and weights, but how those emit what looks like intelligence is not understood by anyone.

j / k navigate · click thread line to collapse