Will the Collapse of SVB End the LLM Hype Bubble?

2 pointspatientplatypus3y ago2 comments

So, question. Given that SVB has collapsed, will this mean that all of the funding for firms that are the chatGPT/stablediffusion versions of pets.com will implode? How much money has the VC/tech industry invested in tertiary firms that are using this technology as a simple plugin to an arbitrary optimization problem? Does anyone have any hard numbers?

2 comments

2 comments · 2 top-level

PaulHoule3y ago

No.

This library can have a Python dev up and running in a few minutes

https://huggingface.co/docs/transformers/quicktour

It supports 13 applications of transformers and text generation is only one of them!

I have been working on text classification on and off for 18 years, for the most part systems better than ‘bag of words’ have been elusive. I was working at a startup maybe 5 years ago where the technology was not ready and today it is.

You see some blog post where somebody writes a prompt for ChatGPT that makes it do a task with 70% accuracy; often I show a simpler model (cheap to run on my own computer) 1000 examples (a day of work) and I am getting well above 90%. It is true that ChatGPT is a superhuman bullshitter, and it attracts the same simpleminded and indolent folks that were attracted to NFTs but there are a lot of people doing amazing things with transformers and not talking about it because they don’t want competition.

anigbrowl3y ago

I don't know; are many of these the beneficiaries of funding largesse, or is everyone rushing to create an 'AI powered dog washing startup' (to use someone else's colorful descriptor) in hopes of racking up users and then turning those numbers into some cold hard cash?

I don't think LLM progress/excitement/hype is going to slow down significantly. They're a breakthrough of the type that only comes along once a decade or two and leads to a phase change. What's happening in AI now resembles the social media landscape back in the early 2000s.