I wonder for how much longer we'll get to see (high quality) open models be shared freely, it's already quite a limited landscape.
Would there even be enough GPU power? Would something like it be doable with a reasonably realistic community effort? The human part of training could also be a part of this effort, which is yet another thing a single company is having to pay a lot of money for (well, I guess not anymore, as everyone is now helping out for free, by using the hosted LLMs).
Then, of course the problem of the data, not sure how far off the data private companies have bought, stolen, ...found is from what's available out there.
Curious if anyone more involved in with the topic has a thought or two about it.
