How can I do that, and where can I download it from?
Though, I too, would rather be able to run the model myself.
I realize it's a charitable interpretation of their behavior, and I used to be very angry about them not being truly open. However, after playing with ChatGPT I think I am beginning to understand and even support their behavior. [1]
My personal sea change was realizing that giving dual-use tools to the global @everyone and hoping for the best might not be the greatest plan. I came to this realization thinking about bio-tech and GNC software, but it may apply to ML products as well. [2]
While I used to think universally/religiously "it should be open, will be one day anyway, etc," I now think about these things on a case by case basis.
[0] https://openai.com/alignment/
[1] https://news.ycombinator.com/item?id=33928400
[2] Imagine how many nodes this bot farm would have if it wasn't limited by the existing C&C bottleneck. ChatGPT is a productivity multiplier. This is productivity I want to put off multiplying as long as possible: https://news.ycombinator.com/item?id=34165350
Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able to manage context/understanding beyond this window length
(Specifically, AI Dungeon type games where ChatGPT is the DM and the human the protagonist, or vice versa. The most common failure mode seems to be that it forgets whether it's playing the DM or the protagonist. To be fair, it performs admirably well despite the limitations.)
>It's around 500gbs and requires around 300+gbs of vram from my understanding and runs on one of the largest super computers in the world. Sable diffusion has around 6 billion parameters gpt-3/chatgpt has 175 billion.
https://gist.github.com/navjack/32197772df1c0a8dbb8628676bc4...
I mean yeah after you set it up like this you still have to prompt engineer to get it to behave like a chat but I mean it's better than GPT - 2
HELLO FRED967, MY NAME IS DOCTOR SBAITSO.
I AM HERE TO HELP YOU.
SAY WHATEVER IS IN YOUR MIND FREELY,
OUR CONVERSATION WILL BE KEPT IN STRICT CONFIDENCE.
MEMORY CONTENTS WILL BE WIPED OFF AFTER YOU LEAVE,
SO, TELL ME ABOUT YOUR PROBLEMS.
Open communities with potential for involving yourself include Hugging Face and EleutherAI, the former perhaps more accessible, the latter an active Discord.
It's been a while since I spent time looking at them, I'm not sure if there is something you can easily get up and running with.
You probably won't be able to run (or especially train) them on typical desktops, though.
GPT-NeoX apparently compares favorably with GPT-3 on some measures, but ChatGPT is a 175 billion parameter GPT-3 model, and the big pretrained GPT-NeoX model available is a 20 billion parameter model. Could you rival ChatGPT with the right settings and training set and sufficient hardware for training? Well, if you want to try, you can with GPT-NeoX, and you can chosr whether or how to filter the output. With OpenAI’s models, you get what they give you, on the terms they are willing you give you access to it, with filters that exist tonprotect OpenAI’s image and liability.
To replace ChatGPT, no they are not good enough.
Since my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large models. Good luck!
"in Google sheets, I have dates in the first column. How would I make the second column indicate if daylight savings time is active on that date?"
ChatGPT will confidently provide non working suggestion after non working suggestion, but ultimately can't make this work.