Also, it's nice to see this mentioned:
> For this 10k-word write-up, I spent around a month finishing up the main parts, and refining/editing it took an extra while. Writing this is indeed a painful process. I spent the entire day on the weekend and 4-5 hours during the rest of the week working on it for around two weeks.
It's the kind of behind-the-scenes effort that often goes unspoken.
Edit: Wow, shocked that I'm being received so negatively about this, non native English is not "bad". It isn't meant as judgement but praise for what he's accomplished.
git clone https://github.com/ggml-org/llama.cpp.git && cd llama.cpp
git checkout c0d4843225eed38903ea71ef302a02fa0b27f048 # Checkout a revision prior to the exploit fix in 1d20e53c40c3cc848ba2b95f5bf7c075eeec8b19
mkdir build-rpc && cd build-rpc
cmake .. -DGGML_RPC=ON
cmake --build . --config Release
cd bin/
./rpc-server -p 50052
nc -lvp 1337
pip install pwntools
python exp.py # From https://gist.github.com/retr0reg/d13de3fde8f9d138fe1af48e59e630a9
I am happy you read it and liked it; more glad you tried it yourself :D
If they're unlucky, the gap converges to zero because they get used to not having to do much work, "fail upwards" because of the raw intelligence, and then can't keep up when surrounded by similarly intelligent people who actually do the work.
Failing at something you were told you were extremely good at, and hence based your entire identity around, is extremely difficult and demoralizing. Some people can never really recover from that, and AFAIK depression / suicide isn't unheard of.
Definitely not a problem for this particular kid though, "lack of hard work" and "coasting" is evidently not what this person is about.
THe middle scenario is kids that do the work, but stay in their community for economic / political / class / "born in the wrong place" reasons. Their talents are mostly squandered, but they might end up doing something very significant for the communities they're part of.
This used to be extremely common, a medieval peasant or ancient slave would most likely stay in their village, regardless of how much of a genius they were. The modern world made it much less so, and that's something worth celebrating.
You lack all the foundational habits and are just used things working out naturally. There are zero consequences, only positive outcomes despite doing the bare minimum. And then it all goes to shit.
The trajectory of everyone ranged from early Facebook employees, a CMU CS PhD, to one literally going to prison for an exploit lol. You can never tell where life will take you.
If they're self-driven like the original author, they'll be good, they not necessarily need the gradient.
It's sad that hacked things that emerge first are way more popular than properly done projects that come later.
This standard of security is what made the exploitation such challenging and rewarding.
None of that is meant to take away from your effort or the success of llama.cpp, but I have spent quite a bit of time reading and working with the internals across layers and have a good eye for quality c++ patterns.