> 3 months later, reading a paper while on board a boring flight home, I have my answer.
I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share.
Any tips or sites for someone interested in picking up more science papers to read.
1. Ideas That Created The Future[1]. It's a collection of fiftyish classic CS papers, with some commentary.
2. Wikipedia's list[2].
3. Test of Time awards[3]. These are papers that have been around for a while and people still think are important.
4. Best paper awards[4]. Less useful than ToT as not every best paper is actually that good or important, and sometimes the award committees can't see past names or brands for novel research.
5. Survey Journals[5]. Students often get their research started with a literature review and some go the extra step to collect dozens of papers into a summary paper. I subscribe to the RSS feed for that one, and usually one or two are interesting enough to read.
6. Citation mining -- As you read all these, consider their citation list as potential new reading material, or if an old paper leaves you wanting more, use Google Scholar to find a papers that cited what you just read.
[1]: https://www.amazon.com/Ideas-That-Created-Future-Computer/dp...
[2]: https://en.wikipedia.org/wiki/List_of_important_publications...
[3]: https://www.usenix.org/conferences/test-of-time-awards
A somewhat similar problem arises with test-of-time and best paper awards. To elaborate on my complaint, imagine the exaggerated case of someone trying to understand modern science by intensely focusing on the work of researchers who won the Nobel Prize. Clearly all very important work, but understanding the 1990 Physics Nobel Prize (on electron-proton scattering) is of no use to understanding the work for which 1991 Nobel was awarded (complex systems and polymers).
There are two things that (I'm assuming the OP's field of interest is computing) a CS education provides: At the undergrad and in the early stages of grad school, breadth of topics, and their modern synthesis. You don't spend much time reading papers (at least in an undergraduate education), but you understand the basics, and get a feel for the problems considered and the sensibilities of researchers. In an intermediate-level graduate seminar, you pick a narrow topic, and focus on papers in that topic. The first papers in the area (like Dijkstra's papers on distributed computing), the best / most important papers in the area, and the latest papers on topical interests (like Merkle trees and blockchains). There is thematic and technical continuity from one paper to the next, and you start to understand the the story being told. Then, late in graduate school, and in the rest of one's professional career, one starts reviewing papers that haven't even been published. At this point, you see the story being written: the steps and the missteps, and the memorable and not-so-memorable papers in a field. To truly understand a field, one needs to read not just the great papers, but also the middling ones.
And one needs to concentrate on a topic. The thing about a forum such as HackerNews is that for every topic of interest, there's likely a person here who's an expert in the area, but it is easy to confuse that observation with the much stronger claim that there's a person here who's an expert on every topic. The last of those people died in the mid-20th century, if they ever existed.
Do they? I suspect that most don't, and those that do are either in specialized careers or are engaged in some kind of scientific research.
Some interesting research gets disseminated via Twitter and chatrooms. Or maybe you follow a podcast that mentions new research. But you might also be following new publications from a handful of reputable journals, or following an Arxiv category, or looking through new conference papers. It's very easy to get overwhelmed with new research to read, and not knowing what's worth your time, unless you're already very familiar with the field and well-versed in the material.
I probably averaged 20 a week back in March when open source AI was booming in the wake of Llama and on the heels of GPT-4.
Once upon a time, I was in condensed matter physics. I was (and remain) interested in a very specific niche within that, and I read a small handful of the papers that were published each week. I’m not actively researching or publishing anymore so I cap this to one or two per month now, and mostly scan over them to see if anything piques my interest.
I was still interested in condensed matter as a whole, at the time, and attended group seminars once a month to see what other people were currently excited about - there wasn’t any hope of me reading a cross section of all condensed matter papers because there is far more published per week than I’d be physically able to even glimpse at, and most of it is stuff I don’t understand or particularly care about.
I was likewise interested in physics as a whole, and twice a year I’d attend a departmental seminar and see what people in the entire department were interested in. Most was far over my head, but it still directed me to a small handful of papers that I’d read for the hell of it. Of course, I couldn’t do this without first hearing people review the research. There’s far more published per day in physics as a whole than I could read in a year, and most of it I’d find unrelatable and uninteresting.
I guess where I’m going with this is that anyone with a specific interest is already reading papers. It’s their job. Anyone with a general interest would find actively pursing paper hunting to be a waste of time with a ridiculously bad signal to noise ratio. Instead, they should use channels that align closely with their own interests, through which they can get recommendations to read papers from the aforementioned specialists who have already filtered out much of the noise themselves. At that point, they should actually read the resulting papers.
There is another trick, though, and that’s to find an individual who publishes two unrelated pieces of work that you find interesting, then read their work and maybe those of their coauthors. Be careful, though, because this is a slippery slope to specialising, after which you’ll find yourself back at the point where you don’t aren’t following 99.9% of the stuff you wanted to follow in the first place.
Honestly many papers are written in a way that's hard to approach and difficult to understand unless you're prepared to reread them a few times.
You're better off just getting your science news from actual science communicators and not the raw source.
Highly doubt that. It’s very hard to actually read scientific papers when you are not actively doing research.
You can’t just read a research paper in isolation. It’s next to useless. You need to understand its context, where it stands with regard to its sources and what it brings which is actually new and valuable. It’s nearly impossible to do properly if you are not fully immersed in a research subject.
I don’t even know how you would scheme introduction and sources to filter articles which are immediately obviously useless without being immersed in a field.
I guess you can obviously go though lists of papers which have be deemed worthwhile by someone else or got prices. That solves the filtering issue but then nearly every time you will be better served reading a text book presenting the ideas in said papers.
I fully expect the HN readership to contain a significant amount of students and actual researchers which explain why you encounter people reading papers but these people aside I would be surprised if the habit is common.
And even then, sometimes you don't understand or care about their procedures, and you just want to look at the pretty results (check out this song they generated using AI!). There's even a very popular YouTube channel that focuses on this (two minute papers).
Finally, you usually hear about these cool papers via Twitter / X
PS: Best papers I have seen are from deepmind where the approaches usually described are novel, varied and path breaking. Worst ones are - well no names but those that just use training and eval sets generated by GPT4 and try to prove things empirically
I completely disagree with that. Spelling out math is literally something out of 12th century. It just hinders understanding, if you have basic STEM-level math literacy, which anyone who reads an ML paper is implied to have (how could you seriously study linear algebra and calculus without it?).
Math may actually be the first thing you recognise in a paper, which can help you cross-reference the text to understand it.
When google doesn't return a good result to a specific question, switch to scholar.google.com and start reading abstracts. Everything may seem like an opaque maze at first, but just keep reading and patterns start emerging quickly and become useful.
https://news.ycombinator.com/item?id=37006967 suggested some avenues for finding some classic papers. The follow-up https://news.ycombinator.com/item?id=37007360 pointed out some circumstances where that's not ideal. But in the process, implicitly assumes that you want to become familiar with current research, instead of just enjoying classic papers for some other motivation.
I mostly read papers in mathematics and computer science. For other disciplines I mostly rely on pop science, like Slate Star Codex or Money Stuff and blogs. There's also The Monad Reader (https://wiki.haskell.org/The_Monad.Reader) if you are interested in functional programming.
There's various blogs with interesting articles. Eg Vitalik Buterin has great stuff, like https://vitalik.ca/general/2017/11/09/starks_part_1.html and he links to the original papers. (I have no conclusive opinions on whether crypto-currencies are useful or good for the real world, but I do find the math behind some of them endlessly fascinating. Especially zero-knowledge proofs.)
Wikipedia is also often a good starting point. Whenever you read about a random topic, Wikipedia usually has an article that comes with plenty of references. Eg https://en.wikipedia.org/wiki/Forth_Bridge#References links to http://www.bath.ac.uk/ace/uploads/StudentProjects/Bridgeconf... and down the rabbit hole you go.
https://gwern.net/ also has great write-ups and links to original papers.
You learn pretty quickly that if you want answers, it's better to just go straight to the source, rather than have it filtered through someone else, where the message can (and often does) get twisted.
What are you interested in reading about? Maybe some people can recommend you some papers to start with.
> Any tips or sites for someone interested in picking up more science papers to read.
Personally, the older I get, the more bored I've been getting with the level of information that "crosses my desk".
Eventually I basically stopped reading blogs et al and started getting my insights from books. Those books would often mention papers. Then I noticed a lot of books (and deep well-researched podcasts) mentioning the same papers. So I started reading those papers.
When you read a couple papers, you notice most of them reference a bunch of other papers. Now you have an exponentially growing queue of interesting papers that you'll never get to. Mission accomplished.
The main trick is to read stuff you're interested in knowing and understanding. Many papers can be quite difficult to read, but getting through a single paper will fuel your brain with more valuable information than 2 weeks of "the internet". In my experience at least.
Ultimately, life is short and papers give you a better information density return on your time than almost anything else. Even the bad ones.
Edit: It seems to've gone on indefinite hiatus but there's a lot of backlog already there and some of it's really quite fascinating.
But I don't see the point of reading a scientific paper unless you're actually curious about a specific topic. They are often hard to read, dense, have so many field-specific jargon that if you're new, you won't be able to read one paper and grasp everything. You would have to read references, or a book/blog that summaries core points.
So find a specific field you're interested in, find a good book/blog/homepage/tutorial/video to get your basics going so that when you start reading papers you won't be completely lost.
Then find a highly cited survey paper to understand what progress have been made beyond what is now basic. Then you can follow your curiously along that survey, decide a branch of research to read upon. You'll probably then realize that a few labs research/publish a lot in a specific direction. Now you can follow those professors (Twitter, Google scholar email notification) to keep up to date. By reading a lot you'll also start to notice papers that are "published just to get my PhD" and soon enough you can just read abstract + intro/result to judge if it is valuable or not.
If ML/LLM is your curiosity probably Lillian Wengs blog [2] is a good start for tutorials / surveys.
[1] https://news.ycombinator.com/item?id=24986727
[2] https://lilianweng.github.io/
Edit: direct link [3] https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPape...
Also youtube and code: Attention is all you need is not a nice paper to read for Joe programmer, but you can understand what it is doing by watching karpathy and reading his code (or someone else who has implemented it, Llama for example). But you need to do some basic torch training first (karpathy again!)
Some sense of urgency helps. Most people will have a medical ailment or physiological issue of some sort. I promise you that there exist useful papers on it.
To get a cold start look for a “survey”, “literature review”, or “systematization of knowledge” papers. Those organize a lot of papers, check out the ones that look cool and read the abstracts.
Rinse and repeat for five years and you get a phd.
When google doesn't return a good result to a specific question, switch to scholar.google.com and start reading abstracts. It'll seem like an opaque maze at first, but just keep reading and it'll start clearing up pretty quickly and become useful.
I imagine this routine comes from people with research backgrounds, where browsing papers is the academic way of googling around for answers.
if they're not too boring, and you're not doing anything important, you'll read it for fun
You only get divergent results if there is some other source of state or entropy: not zeroing buffers correctly, race conditions, not setting rounding mode flags consistently, etc…
From the quality of the code I’ve seen being cobbled together in the AI/ML ecosystem I would assume all three of those issues going on, and maybe more.
(In this particular case, the order in which the numbers are summed up is non-deterministic due to GPU parallelism, which may change the result slightly.)
I would generally refrain from insulting other people's code if you don't know much about the system it's written on.
.
Editing here since all the replies to this are mostly saying the same thing: Yes, CPUs can also be parallel and it can happen there as well, but unlike a CPU where most instructions on their own are deterministic, CUDA provides primitives that aren't. This is very much by design (as they're faster than their deterministic counterparts), and I mostly just take issue with how parent phrased this as a bug caused by bad code.
The behavior in the linked article has to do with the use of atomic adds to reduce sums in parallel. Floating point addition is not associative, so the order in which addition occurs matters. When using atomic adds this way, you get slightly different results depending on the order in which threads arrive at the atomic add call. It's a simple race condition, although one which is usually deemed acceptable.
It literally says that the GPU is deterministic, the NVIDIA libraries on top are deterministic, but it is Tensorflow that introduces variability (errors!) for “performance”.
My argument is that it is the AI/ML code that is introducing non-determinism, usually by sacrificing repeatability to gain performance.
That's precisely what's happening here. Tensorflow introduced a "harmless"[1] data race to improve performance by not having to use a deterministic but slower algorithm.
The individual floating point computations are deterministic, it's the multi-threaded design on top that's introducing the variability in the output.
[1] Used to be harmless, but cutting corners like this will make it nigh impossible to repeatably validate the safety of future models like GPT5. That seems pretty dangerous...
https://pytorch.org/docs/stable/notes/randomness.html#avoidi...
Unfortunately, determinism across devices or even driver versions is not that easy. You'd have to write your own BLAS kernels using only basic operations, which are guaranteed to follow IEEE 754 semantics.
https://docs.nvidia.com/cuda/floating-point/index.html
One gotcha are fused multiply-adds, which the compiler may or may not introduce, so you have to wrap all your floating point operations with __fma* intrinsics to make sure the compiler does not interpret them differently.
If so, this exact same issue happens in CPU code as well: have two or more threads, run the program many times, observe different interleavings that expose race conditions which (depending on the algorithm) may or may not produce different results. This can happen even if you don't use floating point, and has nothing to do with floating point non-determinism itself. For example, have a thread print "Hello" and another thread print "World"; even without tearing, you may see either Hello World or World Hello on the screen.
Now, proper floating point non-determinism happens in two cases. One is that when you run the same code in two different architectures you could have different answers (because of rounding modes, or because some architecture doesn't support subnormal numbers or signaling nans, because transcedental functions like sine are implemented with different accuracy, etc). In this case it's deterministic when run the same in the same machine, but may run differently in another machine with a different architecture.
The other case is that some "optimizations" actually break your code if applied carelessly (you enable those broken optimizations with -ffast-math in C for example). Among other things, this may break numerical stability of algorithms like Kahan summation. And, if you let the compiler decide which exact optimizations will be applied and in what order, you get non-determinism between different compilers. So in this case it's deterministic when compiled with the same compiler, but may run differently with another compiler.
[0] https://stackoverflow.com/questions/50744565/how-to-handle-n...
Well, the general state of how utterly shoddy most of the code in the AI/ML ecosystem is is observable to anyone trying to follow a guide on how to set up Stable Diffusion on AWS. It's a fucking mess of trying various combinations of driver versions, Ubuntu kernel versions, Python versions, and the fact that Python requirements.txt (similar to NodeJS) doesn't pin versions of transitive dependencies doesn't make it easier because it makes for very brittle and not reproducible builds/guides. Oh, and at least some of that stuff won't work without root.
Yeah I'll keep AI shit cordoned off in its own subnet.
People are rushing like crazy to get there first with X for AI all over the place, it would be pretty shocking if there weren’t wires sticking out everywhere.
I don’t think that says anything positive or negative about the hackers involved.
> CUDA (or Compute Unified Device Architecture) is a proprietary and closed source parallel computing platform and application programming interface (API) that allows software to use certain types of graphics processing units (GPUs) for general purpose processing, an approach called general-purpose computing on GPUs (GPGPU). CUDA is a software layer that gives direct access to the GPU's virtual instruction set and parallel computational elements, for the execution of compute kernels.
Is the issue we're discussing because of the GPU or is it because of choices made in software libraries?
The parent is right, there is a deterministic, reproducible way to solve these problems, so if determinism is a desired or expected property, then this is a bug. It's not an inherent problem like you make it out to be. The fact that "workarounds" are given in what you link prove this.
Calling GetTimeOfDay() could do it.
Clock frequency drift between multiple processors could it.
Quantum computation relied on Quantum mechanics.
Quantum mechanics are not deterministic.
So, Quantum computers are not deterministic.
Therefore, unless P=NP, not all computations are deterministic.
I had absolutely no idea GPT4 was nondeterministic and I use it about 2 hours a day. I can see why a cursory looking wasn't cutting it, they "feel" the same in your memory, a lot of similar vocab usage, but are formatted entirely differently, and have sort of a synonym-phrase thing going where some of the key words are the same.
The non-deterministic outputs are really similar, yeah, if you check the gist examples I linked https://gist.github.com/152334H/047827ad3740627f4d37826c867a.... This part is at least no surprise, since the randomness should be bounded.
I suspect OpenAI will figure out some way to reduce the randomness at some point, though, given their public commitment to eventually adding logprobs back to ChatCompletions.
Such as?
Is it saying that part of its more-efficient inferencing relies on mixing tokens from completely-separate inputs – eg, from other users? And then, depending on what other inputs chance into the same grouping, the relative assignment-to-'experts' varies, and thus the eventual completions?
If so, I'd see that as not just introducing non-determinism, but also potentially making the quality of your responses dependent on how-many-concurrent-requests are fighting for the same expert-allocations.
(For example, maybe the parts of the system best at translating/interpreting Hindi give worse results during peak usage hours-of-the-day in India, when the most concurrent inputs are competing for that same competence.)
Perhaps also, this is another possible explanation for perceived quality-degradation over time. When certain tests were reliably succeeding earlier, there was less congestion for the relevant 'experts'. Now, with more concurrent use, those same tests aren't as reliably winning as much of relevant 'experts' effort.
This may also suggest a bit of a quagmire: on whatever domains some sub-experts seem impressively good, initially, even more proportionate use will be attracted. But such new congestion means all the copycat use no longer gets the same expert allocations – and thus the initially-impressive performance degrades.
(And if the effect is strong, & known-but-undisclosed-by-OpenAI, does it amount to a bait-and-switch? Attract users with unrepresentative excellence on an initially-uncongested Mixture-of-Experts system, but then offer them the lower-quality results from a more-congested system.)
I think it groups the batch up differently, so if I have a batch of 10, and it groups it up into 2 groups of 5, if my prompt makes it to the second group or 1st group I get a different answer. But if I’m in the same location in the batch, then I get the same answer.
The whole batch is deterministic given the same batch (sequences and ordering), but if you shuffle the batch then you lose that determinism.
Wait, am I misunderstanding you? I feel like I've had a head injury or something, because I've never heard of an open source LLM that's as capable as GPT-4 (in most scenarios).
That's only true assuming you habe enough data to train a domain-specific model / expertise to train it and test it correctly.
I've encountered cases where an image recognition task could be accomplished well with a very general model like CLIP, but people still fine-tuned another model on their own small data set because that's considered better.
A domain specific model might be more likely to fail on weird outliers not present in the small domain specific training data.
> could spell disaster for OpenAI
Nah I don't think so. They are not all in on one specific model architecture. If the current architecture is found to have serious unfixable flaws then they'll just change architecture.
This is not even close to true for Language models.
The LLM space is just different. There's no guarantee a fine-tuned model will beat a bigger generalist one.
https://github.com/bigscience-workshop/petals
I don't understand how petals can work though. I thought LLMs were typically quite monolithic.
the language models in our heads have not caught up to the ones in our browsers.
as the similarities and associations crystallize a bit better, it won’t look so hard.
bookmark this if you think it bullshit. eight months.
Also I strongly suspect that at least in the case of -me-, an article that was easier for me to understand wouldn't make the underlying theory any easier for me to judge.
(on the upside, at least I -did- understand and appreciate your self deprecating pun :)
Interestingly, on another discussion there was a claim that setting the temperature to 0.0 made gpt-4 deterministic: https://news.ycombinator.com/item?id=36503146
text-davinci-001 and text-davinci-002 were trained through FeedMe and SFT, while text-davinci-003 was RLHF; the models themselves have more variance at high temperature.
Hold up, does this mean that under heavy load the results change? Does this explain why it sometimes feels like the output quality changes?
>In the MoE approach, different "experts" or portions of the model are selected for different parts of the input data. The selection of which experts to use can be influenced by several factors, including the specific content of the input data, the order in which data is processed in a batch, and possibly even minor variations in the internal state of the model.
>This "expert selection" process introduces a level of stochasticity, or randomness, into the model's operation. For example, if you process the same input data twice in slightly different contexts (e.g., as part of different batches), you might end up consulting slightly different sets of experts, leading to slightly different outputs.
The explanation makes quite a bit of sense.
