One thing we do know for certain is that it is necessary, it is needed in "dumb" animals as well as in you and I. If an animal can't sleep it will eventually die.
I don't think that applies to the activity described in the OP. Does their LLM "die" if it can't perform the function described?
That turns out to be un-settled science. No human has ever died from lack of sleep.
People point to “fatal familial insomnia” as a counterexample. But they die to the disease, not the lack of sleep.
In a series of controlled experiments, rats and fruit flies did die from lack of sleep. But no one has yet proven that it holds true for vertebrates except for rats.
In other words, it could be true that “among vertebrates, only rats die of sleep deprivation.”
So “if an animal can’t sleep, it will eventually die” is actually quite hard to prove, and depending on how you look at it, somewhat easy to disprove by the fact that rats and fruit flies were so difficult to kill from sleep depravation alone.
Personally I’m skeptical of the rat study too. Claude amends this:
> What they did not establish: the mechanism. On autopsy, “no anatomical cause of death was identified.” The rats showed weight loss despite eating more, body temperature problems, and skin lesions, but nothing that pointed to a clean cause. So no, they could not say a rat “died from sleep deprivation alone” in the sense of identifying what sleep loss did to the body to kill it. They showed a strong association under tight controls, not a proven causal pathway.
As far as I understand it, there is a disease that destroys your brain's ability to produce sleep. Once you have it, you suffer total, progressive insomnia and die within roughly 6–18 months. Scientists debate whether it's the underlying brain damage or the sleeplessness itself that causes death, but the two are inseparable in practice, and sleep deprivation is considered the leading candidate.
Separately, the longest anyone has stayed awake under controlled conditions was 11 days, which produced severe cognitive impairment, paranoia, and hallucinations; suggesting the body deteriorates rapidly without sleep.
It's probably not wise to state your original claim as established fact.
There is quite literally no prion disease that isn't fatal.
Sleep does a lot of very important things that we probably wouldn't live long without, but it really is unclear to what extent sleep is necessary for them. If we had enough knowledge, could we trigger all the things sleep does without invoking sleep itself ? Perhaps sleep is just a very convenient mechanism.
> People point to “fatal familial insomnia” as a counterexample. But they die to the disease, not the lack of sleep.
It’s a prion disease. It’s established fact that they don’t die from the lack of sleep.
We know more generally that people who get decreased amount of sleep suffer increased rates of physical and mental health issues.
It is not a very big leap from "causes permanent damage" to "enough permanent damage can cause death" and of course, keeping someone awake until they are hurt or killed is deeply unethical, so even if it could be proven in other species, you'd still be here arguing that 'they aren't humans".
For something so incredibly difficult to do (die from lack of sleep) it’s frankly crazy that most people here are saying it like it’s fact.
It’s not a small quibble to point out that the central argument (“animals need sleep or they’ll die”) may be mistaken.
Once energy conserving "sleep time" exists, the genome can postpone or schedule for activity during these times of day, if it turns out to be more effective somehow.
Meaning: it might just provide a big advantage.
I don't want to overextend and assume that any advantage extends to LLMs. That rest-and-recuperate advantage might also extend to LLM-based AIs. Or maybe not, and the rest-and-recuperate is mainly useful for biology-based organisms. But there is some logic to it.
> The function of sleep in animals is largely obscure.
In my understanding, it's well-understood that sleep is used to consolidate and store long-term memories (amongst other functions, like cell and muscle repair). They've found this memory-consolidation-during-sleep even in relatively simple animals like bees.
What is described in the OP is therefore not a specific characteristic of sleep. It may however be a "useful" rhetorical device.
I do however object to the extensive use of such rhetorical tricks in the conversations that surround LLMs. For example, why does a consumer-grade LLM display "thinking" while it is actually sending data from my computer to some datacentre, processing it, and sending the result back? Equally, why does it output human-emotive phrases such as "sorry" when such computation is revealed to be incorrect?
Such rhetorical tricks, and more, likely underlie to a large degree the popularity of LLMs, despite their actual performance being clearly below what the rhetoric implies.
You're talking about different things: biological necessity and evolutionary benefit.
You can find out about the former by preventing an animal from sleeping (but otherwise provide all other needed things), and seeing if it will eventually die.
That is actually almost impossible to do. The rat study was as close as we’ve ever come, and it’s still debated whether the rats died due to lack of sleep or some other mechanism, since the autopsy couldn’t confirm a cause of death. (It could have been due to the way the experiment ran, for example, not the lack of sleep.)
If you don't periodically clean the context, an LLM effectively goes insane in terms of outputs.
If the LLM were fully controlling a physical system (like a robot body) that contained it the resulting insanity of an ever-growing, never cleaned context would likely result in some sort of death-like event.
It's still weak, though. An LLM without constant human input is likely more similar to a bicycle that starts to lose its gyroscopic balance as it moves more slowly, a human can however keep a stationary bicycle upright (while riding it).
Very few animals fail to eventually die even with as much sleep as they want.
But before death, there is a loss of cognitive function from sleep deprivation, and we observe this too with AI whose context windows get too full.
While we don't know very much about sleep, my understanding is that we do have a long list of things that we do during it, we just don't really understand if sleep is necessary for each of them or simply a convenient opportunity for it.
There's lots of things biology does in response to easy-to-detect proxy signals instead of the real thing they care about: Our sensation of needing to breathe more is based on too much carbonic acid in our blood, not lack of oxygen, which is why in general nobody is allowed in an elevator with a liquid nitrogen dewar; Our natural distaste for incest is based on who we grew up with, not our actual DNA; Get too cold and some people suddenly feel warm and want to (and some do) take all their clothes off even though that would just make them hypothermic even faster.
Being asleep may trigger the things we need to get done, but that doesn't mean sleep is *fundamentally* necessary for the things we need to get done. It could be just that it happens to be the way our biochemistry is wired, and we may find some other way to trigger those things.
The quotation given by djeastm would by my guess for what a dream is, and why we have them. But we don't spend all our time asleep, dreaming. And I'd be the first to say that my guess isn't worth much, as I'm not a brain scientist.
We now have evidence for REM sleep in spiders(1). Our last common ancestor with spiders predates the development of nervous systems. This strongly suggests that sleep (and specifically REM sleep) serves some function important enough that it has independently evolved in both protostomes and dueterostomes. (And probably multiple times within the protostomes, being present in both cephalopods and jumping spiders.)
There may be some commonality in the origin of the ion channels, but I'll lay money that the requirements for sleep are more of a result of general information processing requirements.
So, whether the LLM "dies" in any sense may or may not be important for what "sleep" is defined to be in this article. It's quite possible that sleep also affects endocrine system in animals or hormones etc... and that's what's causing death, not necessarily anything to do with how brain functions.
This just obscures the conversation IMO. We know a great deal of functions that sleeping performs. We don't know everything. For some reason this prevents us for using this word in a computing context? What do you think about sleep(1) in Unix?
Also..
> Does their LLM "die" if it can't perform the function described?
what "death" means in the context of a computer program?
Also, there's different kinds/stages of sleep, which probably perform different functions.
For instance, REM may do something like the GP describes, consolidating memories and processing learning. Deep sleep may do something else (I vaguely recall some stage of sleep is used by neurons to clear certain waste products).
It dies in terms of usefulness if it can't stay up to date with new knowledge. That is, it will no longer be used and thus effectively die off.
"Sleep" is just used in their context to describe a non-interactive mode and they didn't lean heavily into zoomorphic - I think you mean - parallels.
You're grinding an axe on a single term. What is your broader hangup with them using the term "sleep"?
> Does their LLM "die" if it can't perform the function described?
We're reaching an age where LMGTFY should now be Let Me LLM That For You. Have you tried asking an LLM this question about the article? I believe it answers it very well.
