The funny thing is I still remember when my brain had spare capacity - I used to fill it with arbitrary facts like what page I was up to when I was reading dozens of books at once. When I was young I never read one book at a time or used bookmarks. I would just remember the page I was up to and whenever I came back to a book I just turned to the correct page. Of course now I have to use bookmarks and every book I read is at the expense of some part of my past :(
Technically, our brains do get full, even this article basically says so.
It's just that we don't experience error messages or shut down when that happens - instead, old memories are replaced with new ones.
The total capacity is limited, but the ability to create new memories is unhindered.
I wish we could choose what we forget, but it seems it's possible to choose what you remember - just relearn that stuff or recall it more often, then it will be at the top of the search results, so to say :-)
I'm finding almost everything to be much easier to understand now, compared to how it was when I was younger. Back then, I could focus, read at much faster speeds and my memory seemed way better, if measured by the amount of information I could remember. Entire texts were memorized with one or two passes. But then I'd have to spend time figuring out what to do with the information.
Now, my feeling is that wherever data I absorb is slotted right next to familiar concepts, so even though I read slower, the amount of 'post-processing' is much reduced.
It feels like a compression algorithm. "This new concept is like A and B, except for X, and Y, which are similar to Z". Sometimes, I caught myself saying that aloud. I've had to develop coping mechanisms however, to avoid jumping to conclusions, as my brain loves to do.
Entirely unfamiliar concepts are now way harder. From foreign languages (with different roots) to... Haskell.
old information is sometimes pushed out of the brain
for new memories to form
That said, a NN can suffer from the trying-to-be-jack-of-all-trades, ending-up-an-idiot problem.
But while you're alive? Assuming you don't just "add more space", to continue the analogy, it seems like what your brain does is probably closer to lossy compression, over and over again. And the important bits remain more-or-less readable.
Something more interesting to think about is the limits of http://en.wikipedia.org/wiki/Working_memory which you probably fill up on a day to day basis.
The brain already consumes something like 20% of our energy.
However this will be done through lossy compression (you forget irrelevant shorter term stuff, you forget details, things get abstracted and generalized etc.)
The human population has gone through more well-studied evolution than other populations; but, for example, novel species such as the London Underground mosquito are unlikely to be rare. You may claim that this is effectively a domestic species, but I claim that's just what we can easily study.
Human lifespans are not very much greater: chimpanzees live to be 60 years old, and they are 8,000,000-12,000,000 years of evolution away. The difference in average lifespan is primarily due to progress with infant mortality rate.
I'm terrified of the possibility that there could be a day when I hit my maximum capacity and have to forget things in order to be able to learn new things.
Of course, if we ever need to reconstruct every detail of old info, we simply cheat, in the confabulation sense [2], possible with disastrous consequences [3]
[1] http://oss.oetiker.ch/rrdtool/
[2] http://www.spring.org.uk/2013/02/reconstructing-the-past-how...
[3] http://www.scientificamerican.com/article/do-the-eyes-have-i...
Reminds me of this Married with Children episode I've seen ;) http://www.imdb.com/title/tt0642312/