Okay so you build a knowledge graph on top of the internet archive. Now you are struggling to prioritize the resources necessary to capture long-tail content that doesn't mesh easily into popular corpuses. I imagine this would lead to the library equivalent of an echo chamber.

The curator being bandwidth-limited is not necessarily a problem if the problem you are solving is an overwhelmed audience in need of a curator. In other words, the Archive missing things may not really be a problem if the stuff is not missing is on average of value.

It raises the issue of governance of the curator, but the IA is already more transparent than Goole & co.

” Web content expands too quickly and too massively.”

If most of it is crap I would call not archiving it a feature.

There is a weird convoluted analogue to CERN particle detectors. They smash particles together and then image the resulting storm of particle contrails via detector that is basically a sandwhiched ccd detector (like you have in camera, but different) the size of a cathedral. Resulting in far too much data for any system to analyze or even store in the first place. Hence they need/needed to runtime filter the massive amount of particle trail signals and only pick out the critical ones.

If there is too much data you simply need to drop the parts you are fairly confident you don’t need.

There is no reason there should be only one internet archive, there might very well be parallel operations filtering a bit different things.

I guess it’s a bit odd Unesco does not already have a parallel effort.

would be nice if we could have a way to navigate just in the old web...