For one, losing in a civil trial isn't the same as being subject to criminal penalties, so you can't self-incriminate unless what you're hiding would itself have been a crime - many contractual disputes have no criminal dimension whatsoever and so your liberty isn't at risk.

But more to the point, archive.org already has a copy of what you previously made public, and there's no reason to burden them with the expense of manually retrieving it as they're not party to the case, and it would be wasteful of the court's resources and archive.org's time to make them sue you for the expense involved.

Here's a more detail breakdown of how the 5th applies in a civil litigation context: http://www.litigationandtrial.com/2013/04/articles/attorney/...

The bottom line is that the 5th can help you stay out of prison, but you can't use it just to avoid losing a dispute.

The government has the right to use force in an investigation.

Things you have in your mind or - in some cases - in an extension of your mind (you're secretly actually guilty, you know where incriminating evidence is, or a safe combination or password that you haven't forgotten) cannot be compelled.

Things you possess (the contents of your home or gun safe, the thumb drive clenched in your fist, the robots.txt file on your server) can be compelled with a search warrant or subpoena. They are your things, and they could incriminate you, but that's not what the 5th Amendment is for.

Laconic.net is a resource I have enjoyed. Here is the 5th amendment passage: http://lawcomic.net/guide/?p=2470 or go back for a history lesson.

A subpeona was already issued and this information could have still been extracted from the Internet Archive directly, just at a higher technological hurdle. If the information was still on the plaintiff's servers they could have just extracted it from there as well (or any backups that they might have had laying around), since it was deleted they needed other ways to recover it.

robots.txt is assumed to only be used to block automated access to a website by search engines and other crawlers, not to hide it from human view as well (in fact, it makes sense to NOT use it to hide things from humans as it announces the presence of content that you are trying to hide). Removing a file that is only to be used to prevent crawlers from indexing certain parts of your site (because they are irrelevant for search results) is hardly self-incrimination (again, because you are advertising the fact that content exists at this location and you don't want it indexed).

It's available in the wayback machine, it was already made publicly available at a previous point in time. This is primarily reducing the courts burden.

Does self-incrimination apply to companies?