Aaron Swartz's FBI File Released (opens in new tab)

Working with that organization in particular will teach you very quickly about the kind of immense roadblocks that exist to even the more "moderate" of conspiracy theories. Not only are they beholden to absolutely everyone, they don't have time to make things up or bury facts. Obviously there are exceptions, but this and most other government agencies are just loose conglomerations of individuals doing a never-ending amount of thankless work with the reward of being scrutinized and accused of every possible crime at every possible turn.

I hold government and its agencies in no high regard, because people are people and we all suck in our own way, but suggesting conspiracy at certain levels is just laughable.

Of course, any one who is interested enough to file for documents may already know what he/she wants and will call you out (with a lawsuit) if it seems you've redacted things unnecessarily. But for other situations, it seems unlikely that the requester will either not go through the trouble of suing or not even know if what was redacted is worth suing over.

An example of a high-profile amusing snafu occurred during the Rod Blagojevich trial when someone using Adobe Acrobat forgot to complete the redaction process, allowing redacted text to be copy-pasted into another text editor: http://capitolfax.com/2010/04/22/blagojevich-hurls-allegatio...

Yes, that was a technology-error rather than one of judgment...but technology errors are much easier to double-check for and yet it's not an uncommon error in the digital age of public documents.

(for those situations when a conspiracy might be necessary)

The situation with the Social Security Death Master File (SSDMF) / Social Security Death Index (SSDI) is very similar to the situation with PACER.

List from Nov 2011, obtained legally by http://ssdmf.info/

https://thepiratebay.se/torrent/7193029/Social_Security_Deat...

The SSA would love to insist that redistribution is illegal, because the price of the SSDMF is outrageous and a cash cow, but they don't have a legal leg to stand on. The SSDI is public record, and because it's a database with no creative element, it's not copyrightable even if it were compiled by someone other than the government.

Official sources for the SSDMF:

https://www.ssdmf.com/

http://www.ntis.gov/products/ssa-dmf.aspx

There are of course auto-redacting solutions available, but due to the nature of the documents (scanned, inconsistent formats), and the potential consequences of either mis-applying a redaction or omitting one, manual verification is required.

"PACER normally carries an eight cents per page fee, however, by accessing from one of the seventeen libraries, users may search and download data for free."

And:

"The two accounts were responsible for downloading more than eighteen million pages with an approximate value of $1.5 million."

~18MM pages at $0.08/page ~= $1.5 million

So downloading a document is stealing printing fees from the courts, apparently.

My personal favorite part of the file so far:

"The Administrative Office of the U.S. Courts reported that the PACER system was being inundated with requests. One request was being made every three seconds."

I actually applied for a programming job with the Federal Courts in San Antonio about a decade ago when I was desperate for work. Cattle-call interview with ~100 easy questions, then they'd call you back that afternoon for a panel interview if you did well. I was criticized for dressing "too casual", despite wearing $400 worth of new clothes I couldn't afford. Apparently, suits are needed for programming, which also involves crawling around courtrooms installing equipment. Got the job offer a few days later for $55K/year. I mentioned that the newspaper ad said the position paid $78K, and politely attempted to negotiate. Five minutes later, I got an email saying they were rescinding the offer. Classy folks.

Think of it like a company selling GNU software. There's nothing prohibiting that, even though once someone has it, they are free to redistribute it for free.

The claim that the docs were worth 1.5 million dollars is based on the arbitrary price set for access to PACER, not any underlying value of the documents.

http://ycombinator.com/newsguidelines.html

"Please don't sign comments, especially with your url. They're already signed with your username. If other users want to learn more about you, they can click on it to see your profile."

edit: DoJ FOIA http://www.justice.gov/oip/foia-request.html