Skip to content

Top New Best Ask Show Jobs

Show HN: Torrenthistory – an open and legal torrent indexer | Better HN

Show HN: Torrenthistory – an open and legal torrent indexer (opens in new tab)

(torrenthistory.org)

23 pointsretonato5y ago30 comments

30 comments

retonatoOP5y ago

There have been quite a lot of torrent indexers already, but most of them are sooner or later closed. So, I decided to create one more, simple enough (so it can work on its own on a relatively cheap VPS) and legal (so it would be much more difficult to close it). Most of the site data can be freely downloaded. Is there something you think any decent torrent indexer should have?

antman5y ago

Could the data be retained on something like IPFS so that cloning of the tracker could happen independent of its existence?

retonatoOP5y ago

I thought about some kind of distributed storage, but the technology is not there yet (=I cannot implement that easily, users cannot use it easily). The site uses a regular PostgreSQL database. Most data, though, is published on Internet archive in a standard format (https://torrenthistory.org/docs/open-data/). There are torrents as well, users can download and seed them. I will upload new archives monthly

Pin it now!

retonatoOP5y ago

Thanks again for all feedback. I added all recent feature suggestions on github (https://github.com/retonato/torrenthistory-public/issues). You can vote for them there (just give a thumbs up for the issue you like) and see the current priority here: https://torrenthistory.org/vote/

tal8d5y ago

It would be very nice if you could search based on file name, as you can with btdig.com. If you could somehow provide BRE without melting down, that would definitely put you way ahead of the other DHT index attempts.

retonatoOP5y ago

At the very beginning (of January) search was done even through filenames, but then:

1. It turned out, that such search requires quite a lot of disk space for index. 2. It is not very useful without showing the actual matched filenames. This requires more code, I didn't have time to create it at the time.

I added it here for tracking, sooner or later that should be implemented: https://github.com/retonato/torrenthistory-public/issues/1

retonatoOP5y ago

What is BRE? Search by regular expressions?

bartvk5y ago

What is this useful for? Genuine question.

retonatoOP5y ago

1. Checking if some torrent exists in principle (by searching for words in a name) 2. Searching for the same files in other torrents (when you click on the file size on some torrent page, it will show all other torrents, which contain files of the same size, it is pretty good method of searching for larger files) 3. Just as an open dataset, which contains a lot of torrent metadata and which you can use in your torrent related projects.

retonatoOP5y ago

Also, just to have a torrent indexer, which will be there forever. Other large indexers had almost 40-50 mln torrents, when they were closed, all that information disappeared almost overnight. In my case, the cost of running is very small, the legal threats shouldn't be the problem either (come on, it is really not possible to download anything from there). And even if something happens to me - others can continue where I left, since almost all site data is open, anyone can create a similar site in a month or so.

not sure what definition of "legal" they are using. I clicked Catalog, then 70, then 80, and found lots of "interesting" titles

> This site and all information on it is legal. There are no:

>

> - files

> - links to files

> - torrent files / info hashes

> - magnet links

>

> You cannot find them anywhere - neither on the site, nor in its source code, nor in the database backups, which are published here.

>

> You cannot download anything from here (really). There are just folder names, file names and their sizes (+some other metadata).

(HN's formatting is seriously bad)

retonatoOP5y ago

That's right, you can see torrent names and file names, your can search through them, but you cannot download anything. There are neither torrent files or infohashes there.

Can files be more specifically identified - more than just filename+size? I.e. hash(es)/checksum(s) or something like that?

Torrent seed files only contain hashes for "pieces" (typically 1-4MB each), which can cross file boundaries. There's no way to pull an md5 for a specific file without actually snatching it.

m-p-35y ago

BitTorrent v2 also includes a per file hash tree https://blog.libtorrent.org/2020/09/bittorrent-v2/

so I assume that if you end up with the same file in multiple torrents, you'll be able to grab the files from seeds of these other torrents.

toomuchtodo5y ago

Pull the files, derive the hashes, and then throw away the files? You only need to retrieve each file once to derive and store the per file hashes.

elastolin5y ago

Why does hosting a magnet link make a torrent index 'illegal'? After all, you're not hosting any actual content on your server.

retonatoOP5y ago

That's a very good question. I think that dialog in filesharing/linking court cases has moved from "link vs torrent vs file" issue to "does this site/person make possible bypassing copyright protection, and, if so, to what extent?" issue.

Part of the torrent community still holds to the old (and comfortable, I must say) idea, that files and links (no matter in which form, http or magnet) are something fundamentally different. Files can be illegal, links cannot. Links are just text, infohashes are just numbers and so on.

The reality, as I see it, is different:

1. If something makes bypassing copyright protection as easy as making 2 mouse clicks and waiting a few minutes - that "something" is illegal, the person which operates it also commits a crime. Indirectly, by facilitation (=providing aid), but still.

2. Even if the law in this area is too old or too general to specifically mention such cases - the court practice is still little by little moving in that direction (=files and links doesn't matter, those are technical details; bypassing copyright - does matter)

hehetrthrthrjn5y ago

So even showing the infohash would be illegal? That's all you really need.

elastolin5y ago

That makes total sense. A more pragmatic and less technical interpretation. I don't condone it seems that's the way we're headed.

It does seem like a slippery slope, though. Someone could argue that your site facilitates bypassing copyright for merely letting users know a particular torrent even exists. They would simply look it up by name in a torrent index that does share magnet links.

Thanks for the insight, and good luck with the website!

j / k navigate · click thread line to collapse