I saw a thread a while ago (linked in README) discussing how Wikipedia does a good job keeping track of the domains of websites like Sci-Hub or The Pirate Bay. Someone mentioned checking Wikipedia to find links to these sites, so I thought this would be a fun thing to automate!
To try it out, install an extension or modify your hosts file, then type in the name of a website with the TLD `.idk`.
For example: scihub.idk -> sci-hub.tw
Cheers!
It would also be useful for identifying cuts of meat, as US cuts and, for example, Italian cuts differ not only in name but in how they are made. Compare the images on this article for an example of what I mean:
How interesting. Bing doesn't do this, which leads me to believe it's not a matter of legality. Is Google simply electing to self-censor results that it'd prefer it's used not to know about? Strange move, especially given the alternative Google does index is almost definitely more nefarious.
As many have pointed out, this just makes it easier for actually malicious sites to get traffic.
[0] https://torrentfreak.com/google-downranks-65000-pirate-sites...
At the bottom of the page click on the DMCA complaint, you'll find all the URLs you shouldn't ever, never ever, click on~
I don't think everyone is seeing this, because sometimes I see this but sometimes not, seems to depend on the query. Searching for "piratebay" doesn't show it in the bottom (I live in an European country [also in EU]) and meanwhile, official thepiratebay website is blocked on a ISP level here.
I'm curious if that's the case for you as well, or if it's my ISP blocking (I wouldn't expect to see the cloudflare error if my ISP was blocking but I don't know).
I bring this up because if the site is unresponsive from wherever you're searching (or perhaps unresponsive for all, idk) then maybe it got de-ranked on google.
[0] https://blog.iusmentis.com/2017/06/19/eu-hof-verklaart-the-p...
- How do you handle ambiguity? e.g what happens when sci-hub.idk and scihub.idk differ?
- Aren’t you concerned by the fact that Wikipedia is open to editing by the public?
Arguably the thrill of uncertainty could add to the fun :D
With the increasing trend to pull data from Wikidata into Wikipedia, this is I think becoming less of an issue – even if nobody is watching the Wikidata item, if some vandalised property is exposed in a Wikipedia infobox, that increases the odds that someone will notice the vandalism. However, there are always going to be more obscure items which lack Wikipedia articles, and more obscure properties which don't get displayed in any infobox, and for them the risk of vandalism is greater. (Plus, it is possible for a Wikipedia article to override the data in Wikidata with its own values; this is done for the English Wikipedia Sci-Hub article, for example – Wikidata is including all the historical web addresses, Wikipedia only wants to display the current ones – I don't think it is technically possible yet to filter out just the current ones, so instead Wikipedia is manually overriding the addresses from Wikidata.)
I used Pushbullet's recipe for "Google acquisitions" up until the night I got the notification "Google acquires 4chan". After being perplexed for a bit and a few more "acquisitions" were made, I discovered the recipe just used Wikipedia's List of mergers and acquisitions by Alphabet[1] page as a source.
[1]: https://en.wikipedia.org/wiki/List_of_mergers_and_acquisitio...
> Not as trivially compromised as it sounds like it would be; could be faked with (inevitably short-lived) edits, but temporality can't be faked. If a system were rolled out tomorrow, nothing that happens after rollout [...] would alter the fact that for the last N years, Wikipedia has understood that the website for Facebook is facebook.com. Newly created, low-traffic articles and short-lived edits would fail the trust threshold. After rollout, there would be increased attention to make sure that longstanding edits getting in that misrepresent the link between domain and identity [can never reach maturity]. Would-be attackers would be discouraged to the point of not even trying.
https://www.colbyrussell.com/2019/05/15/may-integration.html...
What’s a “verified version”? Who verifies?
Edit: this is not DNS over wikipedia. As other pointed out, there is no DNS involved in the linked artifact. One option would be to show alternatives with dates and let user choose.
If an analogy was needed with a network service perhaps this is more like a proxy redirector than DNS.
Keep in mind: with this you will still be misdirected if your DNS/hosts file is pointing the name into a different IP than it should be.
> Resolve DNS queries using the official link found on a topic's Wikipedia page
@aaronjanse: you probably want to correct this. "Resolving DNS records" carry a specific meaning in that you have a DNS record and you "resolve" it to a value, which actually. You're kind of doing, in a way, I suppose.
I was convinced when I started writing this comment that calling this "resolve dns queries" is wrong. But thinking about it, DNS resolving is not necessarily resolving a "name into a IP-address" as @HugoDaniel in the comment I'm replying to is saying (think CNAME records and all the others that don't have IP addresses). It's just taking something and making it into something else, traditionally over DNS servers. But I guess you could argue that this is resolving a name into a different name, that then gets resolved into a IP address. So it's like a overlay over DNS resolving.
Meh, in the end I'm torn. Anyone else wanna give it a shot?
This project is still doing key -> value. It just fetches the value from Wikipedia first, much like your normal dns servers have to fetch non-cached keys from their sources (other dns servers normally)?
- Domains used by the site (first party)
- Domains used by the site (third party)
- Methods allowed per domain.
- CDN's used by the site
- A records and their current IP addresses
- Reporting URL for errors
Then include the public keys for that payload in DNS and in the APEX of the domain? Perhaps a browser add-on could verify the content and report errors back to a standard reporting URL with some technical data that would show which ISP is potentially being tampered with? Does something like this already exist beyond DANE? Similar to HSTS maybe the browser could cache some of this info and show diffs in the report? Maybe the crypto keys learned for a domain could also be cached and warn the user if something has changed (show diff and option to report)? Maybe more complex would be a system that allows a consensus aggregation of data to be ingested by users so they may start off in a hostile network and some trusted domains populated by the browser in advance, also similar to HSTS?
[1]: https://en.wikipedia.org/w/index.php?title=Equifax&diff=9455...
Edit: Unrelated to this issue, but I have a more general idea for the kinds of inputs this extension may accept. It could be an omnibox command [0] that takes the input text, passes it through some search engine with "site:wikipedia.org", visits the first result and finally grabs the URL. So you don't have to know any part of the URL - you can just type the name of the thing.
The rust server set up with dnsmasq is a legit DNS server though.
Overall this is a nifty hack and I like it a lot. Wikipedia has an edit history, and a DNS changelog is something that is very interesting to have. People can change things and phish users of this service, of course, but with the edit log you can see when and potentially why. That kind of transparency is pretty scary to someone that wants to do something malicious or nefarious.
https://lmddgtfy.net/?q=%5Chacker%20news
That's especially useful if DDG is default search engine in your browser.
(I'm not affiliated with DDG)
The purpose of this seems to be to treat Wikipedia as a trusted, reliable source of truth about the canonical URL for websites (debatable, of course). The idea is that you don't trust the search engines, perhaps because you live in a country where your government has required search engines to censor results in some way, but (for some reason?) lets you go to Wikipedia.
This should be Wikidata. Wikipedia does that, but this is more and more moved into Wikidata. This is a good thing, because Wikidata is much easier to query, and the official website of an entity is stored at a single place, that is then reused by all articles about that entity in all languages.
Just as a heads-up of what you could expect to see happening :)
For example looking up "sci hub" on Wikidata leads to https://www.wikidata.org/wiki/Q21980377 which has an "official website" field.
1. Look at past wiki edits combined with article popularity or other signals to arrive at something like a confidence level.
2. Offer some sort of confirmation check to the user.
Regardless, having a system where you can base it off a website could definitely be expanded beyond Wikipedia. Great work!
