undefined | Better HN

0 pointsup6w64y ago0 comments

I believe the Instant View[1] crowdsourcing model where people write templates for each website could boost a lot these parsers (hope they open source it soon). Its just impossible to make these extensions work for every single website with some simple heuristics.

Check the codebase of some popular parsers:

Firefox (already mentioned): https://github.com/mozilla/readability/blob/master/Readabili...

Google Chrome: https://github.com/chromium/dom-distiller

Mercury parser: https://github.com/postlight/mercury-parser

[1] https://instantview.telegram.org/

0 comments

k1m4y ago

Thanks for mentioning Instant View, I hadn't come across that. We actually maintain something similar here: https://github.com/fivefilters/ftr-site-config

We use these in our own tools and also get contributions from others, including Wallabag users: https://github.com/wallabag/wallabag

Before it was sold, Instapaper used to have something similar. A public database of its site-specific extraction templates. We used that as the starting point for our repository.

benzible4y ago

FYI your API pricing page doesn't seem to load https://rapidapi.com/user/fivefilters

freediver4y ago

Thanks to both of you for expanding my 'readable web' toolbox.

What do you fallback to if the rule is not present or doesn't work?

k1m4y ago

In our case, we try to match using the XPath selectors that we have for the site. If we don't have any, or they fail to match anything for the title, author, or body, we then go to Readability and let it do its thing to try and extract whatever we're missing.

1 more reply

bspammer4y ago

Another cool crowdsourced thing I discovered recently is SponsorBlock [1] which is an extension to automatically skip sponsored content in Youtube videos. Users contribute timings to the database that everyone else uses. It works remarkably well, any recent video with more than about 50,000 views is pretty much guaranteed to have timings submitted.

[1] https://sponsor.ajay.app/

nyanpasu644y ago

Is it possible to utilize the database of Instant View per-site parsers in a web browser or extension's reader mode?

j / k navigate · click thread line to collapse

0 pointsup6w64y ago0 comments

Check the codebase of some popular parsers:

Firefox (already mentioned): https://github.com/mozilla/readability/blob/master/Readabili...

Google Chrome: https://github.com/chromium/dom-distiller

Mercury parser: https://github.com/postlight/mercury-parser

[1] https://instantview.telegram.org/

0 comments

k1m4y ago

Thanks for mentioning Instant View, I hadn't come across that. We actually maintain something similar here: https://github.com/fivefilters/ftr-site-config

We use these in our own tools and also get contributions from others, including Wallabag users: https://github.com/wallabag/wallabag

Before it was sold, Instapaper used to have something similar. A public database of its site-specific extraction templates. We used that as the starting point for our repository.

benzible4y ago

FYI your API pricing page doesn't seem to load https://rapidapi.com/user/fivefilters

freediver4y ago

Thanks to both of you for expanding my 'readable web' toolbox.

What do you fallback to if the rule is not present or doesn't work?

k1m4y ago

1 more reply

bspammer4y ago

[1] https://sponsor.ajay.app/

nyanpasu644y ago

Is it possible to utilize the database of Instant View per-site parsers in a web browser or extension's reader mode?

j / k navigate · click thread line to collapse