One approach is to write the filtering engine from scratch. It is what I did, without looking at ABP's code beforehand in order to ensure a clean slate mind.
I didn't get it right the first time, I did spend quite a large amount of time benchmarking, measuring, prototyping, etc.
Once I was satisfied I had finally had a solid code base, I went and benchmarked it against ABP to find out how it compared:
https://github.com/gorhill/httpswitchboard/wiki/Net-request-...
And for HTTPSB's numbers, keep in mind there were an extra over 50,000 rules in the matrix filtering engine (something not found in ABP).
So I think this shows clearly ABP's code can be improved.
I would love to use something better than ABP (in terms of performance, it does content filtering just fine), but I'm not so desperate to abandon the (warning: subjective content ahead) greatest browser of all times to lower the memory footprint of an ad blocker.
Would be nice if someone forked and start to adapt to Firefox. I would of course work in concert as this would allow to identify parts of the code which is browser specific and which parts can be turned into an independent library.
It however would be really awesome for HTTPSB (which I just installed on chromium and fallen in love instantly with) to be ported to firefox.
Seeing all the privacy-related features treated, by domain/site/everywhere rules, being able to block separately js/plugins/cookies (and by the way, the matrix is a great idea, a lot more useable than anything else out there)... in one extension only. Great work already, gorhill!
(I did upvote the comment but I want the creator of the extension to know that more people are interested)
However, I definitely don't want to micromanage things like accepting scripts and cookies for certain sites I use. Is there a way to make it work essentially the same way as AdBlock does? Tried to look in the wiki but didn't find anything.
Now any rules will apply only to that scope. For web sites you use regularly, just lock down these rules for that scope, and they will be remembered every time you visit that particular domain.
This is actually the preferred way to use the extension, but I did not make it the default currently.
Plus I'm afraid that strict settings will break sites I haven't visited before without me knowing they're somehow broken.
Is there a way to configure the extension to just block ads?
This is where I was at an advantage, I was writing new code to fit with new code. There are ideas though that were keys to performance breakthrough which they certainly could borrow.
gmail, aws cloud services panel, and another cloud provider's configuration panel.
Maybe I don't know how to use it, but what are you trying to accomplish by preventing me from loading mail.google.com by default?!
https://github.com/gorhill/httpswitchboard/tree/master/stuff
By default though it comes in block-all/allow-exceptionally mode, just like NoScript and RequestPolicy.
Eventually, as usual, as time permit, I would like to have a wizard for first install which will let the user choose which setup (from the ones above) is preferred to get started.
This is using the abp setup (and a few of the others in stuff/), none of which worked.
https://github.com/gorhill/httpswitchboard/wiki/How-to-use-H...
There's a way to set it up to allow by default.
The plugin looks really interesting, but I wish it would give more detail about the requests that were blocked/allowed and a better explanation if I blacklist something, is that permanent or do I have to save it each time?
There is a request log in the dashboard/Statistics tab where you can see in details (full URL) what was blocked or not.
Someone else at some point asked for a pull request for keyboard shortcuts (and other stuff), but at the time I thought that the extension wasn't stable enough feature-wise, and I didn't want to deal with anything which would need constant revision before the feature set stabilize. For instance, I eventually trashed the context menu code that was put in too early because of this.
But now that it definitely is approaching v1.0, it probably does make sense to address that issue.
A quick question: is there a keyboard shortcut I'm missing to enable/disable the request blocking? For the time being I do Cmd+L, Cmd+C, Cmd+Shift+N, Cmd+V to copy the URL into a private browsing window, but it'd be nice to just temporarily enable full functionality on the page, and then disable when I'm done.
Also, for those using the extension: there's a built-in set of rules to enable browsing Google sites: search, gmail, etc. that you can enable by clicking the puzzle piece in the HTTPSB menu. Very helpful.
Can this app guard against it, ie only send my keystrokes to Google only when I enter a Google URL?
https://github.com/gorhill/httpswitchboard/wiki/Behind-the-s...
By default all behind the scene requests are allowed (or else that would break auto-update and other extensions), but you can make it block everything (except of course for those net requests which are not routed through the chrome.webRequest API).
It is quite useful to also see (and control) net requests made by other extensions.
And discovering Chromium-Updater the other day has plugged the only missing piece in the jigsaw, for me.
[1]http://chromium.woolyss.com/ [2]https://github.com/benbristow/Chromium-Updater/releases
If that's not what you're interested in, more power to you, it's just a suggestion since the use-case isn't obvious especially to a "regular" user.
For one thing, HTTPSB doesn't hide DOM elements (I consider this out of its primary purpose), so if you still want this feature, you will have to keep ABP (or whatever equivalent does element hiding).
As for Ghostery or Disconnect, they still can be useful if you use HTTPSB in a permissive way (allow-all/block-exceptionally).
You could even use HTTPSB to block nothing at all, but as a mere reporting tool (with very low footprint), to find out what the other blockers did not block, as net requests will still be reported in the matrix.
While I have your ear, can I request color blind compatibility?
http://i.imgur.com/WHA3BHw.png
I think there are red and green elements there, but can't tell which is which. Also, I don't know how to set them to red or green specifically as I can't seem them. :)
Could you maybe include a check mark and an X or some other non-color dependent means of portraying that information?
I think it has been available as a mainstream API since early 2012 (Chromium 17).
I am also a bit confused if settings apply to the current domain only or not, some behaviour suggests they are global but I have to test this.
For some odd reason it never did that on my Mac, is there a real difference between firefox osx and Windows?
100mb sounds like a lot of memory for a webpage. Where does all this memory go to?
The most popular websites run quite amazing amounts of code, and JavaScript execution is usually the dominant consumer of memory.
If you want a deeper understanding, visit about:memory in Firefox you'll get a detailed breakdown of how memory is used.
Maybe I just miss the 90s, but I would much rather have the simple content browser than a "full-fledged programming environment", especially since all of that content is coming from the random untrusted intarwebs. Anyone want to write a browser engine that skips all the modern BS and just does a good job delivering mostly-static webpages?
I'm sure if you dug up a copy of an early netscape version it would be very fast on a modern machine, while failing to render most of the internet, if that's what you want.
I also sometimes wish the Web were just serving data in an easily processable format, and let the user decide what and how they are presented. Of course this would definitely and completely defeat the ads-supported scheme on which sits Internet. An ads-free Web would probably require the resources (media files, services, storage space, bandwidth, etc.) to be massively distributed among the PCs of the end-user, who are already paying for bandwidth, processing power and storage anyway. But it would also be quite inefficient as it would require equally massive fault tolerance.
Modern web pages are programs and many are becoming bloated, and there is nothing magical a browser can do to avoid it.
Another is Dillo, although it doesn't have as good of CSS last I tried (version 3 isn't horrible though).
This is one thing I wish more people would try (as I am), and quite honestly don't think is as hard as the big browser vendors make it out to be; my view is that the complexity in mainstream browser engines appears mainly based on how they were designed and implemented, and not as a consequence of the web standards they need to implement. The massive numbers of abstraction layers certainly contribute, as is the notion that everything should be extremely flexible and extensible (Web standards don't change all that often, and when they do they tend to do so incrementally.) It would certainly be interesting to see the limits of how simple (in terms of lines of code, binary size, etc.) and/or fast you could make a web browser and still e.g. have it pass Acid2, which probably makes it quite usable for much of the non-webapp part of the Internet.
text, images, video, audio, animations, 3d graphics, etc.
One only has to look at demoscene productions to see that providing a rich multimedia experience doesn't necessarily require all that much in the way of computational resources, and that's partly a reason why I believe web browsers could be made vastly more efficient than the norm today.
• Reading and discussion. Content-oriented sites. Overwhelmingly, here, Web design isn't the solution, Web design is the problem, and tools such as Readability, Instapaper, and Pocket are increasingly used to address this. Secondarily, content management is a problem, and my suspicion is that a new tool will emerge to displace the current browser model, either one of those mentioned or local clients such as Calibre or Zotero (a bibliographic management tool, though with some document display capabilities). Once content is rendered there's no real call to modify it dynamically. While console-mode / text browsers can be used here, I find they lack the management tools I seek. Discussion would entail some form of standard forum tool, a Usenet 2.0 type concept, ultimately, and would likely be integrated. Editing should support any full-featured local editor of the user's choosing.
• Application platforms. This is where full-featured browsers such as Chrome and Firefox appear headed. I believe they're headed for a rococo collapse -- that is, there's going to be such a degree of overembellishment of the browser that people will eventually abandon them for other, more appropriate tools, as those emerge. This has already happened numerous times in the browser space (Netscape -> Mozilla -> Firefox, MSIE -> anything else), and several times new browsers have emerged whose primary claims to fame were being lightweight and better reading experiences (Skipstone, Galeon, Firefox (initially), Chrome. Most eventually suffer from bloat.
• Commerce. This is currently handled in the browser, and is in fact a principle driver of much of the application functionality. Splitting this from other browsing activity would result in a smaller and saner platform, as well as isolation of financial transactions from other browsing, again, greatly reducing online risks. With a proper payment infrastructure, this should also greatly reduce the general security issues involved with online payments. Arguably this is already happening with the iTunes (Apple) and Play (Google) stores -- apps on mobile platforms.
• Multimedia content. Other than very brief previewing, it's almost always preferable to queue up multiple audio or video streams in an external player. My experience is that this niche is already reasonably well represented, in particular with tools such as VLC which support audio and video content, drag-and-drop queuing, playlists, playback controls, and the ability to correct for audio and video quality (levels, echo, brightness, contrast, etc.). Online players 1) are annoying in general, 2) don't allow for queuing content from multiple sites, 3) have inconsistent (and generally inadequate) controls, and 4) suffer from numerous playback, quality, and performance issues.
More: http://redd.it/256lxu
Over 10 years ago, someone did this with Mozilla Suite.
They called it "Phoenix". Some trademarking issues prompted them to rename it to "Firebird", but that name was also used by a popular database, so they finally renamed it again to "Firefox".
Those who cannot remember the past are condemned to repeat it.
It still is for many of us who complain about browser (and web content) bloat. We are on the web for text and pictures, primarily. The things we do online hasn't changed much since the 90s. Having sophisticated multimedia capabilities is irrelevant when we're in it for the content that matters.
Pages like this: http://www.nytimes.com/projects/2012/snow-fall/ ?
Or pages like HN or wikipedia articles?
We seem to be trading resources for capabilities that we rarely leverage for any tangible benefit.
Yes, I know, and I don't like it.
It makes it more complicated and unreliable to use the web to view text and images.
That's forgetting the DOM, which is often the actual cause for slowness & memory consumption, but gets far less spotlight than JavaScript. This is also a reason why a framework like react.js is so interesting: it circumvents the real DOM when computing (on a virtual DOM), and only applies the end result.
They changed the browser UI, either the browser devs or the site devs via stupid javascript tricks (messing with scrolling, etc). I got driveby download powned and my CC info stolen and five spam toolbars in the UI I can't get rid of. There are too many spammy ads. They make is as hard as humanly possible to use bookmarks to encourage the use of a search engine every single time you go to a site thus monetizing ad views. Typosquatters. "Journalism" sites with one line of the article per page of spam. Spam "answer" websites.
Memory is way, way down the list.
Chrome has a "star" button right in the address bar, FF and IE have Ctrl+B to create a bookmark.
And both Chrome and FF have a pretty good auto-complete in the address bar (I just have to type "ne" to get Hacker News, "ma" for google mail, "youp" and "yout" for video streams, etc pp).
People vastly underestimate the power of keyboard shortcuts. I work as a freelance consultant/support guy and everyone is surprised how fast I work with even the slowest computers. Most can't believe how much time one can save by knowing at least Ctrl+C/Ctrl+V, Alt-Tab and Ctrl-Tab.
Two words for you: Incognito Mode
The Classic Theme Restorer add-on for FF allows you restore the star button to the address bar. On FF, Ctrl-D bookmarks the current page, while Ctrl-B opens a list of bookmarks.
Besides, whatever happened to Web browsing being a lightweight activity accessible for everyone even on his grandma's old PC?
I do. I have 16GB of RAM (the max for my computer) and have 3GB of swap used. About 8GB of that is Firefox, and I (obviously) run more than just a browser on my computer.
It boils down to images, if you think about it a PNG file is small but when you decompress that to a native bitmap for rendering to a screen, you've suddenly consumed a lot of space. A lot of applications render using the CPU and pass the data to the GPU to display.
Now look at a typical page with images, flash (with images inside) all of the scroll and developers that don't optimise their images and then 100MB of data for a single page isn't really a lot.
Plus memory usage is the wrong metric, I don't really care how much RAM my applications use, I care about how responsive they are. Firefox can use all my RAM if I get great page load times and my other applications don't suffer.
if anything I wish more programs would take full advantage of it.
If every single application behaved like it owned all the memory on your machine, you would have a pretty horrible experience as they'll be basically fighting over what's available and likely hitting the swapfile. On the contrary, I do wish developers would care more about the memory footprint of their applications since they are sharing with all the others. Looking at it this way, optimising performance by using more memory aggressively (and thus reducing the amount of memory other applications can use) seems like a horribly greedy and antisocial way to behave, and thus it makes sense to use as little as feasible to give good performance.
(I'm assuming that the use case here is a browser that you have open along with other applications; if you only have a browser running, then it doesn't matter as much.)
I'm always staggered on the flip side that serving a web page using something like XYZ CMS can use a staggering 50MB or more for one page load without caching.
Bizarre when you consider that you could get a small encyclopedias worth of text on a floppy disk.
Too many websites are drowning polar bears...
I actually even started splitting my hosts file: the layout is
/etc/hosts ('compiled' version)
/etc/hosts.d/
/etc/hosts.d/aaa-warning (warning reminding me to run `update-hosts` instead of modifying /etc/hosts. Appears atop the compiled file)
/etc/hosts.d/adblock (that website's hosts file)
/etc/hosts.d/base (original system hosts file)
/etc/hosts.d/dolead (work-related file for development)
function update-hosts() {
cat /etc/hosts.d/* > /etc/hosts;
}
function update-adblock() {
curl http://someonewhocares.org/hosts/zero/hosts -o /etc/hosts.d/adblock 2> /dev/null
update-hosts
}
Works wonders.
I just looked at the adblock-hosts file, and 'tynt' appears several times. One of those lines is commented with a link to an article about tynt[0]. I tried copy-pasting from The New Yorker and TechCrunch (who, as claimed by the author of the article, use tynt), and both times there was no crap appended.
I'd love to move the ad filtering out of my browser though, either into a proxy like privoxy or DNS filtering, if it worked well.
With all the features Adblock Plus gives us, far far more than just blocking xx and yy domains like in a hosts file... Why not just use the extension? Let's leave the host file to the history books.
I think using both is better than either one alone. A hosts file to quickly remove most of the unwanted stuff, and ABP to finish off the rest that does get through.
The net request filters do prevent net requests from being made.
Firefox does support it, and I know that in the past adblock did stop the requests rather than hide elements.
I've just tried the list you provided and it seems to be ok. Will try it for a while to see how I get on.
I'll try to understand it tonight.
Since gethnamaddr.c appears to be BSD-licensed I'm willing to bet that 99% of all OSs out there (including Windows) are going to have similar if not identical code.
Well, for me the whole point of blocking ads is because they are often big flash things that hog cpu and memory. If ABP is no better, then most of the reason is gone. I'd actually like to view ads to support more sites.
People install adblockers because too many idiots became too fucking greedy and ruined ads for everyone else!
Yes! And the cognitive noise. I find it literally impossible to read text if there's some blinking ad nearby. Before Adblock I'd have to put my had up to the screen to block the animation so that I could read the article text in peace.
Unfortunately the Firefox version doesn't have this, and neither do the alternative adblock addons I could find.
If you don't want to screw publishers out of compensation for their product but also don't want Flash running arbitrarily on your browser, change your plugin settings to click-to-play (I know Chrome has it but I'm pretty sure Firefox does too). It has the added benefit (for me) of being able to open most video streams in a new tab the way I do with every other article and then consume it at my leisure later.
You can disable ABP on particular sites. I was going to suggest doing that for sites you frequently visit that don't have ads... but I just did some quick testing and it doesn't seem to change the memory usage at all.
It's sad that the most-demanded feature on every browser, as evidenced by plugin downloads, is ad blocking. However, all the major browsers are produced by companies with their hands in advertising, and this conflict of interest has resulted in this feature request going unfulfilled for a over a decade.
Fork 'em.
Take popup blocking as an example. That's integrated into the browser and has almost completely eliminated the annoying popup ad.
It's not really a work around but some sites do block your [full] access and say to turn off your adblocker if you want access to the [additional] content.
That said I don't think it makes much difference to the matter at hand.
Besides, fewer people have installed the ad block than those who have browsers. This make for smaller incentive to develop workarounds, compared to situation where everyone has some sort of block.
What it did not have, is an auto updated blacklist.
But the actual ad blocking was fast and effective.
I often dream of a world where x% of my ISP bill is paid out in micropayments to the content providers (think about it, we pay ISPs for infrastructure that would be worthless without content, and they get all the cont for free). But that ignores actual bits, so in the end I wish there was a protocol for me to pay for the content I want, and in exchange the provider would give me ad-free content. Now imagine I could choose (or have a browser setting) what level of payment vs. ads I want on each website.
I realize it can't be that simple, the economics of the Internet (from end to end) are much more complicated.
ABP has a huge page-load slowdown on my phone, this is essentially unnoticeable, and still removes the vast majority that I encounter.
Currently this machine has been up for 3 days and 17 hours and Firefox has been running the whole time and it is using 303MB with 4 tabs open one of which is the Daily Mail (don't judge!) which is an extremely busy page. This is perfectly acceptable in my opinion. I only have 4GB RAM which by todays standards in not much either. Obviously reducing the memory footprint is great but I can't say it has ever been a problem I have ever noticed.
On a side note the past few updates to Firefox have improved performance a lot. I am really impressed by how much quicker the browser is, especially with Chrome seeming to just get slower and slower with each update.
I've also seen this in some other comment, so I think it's worth pointing out: if you have a desktop computer then yes, it's ok. However, there are also netbooks and notebooks around with a lot less memory than that. Tablets, while not currently one of Firefox markets, are also commonly seen with less than that. Lots of parents with second-hand computers, too, which haven't been touched in a while because it "just works".
The overall attitude towards hardware is apparently "use it for two years, then throw away and get a new one", and yet underpowered computers surround me in a 5:1 ratio. I sometimes have to install plain old versions of software (Office '97 comes to mind) because the new ones provide 5% new features at the cost of half your memory.
Maybe everyone else has techie friends that only care about the latest and shiniest, or maybe the rest of the world is rich enough to throw away perfectly good hardware just because they feel like it. But from my experience (as biased as it is), 4Gb is not an "only" situation.
And I'm not shy about using tabs either, I often have many dozens open.
-ads take so long to load and when ABP is not enabled bogs down my browser hard core
-when watching videos I have to wait 3-20 seconds for them to load, with ABP enabled I do not have to wait at all
It consistently saves me several minutes every day. If it adds a few extra milliseconds to load some style sheets I have never, ever noticed.
I've never tried converting ABP rules into privoxy, but I did write a crude perl script to grab Ghostery's blocks and convert them into privoxy config files.
GNU/Linux: http://andrwe.org/scripting/bash/privoxy-blocklist
OS X: https://github.com/skroll/privoxy-adblock
Scripts to convert Adblock-filters to the Privoxy-format. They have worked well so far.
I don't know whether anyone has made such a proxy for use in the kind of situation we're talking about here, however.
My computer has 12 Gigabytes of RAM.
I don't give a shit, as long as i don't have to view those terrible stupid Ads.
Advertisers, fix your practises and i will view your Ads again.
It will remove around 90% of obnoxiousness...
Yeah, this would be more or less the same functionality (FlashBlock makes it per item, so you can, for example see a video but not activate the other elements)
There used to be a proxy adblocker that did that, but I don't think it works anymore.
The Kindle browser uses a proxy to pre-render pages on the server in order to lighten the load of the device.
Could AaaS (Adblock as a service) be a viable business? I think I'd pay for it.
It probably could, however I'm sure it would be shut down quickly by advertisers and parties showing ads - after all, you're offering a service to remove their source of income. Unless you'd funnel part of your subscriber's income back to the advertisers, based on whose ads you block. But I'm sure it'd end up being too expensive for the consumer and too much of a hassle for the one offering the service.
Not to mention you'd force your users to funnel all of their internet traffic through your servers, and it wouldn't work with encrypted pages (which large parts of the internet is moving towards)
What legal leg would they have to stand on? There's no law that prevents me ripping out the ads of magazines that get delivered to me…
Tech laws are absurd.
I can understand Mozilla taking some interest in how addons behave, and constructive feedback on extensions is a good thing. However, ABP is the type of extension that is likely to have issues in those areas because of what it does. Which is very important to users, especially those who rely upon it for its privacy and security enhancing capabilities. It is those users who should decide whether the performance and resource usage trade-offs are acceptable. Mozilla shouldn't make, or try to make, such decisions.
The situation with ABP 2.6 (https://adblockplus.org/development-builds/faster-firefox-st..., https://adblockplus.org/forum/viewtopic.php?t=22906) might not be a case of this, but that along with the wider pattern of platform developers being more controlling, does make me somewhat concerned about Mozilla taking too much interest in extensions. I hope my worries are for naught.
> So, it’s clear that ABP greatly increases Firefox’s memory usage. Now, this isn’t all bad. Many people (including me!) will be happy with this trade-off — they will gladly use extra memory in order to block ads. But if you’re using a low-end machine without much memory, you might have different priorities.
If you think this post is unreasonable, I think your skin is too thin.
Did you notice how some people here, and at your blog, proposed modifications what would trade coverage for memory consumption? Even if that would result in known filter hits being eliminated, and also without qualifying that if such an approach were adopted it should be preference based?
https://easylist-downloads.adblockplus.org/easylist_noelemhi...
With these rules firefox's memory usage only goes up from 290MB to 412MB, instead of 1.5GB for the website mentioned in the article for me.
The downside is that this list has a more limited coverage than the full version of the list.
I did try noscript for a while, but I got fed up having to turn it on for sites that just failed to work without JS, and got bored moaning about it.
There might be an architecture problem here. Another solution is to use a proxy to block ads, like GlimmerBlocker for OSX. But I didn't investigate memory usage, though I tend to think it will be lower (plus have the added benefit of working simultaneously for all browsers).
So it's not just memory that can be at risk of hogging.
Maybe it's time to try switching back?
Even with the memory pain, I still (mostly) love FF.
Thank you, really thank you.
You're missing important messages from sponsors.