I wrote a tire search app a few years back and made it work extremely fast given the task at hand. But I did not go to the level that this guy did. http://tiredb.com
http://info.cern.ch/hypertext/WWW/TheProject.html
If we could cram more modern functionality into say...twice or three times the performance of the above, I think the web would be a better place. Instead the web is a couple orders of magnitude slower.
I feel like the big thing I'm missing is smart compilers that can take web app concepts, and turn them into extremely optimsed 'raw' HTML/CSS/JS/SQL/backend. All of the current frameworks still use hand written frequently very bloated or inelegant hand written CSS & HTML, and still require thinking manually about how and when to do AJAX when it's least offensive to the user. Maybe something like yesod ( http://www.yesodweb.com/ ) or something like that is heading in the right direction. http://pyjs.org/ has some nice ideas too... But I'm thinking of something bigger than the individual technologies like coffeescript or LESS... Something that doesn't 'compile to JS', or 'compile to CSS', but 'compile to stack'. I dunno. Maybe I'm just rambling.
On the other hand, there are sites which are conceptually much simpler but incredibly sluggish. Twitter is a particularly bad offender after you've scrolled down a few pages. Or any other site that uses a ton of Ajax with little regard for the consequences.
But, yeah. If webpages would just revert to what TBL had created (yes, I'll allow for images and minimal other frippery) things would be so much more manageable.
Not just performance, but efficiency - both speed and size. Sadly it seems that most of the time this point is brought up, it gets dismissed as "premature optimisation". Instead we're taught in CS to pile abstraction upon abstraction even when they're not really needed, to create overly complex systems just to perform simple tasks, to not care much about efficiency "because hardware is always getting better". I've never agreed with that sort of thinking.
I think it creates a skewed perception of what can be accomplished with current hardware, since it makes optimisation an "only if it's not fast/small enough/we can't afford new hardware" goal; it won't be part of the mindset when designing, nor when writing the bulk of the code. The demoscene challenges this type of thought; it shows that if you design with specific size/speed goals in mind, you can achieve what others would have thought to be impossible. I think that's a real eye-opener; by pushing the limits, it's basically saying just how extremely inefficient most software is.
Right, exactly. It's obvious too that software has scaled faster than hardware in the sense that to do an equivalent task like say, boot to a usable state, takes orders of magnitude longer today than it it used to, despite having hardware that's also orders of magnitude faster.
So when I see demo of ported software that does something computing used to do back in the 90s (but slowly), I'm really only impressed by the massive towers of abstraction we're building on these days, but what we're actually able to do is not all that much better. To think that I'm sitting on a machine capable of billions of instructions per second, and I'm watching it perform like a computer doing millions, is frankly depressing.
All of this is really to make the programmers more efficient, because programmer time is expensive (and getting stuff out the door quicker is important), but the amount of lost time (and money) on the user's end, waiting for these monstrosities of abstraction to compute something must far far exceed those costs.
I'm actually of the opinion that developers should work on or target much lower end machines to force them to think of speed and memory optimizations. The users will thank them and the products will simply "be better" and continue to get better as machines get better automatically.
Well, there will always be demoscene (http://www.youtube.com/watch?v=5lbAMLrl3xI ) which I've always found remarkable.
- I optimized for compression by doing things the same way everywhere; e.g. I always put the class attribute first in my tags
- I wrote a utility that tried rearranging my CSS, in an attempt to find the ordering that was the most compressible
Compression algorithms can do a better job when they're domain-aware. An HTML-aware algorithm could compress HTML much better than a general-use plain-text compression algorithm, without requiring the user to do things like put the class attribute first. Of course, that also requires the decompression algorithm to be similarly aware, which can be a problem if you're distributing the compressed bits widely.
Well not necessarily... An HTML-aware algorithm could for example rearrange attributes in the same order everywhere because it knows it doesn't matter.
Actually that would be a nice addition to the HTML "compressors" out there.
http://code.google.com/p/closure-compiler/wiki/FAQ#Closure_C...
I've seen similar ideas in the demoscene 4k competition world, where code and music is arranged to have as many repeating self-similar patterns as possible so the executable compressors can shrink them optimally.
"Farbrausch" and their series of Fr-X "small" demos would be one example of this kind of "entropy trickery"
http://www.pouet.net/groups.php?which=322
The next target for crunching would be to minimize the actual amount of code given to the browser to execute, versus maximizing the compression ratio only (which is "just" correlated to the running "code" size)
It could make writing for 4k less of a chore?
In any case, this is an outstanding hack. The company I work for has TLS certificates that are larger than the payload of his page. Absolutely terrific job, Daniel.
[0]: https://code.google.com/p/modpagespeed/
[1]: https://github.com/pagespeed/ngx_pagespeed
edit: formatting
This probably says something about compression technology vs. the state of the art in machine learning, but I'm not sure what.
E.g on a thread click:
St4k:
GET https://api.stackexchange.com/2.2/questions/21840919 [HTTP/1.1 200 OK 212ms] 18:02:16.802
GET https://www.gravatar.com/avatar/dca03295d2e81708823c5bd62e75... [HTTP/1.1 200 OK 146ms] 18:02:16.803
stackoverflow.com (a lot of web requests):
GET http://stackoverflow.com/questions/21841027/override-volume-... [HTTP/1.1 200 OK 120ms] 18:02:54.791
GET http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min... [HTTP/1.1 200 OK 62ms] 18:02:54.792
GET http://cdn.sstatic.net/Js/stub.en.js [HTTP/1.1 200 OK 58ms] 18:02:54.792
GET http://cdn.sstatic.net/stackoverflow/all.css [HTTP/1.1 200 OK 73ms] 18:02:54.792
GET https://www.gravatar.com/avatar/2a4cbc9da2ce334d7a5c8f483c92... [HTTP/1.1 200 OK 90ms] 18:02:55.683
GET http://i.stack.imgur.com/tKsDb.png [HTTP/1.1 200 OK 20ms] 18:02:55.683
GET http://static.adzerk.net/ados.js [HTTP/1.1 200 OK 33ms] 18:02:55.684
GET http://www.google-analytics.com/analytics.js [HTTP/1.1 200 OK 18ms] 18:02:55.684
GET http://edge.quantserve.com/quant.js
....and more....
Even with browser cache enabled, stack overflow loads a considerable number of resources compared to st4k. St4k loads 1 api call to get the S.O. data (JSON - ~1KiB), then loads any needed images. Stack overflow is loading the entire HTML document again (~15KiB), along with a lot of other web resources. Without going into their code, I've got no idea on what is lazy loading.
But my point still stands on the speed of page navigation (not first time landing). St4k is faster as each change of page requires less KiBs of information to perform a page render, as well as less content to render: Compressed JSON, vs the entire HTML markup, and Rendering the Changes vs Re-Rendering the entire window/document.
And now we're proud to have a simple functional list compiled into the same amount of memory ...
My only thoughts are that search is the real bottleneck.
html ~200KB (~33 gzipped)
I'd take some trade-off between between crazy optimization and maintainability, but I'd definitely rather do this than slap on any number of frameworks because they are the new 'standard'.
Of course, the guy who has to maintain my code usually ends up crying like a little girl.
I would love to investigate this further. I've always had a suspicion that the aim to make everything reusable for the sake of bite size actually has the opposite effect, as you have to start writing in support and handling tons of edge cases as well, not to mention you now have to write unit test so anyone who consumes your work isn't burned by a refactor. Obviously, there's a place for things like underscore, jquery, and boilerplate code like Backbone, but bringing enterprise-level extensibility to client code is probably mostly a bad thing.
Wonder how we can unobfuscate the source. It would be great if there is a readable version of the source as well, just like we have in Obfuscated C Code Contests. Or perhaps, some way to use the Chrome inspector for this.
That font family should have been
Consolas,Monaco,monospace
Rather then
Consolas,Monaco,serif
But what ever :)
$ curl -s http://danlec.com/st4k | gzip -cd | sed 's/serif/monospace/' | gzip -9c | wc
14 94 4098Try popping open the inspector panel, and the fonts will magically correct themselves.
$ curl -s http://danlec.com/st4k | wc
14 80 4096
$ curl -s http://danlec.com/st4k | gzip -cd | wc
17 311 11547
$ curl -s http://danlec.com/st4k | gzip -cd | gzip -c | wc
19 103 4098 $ zopfli -c st4k |wc
11 127 4050 $ curl -s http://danlec.com/st4k | gzip -cd | 7z a -si -tgzip -mx=9 compressed.gz
$ wc compressed.gz
14 84 4048 compressed.gzDid you try a png data url? Could be smaller.
A specialized compression protocol for the web?