Wired: How has your thinking about design changed over the past decades?
Brooks: When I first wrote The Mythical Man-Month in 1975, I counseled
programmers to “throw the first version away,” then build a second one.
By the 20th-anniversary edition, I realized that constant incremental
iteration is a far sounder approach. You build a quick prototype and
get it in front of users to see what they do with it. You will always
be surprised.
Now, if the name was something like HTTP/1.8-alpha it might be a different thing. At least then it wouldn't carry the label of the "next big thing for everyone". It's sad, but names (and branding) do matter. Forcing a known-broken implementation upon the world is not exactly good engineering.
And I'm very afraid of the following phrase:
"we found out that there are numerous hard problems that SPDY doesn't even get close to solving, and that we will need to make some simplifications in the evolved HTTP concept if we ever want to solve them."
This usually means "ending up with a protocol that has a lot of corner cases and a lot of backward-compatible crap or maybe some half-baked stuff that was left because of some feature that nobody uses"
I am very skeptic of protocols/standards that are born from a committee (take a look at telephony protocols/standards if you doubt me)
It was revolutionary at the time but people have moved on and found many improvements to the original and also outright mistakes.
(As the author seems to have acknowledged when releasing a new improved iteration of his book.)
That hardly means that what it has to teach isn't still valid. Admittedly, I've only read a few chapters in it, but the central point that throwing more man power at a late project only serves to make it later, is at least as relevant today as when the book was first published.
> It was revolutionary at the time but people have moved on and found many improvements to the original and also outright mistakes.
Could you be more specific?
> (As the author seems to have acknowledged when releasing a new improved iteration of his book.)
So, do your points above refer only to the first edition of the book then?
I would also accept an interpretation where that's an indictment on the field.
Few books in our industry have been more prescient.
> In the old days we had different protocols for different use cases. We had FTP and SSH and various protocols for RPC. Placing all our networking needs over HTTP was driven by the ubiquitous availability of HTTP stacks, and the need to circumvent firewalls. I don’t believe a single protocol can be optimal in all scenarios. So I believe we should work on the one where the pain is most obvious - the web - and avoid trying to solve everybody else’s problem.
If we're not careful, we're just going to end up cycling back around again and find ourselves 20 years in the past.
That said, I do think to some extent "that ship has sailed". The future of network programming seems like it will be "TCP --> HTTP -(upgraded connection)-> WebSockets --> actual application layer protocol". See, for example, STOMP over WebSockets. While it is annoying that this implies we've added a layer to the model, it's hard to argue with the real-world portability/ease of development that this all has enabled.
I propose OpenVPN/SOCKS/WebSocket/HTTP/TCP/IP as the new de-facto standard connection protocol. Maybe we can FTP through that VPN connection some time, please wait while I cook up a JavaScript FTP/OpenVPN/SOCKS/WebSocket/HTTP/TCP/IP client.
And thus naturally HTTP must be made to accommodate for all those applications.
You may think that makes sense, but you don't create something as long-lasting as internet-scale architecture based on hacks around incompetence.
Besides, if that's the path we're walking, DPI-based firewall with HTTP-level application firewalling will become the new norm, and we've gotten nowhere further. Except we now have a even bigger mess to work with.
While the OSI-model may be going a bit over board for some aspects, making all future application-protocols be a squashed through HTTP is madness. This thinking is of the same quality and mindset as of PHP developers.
Case in point: 5223/tcp, used by Apple's Push Notification service (amongst other things). Push notifications not being delivered to so many networks made their users angry and tech support calls (and costs) boom, and had soon 5223/tcp unblocked, even on free, non-encrypted, coffee-shop wifi networks.
I understand that most developers just choose to roll with it these days, but I really believe that putting everything atop of 443/tcp because of clueless/incompetent sysadmins is a huge mistake.
Keeping protocol stacks small, efficient and secure should be a design goal.
IP addresses are logical numbers used to communicate with a host. In order to make it easier for humans to know what host to communicate with, DNS records were created to be able to easily communicate with a particular host.
In addition to figuring out what host we want to communicate with we need a way to specify what application we want to use. So service names were created and assigned to a static list of application port numbers. Two of these application service names are "http" and "https", which each provide a different application protocol.
We've built so many things that depend on these two application protocols that developers have finally outgrown what the protocols are capable of. But to get support for a new protocol is "hard". You have to do three things to successfully roll out a new application protocol:
1. Write a server for it and make people want to use it, 2. Write a client for it and get users to want to use it, 3. Get network administrators to open up their firewalls to support it.
Sadly, many developers are playing a cat-and-mouse game with network administrators. They want their applications to be used by anyone, anywhere, automatically. So they employ tricks to get their content past network administrators, like re-using the same application service names used by protocols which are ubiquitously supported throughout worldwide networks.
But this just creates an arms race with network admins. For example, mandating strong encryption to use a service means a network admin traditionally can't restrict what kind of applications work over it. So in order to fight their users subverting their approved applications policies, more and more organizations are doing SSL inspection by injecting CA certs into their employees' computers to block unauthorized applications.
Fighting network admins is a losing game, and does not benefit the users of your applications, or the developers. Instead of hiding more features and overlaying protocols, it would be more productive to create protocols that use their own service names and provide their own functionality and let users demand it be supported by the network admins. This is how all popular application services have been adopted over the years and provides the best quality of service to everyone involved.
It's not like WebSockets runs "over" HTTP, it just uses an HTTP-like handshake; besides that, it's just a simple framing protocol over TCP.
That's why you can use a simple proxy to "websocket-ify" applications that use plain-old raw TCP connections.
Websockets is ridiculous for this reason.
It has a stream abstraction (Websocket API) over a packet based protocol (Websocket Protocol) on top of streaming protocol (TCP) on top a packet protocol (IP).
This shit has to stop.
We had everything we needed. Websockets should have just been exposing regular sockets to the browser.
I don't buy the HTTP proxy argument.
Still, if TCP is the transport layer, and HTTP/SMTP/etc. are the application layer, then what do we call WebSockets when we implement an application layer protocol on top of it? The application compatibility layer?
In the past we had operating systems with network stacks and shared libraries for working with application protocols. Now we have browsers with network stacks and JavaScript libraries for working with application protocols. Wasn't it just a few days ago that someone was proposing an OS that would do just enough to boot a browser, and then let the browser handle most of the "traditional" operating system tasks? How long before we see a proposal for "lowering" WebSocket connections to the TCP layer?
This is what I mean when I say that we risk finding ourselves 20 years in the past. HTTP isn't ubiquitous because it is better. It's better because it is ubiquitous. If we don't pay attention to why things like WebSockets are easier to use today than raw TCP, then history will repeat itself.
Furthermore, if http/3.0 is already being discussed, why not just skip http/2.0 entirely, and live with the current http/1.1+SPDY situation until the work towards a new standard for http is actually done?
That would be a fairly easy bet, but I doubt anybody would take the other side. My own estimate at rolling out a major HTTP protocol revision across the industry would be in the 100's of millions. Take into account that we have approximately 750 million web sites and 3 billion clients.
"So it looks like HTTP 2 really needs (at least) two different profiles, one for web hosting/web browser users ("HTTP 2 is web scale!") and one for HTTP- as-a-substrate users. The latter should have (or more accurately should not have) multiple streams and multiplexing, flow control, priorities, reprioritisation and dependencies, mandatory payload compression, most types of header compression, and many others."
http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/...
"First and foremost, it needs to be recognized that HTTP/2 has been designed from the start to primarily meet the needs of a very specific grouping of high volume web properties and browser implementations. There is very little evidence that ubiquitous use of the protocol is even a secondary consideration -- in fact, the "they can just keep using HTTP/1.1" mantra has been repeated quite often throughout many of the discussions here on this, usually as a way of brushing aside many of the concerns that have been raised. So be it. It's clear at this point that HTTP/2 is on a specific fixed path forward and that, for the kinds of use cases required by IoT, alternatives will need to be pursued."
(although their tendency to fill sites full of third party components may reduce some of its benefits)
https://groups.google.com/forum/#!searchin/mechanical-sympat...
Poul-Henning
http://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/... made me wonder what your standby plan would be – let the people who really need to care about performance use SPDY until a more ambitious HTTP 2.0 stabilizes? One of the concerns I have is that many people want performance now and it seems like HTTP 2.0 might turn into the next XHTML if it takes too long to emerge.
If you really want to gain performance for instance, the way to go is to get rid of the enormous overhead of cookies, to replace the verbose but almost content-free User-Agent header and so on.
Likewise, wrapping all the small blue 'f' icons and their associated tracking icons in TLS/SSL does not improve privacy on the net in any meaningful way.
But the entire focus has been to rush out a gold-plated version of SPDY, rather than to actually solve these "deep" problems in HTTP.
Similarly: Rather than accept that getting firewalls fixed will take a bit of time, everything gets tunneled through port 80/443, with all the interop trouble that will cause.
And instead working with the SCTP people on getting a better transport protocol than TCP ? Stick it all into the HTTP protocol.
Nobody seems to have heard the expression "Festina Lente" in this WG.
Of course we don't live in Plan-9's 9P world, and I don't think we will ever live in that world, but if you think about it, it makes a lot more sense. In every aspect. 9P could make lots of troubled / tied (think XML standard such as WebDAV) / historical standards (FTP, NFS) obsolete. It is sane, simple, fast and secure because it is just a stream of bytes, displayed as a filesystem. There is no need for tons of librarycode. And http/0.2 could be backwards compatible. With http/0.2 you could also have session id. Besides that, with mounting httpfs there is no absolute need for a browser. You could use the standard commandline tools, altough the browser is gonna be used in almost any case.
All I wanna say is that I agree with your ideas. HTTP/2 is probably gonna have a long breath, so thinking it over, and start from scratch would be a good idea IMO. With 9P it could be a real dealmaker.
> Is there an IETF process in place for "The work we're doing would harm the Internet so maybe we should stop?" - http://www.ietf.org/mail-archive/web/trans/current/msg00238....
HTTP/2.0 has been rammed through much faster than is reasonable for the next revision of the bedrock of the web. It was always clearly a single-bid tender for ideas, with the call for proposals geared towards SPDY and the timeline too short for any reasonable possibility of a competitive idea to come up.
There has never been any good reason that SPDY could not co-evolve with HTTP as it had already been doing quite successfully. If it was truly the next-step it would have been clear soon enough. All jamming it through to HTTP/2.0 does is create a barrier for entry for similar co-evolved ideas to come about and compete on even footing.
He wants radical change in the protocol but when given the opportunity submitted a (by his own admission) half baked proposal - there's also the question of what a protocol like HTTP/2 means for his product.
Although HTTP/2 started from SPDY it has evolved, and in different ways e.g. see the framing comments from the thread the OP links to.
We need a better protocol for the web now, yes we could wait around longer for more discussion but where did that get us with HTTP/1.1 - I'd be quite happy if IETF had just adopted SPDY lock, stock and barrel (and no I don't work for Google)
There has never and will never be a point in time where we don't need "a better protocol for the web now." The issue is that canonization was unnecessary, adoption of spdy has been progressing fine without it. And HTTP/2 diverging significantly from spdy does not inspire confidence, either. Rather it just reminds of a famous xkcd [1]and again begs the question of whether trying to turn spdy into http/2 even manages to achieve any of the goals the process was setting out for.
The whole thing just seems like a big fat SNAFU.
There are a few different implementations of SPDY, and a clear use case where it applies. Also, it's a clear standard, made for being used.
What's happening here is that there is a group of very active people that create most of the software we use on the web, and have a use case they want to support. At the same time, there are lots and lots of people that are not as active, with a huge amount of use cases that will be hindered, but since they are not active, they have very little voice.
Nothing about spdy or http2.0 sparks any sort of confidence with regard to proper robust protocol design, keeping things simple nor properly separating concerns.
XHTML 2 is a lot more like what PHK is proposing: an attempt to "rethink" HTML, come up with something simpler, revolutionary rather than evolutionary, "The Right Thing." It was an attempt to reinvent the space from first principles, and had lots of ideas that were theoretically good but unproven at large scale.
When that went nowhere, the world settled on HTML5: evolutionary, incremental, and based on standardizing existing practice. Much less sexy, but more useful in practice.
There is a time and a place for bold new ideas, but a standards body designing v2 of a protocol isn't it. Standards are for codifying proven ideas. When standards bodies try to innovate you end up with XHTML, VRML, P3P, SPARQL, etc.
But "they" (the W3C) didn't do that when people were just complaining about issues with the XHTML 2.0 approach, they did it after a competing approach was developed via an extensive, multi-year process through an outside group (WHATWG), and even then only that after a short period when both approaches were the focus of official W3C working groups.
They didn't adopt a "this is limited, lets throw it away and start over" approach as the original article here calls for with regard to HTTP/2.0.
PHK has experience of HTTP both from the server point of view (the main job of Varnish is acting as a fast HTTP server) and from client point of view (Varnish acts as client to the slow upstream HTTP servers).
As a side note, he also refrained for years from adding TLS support to Varnish after his review of OpenSSL and SSL in general (see https://www.varnish-cache.org/docs/trunk/phk/ssl.html ).
The way I see it though, is not only to have a protocol, but how to get adoption. Especially when you're talking about network protocols, you need rock solid stacks in all major operating systems which is not an easy feat to accomplish.
An aside: I find it odd how HN users jump to agreement when a link to a single mailing-list message is posted, ignoring other discussion on the thread. I think it's because the UI makes it hard to see the rest of the conversation (unlike -say- the comments UI on HN itself)
It does seem a little shocking that the WG chair is proposing last call while still there's serious discussion of things like dropping HPACK.
An incredible aspect of the web is that Tim Berners-Lee's first website back at CERN still works in modern browsers. Same with things like basically the entire Geocities archive.
When it gets to core infrastructure like HTTP you can't just iterate quickly and expect the entire internet to constantly upgrade along with you.
What works for early stage lean startups won't work here.
No standard is perfect, but the worst standard is no standard.
Make up your fucking mind already.