https://www.dreamsongs.com/WorseIsBetter.html
Over time, I have come to believe that the problem is overly-aggressive abstraction. It is very tempting for most developers, especially good developers, to reach for abstraction as soon as things get complicated. And this can pay off very, very well in some cases. However, too much abstraction in a system leads to a very, well, abstract code base that becomes hard to get a handle on: there's no there there. You see this in the java world with AbstractFactoryBuilderLookupServiceBuilders and so forth, and with very elaborate type setups in functional programming languages.
Concretizing the crucial bits of your system, even if that means a few large, complex and gronky methods or classes, ends up often making things more understandable and maintainable and, certainly, debuggable.
John Ousterhout wrote a book that makes this point as well, advocating for "deep" rather than "shallow" classes and methods:
https://www.goodreads.com/en/book/show/39996759-a-philosophy...
Sometimes, yes, overly-aggressive abstractions are a problem. But the author describes policies of v1 and v2 of the linker. And I would say the most critical difference between them is, that the authors of v2 had a better understanding of the requirements that actually mattered. Therefore they were in a better situation to evaluate the architecture and they were able to make better trade-offs.
Deciding to trust object files as inputs might raise a red flag for some people. In principle it could allow attacks/exploits of the linker. But in reality for most threat models this does not matter. Because the compiler generating object files has the same level of trust as the linker. Companies that want to provide an elf linker as a service product are out of scope. Deciding what is in scope and what is out of scope is probably one of the hardest decisions in product engineering. Because many of us software engineers lean towards perfectionism or, especially inexperienced developers are searching for the silver bullet, that set of rules that enables them to develop every product successfully.
[edited typos]
The correct lesson is that the real world is not obliged to conform to your personal model of "better". You have a personal obligation to continuously adapt your model to match the real world. This is the model of science, and is opposed to Platonism. After you have adjusted your model, it is certain to still not be right, and need further adjustment.
Usually "the world" you are obliged to adjust to has its own problems. There are powerful forces making us favor accommodating a Microsoft execution environment, even though that execution environment has always been a cesspit. It represents its own poor abstraction. Posix file system semantics are another example. Von Neumann architecture and the C abstract machine are not the only, or best way to organize computational resources. It is important to recognize when somebody else's pragmatic failure threatens to taint your own models.
Lisp, RG's hobbyhorse, didn't get sidelined because of Philistines. Lisp turned out not to be better, despite how strongly RG felt about it. Instead of figuring out what about Lisp was not right, he called things that were, along axes that matter, more right "worse", preserving his personal model and lessening readers' ability to reason about merit.
All of this is to say: I don't agree with you, but I also agree with you and I suspect he would as well, with qualifications. And he would probably also disagree with you.
Lisp machines were clearly abandoned because of their price. Every time I find some history about someone that actually made that decision, the reasoning was exactly alike, those machines costed more to keep than the Unix ones to install, and were less capable due to outdated hardware.
Yet the essay goes all over the place, citing time to market (that was completely irrelevant, UNIX was the newcomer, Lisp machines were there already), university-based prejudice (yet every single one decided the same at around the same time), and blaming the user. The essay doesn't even talk about money.
But there is no absolute requirement to conform oneself to a social milieu. A social milieu changes. It can be altered. It supports a vast number of models. And milieus overlap so densely that one can just go play somewhere else.
Disclosure: I used to be a Lisp bigot, but I got better.
(Annoying as fuck, that. ;-)
And that seems to indicate that a(n initially) half-assed, but improvable, model of it that is then iterated upon is a good way to build one's model. Feels to me like that is exactly what "Worse is Better" is about -- shouldn't your beef here rather be with "the MIT model"?
> You have a personal obligation to continuously adapt your model to match the real world.
A bit hard to get a consensus model out of that, since everyone's perception of the real world -- heck, everyone's actual "real world" -- varies.
> This is the model of science, and is opposed to Platonism.
Again, what feels like setting up a Platonic ideal to me is more "the MIT model", rather than "Worse is Better".
> After you have adjusted your model, it is certain to still not be right, and need further adjustment.
Ah, dangit, at some point you just gotta say "Screw it, good enough!" (See, for instance, "The saddest 'Just ship it!' story" the other day.)
[Edit: Reduced repetitive weasel wording.]
The top ten languages in https://www.tiobe.com/tiobe-index/ right now are Python, C, Java, C++, C#, Visual Basic, JS, assembly, SQL, and PHP. Of these, the "dialects of Lisp" include Python, Java, C#, VB, JS, and PHP.
Remember that in 01991 all "serious" modern software was either C, C++, Pascal, or assembly. BASIC, whose only data structures were the string and the array, was for amateurs or the MIS department, which mostly used COBOL, assembly, and JCL. Fortran was established but was considered antiquated (except by supercomputer people) and didn't have pointers.
If we compare these languages on the points pg lists in http://www.paulgraham.com/diff.html, the earliest versions of Lisp score 9/9, Python is 7/9, Java is 7/9, C# and VB are Java with different syntax, JS is 7/9, and PHP is 5/9. By contrast, C is 2/9, C++ is 3/9, Pascal is 3/9, assembly is 0/9, COBOL is 0/9, Fortran 77 is 1/9. You can quibble a bit about these numbers (do Pascal procedure parameters qualify as "a function type" even though you can't store them in variables? Does the Python expression "intern(s) is foo" qualify as "a symbol type"?) but the broad division is very clear.
I think it was in the conversation where he originally wrote that essay that Guy Steele said of his own work on Java that they had managed to drag all the C++ programmers kicking and screaming about halfway to Common Lisp, so we should be happy. I've lost my archives from that time, so I can't be sure.
In terms of syntax, Python or Java have nothing in common with Lisp. But in terms of the issues you raise — accommodating a Microsoft execution environment, POSIX filesystem semantics, the Von Neumann architecture, the C abstract machine, or just the tools they give you to analyze problems — they're just dialects of Lisp with slightly different syntax (and more hair on eval, and sort of broken symbols or no symbols).
In terms of "worse is better" of course Python and C# lean just as hard on "worse" as C does.
If we restrict the sense of "Lisp" to languages with S-expression syntax like Emacs Lisp, Common Lisp, Scheme, and Arc, then Lisp did fail to (at least) become popular — but it's plain to see that when people abandoned Common Lisp and Scheme, they were mostly moving to languages like Python and JS which adopted Lisp's most appealing ideas, not to C++.
I also think r-bryan's point in https://news.ycombinator.com/item?id=31346478 is true, beautiful, deep, and merits quoting:
> Your notion "the real world" conflates the physical world with a social milieu. The physical world is (at this scale) immutable, so of course we must conform to it…
> But there is no absolute requirement to conform oneself to a social milieu. A social milieu changes. It can be altered. It supports a vast number of models. And milieus overlap so densely that one can just go play somewhere else.
In that vein, it's worth noting that Linux got pretty far before it ever had to accommodate a Microsoft execution environment, though I did have coworkers in 01997 who thought I was hopelessly unhip for preferring Unix (which they thought of as antiquated) to Microsoft Windows.
It's complex as hell code-wise, but it simplifies the amount of cross-team/cross-company alignment and synchronization that has to happen in order to cut a new version of the application containing hotfix 8495 for the part of the app maintained by Steve's team.
There's actually a decent parallel between that and Microservices. Microservices make maintenance of the whole more complicated by introducing the network between pieces, but allow each piece more flexibility in how it's developed.
The folks at Lucid were starting to get a little worried because I would bring them review drafts of papers arguing for worse is better, and later I would bring them rebuttals against myself. One fellow was seriously nervous that I might have a mental disease.
after over a decade of thinking and speaking about it . . . [I wrote] "Back to the Future: Is Worse (Still) Better?" In this short paper, I came out against worse is better. But a month or so later, I wrote a second one, called "Back to the Future: Worse (Still) is Better!" which was in favor of it.
When those maxima are things like operating systems and programming language ecosystems, those are very very big hops to make.
Code is a tool for exploring and understanding problems as much as it is about solving them. Sophisticated solutions can't be designed before they are validated.
Part of the problem is that it's next to impossible to separate the facts from the religion in software domain knowledge. Software artifacts are simultaneously art, technology. math and religion.
It's also hard to separate the abstract knowledge that is broadly applicable from its original context where it was hard won: the particular operating systems, toolchains and tech stacks.
Even that knowledge which is separable is mired in the jargon of those systems, so that it looks like it is specific to them. If those systems are long obsolete, then it's easy to dismiss anything that is robed in their jargon as being obsolescent by association.
If civil or electronic engineering were like software, then every five years, someone would be reinventing a class B push-pull emitter follower amplifier output stage, or Pratt truss, under different names.
I've got some abstractions out there I wrote early on when I didn't think I coded well and they have lived for years and saved tons of time.
Others don't live long :(
So many times I've taken code that was a mess because someone tried blindly DRY code for the sake of being DRY and rub abstractions on top of it and I've reverted the code back to being simple copypasta and its become so much more clearer and robust. Then you can look at the result and a simple abstraction may pop out which can reduce enough code duplication that the result is satisfactory (it may require a bit more boilerplate in the subclasses or whatever, but nothing likely to be brittle under future fixes).
A person who possesses music theory can explain why they are composing something they way they are, and suggest better notes to someone who is trying to embellish the music with a harmonization or whatever. Just like the people who held the "theory" of the compiler were able to spot better ways of implementing the ideas of the second group without spoiling the design.
Abstraction always costs. This implies any abstraction you put in that doesn't deliver commensurate benefits makes your code fundamentally worse.
If you have an abstraction with a name that seems to say it does X, but to be sure I have to trace through four other source files plus an unknown number of dead ends just to see if it really does exactly just X, and in the end it could have just been coded in place, it has already cost way more than any benefit it could yield.
Abstraction is an engineering tool, not a moral imperative. You can always add abstraction later.
So, "Worse is Better" is at best misleading. Better is better. But the measure of "better" you have been using is likely to be way off. People who think of themselves as smart tend to undervalue simplicity. It is a personal failing.
If you have to verify that "it does X" by following source files and tracing execution then you're talking about indirection.
An abstraction has mathematically sound laws that can be proven and introduce precise, new semantic layers to a program. By definition one doesn't have to think about the layers underneath the abstraction and can instead think entirely in the abstraction.
The difficulty with abstractions is that few practising programmers know how to think even informally about abstractions. The mistake of substituting abstraction for indirection leads to the misguided notion that virtual methods and classes are "abstractions." However if you try to formalize these abstractions with relations and properties I think you will find most of these proofs difficult, if not downright impossible write. That's a good sign you don't have an abstraction.
My take is that one should program in function of what the code should do, and not in function of what it's comfortable to me (as a developer).
Yes, it's a great feeling when your code fits like a jigsaw puzzle, but also more complexity = more code being executed. Behind that RAII, behind that "operator=" and that "p = new Struct", etc. there might be extra complexity for the sake of developer's readability and comfortability. There is little or no added value for the end user or the purpose of the program itself.
Also the code should be written "for the now", not for "that future feature it would be awesome to have someday like making it compatible with every other library X, etc.".
At the end of the day, even without realizing it, your program is slow.
I remember a developer where I work did a C# implementation of an AT command parser, in which every AT command was a separate DLL. It was very complex, and super slow. But the developer argued "if I need a new AT command, I'll just add a new DLL". It might have been better for him as a developer, but it was worse for the end user and the system in general. The code died the day that guy left the job.
The original article called 'worse is better' was pretty successful and widely read, and part of it was the title grabbed people's attention. You could argue it's a rephrasing of "less is more" or "do one thing well" concepts for the click bait zeitgeist. That's not a bad thing - an old concept wrapped up in a new way of expressing can keep the idea alive - anyway it's funny and a little confusing, which might trigger someone to engage with it differently. There's many concepts that I've heard n times expressed in different ways only to have it click on the n + 1 way of phrasing it.
'Yes, it's a great feeling when your code fits like a jigsaw puzzle, but also more complexity = more code being executed. Behind that function call, behind that nested expression and that "p = malloc(sizeof(*p))", etc. there might be extra complexity for the sake of developer's readability and comfortability. There is little or no added value for the end user or the purpose of the program itself.'
See? The same argument can be used against C in favor of assembly. Abstractions allow us to write safer, more correct code in less time, which actually does offer a lot of added value for the end user. C makes many useful abstractions harder or downright impossible, which is one reason why software written in C is generally such a shitshow from a security perspective.
There is no way to avoid most of things you listed. It's not comfort; it's a necessity.
> software written in C is generally such a shitshow
Whatever language you use to code, all your calls will sooner or later pass through a function call, a nested expression or a malloc()/free() call, somewhere in the huge stack of C code your language needs, depends and relies on. There is no escape. Basically, today your preferred language exists and might be able to something just because C code exists.
The folksy software adage for this is YAGNI. (You Ain’t Gonna Need It)
The software industry might be better off if we start making explicit that we have a real trade off of time and effort between designing a system that can handle all contingencies, and one that needs adjudication occasionally, i.e. throws an error or crashes, needs patching, etc. [2]
Notice that Rui’s solution here was to basically allow lld to just crash under rare, extraordinary cases. This seems reasonable when written here, but in the real world suggesting that a system be allowed to crash, ever, often gets very hard pushback.
[1] “Smart Contracts and DApps: Clarity versus Obscurity” https://youtu.be/JPkgJwJHYSc?t=3512
[2] “Hard-assed bug fixin’” Joel on Software (2001)
You could crash, but you can also throw. Somebody somewhere sometime might be able to do a better job with the situation than you can right there and then. Checking status codes all up and down the call graph does not increase the likelihood that something could be done, but costs in the meantime. Hiding the checking behind abstractions does not reduce those costs.
This is a very big assumption.
Edit: The assumption is that this day can ever come. The author decided that in his system that day would likely never arrive.
The central tenet of worse-is-better is to prioritize implementation simplicity over user experience. But if you simplify the implementation since some features are never used and not needed then you haven’t even had to make a choice on that spectrum—you have just cut out unneeded cruft.
In fact the user experience has improved since it is faster...
This post is stating that they tried to generalize too much instead of building to narrow use cases first. There is no need to bring "worse is better" into it at all, IMO
In that sense, every more or less radical simplification that sacrifices some of those fine (theoretical) advantages for simplicity can be said to be an example of "Worse is Better". Because what "Worse is Better" means is just that "theoretically 'worse', but much simpler code is actually better than theoretically 'better' but much more complex code". It's "worse" vs "better" on all those fine more or less theoretical dimensions against "worse" vs "better" on the single dimension of code complexity / simplicity.
So I'd say this article, which was all about how a radical simplification of the code -- sacrificing the (unnecessary) generalization and flexibility they'd first tried to build into it -- turned out to bring a lot of other practical advantages, is a prime example of "Worse is Better". As I understand it.
I find this an odd take on that paper. To me it has always been about how simplicity is more important than correctness. And while the authors take doesn't conflict with WiB, I do think they miss the point.
> "Simplicity of implementation is very, very important, and you can sometimes sacrifice consistency and completeness for it."
They almost get it in the conclusion. They are starting to see the important of simplicity, but still are fixed on correctness. Simplicity is not something you sacrifice correctness and completeness for "sometimes", it always wins if there is a contest. It is always more important (at least in line with WiB).
Not quite: The absolutely simplest code is an empty code file. But that doesn't do anything when compiled, so I doubt it would become very popular; people want their software to do something. And if that "something" is equivalent to "rm -rf /", they'd prefer it to do something correct in stead. So code simplicity doesn't always win; having at least some darn somewhat-correct functionality easily beats it.
Interestingly I have encountered crashes in Ninja (not lld), caused by corrupted on-disk state I had to delete: https://github.com/ninja-build/ninja/issues/1978. I think I traced it down to a memory indexing or null pointer error, which would've been caught by asserts but they were disabled in release builds.
The real world is messy. Things that come into contact with the real world will acquire a little bit of wear and mess. If it's all still brilliantly clean and tidy, you can't have done anything useful yet!
One of the most useful ideas in developing reasonably complex systems that I encountered is treating different types of errors differently. There's a place for both panics/exceptions on one hand result monads/error messages on another.
The problem is that this does't go far enough - decades ago machines were running on a single CPU and OSes were focused on the scheduling of processes. Syscalls were all blocking, so for each individual process there could only ever be one syscall ("request") in flight at a time. Now, we're seeing a change (for example with io_uring) towards fully asynchronous I/O exposed to the userland, which allows submitting multiple requests to various I/O devices simultaneously, which has the potential to improve throughput a lot.
Likewise many programmers don’t seem to get that the incentives of a programmer are quite different than the incentives of a manager. Programmers think “aw man my stupid manager is making me push out features instead of refactoring, if I were in charge I’d focus on code quality and performance”, not understanding that maybe, just maybe their manager might have different incentives and a different perspective.
Worse is better is essentially a shorthand for understanding product management and scoping.
This is one of the reasons sum types are so critical in my opinion. They let you write code in that style, where OOP forces you to make everything look kinda the same.
Also, it's not OK to aim to increase the amount of code. //TODO include both statements in the creed.
Yes; people who unit test.
In fact, neither design is the "better".
In not-so-modern-anymore POSIX, you can choose whether a system call will be restarted after a signal is handled, or whether it will terminate with an error. Both requirements are needed.
It is signals themselves that are "worse". But they let you have asynchronous behaviors without using threads.
Sometimes you want a signal handler to just set some flag. This is because you have to be careful what you do in a signal handler, as well as how much you do. And then if you want the program to react to that flag, it behooves you to have it wake up from the interrupted system call and not go back to sleep for another 27 seconds until some network data arrives or whatever.
In addition to sigaction, you can also abort a system call by jumping out of a signal handler; in POSIX you have sigsetjmp and siglongjmp which save and restore the signal mask. So that would be an alternative to setting a flag and checking. If you use siglongjmp, the signal itself can be set up in such a way that the system call is restarted. The signal handler can then choose to return (syscall is restarted) or bail via siglongjmp (syscall is abandoned). I wouldn't necessarily want to be forced to use siglongjmp as the only way to get around system calls being always restartable.
Anyway, the Unix design showed to be capable of being "worse for now", and have space to work toward "better eventually".
In the present story, the monolithic linker design isn't "worse". Let's just look at one aspect: crashing on corrupt inputs. Is that a bad requirement not to require robustness? No; the requirement is justifiable, because a linker isn't required to handle untrusted inputs. It's a tool-chain back-end. The only way it gets a bad input is if the middle parts of the toolchain violate its contract; the assembler puts out a bad object file and such. It can be a wasteful requirement to have careful contract checking between internal components.
Gabriel naturally makes references to Lisp in the Rise of Worse is Better, claiming that Common Lisp is an example of better. But not everything is robust in Common Lisp. For instance the way type declarations work is "worse is better": you make promises to the compiler, and then if you violate them, you have undefined behavior. Modifying a literal object is undefined behavior in Common Lisp, pretty much exactly like in ISO C. The Loop macro's clause symbols being compared as strings is worse-is-better; the "correct requirement" would have been to use keywords, or else symbols in the CL package that have to be properly made visible to be used.
I don't think that Gabriel had a well reasoned and organized point in the essay and himself admitted that it was probably flawed (and on top of that, misunderstood).
The essays is about requirements; of course the assumption is that everyone is implementing the requirements right: "worse" doesn't refer to bugs (which would be a strawman interpretation) but to a sort of "taste" in the selection of requirements.
Requirements have so many dimensions that it's very hard to know which directions in that space point toward "better". There are tradeoffs at every corner. Adopt this "better" requirement here, but then you have to concede toward "worse" there. If we look at one single requirement at a time, it's not difficult to acquire a sense of which direction is better or worse, but the combinations of thousands of requirements are daunting.
If we look for what is the truth, the insight in Gabriel's essay it is that adherence to principled absolutes is often easily defeated by flexible reasoning that takes into account the context.
3.1415926 is undeniably a better approximation of pi than 3.14. But if you had to use pencil-and-paper calculations to estimate how many tiles you need for a circular room, it would be worse to be using 3.1415926. You would just do a lot of extra work, for no benefit; the estimate wouldn't be any better. Using the worse 3.14 is better than using 3.1415926; that may be the essence of "worse is better". On the other hand, if you have a calculator with a pi button, it would be worse to be punching in 3.14 than just using the button, and the fact that the button gives you pi to 17 digits is moot. A small bit of context like that can change the way in which the worse-is-better reasoning is applied.