Consider an example:
You have made a little web app that beeps at certain times of the day to remind me to do something.
Is it complete?
Oh, you want a calendar integration. That makes sense. You add it. Is it complete?
Oh, you say you want to release mobile versions? Okay, now is it complete?
Sorry, but the iOS version needs to be updated to remain compatible. Now is it complete?
There is a new popular cal app that everyone uses. Should you update it to work with that?
The problem isn't that the software is incomplete. The "problem" is that the world keeps changing.
To the extent that the environment doesn't change (unix), you can "complete" your tool.
If by "Unix philosophy" you mean creating modular software, then sure, we're already doing this. If you mean pushing unstructured ASCII data through actual pipe(2)s, then I'm sorry, but this is not a workable solution in 2017.
I'd also argue that the "change" you are seeing is mostly illusionary, but that is another story altogether.
Contained solutions can be only designed for contained problems. Sure if you define a limited set of requirements you can finish your project and will have a limited software that is only useful in that very limited context.
"change" might be unnecessary but it is certainly not illusionary.
Now, granted, must of us on anything Unix-like is on a BSD or Linux, not on a UNIX, but grep and make are still being patched, here and there. Even 'ls' gets commits. We could get into a pedantic discussion on when something can be classified "finished," but that doesn't seem fruitful. I think we just need to realize that despite following the Unix philosophy (or something similar), which is mainly about modularity and composability, not a single part (modular piece, service, etc.) of modern systems can be left unmaintained, it can never be in a "truly finished state."
Or you build an integration / plugin system, and each new feature is a self-contained plugin to do that one thing well.
When you ship it it should be the example of "the little web app" and when you have others use it. That is "finishing". When you have something that has a modicum of value.
If you add more features in response to users or sales or whatever metric that is the rest of your comment. Or maybe you don't need any more features in the future maybe its only bug fixes like TeX or the Python Requests library.
It's one way to look at things anyway.
Or, to be less terse, don't say "Well, you shouldn't need to solve that problem!" or "You're holding it wrong!" Reality doesn't give you neat problems which map cleanly to the simple solution, and adding complexity is the only way to solve the problem, as opposed to a dishonestly reduced form of the problem.
Even though the library worked perfectly fine several years ago and had many users, several years later it seems to show up some kind of errors that most people didn't experience, certain tasks require workarounds or are down right impossible to do.
And the reason fir it is that the environment where the library operates changed and most likely it is used differently than in the past, but it gives a perception as if the quality of software actually degrades with time.
I feel as though that's somewhat addressed in the essay:
Yes, I hear you claiming that your project is special and
cannot be made functionally complete because, you know,
circumstances X, Y and Z.
And here's the trick: If it can't be made functionally
complete it's too big. You've tried to bite off a piece
that's bigger than you can swallow. Just get back to the
drawing board and split off a piece that can be made
functionally complete. And that's the component you want
to implement.
Once they're small enough the Unix philosophy applies. Write components that do one thing and do it well, and design them in a way to communicate with each other.
Too big for whom? Most people on HN aren't building systems software or embedded software with a strictly-defined purpose; they're building an app business or a service business, where the "purpose" is "to make money" and adding additional features or "checkboxes" are how you attract more customers to make that money.
A program can be done. A product usually can't be, as long as its authors want to continue to rely on it to put food on the table.
Of course that also means slowing down, and obviously that won't do in silicon valley. Gotta go fast, even if you are running around like headless chicken.
If you're designing for the marketplace, the market decides the requirements, and expecting everyone to fall in line behind one thing is a mugg's game.
Or change after it's built. "Oh yeah, I guess we'll also need a floor to set our chair on. And perhaps a foundation for that floor." Later: "I know I said I wanted a chair, but what we really wanted was a glider."
People outside this school of thought don't finish their projects because their projects actually try to solve the underlying problems in computing, which are inevitably hard.
Out of the frustration with AMQP I've started my own ZeroMQ project.
[1] https://en.m.wikipedia.org/wiki/ZeroMQ [2] https://en.m.wikipedia.org/wiki/Pieter_Hintjens
Disclaimer: I knew Pieter personally and met him several times at his home and at conferences.
Three examples I'd recommend:
Life, Consulting, Humility, Calming Down: http://hintjens.com/blog:125 (confessions of a necromancer)
Presentations: http://hintjens.com/blog:107 (ten steps to better public speaking)
Life, Humility, Time: http://hintjens.com/blog:123 (fighting caner)
He's one of the few online writers who keeps my attention no matter the length. His humor is fun and his imagery and story telling style are engaging.
[edit: formatting]
I am not doubting that Pieter Hintjens deserves respect, but the ZeroMQ project was started by Martin. He wrote the vast majority of the code for the first two years, none of which was written by Pieter. Martin's company owned the copyright for the first 2 years, before it was purchased by Pieter's company. The only messages I see on the ZeroMQ mailing list from Pieter in those days are messages of congratulations and enthusiasm. I know Pieter and Martin worked together at iMatix, and that some of the ideas for ZeroMQ spun out of their mutual work with AMQP, but it is fair to say Martin started the ZeroMQ project.
I understand how Pieter presented things, but I think he did so with a revisionist viewpoint. I don't mean to disparage him or downplay the immense impact he had on the later success of ZeroMQ though.
While it's very good to encapsulate functionality, and have stable and well-defined interfaces, there is also a great value to having a unified platform and community that builds interoperable things.
Would you rather assemble your project from 250 different libraries, some of which may be incompatible with other ones? Sure, each may solve a tiny problem, and they may all be orthogonal (best case) but there is a major need for a unifying paradigm above all those components so they can all work together.
Here is a counterpont to solving tiny problems: http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how...
Basically, I prefer to have a growing community working on a growing snowball of components that are all interoperable and can be assembled like lego bricks. I think Wikipedia has such a community. RDF has several such a communities. The Web has such a community. This really creates a lot of value by having an exponentially growing platform where many thing works with many other things.
There’s a package called is-positive-integer (GitHub) that is 4 lines long and as of yesterday required 3 dependencies to use. The author has since refactored it to require 0 dependencies, but I have to wonder why it wasn’t that way in the first place.
Also as a web developer completing projects for different clients regularly I don't see the point of the whole article. Wouldn't a successful delivery of a campaign site be considered a finished project? Especially with the analogy made it's like comparing apples to oranges
I completely disagree with your analogy to a carpenter who builds a chair, this metaphor is wrong and responsible for a lot of misunderstandings when it comes to software development.
I also don't agree that "finish your program" needs to be added to the unix philosophy because a tool that does one thing and does it well is finished by definition.
[0] https://www.tug.org/TUGboat/tb11-4/tb30knut.pdf [PDF, in case it wasn't obvious]
Basically what I'm saying with that example is that you can draw a line around some code and say "this is TeX, and it is finished." But what users mean when they say "I use TeX" will never be finished and will be forever expanding.
TeX is the hardest to argue because it is probably the most stable software in existence for what it's capable of, so any other software you use is much less finished.
505 Bad Analogy. Carpenters work all the time on new designs and better chairs. When a programmer releases a new version of his program, it's like when the carpenter made a new chair and put it up on sale on Amazon or ebay (aren't these kind-of like package managers for physical goods?). You may want to buy that new one, but you may also not. Similarly, nobody forces software upgrades down your throat. But sometimes the people change, they want to sit on divans or nail-beds instead of chairs, the carpenters follow the trends and produce those instead of chairs, and people expect to find these when they come to visit you. You can say "sit on chairs or f*ck off!", but you can't avoid the consequences: loneliness. Similarly, you can hold on to your select programs in select versions, maybe backport patches for them, but the protocols change, the services change, world goes on. You may say eff-off and stick to your programs, but the consequence is that it'll become harder to share experiences and information with others, both ways. Nobody fixes or modifies your stuff should you not want them to, they may try hard to sell and bug you, but you always are the one that buys. Mobile phones do by default, but you can opt-out.
Will they?
If your users are programmers, they might love you for that.
For pretty much all other users, they will want more. "It would be great if your app did...". I would argue that users now expect software to change and add more features.
And why wouldnt they, when pretty much all software they use is doing this? If your app isnt growing and changing, to users it way well look dead: "It's abandonware".
And I think the limited data we have shows that, like the number of people who upgraded their browsers before it was automatic.
Then, every other day, the carpenter turns up at work and tries to improve how they make chairs!
Why can't they just stop changing how chairs are made? I like my chair, but I go back to the store after a few years and they're all different! Carpenters, finish your chairs. Those ideas you have? Stop having them. I like chairs how they are now.
Not sure if you're joking here, but if not - then no, it's not because of that. Companies jsut make new, "better" versions, market them hard, and are happy because when they pull the old ones from the stores, you have no choice but to buy one of the new ones.
If a project is going to be used by a lot of people or become some kind of industry standard, then it makes sense to spend some time up-front and figure out a clean interface and a well-defined feature set so you don't have to make non-backwards-compatible changes. It also makes sense to look at the feature set and figure out if you have the time and ability to complete those features, whether they're important enough to justify the effort, and whether there's an easier option that satisfies the requirement without having to boil the ocean. All this should be driven by the project goals, though, not some abstract ideal that all programs should be complete.
Some of the most useful software is going to be forever incomplete. Web browsers, CAD packages, operating systems, etc... I'm glad there are people working on software like that.
Following his development path (AMQP->ZeroMQ->nanomsg + libmill + others) his interest was in decomposing the larger system (AMQP which included message queues, databases for storing messages, wire protocols, etc.) into its smaller parts so that you could build back up to it in a better way. It happens that when you get to those lower levels, you can call them done.
If you're building a web browser, what's the proper scope for your project? Should it actually consist of all of these things in one project: tcp stack, http client, javascript interpreter, html parser and renderer, keyboard drivers, mouse drivers, touch screen drivers, etc.?
No. You build on the existing OS for the drivers. You build on the OS for the TCP stack. You may build an HTTP client, but odds are you can reuse an existing one, or the one you make could be partitioned off into a separate, and hopefully reusable, library. Your javascript interpreter can also be its own project/library. And most of that already exists, so you can focus on HTML parsing and rendering and glue in the other components. Your HTML parser doesn't need to know how to send an HTTP request, your renderer doesn't need to know it either. Only your browser needs to connect the HTML renderer and the HTTP client.
I think breaking things up into small pieces so that it's modular and you can work on one part at a time and then assemble a working system out of individually-testable working parts is a good thing (especially if you can re-use parts that other people have already made), but I don't think that's quite the argument the author was making.
I think the author was arguing that you should reduce the scope of the task any time you notice open-ended requirements with no clear completion criteria. I think that's good advice most of the time, but there are exceptions. It's okay to have a project that isn't "finished" if it's useful.
Death to evergreen software! Let projects be finished!
Being a command line junkie, what stands out to me is the composability of my tools. This composability, I feel, is the key that let's us separate concerns and write small, standalone tools.
Well, I do :) - given that said editor also has better UX than mainstream operating systems (in terms of efficiency, extensibility and interoperability), and it's because of that it is possible for someone to implement Tetris in it.
I think the point the author is making is to not start projects that aren't likely to have a minimum viable product that is usable and achievable within reasonable time.
I'm not so sure. It's more fun to write the skeleton of a compiler and abandon, than it is to write something minimal sometimes. The only (proper) way to learn what tradeoffs exist in sql database design is to make one. Making a sql parser, btrees and so on is HARD but rewarding. After a week you'll know a lot about a lot of things. But you likely won't have a working SQL database. If you aren't a very special kind of crazy you won't reach the finish line and you'll abandon it somewhere along the way. So was it all a waste? Is exploratory hobby programming somehow a bad practice? I can't see that it is.
Complex systems are constructed from smaller, less complex subsystems. Complete them, let them standalone and build out our final systems from them. We generally don't rewrite tools like grep because it exists and it works well, we may port it to new platforms. And we don't change the effects of the -R flag out from under the users (clients), it is what it is. The interface is, essentially, completed.
If you're leaving incomplete hobby projects in your wake (all of mine are), that's not what he's talking about.
His specific experience was going from AMQP (large, complex specification) to ZMQ (smaller, focused on the essentials and let complexity be built on top) to nanomsg and libmill (even smaller, more focused). So in theory his nanomsg and libmill could be considered complete, as we'd consider TCP "complete", today (from a client perspective at least). And we can work our way back up plugging it into systems to recreate the capabilities of ZMQ, which can be plugged into systems recreating the capabilities of AMQP. Which can then be used to solve our business problems.
This goes far beyond programming. "completeness" is nice but is highly subjective. For work that can be infinitely revised, refactored, recut, "finished" is an idea you impose from the outside, in relation to specific criteria.
The author seems to be saying that there is value in creating smaller, independent modules of things that can be said to be "finished" when they do their thing and no further functionality is added or expected to be added. Further functionality, if needed, would come from connecting "finished" things together, I suppose, or making new things- not over extending a component to handle all cases.
This moves the complexity around a bit and maybe it's more efficient and easier to handle this way.
Artists have a similar problem. It's hard to know when a poem or a song is "functionally complete" but eventually you have to let them go or you never get anything out there.
I'm all for conscientiously abandoning work, especially if the state of the work is well documented so that somebody else could, in theory, pick it up.
This is why I think a "definition of done" is so valuable on a dev team, especially in a startup environment. Without it, there's just natural inconsistency or variability across people and features.
If your work consists of half-finished code that you then attempt to pass off as a usable product, the expected reward is shunning.
There's a difference between pushing your CS101 homework to github and publishing your package to npm. As with academia, once you publish you implicitly vouch for the quality of your work, and your reputation is permanently tied to it.
Yes, it’s important to generally finish what you start. Equally important is the capacity to accept, learn and make changes
That problem along with I'm sure hundreds of others were fixed along the way.
You don't achieve perfection in software development. Ever. Even when developing something for yourself where you define the scope and you leverage it for a menial task. There's always better. There's always more robust. There's always something else you can do.
That's the draw to software development for many of us. There is a constant challenge awaiting our brain. You can consistently push yourself in a new direction that you care about.
The downside is that feeling of a completed accomplishment really is just an arbitrary release ceremony. What software developers need to get good at is letting go and moving on.
Software engineering is not flawed. It just lacks hundreds of years of experience which it will gain with time.
So the problem is not so much to solve a problem but to FIND a problem whose solution is useful on its own and perhaps also as a sub-solution to bigger problems.