That said, the actual problem described in the article has this issue at the very core - we have more software we need written than we have people who can write good Haskell. This means that a large percentage of software is going to be written poorly - and the only real solution would be to simply have less software.
Supply-Demand overrides this completely, and so bad software will remain. The true challenge is therefore making software that can be written by the semi clueless and yet still be decipherable enough to maintain. Java and C# seem to have cornered this market to great effect in the business landscape from my experience.
I think current language development trends towards functional programming is a mistake as it caters to such a tiny audience, and more emphasis should be placed on new languages that can force people to write good code.
Is this possible? Dumb example: If someone is hacking together computations of circles without knowing about PI, is there any possible way for a language to help here?
I think a language that can make it clear what the code is trying to do, even when written by someone who has no clue, would go a long way to being able to filter out and fix bad code quicker. If your code is in Haskell, it's going to take a lot of effort to work out what bad code is even trying to accomplish.
I interviewed for jobs a few months back. When people asked me "solve this puzzle in any language you like", I usually used haskell. The solutions were just simpler and cleaner in Haskell than in Python/Java/etc. Apart from minor conventions they were unfamiliar with (e.g., "What's Maybe"), not a single interviewer had a problem understanding what I did, even though most had never used Haskell.
One company even had me demo some code I wrote (great interview technique, BTW), and I picked a Haskell project [1]. They literally stopped my detailed verbal explanations and told me: "no need to explain getFileHandleOrNothing, this all looks pretty clear and obvious. The only thing we needed an explanation of was your monad, the rest is clear."
You advocate that more emphasis should be placed on new languages that force people to write good code. That's what Haskell and other FP languages do.
Edit: actually when interviewing people I show them some Haskell code and ask what it does. Most of them do, some answer that they don't know the programming language. Of course I don't give them very difficult stuff, but still; a lot of well written Haskell code is incredible readable, even for people who never saw it. Also, I really assume if you have a CS degree that you played around with functional languages and recursion. My disappointment with a certain country was that almost none (and I interviewed 100s in the past 10 years) of the Masters of CS from that country even have a clue what recursion is, let alone functional or logic programming languages.
I don't get to write haskell at work, but I see its principles everywhere with an additional layer of cruft over the top. Examples include doing functional style programming in python, linq in C#, passing around functions and callbacks in JavaScript and trying to reason about side effects.
http://uncyclopedia.wikia.com/wiki/Haskell#Haskell_Code_Exam...
It's like using code from the IOCCC to make a point that C code is not maintainable.
pow = 1 : [ 2*x | x <- pow] pow = iterate (* 2) 1In my limited experience, Haskell is like any other PL, you can write clean, readable code or a horrible mess.
My major concern with Haskell right now is with maintainability/changability of code. One recent example that bit me in the ass was: I wrote an elegant solution to check whether a list of files exists, using sth like foldr (&&) True doesFileExist. Then I realized I really need to print out which file is actually missing, but only the first N to avoid flooding the user with error messages. At this point FP becomes a pain IMO, and it'd be much easier to add a counter variable and printfs in the imperative program and move on. Actually this is fairly common, think about adding printfs for debugging: not so easy in Haskell, as IO changes the type signatures of the function and its callers. So, in my limited experience, changing code in Haskell is expensive and its effect are not localized.
First, I think your problem really has two tasks to perform: getting a list of files that don't exist, and selecting only a part of those inexistant files. It would make sense to decompose these two tasks into two different functions, and since Haskell has non-strict evaluation, we don't have to worry about generating too much data (list of inexistant files) for nothing. Starting with the function to select inexistant files:
inexistantFiles :: [FilePath] -> IO [FilePath]
inexistantFiles files = filterM (\f -> doesFileExist f >>= return . not) files
takeInexistantFiles :: Int -> [FilePath] -> IO [FilePath]
takeInexistantFiles n files = takeM n (inexistantFiles files)
where takeM n = liftM (take n)
Have a Merry Christmas!
Some things I find help with maintainance : * the type system allows the compiler do a lot of work for you, * types have to be concrete rather than abstract, * if a little effort is applied the unit tests can be amazing, * everything's clearly defined, * codebases tend to be a bit smaller, and once you're used to it, more readable.
In the particular situation you describe, maybe using the "take" function could work?
Yes, the IO stuff can be tricky, I agree, but with practise it's not too bad, and separating IO from pure code gives all sorts of goodness back. There's always Debug.Trace if you need to do a quick bit of debugging by print statements.
I feel the article was being unfair on Haskell there, after all the code example was being deliberately unreadable, and that's possible in any language.
It's not really that hard:
map (2^) [1..]
A "first n files that don't exist" function could be something like:
nexist :: Int -> [FilePath] -> IO [FilePath]
nexist n l = take n . map fst . filter (not.snd) . zip l <$> mapM doesFileExist l
That's a feature, because it protects against spaghettification. Explained here: http://michaelochurch.wordpress.com/2012/12/06/functional-pr...
When a manager walks in with a new requirement, you'd rather be able to do it in 15 minutes than 2 hours, because managerial injections are annoying and you'd rather get back to more interesting work. Sure. I get that.
However, once enough hands have passed over code, with hackish fixes thrown in to satisfy dipshit flavor-of-the-month requirements, then if it's easy to inject complexity without consideration of the loss of conceptual integrity, people will do so. What you get over time is spaghetti. It doesn't happen overnight. Almost no one sits down and says, "I'm going to write a bunch of shitty spaghetti code."
Spaghetti code is rarely written by one person. It's an "all of us is dumber than any of us" phenomenon. Even Google has some spaghetti code, and I've never met an incompetent engineer at Google. (Project and people management would be a different discussion.)
The nice thing about FP is that changing behavior (except for performance) actually requires altering an interface. In a statically typed language, this requires that you make changes elsewhere in the code to accomodate the new information flow. Yes, it's time-consuming, but it also forces you to think about how your change is going to affect the whole system. That's a good thing.
Writing code shouldn't be easy, because reading average-case code is impossible. If it takes 50% longer to write code, but the result is that people can still comprehend it at scale, I call that a major win.
I otherwise agree with your b) and c) points.
I live in London, arguably the current world financial capital, but I'd have to be fairly desperate to work for a bank. (Not suggesting I'm the very best, of course, but rather taking banker money is like being a whore.)
- Correcting the interviewer on C++ intricacies IN the interview. - Very poor process of dev/test/release cycles - Management of 'uptime' by having c++ devs on call 24/7 with Limo's, hotel rooms etc but no thought to proper testing, regression or postmortem. - Managers that managed by fear and intimidation - People who were clearly there because they were the only ones who could understand the shambles of code they'd written and were paid astronomically to not leave. In general they were mathematicians who were presumably great at maths and poor at engineering software.
While they were both very good and have moved on to more interesting ventures they earned more in a few years of pain then they would have in 20 years of game development so it was worth it for them.
What if you're a barber who cuts a banker's hair, is that "like being a whore?" What if you sell software to a banker, or you're a nanny for the children of a banker? Also: do you have any respect for actual women who engage in prostitution?
The other jobs you mention have different power relations compared to the employee / employer relationship, and it's the dynamics of that relationship in investment banks, and their culture, that disgusts me.
No.
Ditto for drug dealers.
Would it be whoring if you took money from Zynga instead? Groupon? Facebook? Yelp? Or any number of other disreputable companies that have engaged in allegedly immoral/illegal behavior?
FWIW I don't even work in banking.
"The technology people employed at these companies are considered to be the very best..."
Who considers these people 'the very best'? Compsci majors? The hiring manager? Of course people in that sector are going to say "we only hire the best", but at best they're only hiring the best who bother to apply or that can be recruited. Very very very likely "the best" (by other measures) stay the hell away from finance.
In some industries, having extreme expertise with specific algorithms and other very deep-knowledge of specific business rules coupled with top-notch programming skills is going to be required for some tasks. Outside of that, by and large, superhuman math/algo skills aren't needed as much as other broader skill sets (inter-departmental relations, listening, etc).
It's the mentality of the financial sector: If you spend the most money, you are going to get the very best. If you earn the most money, you are the best. They assume that because they pay more than anybody else, they obviously must have the best talent. It's probably unfathomable to a lot of these people that there are motivations for developers other than financial ones.
That said, he's got a point, but the only solution he gives is "Include some comments!" I've seen so many shitty comments that I really think this is no advice at all.
Four levels of software maturity. (There are others. This is a simplified model.)
Level 1: Your work doesn't exist until it's in version control. It might be amazing to 2012-era programmers that this could be controversial, because even the laggards have reached L1 maturity, but at one time, it was. (See: the Joel test.)
Level 2: Your work doesn't exist until it has tests at multiple levels (e.g. unit and integration). Otherwise, it has no enforced semantics.
Level 3: Your work doesn't exist until it has well-documented semantics in addition to the tests, which cannot cover all cases. Otherwise, how does one even decide what a "bug" is?
Level 4: Your work doesn't exist until you've provided the resources (e.g. a code walk or an interactive module) to teach people how to use it.
If you're at Level 1, you won't lose source code. If you're at Level 2, your breakage rate will be reduced, and you can run CI. At Level 3, your system integrity will remain decent because you don't end up with those software-as-spec modules that, over time, corrupt the whole system.
Guess what? It takes Level 4 to get good code. Unless there is a culture of teaching (incremental code reviews are not enough) within the company, you will not have good code for very long. Since engineers are intolerant of context switches and in-person teaching doesn't scale, the teaching should be automatic. Interactive modules or, at the least, code walks should come with the technical asset being demonstrated. If people don't know what they are looking at, then how can they be expected to review or maintain code. With the worst code, it's not even clear why it's bad. It's just incomprehensible.
Level 5: your work doesn't exist until it has been formally proven to be correct.
It's obvious that the amount of effort involved in this is so huge that it should only be considered in a select few problem domains. The same is true for the other levels. They're just a scale of increasing formalization/rigor, but you've said nothing to justify why they should be applied universally.
I think it's important to keep in mind that these levels (or something analogous) exist, but different domains have different sweet spots and a deliberate choice needs to be made for each project.
I think it's easy to judge from academia where you do an assignment, then move on to the next. But it only takes a couple of years in a real-world environment for code to get incredibly f'd up even if the developers were trying to be diligent. People come and go, platforms go out of style, programming fads come and go and, of course, one or two mediocre people on the team add their code to the mix.
Very insightful observation. Bad code is a problem that is both technical and social. Purely technical solutions won't cut it.
Consequently, business people don't trust us to do anything that resembles leadership. It's not just that they won't rely on us. They won't even let us take the time (which is why it's better to just do and ask for forgiveness, not permission) to do anything that involves future orientation or leadership.
The reality is that programming is a leadership role. When you write code and design systems, you will affect how other people work, unless your work is of such low quality that no one can use it.
The result of this is that we have a Greenspun's Tenth Law situation where the failure to address a capability results in it being met in an ad-hoc, ineffective way. Programmers aren't supposed to be multipliers, but such responsibilities emerge naturally and, when they do, the people picked to handle them (usually for political reasons, since business people are hopeless at evaluating software ability) are usually incapable of doing it.
Blog plug on this: http://michaelochurch.wordpress.com/2012/12/14/the-unbearabl...
Reasoning about your semantics and having the learning of the code be reified are certainly important. So then the question becomes how to incentivize helping that learnng
I think it does ruin the author's credibility. They didn't do the most basic research before including that code snippet, and are just holding it up as a distracting side show. The rest of the screed might have some value, but I'm pretty skeptical of that after reading the last paragraphs.
He follows this with a line of intentionally hard-to-read code.
Everyone who is familiar with Haskell knows that it's possible to write code like this, but it's not common. You have to actually work at it to make truly awful code in Haskell or ML.
The rest of the article is just random presumptions thrown together, for what reason I couldn't say. The best developers work in finance because the money's highest? Yeah, OK then... (To be fair, I think there is a qualifying "considered to be" in there somewhere) K is impenetrable? Hmmm. Haskell is an offshoot of ML and unreadable? Umm, factually wrong.
There's lots wrong with software at scale, and the pressures of financial institutions along with high budgets etc do produce an interesting set of problems, but this article offers nothing new about anything at all in the field.
As an aside, I already don't trust an author when they start off with a claim that textbook code is good code! It's almost universally accepted to be trivialised and simplified to the point of "don't do this in the real world" simply because it has to be. Some of the worst code I've seen has been in textbooks (and that's worst as in "dangerous" rather than as in "badly written and in need of refactoring/replacement/enclosing in concrete).
Edit to note: I think it's actually beyond incompetent journalism and in to plain lying to present a piece of code as evidence of Haskell unreadability when the link to that code on SO refers to it clearly as "obfuscated Haskell code".
YOU AND ME know that. The title of the article is "What Compsci textbooks don't tell you", so I guess it is targeted at people still in college, to show them how things are in the "real world".
I would call it a hatchet job if I could tell by reading it what the target is, but it isn't even good enough for that.
http://www.drdobbs.com/architecture-and-design/interview-wit...
Pop culture is all about identity and feeling like you're participating. It has nothing to do with cooperation, the past or the future — it's living in the present. I think the same is true of most people who write code for money. They have no idea where [their culture came from] — and the Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made. When was the last time a technology with a scale like that was so error-free?
I appreciate this categorization on many fronts:
- It explains how non-computer science majors have frequently succeeded. In music, you will never find a classical musician without formal training, but frequently find pop musicians who have little or none.
- It shows how some of the same ideas keep being "discovered" by one group while another group sighs and exclaims "we knew about that years ago..."
- Pop culture is inherently youthful, prone to extremes, sensitive to new trends. Sound familiar?
- It explains a vast amount of decision making that goes on in industry. Rather than choosing the absolute best solution from a somewhat abstract technical perspective, a solution that is new and hip (or fits what I was doing in the hey-day of the bosses youth) is selected.
It is over simplistic to simplify real world code as "sucking" (though of course much does). It is like comparing Bach and <fill in your favorite rock/pop star here>. Both produce valid creations that accomplish something, but were made with vastly different intentions and for different purposes.
http://queue.acm.org/detail.cfm?id=1039523
In the last 25 years or so, we actually got something like a pop culture, similar to what happened when television came on the scene and some of its inventors thought it would be a way of getting Shakespeare to the masses. But they forgot that you have to be more sophisticated and have more perspective to understand Shakespeare. What television was able to do was to capture people as they were.
So I think the lack of a real computer science today, and the lack of real software engineering today, is partly due to this pop culture.
And to return to the issue of code quality, academia is not a magical land of perfect code. Code I read in academia, in scientific and bioinformatics code, was easily one of the worst I ever read, and I currently work on a huge codebase that started 10 years ago as someone's attempt to learn a new programming language. The "best" code comes either from obsession, or from requirements actually going down to the code quality itself — either because it's used for didactics, or because e.g. reliability requirements demand actual proof of correctness (which is rare, but there's some industrial code that actually has this requirement, see Maeslantkering).
The closest I can recall are those with heavy emphasis on refactoring, because gradually moving from bad to good is usually the best one can hope for in the real world. I'm thinking in particular of Bob Martin's Agile Software Development, because I liked how he made the point in the book that he didn't just arrive at this beautiful code, but achieved it through TDD and refactoring.
I think "bad code" is actually what the most talented developers excel at (see also Duct Tape Programming http://www.joelonsoftware.com/items/2009/09/23.html). A talented programmer knows when it's worthwhile breaking the rules and, by necessity of getting things done, does so on a frequent basis.
Feathers (the author) methodologically chose to copy / inspire some of his examples from code he's worked with as a mentor / developer (some of it from his time @ Thought Works w/ Beck, et al.)
The book is similar to Fowler's Refactoring in that it provides some core thought processes / ideology and then a long list of heuristics and techniques for revising/updating legacy code. (I find his definition of "legacy code" as any software without automated testing to be challenging and insightful, especially since it indicts most code written ever. ;)
If you get the chance, you could also consider reading "Clean Code", it doesn't have /as much/ "real world" source but is still excellent.
This is a fairly natural series of events that can be helped by periodic refactoring. Unfortunately, because each change is small, none of them on their own justify the time required for the refactor. That's when you've got to hope you've got a management team that trusts your judgement, and is willing to carve out the time necessary for this stuff. Most people, alas, don't have that.
You just can't compare the requirements of "science" part with requirements of "real world business system". The "science" requirements are almost laughably trivial. And the documentation of business system will run into thousands of pages.
Now, you say if you just refactored the code, if you saw the "grand design" and rewrote the code to accommodate every little requirement elegantly and with proper abstractions and so on... I will say you would just end up with system so abstract and general, no-one would understand exactly how it is meeting those requirements. Because you would had generalized every special case (where it should've stayed a special case, and that's that).
So in a way, the simple hacked-together awful program might actually make more sense than the elegant and grand abstract program. Because you can follow the hacked-together program, and you can't follow the arhitected program. It's this "abstractions all the way down", where you can hunt abstractions down for "how exactly does it do this one thing" all day, and then find out it's a little config setting in obscure .properties file that actually covers this one business case. (And if it was there right in code, you would be done in 5 minutes. But alas - it's a config option, so to change behaviour, now you must write all infrastructure to support another possible config option.)
I say - bad code is just fact of life in real-world business. You should strive to clean it up, make it elegant, abstract and generalize... but really business just needs to to chug along.
Equally, I don't think that the opposite end of 'hacked together' is some kind of AbstractAbstractFactoryFactory extravaganza - that's just a different kind of bad code. To me, good code is code that is simple for other people to understand, and takes basic precautions with respect to being not-too-brittle to change. Every so often, I find that once the code has received enough alterations, it needs a bit of TLC.
Indeed, and even then usually crap is delivered.
Another assumption: the [thing to get done] will likely eventually be transformed via evolution or destroyed, as all natural things like rocks and species and hydrogen atoms and gluons and the entire known universe starting[?] with the big bang transforms via evolution or gets destroyed.
A conclusion: code that can not be easily changed and evolved will be conveniently destroyed via a complete re-write. This is why "facades" and opaque APIs are so popular - it is theoretically possible to rewrite different components of the code without rewriting the whole entire dang thing.
Case in point: http://stackoverflow.com/questions/810129/how-does-whiles-t-...
>>>while (s++ = t++);
This is code that is hard to understand(hence the stackoverflow question), making it not easily changed. Not a good sign for the survivability of the code.
JoelonSoftware.com makes a sparse point with this example, in that a "good" coder must understand what this does. He leaves the reasoning up to the reader, likely because he is clever enough not to make any claims that can be disagreed with. Luckily I am not this clever and I will make a claim that follows from my assumptions and conclusion above:
Claim: a "good" coder needs to understand the above type of obfuscated code so that they can evolve it or destroy it safely when they see it.
Note that my claim does not say whether a "good" coder would write code like that. A "good" coder gets the job done, whether it's a write-once-use-once-read-never perl one-liner or a big part of a big program. I only claim that a "good" coder must be able to read it, so that they can change it or destroy it safely.
is absolutely trivial to understand if the reader has taken the time to actually learn the language in which the code is written. Saying this is "hard to understand" is like saying that "Ich liebe dich" is hard to understand. I'm sure it might be if the reader has never encountered German, but that's not real complexity, and it's not unreasonable for someone writing in German to assume that her reader has some competency with the language.
Yes, you can understand it, but it takes more brainpower to do so than it should, especially once you get beyond trivial cases. It is much better just to write it the long way.
[1] In terms of needing refactoring, poorly named variables and functions, just doing The Wrong Thing you don't ever do in real life.
As a Haskell programmer, I thought the Taliban comment was funny.
Can we stop posting this crap, please? TheReg was kinda edgy and cool circa 1998, but now it just seems a forum for trollish and content free articles.
Bad code is buggier, so it will require fixing more often.
Bad code is more difficult to understand and sometimes has non-intuitive side effects or method of operation and so working on it will take longer.
Because of the convoluted way that bad code often works, adding features or doing maintenance on it often makes it even more convoluted and difficult to work on.
Bad code typically has lower performance, so it will require more attention to improve performance. Due to the above 3 factors this becomes a vicious cycle of sinking effort into the code only to make it that much harder to work on in the future.
For these simple reasons bad code becomes a huge time sink. Programming tasks involving good code end up taking up a much smaller portion of the developer-effort budget because it's so much easier to work on that code, while the bad code takes so much more effort and often the code base just ends up even worse than before, perpetuating the phenomenon through to the next cycle of development.
The only way out of this cycle is to identify the code that is the main source of bugs and the main sink of effort then budget out more than enough time to tackle it and improve it. But most development shops aren't proactive enough, introspective enough, or set aside enough dev-resources to be capable of doing that. Even though such efforts tend to have an enormous impact on future development.
Code in real world companies is mostly horrible, especially when the company gets bigger and older. Spolsky has a bunch of articles about that. To counteract or prevent that you need heaps of money put into it and most companies really don't see the need or actually don't have the funds.
The project I work for depends highly on usually inexperienced developers and we're using a development process that is task-oriented but at the same time based on continuous feedback and regular code reviews. Should it be any other way, things would quickly degenerate. That's what happened before I arrived, and we're still (years later) recovering from the damage that that has caused to the code base.
Outside the IT industry, most bosses don't want to hear about code. Coding is actually seen as a pretty trivial step in many cases. It's OK to hire an intern to add new feature X. Writing good code is not properly rewarded - as long as it works, it's stable, no-one complains, it's OK.
Not to mention stuff such as niche technologies, that usually create a barrier to entry that would-be employees have to overcome, thus concentrating the know-how, power and pay in the hands of a few bored people that won't bother learning the next thing.
If you would take the first version of most production code put it in a book and write the problem it solves, I'm sure you wouldn't think it sucks. My exp developer 15years, finance industry 7.
We've all seen it, at least those in the audience that have worked at some corporate of any size.. maybe "start-ups" face similar issues, it's difficult to get a consensus there.
The good analogy is with writing a poetry and professional writing in general. There are many examples of technical and scientific books produced by two or free authors (they could list ten, but actual work was done usually by no more than a couple). But we haven't seen any decent book produced by a mediocre writers on 9 to 5 shifts. The idea that this will work for programming is very naive one.
Another analogy is engineering, which is also done by few capable individuals with help of others. There are nothing but failures when mediocre groups trying to engineer anything. On the contrary, we had lots of things invented by capable individuals, things that later were polished by later generations.
So, if one looking for an example of good software look at individuals, or small groups lead by one capable visionary, and avoid anything created in sweat-shops. Most of the really good software we have (Lisp, C, Unix, Plan9, Emacs, vi, nginx, redis, postgres, Informix, etc) was created by an individual effort of men of unusual capabilities, and then polished by community.
Good software engineering is as rare as good poetry, and it cannot be produced by any amount of a manual labor.
There are people who can sell you products and solutions, even ready processes which, they say, will guarantee that you can create good software employing mediocre coders in 9-to-5 shifts. It is all Java or Scrum scams is all about.
It is not tools that create software or invent things, it is minds, the same way no typewriter or fountain pen could make one a poet.
Worse, it goes against his point just a paragraph earlier. The whole advantage of Q/K/J/Haskell style languages is that once skilled in them, the code is much (as in 10-20 times) denser than the equivalent OO-style code, and the entire flow can be grok'ed with ease. There is much less danger of over-engineering in those languages.