Ask HN: What’s the biggest difference between professional coding and academia?

Academic code typically just has to work once or a handful of times, for a small number of highly expert users, frequently just for the author. Ease of update is of the essence - you'll rewrite most of it many times, as your understanding the problem change. You can use all sorts of ugly hacks so long as you get what you're after.

If any of it ever becomes commercially released or whatever, there'll need to be a complete rewrite that makes it usable and maintainable by people other than yourself. But most of the code will never get to that point because most of what you've done up until about a week ago is wrong and worthless, and the current, correct-until-next-week iteration is stuck together with duct tape.

Speed only matters on the infrequent hot paths, which is why Python is popular. The rule of thumb is nobody cares about speed / resource consumption until it needs to run on a cluster, but then you care a lot because cluster time is metered and simulations can get huge. Fortran is still fairly popular because many math libraries are on it and porting would require huge effort from a very small group of very busy people.

Most of the coders are not software engineers and don't know / don't follow best practices; on the other hand the popular best practices are not designed for their use-case and frequently don't fit. Versioning (of the I-don't-know-which-of-the-fifty-copies-on-my-laptop-is-the-right-one type) is a big issue. Data loss happens. Git/Github/etc has steep learning curve, but so does all the various workflow systems designed for research use.

In academic systems papers, every performance claim needs to be backed up by an experiment. But you can get credit for features even if you argue that it is possible to implement that feature with your design, even if you didn't actually do it.

In production software, this is flipped. Every feature claim needs to have an associated test, as it's a contract with your user. But when it comes to performance, everyone just waves their hands.

I'm being a little glib. But production software has to work. You'll spend far more time dealing with all of the "less interesting" details and edge cases than with research software. As ams6110 points out, this means more focus on testing, maintenance and good design. But I do want to emphasize testing - sometimes you'll spend more time testing something than actually implementing it. There's also often many more residual effects from dependencies elsewhere in the ecosystem you're working in. That's not typical in academic software.

Maintainability. A lot of academic code only has to last long enough for one project or thesis, and the only maintainer will be the original author. Real-world code will last longer[1], and be worked on by more people, including people of lesser skill without the original author around to guide them. Often, that code also has to run in more environments. This difference is reflected not only in the code itself, but even more importantly in the infrastructure around it - source control, tests, documentation, bug trackers, etc.

Ironically, an academic might get to spend a higher percentage of their time on pure coding than a professional coder does. They have other concerns. Maintainable code is not part of the desired outcome. It's consumable and expendable, not durable, so any time spent making it any better than "just barely good enough" is wasted. Why build a tank when all you need is a bicycle?

[1] At least the expectation. Some academic code lives on far longer than its authors intended, and some non-academic code vanishes pretty darn quickly. But in general, both the intent and the expectation is that non-academic code will live longer.

"Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth.

This is the biggest difference between academic and professional programming in a single pithy statement, from a paper that Knuth wrote.

Academia only worries about getting results for publishing. Testability, maintainability, clean design, all take a back seat because once the paper is done the author will likely never touch the code again.

A number of Linux kernel developers have been working with a subset of the Usenix FAST (File systems and Storage Technologies) community. We hold a Linux FAST workshop after the FAST conference for the past few years.

A few years back, some of the researchers (professors and graduate students) claimed they were interested in more testing and possibly taking some of their work (Betrfs[1], specifically), and productionalizing it. In response, I spent a lot of time with kvm-xfstests[2] and gce-xfstests[3][4] testing infrastructure, cleaning them, making them work in a turn-key fashion, and providing lots of documentation.

[1] http://betrfs.org

[2] https://github.com/tytso/xfstests-bld/blob/master/Documentat...

[3] https://github.com/tytso/xfstests-bld/blob/master/Documentat...

[4] https://thunk.org/gce-xfstests

Not a single researcher has used this code, despite the fact that I made it so easy that even a professor could use it. :-)

The problem is that trying to test and productionalize research code takes time away from the primary output of Academia, which is (a) graduating Ph.D. students, and (b) writing more papers, lest the professors perish. (Especially for those professors who have not yet received tenure.) So while academics might claim that they are interested in testing and trying to get their fruits of the research into production code, the reality is that the Darwinian nature of life in academia very much militates against this aspiration becoming a reality.

It turns out that writing a new file system really isn't that hard. It's taking the file system, testing it, finding all of the edge cases, optimizing it, making it scale to 32+ CPU's, and other such tasks to turn it into a production-ready system which takes a long time. If you take a look at how long it's taken for btrfs to become stable it's a good example of that fact. Sun worked on ZFS for seven years before they started talking about it externally, and then it was probably another 3 years before system administrators started trusting it with their production systems.

Academics aren't paid to code. Academics are paid to do research.

Professional coders are paid to code.

JavaScript and the web generally are a really big deal. Unless you are working close to the metal at some point you are probably going to have to write code that somehow works in a browser. The problem with this is that it doesn't resemble C++ or Java and very few people figure out how these technologies actually work.

Academia isn't preparing developers for this reality. Many will try to fake it or hide under imposter syndrome, which is fine if everybody in the company is an imposter, otherwise it is plainly obvious you are incompetent.

I suppose it depends on why the academic is writing code. I've written simulations for social science research -- basically to extrapolate the results of certain decision making strategies by an idealized decision maker in a toy problem. The theory is the subject, not the code. Many people who read my paper will not be programmers able to critique the code, and few will care whether it follows best practices. I've made it open source (https://github.com/joeclark-phd/bandito) because I think it's a good practice to promote, but I'm kind of an oddball. When I was studying other well-known simulation papers, I found their models were incompletely described and I had to contact the original authors to get implementation details in order to replicate them. This is a lot like the problem of data sharing -- all academics should be willing to share their data in order to prove that their results are as reported, but it's not an easily enforceable principle.

If you are talking about computer science academics, of course, that's a horse of a different color. In that case, the code is the topic, so I would guess that they're providing it! On the other hand, the majority of such research is probably solving niche problems and special cases, so it may not be very usable in your professional coding.

In academia you get to choose your problem. This means that if something isn't working you can restrict the inputs or only operate on some subspace of the problem.

In contrast, industry doesn't let you choose the problem: you need to solve whatever the problem is that the client has. This means generalising a lot further and having a less optimal solution that is more robust to input error or poorly calibrated measurements. Even if it does fail you should be able to identify why and explain to the user what they did wrong.

In academia this feedback process is generally to the person who wrote the software, so a cryptic error message including some algorithmic details might be sufficient to debug the inputs sufficiently.

In one sentence: Scientists build so they can study. Engineers study so they can build.

My primary observation so far has been that academics are going to want to reach into the guts of my data analysis pipeline at every single step and inspect what's going on.

This informs my design choices quite a bit.

In academia you don't care about the quality of your code, in professional you do care but don't get the time to fix it ;).

On a more serious note. In addition to what is already mentioned by others on quality, performance and so on I'd like to add that in professional career you most likely work with a (larger) team. Which means you will run into code conflicts where code is reused for different purposes and you cannot simply change it. In addition you have to think about readability and documentation as your colleagues have to be able to understand the code without losing too much time or needing you.

You will also always have to work with legacy code. Most likely code you want to change but can't considering the timelines.

You will have to sync your design with many others. You might have to convince them or discuss issues with conflicting requirements or deadlines. There will be times you can't finish your entire design and have to think of a staged introduction or even harder, change it so it can work with only 50% of the design.

Also, your code has to run for many years. You can't simply take an expirimental third party package maintained by a single person. Too risky. You have to think about hardware expiring or no longer being supported (especially with gpus).

You gave to think about licenses. Academia is usually free. With professional you have to take a close look.

Operations. Forward and backward compatibility concerns. In professional coding, good enough beats cute implementations that no one will see.

The consensus here is that "professional" code is more maintainable than "academic". That's probably the ideal, but not entirely sure it holds up in practice. In particular, approaches which put a lot of emphasis on clarity and "testability" of individual functions/"units"/whatever can make it harder to understand and reason about what the program as a whole is doing.

Also, the focus on building software in teams seems to lead to architectures that need teams (vs. suites of manageable-size, "do one thing well" tools).

Slightly different take on this: http://yosefk.com/blog/why-bad-scientific-code-beats-code-fo...

What does "academia" mean to you? Are you an undergraduate student, a grad student, a postdoc, a research scientist, a professor (and if so, which level)? Relationships to software creation differ at each level.

See: Why do many talented scientists write horrible software? - Academia Stack Exchange https://academia.stackexchange.com/questions/17781/why-do-ma...

Wrote this recently, kind of an answer to this question (if you don't mind another disgruntled JavaScript rant): https://guscost.com/2017/06/19/future-driven-development/

Academic is often theory

Professional is often whatever works

This is fairly common with many academic vs professional differences, btw

In most cases, not enough testing

For what it is worth, LLVM was birthed in academia.

the biggest difference, in academia you code on sunday night and actually, a very small number of people read your code. But in industry, you write a code and then it goes through a pipeline of multiple reviews, so you end up spending most of your time addressing all those comments, and also you do the same for the codes written by other people.

Obligatory xkcd: https://xkcd.com/664/

The super structure of a Software project, size and density of crew.

The number of // @TODO's in the code.

Testing and review.

shipping

Previous HN discussion: "Why can't you guys comment your fucking code"

https://news.ycombinator.com/item?id=14692691

Copy&pasting my response there:

---

Why is code coming out of research labs/universities so bad?

1. DON'T SEE WHY CLEAR CODE MATTERS

Academic projects are typically one-offs, not grounded in a wider context or value chain. Even if the researcher would like to build something long-term useful and robust, they don't have the requisite domain knowledge to go that deep. The problems are more isolated, there's little feedback from other people using your output.

2. DON'T WANT TO WRITE CLEAR CODE

Different incentives between academic research (publications count, citation count...) and industry (code maintainability, modularity, robustness, handling corner cases, performance...). Sometimes direct opposites (fear of being scooped if research too clear and accessible).

3. DON'T KNOW HOW TO WRITE CLEAR CODE

Lack of programming experience. Choosing the right abstraction boundaries and expressing them clearly and succinctly in code is HARD. Code invariants, dependencies, comments, naming things properly...

But it's a skill like any other. Many professional researchers never participated in an industrial project, so they don't know the tools, how to share or collaborate (git, SSH, code dissemination...), so they haven't built that muscle.

The GOOD NEWS is, contrary to popular opinion, it doesn't cost any more time to write good code than bad code (even for a one-off code base). It's just a matter of discipline and experience, and choosing your battles.