Comments should explain things that are not obvious from the code. Comments should be things like
// Note: PCI-DSS requirements apply below
// Must check status register and FIFO to determine completion due to flaky hardware
// Algorithm below is modified Knuth-Morris-Pratt
// This is O(scary), but seems quick enough in practice.
(Now, if someone lets me put images in comments, that would be amazing. Good for state transition diagrams and random scribbles)
In his case, his algo said X, and his code did Y.. very easy to see the mistake.
In your case, lets add a comment
// Note: PCI-DSS requirements apply below
Now 3 years later, the law changes and the requirements do not apply. So you are at the same situation. Code does X, comments say Y. Which is right?
Not a very easy to keep comments and code in sync
But at any rate, why is relevant, and code doesn't express the why.
Comments can be useful for annotating algorithms or referencing stack overflow though.
Requirements are usually exogenous to the code.
1. Specs&design docs and code should be in separate files, because I believe the separation of concerns should be applied there. That's indeed the opposite of literate programming.
2. There should be two-way links between documentation and code: in the code, one should have links to the spec; and from the spec, one should have links to the code.
3. If the specs or the code changes, those links should be displayed in a different way to warn the reader that things are potentially not in sync. How to do that: check if the links point to the latest version. The maintainers have to update the links to remove the warnings.
Specification and design/implementation are not separate concerns. They are dual.
A sufficiently detailed specification is an implementation. Prolog does this (and Eve has a very similar feel).
As engineers, we traditionally work declaratively at the top of the "V" and imperatively at the bottom of the "V" -- but the reasons for this are largely historical/cultural.
We could (in theory) carry out the analysis/refinement process using entirely declarative notation.
The problem domain has primacy. Analysis separates problem-domain concerns and the duality takes care of the translation between problem and solution domains.
(OK -- so this is basically just a reiteration of the thesis of good old-fashioned AI -- that with a sufficiently powerful theorem prover and a sufficiently large and detailed knowledge base -- solutions will just pop out of a largely mechanical analysis process -- and I'm pretty sure this isn't at all trendy right now ... so I should relegate this to the "thinking out loud" bucket ...)
But there's ways to mix the two: Python doctests is one. The lp approach is to "escape" the code, not the comments. I really do think some richer data structure than flat text files is needed - simple multi-/hyper-text seems like a minimum. Not only links (most ides have this: hoover to display help, click to goto definition etc).
Why so few seem to successfully add vector graphics and diagrams I don't know. I guess jupyter / ipython might be a rare breath of air. But who really wants to maintain 10k locs of code and 15k locs of tests in ipython notebooks?
We need something more (lively kernel might be more interesting here - webdav for persistence Web browser for run/edit/view - I'm not sure if the required power Canberra realised any simpler).
The best example I'm aware of is probably Smalltalk coupled with monticello version control and a solid object database like gemstone/s (I'm not aware of a real open/free workalike for the last part).
That's not not say relational or logic databases aren't useful - but it's more difficult to manage the data, query code and program code in one integrated system if the database takes the form of an external server. That said, Ms visual studio and the SQL db gui does a pretty good job of presenting a somewhat coherent environment.
I used to be strongly in the Linux/Unix camp - thinking that the datastore we call the filsystem is a good abstraction. But I've come to believe that even the strong legacy of user familiarity isn't a good enough reason to stick with it. Even if we were a little bit more serious and at least went all plan9 - rather than the half-hearted state of Linux/Unix (everything is a file, or a database file, or a binary file or an archive of a filsystem or...).
That said a "filsystem" might very well still be a decent building block for a higher level system (with decent search, for example).
But yeah, I don't think jupyter is the be all, end all of development. But it is an interesting way to move legacy programming environments forward in a low friction manner.
Sometimes a piece of code needs to be highly optimized to the point of being basically unreadable, at that point it's probably worth adding explanatory comments that you would normally avoid.
The comments should be the things that you would tell a new team member during a pair programming session. You assume she knows what the syntax of the language is, but you don't assume she knows the business intent behind the code or the history of trial and error that led to its current state.
That's exactly what JetBrains MPS (https://www.jetbrains.com/mps/) does. Even better: the state transition diagram is the code. Some of my colleagues are playing with it - not sure if it's used in production yet.
