When I started learning programming from books, the internet and later formal study this wasn't stressed nearly enough. A program transforms data. So much else starts falling into place about all the various abstraction techniques useful to perform that kind of transformation (procedural, oo, functional etc) and technologies used to assist, (testing, debuggers, etc) once you keep that thought front and central in your mind. At least I feel it really did it for me.
Reading code is the same. Where are the places the data coming in? Where is it going out? Sometimes this is a better place to start than "main"
I'm sure that's really obvious to many but I'm not sure it is to the author in a way that is being successfully communicated:
”programming is a tool to solve problems that you have in the domain of computers”
Taking a baseball bat to a computer, on some level can meet the suggested definition of programming, but I don't like it that it does. I also don't much care for the use of "problems" there either.
The above is especially opinionated because I'm particularly interested if people disagree!
edit: I'm definitely still learning programming here.
I completely agree that the essence of a "program" is transforming data. In my previous article which I linked to in the main article, I specifically state "The purpose of a program is, and ought to be, something that transforms data into other forms of data".
But the essence of "programming" is different, and I stand by statement that "Programming is a tool to solve problems that you have in the domain of computers". And this article was an extension of the previous article meaning that taking it out of context has caused this confusion. But the term "programming" embeds the concept of "a program" already: programming produces a program. Programming does not start with hitting the computer with a baseball bat.
1. Data. It can be in many different forms and structures, more or less strictly typed, and a big part of writing a program is determining what it looks like, and what it should look like.
2. Procedures to work on that data. They can transform the data, remove it, reshape it, convert it, or do any number of other operations on it, simple or complex.
That's all there is to programming.
Software development, in my understanding, is about designing and implementing tools to solve particular (classes of) problems. Exactly what is being described in the article.
I think Programming in general is about writing instructions that (maybe not only) a computer can execute to produce (the desired) effects. These instructions may or may not involve input data and may produce side-effects along the way and may not even produce output data (although one could say that “nothing” itself is data similar to how we use 0 as a number).
I don’t think that we need a purpose for writing programs. We can program just about any nonsense and we don’t even have to ever execute it. I think “problem solving” is already too specific.
Even though it makes a lot of sense to think about programs as mathematical functions that transform data, underneath our abstractions are building instructions for the computer to execute and they tend to produce the desired output if we are doing it right and there are no interfering environmental circumstances.
A simple substitution of input data with output data would be the most primitive example, although it would be trivial. To produce non-trivial outputs, we need to replace the substitution with a (more complex) algorithm. Algorithms are tools that solve particular (classes of) problems. So here we are, doing software development, because we actually need a tool.
Yes, information technology is for processing information.
Any transformation can be decomposed into lossy and lossless components (aka bijectve and non-bijective; reversible and non-reversible). The nice thing is that the lossless component can be automatically checked for that property.
> This is one of the few programming quotes that is not just abstract crap, but one thing you can use to improve your programming skills 10x IMHO.
> Something like 10 years ago I was lucky enough that a guy told me and explained me this stuff, that I was starting to understand myself btw, and programming suddenly changed for me. If you get the data structures right, the first effect is that the code becomes much simpler to write. It's not hard that you drop 50% of the whole code needed just because you created better data structures (here better means: more suitable to describe and operate on the problem). Or something that looked super-hard to do suddenly starts to be trivial because the representation is the right one.
Output can also be an action performed through an actuator (robotic arm, welder, coffee pot, ...). Could be considered as data (signal coming to the actuator) but the "problem solved" is the action itself (and usually involves a feedback loop).
I don’t agree here - in my experience data-oriented programming is only a subset of all useful programs.
For example, a game is in my opinion not like that. Sure, very pedantically it may be considered a “mathematical function”, but then so is every human interaction, and that’s not a useful pedantism.
Approaching the issue of solving problems with incomplete information, they end up with circular conclusion, like the essence of programming being operating the computer.
At some point some realize that programming is modeling a domain and it's transformations into a format that is computable and then engineering starts to happen.
"Programming is not a craft." It is a craft and ought to be treated as such. Not doing so leads to the mess in software we have today. Software that is no more complicated than software from 20 years ago but runs 100x-1000x slower than its older counter part. This is the direct result of not understanding what art and craft of programming is and not treating it as such.
I highly recommend reading the previous article to this one posted (Pragmatism in Programming Proverbs) to get a better understanding of what I am expressing, since "The Essence of Programming" is a sequel to that article.
Counterpoint: “the mess in software that we have today” (we at least agree on this) is precisely generated by the fact that people treat programming as a “craft”, where everytime a (apparently) new problem arises, people feel the need to “craft” together some new framework or way of doing things, only looking at the small problem at hand and without considering the bigger picture.
In other words, people treat programming as an artisanal activity instead of what it should be: an engineering discipline and possibly still a scientific pursuit.
Let’s say this: Programming could be approached as a craft by the people who apply it to real world problems. BUT, there is a very different job, which is the one of the Computer Scientist (the name has a meaning), which should instead treat programming as a science. This is currently not happening, or not fast enough, exactly because so-called computer scientists treat programming as a craft, without truly exploring the fundamentals.
I have expressed this view several times in past comments here on HN, but we are still far from having a proper, theoretically grounded framework to do coding, and crafting more short-sighted solutions to pile up on top of each other won’t help…
"Programming is a tool to solve problems that you have in the domain of computers"
Again, this is wrong. Computers are tool with well defined set of operational constraints. You use such tool to model problem _outside_ of the domain of computers, and while it's true that programmer primary effort is in understanding the model and writing transformations that produce useful result, that process is a craft only if the practicioneer is an artisan.
Writing a computer program serves two separate purposes: to communicate a set of functions to the machine, but also to communicate the purpose of these functions to a human. The former is for execution and the latter is for code review and, later, debugging.
Any old clown can get a computer to do something approximating a solution to the problem at hand. If it has bugs then the author — or more likely some other poor sap down the line — will be on the hook to first understand the intent, and then make the code match the intent.
In a professional shop you will have code review where this “debugging” step happens first. Your reviewer will read your code to infer your intent and upon agreeing with that, check your style and composition for inconsistencies or lack of clarity.
To that extent, a large codebase is akin to a mathematical proof where each new module and function represents a lemma on which the final proof depends. No serious scientist would present their thesis without a progression of isolated and carefully laid out lemmas and corollaries, and no serious programmer throws up a thousand line function with 15 arguments, global state, and — if in a dynamic language like Python — inconsistent return types.
Your code is a proof to your reviewer that you are on the right track. The fact that it happens to execute without bugs and produces the correct output is far less important than the code being readable, comprehensible, and consistent as part of the larger system.
It's more important that a function looks good, than that it actually works?
I'm sorry, but that sounds stupid. A computer is a functioning machine first.
If it worked from the start, it wouldn't need to be fixed later.
You can always study someone's work to figure out how it works. There's no point studying a system that doesn't.
True. But you ignore the fact that NO SOFTWARE IS EVER DONE. Software always has bugs, and even if it didn't, it will bitrot as the business needs change.
In theory, it's better to have 100% working software. In practice, that never happens (or only happens for a few weeks at best). Eventually the software needs to be changed. In that case, software that is "written for humans" will always be easier to change than "software that used to work, but now we need to change it, but nobody understands it".
So yeah: code readability becomes the deciding factor for both what to accept: and what will be (most importantly) the most efficient use of development hours on fixing…
Or a better way of putting it… We are monkeys that bribed rocks and copper to think for us with electricity: design code and systems that even a monkey could maintain.
It is not stupid - just counter-intuitive, which should give us a pause to think. You say, "A computer is a functioning machine first" - yes - but consider why SICP says, "Programs are meant to be read by humans and only incidentally for computers to execute." (see a discussion on the quote https://news.ycombinator.com/item?id=16431701).
Once the software is done, sure, throw away the source code entirely (if you dare).
I highly recommend reading the previous article to this one posted (Pragmatism in Programming Proverbs) to get a better understanding of what I am expressing, since "The Essence of Programming" is a sequel to that article. I originally wanted to write the previous article in normal prose article, regarding the topic of "Pragmatism in Programming", however I thought I’d experiment in style by writing in a proverbial style.
> The fact that it happens to execute without bugs and produces the correct output is far less important than the code being readable, comprehensible, and consistent as part of the larger system.
I partially disagree with this. If the code executes without bugs (very rare) and produces the correct output, then that code solves the problem. The next task is to make it readable, comprehensible, and easy to maintain whilst not trying to introduce other bugs.
I recommend reading Casey Muratori's article on "Semantic Compression" https://caseymuratori.com/blog_0015
This article is a good overview of the idea I am trying to express in this reply. It's very important to get things working BUT then actually improve it, don't just stop there!
a large codebase is akin to a mathematical proof iff you are using something like an lcf/λc environment; the curry-howard isomorphism does not yield meaningful programmer-level insights about the average monad transformer lens abomination implementing a rest api endpoint
mathematicians do not have to worry about dependency hell in their proofs, or whether they should be importing wiles's proof of fermat's last theorem at version 3.8.21-rc2 or -rc3
if i write a window manager, its source does not primarily function as some indication that i am on the right track to competently wrangling the x11 protocol, unless i am trying to impress a headhunter from red hat. it is, in fact, primarily meant to produce a program that competently wrangles the x11 protocol. making the code clean and sensible is akin to the courtesy demonstrated by not rendering the blueprints to an air conditioner in chicken scratch with a crayon, and ensuring that part sheets are easy to follow to the assembly
framing code as in the service of the reviewer or debugger takes what is a self-evident prerequisite for getting anywhere, and promotes it to a counterproductive mind game
“oh i'll break this function out into multiple that are called exactly once, even though all that does is make the reader scroll a bunch to follow the logic, because i know jeff is gonna get this one, and jeff is a cargo cultist who thinks robert martin is the second coming of christ”
> Any old clown can get a computer to do something approximating a solution to the problem at hand.
in my experience, far more clowns are capable foremost of endless bikeshedding -- they adore giving off the impression that they are contributing something to the effort, and in the absence of insight, they substitute a performance. “hah, bet you're glad you had me to tell you to write every item in this array on its own line” (the items in question are numbers averaging two digits)
yes, code is for human beings to read, but experience has made me wary of anyone who shows up to passionately insist to that effect
You may get the impression from reading some online Q&A sites that everyone wants "the best" solution because of the way a lot of questions get asked. But in fact, phrases like "a good way" or "the best way" are more often than not purely rhetorical devices whose purpose is to appear competent.
The person is trying to ask the question in a way which suggests that he or she does know how to solve the problem in ways that are not so great, and is just looking for a better way. In fact, the reason they are asking is that they don't know even a bad way to solve the problem. They may have a partial solution, which has hit some roadblock or whatever.
People in this type of field sometimes have a hard time admitting they don't know how to do something. Particularly if they are not such beginners and have a track record of solving problems on their own, which has become part of their self-image.
I try an approach, but I run into X. I try a different approach, but I can't make it work because of Y. I try a third approach, but Z. So I ask around about a good approach - one that won't run into these issues. When I do, I'm really asking for an approach that avoids X, Y, Z, W, and however many more are out there. I'm asking for something close to a best practice.
If you buy that approaches that are free from major pitfalls are rare, then I think you and I are in agreement.
All I ever get from beginners are questions about the "best tool" by which they usually mean "a set of rules and procedures so that I don't have to think about the problem".
> The person is trying to ask the question in a way which suggests that he or she does know how to solve the problem in ways that are not so great, and is just looking for a better way.
It's pretty much always the exact opposite since they rarely understand the problem that they are trying to solve to begin with. And that's the issue! They want a "tool" to just do it for them, even if that tool is not even appropriate for the task at hand. And the questions asked are usually in the form of an XY problem: "how can I use X to do Y?". You might be able to bodge the tool X until problem Y is solved but that doesn't even mean it is appropriate to begin with.
For example: I get a lot of questions at the moment about Entity Component Systems (ECS) and asking what is the best way to make one (or which to use) because they want to make a game. But their games don't even require one in the first place since they have such a low number of entities and components that an ECS would probably be detrimental to them in the first place (performance and productively). But to clarify, if the individual wants to just learn about the topic rather than apply it, then I usually recommend them to learn about relational databases first and how they are implemented.
Many people want to use a tool because they've heard someone else (usually a semi-famous programmer or a big company) using it and then following it in a "flash fad" rather than asking whether or not it is even appropriate to use. I will state that this perfectly _rational_ to do by the way since you are trying to outsource wisdom to others that appear to have it. However, programming as a profession is a best 70 years old, and as a result there has been not been enough time for evolutionary selection pressure to reveal the wisdom, so many people (especially beginners0 are effectively walking around following whatever the latest Fad is hoping for wisdom and a set of rules to follow.
> People in this type of field sometimes have a hard time admitting they don't know how to do something. Particularly if they are not such beginners and have a track record of solving problems on their own, which has become part of their self-image.
Humility is a virtue but also a very difficult one possess. For beginners reading: don't be afraid to make mistakes and look foolish. It is a heck of a lot better to look ignorant but honest than to look ignorant but arrogant.
I highly recommend reading the previous article to this one posted (Pragmatism in Programming Proverbs) to get a better understanding of what I am expressing, since "The Essence of Programming" is a sequel to that article.
My usual approach to an XY problem is to treat it as an AZ problem, asking myself if there are any Ms for which I think I might be able to solve AM or MZ ... but it just occurred to me that this process may not be as intuitively obvious for anyone who has grown up with GPS navigation!
Also, about the 'best way' - given the 'right' set of constrains, there is the best way to do almost anything. Unfortunately, most of the times, there is almost no constrains.
To clarify what I mean (which I also wrote here: https://news.ycombinator.com/item?id=32507580)
All I ever get from beginners are questions about the "best tool" by which they usually mean "a set of rules and procedures so that I don't have to think about the problem". It's rarely actually finding the "best way" to solve the problem but the "best way" to not have to understand the problem at hand.
Beginners want the shortcuts to solving the problem whilst at the exact same time not even knowing nor understanding what their problem is in the first place.
For more clarification, I highly recommend reading the previous article to this one posted (Pragmatism in Programming Proverbs) to get a better understanding of what I am expressing, since "The Essence of Programming" is a sequel to that article.
Since I am developer I was always starting from position where I need "reusable and flexible" script. This approach got me stranded in doing stuff like updating TLS certificates by clicking in IIS settings - because I did not want even to start if I did not have "perfect script".
At some point I went "OK I am just going to do script for this single server" - well script is reusable because it is simple and for another server I just change variables inside of the script. It saves me a lot of time already.
Lots of business line software does not need flexibility but developers tend to have need to build a framework because somehow they feel like they are not real developers.
Of course nothing is lost, and we are all richer for the adoption of new ways of seeing and thinking.
The experience of the '90s and the then insistence on primacy of "paradigms" should stand as a red warning to us all. Programming is nothing if not a creative endeavor, with as much scope for original thought and insight as any other. If you are not learning new "essences" on a regular basis, you are wasting your life.
The essence of programming is data.
I quote Rob Pike along with many others in the previous article I wrote (which is linked in the article): "Pragmatism in Programming Proverbs" https://www.gingerbill.org/article/2020/05/31/progamming-pra...
And to be clear, I am making a distinction between the essence of "programming" and the essence of a "program". I agree that with Rob Pike that (and to quote myself) "The purpose of a program is, and ought to be, something that transforms data into other forms of data".
