His problem was syntax getting broken and not about "questionable data". In the context of dynamic vs static compile, he's worried about scenarios like this:
x = customer.last_name // worked on Friday and x=="Smith"
- remove field customer.last_name // Saturday refactor
+ add field customer.full_name // Saturday refactor
x = customer.last_name // broken on Monday with a runtime error
This essay is a "solution" to a different problem.
What's the proposed fix in this case? Should all clients accessing that field switch to `customer.full_name`? Do you control all of the call-sites? If so, that change is pretty easy to automate in dynamic languages by marking the field as deprecated, adding the new field, and forwarding `customer.last_name` to `customer.full_name`. That way, running systems don't break but you get the new behavior. You can automate logs to track when the deprecated field is accessed and, if necessary, fix call-sites after a reasonable period of monitoring, reducing risk (assuming there's not "once-in-a-century" paths in the code that invoke the call site).
It seems reasonably likely that this is not the desired behavior - you may have external clients, and some callers may be relying on this field to actually return only a last name. In that case, deprecation is still the correct path, I think, since it's unreasonable to expect external clients to immediately fix systems relying on that field being available.
Static typing can tell clients where they need to make changes, but it's not realistic demand that they immediately update every call site, every method depending on that behavior, and every integration with their external dependencies that relied on the interaction between your code and another system. This is why we have (semantic) versioning, deprecation warnings, and support agreements for larger systems. API's should evolve slowly and gracefully over time, not shift suddenly overnight.
My reading of jsode's example is in a shared codebase, where it would be reasonable for the person performing the refactoring to update all call sites, and where checking in a change that does not update all call sites to whatever branch is released to production should be unacceptable, as it will provably cause problems in production.
If the change was made in a standalone library that other teams depend on, the type-checker error would be produced when they update the library to the new refactored version, and it's the same story: prohibit deploying known-invalid code to production until you've updated all call-sites to match the changes in the library API.
If updating all call-sites immediately is implausible for whatever reason, leaving the old field in place with a deprecated annotation would be quite reasonable, but the purpose of a type checker in this context is to prevent you from failing to handle this in at least some way. As you say, responsible maintainership of an API should facilitate gradual transition; the use of a type-checker here is to prevent deploying code that will fail at runtime when someone has made a mistake, and can help facilitate responsible maintainership.
For the purposes of my reply to the author, the type of fix doesn't matter and its discussion would be a distraction away from my main point: the blog author Curtis "Ovid" Poe misintepreted the comment by Samuel Falvo II and therefore wrote the wrong "solution" as a response. Ovid's health data example of obj.fetchDataFromServer() error and reconstructing data from other sources with a threshold score is not the same abstraction level as the commenter's example of obj.last_name getting broken because the plumbing API was renamed/refactored.
(My guess is that Samuel Falvo II's hostile tone makes it easy to miss that he was actually describing a syntax error and not a data interpretation error.)
Regardless of whether the fix is:
- stringsplit(full_name) to extract a last_name
- or change the client code's usage of last_name and only use full_name
- or re-architect the upstream customer object to include both last_name and full_name so there's a gradual migration
...it isn't the issue.
What the angry commenter was trying to communicate was, "tell me about the last_name refactoring causing a syntax error _immediately_ instead of letting it sit there as a ticking time bomb that will blow up and wake me up at 4am".
He doesn't necessarily have to immediately fix it, but he does want to immediately know of the error's existence before a crash at runtime.
In a statically typed world, Ted tries to compile his project and gets an error saying, "last_name doesn't exist". In a dynamically typed world, Ted ships his code into production (or possibly testing) and gets a call at 4:00a.m. saying it threw an exception or just died. (The testers are in Bangladesh.)
In Alan Kay's world (or at least that of the author), the code goes on robustly producing whatever results it produces in the case where no one has a last name. Better?
When you are dealing with external systems, such as a third party API, the message passing paradigm is what you are dealing with. There is no static typing for someone else's API (even if you are using their SDK).
The merits of Alan Kay's message passing are fine but it is not relevant to this case of the author writing an inappropriate solution because he misunderstood a commenter's feedback.
The "this" in your "_this_ is about message passing at an abstract level" comment seems to only refer to blog article #2 in isolation -- instead of considering how it was provoked by the author's previous misunderstanding.
>You are the one missing the point
You want to lecture me on being wrong but it seems that you are the one jumping into the middle of a conversation without realizing what communications have transpired so far. If you didn't completely read both blog articles and the user comment, you're missing the full context. Let's recap:
The blog author wrote 2 articles.
Blog article #1[1] didn't say anything about "external systems 3rd party API". The blog author himself brought up the issue of dynamic vs static in regards to "hot swapping" code without restarting the system. It was not "external systems". It included an example:
print customers.fetch(customer_id).last_name
Blog article #2[4] then responds with an elaborate solution to reconstruct missing data which is a totally different abstraction level from the error scenario the commenter was asking about. This 2nd article introduces an "ETL workflow" that possibly comes from servers of external partners. The 1st article did not.
My top-level comment was explaining to the blog author that he misinterpreted what the angry commenter was asking.
[1] https://ovid.github.io/articles/alan-kay-and-oo-programming....
[2] https://ovid.github.io/articles/alan-kay-and-oo-programming....
[3] screenshot of Disqus comment in case it gets deleted: https://imgur.com/a/D09ma0x
[4] https://ovid.github.io/articles/alan-kay-and-missing-message...
When they get an unrecognized message, Objective C objects call their doesNotRecognizeSelector method, and Smalltalk objects call their doesNotUnderstand method.
And the object sending the message can first check with respondsToSelector in Objective C or respondsTo in Smalltalk, before sending the message.
But validating and sanitizing input parameters is a totally different thing than handling unknown messages, orthogonal to object oriented programming.
Objects and messages are kinds of types. Types can have constraints and conditions and there is all sorts of nuance.
Difference, yes. But completely different? No... when designing a system there is a lot of freedom in where to draw the shapes of the system, how much information do we contain in the objects themselves, a hierarchy or inside parameters.
"Stringy interfaces" is one extreme. There are many others.
http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay...
>One of the things I should have mentioned is that there were two main paths that were catalysed by Simula. The early one (just by accident) was the bio/net non-data-procedure route that I took. The other one, which came a little later as an object of study was abstract data types, and this got much more play.
>If we look at the whole history, we see that the proto-OOP stuff started with ADT, had a little fork towards what I called "objects" -- that led to Smalltalk, etc.,-- but after the little fork, the CS establishment pretty much did ADT and wanted to stick with the data-procedure paradigm. [...]
>(I'm not against types, but I don't know of any type systems that aren't a complete pain, so I still like dynamic typing.)
>OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things. It can be done in Smalltalk and in LISP. There are possibly other systems in which this is possible, but I'm not aware of them.
https://computinged.wordpress.com/2010/09/15/alan-kay-on-mot...
>If you are “setting” values from the outside of an object, you are doing “simulated data structure programming” rather than object oriented programming. One of my original motivations for trying to invent OOP was to eliminate imperative assignment (at least as a global unprotected action). “Real OOP” is much more about “requests”, and the more the requests invoke goals the object knows how to accomplish, the better. “Abstract Data Types” is not OOP!
https://news.ycombinator.com/item?id=10967103
>An interesting historical note is that the two inventors of Simula had completely different views of what they were doing and how it should be used for programming. Dahl was brilliant and conservative, and later wrote papers about using class definitions to make Abstract Data Types (and that is how a lot of so-called OOP programming is done today). Nygaard on the other hand was quite a wonderful wild man and visionary -- beyond brilliant -- and was into the abstract "simulate the meaningful structures" idea. Dahl was trying to fix the past and Nygaard was trying to invent the future.
http://worrydream.com/EarlyHistoryOfSmalltalk/
>"Object-oriented" Style
>This is probably a good place to comment on the difference between what we thought of as OOP-style and the superficial encapsulation called "abstract data types" that was just starting to be investigated in academic circles. Our early "LISP-pair" definition is an example of an abstract data type because it preserves the "field access" and "field rebinding" that is the hallmark of a data structure. Considerable work in the 60s was concerned with generalizing such structures [DSP *]. The "official" computer science world started to regard Simula as a possible vehicle for defining abstract data types (even by one of its inventors [Dahl 1970]), and it formed much of the later backbone of ADA. This led to the ubiquitous stack data-type example in hundreds of papers. To put it mildly, we were quite amazed at this, since to us, what Simula had whispered was something much stronger than simply reimplementing a weak and ad hoc idea. What I got from Simula was that you could now replace bindings and assignment with goals. The last thing you wanted any programmer to do is mess with internal state even if presented figuratively. Instead, the objects should be presented as sites of higher level behaviors more appropriate for use as dynamic components.
https://medium.com/@richardeng/goos-is-looking-at-it-from-th...
>GOOS is looking at it from the perspective of Abstract Data Types. In Alan Kay’s conception of OOP, instead of static structures that are easy to reason, aliasing gives you dynamic systems of collaborating objects that are endlessly flexible and scalable, just like in nature’s biological systems of cells or the Internet of web servers. Proponents of ADT-style thinking, who use languages like C++ and Java, can’t imagine such complex systems, or they’re afraid of them.
http://mythz.servicestack.net/blog/2013/02/27/the-deep-insig...
>Where the big missing piece lacking in mainstream typed OO languages today is:
>The big idea is “messaging”
>He advocates focus should instead be on messaging and the loose-coupling and interactions of modules rather than their internal object composition:
>The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be.
>And finds static type systems too crippling:
>I’m not against types, but I don’t know of any type systems that aren’t a complete pain, so I still like dynamic typing.
>Other popular languages embracing Smalltalk’s message-passing and late-binding and having implemented its message-based doesNotUnderstand construct include: forwardInvocation in Objective-C, method_missing in Ruby and more recently noSuchMethod in Google’s Dart.
https://queue.acm.org/detail.cfm?id=1039523
>Some people are completely religious about type systems and as a mathematician I love the idea of type systems, but nobody has ever come up with one that has enough scope. If you combine Simula and Lisp—Lisp didn’t have data structures, it had instances of objects—you would have a dynamic type system that would give you the range of expression you need.
>It would allow you to think the kinds of thoughts you need to think without worrying about what type something is, because you have a much, much wider range of things. What you’re paying for is some of the checks that can be done at runtime, and, especially in the old days, you paid for it in some efficiencies. Now we get around the efficiency stuff the same way Barton did on the B5000: by just saying, “Screw it, we’re going to execute this important stuff as directly as we possibly can.” We’re not going to worry about whether we can compile it into a von Neumann computer or not, and we will make the microcode do whatever we need to get around these inefficiencies because a lot of the inefficiencies are just putting stuff on obsolete hardware architectures.
https://news.ycombinator.com/item?id=19416424
>"I'm not against types, but I don't know of any type systems that aren't a complete pain, so I still like dynamic typing.)"
>That's probably more controversial here than his views on OO.
A terrific 2014 paper, Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems [1] found that most catastrophic crashes in distributed systems are a result of catching exceptions and not paying thought to how to handle them.
[1]: https://www.usenix.org/system/files/conference/osdi14/osdi14... (talk: https://www.usenix.org/conference/osdi14/technical-sessions/...)
For example, if you're writing an HTTP handler for an API, you should catch any exceptions at that point in your handler and return a 500 with an appropriate response to the client, since returning 500 is something productive we can do (vs letting it pass and crashing the server).
Data importers and sanitizers are de-rigueur when dealing with real world data, of course. You have to expect to find crazy things inside, and be expected to just cope with it. That's not a problem.
But when you're dealing with purely internal systems, the calculus is different. If you had to keep chasing down fopen(), printf(), and malloc() calls, you'd spend more time in administration than you would getting actual work done.
So if I'm understanding this correctly, we're talking about purely internal code vs code that traverses domains (much like in DDD), which require different styles of solutions.
Or am I still missing the point?
I'm primarily talking about external data/services. Anything which you would ordinarily consider "suspect", such as "reading from a third-party API", is the target here.
The odds of printf() failing are so ridiculously low that I am not going to write tons of code to handle this case. It's not worth the money.
The odds of putMoneyInCustomerAccount() failing are much higher, and the consequences are much graver, so that's a perfect candidate for saying "maybe just throwing an exception ain't the best solution here."
In the case of OOP, particularly in Kay's vision, these objects might be written by someone else, using code we don't see, and thanks to isolation, might be connecting to third-party services which do all sorts of interesting things we don't know about or need to care about. Thus, they're not trusted.
If I wrote the object or I can read its code, and I know what it's doing, I'm not going to sweat it.
I'm also not arguing that we should always make the code that robust. Some things you can recover from or your budget might not allow you to make the system more robust.
Oddly enough, I'm currently working on exactly this sort of thing, for financial data. And I've been cursing the guy who wrote the initial pass at it, the state of "software engineering" education, and the Pacific Ocean for good measure. It turns out that Mr. Guy is an expert at modern software engineering best practices, dependency injection, unit testing, and ORMs, but had apparently never written a parser. Being an unpleasant and perverse excuse for a human being at this stage of my career, I decided to mostly keep his parser and wrap it in enough error handling to do something useful is some transaction record doesn't have a supplier. That is, rather than rewriting it as a normal-human-being parser or even (heaven forfend) breaking out and learning to use ANTLR. (Whew. I feel better now.)
Anyway, so, yeah, I've been there. I know what you're talking about.
But...
Samuel Falvo II is commenting on a completely different topic. He's not talking about the problem of making a system robust against crazy-pants input or even crazy-pants programmers (well, indirectly...). He's not talking about
"So when fetchDataFromServer() fails, or when an object doesn't respond to the message you sent, what do you? There's a simple way to approach this: ask yourself what you would do in real life."
He's talking about "when an object doesn't respond to the message you sent," how do you know? Don't have a static type system of some sort? If you type "entrie_name" when the message handler is called "entire_name", you won't know until some process tries executing that message send. And if you don't have something like a exception system, you won't even know then. (SIGSEGV, SIGILL, or SIGBUS and the like are really indications that you have just made some grievous error in how you are living your life.)
Falvo, with all his vitriol, is pointing out that the systems of Kay's vision are actively trying to prevent you from writing robust systems. And you have done exactly what he's accused you of doing:
"Don't resort to changing the topic either [...]; the issue is, right then when the error happened... What happens right then and there?
"Until you can answer that question, you have done nothing but bloviate about what is possible in some utopian system...."
It seems pretty important to be clear about static assumptions you can trust (because it's compiled in) versus things that can change.
I had all kinds of issues - variations of names, mispelled names, nicknames, names with the middle name used as a surname (and vice versa) and a few lacking even that. I recall leaning heavily on tables of common names, various Python string normalisation methods as well as soundex. In the end I was left with a dozen or so names which needed follow up but it was pretty good for the ~1000 I had started with initially.
The most harrowing part of it all was attending the crew screening - it is actually possible to address issues on an end crawl (provided you catch it early enough) but I really didn't want to be the guy who screwed up the credit of someone who had just spent the previous three months in crunch to get the picture out the door.
He is advocating Queues and one of his arguments is that Queues decoupling the requester from the receiver. So you don't really have to worry about things like this by using a queue.
thingy.spamford()
"Kay argues (correctly, IMHO), that the computer revolution hasn't happened yet because while our bodies are massively scalable meat computers, our silicon computers generally don't scale in the slightest. This isn't just because silicon is slow; it's because of things like print customers.fetch(customer_id).last_name not actually having a last_name method, throwing an exception (assuming it compiled in the first place) and programmers getting frantic late-night calls to bring the batch system back up. The only real upside is that it offers job security."
This has nothing to do with network protocols.
(Not knocking the author...the practice is unfortunately needed because providers won't provide good data)
It wasn't quite like that. It was to help the pharmaceuticals work together to "pool" talented researchers who had the required training to proceed in the clinical trials. If a bunch of companies don't share this data, they drive up their costs tremendously in trying to find and recruit qualified doctors for their trials, pay to train them, and then have them conduct studies, only to find out that the doctor in question doesn't recruit any candidates themselves, or fails to report results.
If, however, they pool their resources, they can find qualified doctors who have already taken the training and are known to participate well in trials. They can then correlate that with populations who might benefit from the drugs in question and save a fortune (and possibly many lives), by reducing the cost-to-market.
The matching algorithm was actually very strong, but there was still a human step, from the pharmaceutical companies, to verify that these were the researchers they were seeking.
What about private methods? Decently written objects in the wild tend to have some private methods that they call on themselves. The highfalutin metaphor of a message goes out the window there, because it's an individual object doing something to itself, not a communication.
Whether some actual bonafide communication ends up being done one level removed (e.g. inside such a private method, or chained off of a field access) is immaterial because when appraising the conceptual integrity we cannot continue after seeing a fail.
(edited)
If your agent that send a "must respond" message doesn't get a response within some customizable time, then it itself fails loudly.
Cross system require much flexibility and to allow for error.
http://gigamonkeys.com/book/beyond-exception-handling-condit...
In the context of the example, I can easily imagine how the system might have processed most rows without issue, called out to a scoring system for most exceptional rows, and raised a few rows to human attention when even the scoring system couldn't figure it out.
The "if you don't know the answer, find an approximation from another route" is .. situational. In this case it's exactly what the customer wants. In other cases (finance, security, aerospace) it could be a disaster waiting to happen.
I worked on a point-of-sale system where it was a matter of design philosophy that any inconsistency should be an immediate crash-and-reboot; since it also operated as a save-everywhere and audit-everywhere system, if you did manage to crash it then within 30 seconds you'd be back at the screen just before the last button press with no data loss other than the button press that caused the crash. I believe this crash-and-recover approach is very Erlang: https://ninenines.eu/articles/dont-let-it-crash/
Thinking of exceptions and message validation also makes me think of "in band" versus "out of band" signalling and errors. Exceptions are "out of band" - outside the normal function return process. Internet communication is all "in band" and the whole approach of separate control and data planes has almost entirely gone away, apart from SIP and (nearly dead) FTP.
IMO this counts as a kind of out-of-band arrangement because it doesn't involve using one of the normal return values to signal error.