Automatic parallelization is very possible. The problem is tends to be less efficient. A decent developer can often do a better job than the compiler by performing manual code restructuring. The compiler cannot always determine which changes are safe without pragmas to guide it. With that said, our top compiler devs did some amazing work adding automatic parallelization to some awful code.
We inevitably sold our supercomputers because we had application experts who would manually restructure the most mission-critical code to fit cache lines and fill the vectors. Most other problems would perform quite adequately with the automatically-generated code.
What this article lacks is a description of why Erlang is more uniquely suited to writing parallel code than all the other natively parallel languages like Unified Parallel C, Fortran2008, Chapel, Golang, etc. There are so many choices and many have been around for a long, long time.
The people who really care about performance are using those. The ones who don't are honestly mostly still writing code that has large constant factors of _sequential_ performance available as low-hanging fruit. Sure, they'd take free performance, but the rewrite/porting/debug costs (even in automatic parallel compilers for the same language) are at least as high as just firing up a profiler.
I'm increasingly of the opinion that if you can't win a top spot in the supercomputer performance fight, you have to have a unique application domain. Erlang's seems to be reliability. I suspect that a parallel variant of javascript that runs in every browser will end up being the next compelling parallel language, as opposed to all of us who are either inventing or attempting to resurrect languages that target x86 or GPUs.
Debugging a non-concurrent program can be difficult, now throw in threads, shared memory, pointers and it quickly becomes a nightmare. The system could be fast, but if it crashes every week, is it useful. Often the answer is yes. But in some cases the answer it no.
There is no free lunch. Shared-nothing architecture doesn't come for free. You pay a toll in _sequential_ performance. It might or not matter to you.
> Automatic parallelization is very possible.
For numerical algorithms many and for small function scope scale I can see that. Numerical code. But the problem is (and what Joe was pointing out) is that applications and algorithm design has to be build concurrently to start with.
A compile will not re-factor your code to not access a single database and acquire a lock from 100k clients into using some eventually consistent or event-sourcing data store. It is something that has to be built from ground up.
Same thing with fault tolerance, it has to built in from ground up. Adding it later is not easy.
Every time someone conflates parallelism with concurrency...everyone gets very confused.
We can do some of this now in most languages with hot-spot profiling, basic block analysis, selective inlining, and other innovations. However, you really can't beat low-level languages that explicitly "hint" at their execution paths.
By the same token, Cray's applications were...so...slow if you were foolish enough to run them on the expensive hardware and not the FEPs.
I completely agree with your last sentence. For those of us who have dived in a way, the advantages become clear, but TFA was really just preaching to the choir.
The first part of this statement is plain wrong. Single thread performance has improved a lot due to better CPU architecture. Look at http://www.cpubenchmark.net/singleThread.html and compare CPUs with the same clock rate, where a 2.5 GHz. An April 2012 Intel Core i7-3770T scores 1971 points while a July 2008 Intel Core2 Duo T9400 scores 1005 points. This is almost double the score in less than four years. Of course, one factor is the larger cache that the quad core has, but this refutes Armstrong's point that the multicore age is bad for single thread performance even more.
For exposure to a more balanced point of view, I would highly recommend Martin Thompson's blog mechanical-sympathy.blogspot.com. It is a good a starting point on how far single threaded programs can be pushed and where multi-threading can even be detrimental.
Also, I think that fault tolerance is where Erlang really shines. More than a decade after OTP, projects like Akka and Hysterix are finally venturing in the right direction.
A better example would be C2D E8600 @ 3.33Ghz and i5 3470S @ 2.90GHz (3.6Ghz turbo). They are both 65W desktop parts, and the single threaded clock speed is similar. You can see that the C2D gets 1,376 in the single threaded benchmark, while the i5 gets 1,874. The difference is not as drastic (the C2D launched at a significantly higher price point as an enthusiast level chip, while the i5 is a budget chip) but definitely still significant. There are probably even better comparisons but I didn't spend too much time picking out comparable CPUs from different generations.
So while obviously the most literal and absolute interpretation of the statement "sequential programs are getting slower" is nonsensical, I think there's a very valid point being made.
http://www.anandtech.com/show/5057/the-bulldozer-aftermath-d...
Of course you realize even bigger gains on many common workloads using parallelism, but this part of his argument doesn't need the first part, which was wrong.
Erlang solved a problem really well over 20 years ago, its the sanest language by far that I have used when dealing with concurrent programming. (I havent tried go or dart yet) and I owe a lot of what I know to the very smart people building erlang.
However it has barely evolved in the last 10 years, will 2013 be the year of the structs? (I doubt it), every new release comes with some nice sounding benchmark about how much faster your programs will run in parallel and there is never a mention of whats actually important to programmers, a vibrant ecosystem and community, language improvements that doesnt make it feel like you are programming in the 80's. Better constructs for reusing and packaging code in a sane way.
Its fairly trivial in most languages to get the concurrency you need, I think erlang is solving the wrong problem in 2013.
Eh... It is not perfect but considering everything else it brings to the table (fault tolerance, distribution, concurrency) I can easily overlook its warts.
There is always Elixir or some LISP like dialects running on the BEAM VM as an alternative.
I for one like its syntax. It makes sense to me somehow. I which maps were there and I think one day they will be. I would rather they'd work on concurrency, speed and distribution.
But Ericcson dont know how to manage an open source ecosystem, I dont think they particularly want to. It only started using an open source control a few years ago, still no open bug tracker, half the standard library is in terrible shape, there is no good support for 3rd party library integration.
A few years ago I wrote a UI to the documentation that most of the community seemed to massively prefer (http://erldocs.com). Every year I asked the OTP manager how to get it merged but instead they wrote their own (imo) sub par version while every release changing the documentation format which broke mine without warning.
Well Go uses shared-memory concurrency and no other so...
Rust still looks more interesting there, though they still have to deliver the language (it's still heavily in flux)
It doesn't force you to do so.
In fact, the Go team explicitly tells you not to do that: "do not communicate by sharing memory; instead, share memory by communicating." (http://blog.golang.org/2010/07/share-memory-by-communicating...) i.e. they tell you to use the Erlang model.
But as everything, there are trade-offs. Even Go team uses shared memory protected by locks in some cases (see Go standard library) because it's faster or easier that way.
Every good idea (sharing memory between threads is dangerous and therefore should be avoided) taken to extreme becomes cargo cult programming.
Yes, it is dangerous, but at the same time there are plenty of successful projects that do it because there are programmers that can contain that complexity despite the somewhat popular view that this approach dooms you to failure.
Erlang's core concept of concurrency seems like something that'd be better suited as a library and app server than a whole language and runtime.
I've yet to hear of any Erlang-specific magic that cannot be implemented inside another language.
It's also preemptive too. I don't believe JVM have this nor any other languages.
This kind of belligerent rhetoric (we're solving the right problems, everyone else is dumb) is the kind of drivel that gives momentum to language zealots that think language X is better than language Y.
I've contributed to Google Go in the early phases and I was naïve and really believed that Go was the "next big thing." But it turned out to be yet another general-purpose language with some things that were really interesting (goroutines, garbage collection, etc.) but some things that were just same-old same-old. Now, I'm editing a book about Dart and I've since lost my enthusiasm for new languages; I can already see that Dart solves some problems but often creates new ones.
And in a lot of ways Erlang sucks, too. The syntax is outdated and stupid (Prolog lol), it has weird type coercion, memory management isn't handled that well (and many more). Of course, since Facebook uses it, people think it's a magic bullet (Erlang is to Facebook like Python is to Google).
The article also forces readers to attack a straw man. Often times, algorithms simply cannot be parallelized. The Fibonacci sequence is a popular example (unless you use something called a prefix sum -- but that's a special case). So in many ways, the rhetorical question posed by the article -- "but will your servers with 24 cores be 24 times faster?" -- is just silly.
For some context:
http://joearms.github.com/2013/03/27/promoting-erlang.html
He's new at "rah rah!" promotion of his language, so cut him a bit of slack.
How is it any more or less stupid than curly bracket.
Show me another production ready language that has the same level of pattern matching as Erlang.
> Often times, algorithms simply cannot be parallelized.
Who cares. How many people here have implemented individual algorithms and delivered them as units of execution. Sure middleware companies maybe sell a cool implementation of A* or some patented sort algorithm.
You can think of parallelization at system level. Can you handle client connections concurrently (and in parallel?). If yes, that covers a large chunk of the usage domain for platforms these days.
> memory management isn't handled that well (and many more).
Can you expand on that?
When you move a few lines around to refactor, you have to pay a lot of attention to the "ant turd token" line endings. I like Erlang, but still find this aspect of it annoying.
So... how would you have handled it, out of curiosity.
Imo separate heaps is the first big mistake. Even implementations like Erjang (Erlang on the JVM using the Kilim microthreading library -- which I've also contributed to) improve on the copy-from-heap mechanism prevalent in vanilla Erlang. Not only that, but Erlang's memory allocator isn't that well-suited for multi-threaded allocations, which also means that Erlang doesn't (can't?) take advantage of tcmalloc, umem, hoard, etc.
It is understandable though. Just think about how much resources have been put into development of Erlang VM and the runtime/libraries(OTP), and compare it with JVM/JDK. There is just no magic in software development. When talking about high concurrency and performance, the essential things are data layout, cache locality and CPU scheduling etc for your business scenario, not the language.
Use it for what it is built for!
Using C, you might be able to get parallel, but it'll be a lot of work to make it distributed and fault tolerant.
The underlying data structures have little to nothing to do with what's being said in the article.
I'm just taking issue with the bit at the end, where they're bragging about removing a serial bottleneck by rewriting zlib in Erlang in order to remove a lock. Rewriting it in Erlang really doesn't have anything at all to do with switching to a lock-free data structure.
It sounds like this is a zlib usage issue more than anything else.
By contrast, C is a very low-level language that can do anything. You can implement any model of concurrency in C. But you'll be doing all the plumbing yourself, or using a library that does the same. Erlang's model is not the only possible way to be lock-free, and you can pursue other options in C, if you want to.
Yes you can do it. But after a while it is like plugging wholes in a piece of swiss cheese. That is what Joe was saying you start with code that doesn't run well concurrently because concurrency was added later. It is better sometimes to start from scratch with a language that makes concurrency the default and the sequential sections are the exception.
> AFAIK Erlang isn't some magical language that allows traditionally-locked data structures to become lock-free.
There is some magic in how it has separate heaps and how it maps schedulers n:m (n cpus say 2 to 24 to m processes say 2 to 300k), how it provides concurrent garbage collection without stopping the world, how it provides hot code reloaded if that is what you need.
No it won't make coffee for you and it might not work well for a lot of tasks but it just happens to be the right tool for the right job lately as reliable concurrent back-ends becomes more important (as opposed to say single threaded desktop applications)
Unlike another general purpose language (like say, C++ or C#) allow me to grasp what's happening after staring at it for 30 seconds. This is the same problem, I have with Lisp.
Maybe I'm just dyslexic, but these rhetoric pieces for one language or another that says it's concurrent (which it is), fast (obviously), more C than C, will bring the dead to life, create unicorns and other wonderful, fantastic things that I'm sure are all true, just don't seem to be capable of passing into my grey matter.
You know another thing all these amazing super power languages haven't been able to do that even a crappy, broken, in many ways outright wrong, carcinogenic etc... etc... language like even PHP has allowed me to do? Ship in 48 hours.
Before, I get flamed, I already tried that with Nitrogen (http://nitrogenproject.com). It didn't end well, but maybe it will work for someone already familiar with Erlang.
It's like you've written the Mahabharata; it'a a masterpiece and it's one of the greatest epics of all time. Unfortunately, it's written in Sanskrit.
I had the same problem with Lisp (Scheme, to be more specific) and I thought that it'd be impossible to reason about run-times and such. That is, until I learned the language and the standard libraries. I've never looked at Erlang, but I'd bet it's the same issue.
A C++ programmer can look at C# code and figure out what it's doing because they have similar syntax and vocabulary. Just because Erlang isn't immediately accessible to you, it doesn't mean it isn't any good for shipping in 48 hours.
Perhaps if you spend 24 hours sharpening your axe, you'll chop that tree down in another 4 hours instead of using the full 48.
But here's the thing: I switched to C# from VB.Net. Before that, it was VBScript (ASP pages) and even before that it was PHP and JavaScript. At no point did I stop in the language I'm currently working in to start learning another.
Until that changes, I don't see how it will help me.
No, that is not the problem. The problem is total disregard to what most people consider valuable to them. And if they don't get it, framing that as their stupidity rather admitting the fact that the syntax is a little strange to wrap your head around(which is true).
>>Perhaps if you spend 24 hours sharpening your axe, you'll chop that tree down in another 4 hours instead of using the full 48.
In all fairness sharpening your erlang axe might take 24 months not 24 hours.
And there are languages that fill those niches. When people say crappy language, more often than not, crappy for their needs(or sometimes their expertise). There is no need to fight about it; its like saying who prefers which color.
The reason you do lisp is not to do coding in it(given the current state of affairs), but to reshape one's mind by thinking in terms previously unseen because we could not see forest for the trees.
And that ability to see something from different view points arms you with weaponry to solve challenging problems. There is atleast a physicist who agrees with me on that :) http://www.youtube.com/watch?v=IfUhkSEoku0
Very nice video.
Try Webmachine and erlydtl w/Mochiweb. Webmachine is like Sinatra, but more, and erlydtl is just Django templating.
Erlang is absurdly easy to read, but it doesn't look like algol, it looks like prolog. You get used to it, and after you do, coding in it is really fast.
Back to your point though, I think once you understand the language it is actually a lot simpler to understand what is going on. Modules are usually very self contained, and you don't get the layers upon layers of indirection you see in other frameworks (I'm looking at you Rails). I think the functional style of programming as well helps to keep things simple, it doesn't make sense to have a 20 line function in Erlang.
Ignoring that point, this seems like a poor point for comparison as it's a trivially parallelized task because zlib operates on streams and shouldn't have any thread contention. There's very little information in the description but unless there are key details missing, this doesn't sound like a problem where Erlang has much interesting to add. The most interesting aspect would be the relative measures for implementation complexity and debugging.
1. Erlang has locks and semaphores [1], receive is a lock, actors are semaphore. Erlang chose a 1 semaphore/ 1 lock per process model
2. Erlang scales better not because of being lock-free (see above), but because it easily uses async compared to other languages
3. Async prevents deadlocks not Erlang being lock-free (see above)
> The problem that the rest of the world is solving is how to parallelise legacy code.
As member of the rest of the world, I can assure you that I'm not trying to solve either of these problems. :p
It'll be the same over the next 20 years as well.
I predict that we'll see a lot of technological leaps which will serve as much to maintain the ability to run "old code" in new and interesting ways as to enable a brave new world of purpose-built languages.
In the next few decades we'll see advances in micro-chip fabrication and design as well as memory and storage technology (such as memristors) which will result in even handheld battery powered devices having vastly more processing power than high-end workstations do today.
Is that an environment in which one seeks to trade programmer effort and training in order to squeeze out the maximum possible efficiency from hugely abundant resources? Seems unlikely to me, to be honest.
Indeed, it seems like the trend of relying on even bloatier languages (like Java) will continue. Do you think anyone is going to seriously consider rewriting the code for a self-service point-of-sale terminal in Erlang in order to improve performance? That's not the long pole, it never has been, and it's becoming a shorter and shorter pole over time.
In the future we'll be drowning in processor cycles. The most important factor will very much not be figuring out how to use them most efficiently, it'll be figuring out how to maximize the value of programmer time and figuring out how to use any amount of cycles to provide value to customers effectively.
(I think that advancements in core, fundamental language design and efficiency will happen and take hold in the industry, but mostly via back-door means and blue sky research, rather than being forced into it through some impending limitation due to architecture.)
The "mainstream" has been relying on incremental improvements for decades, and in doing so avoided rewriting legacy code until last possible moment
Some people have taken concurrency upfront and anecdotally seen cost / performance benefits plus more modern codebases and have anecdotally enjoyed competitive advantages in areas where concurrency makes a difference
We will never see the average, user interface bother with concurrency and legacy rewrites because the competitive advantages are low.
There are likely to be areas where the concurrency advantage is great enough - if you like erlang look for those niches
It's like designing a race car, or a fighter jet. Sure, they are amazing things. But are people ever going to commute to work in anything resembling a Bugatti Veyron or an F-22? Of course not. Neither maximum automotive performance nor air combat effectiveness are the sorts of things that are normally necessary to optimize for in daily life. Some time in the far future we're going to have both the tools to write amazingly efficient programs and to do so with a minimal amount of fuss from the programmer's perspective, but it'll be a long time getting there. And in the meantime there are going to be plenty of cycles of figuring out how to produce performance gains with the least disruption to existing ways of doing things.
The problem with these languages remain unchanged. The syntax is so strange and esoteric, learning and doing anything basic things with them will likely require months of learning and practice. This lone fact will make it impractical for 99% for all programmers in the world.
No serious company until its absolutely unavoidable(and situation gets completely unworkable without it) will ever use a language like Erlang or Lisp. Because every one knows the number of skilled people in market who know Erlang, are close to zero. And those who can work for you are going to be crazy expensive. And not to mention the night mare of maintaining the code in this kind of a language for years. There is no friendly documentation or a easy way a ordinary programmer can use to learn these languages. And there is no way the level of reusable solutions available for these languages as they are for other mainstream C based languages.
In short using these languages attracts massive maintenance nightmares.
The concurrency/parallelisation problem today is very similar to what memory management was in the 80's and 90's. Programmers hate to do it themselves. These are sort of things that the underlying technologies(Compilers/VM's) are supposed to do it for us.
I bet most of these super power languages will watch other pragmatic languages like Perl/Python/Ruby/Php etc eat their lunch over the next decade or so when they figure out more pragmatic means of achieving these goals.
You know, Lisp's syntax is weird but it is exactly this what makes it so flexible. It's easy to manipulate code as data, because the syntax is very regular. Try to do that with C's syntax...
So, unless someone knows how to solve this in a easy way, I'd say that the lot's of parentheses are actually a pragmatic decision (i.e. you want easy macros... so you have to use this uncommon syntax).
If popularity is the goal, then maybe those languages were not pragmatic. However, It seems the language designers of such powerfull languages (e.g. Lisp, Erlang, Haskell) were looking to solve other problems where popularity is really not a concern.
Why you'd "manipulated code as data"? To write macros? A good template system can help with that (if you need it) without homoiconicity.
For me, the level of manipulation of "code as data" (and vice versa) you get with JSON/JS is enough for a lot of use cases.
I can't speak for Erlang, but Lisp, really? Strange syntax? It's about as straightforward as it gets. And you can learn it in a day, a week tops.
>No serious company until its absolutely unavoidable(and situation gets completely unworkable without it) will ever use a language like Erlang or Lisp.
Lots of serious companies used both. Lisp was widely used in academia and in places like the JPL. AutoCAD worked with Lisp. Heck, even more obscure languages like OCaml are widely used in the financial domain. And today, lots of startups use Clojure. This very site (HN) is made in a Lisp.
Sure, using LISP has some drawbacks and is avoided by the mainstream enterprises today, because of lack of developers and commercial support (compared to C, Java, .NET etc).
>I bet most of these super power languages will watch other pragmatic languages like Perl/Python/Ruby/Php etc eat their lunch over the next decade or so when they figure out more pragmatic means of achieving these goals.
I'm sorry to break it to you, but Perl, Python, PHP and even Ruby have already peaked. They are not going anywhere (upwards) in the next decade or so.
Yes and no. Where it's true, sure, arrogant jerks.
But the Lisp weenies have realized they know something the rest of the world doesn't. They think it's important, and it is in a sense. If only you knew that... you'd agree. (Ok, try not to take that too seriously.)
Unlike the impossibly abstract Lisp Truth™, this Erlang bit is centered in a very concrete fact that affects all of us. If you don't care about it today, it will affect you tomorrow all the same. You might as well argue that you never much cared for oxygen and who cares if the atmosphere is slowly turning to methane?
Joe wasn't saying you have to write Erlang, he was saying you need to write concurrent programs. If another language eats his lunch, it will probably do it in the same way that would be done in Erlang. There are alternatives, but the actor model is by far the most programmer-friendly that I have ever seen.
>>I have heard this comment millions of time from management people.
Really?
>>These so called pragmatic languages are good to build "applications" which are just a bunch of API calls bound together
Guess what?
All software is building on top of something. Every heard of system calls?
>>and most of the so called programmers are building "applications", they have no idea of how to build real "systems"
May be you being such a great programmer can show us how to build a "real system", without ever using a API of anything ever.
>> Try to build "systems" in your so called pragmatic language and you will find what I mean.
Something like 99.99% of the world does that. Erlang is not even in the list top 10 languages in the world today.
There are plenty of companies that already use Erlang and/or Lisp. In fact, Erlang comes from industry, designed to solve Ericsson's business problems.
vs three paragraphs later
> Alexander’s talk gave us a glimpse of the future. His company concurix is showing us where the future leads. They have tools to automate the detection of sequential bottlenecks in Erlang code.
why is that not a contradiction? because an erlang program isn't "sequential" to start with?
Yes. The point is that in a well-coded erlang program only bottlenecks should be sequential (and the bulk should be concurrent), the goal's tool would be (I haven't seen the presentation so I'm throwing ideas to the wall) to see what dependencies lead to sequences in the system reducing overall concurrency and leaving the developer to handle fixing this part if possible.
It doesn't try to automatically parallelize a sequential program, and it does not start from fully sequential programs in the first place.
(not saying I agree with Joe's assertions, they're quite inflammatory and at a very fundamental level lack solid evidence. I have to say I prefer his milder tone to this new "rha rha" one, though this one may yield more visibility for the language I fear the drama)
That's not particularly impressive you know.
So, there was an error in someone's code which you rewrote without the error and it ran faster? Well done detective...
We need more parallel programs, no doubt, but we need more, better programmers, who are willing to write in compiled languages with low-overhead.
Erlang allows you to create concurrent programs, i.e.: programs where the result is schedule dependent.
One right problem is allowing people to write deterministic parallel programs. This gives you the speed (from parallel) with the reliability (from deterministic).
- break program into function calls that match the steps that can happen in parallel
- wrap the function calls in messages passed over the network
+ i.e. process(thing) -> post(thing)/poll_for_things()
- split the sender and receiver into different processes
I've spent many years developing and reviewing products in the telecoms realm, and have found that failing to realize when something like Erlang brings life-saving concepts to your project may well make the difference between delivering on time and disappearing into a black hole of endless complexity. It's not for everyone, but when it fits, boy does it help!
Donald Knuth: "During the past 50 years, I’ve written well over a thousand programs, many of which have substantial size. I can’t think of even five of those programs that would have been enhanced noticeably by parallelism or multithreading."
Also a recent book using other Erlang web technology: http://www.amazon.com/Building-Web-Applications-Erlang-Worki...