Very off topic, but I have read several times the argument that the rise of functional programming is due to it's easy concurrency (since functions don't have side effects) and that concurrency becomes more and more important due to moores law being dead (i.e. we can't scale the hardware up, we have to add cores to our processors).
Could someone with more experience comment on that? Is concurrency really easier in functional languages and is the rising importance of concurrency a valid reason to look into functional programming?
I think it depends on who's writing the code. A little bit of shared mutable state here and there isn't gonna kill you, but if the program's architecture is poor, that'll blow up and spread everywhere and the next thing you know is you're spending half your time worrying about data races, deadlocks, and a locking nightmare.
Shared-nothing architectures fix that; your synchronization will happen over a narrow range of well defined primitives (e.g. message queues), though this doesn't prevent race conditions.
Functional programming that encourages pure functions & immutable state is a bit of a straight jacket that keeps you from making a mess that blows up. I think it helps, and I wish $stuffAtWork were written in such a language because I'm tired of wrestling with terrible architecture (that I'm not really allowed to fix, given the time constraints).
The other thing that makes a difference is what your problem looks like. Sometimes concurrency is absolutely trivial, like dispatching sections of an array for threads to perform (independent) compute on, and come back with a result. Sometimes it's way more complicated...
Many of them provide a concurrency monad which allows you to write expressions like this:
read_from_socket_into_buffer params >>=
process_buffer >>=
...
BTW, It's quite easy to implement your own concurrency monad, however: the real work is the wrapping of all the blocking system calls into this framework.
Specifically, concurrent Haskell is good and also quite fast. It doesn't quite compare to Erlang in the concurrency department although the languages are very different in other ways, so there are reasons to prefer Haskell anyway. I think this shows that just having a functional language that emphasizes effect-free programming is not sufficient. If you want a good concurrent language, then you have to design it like that, and not simply rely on the absence of shared mutable state.
In practice, people write massively concurrent systems all the time, in the form of dynamic web sites that handle many concurrent requests, and synchronise access to a shared mutable state via a transactional database of some kind. These don't depend on fancy languages, but on a rigid architecture that tries to isolate synchronisation issues in the web server and database components.
While this article is about concurrency, I'll also add a note on parallelism: Parallel functional programming has IMO not been shown to work well in practice. While it is trivial to parallelise more or less any pure functional program correctly, it is very hard to actually gain meaningful speedup from it. The single point of parallelism is performance, and most current implementations of functional languages (including all general purpose functional languages I know of) simply have too many bottlenecks or pitfalls to scale for computational work as well in practice as one might expect in theory. For example, concurrent Haskell is fast because most of the actual work tends to be IO and GHC has an excellent parallel IO manager and scheduler, while parallel Haskell is hampered by, among other things, a slow garbage collector.
Btw, what are some of those other things hampering Haskell's performance with parallel computing? Wouldn't the garbage collector mainly produce pauses instead of really destroying parallel performance? Although a concurrent garbage collector has recently been merged to the nightly GHC.
Another advantage is, that you can unit test each function on its own, without setting up a whole host of preconditions. That's because of referential transparency.
IME locking isn't really all that hard until you need to squeeze out performance. If you can handle one big dumb lock around everything, it's easy. The finer grained your locking gets, the harder it is to get right(even then, lock hierarchies can help to a great degree).
For example, iterator invalidation. Which is probably the most common thing I've seen when people use dumb locks to make a concurrent program sound. Either your iterators need to be aware of the lock and hold it through their lifetime (which means practically only one iterator can exist, which is undesirable) or your datastructure needs to be aware of outstanding iterators to it and update them (which means you have added logic to simple mutations, which is also undesirable).
It's problems like that you look for in non-functional languages that require library authors to say things like "this is/isn't thread safe" in their documentation, while providing those guarantees manually. And as a consumer of the library you need to be aware of these things and look for them, and what an error looks like when someone messes up.
I don't know what you're trying to say about stacks/the call stack in functional languages, because they implement their patterns in variety of ways - including locking. It's just about providing particular guarantees in the API, and the benefits of those guarantees are that it prevents entire classes of bugs like iterator invalidation and data races.
There are also all sorts of clever data structures you can use that don't require duplication of the underlying data. The easiest is a singly linked list without insertion. No locks required.
Only you know if concurrency is relavant to your work. A lot of internet stuff is deeply concurrent, but there's a lot of sequential work out there, too.
I have experience with Erlang. It's amazing for concurrency, but the reason is really not anything it gives you, but everything it prevents you from doing, or requires you to do.
Erlang's immutability is important because it means you can't save a reference to something and have it magically updated; if you want a value updated somewhere, you need to send the current value or request the current value at time of use; explicit synchronization makes concurrency clearer, and helps influence system design to reduce synchronization. Immutability also makes message passing easier; the content of a message can't change after it's composed which means the implementation is free to make a copy of the message if it's convenient and as a programmer, there's no need to consider the effects of changing something that you sent to another process. Immutability also makes garbage collection very simple; it's impossible to form a looped datastructure, so there's no need for mark and sweep, a simple copying collector is sufficient. Individual stacks per Erlang process (like a green thread) means GC pauses don't stop the world, only individual processes, and pause length is bounded and proportional to the memory use of the process.
You could certainly set up a similar environment in C or whatever traditional language; the problem would be enforcing your system design on your own code, and partitioning off library code (including libc and/or kernel syscalls) to ensure it doesn't impact your system design either.
You can use the same approaches in non-functional languages. You can only use immutable data structures, and pure functions everywhere. Then you get the same benefits of easier reasoning about concurrent execution, as you get from a functional programming language.
So the point of using an actual functional language, is that the compiler pushes you very strongly towards programming this way. It's kind of like using a language with built in garbage collection. You could implement a garbage collector in C, and use it everywhere in your program where you allocate and release memory. But having it build into the language frees you up from having to think about it very much.
So, deep down in the guts of the language's libraries there must be mutable operations. Which kind of defeats the whole point.
I don't get point of functional programming. Machines don't work that way.
Data races can be statically removed by carefully restricting certain parts of the language design, see Pony. https://tutorial.ponylang.io/#what-s-pony-anyway
Bonus: learn aspects of deadlocking by playing a game: https://deadlockempire.github.io/
It's because of my concurrent C homework assignments that I was able to really appreciate Go's channels in my first internship.
It’s like with cryptography - don’t roll your own solutions just because you know what xor is. You’ll fail miserably.
Why copy techniques meant for Netflix and Facebook when your org or app is most likely 1/1000th their size. Phallic size jealousy at work.
Most concurrent and parallel work can and should be done on a true-and-tried RDMBS for most orgs and apps. Use transactions/rollbacks properly and let the RDBMS manage most the grunt work instead of reinvent the wheel in app code.
K.I.S.S. and use-the-right-tool-for-the-job.
I like the attitude the article takes. To successfully use concurrency, you should understand the tools available to you and their tradeoffs so you can make the right decision for your requirements.
Not necessarily, but it is fine for this purpose I suppose. See https://news.ycombinator.com/item?id=21959692
Glad to see lock hierarchies mentioned. Barriers are new to me so that was nice.
IMO, it would be nice to at least have a mention of lock-free techniques and their advantages and disadvantages.
Every multiprocessor system nowadays has coherent caches for normal memory, meaning that if you have reads that you perceive as stale, it isn't because of the cache, but because the hardware doesn't guarantee sequential consistency.
You can have problems due to relaxed consistency on a system without caches, and the problems on systems with caches aren't due to the caches.
There are situations when you have to worry about lack of coherency for other memory types, but AFAIK these are rarely exposed to userspace.
1. std::thread::hardware_concurrency
This is the number of threads that can execute in parallel, no?
2. "Memory-level parallelism"
How many memory operations can be "outstanding" at once - seems comparable to a single core issuing multiple disk reads. The memory operations aren't really serviced simultaneously, they just have overlapping lifetimes.
For more fun, some people refer to the case where performance is limited by the amount of memory-level parallelism available as "concurrency-limited": https://sites.utexas.edu/jdm4372/2018/01/01/notes-on-non-tem...
SIMD systems effectively give you parallelism without concurrency - only one instruction is executing, but it's operating on multiple dataflows.
Your linked definition of "concurrency limited" seems to refer to utilisation. In the scenario described, how effectively the processor can be utilised depends on how many concurrent tasks it has in progress so it has something to do while one of them is waiting for a cache miss.
Databases provide transactions. This mechanism is also an inspiration for a synchronisation model called Software Transactional Memory proposed for Haskell, and used as "the" synchronisation model in Clojure. Locks and semaphores are rather lower-level primitives and it's harder for humans to reason about them with an ease comparable to using CSP or STM.
[0] https://en.wikipedia.org/wiki/Dining_philosophers_problem [1] https://rosettacode.org/wiki/Dining_philosophers
I was hoping for more real life examples where you can see the effects of concurrency parallelism.
Why is that? I'm pretty sure that the author's intention is not to equip the readers with the tools to make buggy programs, yet that is exactly what happens here.
When you need to start worrying is when you start implementing your own synchronization (either rolling your own primitives or going lock-free).
'moring needs to clarify what they are talking about. It's perfectly possible to write correct code using pthreads on modern hardware with no understanding of memory consistency.