undefined | Better HN

0 pointsJtsummers4y ago0 comments

I'm not sure there's too much debate on what parallelism is (simultaneous execution of 2 or more processes/threads). The broader debate seems to be on what concurrency is. Is it a design feature (in which case coroutines in a single threaded program offer concurrency) or is it an execution model (in which case it intersects with or is the same as the common definition of parallelism).

The other debate is on how parallelism (and concurrency) should be done. Shared-nothing or shared-memory? It'd be foolish to say that only one of them is parallelism, but it would be reasonable to debate the merits of each approach in different systems and circumstances.

0 comments

7 comments · 3 top-level

bullen4y ago· 3 in thread

I'm trying to name the version of parallelism that can share memory between threads, and so far I have tried "Joint Parallelism" and now lately "Atomic Parallelism" without much success.

What would you suggest?

JtsummersOP4y ago

I'm not sure it needs a special name. It's the default model that almost everyone learns first, shared-memory parallelism/concurrency. It's the whole reason that semaphores and mutexes were developed (to avoid contention for the same resource or resource set from multiple threads of execution).

bullen4y ago

Yes, but I also mean that parallelism to be atomic/non-locking. afaik only Java/C# can do this on a high-level (without going back to atomic arrays in C) but I still can't explain why, the best I found so far is:

"Many lock-free structures offer atomic-free read paths, notably concurrent containers in garbage collected languages, such as ConcurrentHashMap in Java. Languages without garbage collection have fewer straightforward options, mostly because safe memory reclamation is a hard problem..." - Travis Downs https://travisdowns.github.io/blog/2020/07/06/concurrency-co...

But it seems there is also some benefit to have a VM in addition to a GC for this joint/atomic parallelism.

2 more replies

kazinator4y ago

Historically, this is known as "multiprogramming". Multiple programs making forward progress inside the same computer, via interleaved or concurrent execution, the latter being called "multiprocessing".

vbezhenar4y ago· 1 in thread

> The other debate is on how parallelism (and concurrency) should be done. Shared-nothing or shared-memory?

Obviously it should be "shared nothing" as it would allow you to horizontally scale your application and code will be much easier to develop and test. But in reality you inevitably need to have a shared state in one way or another. You could try to move that state into some dedicated service, like database, but it's not always performant enough (or even applicable). So you end up with shared memory because of performance.

vlovich1234y ago

I think Rust shows the advantages of shared nothing can be mitigated through novel language design.

capableweb4y ago

Thanks, I agree with you overall. My comment was more focused on the (old and "well-used") adage of "X is not true Y" that gets used over and over again to look down on X, while the definition of anything "true" is most likely just personal opinion and is different depending on who you ask.

j / k navigate · click thread line to collapse