Aeron: High-Performance Open Source Message Transport [video] (opens in new tab)

(infoq.com)

38 pointsenginous11y ago26 comments

26 comments

The described log buffer implementation does not seem wait-free to me. If one of the writers fails all other subsequent writes will fail, too. Subsequent writers can still advance the tail pointer and copy their messages into the buffer but the reader will never see their messages because the failed writer did not update the length field in its header.

So physically the operations for the subsequent writers completed after updating the length fields in their headers but logically they are not completed until they become visible to the reader and that requires updating the length field in the header of the failed writer. The situation seems even worse when the writer responsible to rotate the logs fails.

I did not look into the source code, they are probably dealing with failed writers, but the presentation alone does not provide any hints to me how they do it.

mjpt77711y ago

Can you expand on how you see the writer can "fail"? Also please relate this to other algorithms you see as wait-free but don't have your failure issue. The code is very simple and does not have external dependencies. It is a three step process of advance tail, copy in message, apply header. All within the implementation, all threads make progress and completes in a finite number of steps, and thus wait-free.

https://github.com/real-logic/Aeron/blob/master/aeron-client...

danbruc11y ago

I will kill or at least pause the thread after it advanced the tail pointer but before it updated the length field in the header. In other wait-free algorithms all the other threads will usually help failed threads to complete their work.

mjpt77711y ago

Can you point to a respected publication that states that is the definition of a wait-free algorithm, i.e. that other threads must help and it must cope with killing the thread externally?

By kill a thread I assume you mean use the OS to interrupt and terminate the thread from the outside while in such a simple algorithm and yet expect it to cope. Have you tried this on any java.util.concurrent classes and reported the "bugs" you have found? I'd be interested in the feedback you got. :-)

1 more reply

anton_gogolev11y ago

Half-expected something related to Herman Miller's Aeron.

http://www.hermanmiller.com/products/seating/performance-wor...

j / k navigate · click thread line to collapse

26 comments

danbruc11y ago

I did not look into the source code, they are probably dealing with failed writers, but the presentation alone does not provide any hints to me how they do it.

mjpt77711y ago

https://github.com/real-logic/Aeron/blob/master/aeron-client...

danbruc11y ago

mjpt77711y ago

Can you point to a respected publication that states that is the definition of a wait-free algorithm, i.e. that other threads must help and it must cope with killing the thread externally?

1 more reply

anton_gogolev11y ago

Half-expected something related to Herman Miller's Aeron.

http://www.hermanmiller.com/products/seating/performance-wor...

j / k navigate · click thread line to collapse