undefined | Better HN

0 pointsboredandroid12y ago0 comments

This is not correct: http://kafka.apache.org/documentation.html#design

:-)

0 comments

2 comments · 1 top-level

idunno24612y ago· 1 in thread

Which part is not correct? They go into great detail about their design for throughput. Re: reliability, "Not all use cases require such strong guarantees. For uses which are latency sensitive we allow the producer to specify the durability level it desires."

Pre-0.8, if a machine fails you lose all the data on that machine, only the lower durability levels were available. It guarantees at least once processing, while other queues generally make stronger claims. etc.

It's still very good at what it does.

mumrah12y ago

Before 0.8, you are correct that if you lose a disk, you lose the data. However, just because a broker goes down, that doesn't mean you lose the data on it - it just becomes unavailable to consumers. The log files backing Kafka partitions are fully durable append only files. So the durability guarantees are pretty good, even before 0.8.

What you're talking about is failover and fault tolerance, which are greatly improved in 0.8 with the addition of replication.

j / k navigate · click thread line to collapse