undefined | Better HN

0 pointspierrekin2mo ago0 comments

I think there are two kinds of partition based approach which may cause some confusion if lumped together in this kind of comparison.

Insert and delete with old partition drop vs insert only with old partition drop.

The semantics of the two approaches differ by default but you can achieve the same semantics from either with some higher order changes (partitioning the event space, tracking a cursor per consumer etc).

How does PgQue compare to the insert only partition based approach?

0 comments

1 comments · 1 top-level

samokhvalov2mo ago

1. partitions are never dropped – they got TRUNCATEd (gracefully) during rotation

2. INSERT-only. Each consumer remembers its position – ID of the last event consumed. This pointer shifts independently for each consumer. It's much closer to Kafka than to task queue systems like ActiveMQ or RabbitMQ.

When you run long-running tx with real XID or read-only in REPEATABLE READ (e.g., pg_dump for long time), or logical slot is unused/lagging, this affects performance badly if you have dead tuples accumulated from DELETEs/UPDATEs, but not promptly vacuumed.

PgQue event tables are append-only, and consumers know how to find next batch of events to consume – so xmin horizon block is not affecting, by design.

j / k navigate · click thread line to collapse