Alert Evaluations: Incremental Merges in ClickHouse (opens in new tab)

We insert from our alerts worker because we want the aggregation to happen per alert (with the aggregated data filtered by the particular alert definition). As each alert is evaluated, we run the following [1] INSERT INTO ... SELECT ... statement based on the alert definition. We can't aggregate with an MV since we'd need to create an MV per unique alert that a customer may set up.

[1]: https://github.com/highlight/highlight/blob/c526daea31fdf764...

Precisely, whats stopping them from using simple Incremental Materialised view?

The quantile() method in clickhouse is also approximate although it uses a more simplistic reservoir sampling model than GK, but quantileGK() is also available. quantileExact() exists but indeed becomes impractical as you point out.