Ten years of ClickHouse in open source (opens in new tab)

ClickHouse replacing Loki finally made our observability stack feel 'right'. It really is a powerhouse for logs and general analytical queries.

and yet it's too difficult to package https://repology.org/project/clickhouse/versions

I discovered ClickHouse around 2017-18 and built a PoC to replace Elasticsearch: 5x better storage and qps, in a couple of weeks.

Managers rejected it because it wasn't well known and was seen as "some database made by Russians."

On a personal level, it's quite sad to have seen that train coming so early and not been able to get on board.

ClickHouse recently has been a breath of fresh air compared to using timescaledb for a long time. Although psql is the greatest there is and I really enjoyed the fact that I could rely on a single database system to run everything, when it came to migration maintenance and deployment it's really a pain and it also feels like development on timescaledb is a bit wishy washy with all the structural changes from version to version and it really feels like an alpha product sometimes.

For our metrics and autoscaling engine at Cloud 66, we went through 5 iterations before settling on Clickhouse: 1. Redis 2. Cassandra 3. Handrolled: Ruby + RabbitMQ 4. Handrolled: Go + RabbitMQ 5. Clickhouse

Every time we reached some limit or huge optimization burdens that were unfeasible. Clickhouse has been rock solid for the past 4 years.

> You can open a pull request as an experiment, without aiming for it to be merged - it will be tested with the same level of scrutiny as production releases. Found a new memory allocator, a new compression library, a new hash table, a data format, or a sorting algorithm? - bring it to ClickHouse, and it will expose it inside-out

Wow

It's interesting that the blog post places SQLite and Ladybird on the spectrum, but omits it's chief open source rival: DuckDB.

Agree that Level 3 is what inspires confidence. But we need to invent new business models to sustain in the era of vibe-coded databases.

It is sad they are afraid to mention on the page that "data processing for a web analytics system ... similar to Google Analytics" was actually something used in Yandex.

On top of being a really good OLAP database, what has been a game changer for me is the built-in connectors for bringing in data from remote sources. It can manage automatic recurring import of a s3 folder containing parquet/json files and it can also connect directly to Postgres. For our data warehouse at a medium sized newspaper, we switched from Druid+postgres+trino to just one big clickhouse node and I’ve never looked back. Much more performant, practical, and a lot less maintenance.

Clickhouse has been a game changer for some of the companies i have worked in the past. This reminds me of this podcast episode (1) from the Rust in Production pod about their Rust adoption.

1. https://open.spotify.com/episode/0TBKDUhO0KihBxEzZqnQx1

Clickhouse is *really* gatekeeping the "zero copy replication" where you store data on object-storage and have high availability from the open source version.

I've been using clickhouse for the last year for in-house analytics and found it a really pleasant experience, thanks for all the progress you've made

clickhouse is the low key amazing tech people are busy using instead of posting about. keep it up!

I used to keep all of OnlineOrNot's timeseries data entirely in a hot postgres db with the rest of the relational data.

Used to take a few seconds to get a week's uptime data and do some useful analysis.

Since moving to Clickhouse I think I can grab a full year's data in around 200ms (probably less if I try optimising it). Still completely blows my mind everyday.

If your data is too big for postgres, it seems like moving straight to Clickhouse is the best option. We have been through an whole array of distributed database technologies, and Clickhouse might be first one that doesn't have too many compromises.

We use Clickhouse in a rails app for our customer facing dashboard analytics, logging, and datalake type stuff where Postgres is too heavy and expensive. The web admin panel they built is great and we’ve had solid performance.

CHDB deserves a lot of credit too, wish they would unmark the rust version as experimental. The python version works very well!

what are you guys using it for other than collecting analytics?

The query speed deserves the praise, but the JSON ingestion path has quiet footguns nobody mentions here. Every numeric column comes back as a string over JSONEachRow, so a forgotten Number() cast silently turns arithmetic into string concatenation, and with input_format_skip_unknown_fields enabled a single typo in a column name drops that field with no error at all. Worth wiring an assertion that inserts a row and reads it back into CI before trusting the dashboards.

10 Years! quite a long journey, specailly observeability part is need of hour