Inside PostHog: SSRF, ClickHouse SQL Escape and Default Postgres Creds to RCE (opens in new tab)

(mdisec.com)

110 pointsarwt3mo ago35 comments

35 comments

I work on security at PostHog. We resolved these SSRF findings back in October 2024 when this report was responsibly disclosed to us. I'm currently gathering the relevant PRs so that we can share them here. We're also working on some architectural improvements around egress, namely using smokescreen, to better protect against this class of issue.

piccirello3mo ago

Here's the PR[0] that resolved the SSRF issue. This fix was shipped within 24 hours of receiving the initial report.

It's worth noting that at the time of this report, this only affected PostHog's single tenant hobby deployment (i.e. our self hosted version). Our Cloud deployment used our Rust service for sending webhooks, which has had SSRF protection since May 2024[1].

Since this report we've evolved our Cloud architecture significantly, and we have similar IP-based filtering throughout our backend services.

[0] https://github.com/PostHog/posthog/pull/25398

[1] https://github.com/PostHog/posthog/commit/281af615b4874da1b8...

thenaturalist3mo ago

Wow, chapeau to the author.

What an elegant, interesting read.

What I don't quite understand: Why is the Clickhouse bug not given more scrutiny?

Like that escape bug was what made the RCE possible and certainly a core DB company like ClickHouse should be held accountable for such an oversight?

matmuls3mo ago

ssrf was the entry point, and clickhouse is supposed to be an internal only service, but one could reach it only with that ssrf, so hence less of "scrutiny". The 0day by itself wouldnt be useful, unless an attacker can reach clickhouse, which they usually can't.

thenaturalist3mo ago

But if they do, prohibiting SQL injection, a critical last mile vulnerability, seems trivial?

3 more replies

simonw3mo ago

The ClickHouse bug was fixed here: https://github.com/ClickHouse/ClickHouse/pull/74144

lkt3mo ago

Out of interest, how much does ZDI pay for a bug like this?

rs_rs_rs_rs_rs3mo ago

They probably don't accept something like this. Not that many Posthog self-hosted instances out there...

lkt3mo ago

That's what I thought too, but the article says it was submitted to ZDI and they handled the communication with Posthog

wtfse3mo ago

All of these vulnerabilities accepted by ZDI.Feel free to search the following codes. ZDI-CAN-25351. ZDI-CAN-25352. ZDI-CAN-25350. ZDI-CAN-25358.

anothercat3mo ago

Does this require authenticated access to the posthog api to kick off? In that case I feel clickhouse and posthog both have their share of the blame here.

nightpool3mo ago

It looks like the entire class of bugs here are "if you have access to Posthog's admin dashboard, you can configure webhook URLs that hit Posthog's internal services". That's not particularly surprising for a self-hosted system like the author's, but I expect it would pretty bad if you were using their cloud-hosted product.

anothercat3mo ago

Ah of couse! I forgot about the cloud hosted option.

1 more reply

yellow_lead3mo ago

Need an edit here

> As it described on Clickhouse documentation, their API is designed to be READ ONLY on any operation for HTTP GET As described in the Clickhouse documentation, their API is designed to be READ ONLY on any operation for HTTP GET requests.

wtfse3mo ago

hi, this is the author of the article. Thanks for the feedback mate. fixed it.

yellow_lead3mo ago

Thanks! Great article

taw_12653mo ago

PostHog does a lot of vibe coding, I wonder how many other issues they have.

Nextgrid3mo ago

Not that I’m disproving it but do you have a source? Companies say all kinds of things for hype and to attract investors, but it doesn’t necessarily make it true.

matmuls3mo ago

looking at their commits, there are about 300+ commits tagged with " Generated with https://claude.com/claude-code" attribution.

1 more reply

danr43mo ago

Very nice write up!

wtfse3mo ago

I'm glad you liked it mate.

j / k navigate · click thread line to collapse

35 comments

piccirello3mo ago

Here's the PR[0] that resolved the SSRF issue. This fix was shipped within 24 hours of receiving the initial report.

Since this report we've evolved our Cloud architecture significantly, and we have similar IP-based filtering throughout our backend services.

[0] https://github.com/PostHog/posthog/pull/25398

[1] https://github.com/PostHog/posthog/commit/281af615b4874da1b8...

thenaturalist3mo ago

Wow, chapeau to the author.

What an elegant, interesting read.

What I don't quite understand: Why is the Clickhouse bug not given more scrutiny?

Like that escape bug was what made the RCE possible and certainly a core DB company like ClickHouse should be held accountable for such an oversight?

matmuls3mo ago

thenaturalist3mo ago

But if they do, prohibiting SQL injection, a critical last mile vulnerability, seems trivial?

3 more replies

simonw3mo ago

The ClickHouse bug was fixed here: https://github.com/ClickHouse/ClickHouse/pull/74144

lkt3mo ago

Out of interest, how much does ZDI pay for a bug like this?

rs_rs_rs_rs_rs3mo ago

They probably don't accept something like this. Not that many Posthog self-hosted instances out there...

lkt3mo ago

That's what I thought too, but the article says it was submitted to ZDI and they handled the communication with Posthog

wtfse3mo ago

All of these vulnerabilities accepted by ZDI.Feel free to search the following codes. ZDI-CAN-25351. ZDI-CAN-25352. ZDI-CAN-25350. ZDI-CAN-25358.

anothercat3mo ago

Does this require authenticated access to the posthog api to kick off? In that case I feel clickhouse and posthog both have their share of the blame here.

nightpool3mo ago

anothercat3mo ago

Ah of couse! I forgot about the cloud hosted option.

1 more reply

yellow_lead3mo ago

Need an edit here

wtfse3mo ago

hi, this is the author of the article. Thanks for the feedback mate. fixed it.

yellow_lead3mo ago

Thanks! Great article

taw_12653mo ago

PostHog does a lot of vibe coding, I wonder how many other issues they have.

Nextgrid3mo ago

Not that I’m disproving it but do you have a source? Companies say all kinds of things for hype and to attract investors, but it doesn’t necessarily make it true.

matmuls3mo ago

looking at their commits, there are about 300+ commits tagged with " Generated with https://claude.com/claude-code" attribution.

1 more reply

danr43mo ago

Very nice write up!

wtfse3mo ago

I'm glad you liked it mate.

j / k navigate · click thread line to collapse