I think that might make sense ingest side, but that's very expensive to deal with if you're doing anything remotely large.

I think sinking into something like delta-lake or iceberg probably makes sense at scale.

But yeah, I definitely agree that CSV is not great.