You might think the world doesn't need another observability solution. There are already great ones like DataDog, Elastic, and New Relic. But we’ve come to believe these tools give you suboptimal capabilities. Each modern application consists of many services and “big data”-scale telemetry, and in this landscape, there’s a huge benefit to analytics, event correlation, and advanced OLAP queries.
We think that:
1. There’s a need for an analytics solution for telemetry data (logs, metrics, traces). Once such a solution exists [with low barriers to entry], it could change how we approach production debugging. 2. The mainstream existing solutions can’t provide analytics because their underlying databases are optimized for storage, search/filtering, and basic aggregation. 3. Asking questions in natural language about telemetry data will be extremely useful—if it works well and reliably.
We’ve created a solution where you can get answers to questions like:
- Show me all logs from the last 5 traces where error X happened. - Look at requests where there was an out-of-memory exception in the last day and break them down by data center. - How much is the error rate increasing in service “MyService” when the CPU is over 70%? - What were the most common cart items in sessions where the user added items to the cart but didn’t buy?
Here’s the Obics solution in simple terms:
- Save all telemetry data in a centralized columnar database that allows real-time analytics on big data. We chose ClickHouse. - Allow ingestion from OpenTelemetry, Prometheus, cloud providers, etc., so you can keep it vendor-neutral. - Train an AI model on your telemetry data.
And here’s why we think it will work well:
1. ClickHouse uses SQL, and LLMs are very good at generating SQL. 2. Since LLMs aren’t 100% reliable, you can’t just ask questions and get answers you can fully trust. But if the LLM generates an SQL query, you can verify it, making the solution as reliable as the query itself. 3. ClickHouse doesn’t require a lot of indexing, and it integrates with cheap blob storage like S3. In other words, customers can enjoy the best of both worlds: advanced analytics capabilities and lower pricing.
That’s basically it, looking forward to your comments. You can sign up for a free trial at https://obics.io/freetrial/. We’re also looking for design partners. Thanks.
