Ubiq: A Scalable and Fault-Tolerant Log Processing Infrastructure (2016) (opens in new tab)

Let me take this opportunity to say I'm appalled all research at Google has been rolled into "AI." Ubiq has nothing to do with machine learning, but you can't access it without wading through a swath of Google-branded AI marketing material. The domain research.google.com now redirects to ai.google. If you want to search for any research publications by Google employees, you will be searching on a domain that first and foremost highlights the AI research teams. In a particularly egregious example, all quantum computing research has been filed under "Quantum AI." The "Recent Publications" section exclusively highlights machine learning research despite there being more recent research by other teams. I can go on.

In my opinion, this move threatens the scientific integrity of Google's research. It's clear that researchers in distributed systems, fundamental theory, security and privacy, networking, language development, etc. are second class citizens. I wouldn't be surprised to learn that there's organizational pressure to inject machine learning into as many publications as possible, even if it dilutes the overall diversity of research.

> It also guarantees exactly-once semantics for application pipelines to process logs in the form of event bundles.

Well, it's got my attention.

huh...I interviewed with Google about 4-5 years ago and one of the interviewers spent a bunch of time asking me questions about something just like this, but from a "theoretical" perspective.

Pretty interesting to see this actually happen and see where Google ended up with it.

I wonder if the name is a nod to Ubik by Philip K. Dick

Published in 2016

The most scalable, fastest reaction time and simplest log processing tool is awk.