Amazon Time Sync Service (opens in new tab)

The combination of a time and guaranteed error bound can be used as a primitive in distributed systems. You can get timestamps from requests coming from different systems and relate them in a meaningful way. At least, in a false negative fashion. E.g., if A occurs at T + 100ms with 1ms uncertainty and B occurs at T + 105ms with 2ms uncertainty you can conclude that A happened before B. This does not allow you to always establish an order (e.g., if there is overlap) but this isn't always necessary. For instance if instead you are B and you want to have something occur after some time A with uncertainty A_uncertainty, then you can sleep until B with uncertainty B_uncertainty such that B - B_uncertainty > A + A_uncertainty.

A person with two watches never knows what time it is.

Time is hard.

By asking both super-microscopic stuff and stuff way out in space, you can now find out what time it is!

Yeah, I was thinking about this too. Without a real-time OS on separate hardware, and/or some kind of guaranteed "will not context-switch out of your process for more than X time" + strict coding habits, this seems like at best a practical (>?)99% improvement and at worst snake-oil. Certainly not reliable in any case.

edit: well, after reading Spivak's comment[1] a bit, I guess it does provide strict upper bounds. May be useful to reduce how often you need to use fallback behaviors / get more byzantine. Though I'm not yet sure how to turn that into something useful. No doubt there are some though.

[1] https://news.ycombinator.com/item?id=29103093

Is that really the API? I'm sure I'd be writing:

  lowerBound, upperBound = readTime()

often without noticing my error.

I mean it’s not that hard even on non-rt preemptive schedulers. You can’t avoid false negatives because you could be preempted after func completes but before the time is fetched but if you get a result it will be valid.

    def must_complete_before(func, deadline):
        result = func()
        lower, upper = time.now()
        if upper < deadline:
            return result
        else:
            throw MissedDeadlineException(“Can’t guarantee func completed before deadline”)

You can use rseq on linux to avoid this.

To be fair your example would be unsafe with and kind of time API.

The APIs provided by TrueTime and Time Sync are useful to compare two events, each with their own uncertainty intervals. Then you can be sure if any event "happened before" the other, or if they're concurrent.

How about just

    syncTime()

And they take care of the mess

It's rare to see a comment on HN that misunderstands basic distributed systems concepts. MapReduce the paper has nothing to do with a database. You're likely conflating the fact that to achieve fault tolerant distributed computation, hadoop and hadoop like systems use a database like filesystem.

However, no one is looking at Map Reduce type jobs as a replacement for a database and vice versa. That's like saying "wow linkedin made kafka why do we need a webserver too". Those two technologies are only related in the loosest sense.

Those aren't really equivalent, are they? Hadoop is for analytics, spanner is transactional.

In terms of popular nosql vs google sql products, it's more Hadoop : BigQuery :: Mongo? : Spanner

You're pretty explicitly not supposed to run OLAP queries on spanner.

Google uses map reduce extensively... where it's appropriate.

True time helps with things like spanner transactions. It's just a totally different use case.

amazing. you literally got everything wrong.

MapReduce predates TrueTime by a decade or more. MR was critical to scaling internet systems at the time it was released.

However, Flume + Spanner was a much nicer system to work with than MR + GFS, I'll give you that.

Amazon time sync likely also requires extremely accurate clocks on the system board as well to prevent there from being too much drift.

Them providing PTP would be really interesting

Not quite: Truetime has guarantees about ordering, rather than accuracy.

If you ask for the time and get A, and then ask for the time again and get B, then Truetime guarantees that A is less than B.

Obviously, in a distributed setting this is much easier to do if you have accurately sync'ed clocks, but that accuracy goes to reducing the uncertainty in the time (and hence making truetime faster) rather than providing accuracy.

Sorry if it's a bit sensationalized. I wanted to give some color as to why this release might be interesting and took this tweet I saw at face value https://twitter.com/rbranson/status/1455923426359578631

so... GPS ?

I'm pretty sure if you use the GCP dataflow stuff, the dataflow worker VM's will be given access to an RPC server which has the necessary endpoints for TrueTime.

Obviously it's all a matter of reverse engineering rather than documented API's.