undefined | Better HN

0 pointsLazare5y ago0 comments

Right, your initial post said:

> ULIDs are sortable (time component), short (26 chars) and nearly human readable, and good enough entropy/randomness for everything I'd ever be working on.

1) Being roughly sortable is a valuable characteristic if used as a DB index in most DB systems, and ULIDs meet this requirement (as do others). However, they are NOT strictly time sortable except under very specific conditions. If you need that, you should avoid ULIDs (because the design is broken). But if you don't, then...well, ULIDs will work fine for you! Of course, so would other options. :)

2) 26 characters is fine. However, the use of base32 is a bit unfortunate. Some implementations generate lowercase IDs; others generate uppercase. The spec officially recommends generating uppercase IDs, but if you're not careful (and are generating them on different platforms), you may end up with a mix of both, and naive lexical sorting will yield incorrect results. You need to normalise all your IDs; doable but overhead. Conversely, if a denser encoding was chosen (like base62 or base58), you could either get the same entropy in a shorter string, or more entropy in the same string, but ALSO not need to worry about case. Win/win, from my point of view.

3) Human readable...eh, not sure "01ARZ3NDEKTSV4RRFFQ69G5FAV" is very human readable, but okay. I'd score it no better or worse than a normal UUID (like "fe9c8a21-61c1-44dd-b552-95616d62404d") or a KSUID (like "0ujssxh0cECutqzMgbtXSGnjorm").

4) ULIDs are fine in terms of entropy. Again, this doesn't set them apart from others.

Apart from the question of being strictly sortable (if you need that, you need a different tech), ULIDs are fine. But they're also not unique. Some people re-order a v1 UUID so it starts with a timestamp, and obtain the same benefits. Others use KSUIDs and obtain the same benefits. Others roll their own "slam a timestamp and some random data together and base whatever encode it" and...obtain the same benefits.

It's cool to have an ID which is roughly sortable by time, is safe to generate on multiple systems, and can be encoded as a relatively short alpha-numeric string, and I think a lot of people can benefit from this. Certainly I've found uses for this, which is why I've opted to use KSUID.

...that being said, ULID also works for this. It's just...odd. Like, 80 bits of randomness per millisecond is fine! But actually, if you follow the spec and generate them on a single system, only the first ULID is random; the rest just increment until you run out of room, which means you can generate a random number of ULIDs per millisecond. Anywhere between a bit over one septillion and one. Of course, the odds of getting a small amount of room is low, so as a practical matter it's fine. But it's odd that, if you ever did try and generate several ULIDs at once, there's just a random chance it might not work! Of course, you can just wait a millisecond and try again, so no big deal.

Of course, that opens another issue. If you tell me a ULID, then if you're following the spec, and also generating more than one ULID per millisecond, then if I have one ULID I now know the other ULIDs! For many people a major attraction of IDs like this is that they're unguessable even if you have seen a bunch, but a ULID is actually, in theory, open to enumeration attacks. That's abasolutely crazy! If you managed to make a system you don't control generate a ULID for you at the same millisecond it generates one for your target, you can now trivially guess the target's ULID. If you were relying on that ULID being unguessable for security reasons (which you would be if you were using it as an API key, session identifier, or unguessable URL), then it's completely compromised. I admit, hard attack to pull off, but wow, imagine it even being possible to do that!

Of course, in the real world, it's fine. I don't like base32, but it's good enough. The crazy increment logic is a wtf, but whatever. ULIDs are most broken when you use them for what they're designed for (guaranteeing strict monotonicity at scale), but the design is so impractical you'll give up before you hit the landmines. And if used more "normally", then they'll be good enough. I still wouldn't choose ULIDs for a project I control, just because it's so odd. But I doubt you'd gain any benefit from switching to, eg, KSUIDs now.

0 comments

Vanderson5y ago

Wow, thanks this was more than I expected. Here's my considerations on your reply, though it's likely not useful for you, but it is good for me to write this out.

1) DB performance has as much (or more) todo with time sorting as anything. But I am not an expert, I just rely on what I read from others. (hence, my questions to you, and I appreciate your time/effort on your responses here)

But, I do appreciate the inherit use of a time part, just makes me feel better about having a "randomish" id, but not loosing what I felt I had previously with incremented integers.

2) This is one reason I _prefer_ ULID to other options. I am using ULIDs in low count generated IDs per second, and I want to use in them in URLs. All lowercase is a trivial thing to implement. And I remove ambiguity about content (ie, 1,i,I,L, etc...)

So for human readable ULID is almost unparalleled from what I have found.

Is there another UID system that is just as human readable? I'd love to find something else to compare it to. (that I haven't already)

3) #2 dove-tails into #3 I guess, but yes, I look at data all day long, and the shorter the better. The fewer conflicts, misunderstandings, etc... the more human readable it is.

Is there another UID that is _more_ human readable?

4) I don't need my UIDs to be special, I just need them to be useful.

Why is something "odd" to you because of an arbitrary number like "80 bits"? Am I missing something about unique ids that require a specific hex/octet/base-10 value or they don't work properly?

As far as I know _all_ unique ids have the potential for conflicts, there is no getting around this.

A) I don't have any issues with making more than one ULID per millisecond, I spend way more CPU time on other things, I have no issue holding back ULID generation to avoid the "countable" error of multiple ULIDs that can be guessed.

Thank you for pointing out this flaw, it would be nice if it didn't exist, and maybe the libraries should have some kind of flag/option to enable guaranteed random values at the cost of speed. (which considering how long it takes to download very large background video on a splash/hero banner on a site, I think solves what seems to be 100% of the criticisms you have against ULIDs.

B) Here's a discussion about ULID vs KSUID, which includes replies from the author you linked to from Segment, Rick Branson.

https://github.com/segmentio/ksuid/issues/8

Rick says this about ULID vs KSUID:

"If one is concerned about the extra 4 bytes of data or if millisecond precision is needed, then ULID is probably a better choice."

It seems that ULID has some advantages over KSUID, and it's not so cut and dry that KSUID is superior in all ways.

j / k navigate · click thread line to collapse