Considerations when building embedded databases (opens in new tab)

Since it’s not clear from the title: embedded here does not mean sqlite-inside-your-app but small devices with microcontrollers

> However, if you compile on an AWS EC2 instance (as you may want to do if there is a cloud component to your fishtank), then you get a 16-byte structure because int there is 8 bytes

I'm not familiar with any OS that EC2 would offer that uses an 8-byte int. A better example might be that long is the size of a pointer on Unix and microcontroller systems, but always 4 bytes on Windows.

> What we learned is that, unless you need to support some very exotic big-endian architecture, you’re better off ignoring endianness and just letting all your structs be little-endian. This greatly simplifies maintainability, reduces confusion for developers unfamiliar with byte-order, saves CPU time, and allows working with constant record data without RAM copies to reverse the byte order before write to flash.

I was hoping to read this. Thank goodness. Although I wonder what embedded systems dev isn’t familiar with endianness? The overlap between network programmers and ES must be close to a circle, for one. Regardless, “just keep it LE” is a good move.

The chosen solution isn’t ideal for all cases, especially those with many or large records. It doesn’t require parsing all the bytes of a payload but it does require reading them (because it’s a sequential scan of all content).

A better solution would separate the keys from the values (à la MFT) because reading a page of flash is going to be the slowest step. If you only have to sequentially scan the header table, you have to read an order of magnitude or more less data to find the record you are searching for.