1
I'm working on an evolutionary, accumulate-only database system called SirixDB written in Java (and a module in Kotlin) in my spare time and in my day to day job I'm also working as a software engineer (embedded; besides I'm dancing a lot -- swing dances, mainly Lindy Hop and travel to festivals in Europe often times in my holidays) :-)
Furthermore, I'm working on Brackit (http://brackit.io), a query engine for JSON and XML using a JSONiq like query language. It can be used as ad-hoc in-memory query processor as the query processor of a data store. We envision, that it encapsulates common proven optimizations whereas the individual data stores can add further stages to rewrite the queries for index matching and physical optimizations.
SirixDB (https://sirix.io or https://github.com/sirixdb/sirix) is all about efficient versioning of your data. That is on the one hand it reduces the storage cost of storing a new revision during each transaction-commit while balancing read- and write-performance through a novel sliding snapshot algorithm and dynamic page compression. On the other hand Sirix supports easy query capabilities for instance to open a specific revision by a timestamp or several revisions by a given timespan, to navigate to future or past versions of nodes in the tree-structure and so on. It basically never overwrites data and is heavily inspired by ZFS and Git and borrows some ideas and puts these to test on the sub-file level.
In stark contrast to other approaches SirixDB combines copy-on-write semantics with node-level versioning and does not require a write-ahead-log for consistency.
It all started around 2006 as a university / Ph.D. project of Marc Kramis and I worked on the project since 2007 and already did my Bachelor's Thesis, Master's Thesis as well as several HiWi-Jobs on topics regarding the project and I'm still more eager than ever to put forth the idea of a versioned, analytics plattform to perform analytical tasks based on current as well as the history of the data.
e-mail address: lichtenberger.johannes@gmail.com
That said, whenever I find some time, I can work on my vision much more efficiently (mostly as a product owner + architect in one person rather than writing everything "by hand"). So, I of course wonder if our jobs are safe in the future. I think you'll always have to heavily guide the agents and stop them immediately whenever they're about to get haywire and thus you have to have the skills of a senior software engineer, but on the other hand I'm sure that small teams of senior engineers can be much more efficient than before. So, either it's that you'll need less software engineers at some point, or if it's rather that you can deliver products faster with more ideas implemented or simply that new ideas can be explored much more efficiently as before => more small startups?). I really don't know...