Author here--from discussions with Capela's team, I think this sort of early testing can be remarkably helpful, because it offers a test suite that Capela's team can check their work against as they move forward.

I would suggest against this kind of integration test when the data model or API are in constant flux, because then you have to re-write or even re-design the test as the API changes. Small changes--adding fields or features, changing HTTP paths or renaming fields--are generally easy to keep up with, but if there were, say, a redesign that removed core operations, or changed the fundamental semantics, it might require extensive changes to the test suite.

We thought a lot about this, and decided to not wait since we are a pretty small team and having more hands helping us to catch any problems early on would help us to make better technical decisions as we continue evolving the core platform. In addition to that, we gained a pretty robust CI step to keep us accountable around the guarantees that we want to provide. Reliably and consistently storing data is of utmost importance for us.

The plan is to engage with Jepsen again once we have a system that passes the current suite, expand the test surface even further, and continue iterating until we are satisfied with the results. There won't be a public release before that is true.

Working with Jepsen also sparked a couple other interesting ideas, like building a Python language fuzzer to ensure that many shapes of Python programs work as intended in Capela. That's something we would love to do in the future.

Re: Antithesis - absolutely. Kyle mentioned them to us and we think it will be a very interesting product for us to adopt to further ensure we're delivering a reliable product.