For developer tools config files are user interfaces and sometimes the only user interface. Therefore config should be designed for humans, not machines.
This has a few implications.
We avoid programming languages like JSON or TySON. Instead we use HOCON. It's nice but not well known outside the JVM ecosystem which is a pity. The "H" in HOCON stands for Human and the syntax has various conveniences that you might not want in a programming language or data serialization format, but which are great for config files. For example strings can be unquoted as long as they don't contain any special characters.
We avoid a strict type system. Conveyor's config has places where a key can be set to a string, or an object, or a list of strings, or a list of objects. The "canonical" form is the list of objects, but sometimes those objects can be conveniently expressed using a string shorthand, and a single string or object is treated as equivalent to a single item list containing that thing. This is highly convenient. Obviously configs become less verbose and more intuitive, but it also means the user interface will Do What You Mean™ instead of moaning at you because you forgot that, for example, this key happens to support more than one item.
In turn this means that we shouldn't directly map config to internal objects using a deserialization framework. Instead config is mapped field-at-a-time using Kotlin delegated properties. Casting is done on the fly, so for instance if internally a field is a string but you happened to specify a number then it's read as a string. The goal is to present the user with type errors as rarely as possible. In a programming language this sort of dynamic typing can yield surprises in large codebases, which is why TypeScript is useful in the first place, but it's rare that config files become as big as codebases.
HOCON has many of the same features as TySON but in a more convenient form. For example, instead of having to use an import and export statement to import and override config, you can just write:
include "other.conf"
foo = bar
or foo {
include "other.conf"
}
A big question config formats always face is how much programmability to support. On one hand, pure declarative formats are often too limiting and end up requiring workarounds like preprocessors or hacking programming languages into the config syntax. On the other hand, programming is a problem of potentially infinite scope. We solve this using hashbang includes: foo {
include "#!your-script --flags"
}
This executes the program and includes the standard output as config. This can be used in several ways:• Convert from other config formats like jsonnet.
• Run little shell/python/kotlin scripts to generate config.
• Invoke a build system task to reflect build metadata e.g. `include "#!gradlew printConfig"`
What it does is pretty intuitive when you look at it, because of the visual resemblance to the start of a UNIX shell script. The nice thing about this is that the external code can be written in anything. It also naturally supports executing tools in parallel (though we don't do this), sandboxing them, and the model could be extended to pass the config without hashbangs on stdin as JSON, so the tools can transform config instead of just generating it.
HOCON has some level of editor and IDE support, though not as much as TypeScript would have. For it to be more widely used would require compiling the library down to native code and exposing a C API for it, I think.