Manifold: Java Type-Safe Metaprogramming, Structural Typing, Extension Methods (opens in new tab)

(github.com)

126 pointsjeanlucbernard6y ago43 comments

43 comments

Manifold is an amazing piece of engineering. Scott McKinney, the author, used to work on Gosu but mostly focuses on this now.

He added GraphQL support a few weeks ago:

https://github.com/manifold-systems/manifold-sample-graphql-...

You can just drop graphql files in your resources directory and start coding against them as type-safe interfaces immediately, no code-gen required.

The @Jailbreak annotation is awesome too.

mindcrime6y ago

Looks pretty cool. Reminds me a lot of the ways in which Groovy improves Java.

vorg6y ago

Apache Groovy really improved on Beanshell (by adding closures, in 2003), which is what improved on Java (by adding dynamic typing, in late 1990's).

brabel6y ago

I thought I knew all about Java, but this thing makes me feel like I haven't got a clue! How can it achieve this stuff by just being in the classpath?? No Java agent can be involved, no codegen during the build?! This shouldn't be possible, or??

brabel6y ago

Ok just checked this deeper. The [docs](http://manifold.systems/docs.html) actually show this is a javac plugin (I didn't know about the `-Xplugin` flag, but you ALSO need to pass that to javac for Manifold to work)... you also need the `tools.jar` file (which includes the JavaCompiler) on the runtime classpath... so suddenly, there's no magic... it's a bit misleading to say this does not require a build step, it clearly does, but as it integrates with javac itself, it "feels" seamless...

mckinney6y ago

Note tools.jar is needed only for Java 8, and only for compile-time, unless you use Manifold's dynamic loading feature. The JREs for Java 9+ bundle what used to be called tools.jar.

the84726y ago

How much of an update hazard is this? Does it break with every major java upgrade?

mckinney6y ago

There's a significant difference between a code gen build step and what Manifold is doing with javac. Resource files are virtual source files, this represents a leap in terms of dev productivity.

1 more reply

mckinney6y ago

See my earlier comment re javac plugin API, Manifold uses that as a means to hook into the compiler.

Probably the most interesting aspect of Manifold is its role as a type name resolver for the compiler. Manifold provides an API to dynamically resolve types corresponding with, well anything, but mostly with resources like JSON files, GraphQL schemas, SQL, DDL, CSV, scripting languages, you name it. So from the compiler's perspective it has no idea what is going on beyond normal type name resolution, but the source files it's loading up are coming from deep, dark places ;)

But yeah, no code generation step in your build, no agents, no custom class loaders, etc.

eatonphil6y ago

If you like this, check out Project Lombok [0] as well. It gives you var/val for local type-inferenced variables, getter/setter code generation, and others to help eliminate boilerplate.

[0] https://projectlombok.org

rb8086y ago

Its interesting with the other thread about Java EE dying and Oracle politics. The ecosystem around Java and JVM is huge, so much cool stuff going on. Definitely a great platform.

stewbrew6y ago

How is the user experience with that if you don't use Intellij? It seems to require explicit support by the IDE, which is why it doesn't seem to work with eclipse.

mckinney6y ago

(Hi, I'm the author of Manifold) Manifold works equally well with Java 8 - 12. While no IDE is required, it is designed to fully leverage Java's static type system, which heavily benefits from the static analysis provided by a topnotch IDE like IntelliJ IDEA. Code completion, deterministic refactoring, usage searching, feature navigation, incremental compilation, etc. -- suddenly these productive features are now available to any resource connected with Manifold e.g, GraphQL, JSON Schema, Javascript, etc.

Re Intellij v. Eclipse. I'm just one guy banging on a project; I only have time to build and maintain one IDE plugin and right now my focus is on IntelliJ. I'd like to start working on an Eclipse plugin at some point... or trick someone else into doing the work ;)

orbifold6y ago

First they came for the particles (electron, gluon, atom) now they are coming for mathematical constructs and give them mundane everyday meaning .

iSnow6y ago

Or car parts, who can tell...

isharamet6y ago

FYI: Manifold is a pretty popular Clojure library by Zach Tellman.

https://github.com/ztellman/manifold

iSnow6y ago

Wow, this looks like a god-send. The @Jailbreak annotation would give mid-2000's Java coders a heart-attack :)

The only contentious point I find is that they are basically replicating Project Lombok's @SneakyThrows annotation via "Xplugin:Manifold stringsexceptions" or ModelMapper in their Structural Typing.

Furthermore it seems it should be structured in sub-projects, eg. the String-templates and ManTL in one lib, Structural Typing in another so we can mix & match.

But overall, very impressive!

mckinney6y ago

Thanks!

The 'exceptions' option is quite a bit different from SneakyThrows. The 'exceptions' option completely mutes checked exceptions in your project -- you don't have to declare a throws clause or catch a checked exception anywhere in your code, thus with the option enabled checked exceptions behave exactly as unchecked exceptions. This behavior is identical to exception treatment in other JVM languages such as Scala, Kotlin, Ceylon, etc.

Regarding sub-projects on extensions, I've been meaning to do that. The project is already divided into separate targets e.g., manifold-graphql, manifold-json, etc. But manifold-ext has probably grown too big for one target.

iSnow6y ago

My god, I just read a bit more about the Extension Manifold, and that is crazy good. I am doing both Java and JS, and I started to like duck-typing and extending prototypes so much in some situations. Now, Extensions give me something similar in Java?

I have the highest respect for your work, and no idea how you are doing that more or less on your own. I am just a bit sad your upcoming SQL/DDL extensions are going to kill one of my side-projects :) https://github.com/iSnow/sqlWebservice - if you can use a SQL schema as a type provider, you are leaps and bounds ahead of what I am doing there.

So, hats off, great job.

sisu20196y ago

This is what I wanted all my life! I wonder what kind of dark magic makes this go and how this will come back to bite me if I use it?

mckinney6y ago

Manifold hooks into the Java compiler via the javac plugin API. This is similar to the way annotation processors work, but without the overhead of additional passes (or rounds) in the compiler. The plugin route also provides access to earlier stages of the compiler, for example this is how Manifold's string interpolation feature works. The overall details are a bit deep and dark for a comment, however ;)

brabel6y ago

I've played with the compiler's API before and found it incredibly difficult to work with... and completely undocumented... how did you manage to get through that? Great job!

1 more reply

der_Fidelis6y ago

At that point, why not just use another jvm language with these features like Kotlin?

_old_dude_6y ago

Scott McKinney, the guy behind that library, is also the guy behind Gosu [1].

Selling a new language is hard, selling a library is easier, even if this one has a hook deep in the code of javac.

[1] https://en.wikipedia.org/wiki/Gosu_(programming_language)

sisu20196y ago

the main feature of this allows you to transparently use structured data as types in your code like type providers in .net. I love kotlin but it can't do anything remotely like that.

jillesvangurp6y ago

Yep, I'm wondering if this actually works in Kotlin as well. I'm thinking it may probably play nice with this as is since Kotlin can interact with Java classes. However, the compiler plugin might be a problem. If so, this would be an awesome thing to add to Kotlin.

iSnow6y ago

Ever tried to "sell" an alternative language in BigCo Enterprise? Where I work, Javascript is still a front-end only language and just tolerated as a toolchain for compiling LESS, ECMAScript etc.

based26y ago

or Scala https://www.scala-lang.org/

marcinzm6y ago

Scala comes with a lot of overhead in terms of features (and breaking changes in versions) and is still missing the meta-programming aspect of this library. In fact, scala has now deprecated all it's previous attempts at meta-programming and is still trying to work out it's next attempt (which is I believe black-box only so can't do what this library does).

james_s_tayler6y ago

at that point why not use C#?

tigershark6y ago

C# doesn’t support type providers, only f# supports them.

kitd6y ago

This feels a bit like Project Lombok: adding cool features to standard Java.

I like the Extensions option. It works very much like D's UFCS mechanism.

chii6y ago

Removing checked exceptions - what a dangerous feature!

mckinney6y ago

It's optional. It seems scary, but checked exceptions are a feature exclusive to the Java compiler -- the JVM does not enforce them. This is how JVM languages like Kotlin and others only use unchecked exceptions, and also how Manifold gets away with muting them as a compile-time feature.

_old_dude_6y ago

It should not be optional :)

Checked exceptions is a stupid feature because it makes an implementation detail, the fact that a code raises an exception, part of the type of a method.

Scala, C# or Kotlin have no checked exception and it doesn't make them less safe.

2 more replies

ilitirit6y ago

See the section on Lambdas:

http://manifold.systems/docs.html#checked-exception-suppress...

j / k navigate · click thread line to collapse

43 comments

throwaway_x13zd6y ago

Manifold is an amazing piece of engineering. Scott McKinney, the author, used to work on Gosu but mostly focuses on this now.

He added GraphQL support a few weeks ago:

https://github.com/manifold-systems/manifold-sample-graphql-...

You can just drop graphql files in your resources directory and start coding against them as type-safe interfaces immediately, no code-gen required.

The @Jailbreak annotation is awesome too.

mindcrime6y ago

Looks pretty cool. Reminds me a lot of the ways in which Groovy improves Java.

vorg6y ago

Apache Groovy really improved on Beanshell (by adding closures, in 2003), which is what improved on Java (by adding dynamic typing, in late 1990's).

brabel6y ago

mckinney6y ago

Note tools.jar is needed only for Java 8, and only for compile-time, unless you use Manifold's dynamic loading feature. The JREs for Java 9+ bundle what used to be called tools.jar.

the84726y ago

How much of an update hazard is this? Does it break with every major java upgrade?

mckinney6y ago

There's a significant difference between a code gen build step and what Manifold is doing with javac. Resource files are virtual source files, this represents a leap in terms of dev productivity.

1 more reply

mckinney6y ago

See my earlier comment re javac plugin API, Manifold uses that as a means to hook into the compiler.

But yeah, no code generation step in your build, no agents, no custom class loaders, etc.

eatonphil6y ago

If you like this, check out Project Lombok [0] as well. It gives you var/val for local type-inferenced variables, getter/setter code generation, and others to help eliminate boilerplate.

[0] https://projectlombok.org

rb8086y ago

Its interesting with the other thread about Java EE dying and Oracle politics. The ecosystem around Java and JVM is huge, so much cool stuff going on. Definitely a great platform.

stewbrew6y ago

How is the user experience with that if you don't use Intellij? It seems to require explicit support by the IDE, which is why it doesn't seem to work with eclipse.

mckinney6y ago

orbifold6y ago

First they came for the particles (electron, gluon, atom) now they are coming for mathematical constructs and give them mundane everyday meaning .

iSnow6y ago

Or car parts, who can tell...

isharamet6y ago

FYI: Manifold is a pretty popular Clojure library by Zach Tellman.

https://github.com/ztellman/manifold

iSnow6y ago

Wow, this looks like a god-send. The @Jailbreak annotation would give mid-2000's Java coders a heart-attack :)

The only contentious point I find is that they are basically replicating Project Lombok's @SneakyThrows annotation via "Xplugin:Manifold stringsexceptions" or ModelMapper in their Structural Typing.

Furthermore it seems it should be structured in sub-projects, eg. the String-templates and ManTL in one lib, Structural Typing in another so we can mix & match.

But overall, very impressive!

mckinney6y ago

Thanks!

iSnow6y ago

So, hats off, great job.

sisu20196y ago

This is what I wanted all my life! I wonder what kind of dark magic makes this go and how this will come back to bite me if I use it?

mckinney6y ago

brabel6y ago

I've played with the compiler's API before and found it incredibly difficult to work with... and completely undocumented... how did you manage to get through that? Great job!

1 more reply

der_Fidelis6y ago

At that point, why not just use another jvm language with these features like Kotlin?

_old_dude_6y ago

Scott McKinney, the guy behind that library, is also the guy behind Gosu [1].

Selling a new language is hard, selling a library is easier, even if this one has a hook deep in the code of javac.

[1] https://en.wikipedia.org/wiki/Gosu_(programming_language)

sisu20196y ago

the main feature of this allows you to transparently use structured data as types in your code like type providers in .net. I love kotlin but it can't do anything remotely like that.

jillesvangurp6y ago

iSnow6y ago

Ever tried to "sell" an alternative language in BigCo Enterprise? Where I work, Javascript is still a front-end only language and just tolerated as a toolchain for compiling LESS, ECMAScript etc.

based26y ago

or Scala https://www.scala-lang.org/

marcinzm6y ago

james_s_tayler6y ago

at that point why not use C#?

tigershark6y ago

C# doesn’t support type providers, only f# supports them.

kitd6y ago

This feels a bit like Project Lombok: adding cool features to standard Java.

I like the Extensions option. It works very much like D's UFCS mechanism.

chii6y ago

Removing checked exceptions - what a dangerous feature!

mckinney6y ago

_old_dude_6y ago

It should not be optional :)

Checked exceptions is a stupid feature because it makes an implementation detail, the fact that a code raises an exception, part of the type of a method.

Scala, C# or Kotlin have no checked exception and it doesn't make them less safe.

2 more replies

ilitirit6y ago

See the section on Lambdas:

http://manifold.systems/docs.html#checked-exception-suppress...

j / k navigate · click thread line to collapse