Spinel: Ruby AOT Native Compiler (opens in new tab)

It's a very pragmatic design: Uses Prism - parsing Ruby is almost harder than the actual translation - and generates C. Basic Ruby semantics are not all that hard to implement.

On the other extreme, I have a long-languishing, buggy, pure-Ruby AOT compiler for Ruby, and I made things massively harder for myself (on purpose) by insisting on it being written to be self-hosting, and using its own parser. It'll get there one day (maybe...).

But one of the things I learned early on from that is that you can half-ass the first 80% and a lot of Ruby code will run. The "second 80%" are largely in things Matz has omitted from this (and from Mruby), like encodings, and all kinds of fringe features (I wish Ruby would deprecate some of them - there are quite a few things in Ruby I've never, ever seen in the wild).

> eval, send, method_missing, define_method , as a non-rubyist how common are these in real-world code? And how is untyped parsing done (ie JSON ingestion?).

They are pervasive. The limitations are similar to those of mruby, though, which has its uses.

Supporting send, method_missing, and define_method is pretty easy.

Supporting eval() is a massive, massive pain, but with the giant caveat that a huge proportion of eval() use in Ruby can be statically reduced to the block version of instance_eval, which can be AOT compiled relatively easily. E.g. if you can statically determine the string eval() is called with, or can split it up, as a lot of the uses are unnecessary or workaround for relatively simple introspection that you can statically check for and handle. For my own compiler, if/when I get to a point where that is a blocking issue, that's my intended first step.

Quite a lot, that's what allows you to build something like Rails with magic sprinkled all around. I'm not 100% sure, but probably the untyped JSON ingestion example uses those.

Remove that, and you have a very compact and readable language that is less strongly typed than Crystal but less metaprogrammable than official Ruby. So I think it has quite a lot of potential but time will tell.

The interesting bunch (to me, based on experience) is `eval`, `exec`, and `define_method` (as well as creating new classes with `Class.new` `Struct.new`). My sense is that the majority of their use is at the time of application boot, while requiring files. In some ways, it is nearly a compilation step already.

This depends on the individual writing code. Some use it more than others.

I can only give my use case.

.send() I use a lot. I feel that it is simple to understand - you simply invoke a specific method here. Of course people can just use .method_name() instead (usually without the () in ruby), but sometimes you may autogenerate methods and then need to call something dynamically.

.define_method() I use sometimes, when I batch create methods. For instance I use the HTML colour names, steelblue, darkgreen and so forth, and often I then batch-generate the methods for this, e. g. via the correct RGB code. And similar use cases. But, from about 50 of my main projects in ruby, at best only ... 20 or so use it, whereas about 40 may use .send (or, both a bit lower than that).

eval() I try to avoid; in a few cases I use them or the variants. For instance, in a simple but stupid calculator, I use eval() to calculate the expression (I sanitize it before). It's not ideal but simple. I use instance_eval and class_eval more often, usually for aliases (my brain is bad so I need aliases to remember, and sometimes it helps to think properly about a problem).

method_missing I almost never use anymore. There are a few use cases when it is nice to have, but I found that whenever I would use it, the code became more complex and harder to understand, and I kind of got tired of that. So I try to avoid it. It is not always possible to avoid it, but I try to avoid it when possible.

So, to answer your second question, to me personally I would only think of .send() as very important; the others are sometimes but not that important to me. Real-world code may differ, the rails ecosystem is super-weird to me. They even came up with HashWithIndifferentAccess, and while I understand why they came up with it, it also shows a lack of UNDERSTANDING. This is a really big problem with the rails ecosystem - many rails people really did not or do not know ruby. It is strange.

"untyped parsing" I don't understand why that would ever be a problem. I guess only people whose brain is tied to types think about this as a problem. Types are not a problem to me. I know others disagree but it really is not a problem anywhere. It's interesting to see that some people can only operate when there is a type system in place. Usually in ruby you check for behaviour and capabilities, or, if you are lazy, like me, you use .is_a?() which I also do since it is so simple. I actually often prefer it over .respond_to?() as it is shorter to type. And often the checks I use are simple, e. g. "object, are you a string, hash or array" - that covers perhaps 95% of my use cases already. I would not know why types are needed here or fit in anywhere. They may give additional security (perhaps) but they are not necessary IMO.

We talk a lot about AI building programs from soup to nuts. But I think people overlook the more likely scenario. AI will turn 10x programmers into 100x programmers. Or in Matz’s case maybe 100x programmers into 500x programmers.

https://www.youtube.com/@rubykaigi4884/videos

https://en.wikipedia.org/wiki/Spinel

The problem is that people often do not have the time to refactor once they have gotten the thing to work. And the mess keeps growing.

For example infer_comparison_type() [1]. This is far from the worst offender - it's not that hard to read - but what's striking here that there is a better implementation that's so simple and obvious and Claude still fails to get there. Why not replace this with

    COMPARISON_TYPES = Set.new(["<", ">", "<=", ">=", "==", "!=", "!"])

    def infer_comparison_type(mname)
      if COMPARISON_TYPES.include?(mname)
          "bool"
      else 
        ""
      end
      # Or even better, strip the else case
      # (Which would return nil for anything not in the set)
    end

Other files have much better code quality though. For example, most of the lib directory, which seems to correspond to the ext directory in the mainline Ruby repo. The API is clearly inspired by MRI ruby, even though the implementation differs substantially. I would guess that Matz prompted Claude to mirror parts of the original API and this had a bit of a regularizing effect on the output.

[1] https://github.com/matz/spinel/blob/98d1179670e4d6486bbd1547...

But I am not sure that huge files are easy for the AI to work with. I try to restrict the files to 300 lines. My thinking is that if it's easy for a human to understand the code, it will be easy for coding agents, too.

However, send/method_missing is in common use in preexisting libraries and it shouldn't be particularly difficult to implement (via in memory lookup tables at "compile" (to c) time etc), so either they're ommitted for the reasons you say, or he just hasn't gotten around to it yet. I'm hoping the latter, but only for compatibility sake as I won't be able to use it for any real work at least in the short term otherwise.

It was having less code to read.

Matz: gem env|info and find should do

This is for a limited subset of Ruby - almost no popular Ruby gems would run under it. It's more like PreScheme [1] (ie. a subset of a language oriented at C compilation).

I don't think these compete in the same niches right now. Full Ruby almost certainly requires a JIT.

[1]: https://prescheme.org/

The revenge of Rational Unified Process, Enterprise Architect and many other tools.

Instead of UML diagrams it is markdown files.

For Ruby and c development it’s the best llm we got right now, the others lag behind by a lot sadly.

With openAI it’s so bad for my usecase it’s basically unusable.