He was talking about how deep the transformation pipeline in scalac is, compared to javac.
I would say that something close to the opposite statement is justified: databases are interpreters, e.g. sqlite is certainly an interpreter with a storage layer.
I didn't watch enough of it to see why he thinks that compilers are databases. Where's the storage layer? I'm interested in compilers and databases, but TBH I am not that interested in Scala, so I stopped watching.
It's misleading to say that compilers are databases if they're not anything like databases people use and know. Probably a more accurate title would have been "Compilers are like functional databases".
Storage is obviously an important part of database architecture. If a compiler doesn't deal with storage, there's only so much resemblance that a compiler and a database can have.
Not that compilers can be used as a general DBMS. I think.
A database is an interpreter because it has a lexer, parser, byte code compiler, optimizer, interpreter loop, etc. It also has a storage layer (i.e. the B-tree implementation, paging, etc.).
A compiler (like javac) or interpreter isn't a database because it doesn't have a storage layer.
Even a "functional database" has to have a storage layer!
- every non-trivial software is a compiler
- every non-trivial software is a database
Every non trivial software have to read/parse/manipulate/store a lot of data. I think it ultimately permit to simplify thinks to just admit that we are doing a specialized database/compiler. The fact that input data are coming from file, database, network, command line or the code itself and are going to files, database or network is just an implementation detail.
The solution is to design your software as a compiler from the start.
Too bad. It's an interesting talk on the architecture of an ambitious compiler redesign. The title is a simplification for the sake of being a keynote.
There was a lot of interesting comments yesterday on a post called "Presumption of stupidity" (https://news.ycombinator.com/item?id=10034883). This could also applied to comments: Do not assume the worse you can imagine by default.
And yes, it's indeed a very interesting talk.
Actually this is even more true than you probably think it is.
It strikes me that bringing functional programming to the JVM can / has been done simpler: Clojure being the obvious one, but F# on the CLR is another good template.
Third, Frege is a Haskell implementation for the JVM which to my mind is a much better long-term bet than Scala: it's already a tried-and-tested language, and the work involved in the translation has taken an order of magnitude less time.
It's all very well having some new great language on the JVM, but simplicity is a huge virtue if you want people to build an ecosystem around it.
> Clojure being the obvious one
Clojure, to be honest, is an example of a very convoluted, not very well thought out compiler design.
Disclaimer: I know nothing about the actual Scala compiler passes, just commenting on the number of passes as a proxy for complexity.
Scala's FP is more expressive than all of them, by a large margin. Additionally, Scala also supports an useful OO system, unlike for instance the stuff F#/OCaml/... ships.
> It's all very well having some new great language on the JVM, but simplicity is a huge virtue if you want people to build an ecosystem around it.
You realize that Scala has probably the largest, and most meaningful ecosystem of all (non-Java) JVM languages, and is driving a lot of the changes in the Java and JVM space? (Reactive streams, value types, specialied generics, ...)
I expect you're right about the expressiveness, although I don't really know exactly what you mean by this. The "buffet of abstractions" available in Scala actually put me off it. For me, there are too many options available to expect to have readable, maintainable code at the end of it all, especially if you work in a team which regularly creates almost wilfully hard-to-comprehend java.
This video is 2 years old now, but I think it marked the point when I stopped investigating scala. https://www.youtube.com/watch?v=TS1lpKBMkgg&feature=youtube_...
Yes, compilers indeed have a lot in common with databases. Datalog (and therefore a relational algebra) is a very powerful tool for implementing compiler passes, for asking questions about the AST or various IRs.
EDIT: I'm talking about things like this: http://people.csail.mit.edu/mcarbin/papers/aplas05.pdf
Good paper BTW, though I wish you hadn't prefaced the post by calling people post-literates. This paper deserves more reads.
P.S. I'm really getting tired of the ever growing proportion of video vs. text in the otherwise valuable content. I'm genuinely interested in the topic but there is no way I can watch it. Dreaming of a service which would transcribe arbitrary videos for a tiny fee.
http://openjdk.java.net/projects/mlvm/jvmlangsummit/agenda.h...
And there is an You Tube playlist for the talks already
https://www.youtube.com/playlist?list=PLX8CzqL3ArzUo2dtMurvp...
