Elixir in Depth – Reading and personal notes (opens in new tab)

I learned ruby on a big project about...5 years ago. I've been using it ever since at work and I definitely understand the pain points. With Ruby you really have to learn to understand and appreciate where its strengths are to be productive with it (how close it can bring you to Aspect Orient Programming is probably the biggest).

The only similarity that Elixir honestly has with Ruby is clean syntax and general emphasis on productivity. Everything else is different. Same with Phoenix. On the web side of things it has routes, controllers and views...but there's no magic involved. It's all very clear, explicit and flexible in how it does things. The view layer is honestly one of it's biggest strong points because the way it's handled is the core reason for all of those "My site was faster without caching" articles...but you don't have to work differently to enjoy those gains.

Big Nerd Ranch did a solid write up on the ins and outs of why.

https://www.bignerdranch.com/blog/elixir-and-io-lists-part-2...

This Choosing Elixir for the Code post that came out last month does a really good job of emphasizing a lot of it.

https://pragtob.wordpress.com/2017/07/26/choosing-elixir-for...

I came to Elixir from Clojure and was not a big fan of Elixir's syntax, but I've found that after a while it mostly fades into the background (much in the same way that Clojure's parens do).

The language and Erlang's virtual machine are generally a joy to work with, and I have no regrets pushing past the syntax.

I also come from Clojure and Erlang and, tbh, couldn't get into Elixir for various reasons (I've followed the development closely since its inception):

* Most of the Elixir community comes from the Rails world, so most of the libraries focus on the web.

* Coming from the Ruby world, a lot of these new developers tend to be ignorant of the underlying Erlang machinery, believing that Elixir is the magic that makes Erlang a "modern" and usable language, which is simply not true.

* The tendency to favor frameworks over libraries.

* I'm probably alone in this one, but I find the syntax off-putting. It adds lots of sigils and complexity to Erlang's very succinct syntax.

* In the end, I think Elixir adds very little besides the excellent tooling to the Erlang ecosystem.

If you're not fond of the syntax (and I don't blame you at all, I'm no fan) then you should give Erlang a try.

Elixir seems much more like Clojure without the parens to me than Ruby. (Yes, I know Clojure can actually use fewer parens than the equivalent Java, just referring to the Lispy syntax in general.)

Dynamically typed, functional language with immutable data structures and a strong philosophy about how to best handle concurrency and scaling.

The key difference is JVM vs. Erlang VM as the base and all that implies.

It's a functional language! And it's based off of Erlang, so if you like Erlang, you can use call Erlang directly in Elixir. And if you like piping things / pattern matching (common in F#), I think you'd enjoy it :)

First of all, Elixir's relationship to Ruby is only skin deep. The languages aren't as similar as some like to suggest, in my opinion.

Personally, I like Elixir better than Clojure because memory in the BEAM VM is isolated. The JVM uses shared memory, which has caused me issues before. I also like the tooling and build tools for Elixir a lot better. Clojure is weak in this area, I think. I like Elixir's syntax much better too. For me, it feels a lot cleaner and easier to type, but that is subjective.

Regarding Rails vs. Phoenix, I cannot really speak to that. I don't do monolithic apps anymore, so I don't use Rails or Phoenix. Elixir has a library called Plug built into it that lets me easily make REST APIs for micro-services, so that is what I typically use. For me, this eliminates a lot of the issues I had with Rails.

You should hear Chris McCord (Phoenix author) talk sometime -- he is extremely wary of sprawling design in a way that might please you.

I came to Elixir from Scala but also having a Rails background. I was quite sick of Railsy magic, because I'd gone deep with the Ruby metacrap and I got to taste the other edge of the sword. Elixir was a breath of fresh air.

If you enjoy concurrent programming with nice syntax and few surprises, you might end up liking it. Elixir is basically Erlang with modern syntax and vastly improved tooling.

I disliked Rails and did not get into Ruby but I love Elixir. YMMV

The syntax is bad (just like ruby), and there are some annoying limitations with macros, but the semantics are very good, and the ability to keep thousands (or hundreds of thousands, or even millions if you know what you're doing) of stateful connections to clients is very useful in practice.

I’m a Python guy who strongly dislikes Rails, feels meh about Ruby, yet loves Elixir (and likes Phoenix).

You can look into drab, an elixir server-side DOM manipulation library: https://github.com/grych/drab. I've never used it, but many people in the Elixir community think it holds a lot of potential.

I think one of the difficulties here is that much of the functionality that comes from, for example, Docker + Kubernetes exists as part of the BEAM VM. A lot of the development being done today in deployment is around Docker and Docker clustering technology, which means that it often feels like you are doing things the old fashioned way when using BEAM clustering & networking technology.

There are ways to set up these systems in symbiotic configurations, like having an Elixir/Erlang/BEAM cluster that has auto-joining thanks to external service discovery. However, to use the hot-swap functionality, you are going to be working with the BEAM VM directly, which means going through the Docker container boundary.

Know that if you choose to go with BEAM and no Docker, you will still have access to amazing deployment features (like hot swapping, as you mentioned), but also clustering, service discovery, monitoring, etc. You're just going to be spending a lot more time at the command line without the pretty interfaces and the sleek modern tools.

To get a better understanding of running BEAM, I highly recommend "Designing for Scalability with Erlang/OTP". You'll have to learn Erlang to get through it, but IMO it's the best introduction.

I've been running an Elixir app in production on about 20 nodes for 18 months. We just build Debian packages and deploy using Salt.

Additionally, most of the cluster nodes run on Spot Instances in AWS so they are relatively inexpensive also. When a new instance comes up it connects to the cluster and starts serving requests. When an instance is killed, traffic is routed to the remaining nodes. Works great.

My first ever docker build was a multi-stage elixir compilation. The end result is a docker image that can be deployed to Heroku or k8s that works great. It's not that hard to get a good docker setup for it. It's even easier if you use Mix and not releases (like I did). However, releases are cool.

Code hot swapping is not really docker friendly (obviously). I haven't done that, but work's general philosophy has been to have recoverable processes such that starting up the app will bring up the correct processes loaded with the right data. However, it's definitely possible to setup hot swapping but is going to be a learning curve to it.

One new service that tries to make Elixir deployment easier is https://gigalixir.com. If Heroku was built with only Elixir in mind, this would be it. Disclaimer: I'm the founder.

It's not very hard to deploy. Just use the proper tools to compile and build a release (on a VM emulating the same architecture as your server), rsync the resulting files into the server and strat the app.

Getting hot code reload to work is harder, but for a "normal" basic deployment it's just this. App restarts take less than one second of downtime for my app, and that's a cost I'm willing to pay.

You shouldn't do this by hand of course. Write an Ansible playbook or something else that does this for you

There are already recipes for deploying to Heroku, and I've had no problem using Elixir and Phoenix apps on Heroku for the past 2 years. If you use EngineYard or other bare-metal solutions like AWS or Google Cloud, I would imagine there's something you can glean from that.

I actually think phoenixframework.org is pretty robust on this at the moment, and using the #elixir chat in Slack is also awesome.

Um...you know this is basically threads right?

(not disagreeing, but having watched the past few years as "threads bad, don't know why though..." it is amusing to see thread-style concurrency rediscovered with a new name).

What do Promises/Futures/Async/etc. have to do with monads? Are monads a good abstraction to model those things?

Not just scheduling — the ability to forcibly terminate processes itself is a key part of why Erlang can be resilient to failures.

It's a key area where I think Go went wrong. Without the ability to kill a goroutine, you can't implement caller-enforced timeouts and canceling — anything that should be able to be canceled has to explicitly check for some signal sent by the controlling party. This is why Go ended up with Context, which infects everything it touched.

And you also can't create Erlang-style supervisor trees; you can't create self-repairing hiearchies of control. And you can't do things like use a debugger to attach to a process and terminate runaway goroutines.

You may have been able to bump the process priority to give that process more scheduler time if it's a critical process in a hot path. Perhaps you tried that though. Definitely have to be careful with it, but always an option to keep in mind!

Do you mind me asking what kind of throughput you are seeing for a process like that where you unpack video across some processes? I'm working on something similar using Go and it'd be great to know if Elixir is a good alternative or not.

Pre-emptive scheduling is indeed the approach used by most operating systems (those of us who are bit older will remember when the most common version of Windows was co-operative scheduling - that was fun, any single application misbehaving could hold all the resources on the machine and that happened quite a lot!).

The genius of Erlang is that instead of a per machine it was designed to be distributed from the start so you have a lot less inter dependencies when you want to scale across machines (or cores since that's where Erlang/Elixir shines at the moment), that was one of the design goals since it was designed for the kinds of software where that was a desirable quality (back when hardware was way slower than now), dropping a single call/circuit wasn't the end of the world if the rest stayed alive.

Lifted from the wikipedia article :-

    Everything is a process.
    Processes are strongly isolated.
    Process creation and destruction is a lightweight operation.
    Message passing is the only way for processes to interact.
    Processes have unique names.
    If you know the name of a process you can send it a message.
    Processes share no resources.
    Error handling is non-local.
    Processes do what they are supposed to do or fail.

I think when you get down to it there isn't really anything you can do in Erlang you couldn't do in another language, at one time Erlang went through C as a build step, it's that it was a fairly radical departure as a solution to a problem at the time (telephony) that also happens to fit a fair few problems we run into on the internet.

I'm not an Erlang/Elixir programmer (so I've no dog in the fight) but I've been fascinated by the language since I read about it in the early 90's in a programming journal, it seemed alien then and it seems alien now (much like APL).

Amongst my programmer friends Elixir seems to be really popular with Ruby programmers (I'm sure there is a reason but I'm not a Ruby programmer either so I couldn't tell you).

For databases, you can provision one through Gigalixir, or you can provision one yourself with Google Cloud SQL in the us-central1 region.

For other apps, anything running in GCP us-central1 will be in the same high latency network, but some customers interop with AWS or Heroku just fine albeit with slightly higher latencies.

We'll be waiting for you!

Right now, almost everything is done from the command-line interface. We're building a web interface, but we wanted to be command-line first so it integrates with scripts and tools.

Thanks, what about security issues? Do you use some firewall with your Erlang deployments?

You have persistent processes which you can send and receive messages from which can be treated like a data structure store, as well as ETS if you need more advanced querying and Mnesia if you need embedded persistence.

I'm not sure how these compare to Redis in terms of performance/latency but Mongo is more of a "proper" database than any of these.

We definitely use databases such as Postgres, MySQL, Riak, etc. It is just that when it comes to ephemeral data, the tools that ship with the Elixir and OTP may be enough. See how Phoenix does the whole pubsub and presence without Redis or similar.

Seconded. I've been trying to find good write ups on production deployment best practices, especially people who have used it commercially for over a year.

The only reason I say that is that every project doesn't need hot-reloading. It's one of those things that is only recommended where you need it.

Just think of it in the same way that you don't need every available jar file for a Java project. The capabilities of the system are expansive, but not always called for.

Distillery is the main deployment related library out there for Elixir and reduces hot reloads to a an argument on the build.

https://github.com/bitwalker/distillery/blob/master/docs/Wal...

Thanks! Much appreciated.