I did this and I found that as I increased the paralellism of my tests, the genserver got bottlenecked and the test suites started throwing heisenbugs, which is not what you want. You should really be using the Mox library, which I don't believe suffers from this problem.
Echo use of the Mox library! I found testing in Elixir to be by far my favourite experience (outside of Scheme, I guess, but I haven't done production Scheme). The REPL is really nice as well for exploratory development.
