I do agree, the JVM is a solid language choice. I also don't think going with too esoteric of a language is a good thing. Probably a little early to bet a company on Idris or Ceylon.
Still, I apologize for my tone. Let me give you purely anecdotal information about me, so you can at least see where I'm coming from with regards to my anti-java stance.
After doing java/C# for many many years, I will not do it again. I'm much more productive in Scala/F#/ML, it's more pleasant to use, and I believe the average programmer using those languages ends up being a different caliber than the java programmers.
There are absolutely good Java programmers. They just probably work at Google or Facebook and you'll be competing with Google. I would take less money (and have, though not too much less) to not work at Google because I get to use a functional language.
shrug
If a startup is using java, I think to myself 'why java? why not Scala, F#, or C# at the very least?'. Usually the answer is 'scala/F# programmers are too hard to find', but that's not really true, what they mean is they're not willing to pay the 20% premium for them. That's a strong indicator that a start up doesn't value talent.