Most of these switches are driven by a shift to some new programming paradigm. We started with procedural programming, then made a big shift to object-oriented programming, and now we're starting to see a big shift toward functional/declarative programming. Who knows what'll be big next?
If this happens too frequently, then there are certainly going to be problems, yes. The current trend, however, indicates that it's not happening too frequently, at least not yet; the current shift from object-oriented imperative to functional/declarative programming (with some object orientation here and there, though this isn't very pronounced in Erlang and its family tree) seems to be by necessity rather than adopting shiny for the sake of shiny, and I think that's what makes the difference here.
Basically, the more dramatic the switch is, the more likely the switch to be a net gain rather than a net loss, since it indicates that the switch was truly necessary. Switching from Python to Ruby, for example, would be a net loss, since - while they have different syntax and in some cases different programming styles and ecosystems - they're both imperative scripting languages and you're not really changing all that much besides a bit of syntax and semantics. Meanwhile, a switch from Python to Haskell, for another example, is more likely to be a net gain, since you wouldn't even consider that switch unless you felt the need for the massive paradigm shift that's required to make such a switch.
As a disclaimer, my claims above are mostly conjecture, though I think they correspond well with reality, at least by my own observations.
Most of the switches should be driven by one thing, and one thing only: reducing the cost of software development (or increasing the quality, which is really the other side of the same coin). Whether a new programming paradigm actually achieves that or merely advertises itself as achieving that is the cause for the very high churn in fashion-driven SV and low churn elsewhere (where people are actually interested in cost reduction only).
> the current shift from object-oriented imperative to functional/declarative programming
As someone familiar with the software industry, I can tell you that there is no such shift happening other than in the minds of some wishful-thinkers. Are functional concepts being adopted by OO languages? Sure! (although Gilad Bracha would tell you, correctly, that most of these concepts were already in some OO languages -- i.e. Smalltalk -- long before people in the industry started using the term FP). But people aren't really switching languages outside those sectors that have made switching languages every few years a lifestyle choice.
> Basically, the more dramatic the switch is, the more likely the switch to be a net gain rather than a net loss,
I completely agree with that.
> since it indicates that the switch was truly necessary.
But not with this. Dramatic changes do have the potential to make a dramatic impact, but 1/ they rarely do, and 2/ when they do, it's mostly in the runtime -- not the language. Erlang is certainly an excellent runtime that has a big impact (though personally, I prefer the JVM, which can do anything BEAM does only better), but Haskell? Haskell is a very, very unproven language. I remember that about 15 years ago everyone was talking about it, saying how it's going to be the next big thing. 20 years into its existence, though, the largest single Haskell program is still its own compiler, which is not particularly large. The truth is that nobody really knows Haskell's actual benefits (in terms of real impact to project development) because in twenty years no one has tried to actually use the language for anything big (and small projects are usually cost-effective in any language). Its impact could be great but it may well be negative. Nobody knows. So if anyone tells me they've switched to Haskell out of necessity, I call BS.
But switching to BEAM is always a good idea (sooner or later) for large projects (unless you're already on the JVM), although if you're making the switch, I'd advise on the JVM, as it's a safer bet (both in terms of future, as well as likeliness of addressing the needs you may actually face; BEAM is a slow runtime, and many Erlang shops rely on C for important parts of their code).
> though personally, I prefer the JVM, which can do anything BEAM does only better
In my experience, this couldn't be farther from the truth. Java's threading model is bloated and slow compared to Erlang's process model, and that process model - the ability to spin up processes so lightweight that they make even Java threads look heavy in comparison - is the key to its success when it comes to parallel and distributed computing.
Now, this isn't to say that a JVM implementation can't do the things BEAM can (otherwise, projects like Erjang wouldn't exist), but I've found Erlang applications with BEAM to perform far better in that domain. This isn't particularly surprising, seeing as BEAM was designed to run declarative, distributed, concurrent languages (like Erlang and eventually Elixir and LFE and the like) right from the start, whereas the JVM, .NET CLR, etc. were originally designed for imperative languages (like Java and C#, respectively) and have been gradually adapted for fuctional/declarative programming (like with Scala or F#, respectively).
> no one has tried to actually use [Haskell] for anything big
Maybe because the equivalent Haskell codebase to a "big" project in most non-declarative languages ends up being "small" in comparison? Or are you talking about importance?
