Microsoft the software company has a competing technology to sell you. Microsoft Research does not ship code. IBM and HP are professional services, not technology R&D companies. RedHat's JBoss division had been (very positively) driving the direction of Java EE, but they're much more of a consulting/software vendor (focused on enterprise software, not on programming languages) than an R&D shop.
Google would be the most likely candidate: they use Java extensively, make their own JVM and contribute to Apache Harmony. They do highly advanced R&D work (you could say they're the modern day Bell Labs), but most fruits of it are internal (with some work ending up as research papers and a very tiny fraction going out as open source).
Seems like something is missing: a first-class R&D shop that's a home to top technologists (who aren't interested wealth through entrepreneurship, but would rather work on many different, challenging projects -- something focused start-ups can't provide) and which ships software and hardware to the world. I'd love to see Google step up to that plate, but is that realistic?
I think this comparison is unduly favorable to Google. http://en.wikipedia.org/wiki/Bell_Labs#Discoveries_and_devel...
One of the biggest complaints about Microsoft Research (or whatever it's called) was how little gets out of the lab into the "real world", with things like F# being recent exceptions, and their funding of the GHC is another example of how they indirectly ship code. Although this mostly legit complaint is precisely your complaint about "not just research shops".
I'm pretty sure IBM is still doing some tech R&D, how much in the coding area I'm not sure, let alone for how long.
Hmmm, has such a beast other than Sun for a brief time ever existed? Obviously Bell Labs, and the UCB Unix project (and some other DARPA projects that had useful code as a required deliverable, e.g. their general VLSI infrastructure push: http://en.wikipedia.org/wiki/VLSI_Project). MIT's exokernel project found itself reified in the real world as Xen, through yet another university lab.
Plus today there are other business models, e.g. look at Clojure (admittedly a one man project for the first few years), the cost of doing a R&D project that ships real world code is fantastically lower than it used to be when e.g. XEROX PARC was doing most? of its work on bit slice (i.e. not too fast) 16 bit Altos with 1-4 banks of 64KB RAM.
I came to the conclusion in the early '80s that some of the differences in the granularity of traditional Lisp and Smalltalk objects (the latter are larger) is in part due to the machines these people did their work on. Lisp has always gone for a big flat address space, Smalltalk had bank switching to contend with an I think that encouraged larger objects (larger than a cons cell, atom, numeric immediate, etc.
The architecture of X has a lot more to do with GAO "most preferred customer" rules/law than anything else.
There's certainly something to be said about it being possible -- more so now than ever -- for hackers to work alone, without affiliation with a large R&D organization or academia. Rich Hickey's work has been amazing and will be influential (the persistent immutable data structures are beginning to spread beyond Clojure). It's also interesting that Hickey was able to "bootstrap" himself from consulting rather than full-time work, regaining full rights to his work (not being subject to the all-too-common agreements which require one to give the fruit of all of their ideas to their employer).
Academia is still doing well (Scala from EPFL, some fairly interesting distributed systems work going on at Berkeley) and there are also university-based spin-offs e.g., Stonebraker start-ups (even though I disagree with some of his approaches, it's still great to see companies built to solve difficult and interesting problems). Historically these have been big on the West Coast: Google, Inktomi, Ousterhout's start-ups (Scriptics, Electric Cloud). I haven't seen any emerge recently, but perhaps I haven't been looking in the right places (or have been too cynical about what I saw).
Microsoft Research is excellent as a research organization, but it seems that it's goal isn't to develop new products based on this research but rather to make sure that the scientists aren't developing products based on their research elsewhere.
Note: there's also lots of research work that needs to be done that isn't going to be applied immediately. That's certainly fine -- but there's been research that has went on at Sun that has been turned into industry changing products (from NFS to Java).
I think it depends on the road-map of javaBy the end of the year the pattern will be clear; me, I'm watching to see if they keep Fortress and Guy Steele.
Well... It's better than Microsoft, I guess. The thought of a "Windows 7 Server for SPARC Enterprise Edition Plus" is frightening.
The only reason Windows only runs on x86/x64 now is that customers weren't interested in it on other platforms. Microsoft really tried to make it cross platform.
I hope you take much deserved time off then maybe end up at one of the newer brain trusts like Google.