Who was it that turned GC off entirely, minimized allocation and just restarted their VMs when they ran out of RAM every couple of hours, was that Netflix?
Either way. It makes me excited for Rust and the languages it'll inspire, all this labor gone away.
Every single financial firm out there, using Java for sub-microsecond tasks. Really, there is no other way to keep low latencies if you have your GC messing around every few milliseconds.
This may surprise some people, but Java is ubiquitous in low latency environments such as trading firms. It offers performance close enough to C++, but the developer pool is way larger. Also, when one needs to comply with extremely low latency requirements, the way to go is always dedicated hardware anyway.
If you are interested in the topic, there is this project called OpenHFT that aims to provide high frequency trading tools for Java. Particularly, their Chronicle Queue implementation tries to handle the GC latency issue by storing stuff off heap. Its co-founder, Peter Lawrey, has also delivered a handful of good talks about low latency Java.
The only firm that widely uses Java over C or C++ on the hft fast path is Virtu, and the sense I get from some of my friends there is that they regret the decision (and now it’s mostly used to guide fpgas).
There is a TON of Java usage in software that would have been considered HFT maybe 10 years ago, especially at big banks, but the fastest I’ve ever seen somebody make a Java trading program that actually handled the whole tick-to-trade path is 7us, which would be at best ok by C or C++ trading platform standards.
I’m sure you could write some dumb trigger program operating on off-heap bytes that had similar performance characteristics to a C hft program, but unless you accomplish this via code generation then you’re just writing C in Java with a runtime actively battling your goals.
Edit: Jane street uses ocaml + fpgas but they aren’t really in the HFT business in the same way that say Virtu or Tower is.
Otherwise, if you wrote your image processor in C# or Java, it becomes hard to call your library from Python or Node because you have to require the entire VM. Likewise, you can ship an application binary that has no requirements for a runtime. (Your application binary doesn't require a JVM, CLR, Mono, Python, Node, or some other runtime.)
I've been through the Rust book twice but I'm just getting to the point of trying to write something in it. The mental model is very different. Coming from C# / Java / Javascript / Objective C; I'm wondering how many hours I need before I can get my head into Rust?
I found C++ to be treacherous around corners cases on this subject.
OTOH, if you think other languages let you do away with a GC without pretty significant extra work, especially in concurrent systems, well, then you haven't had experience with those languages.
I'd say the opposite is true. Relying on GC requires significant extra work. Because you always need to think about memory (exception: small script-like applications). The only thing a GC does is that it enable you to not think about it, but the moment you don't you will write bad code and realize it was a disservice all along. And by then it is too late.
So in a GC language you need to constantly be aware of when you take something for granted. Which is more work than just doing it manually yourself.
Steve lays it out far better than I could, coining the term "static garbage collection": https://steveklabnik.com/writing/borrow-checking-escape-anal....
And to your "without pretty significant extra work" qualifier: I really don't find that to be true with Rust. The initial learning curve was a bit rough, but certainly far less so than other languages/platforms I've picked up (hello, ML). In the end, I find that it's just a nice, helpful, productive, and ludicrously performant language.
This is still a lot of work for a video game (because you never want any latency, the only way to achieve this is with an arena allocator or going full @nogc).
But for apps where the latency requirement is bounded, D doesn't make it hard and the language is nice and ergonomic.
The memory work remains the same, you can do it yourself or let the GC handle it. For 99% of applications, the GCs are good enough and getting better every year, but low-latency still needs predictability and would ideally choose manual memory management.
The realities of the job market and IT deployments are different though and that's why we still have JVMs involved with low-latency scenarios because of talent, tooling and productivity.
This was common practice in trading firms that got on the Java hype train. Turn off GC and just restart the JVM outside of trading hours.
> Either way. It makes me excited for Rust and the languages it'll inspire, all this labor gone away.
The JVM gives GC a bad name. There are plenty of GC languages which don’t have the level of pain of hotspot except in extreme cases. For the vast majority of GC languages you never even think about it. Rust / C++ are great when you need full control but it’s not necessary for most things.
Whomever they were, they were rotating pre-warmed jvm images with disabled GC, and were reaching quite respectable latency figures.
To not have to recycle them quickly, you'll want to not generate too much garbage objects, and that's actually easier than one might think in Java. Especially if you accept to restart the jvm from time to time, as you only need to be mostly statically allocated.
Rare error paths can freely use dynamic allocation as long as most of the service doesn't.
Nowadays you can also get away without using strings in most places, using only char sequence flyweights over "statically" allocated char sequences. Otherwise strings were a pain, especially API's that really doesn't need a string (ownership) but had string method arguments nevertheless.
Used like that, as you would on an embedded platform, theres nothing I know of that actually beats the JVM in raw performance while still being somewhat practical in terms of tooling and hiring. Rust might take that crown, we'll see, but I hope so.
Here we go again ...
Neat that they were able to get a dramatic improvement in GC latencies on both G1 and ZGC.
No mention of the Shenandoah GC. Would the same trick help out there too?
Exactly... I thought they may be talking about Project Loom's Virtual Threads (which are going to be true green Threads) which are available experimentally as of Java 15, given they did use Java 15, but nothing in the post indicates they used that.
And then say it's comparable. "This basic design is also present in the concepts of green threads and coroutines. In Hazelcast Jet we call them tasklets."
From https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.ht...
-------------
An alternative solution to that of fibers to concurrency's simplicity vs. performance issue is known as async/await, and has been adopted by C# and Node.js, and will likely be adopted by standard JavaScript. Continuations and fibers dominate async/await in the sense that async/await is easily implemented with continuations (in fact, it can be implemented with a weak form of delimited continuations known as stackless continuations, that don't capture an entire call-stack but only the local context of a single subroutine), but not vice-versa.
While implementing async/await is easier than full-blown continuations and fibers, that solution falls far too short of addressing the problem. While async/await makes code simpler and gives it the appearance of normal, sequential code, like asynchronous code it still requires significant changes to existing code, explicit support in libraries, and does not interoperate well with synchronous code. In other words, it does not solve what's known as the "colored function" problem.
Soon to be almost 20 years ago we pushed >10k messages per second on a JVM, using essentially pentium pro class hardware, and the messages spread over thousands of TCP consumers, yielding average latencies well below 0.5 seconds. Not really low latency, but low enough that we didn't need much lower
This was on a purely blocking implementation, because that was before almost anyone did anything like that on the JVM.
With the advancement in async IO, it's got to be possible to drive many millions of sockets, or have really low latency targets before you have to start being really careful?
So what are you guys doing that seems to need so much async code?
With that I mean actual async code, not code having locks but pretending to be async by using callbacks everywhere, because that's somewhat common.
I'm not trying go be rude, I honestly don't get what people are doing that needs more than the JVM should rather easily provide, unless possibly you have super low latency targets?
There seems to be too many that have to resort to quite cumbersome implementation strategies, so I'm starting to think there's some corner of the industry which I have completely missed, and which requires these strategies regularly?
What? We've (JavaScript and developers dealing with asynchronous patterns) been able to build scalable (in terms of code and its maintenance) for many many years, probably 10+.
async/wait is simply syntactic sugar and doesn't drastically change anything, you still need to understand the asynchronicity underneath it all, and if you do, you won't have any problems building scalable apps using your knowledge.
It's a complete shift in application design...
If I have to flatmap one more time...
BTW, many if not most of the cutting-edge advances in compilation, low-overhead deep profiling, and garbage collection are done on the Java platform, so it's still the technology leader in those areas.
Ah and the beloved OS around here is reaching 60 as well.
Plenty of legacy love.
"Aegis Battleship Weapons System"
http://www.artist-embedded.org/docs/Events/2011/JTRES/Slides...
"French radar system for ballistic missile tracking and measurement"
https://www.militaryaerospace.com/defense-executive/article/...
"NASA Ground Control Station for Multiple UAVs Flight Simulation"
https://www.semanticscholar.org/paper/Ground-Control-Station...
True that one can end up writing terribly inefficient Java code, but one can write terrible code in any language. If I need to write server code where performance is particularly important and I don't want to deal with the cost (in debug time and dev expertise) of C or C++, Java would be my first choice.
Also I'm of the school of thought that performance always matters. Autoscaling in cloud providers sure makes it easy to scale horizontally to make up for slow server code, but once you reach certain size, go have a chat with the finance team about the AWS bill.
