I'm surprised you did not look at "Dear ImGui", "Noesis", and "JUCE". All three of them are heavily used in gaming, are rather clean C++, use full GPU acceleration, and have visual editors available. Especially JUCE is used for A LOT of hard-realtime professional audio applications.
"When we started building Zed, arbitrary 2D graphics rendering on the GPU was still very much a research project."
What are you talking about? JUCE has had GPU-accelerated spline shapes and SVG animations since 2012?
BTW, I like the explanations for how they use SDFs for rendering basic primitives. But that technique looks an awful lot like the 2018 GPU renderer from KiCad ;) And lastly, that glyph atlas for font rendering is only 1 channel? KiCad uses a technique using RGB for gradients so that the rendered glyphs can be anti-aliased without accidentally rounding sharp corners. Overall, this reads to me like they did not do much research before starting, which is totally OK, but then they shouldn't say stuff like "did not exist" or "was still a research project".
https://github.com/juce-framework/JUCE/blob/2b16c1b94c90d0db...
https://github.com/juce-framework/JUCE/blob/2b16c1b94c90d0db...
It also does stroke handling on the CPU:
https://github.com/juce-framework/JUCE/blob/2b16c1b94c90d0db...
Basically, this isn't really "GPU accelerated splines". It's a CPU coverage rasterizer with composting handled by the GPU.
CoreGraphicsContext::createPath will convert the CPU spline segments to CG spline segments which are then rasterized by CoreGraphics using Metal on the GPU.
https://github.com/juce-framework/JUCE/blob/2b16c1b94c90d0db...
And on Windows 7 and up it'll use the hardware-accelerated Direct2D APIs:
https://github.com/juce-framework/JUCE/blob/2b16c1b94c90d0db...
You’d surprise a lot of people if you were right about it using GPU acceleration for its UI framework. It does have an OpenGL container that fits into its view object hierarchy if you want to write your own accelerated component, but the rest of the UI is pretty much standard event-loop -> platform-API dispatch. It’s accelerated where the underlying platform-API is accelerated but not in any kind of explicit or fine-tuned way. The focus has always been more on portability than performance, even on the audio side.
(And of course the high-priority audio processing runs in an entirely segregated context from the UI, so the performance characteristics of the two pieces are decoupled anyway.)
That JUCE is "heavily used in gaming" for starters is widly innacurate
Also, you can use OpenGL for GUI compositing, too, not only as a 3D context.
My 2 cents: it's nice to have smooth rendering in your editor, but I'm currently mostly using a Java-based IDE (IDEA family), and it's responsive enough for my taste. If I were to use the current prototype of their editor, I'm afraid the usage of fixed-width fonts all over the place (which I assume can be fixed, but may also be due to constraints of the UI framework?) would probably bother me more than the 120 FPS UI would impress me (assuming that I had a monitor that was fast enough, which I don't). On the plus side, it sounds like they support sub-pixel rendering - if that's the case, kudos to them!
Zed is not constrained to use fixed-width fonts and supports all kinds of fonts, monospaced and variable-spaced alike (ligatures and contextual alternates included). Even though we use a glyph atlas, we rely on CoreText for shaping and rasterization (rendering sub pixel variants as well). In practice, this means that text is indistinguishable from a piece of text rendered by the operating system.
And egui isn't tied to Apple proprietary frameworks.
This is known as a “multi-channel signed distance field”, or “msdf”.
This is not to diminish the quality work that went into the library, but I passed over it for those and other reasons. I used nuklear for a bit since its typography was extensible but in the end it was too hard to overcome its other issues, so I use a custom mixed immediate/retained library now with rich text and accessibility support.
I'm always interested in learning about GUI toolkits that have accessibility support. What level of accessibility support does yours have? e.g. does it implement platform accessibility APIs, and if so, on which platforms? Is your library open source? If not, can I see it in action in any commercially available product? Thanks.
Is this stuff written down somewhere other than the code?
I use KiCad all the time and certainly never noticed this.
I'm not familiar at all with JUCE, but the state of the art in GPU-accelerated 2D graphics has dramatically improved over the past 10 years so I doubt what JUCE did in 2012 is really comparable.
They are not using SDFs for the text, they render the glyphs at the specified font size directly.
I mostly stick with them because the flexibility and image quality feel worth it.
What causes user interfaces to hick up is that it's too easy to do stuff in the main UI thread. First it doesn't matter but stuff does accumulate, and eventually the UI begins to freeze briefly for example, after you press a button. The user interface gets intermingled with the program logic, and the execution of the program will visibly relay its operations to the user.
It would be very much possible to keep the user interface running in a thread, dedicated to a single CPU on a priority task, updating at vsync rate as soon as there are dirty areas in the window, merely sending UI events to the processing thread, and doing absolutely nothing more. This is closer to how games work: the rendering thread does rendering and there are other, more slow-paced threads running physics, simulation, and game logic at a suitable pace. With games it's obvious because rendering is hard and it needs to be fast so anything else that might slow down rendering must be moved away but UIs shouldn't be any different. An instantly reacting UI feels natural to a human, one that takes its time to act will slow down the brain.
But you don't need a GPU for that.
The latency that matters for a text editor is how long it takes for a new keypress to show up. And that's usually a 32x32 pixel area or less.
You could have some wins around scrolling and any animations that you can commit ahead of time
I do think that they will need to do something similar with a higher level language support for extensions. Could probably piggy-back the Deno work for using V8/JS/TS for extensions similar, maybe even compatible with VS Code extensions. It's a massive user space, but seeing the core feature set in another editor could be really nice.
For me the killer feature of VS Code is really the integrated terminal. Not having to switch windows, or leave the app to run a quick command is really big imo.
No we didn't. We had nothing close to that in the 90s.
Typing was responsive, but that's not anywhere close to 60fps and a very small region to update to boot (120wpm = 600 keys/minute = 10 keys/s, or 10fps). Scrolling benefits from 60fps+, but in the 90s scrolling jumped by lines at a time because it couldn't do anything better.
You need a GPU to keep up with smooth scrolling at modern resolutions. But this also shouldn't be a surprise. What the article talks about is bog standard stuff for all current UI toolkits. It's what mobile platforms like Android have been doing for a decade now.
Was amazed at how good my latest iPhone feels with 120hz scrolling. It's like magic.
Nowadays I still regularly see applications with (temporarily) frozen windows and I just don't understand how that's possible. When I was developing my winforms apps, anything that would do more than perfectly predictable UI manipulations was run on a background thread (in a task), would be forbidden from starting twice at the same time and only updated the UI when done.
The reality is that moving things off the UI thread is always a calculated engineering decision. It can be worth it, but it complicates the code and should only be done if there's a realistic chance of the user noticing.
But the HW accelerated brag is pointeless. Even if they manage to squeeze some extra performance over native toolkits, that is not necessarely going to matter in the grand scheme of things. Drawing the UI is never the bottlneck in a text editor...
> Drawing the UI is never the bottlneck in a text editor...
That would be my advice to anyone making a gpu-accelerated ui library in 2023: Try to support accessibility, and even better: make it a first class citizen.
And it's not just about electricity cost and heat stress, it will conflict with everything else that requires the gpu to do stuff, including watching 4k videos on the second monitor, which does have a legitimate case for requiring hardware acceleration since they move a lot of data 60 times per second, and your editor doesn't.
And the limited resource is not the gpu itself but the nearby onboard memory is a scarce resource on its own. I'd be real mad at a software that prevents me to multitask.
Additionally, using your GPU doesn't mean locking your GPU on that task. Your 4K video most likely isn't taking up 100% of GPU time, nor 100% of the nearby onboard memory. Everyone already gets a share of that time.
I’m honestly thankful for this bit of lying. Windows 8 and below had horrible, horrible issues with fullscreen programs, especially games - they’d change your screen resolution and thus move and resize all your other windows, they’d freeze for ten seconds when you tried to alt-tab out, and they’d frequently just crash when you switched back to them. Those problems are essentially nonexistent nowadays, and it’s worth the small performance cost that comes from going through the compositor.
Chasing 120fps rendering here is the problem.
Then again the issue is the focus of rendering at 120fps.the rendering context should be paused between the vast nothingness that happens in between user keystrokes, that happen at something like 4kp, and in burst.
The insistence on 120fps is the issue here, not the rendering pipeline.
Zed and GPUI use energy very judiciously and only perform updates when needed. The idea is that we can render the whole application within ~8ms, and that shows everywhere: from typing a letter to scrolling up and down in a buffer. However, if the editor is sitting there idle, we won't waste precious CPU cycles.
Thanks for the feedback!
I mean, the win16 api from ages ago had support for invalidating specific screen regions etc. It probably got lost somewhere in the transition to javascript...
Another option in this front could be Flutter if write-once run-everywhere is a need for the project. Another advantage is that it's not only GPU accelerated but it's also retained mode.
Which is kind of a shame. But it's the result of years of product management neglect as well as the pull away from desktop UIs to web UIs.
Native AOT still can't compile WinUI, while C++/WinRT is like doing ATL in 2000, while bug issues grow exponentially.
Only WinDev themselves, and WinUI MVPs, can still believe this is going somewhere.
The rest of us have moved on.
The article is talking about the same generic hybrid GPU-accelerated rendering architecture that everything uses. Seemingly the only "new" part is "in Rust!"
(Also GPUI seems to be retained, so completely unrelated.)
https://nick-lab.gs.washington.edu/java/jdk1.5b/guide/2d/spe...
Similarly, if you're dragging something around with your finger, 120fps keeps that thing closer to the finger. Just improves the UX.
I love immediate feedback but getting it ranges from hard to neigh impossible. E.g. I have a complex Emacs setup for rendering Pikchr diagrams, but there are a lot of problems to solve from diagram conception to the end result, so I thought, hey, why not make my own cool RT editor - in Rust obviously.
Unfortunately I learned that GUIs are though problem especially if idea is hobby-based so there's only one developer inside. Ultra responsive GUIs cool, I have a prototype in egui (not sure if that's as fast as Zed's premise but feels fast nonetheless) and yet it doesn't support multiple windows, which I wanted to have.
120 FPS with direct rendering sounds AWESOME just for sake of it, but I believe that for the end-user layout will be more important than refresh rate, and that's different beast to tame.
Personally I "almost" settled for Dioxus (shameless plug: [1], there's link to YT video) and I'm quite happy with it. Having editor in WebView feels really quirky though (e.g. no textareas, I'm intercepting key events and rendering in div glyph-by-glyph directly).
You're right that rendering is only part of the story. To stay within the ~8ms frame budget, however, every little bit counts. Maintaining application state, layout, painting, and finally pushing pixels to screen, all need to be as performant as they can be.
For layout specifically we're using an approach inspired by Flutter, which lets us avoid complex algorithms but still have a lot of flexibility in the way elements can be positioned and produce a rich graphical experience.
Thanks for reading and commenting!
Rebuilding whole app to ensure that some box doesn't get cut off ruins development experience, especially for big apps.
Kudos to you guys, I hope you'll make Zed extensible, so that instead of writing my own editor I can use yours ;-)
Curious if you guys have thought about VR / AR possibilities with GPUI?
Would probably just be stupid gimmick, but I want to see somebody do it :-D
Windows uses GPU-based rendering in most/all of their GUI frameworks. Chrome also makes heavy use of hardware acceleration. Rendering performance isn't left to the window manager. If you're making your own GUI this is exactly the kind of stuff you need to do to make it fast.
Could you please point me here as to what I am missing ?
After our past experience with Atom, getting the plugin system right is a top priority for the editor.
The thought of cross compatibility with VSCode plugins definitely crossed our mind and it's not out of the question, although our current plan is to initially support plugins using WASM.
Is their GPUI library open source?
It should be noted that the main person behind Zed is Nathan Sobo, who created Atom while he was at Github, which is the basis of Visual Studio Code today.
As such, I have high hopes Zed will be a much faster version of Visual Studio Code and am excited to see what him & his team make.
I suspect a lot of time is likely to be spent on the CPU side updating vertex and other data and pushing it to the GPU so it would be useful to have some more detail on how they are handling that.
I'd like to see "gedit", for Linux, fixed. It can stall on large files, and, in long edit sessions, will sometimes mess up the file name in the tab. Or "notepad++" for Linux.
I'm surprised nobody pointed out lite/litexl here either it's rendering of ui is very similar (although fonts are via a texture; like a game would) and doesn't focus overly on the GPU but optimises those paths like games circa directx9/opengl 1.3
There are great details of the approach taken with lite at https://rxi.github.io
Lite-xl might have evolved the renderer but the code here is very consumable for me.
IME the main theme with achieving high performance not just in games, and not just in rendering, is to avoid frequent 'context switches' and instead queue/batch a lot of work (e.g. all rendering commands for one frame) and then submit all this work at once "to the other side" in a single call. This is not just how modern 3D APIs work, but is the same basic idea in 'miracle APIs' like io_uring
This takes care of the 'throughput problem', but it can easily lead to a 'latency problem' (if the work to be done needs to travel through several such queues which are 'pipelined' together).
Second, forgive a naive question since I know nothing about graphics, but would the method described in the article perform better than Alacritty + Neovim?
I was really excited when I saw that demo. Why didn't this turn into a final product that people could use?