undefined | Better HN

0 pointsJweb_Guru5y ago0 comments

Renderers need to deallocate memory all the time. For example, they often need to store per-frame data that gets dropped when the frame is over, debug strings, track temporary access to texture/buffer data, manage command buffers, etc. They also have to track resources for lots of data types external to the program (such as GPU data) which often forces them to work with external allocators, as well as having to synchronize stuff like resource destruction using GPU-CPU fences. Moreover, most resources are either immutable or can be accessed optimally when they are immutable, so you often end up benefiting significantly from being able to destroy and reclaim the memory of stuff you're no longer using rather than trying to update in place.

Even if you try to only allocate one buffer (for example) to split up and use for all your logical vertex buffers, you still have to manage memory within that buffer, which ends up leading to pretty much the same kinds of complex reasoning you need for ordinary dynamic allocation. Failure to do this properly results in resource leaks on both the GPU and CPU side which can be really nasty.

Of course, this all depends on the complexity of your renderer. For simple usecases you might be able to get away with no deallocation until exit. But they'd have to be really simple.

0 comments

4 comments · 2 top-level

skohan5y ago· 2 in thread

> For example, they often need to store per-frame data that gets dropped when the frame is over, debug strings, track temporary access to texture/buffer data, command buffers, etc.

Exactly. You can just bulk allocate all the memory for the frame at the beginning and drop the entire thing after the frame is finished. This is a very easy case for manual memory management where you have one allocation and one deallocation per frame.

Or you can do one better, and keep an arena for each framebuffer, and then just recycle them and never deallocate at all. If you really need some level of dynamism in terms of memory usage, you can just double the size of the arena every time you fill it, and you still don't need to deallocate, since chances are you will need that space again for some frame in the future.

Jweb_GuruOP5y ago

I love arena allocation (which, incidentally, does entail dynamic resource destruction if you use it per-frame, and which is pretty easy to get wrong if you're just storing raw pointers everywhere--so good bye nice use after free properties), but you absolutely cannot just use arena allocation for everything. GPU resources in particular need to wait on fences and therefore can't have a neatly defined lifetime like what you're proposing unless you block your whole program waiting for these fences, while other GPU resources are (like I said) immutable and need to live for multiple frames. Where exactly do you store this data? How do you access it? How do you free it when it's done? How do you perform slow tasks like recomputing pipelines and shaders in the background without being able to dynamically track the lifetimes of the old/new backing GPU allocations? More generally, how do you track resources other than the framebuffer/surface, which are usually managed very dynamically and often appear and disappear between frames?

I think maybe it would be helpful if you would go look at the actual implementation of a nontrivial renderer and try to find one that doesn't allocate. Because the strategies you're describing don't match my experience working on renderers, at all.

skohan5y ago

I am speaking from experience, I have written a lot of renderers.

> GPU resources in particular need to wait on fences and therefore can't have a neatly defined lifetime like what you're proposing unless you block your whole program waiting for these fences, while other GPU resources are (like I said) immutable and need to live for multiple frames.

I don't quite understand why fences are a problem here. As long as you know what frame a resource corresponds to, you just to have to guarantee that the entire frame is finished before reusing. You don't need to know what's happening at a granular detail within the frame.

As far as static resources, that's easy: another arena for static resources which grows as needed and is shared between frames. Most of that data is living on the GPU anyway, so you just need to keep a list of the resource handles somewhere, and you're going to want a budget for it anyway, so there's no reason not to make it of finite size.

> More generally, how do you track resources other than the framebuffer, which are usually managed very dynamically and often appear and disappear between frames?

See the whole issue is trying to solve the issue "more generally". In many cases, you can know exactly the resources you will need (or at least the upper limit) when you start rendering a given scene. If you need to do any memory management at all, you can do it at neatly defined boundaries when you load/unload a scene.

The only time when you need some kind of super dynamic renderer is when you are talking about an open world or something where you are streaming assets.

Most of the serious renderers are written in C++ using custom allocators, which are just some kind of arena allocator under the hood anyway.

1 more reply

habibur5y ago

> Renderers need to deallocate memory all the time. For example, they often need to store per-frame data that gets dropped when the frame is over

Zero the buff and reuse this same frame as the new frame.

j / k navigate · click thread line to collapse