Dynamic height of virtual scrolling elements is a thing. You just need to recalculate the scrollable height on the fly. tanstack's does it, as do some of the nicer grid libraries.