Well, the constraint layout system that Apple introduced (sorry, I am drawing a blank on the name) is sufficiently different that I'd say that although technically you could call it a layout tree, that is really missing the point.

Also, the render tree concept doesn't exist in the internals of GTK (at least; I could speculate about Qt but will not). Rendering is done by invalidating rectangles and then finding intersections with widgets; since there is no z-axis stacking, there's only ever 1 widget to be drawn at a given pixel coordinate.