I agree with you. It seems wasteful to spending a whole "wait around period" for that <div id="root"/> to eventually load once the js gets around to it.
I also agree that server components make a lot of sense to solve this problem. It's the obvious optimization (and in some ways, a good re-learning from past wins with PHP, Ruby, etc).
I also still feel we have a long way to go before it becomes elegant and obvious how all this is working. It hasn't become standard in the way that everyone easily groks the concept of how it all works and expects it everywhere on every web stack.
Still waiting to see if we go down another rabbit hole of complexity, or see more cautious and careful improvements towards an easier, more maintainable environment.