In your analogy, the client JS code is like the serverside code, fetching over the network instead of directly from the DB, and then doing essentially the same work from there... materializing html and a set of controls for the user to interact with.
In a sense, I see your point.
But there's a difference: When you materialize the html on the server and send that over the wire, the browser does all the work for you. When you take the SPA approach, you must re-implement much of what the browser does in JS, and hence the well-known trouble with routing, history, and so on. You can argue that React/Angular/whatever takes care of this for you at this point, and to some extent it's true, but you're still cutting against the grain. And even as mature as the frameworks are, you will hit weird edge cases sometimes that you'd never have to worry about with the browser itself.