Well, the environment isn't the only factor, but it's the biggest one. You have to take care of routing, permissions, timeouts, errors etc on the server as well. Data being "an sql query away" on the server is similar to being "an AJAX call away" on the client.
If you have a UI, you're going to have to manage UI state somewhere. Even if you have no client-side code, you've marshal that state into strings to send to the client, and then parse it all out again on the server when you get the next request. Remember hidden form fields? Heck, this very site has some pretty interesting history about attempts to solve that problem elegantly. It was a PITA.
Ultimately, none of that has to do with Javascript; it's inherent in a client-server architecture. The one thing that is specific to JS is asynchronicity. If you don't like that, ok, fair enough, but that's a matter of taste. It has advantages as well as disadvantages, and with Promises and async/await, it's actually pretty elegant. It's certainly not the case that asynchronicity makes you write shit code.
