You can have a front-end web server that is also a client of the API server and keep the separation as if it were any other API client. This is the approach I'm taking in my current project since I have to support some obsolete browsers, but more generally it lends itself to a cleaner decoupled architecture

It's not only possible, it's pretty trivial in most cases. Just endow each user facing object with a .ToHTML() and .ToJSON().

This is actually one of the core benefits of REST, you send me a request for a resource along with some desired media type(s), and I send you back a representation in the media type of your choice (or as close as possible).

The trap a lot of developers fall into is going around the existing API for server-side applications though, thinking they'll get more performance by (for example) going directly to the persistence layer. That's how most server-side apps are written, actually; api and front-end tightly coupled.

I agree, but there are other benefits too :)

Yes it is, that's my point. Our designers push new templates and our developers push new binaries completely independently. We literally have an API doc that the designers put "I want this" into, and the developers implement it and update the doc to reflect it being finished. People just use template systems that make this unnatural. See heist for an example of a system that makes it natural and straight forward.