When I talk about using it for the entire end-to-end build process regardless, I'm talking about people using it to do things that should really have been written in C# or Python, such as manipulating multiple configuration files. The result is often gratuitous quantities of copy and paste boilerplate code that can be very painful to maintain.
Another common problem with many MSBuild scripts is that they border on the unreadable. Besides the syntactic noise of the XML angle bracket tax, almost every line in almost every MSBuild script that I've ever encountered is hundreds of characters long for instance.