…and money to go to conferences and summer schools, and money for software licenses (especially in applied programs), and department funds to bring visiting academics, and the following things that get lumped under administration: money for grad student food pantries and childcare because funding streams for PhDs don’t provide for good salaries outright, and job advising centers because the math job market is a crapshoot, and free student health clinics for psychological and physical health because doing a PhD in any condition is rough…

Who pays for all that? Usually it's not the students or even private funders / donors. Most of the money comes from one level of the government or another, and it comes with all kinds of regulations and requirements. Complying with that requires a lot of specialized administrative staff.

Most of the time, when you hear a politician saying that universities should / should not do X, they are effectively saying that universities should spend more on administration.

Universities with a residential campus have a lot of staff in functions unrelated to the academic mission, such as student housing, food services, healthcare, or sports facilities. If they have to compete for students instead of most people just automatically choosing the nearest university, focusing on these tends to make them more competitive. And while student amenities are not particularly important to PhD students, they are important to the university if it also educates undergraduates.

Then there is the organization chart. In a traditional university, the faculty senate (or another similar body) is in charge and all administrators are subordinate to it. But the modern world prefers centralized organizations, with administrators at the top. And whoever is in charge also determines the priorities of the organization.

"You could cut university admin costs by 75% and lose nothing."

People say that, but could you really? I'd love to see a breakdown on how you pull this off.