Mathematicians do care about how much "black magic" they're invoking, and like to use simple constructions where possible (the field of reverse mathematics makes the central object of study). For example, Wiles' initial proof of Fermat's last theorem used quite exotic machinery called "inaccessible cardinals", which lie outside of ZFC. Subsequent work showed they weren't needed.
Another good example of mathematicians caring which 'house of cards' their results are built on is the search for an "elementary" proof of the prime number theorem (i.e. showing it doesn't rely on complex analysis).
Edit: here's a great related discussion on MathOverflow, bringing in analogies from CS: https://mathoverflow.net/questions/90820/set-theories-withou...
In a way mathematicians can afford to do this more readily than people in software development, because if something is actually proven, then you can 100% rely on that. With software not so much. Or rather: Software usually is not proven to be correct, because that's usually expensive. In mathematics they don't have to consider the runtime of an algorithm, when they "merely" need to prove correctness. The time it needs to run is irrelevant for its correctness. And so they can stack and stack and stack, provided that each piece is proven correct, and it won't have negative consequences. Well, almost. There is some negative consequence in that another human being, wanting to understand a proof, needs to know perhaps many concepts and other proofs, in order to be able to do so. But that's probably the only reason to pursue simplicity in mathematics.
This is a very naive take - the very direct translation of what you're saying doesn't happen does in happen in analysis all the time: there are many inequalities which can be "stacked" to prove a bound on something but their factors are too large so you cannot just stack them if you need a fixed bound for your proof to go through. Unsurprisingly this is exactly how actual runtime analysis also works (it's unsurprising because they're both literally math).
I'd be careful about generalizing that to all or most 'mathematicians'. E.g., people working in a lot of fields won't bat an eye at invoking the real numbers when the rational or algebraic numbers would do.
most mathematicians act as though they are Platonists, even though, if pressed to defend the position carefully, they may retreat to formalism https://en.wikipedia.org/wiki/Mathematical_Platonism
Mathematicians begrudgingly retreat to formalism and foundations when pressed because its easier to defend, but the day-to-day of contemporary mathematics is much more an explorative process of a "real" mathematical landscape. They aren't concerned with foundations because it "feels" self-evident that the mathematics they are discovering is true (because their means of discovery, rigour and proof, "guarantee" it to be so).
A lot of the comments here are making false assumptions like "but surely mathematicians all know that their field is ultimately justified as a symbol-pushing game from some axiomatic system right?" in the same way one might say "surely all computer scientists know that every language ultimately compiles down to 1s and 0s processed by a CPU" but that is not at all how most mathematicians think about doing mathematics.
Architects know and care that they are building things made of atoms. And then ... pretty much don't think about atoms because the objects and relationships they are working on are abstractions well above the fine details of atoms.
And having designed many structures with architectural methods, and seeing those buildings built and stand, it doesn't worry them to hear physicists arguing that maybe atoms are different than they thought. They figure, their experience with architecture and its artifacts has proven to be reliable, so there is no realistic threat of some new quantum theory undermining their work.
On infrequent situations, where their work needs to deal with some special property of some material, they don't have any issue dipping down any number of levels of abstraction. But as a practical matter, that is infrequent for most math.
I know TFA says that the purpose of foundations is to find a happy home (frame) for the mathematicians intuition. But choosing foundation has real implications on the mathematics. You can have a foundation where every total function on the real numbers is continuous. Or one where Banach–Tarski is just false. So, unless they are just playing a game, the mathematicians should care!
You say you have a foundation where that is in fact what I am doing? Great, if that floats your boat. I don't care. That's several layers of abstraction away from what I'm doing. I pretty much only care about stuff at my layer, and maybe one layer above or below.
Or what of commutative algebra and their beloved existence of maximal ideals!
So if many mathematicians can go without fixed definitions, then they can certainly go without fixed foundations, and try to 'fix everything up' if something ever goes wrong.
But the key is that proponents of both definitions can convert freely between the two in their understandings.
I mean, mathematicians do care about the part of the foundations that affect what they do! Classical vs constructive matters, yes. But material vs structural is not something most mathematicians think about. (They don't think about classical vs constructive either, but that's because they don't really know about constructive and it's not what they're trying to do, rather than because it's irrelevant to them like material vs structural.)
