>What if it were a Chinese mine that they hit?

Laying a minefield in international waters (from the point of view of everyone but China, anyway) is an act of war. It's not necessary for there to be any interpretation or misunderstandings at that point, it's pretty much the same as if the Chinese fired ballistic missiles with real warheads at Taiwan, or engaged foreign navies in the South China Sea with swarms of anti ship missiles. At that point, the war has already started, so no one would do that unless that's what they wanted.

Which is why China hasn't placed any mines there, and your question doesn't make sense.

It seems equally unlikely to me that the Pentagon would let loose the nukes after losing a single SSN and having no other indication of an invasion. For one thing, it would probably take days to confirm the sub was sunk, and by that time the lack of an invasion should be clear. All the other American ships in the area being unscathed would also be a pretty big clue.

To be frank I'm not even sure I believe America would start a nuclear war at all even if Taiwan really were being invaded.