With regards to SpaceX ratting on our location, I don't think that would be a serious worry but in any case whenever shit gets serious a warship will go into "black hole" operations that block any non-essential comms. I no longer work for the navy but I can imagine that would involve physically cutting power to the starlink dish.
As a practical matter as well, both in theory and based on usage in Ukraine, Starlink appears to be a somewhat challenging target too. A phased array doing 10-12 GHz is a fairly tight beam and it's tracking very fast across the sky, jumping around between LEO@550km (and in the future VLEO@~350km) sats. In a naval setting it's not clear that'd be much of a limiting factor: something capable of seeing that would probably need to be at such an altitude and angle to ships on the ocean that it could also just plain see any surface naval vessel directly optically or via radar. The stealth ship proposals Skunkworks suggested back around the F-117 never went anywhere since the US Navy is dumb^Wtraditional.
But as you say either way they can always just turn it off as needed. It'd be very helpful the much higher percentage of time that things are boring.
I had a hard time finding confirmation of this online (lots of hits about Starlink potentially being used _as_ a GNSS), but one of the photos of this teardown of a terminal highlights the GPS receiver: https://arstechnica.com/information-technology/2020/12/teard...
