Thing is, as long as there is one honest person with a connection to both networks, they can easily serve as a relay between them, and keep both networks fully synchronized.

If an attacker is on both networks and trying to double-spend, it's extremely likely that someone honest is also on both networks who is successfully keeping the two networks fully synchronized. In this type of situation, you can make yourself safe just by waiting for 4-6 confirmations instead of the usual 0-2 confirmations.

Which is my point: if an interconnect is possible, miners are going to have a very high incentive to use it as well. Not just double-spenders. And it wouldn't take many miners doing it to prevent the fork in the first place.