The sending mail server was configured with a zero timeout for connections. If it didn't IMMEDIATELY get a response from the destination mail server it would fail. This immediate failure took 3 milliseconds, a long enough time that some servers could actually respond back before that happened... but if the server was too far away (more than 500 miles) the connection would fail before the first packet could even get there due to the finite speed of light.