Wouldn't initial page load be affected? On the user side this only really happens on the very first load when the font isn't cache'd but this might affect SEO bots more?
It makes no difference other than download/parse performance. You have to weigh the latency/bandwidth of your origin vs that of Google's CDN and the extra connections.