The problems are 1) side channels. Caching those would mean that a malicious script could use timing to find out if the user accessed specific other websites. Also it enables a channel for cross-domain communication. 2) malicious user tracking would not work as well anymore, which would be good but Google probably will not support that. Currently they get all those nice http log entries from sites that only include fonts etc.