Defense in depth, essentially. But yes, checking site X to see if site X gave you Y correctly is rather weak confidence.

To argue in its favor tho, file-hosting and site-serving may be handled by different systems with different security characteristics, and potentially even different datacenters (e.g. a CDN). If you only have to compromise one system, it's generally easier to do so than when you have to compromise N and make them all agree with each other.