It is much more difficult (perhaps impossible on a properly-implemented site?) to interact with a site on behalf of a user in harmful ways through an img tag -- Like, say, getting their checking account number, or transfering money from their checking account to yours, or resetting their password or email address. It's difficult/impossible to do that kind of attack only through an <img> tag, because Javascript can't see the body response of the URL in an <img> tag even if you point it at a non-image, and additionally an img tag can only be used for GET requests.

I didn't say that XmlHttpRequests "broke any assumptions that were valid before", I don't know exactly what that means.

If an XmlHttpRequest were allowed to do arbitrary cross-site requests, Javascrpt could use them do some of the kind of attacks mentioned above. Which is why XmlHttpRequests can't do cross-site requests. CORS exists to let XmlHttpRequests do cross-site requests only in an opt-in way, so a site can opt-in to it in ways that it knows/believes are secure from attacks like above.

I'm not saying the overall web security story is perfect (nobody would ever say that!), but that's where CORS comes from, and why it's different than img tag.

The common misconception is that CORS exists as some kind of general access control to your website. It only exists to keep the user of a well-behaved browser safe (in certain ways) from malicious Javascript on a site they are visiting. CORS has nothing at all to do with any scenarios involving malicious browsers, which are of course free to ignore CORS.