Legible, maybe, but I don't think you'll be able to collide the hash with natural language, so you will need an excuse as to why the revealed text has weird nonsense in it that is actually there to collide the hash.
You can also "see" this, once you suspect it's going on, if you're a cryptanalyst and have the tools to see inside the hash, you can see basically such collisions involving getting the internal state into an awkward place, and then forcing it from there where they want to go, that could happen by accident, but only by truly astronomical bad luck, so it's a pretty good smoking gun.
MD5 is a bad idea for situations where a machine will be verifying, because machines aren't good at saying "that's odd...", enhancing them to do so is _way_ more effort than just using SHA-256 instead. But for something like this where a human will be examining things by hand, it's fine.
For SHA-1 (which we knew at the time would be broken shortly, and then in practice it was less than a year before Google and some academics announced their full collision) we reviewed special "Exception" SHA-1 SSL issuances by hand on m.d.s.policy after the official deadline for SHA-1 issuance and I asked for one application to be explained or rejected on the basis that the requested certificate had bizarre short gibberish values for "Organizational Unit". The applicant provided an explanation (which was maybe plausible but not up to the standard of transparency needed in the circumstances) but agreed to accept certificates missing the OU value altogether instead. That's the sort of thing you'd catch if a human examines what was hashed rather than a machine. I had no reason to believe that applicant was trying anything sinister, but the point of the manual examinations was to ensure everybody can see there were no shenanigans going on (and to make it a complete pain -- if the process was easy it would have become routine and defeated the purpose of prohibiting new SHA-1 issuance, being annoying was a feature).
