Yes, but users can compare fingerprints (sure, most probably don't, but it's definitely a deterrence against MITMing all conversations by default), receive warnings whenever fingerprints change etc.
There's also supposedly a key transparency service deployed (similar to Certificate Transparency), but I haven't looked into that in detail.
