Exactly. This is also what makes SPF and friends a bit of a pointless exercise. Even if they had global unanimous support end users don't really care about the envelope from anyway.
Stopping phishing is hard. End users mostly are fooled by a little padlock in their web browser, and that's a much simpler trust model. Eliminating email dressed up as web pages would probably do more to combat that than authenticated sender models ever will, but nobody really wants that.
