Let's say someone with an older irc client or who doesn't want e2e messages you,what then? Now your irc conversations become opportunistically secure. That means only arbitrary conversations are secure. Email does this and it sucks.
Another problem,clients that don't have e2e almost always also store logs in plain text. Yet another problem - no protocol level e2e plans as of yet.
So why try to fix it? This approach does not work. The least we can do is learn from history. Unlike email or ftp, there aren't a whole lot of people that depend on irc being the current irc protocol. What people want is a distributed server architecture with a user interface that is similar to the current irc. What protocol engineers seem to get wrong is that they use the opportunity to also add other fancy features which simply ends up working against adoption.