Even people who've spoken English daily for decades will do a double take if they're not used to it.
If you listen carefully, the range of vowels is quite wide, for the same word in different accents. The probably throws off a lot of people.
When they visited Texas for a family reunion, several of my cousins who have not spent much time around non-Texans honestly asked about the language she was speaking.
I think this diversity, while large, is still less than in many other languages. As one example of many, just within Europe there are accents/dialects of German in Switzerland that are largely mutually unintelligible even to some other Swiss German speakers, and nearly entirely unintelligible to most Germans. This is also true for dialects of Italian, or even Dutch, according to friends from those countries. It's generally even more fragmented outside of European languages.
Unlike English, some of these cases don't even have easily standardized registers. In my experience, people in Birmingham or Glasgow, for instance, can fluidly switch to a more standardized accent and dialect that's natively or "intuitively" understandable to a speaker from North America or South Africa.
Dialect continuums between languages are basically the norm around the world. You can even connect Sicilian to Portuguese for example.
I would assume the primary reason English isn't part of any major one is because Britain is an island, otherwise English would probably connect to Dutch and join the West Germanic continuum. It would then appear just as fractured as any other language.
The thing is though, you can ask people to speak High German instead of mundart, eg at meetings. They admit it's different enough to warrant explicit statement.
You can't be telling your Liverpudlian or Brummie to speak the Queen's English, that would be rude.
There's no real trick to it other than paying attention. Record yourself speaking, listen closely to how the sounds you use differ from the accent you want to have. Then practice saying those sounds over and over until they become natural. Keep at this for several years.
It's painful to do (though I find it less so if you focus mostly on specific sounds rather than listening to your full speech -- it's easier to separate the sound from the revulsion of hearing your own voice that way) but it works.