Is that really the story? I think it was more like "backward compatible solution soon about more pure, theoretically better solution"
There's enormous non-xhtml legacy than nobody wanted to port. And tooling back in the day didn't make it easy to write correct xhtml.
Also like it or not, HTML is still written by humans sometimes, and they don't like parser blowing up because of a minor problem. Especially since such problems are often detected late, and a page which displays slightly wrong is much better outcome than the page blowing up.