If instead, every site I had ever logged into kept track of my tokens I would need to visit each of them and do the same thing.
(It's already messier than that because some accounts I have--GitHub and Facebook--don't accept SSO but are important enough to be worth protecting with hardware tokens. But I don't want to go farther in this direction!)