And for a while those states were subject to ongoing raids from the federal government. The Obama administration has held off, but the legal framework that justifies said raids is very much in place. If opinions blow back the other way they could resume any time.
When you're not sovereign, then your sovereign can tell you what you can and can't do. If you're mostly on their side, it's not a problem, but there's no guarantee it will stay that way forever.
Say a conservative supreme court comes to power and says, "Yeah, pot is totally banned and states can't opt out of it." End of discussion.