The Supreme court decided that the Fed could regular intrastate commerce in Wickard v. Filburn, and that ruling has also been used since to go after marijuana that stayed intrastate. [1]
I think it's a bullshit interpretation of the text of the Constitution too, but that doesn't change the current legal landscape, however much I or anyone else might want it to.