Very few know that swallowing two or more small magnets can result in death in a few days, even with medical treatment, especially if the treatment is "too late". That's very much unlike other "small bits of metal", to which I'd include small ammunition (although the commonly unplated lead bullets in .22 LR ammo would add a lead exposure threat).
And then there's the little matter of the 2nd Amendment, we have no Constitutionally enshrined right to keep and bear small magnets, which are also fantastically less useful than guns.
For that matter, in the 1970's heyday of US gun control, the CPSC was explicitly banned from the area of firearms and ammo. This is not unique, there are many other things like "tobacco, motor vehicles, pesticides, aircraft, aircraft engines and boats" (http://www.nssf.org/newsroom/writers/guide/regulations.cfm) where the regulation is done by other units of the government, generally ones with domain knowledge. E.g. pesticides touch on at least three areas, safety to those handling and using them, plus food and environmental safety, that's more like the remit of OSHA (which can cover the much smaller consumer uses) plus the FDA and EPA.
