We also can't realistically expect recreational alcohol use with no long-term issues either: alcohol kills about 100k people per year in the USA.
That's not a reason to ban it. The issues are caused by the user, to the user. If users are causing issues to others (violence, theft, drunk driving, et c), those are separate and already-illegal things that can and should be enforced. Arresting people for possession is basically like arresting them for pre-crime: "some other people who possessed similar things later went on to commit actual crimes, so you go to jail out of fear" is not a sane approach.
The possession and use of substances for the human body is plainly the wrong place to legislate: the war on drugs in the US has been a costly, racist, unmitigated disaster.
