Yes, that's pretty normal, that's actually expected behavior. I think people saying it changed stuff are the ones having a problem in the first place (reality perception, lack of sensitivity or whatever).
Just like alcohol, those things are basically a poison, that you do not die from, provided you don't overdose (you probably have more chance dying of alcohol than mushrooms but still...).
People should take them to have a bit of fun, just like getting inebriated with alcohol and moving on with their life.
Beware of people who tries to pretend more, that are most likely full of shit and you can probably ignore most things they say if they lie about that...
