Yeah, during school it's difficult since you are forced together with potentially toxic people. As an adult you can choose at least in personal life and to an extent workplace, although sometimes workplace can also be difficult to get right.
I'd 100% rather be alone than around people who might judge or use in someway against me anything about me. It would feel internally disgusting for me to think that someone might be trying to get at my expense and that I'm not around people who are there to try and build each other. What a waste of time.
