You can't be that surprised that in a situation like this, most people would tend to be very conservative (not republican conservative, but careful, do no harm, don't rock the boat kind of conservative).
Most people are "conservative" on this issue because they have no idea what they're talking about, just like every almost every other issue related to trans people.
My mom said something similar about "curiosity" to try and convince me that I couldn't be trans. As it turns out, what she meant was "when I was your age, I wanted to cut my hair short." Likewise, pretty much every story I've heard like this turns out to be either obvious lies meant to discredit trans people, or something that would have been easily sorted out by a psychologist. Eg. I've heard of gay people wishing they were another gender so that they wouldn't get bullied for their orientation. No decent professional would mistake that for genuine dysphoria.
But even supposing that your psychologist was incompetent, you still would have socially transitioned in some capacity before you ever started hormones. This almost certainly would have let you know if transitioning was really right for you. And even if it didn't, and you went all the way to starting estrogen, you could always stop if you didn't like what it was doing. Estrogen works very slowly. I've been on it for three years, and I know cis men whose boobs are bigger than mine. If I changed my mind, I could easily start living as a man again.
It's absolutely unfair to impose life-ruining misery on trans teens because of the off-chance that it might do a fraction of the same harm to a cis person.