Science isn’t the club; academia is.
Normally, when we say “scientist”, we mean “academic scientist”—i.e. someone who does science specifically for the enculturated ideals of academia (e.g. the advancement of common knowledge) using the process of academia (e.g. writing papers and sending them off to journals, attending conferences, etc.)
(You might define “academic scientist” in other ways, with something about a treadmill of papers and grants, but I would point out that any definition of academic science has to include tenured science professors, who aren’t subject to the same forces.)
Since academia (i.e. the global cooperation of academics to further knowledge by writing, reading and reviewing papers) is seen, by academics, as an unalloyed good, they see scientists who don’t take part in said academia to be doing something strange, something perhaps sub-optimal for the furthering of scientific knowledge.
And I feel like that’s maybe true, but also often misjudged: many “citizen scientists” are actually academic scientists. They may not be funded by a university or submit papers to Nature, but they collaborate and are in constant communication with people who do, such that their work isn’t advancing the boundaries of science any less optimally than their own work is.
Just the first example off the top of my head: Destin Sandlin, who some might call a “citizen scientist” (he’s a YouTuber who does science journalism but also often his own scientific experiments) works more heavily with “academic scientists” to do his experiments, than most University grad students ever even bother to do. He might be a “travelling” scientist instead of belonging to any particular institution, but he’s certainly an academic through-and-through.