I think that most people are attached to the idea that humans have some type of "soul", even if they don't call it that. This soul, as the thinking goes, is completely separate from material reality, and allows people to have ideas that come "out of nowhere" and are not inspired by anything in particular. In this way, humans are supposedly different from machines, whose outputs are a mere function of their inputs.

This is the root of the objection to AI. To say that humans are similar would imply that a person's thoughts are a function of their sensory experiences, and that is not a popular view.

Similarly, people like to think that a person's behavior is not a mere function of their experience- that they are capable of "choosing" how to react to their circumstances. They will accept no explanation that appeals to material conditions or biology. A man steals bread not because he is poor and hungry, nor because of some chemical reaction in his brain, but because he "chose" to steal. The part that chooses would his "soul", which cannot be explained in terms of cause-and-effect.