Regardless of Ted Chiang's qualifications as a cognitive or computer scientist, he's correct. Claiming that a computational model is conscious falls squarely in the category of "extraordinary" and as Carl Sagan said, "Extraordinary claims require extraordinary evidence." Because the claim is extraordinary, the burden of proof is on the claimant of consciousness, not on those who deny that LLMs are conscious.

The fact that LLMs remind us of consciousness means nothing. Eliza made people believe they were talking to a conscious intelligence, and the code for Eliza was less than 1000 lines long [0]. It doesn't take much to fool humans. Ask any magician.

Claiming that LLMs are conscious is tantamount to watching a Penn and Teller performance and claiming they have tapped into paranormal powers. It's an idiotic claim. And it's embarrassing coming from people who supposedly understand how science works.

[0] https://sites.google.com/view/elizaarchaeology/code