Imagine a device that you put on your head and press "record". After 5 minutes, you press "stop" and then I put it on my head and press "play". I then experience those 5 minutes as you experienced them. You could also replay your own 5 minutes and confirm the recording is accurate.

That device doesn't exist today -- it's sci-fi -- but there's no known law of physics that forbids it. If it is built it would be a lot of progress towards, perhaps even a solution for, the hard problem.

The only realistic way we either build that device or prove that it can't be built is science (probably neuroscience). So, my opinion is that, to a large degree, when we opine about "subjective nature of experience" we're bloviating a bit. Our experience is subjective now but there's no law of physics that says that must always be true.

Back in the XVII/XIX century, a similar problem existed regarding life - the problem of "what makes living things tick". The assumption at that time was that while we can understand the biological processes around life, we will never understand the so-called "vital force", which causes things to live - life itself. I know it sounds weird now, but back in the day the mental models were different. Phenomenas like "water boils" and "organisms self-replicate" were treated as completely different domains of reality, without an overarching uniform scientific model.

It turned out that after around 100 years, we can figure out the chemical/physical processes and the need for the term "vital force" became redundant.

While this is certainly not an argument proving that the Hard Problem is not in fact hard, it is an interesting idea to think about. Perhaps its all a matter of developing better, higher-resolution neurological models which will at some point give us the tools to decompose qualia.

What we could do is break down other facets of what we talk about under the umbrella of consciousness, and find measurable subsets.

E.g. a lot of people insists LLMs can't reason either. Coming up with a testable definition of what that means might be doable.

Overall also just separating the subjective experience from the rest leaves us in a position where proving the possibility of AGI "just" rests on whether or not human brains exceed the Turing computable.

If they don't, then subjective experience or not is irrelevant for the question of reasoning and intelligence, as in that case a subjective experience either can't affect the computation or must itself be at least possible to fully simulate by any Turing complete system.

The problem of subjective experience then would largely be down to faith and feelings but would also be entirely orthogonal to the rest.

Billions * billions of pipes and valves can result in emergent behavior that appears conscious while at the same time the sound of a single independent water pipe can moan and sound like human speech or otherwise lifelike and evoke human emotions.

I think LLMs are doing both of these things and often people are more impressed by the independent fixtures (the moan) rather than the emergent behavior. Both the sound and the emergent behavior can be built on purpose or on accident.

I think it helps to look at this through an Information Theory lens. What information is coming into the system (the human or the machine)? What information goes out of the system which is novel? How much of this can be attributed to attempting to parse random noise aka. `Random_Imagination_Engine` vs something else? The number of inventors who come up with a breakthrough idea after mis-hearing someone is surprisingly high.

If we make the distinction between phenomenal consciousness from access consciousness we can see that LLMs clearly can make decisions based on input (A-Consciousness) but they probably don't have raw feelings and sensations (P-Consciousness).

And yes, turing equivalence is turing equivalence, I don't see why a system of pipes can't make an AI.

We can't prove it, but given the total absence of evidence of anything exceeding the Turing computable, as a hypothesis it is a reasonable one that would require truly extraordinary evidence to rise above "magical thinking".

Now, that is also far from proving they are "conscious".