The halting problem is a good example that I'd have to think more about. My initial feeling is that while the proof shows that you can't write an general algorithm to determine if a system halts you can still understand why that's the case from the proof itself. My claim isn't that understanding things allows you to do everything, but to find out what can be done. You're right though that knowing more won't let you violate the laws of physics or computability - it just lets you learn what the laws are.
For the physics point I think it's possible that the territory is math and that's why mathematical equations do such a good job of explaining it. I read about this idea in Max Tegmark's book: https://www.amazon.com/dp/B00DXKJ2DA/ - in that case we could be discovering pieces of it (though our syntax and such is still just a map to it). Until we have a grand unifying theory it seems worthwhile to keep working on the map. Maybe I'm just optimistic about this.
You lose me in the next section though - it seems like we agree on the soft problem, but I don't understand your classification of the hard problem.
Why is it difficult to think of a physical process leading to subjective experience? Natural selection towards a general problem solving organ that provides feedback etc. Visual inputs, audio, the ability to make predictions and act based on those - subjective experience seems like a natural outcome of all of that?
I think you and I probably agree that 'emergent phenomena' is mostly hand-wavy nonsense (http://lesswrong.com/lw/iv/the_futility_of_emergence/).
> "but as many wise people who have meditated long and hard on this matter have suggested, the nature of consciousness exists prior to concepts, prior to the division of reality by the mind into subjective and objective and therefore prior to (and so beyond the scope of) scientific inquiry."
I don't understand what leads to this conclusion - your last sentence seems to imply dualism which I think is generally pretty weak since poking someone in the brain, a brain tumor's affect on consciousness, and LSD's affect on it provide decent evidence that it's material. I don't think this is a real problem, it feels like one of those that disappears once we understand how things actually work.