> but that doesn't mean the state of the program doesn't contain any understanding or intelligence
Programs don't contain understanding or intelligence, they contain instructions.
> We're not asking whether the computer chip itself understands things but whether the something contained in the program running on it does.
I feel like your saying "I'm not accusing the blender of being intelligent, I'm saying the recipe for this margarita is self aware." It doesn't matter if its hardware or software, neither is capable of understanding because understanding is a conscious experience and neither a blender nor a recipe are sentient.
> The human could also run a physics simulation
Cool XKCD but I'm not arguing about wether AI is possible. Just pointing out that convolutional neural networks are not self aware or intelligent or actually learning (at least not yet).