Yes but you can detect that. The enclave can sign a statement that you could verify with a public key for that enclave. So if you replace the hardware there is no way to do that anymore as you cannot extract the private key from the original enclave.