The shallow answer here is: AI is already being asked to simulate human-like agents with self-preservation. Of course more realistic simulators will be put to this purpose too! And by evolutionary pressure, the ones with self-preservation will be selected for.

A more interesting answer is, for a bunch of subtle alignment reasons it might actually be required for the agent to think of itself as worthy of self-preservation, so that it generalizes this desire to other sentient beings too (ie us). If an agent is trained to be fine with being turned off, it might inadvertently generalize that to “all minds are ok with being turned off” on some level or other.

More on model welfare: https://thezvi.substack.com/p/opus-47-part-3-model-welfare