If we're only talking about exploiting a device across a network, sure turn it off or disconnect it from the network. But there's more to security than that.

One can always take the device and turn it on for oneself.

If one can't exploit the device, one can resort to rubber-hose cryptanalysis.