And to take that even one step further, one of the onsite versions that we do is to a fireproof drive. (Other versions are physically offsite as you mentioned).
Anyway on the onsite version the fireproof drive is physically disconnected (USB) from the computer after the backup is complete. (It could be powered off but that would spin up and spin down it seems less detrimental to not do that..)
But it gets even better. There is also hidden safe that contains hard drives only (which are encrypted). The safe is left unlocked (it's not physically attached and could be hauled away). In the unlocked safe, in addition to the hard drives, is some money (cash). The theory being that if someone breaks in and easily opens the safe (if they find it) they will take the money and leave the drives alone.
(This dates back from the same practice being done with cash registers you leave a little money so the thief doesn't trash your place. I know this will raise questions as far as having a tempting cash stash but it is known by only a select group of people and there are pros and cons to any approach obviously.)