This is why the "if you have a loan you're just renting from the bank" people aren't right - sure, in a way it's true but there really IS a legal difference, and it DOES come into play when things go south.
Unless they are a trespasser. Generally, you owe no duty of care to trespassers other than to avoid intentionally harming them, unless it’s a kid and you have a pool.