Easy to say when defending your point on Hacker News. We're already talking about security, and about someone with malicious intent having access to your phone. In the context of this conversation, you're already in simulated high security mental mode.
It's completely different when you're relaxed (maybe having some fun), and the person asking for your phone is someone you know.
In a relaxed social environment, you will probably not be that rude to someone you know.