I've interviewed people who, though they stayed the whole interview, ultimately declined the position. I still have much respect for them because I could see that they are capable and motivated, and were I to encounter them again (which is likely in this industry), I'd be more than willing to put in a good word for them if it would help. Now if someone had up and left halfway through? Well, human nature and emotions are tricky things. Are they within their rights? Of course. Will it strike at my ego or make me feel rejection? Absolutely. Will it form a low level resentment, no matter how minute? Unfortunately, yes. Anyone saying otherwise is simply deceiving themselves. Am I in the wrong for this? Yes, but it happens anyway.
You're dealing with humans, not machines. Maximize your advantages by being polite and civil and bending a bit even when you don't have to. It's a small world in tech.