It's definitely a matter of getting used to it. I moved from Canada where contactless has been standard for a couple of years now and being unable to get a contactless UK card for now I'm super annoyed every time I have to punch in PINs to pay £1 - £3 for something small. The extra 30 seconds just feel totally unnecessary.
A few coins are much heavier and thicker than a card, for one. Just tapping a card is faster than picking out coins then waiting for change. And when paying with card I know I won't get any two pence coins back.