there was one former boss who i felt was always a bit cold and standoffish (but always professional) with me and i could never quite put my finger on why until i did show up at one of the weekly happy hours i almost never went to and realized that he loved them and spent the whole time laughing and smiling and chatting with everyone.
since i was rarely at those and especially missed most of them my first 6 months on the job (when i had some ongoing family issues that required my non-work time and focus), i suspect he took that as a bit of antisocial behavior on my part and prevented him from really trusting me