That's not really much of a defence though, is it? British construction workers have decided for themselves which jokes are normal, which are funny, which jokes cross the line. Their tastes are biased in a very sexist way (probably more to do with subjective bias than actual misogyny, but still sexist in content).

Think about your list of things to joke about. If you're not a straight, masculine, white, English man then they joke about whatever makes you different. Which means sexist, homophobic or racist jokes. If you are a straight, masculine, white, English man, then, ha, ha, you like Manchester United. It's not ha, ha, your mum's got cancer, or ha, ha, you lost your wife in a car accident. No, hurtful jokes only get made to people who are different. People who are like the majority get made fun off for having different taste with regards being a sports fan.

Just because it's a tradition, and they don't mean it, doesn't mean it's okay.