I'm not disputing that it is used that way in certain circles, but it isn't common. Also, ladyboy is often used as a term of hatred and abuse against transsexual people, so if you keep using it like that eventually you're going to be in a situation where you come across like some old guy casually talking about "niggers in the woodpile" and that would be bad. (I'm not saying it's equivalent, just that it could land you in equivalently hot water. You know what I mean.)