I have considered the same thing. I've never really liked milk. Chocolate milk was always worse - it tasted like someone mixed something rotten with a chocolate flavor. I always think milk smells like it has gone off. I've never had an upset stomach or other gastric symptoms from dairy like my sister, who is lactose intolerant. I always kinda figured I wasn't.

Then I tried some lactose free chocolate milk, with no sugar added. I think this was the first time I've actually enjoyed milk. I still do not drink milk often, but now I will actually drink it.