Agreed, I'm Indian and thus often it is the case that other Indians try to speak with me in Hindi, but I end up having to insist on English in certain contexts.
It's unnecessarily rude and exclusionary towards others to speak a language others cannot understand in an environment where they have an interest in participating. All it does is form cliques and encourages toxic dynamics to form.
But on the other hand, I previously encountered the opinion that people who insist on speaking in English (especially without the "quirks" and accent of Indian English) despite being able to do Hindi are just trying to be pretentious.