It's hard for people to understand how deeply woven caste-based motivations are in Indian society. At a large scale it may seem like some globally stratified layer cake. On the ground, it devolves into regional tribes.
The "caste associations" rural people associate themselves with in rural areas are loosely formed tribes of 10 thousand to 100-thousand strong members.
The perspective of many who choose to be invested in this system - which is most people living in those areas (what else are you gonna do? not support your tribe? who else is gonna have your back if there's trouble?) - is one of trying to gain status for their caste/tribe over others.
Caste rivalries are often regional, and localized.
Identifying India's current PM as not coming from one of the well-recognized "higher castes" is really meaningless. It'd be sort of like saying "yeah, this politician is unlikely to be a hardline christian dominionist, he's a 7-th day adventist, which is clearly a minority sect".
This does not change anything. Modi is just a pawn. He does not have any original ideology, he is there for power.
Look at the number of brahmins and non-brahmins in the key position in the BJP. Look at the analysis here https://theprint.in/politics/ambedkar-on-its-agenda-but-bjp-...
If you are still not convinced, look at what "Subramanian swamy" had to say when every one in BJP party was changing their twitter profile last name to "chowkidar"(meaning soldier or protector). He said he is a brahmin and its brahmins duty to advice how the country should be run and not be a soldier as vedas consider brahmins as superior to all the other varnas.
https://www.indiatoday.in/elections/lok-sabha-2019/story/can...
The point is the ideology that is driving BJP is all about bringing back the privilege based on the birth.
I don't expect him to be post-modernists egalitarian.
However at the same it doesn't mean he can't say his casteist comments openly. ;)
After all, its better if people say it openly so that they can be criticized for their casteist comment. The likes of him is politically insignificant as he is NOT competing in elections.
Don't just rely on Indian fake news media.