Upholding the Constitution
can be a dogwhistle for various malign ideas. But there are
also people who see, for example, the "living Constitution" jurisprudence as not actually upholding the Constitution, but rather just saying what you want and calling it the law. There are people who see executive orders (whether by Bush, Obama, or Trump) as not the way the country is supposed to be governed, and worry about the constitutional legitimacy of those orders. Probably the majority of the people who worry about such things are conservatives; my feel is that the majority (at a minumum) are not concerned about such issues as a cover for white supremacy.
Note well: I take no position on whether Oath Keepers is using "upholding the Constitution" as a cover for white supremacy.