This absolutely works both ways and erases people in favor of a label.
I talk to a lot of conservatives. While many of them oppose abortion, the issue that sent so many of them over the edge is the lack of willingness of Democrats to agree on a limit. The bulk of opposition is to late term abortions.
The lack of reasonable discourse is what sets extremes in place.
Another example: Tim Scott’s police reform bill that he thought could move forward with bipartisan support. It didn’t solely because Democratic leadership wanted to ensure that it remained an election issue.
We still have no movement on an equivalent bill since Biden took office.
Examples like this make it pretty clear that the party leadership doesn’t actually want to resolve issues.
Allowing the people who benefit from division to keep doing it will only make things worse.