The tricky thing about that is it depends "what period of time" you choose to look at. When the Serbs had the upper hand in the 90s, yes - a lot of the aggression was due to their actions... But if you choose to look 50 years before that, it was the Croat ustazi allied with Nazis slaughtering Serbs wholesale, to the point where even the Nazis themselves were appalled... A little while before that, it was the Bosniaks/Muslims impaling everyone else with the backing of the Ottomans.
It's more accurate to say "whoever had the upper hand at a given time" was using their temporary advantage to terrorize the others over the last couple centuries.
Given this, it's easy to understand why Serbs wouldn't want their friends and families living in states administered by people who were massacring them with the backing of Nazis and/or Ottomans within a generation.
It doesn't justify the atrocities of the Milosevic era, and it's still technically correct that "yes, the Serbs were the lone bad guys" but only if you choose to look at a certain decade and pretend history doesn't exist before that: which is very much how the American news media at the time "sold it" to justify U.S. involvement in the region.