I think the difference matters. "I'm not living with those people in charge" and then fighting for the better part of a century to throw them out, gets less sympathy from me than "they threw me out".
On the other hand, I have some sympathy for Cuban exiles, so maybe I'm inconsistent...