Exactly. I've heard this same type of criticism about Native Americans. "their towns are filthy, you will get robbed, (they aren't proper people, and being friends with them lowers your social status)".
Having experienced them, US reservations are definitely shitty places, but everything the OP said applies. Yes, they look like 3rd world counties, and passers-by attribute that to the people living there. Rather, the real reason these places look terrible is poverty and lack of investment.
I had the misfortune last year of having to attend a funeral on a reservation (misfortune in the sense that my friend died), and there was a stark line between the ranchland surrounding the res and the res itself.
I can't speak to Europe, but even in NA it really isn't about color of skin. I'm of French Mediterranean background, so during the summer I'm browner than most of my Hispanic friends. Doesn't matter, I 'pass' as a white person, while my Hispanic friend who is whiter than me gets treated like a second class citizen. It's fucked, but in America I don't actually think it's about race, americans just hide classism behind race since racism is more acceptable than classism.
Edit: not to say that reservations are unequalled places of filth. In my experience, even if you aren't native people are willing to help out. I was cross-country traveling on a garbage old Harley that needed help every 75 miles, and I had the most generous offers when I was in Native country vs. anything else. This agrees with the OP, in these places people are more willing to help out. For example, in a reservation town in SW SD my magneto died. I struggled with kicking the bike for 10 minutes, somebody eventually saw I was struggling and offered to call their cousin with a truck. Towing the bike behind a truck got it to fire, and everyone involved refused money (common response was "you'll need it later"). I get I was driving an absolute crap piece of machinery, but the reservation towns were much more helpful than other towns that I broke down in over that trip (and I broke down in a lot, I was driving a 71 harley sportster, they weren't even reliable when new).