I think racism plays a part, but there are are factors to consider as well, including victimology. The current way we are approaching race in america seems to be part of the problem.
We seem to be telling people of certain races that they are born disadvantaged, that they are somehow second-class citizens and not fully responsible for their actions or outcome of their life. We then distribute aid and provide services based not only on economic or educational status, but also on race, and we form organizations to benefit members of that race. We become outraged when their rights are violated (but no so much when the rights of members of another race are).
I don't think we'll truly eliminate racism in our society if we continue to focus on race the way we are. Instead we need to care just as much about the rights (or the violations of those rights) of a black man as we do a white man (or any other race). Race needs to not be considered when creating programs to benefit the poor or uneducated. After all, shouldn't a poor white man be just as entitled to our compassion and assistance as an equally poor black man?
Martin Luther King said "I have a dream that my four little children will one day live in a nation where they will not be judged by the color of their skin, but by the content of their character." For me, that cuts both ways. Not only should we not be judged unfairly in a court of law or be treated different by police because of the color of one's skin. It should be a non-factor in determining any sort of benefit or assistance as well.