Whites are actually unaffected by affirmative action. The transfer is from Asians to blacks/Hispanics. Here's some hard data:
http://www.princeton.edu/~tje/files/Opportunity%20Cost%20of%..."White applicants would benefit very little by removing racial and ethnic preferences; the white acceptance rate would increase by roughly 0.5 percentage points."