I'm torn on this issue to be honest.
On the one hand, taking jobs from one person to give it to another based on race, gender, disability, age, sexuality etc... is not exactly moral.
However, the spirit of Affirmative action is to try to break down social segregation.
Of course, complaining about affirmative action denies the possibility that out of a choice of two equally qualified people, that the job could be given to the person in the less privellged position. So, if a white person and a black person were to be equally qualified for a job, just because the black person got the job, doesn't mean that it was due to affirmative action (say for example, the black person had a personal quality that was slightly better suited to the job). However, it seems that people against affirmative action in this instance might argue that it was just because of the person's race, but in actual fact are missing the point completely.
On the whole idea of having a broader spectrum of people working in the same company, it'd probably be better of image-wise.
So yeah, it's a dicey subject area to be honest. Of course, we wouldn't need affirmative action if society were already egalitarian.