I mean think about it. A lot of girls say to some dudes like "If you were thirty feet taller I'd date you" or something like that. How do you feel that is different from "I'd date you if you had fair skin"
I mean, taking the example above, we can see that the other person is being shallow, yes, but both of those characteristics he / she's rejecting the partner for are completely genetic.
I say it again, I feel like rejecting someone that you personally connect with due to his or her aesthtetics it's completely shallow. But at the end of the day, we all have our preferences.
That's why I feel preferring white or black color due to their skin isn't racist. If it was like "I would date you if you weren't american / irish / european / whatever" then THAT would be racist.
But skin color is just aesthetic. If someone prefers white skinned people, it's like preferring black hair over yellow hair.