It's a Prisoner's Dilemma[0] situation. Even if one arbitrary group is inclined to behave in a unbiased way, they know that they stand a lot to lose if the other groups take advantage of the situation. The other groups knows this as well. It takes a lot of trust on all sides for the "prisoners" to cooperate and be better off for it. The group with the weakest cohesion loses, unless they all weaken at the same time. And too many interests benefit from keeping everyone tribal and mistrusting.
Your explanation begs the question. Racism is at its core, based in the idea that superficial characteristics somehow convey deeper information. Your explanation assumes that this is true, that superficial characteristics correctly signal group affiliation, something that is simply not true.
> Your explanation assumes that this is true, that superficial characteristics correctly signal group affiliation, something that is simply not true.
It is, because in a society that has racism as status quo, you can be sure that these superficial characteristics are shared between you and your immediate, as well as extended family - people who you can rely to be in your group.
What? Extended family is a tighter group than any group based on physical resemblance.
My argument isn't much changed if you recast it as the generalization from family to "race" being incorrect, because races aren't actually coherent groups in the same way that many (extended) families are.
My point is that the generalization from a real in group of people that look like you (extended family) to the group of people that look like you, is not really a useful thing to do, it mistakes the superficial information (similar appearance) for useful information.