Because race really matters when your worldview is all about creating post-facto equity between races, not equal treatment and opportunity.

When you have an a-priori assumption that every difference in outcome is due to oppression of some sort, the argument is over -- you've already allowed that discrimination on any basis is OK, as long as it's against whoever you've labeled the "oppressor" group.

As a final coup de grace, you can label anyone who disagrees as an "oppressor" (or at least "fragile") without engaging their argument -- it's all circular reasoning supported by an ad-hominem fallacy.

Respectfully, you didn't really answer my question at all. The question, which I'll phrase slightly differently, is "why can't a person who's not from a particular race or group (i.e., an outsider) observe that people in that group are treating each other poorly, unfairly, or unjustly?"

What you said, on the other hand, is a bunch of familiar, well-worn complaints dating back to the 1990s from privileged white people who think they're being oppressed, even though that's not even the subject.