It sounds racist because they phrased it very bluntly, but that aren't saying that black people can't do math. One of the reasons why Americans are bad at math is because our expectations of math are abysmal. Raising these expectations will reveal just how far behind black students are in math, which will get you accused of racism, so schools are incentivized to keep them low. One can argue that this is actually a form of systemic racism because the schools that are failing to properly educate black students are shielded from scrutiny.