There are cases where AI can recognise gender on an X-ray when humans can't, find tumors that experienced doctor's can't. This must mean that human doctors looking at Xrays use just boring pattern recognition and AI has actual concepts of what it's seeing.
But does it really? Or is it more observant than a human doctor and more thorough, but only at the limited task of deciding if this X-ray looks like the million other X-rays of a male abdomen versus the million X-rays of a female abdomen.
I assume counting the number of ribs is not what is meant …
“We found that even state-of-the-art models which are optimally performant in data similar to their training sets are not optimal — that is, they do not make the best trade-off between overall and subgroup performance — in novel settings,” Ghassemi says. “Unfortunately, this is actually how a model is likely to be deployed. Most models are trained and validated with data from one hospital, or one source, and then deployed widely.”
It's simple math. If the correlation between gender and sex is 0.99 then if a method can determine your sex with say a 90% accuracy then it can determine your gender with an 89% accuracy (very roughly). The difference is negligible.
