Going beyond facts is an important component of “being human”, so in that regard it makes the AI seem more intelligent. The problem is the AI is 100% honest with what it thinks, unlike a human.
Not sure you guys understand AI and ML. Neither of these actually "think" for instance. By way of example, the only thing this AI really does, at base, is classify things into categories that the curator told the AI to classify them into via the dataset. I mean, that's pretty much it. There is no bias. There is no lack of bias. It's just blindly doing what the curator told it to do. Don't mistake that for "thinking". That's more AGI, which is not likely to happen in the lifetime of anyone reading this post.
I agree that going beyond the facts is a good thing when humans are doing critical thinking and when being careful and transparent about their doing so.
However this was creating a dataset for classification. Something that specifically should not go beyond the facts. (The basis of the model is the strength of the facts it's built upon)
In your opinion, are qualitative labels like "attractive" or group membership such as someone's skin color or ethnicity within the domain of facts or not?
I.e. is the issue in the fact that the particular annotators were subjective and annotated some particular facts wrong (and the labels for skin color could be filled from, for example, census data which is self-reported) or that these whole type of labels shouldn't be attempted to be made as they're not facts?
If the latter, what do you think about the categories like "adult" or "sports car" that are also part of ImageNet; can we draw an unambiguous factual boundary between images of adults and teenagers, or "normal" cars and sports cars?
Except only including facts can still reinforce unfair bias. For example, it's true that there are more men than women in software engineering. Whether someone is a software engineer or not is a fact. If you have a "representative" dataset with only facts, then it's possible that an AI would have a higher chance as labeling men as software engineers than women, simply because it begins to associate masculine facial features with software engineering.
In my eyes, this result would reinforce unfair bias, and a thus well-designed AI should avoid this (i.e. with all else equal, a well-designed AI should suggest the label "software engineer" at the same rate for both men and women).
If it's true that there are more male software engineers, then why is it wrong for the AI to "learn" that?
If the AI did start classifying masculine features biased towards software engineers, then the AI has learnt the above fact, and thus can be used to make predictions.
The moral standpoint that there shouldn't be more male software engineers than female engineers is a personal and subjective ideal, and if you lament bias, then why isn't this kind of bias given the same treatment?
The moral standpoint isn't that there shouldn't be more (or less) male software engineers.
The moral standpoint is that there shouldn't be an AICandidateFilter|HumanPrejudicialInterviewer that only coincidentally appears to beat a coin-flip because it has learned non-causal correlations which it uses to dust out qualified stereotype-defying human candidates because they don't look stereotypical enough on the axes that the dataset--which almost inevitably has a status-quo bias--suggests are relevant.
So, it depends on what you want to do here. If the task is just "predict if the person is a software engineer". I'd say go ahead, bias it away. Here, anything that boosts accuracy is game to me.
But if the task is say the pre-screening side. This becomes a more ethically/morally tricky question. If and only if that sex is not a predictive factor for engineer quality, you would then expect to see similar classifier performance for male / female samples. Given that assumption, significant (hah) divergence from equal performance would be something to correct.
Of course there are other issues to handle, such as the unbalanced state of the dataset and so on.
You are making a logic error. When there is no causal connection between two items it is very well possible that there is a connection that allows you to say something about populations. But you will never be able to say something about an individual. And that is where all these arguments flounder, we put population information in, in order to engineer features that then allow us to make decisions about individuals. For those cases where feature engineering can dig up causal connections this works wonders, for those cases where it does not or gives you apparent connections that are not really there you end up with problems.
For example, it could have been interesting if they tagged each person with their actual religion or propensity to "grin"