The people who curated the first training set used subjective words like 'attractive' to tag the images which means the AI tagged all images it deemed 'attractive' to the people who made the training set. as this is a very biased and homogenous group it means the AI turned out biased. Maybe if they randomly sampled millions of people from all countries in order to create the training data then they could effectively train an AI to guess what YOU might find attractive. However even then I somehow doubt it. Beauty is in the eye of the beholder. We dont consiously know the rules of what we are attracted to, nor does an AI have secret information if you just supplied it with enough words and images.
If they'd stuck with simple classifications like 'black' 'white' 'man' 'woman' then they would have less subjective judgement values about the original training set.
Data is data and cannot make value judgments so not sure how it can be racist. If the data is how racist people label things it still is not racist data it is data is perfectly valid for what it is racists. Removing the images from ImageNet seems absurd.
In practice, "data" means a set of observations collected by humans- who have inherent biases that influence the collection.
I'm not talking specifically about cultural biases like racial stereotypes etc. Confirmation bias is a thing, there's nothing stopping a researcher from making those observations that confirm their favoured theory and contradict all others.
Then of course there is sampling error. Just because you have a set of data that you collected "at random" doesn't mean that this dataset is representative of the population you are interested in. Let alone the fact that it's very hard to collect a truly random set of observations about processes that we don't understand to begin with.
The kind of data you're describing is an ideal, a principle that we all aspire to. It's far from the reality in practice.
I mean, it's very common colloquially to describe non sentient things as racist because they're based on either purposely or obliviously racist ideas and stereotypes.
Do you hear yourself ?! ImageNet was not based on either purposely or obliviously racist ideas and stereotypes. If it was, sure, I would have some patience for the claim it was racist. But it was not.
Going beyond facts is an important component of “being human”, so in that regard it makes the AI seem more intelligent. The problem is the AI is 100% honest with what it thinks, unlike a human.
Not sure you guys understand AI and ML. Neither of these actually "think" for instance. By way of example, the only thing this AI really does, at base, is classify things into categories that the curator told the AI to classify them into via the dataset. I mean, that's pretty much it. There is no bias. There is no lack of bias. It's just blindly doing what the curator told it to do. Don't mistake that for "thinking". That's more AGI, which is not likely to happen in the lifetime of anyone reading this post.
I agree that going beyond the facts is a good thing when humans are doing critical thinking and when being careful and transparent about their doing so.
However this was creating a dataset for classification. Something that specifically should not go beyond the facts. (The basis of the model is the strength of the facts it's built upon)
In your opinion, are qualitative labels like "attractive" or group membership such as someone's skin color or ethnicity within the domain of facts or not?
I.e. is the issue in the fact that the particular annotators were subjective and annotated some particular facts wrong (and the labels for skin color could be filled from, for example, census data which is self-reported) or that these whole type of labels shouldn't be attempted to be made as they're not facts?
If the latter, what do you think about the categories like "adult" or "sports car" that are also part of ImageNet; can we draw an unambiguous factual boundary between images of adults and teenagers, or "normal" cars and sports cars?
Except only including facts can still reinforce unfair bias. For example, it's true that there are more men than women in software engineering. Whether someone is a software engineer or not is a fact. If you have a "representative" dataset with only facts, then it's possible that an AI would have a higher chance as labeling men as software engineers than women, simply because it begins to associate masculine facial features with software engineering.
In my eyes, this result would reinforce unfair bias, and a thus well-designed AI should avoid this (i.e. with all else equal, a well-designed AI should suggest the label "software engineer" at the same rate for both men and women).
If it's true that there are more male software engineers, then why is it wrong for the AI to "learn" that?
If the AI did start classifying masculine features biased towards software engineers, then the AI has learnt the above fact, and thus can be used to make predictions.
The moral standpoint that there shouldn't be more male software engineers than female engineers is a personal and subjective ideal, and if you lament bias, then why isn't this kind of bias given the same treatment?
The moral standpoint isn't that there shouldn't be more (or less) male software engineers.
The moral standpoint is that there shouldn't be an AICandidateFilter|HumanPrejudicialInterviewer that only coincidentally appears to beat a coin-flip because it has learned non-causal correlations which it uses to dust out qualified stereotype-defying human candidates because they don't look stereotypical enough on the axes that the dataset--which almost inevitably has a status-quo bias--suggests are relevant.
So, it depends on what you want to do here. If the task is just "predict if the person is a software engineer". I'd say go ahead, bias it away. Here, anything that boosts accuracy is game to me.
But if the task is say the pre-screening side. This becomes a more ethically/morally tricky question. If and only if that sex is not a predictive factor for engineer quality, you would then expect to see similar classifier performance for male / female samples. Given that assumption, significant (hah) divergence from equal performance would be something to correct.
Of course there are other issues to handle, such as the unbalanced state of the dataset and so on.
You are making a logic error. When there is no causal connection between two items it is very well possible that there is a connection that allows you to say something about populations. But you will never be able to say something about an individual. And that is where all these arguments flounder, we put population information in, in order to engineer features that then allow us to make decisions about individuals. For those cases where feature engineering can dig up causal connections this works wonders, for those cases where it does not or gives you apparent connections that are not really there you end up with problems.
That illustrates the depth of the issue - while directly racist data can possibly be removed there are many proxy/correlated attributes (as any insurance/mortgage/etc company knows), and to find correlations is the core nature of the machine learning systems (at least the ML as it is currently known to humans).
Tangently related, this reminds me of the women who had her picture photoshopped by people from different countries to make her look beautiful in their eyes:
I think it's important context that "had her" means "she initiated", rather than "was imposed upon her":
"Make me beautiful," she said, hoping to bring to light how standards of beauty differ across various cultures"
She specifically asked for some kind of alteration, so leaving the picture unaltered was implicitly not an option. Furthermore, this assumes a random (but distributed) selection of Fivver artists represent worldwide beauty standards.
This is pretty close to "does the Chinese room know Chinese". https://en.wikipedia.org/wiki/Chinese_room Going too deep down that path is fun for philosophy, but not super useful...
> If they'd stuck with simple classifications like 'black' 'white' 'man' 'woman'
Interestingly, I was just thinking about the "black and white" issue today. A while ago I watched an interview on Youtube with a young woman who was born in Japan. Her parents were American and growing up she new she was different, but basically it boiled down to "the English teacher's daughter", or "part of the foreigner family". But she didn't speak English very well and her parents didn't speak Japanese very well, so she identified very much more strongly with her friends in the area than her family.
When she was 12, her family moved back to the US. When she went to school the new children were encouraged to say what nationality they were. One person said they were Canadian. One said they were Mexican. When it came to her turn she said, "I'm Japanese". One child in her class corrected her: "No you aren't. You are black." Of course, this was a source of considerable confusion for her.
One of the things that's kind of weird about Japan is that some Japanese people have very dark complexions -- darker than what would be called "black" in the US. Some have very light complexions. In my opinion, considerably more "white" than I am (and most people would call me "white" I think). In fact, after I moved to Japan, I realised that I wasn't white at all. I'm pink. I mean, I'm super pink. I seriously never noticed it until I spent 5 or so years living in a place where nobody else was pink. I literally avoid wearing red now because it makes me look like a tomato.
When I was about 3, my best friend was black. My grandmother asked me, "Do you notice anything different about your fried? Their hair or something?" I didn't understand the question. All of my friends had different hair. Then she said, "Can you tell that he is black and you are white? Or do you not think about it that way?" My grandmother was just curious, but this question completely blew me away. From that time, I realised that people didn't each have their own skin color. Instead, they were categorised and my friend was different than me. I think my friend noticed that I looked at him differently (though not necessarily badly). We stopped being friends for a long time. Somewhat strangely, I just recently realised that he was one of my best friends in high school... I never actually made the connection that my friend at 3 and my friend in high school were the same person until recently. I wonder if he ever realised.
But anyway, there really isn't a classification like "black" and "white". When I have a tan, my skin is darker than my wife's. But she is very tan and so her skin looks brown. When I compare my skin to hers, my skin is still red, even though it is dark. But if I were to compare my skin to an indiginous American, I think their skin will be more red than mine when tan -- and less pink when not. And my wife, when compared to someone indiginous from Africa has more a more of a curry brown than a deep chocolate brown.
As I was saying, Japanese people don't call dark Japanese people "black". They don't treat them as a different race. Neither are very pale Japanese people "white", even though they may say "your skin is very white". They aren't different. What I found interesting about the young girl who discovered that she was black was that she wasn't "black" in Japan. Or, at least, not "black" in the way we use the term in America. Japan does not have that cultural history (it's got enough of it's own baggage, thank you very much ;-) ).
If a computer were to compare skin tones objectively, it would simply tell you the color. If it decided to classify in terms of "black" and "white", it would be classifying based on cultural labels, not color.
Perhaps instead of using human categories of race, we should ask the neural nets to provide those categorisations?
It would be based on a larger selections of individuals (non-local), based purely on appearance, and un-biased by human conception (we are really good at processing faces/facial features).
The people who curated the first training set used subjective words like 'attractive' to tag the images which means the AI tagged all images it deemed 'attractive' to the people who made the training set. as this is a very biased and homogenous group it means the AI turned out biased. Maybe if they randomly sampled millions of people from all countries in order to create the training data then they could effectively train an AI to guess what YOU might find attractive. However even then I somehow doubt it. Beauty is in the eye of the beholder. We dont consiously know the rules of what we are attracted to, nor does an AI have secret information if you just supplied it with enough words and images.
If they'd stuck with simple classifications like 'black' 'white' 'man' 'woman' then they would have less subjective judgement values about the original training set.