More to the point, “procedural fairness” or “equality of opportunity” (if I understand it correctly) is directly opposed to “representative fairness” or “equality of outcome”. In many situations, prioritising one will compromise the other. As an example, Harvard admission of Asian students.
Perspectives differ on which one is more important. Personally, I’m absolutely for the former, but it increasingly seems I’m a part of the minority (or the silent majority).
I think the most useful way to think about this is that unequal outcomes almost always lead us to unequal opportunity upstream.
If, for example, you made admission to elite colleges(which is a whole other can of worms) ignore ethnic backgrounds, you'd get a result where far fewer applicants of certain ethnicities would be accepted. In an arbitrary sense, this could represent "equal opportunity" if you believed that these people materialized from the ether the day they submit their application. But if you take into account their lives leading up to submitting their application, "equal opportunity" requires counteracting powerful structural factors that make it harder for people of some backgrounds to get into college.
The reason Harvard admission of Asian students is cited to such a tiresome degree is that it's a rare case where there isn't a clear upstream justification. That's why there's a big lawsuit and Harvard is getting tons of awful press about it. But it also lines up with the fundamental problem we're trying to fix with affirmative action. Harvard and its admissions systems were invented by white people, and they have proven to relentlessly favor white people, who then have easier lives and more money they can use to pay SAT tutors to help the next generation of white kids get into Harvard. It's not surprising that Harvard would warp the affirmative action system away from its original purpose, to correct for historical oppression, in order to... help white people get into Harvard. The fact that Harvard is discriminating against Asians is not a reason to implement a fake-fair system that has the end result of discriminating against all other non-white people.
It's theoretically possible that there are some other cases somewhere that show large inequality of outcome but have no unfairness upstream. But I'd rather have the task of discovering and fixing those rare cases than the present system, in which being born with a certain background gives you radical advantages or disadvantages on the basis of, basically, ancestral violence.
Can you please explain how Obama’s daughters are disadvantaged based on ancestral violence?
The point is, racist discrimination, even if “positive”, produces unfair outcomes. Obama’s daughter don’t need even more advantages. If you want to help kids from disadvantaged backgrounds (and I think that’s a worthy cause, even if you ignore “fairness”, simply because it increases the odds of finding the next Ramanujan and improves the well-being of the society), you should help people from disadvantaged backgrounds. Admitting them to degrees they don’t deserve isn’t actually helping them - it’s mainly just fixing superficial statistics and perpetuating harmful stereotypes (e.g. people not wanting black doctors because they suspect they’re worse because they weren’t subject to as strict standards as white, let alone Asian, doctors). To ensure actual equality of opportunity, you need to help them when they’re young, improve their school system, help their families, etc. Everything else is just a bandaid, at best masking rhe underlying problem.
Yes, that's very easy! Let me count the ways. First of all, Obama's daughters are much more likely to be murdered, abused or falsely accused by police than they would be if they were white. Even iconically famous black people are sometimes mistreated by police officers who don't recognize them. I'd say being at a high risk of murder counts as a disadvantage! I don't even need to draw a line back to ancestral violence for that one—that's just regular old present day violence.
Second, although the Obama sisters enjoy substantial family wealth due to their parents' incredible achievements, the fact that they were born black means that, statistically speaking, they would have been far wealthier if they'd been born to a comparably wealthy white family. For example, if the Obama's are the X richest African American family, they'd be far richer if they were the X richest white American family. Bingo! Disadvantage! Why does that wealth gap exist? There are many recent crimes, many tied to violence, that exacerbated the situation. But we can start by looking at the fact that Black people, including Obama ancestors, were violently forced to build wealth for white people without any compensation.
There are many more ways in which contemporary American society puts extra pressure and psychological burden on African Americans, yes even rich African Americans, all of which can be traced back to historical violence in our history, and all of which clearly affect the outcomes we associate with "success."
Whether you admit it or not, you're arguing that the person who "deserves" to win in our economy is the person whose ancestors perpetrated horrific crimes against his competitor's ancestors and who doesn't have to face the constant daily effects of ongoing racism in our society.
> First of all, Obama's daughters are much more likely to be murdered, abused or falsely accused by police than they would be if they were white.
I don't think this is true. If you compare Obama's daughters against Bush's sons, probably. But against a random white person with no media exposure, unlikely.
> they would have been far wealthier if they'd been born to a comparably wealthy white family.
Again, why are we comparing against comparably wealthy white families? The reason why Obama's daughters were brought up in the first place is because we recognize that they are incredibly privileged compared to the general population, and they don't need more advantages. We don't care whether they are privileged compared to Bill Gate's children or someone comparatively more privileged.
If you want to understand how anti-black bias affects even wealthy African Americans, you have to use methods like this. The GP was saying, basically "if the Obama's succeeded, it doesn't matter that their competitors were cheating." You can be disadvantaged and still be successful.
Why would it be useful to compare the most famous and rich African Americans against "a random white person?" Is that the way we prove that African Americans are not disadvantaged, by comparing an anomaly from one group to the average of another? We are comparing comparably wealthy people in order to eliminate other forms of difference so we can see just the effects of ethnicity...
What if Obama's daughters don't find being more likely to be murdered to be a worthwhile exchange for whatever advantages you're offering them? It doesn't seem like a fair negotiation to not include them.
We are talking about large, diverse groups and their median or average.
That said, usually the correct approach is not discriminating based on race, but discriminating on the actual "disadvantegedness" of the path taken by the applicant.
Accidentally that's how it's done around here for university (you get points for being disabled, poor, living far from the campus, and so on).
It's great to account for what structurally led to the moment of a decision. What's tricky is that it's hard to get the right level of granularity. If you don't have data on how historically disadvantaged a given person's path was, you're going to get false positives and false negatives, and you can't make the system fair across both.
Changing the discussion to be structurally fair, not just transactionally, you still have the same impossibility issues arising if you don't have an oracle model with omniscient features. It's the same issues, but with a larger scope of features and outcomes.
with regards to Harvard et al., they have a harder time to do positive discrimination since some court decisions in recent years. all in all, they look at extracurricular stuff to try to give extra points, because they can't discriminate - even positively - based on race, as far as I know. (the legal history of affirmative action is pretty long: https://en.wikipedia.org/wiki/Affirmative_action_in_the_Unit... )
The problem is equality of opportunity is it's invariably applied in a way that ignores some very racist and uneven starting conditions and the effects of the past. Consider as a small example how schools are funded in the US, which is largely by local property taxes, so poorer places have poorer schools. These areas are heavily skewed towards minority populations because of some extremely racist issues in our past like redlining which kept blacks and other minorities out of the housing boom which drove a lot of middle class wealth growth through the 60s to 00s. So we build on this cycle generation after generation where kids in poor areas go to poorly funded schools and experience poor educational outcomes.
Even if we solve the funding problem it's still not actually equal opportunity because the poorer kids don't get the same support outside of school due to parents working longer hours at lower paying jobs because they got a bad education or living in a crummy house with lead paint (because that's all your parents could afford) or living in a town with contaminated water (Flint is just the one to grab headlines cities all over the country have abysmally high lead levels). Even if you want to blame community culture for some of these issues the question becomes where do you think that culture comes from, perhaps just maybe it comes from decades of lack of opportunity and systemic abuse?
How do you possibly hope to close the gap in our system where wealth almost invariably builds on wealth?
tl;dr: Equality of outcome doesn't mean everyone has to have the exact same exact life, it's just acknowledging our opportunities are directly tied to the outcomes of the generation before us.
I know of several school districts nearby that contain nothing but relatively new $1mm+ houses and abysmal school ratings. I would believe that there's some kind of complex nonlinear correlation between property values and quality of school, but it is very clearly not an easy "more expensive houses == better schools" calculation.
> perhaps just maybe it comes from decades of lack of opportunity and systemic abuse?
A fine point, and certainly not limited to any one race. I'm all in favor of helping those with fewer opportunities growing up, and I don't really care what race they are. If they happen to be mostly $RACE, it doesn't matter to me.
My point was principally about the issues with 'equality of opportunity' in the US because of the massive inequality already present.
On your note though yes with unlimited money we could do that, given limited budget though targeting the assistance to those most affected makes sense. Also a program like that also does little to nothing for people who aren't very young when it's started. Also ultimately that's an equality of outcome style solution because it's giving additional resources to some.
(this is more about things like the Harvard affirmative action program) I think there's also a meaningful distinction to draw between inclusive discrimination like affirmative action and exclusive discrimination of groups. I know it's an extremely tough line to define at a government level but they feel like very distinct categories of activities.
> meaningful distinction to draw between inclusive discrimination like affirmative action and exclusive discrimination of groups.
The only distinction I see is in the rethorical spin that people make. A preferential treatment of a set of people is exactly equal to a discriminatory treatment of the complement of that set of people (e.g. positive discrimination of black people equals negative discrimination of non-black people).
"The problem is equality of opportunity is it's invariably applied in a way that ignores some very racist and uneven starting conditions and the effects of the past."
Then that's where the energy ought to go. Adjusting the end result (e.g. by adjusting the college admission criteria based on race/ethnicity) doesn't really fix the root cause; it's a band-aid on America's broken leg.
If the starting conditions are unequal, then that is by definition not equal opportunity, so correcting those conditions should be Priority 1.
Sure he did great doesn't change the fact that individual examples like Frederick Douglass don't negate that fact that intergenerational incomes are pretty inelastic in America. It's a statistical problem across all slices of the population.
I am also interested in equality of opportunity- but how do I measure that everyone has an equal opportunity to achieve something? How do I measure the opportunity to consume nutritious food? The opportunity for access to the same healthcare?
It goes deeper than that if you go into cultural biases prevalent in poor (regardless of race) communities. Cigarettes, scratch-offs, crappy food.. what counts as opportunity?
Is being raised a certain way tantamount to having 'less opportunity' to make better micro-decisions?
In order to create a situation where people have equality of opportunity, it's often necessary to ensure that the individual had an equal outcome that creates the necessary starting state.
Conflating those two inherently different concepts for the sake of convenience will not net good systems. Equality of outcome is NOT a substitute for equality of opportunity. Even if attempting to create as equal as possible of a starting state has some surface similarities with equality of outcome, it's important to maintain the distinction.
Doesn't this mean Harvard should accept everyone? If equality of outcome is important - and I'd argue that it is to a much greater extent than most people would consider acceptable - doesn't it stand to reason that we should care about it at the level of individuals, not merely populations? What's strange to me is that people seem to think that equality is only important when talking about small differences across arbitrary social constructs, while entirely ignoring much larger gaps in individuals.
In theory I agree but in practice that often leads to racism. For example, some people support discrimination for blacks (& against whites) at e.g. college admissions, purportedly to offset their (statistically) worse family background (in terms of education and wealth). Clearly, that's a terrible and racist solution, with the better and obviously correct solution being... to support people from worse family backgrounds! Instead of targeting the proxy for the problem, you solve the problem itself.
The flaw with this argument is that if you recognize that for example, black people are generally worse off in family background than white people, then you end up back at the point where you on average offer more aid for black families than white ones which by your argument would be racist.
You're not solving the problem at all by attempting to sweep racism under the rug; quite the opposite in fact because it fails to address how race affects family background and discrimination in the first place.
You seem to have misunderstood my argument; maybe I wasn’t really clear in that part. I think a policy that helps lots of poor black people, some poor white people, and doesn’t help Obama’s daughters (because they don’t need help), is a good, fair, and most importantly effective policy. A policy that doesn’t help poor white people is racist. A policy that helps Obama’s daughters is ineffective (and I don’t want my tax money to go to that cause).
A policy that affects one group of people more than others by racial lines sounds like a racist policy, don't you think? Which was my entire point. You're in effect, arguing for a policy that is racist by your own prior argument.
We have affirmative action policies and the like because we admit that some groups of people are disadvantaged due to a multitude of circumstances and we attempt to do things in such a way as to ensure that they can mitigate those circumstances.
I think you're conflating the two different definitions of racism here. In one, something is racist if it has different impacts on people of different races - not due to their race, but due to some correlated variable. In another definition of racism, something is racist if it uses race as a determining factor in behavior or treatment.
Providing aid to low-income families is racist by the first definition, but not by the second. But (damn near) everything is racist by the first definition.
Think of it like this. If you wanted to support all individual poor people in America equally, you'll end up helping a disproportionate number of black people, because black people are disproportionately poor. So it's actually reasonable to say "black people need a disproportionate amount of help" if you're trying to help poverty in an unbiased way.
Why? There are many, if not the majority, of examples of mobility across outcomes. People don’t have to be born rich to become rich. Nor poor to remain poor. So many millions of people have taken advantage of equal or even unequal opportunities to create better outcomes for themselves and their descendants, why now are those successes all invalid and only by shoveling equality of outcome onto the “disadvantaged” are we able to make things fair?
...what? No. I'm asking about the equality of opportunity vs. outcome. Parent said he's for equality of opportunity rather than which accessible parking is most definitely not. I'm suggesting we need more nuance than just categorizing one as desirable and one as undesirable.
I mostly disagree with what “gdy” said but he might have a kind of a point. We, the society, choose to unfairly advantage disabled people, recognising that even with all their advantages, their lives are still pretty shitty. On the other hand, there’s plenty of black people who can, and do, raise to the top of whatever success hierarchy you want, so they don’t need to be treated better; they just need fo be treated the same. (As a thought experiment, would you rather be black like Obama or Kanye, or some disabled guy like Hawking or Roosevelt?)
I don't think this contradicts what I wrote. Like I wrote in the previous comment, all I'm saying we need more nuance on whether equal opportunity or equal outcome is better, because it's situation-dependent, and often some combination of both. On this point you guys are agreeing and then analyzing the situation which is... exactly what I've been saying.
Contrary to what you read online, very few people should discuss those concepts in contrast with each other. Almost all of the public debate is contained in the former. Therefor being for the former doesn't really say much, at all, and any encyclopedia will tell you why.
Edit: I will make it easier for you. If you do not understand most of this page[0] you are ignorant of one of the most fundamental debates in society. If you do you would understand my position. The wikipedia version might do as well [1]. It is ironic how people who don't "believe in outcome" are often ignorant.
I was interested in the slide deck and wanted to hear the author expand on some of his thoughts, found a video of his presentation if anyone else is interested.
Wow, this is really good. I hadn't thought of some of these things and he exposes some points that should have been obvious to me. For example: You aren't guaranteed to be able to maximize two functions at the same time.
I don't think so. This is simpler, it's just stating that an optimization problem on a constrained information set must be weakly dominated by the full information set.
Arrow's theorem is a more domain specific and elegant proof regarding political properties that can be simultaneously held.
More like the orthogonality thesis, I think. These different value axes are totally unrelated, and building an algorithm that maximizes one does not necessarily mean that it maximizes the others as well, and this mismatch can have disastrous consequences, a la "The Paperclip Machine"
As I understand it, Asians are "underrepresented" at Harvard relative to what they would be if Harvard judged candidates based on SAT scores, scholastic achievement, and other seemingly-objective-and-reasonably-sensible metrics. They're "underrepresented" relative to the pool of candidates who are qualified to be at Harvard. But they are "overrepresented" compared to the general population of the United States. Which comparison is the right one?
Surely their high SAT scores and over-representation at Harvard compared to the general population are both caused by some heretofore unidentified analogue of white-privilege - what other explanation is there?
'Surely'? Why must it 'surely' fit into a preconceived ideological straightjacket? (One who's adherents are like 90% white, FWIW).
The answer is that asian immigrants are a self-selected subset of 1.5B asians in asia.
The social justice mental toolkit doesn't have an answer to this anomaly. American census numbers and the assumption of a normal distribution within them don't square with the highly-motivated subset of Asians who immigrated.
1) Cultural factors often raise people from poor to middle class
2) The above often happens with little monetary input
3) Culture is transmissible
In contrast, giving people money and resources has a very poor track record for increasing wealth long term. Why aren't social justice people all about the transmission of wealth building cultural knowledge? If anything, on the whole, they seem ideologically in favor of policies that erode the transmission of wealth building cultural knowledge.
I hope you're not asking me, because I don't understand the social justice people at all.
My guess is a combination of social media obsession and limited life experience cause them to optimize for 'sounding woke' over 'things that help real people'.
> if Harvard judged candidates based on SAT scores, scholastic achievement, and other seemingly-objective-and-reasonably-sensible metrics
I think this type of statement represents the main problem that folks who run down the “Asians at Harvard” rabbit hole.
People assume that Harvard admissions is all just based on who is the biggest brainiac based on grades and SATs, and that’s simply not the case at Harvard or any other elite school.
High SAT and high grades gets you merely a 2 (1 being best) in one of four categories (academic, athletic, extracurricular, and personal/leadership). If you get a 2 in three of four categories, it still only gets you a 40% chance of admission.
That said, getting a 1 in any category gives the applicant a bump to anywhere between 48% and 88% chance of admission. Note that only about 100 people per year get a 1 on academic, and that still only gives them a 68% chance of admission if that’s their only 1. Standards are sky high.
To anyone who has high scores and great grades in a rigorous curriculum and wants to apply to an elite school, I advise them to differentiate themselves in ways other than academics — it’s a much easier path to acceptance. In my opinion, these non-academic selection criteria is one of the things that makes top schools amazing places.
Specifically:
- Great grades and great SAT scores (750+ in Math and Verbal) are a baseline for being considered.
- The more of the following list below an applicant can check off, the more likely they are to get admitted. If they have several high quality checks, the requirements for grades and scores can actually decrease to a surprisingly low level:
(rough order of importance... non-exhaustive list)
1. Recruited athlete.
2. Diversity candidate (race).
3. Notable academic achievement (e.g., publish a paper).
4. Demonstrate leadership via some verifiable and substantial project.
5. Be a skilled-but-not-recruited athlete (esp. in non-varsity sports), entertainer, or person of interest (e.g., children of famous or powerful people). Accolades or championships help.
6. Create a substantial and successful business or non-profit.
7. Diversity candidate (geographical).
8. Have an incredible personal narrative (e.g., Malala) — this might be more important... it’s hard to tell.
9. Be the child of an alum.
10. Be the child of a substantial donor (also hard to rate in importance... it really only comes into play if they aren’t admitted normally).
11. School/department applied to (depends on university).
It boggles my mind when someone (Asian or otherwise) points to great grades and great SAT scores and gripes about not getting into an elite school. Elite schools explicitly state that they are looking for well-rounded applicants or “multidimensional excellence”. Why is that person/applicant surprised when they only focused on one dimension and did that in a non-exceptional way (compared to other applicants)?
Anyway, this is a topic I know quite a bit about. Please feel free to ask follow-up questions.
Then why do their admissions numbers line up neatly into racial buckets that match exactly what a social justice person would like to see?
I'm not saying you're wrong, but when their internal paperwork seems to indicate an ASTOUNDING difference in "personality" for blacks vs Asians, at a large-numbers level.. I mean do we really buy that? Can we even just consider for a moment how racist that sentiment is? Against both races?
1. Asians are over-represented by ~4x versus their percent of the population (5.6 in US and 22.9 at Harvard)... not that this is a useful metric. Is this a stat that SJWs are happy about? I genuinely wonder.
2. Affirmative action is a thing. If you stop the so-called “Asian quota” (e.g., by going race blind), then any slots gained will most likely result in losses for slots from other ethnic minorities. Some people believe that shaping the student body a certain way is good for the university. I personally agree with that general statement, but the specifics are not easy to hash out — there are many trade offs, and it’s not easy to determine where the magic cutoff is for too few or too many.
3. If Harvard goes race blind, don’t believe for a second that it will turn out like Stuyvesant or Berkeley. IIRC, when the change happened, Stuyvesant did not focus on the “multidimensional excellence” that most other elite schools do. Berkeley just lets in too many people to need to review to that level of detail. Harvard and other elite schools might see a very small Asian bump, but my guess is that it won’t be what people expect.
Note that I strongly advocate for fairness in the admissions process for all people, and I definitely think that some Asians get stereotyped poorly. That said, I also think that a lot of people, Asians as well as non-Asians, simply don’t make efforts to shape themselves in a way that optimizes their chance for elite school admission.
I can't tell if you're disputing my assertion that they first decided what their racial buckets "should" look like, then stack ranked within each bucket and fudged the numbers to make it look less discriminatory.
It's that last part that I have a real problem with. If Harvard wants to say, "we want x% black and y% Asian, etc".. ok, I guess, but if they're going to say "black people have lots of personality to make up for their lack of science skills", and vice versa to Asians, that's just racist as hell to everybody involved. Own your decisions.
While there are foreign nationals who come in as undergrads, it doesn’t sway the percentage that much (x domestic and 3x international). Most undergrad Asians I’ve met at elite schools are US residents/citizens who have been in the country for a while (i.e., not just someone who came for high school).
That can be quite different for grad schools, though.
Thanks for weighing in with the detailed information.
I actually had extracurricular activities in mind as one of the "seemingly-objective-and-reasonably-sensible metrics". (It can be harder to judge—e.g. "president of the choir club" could mean anything from "four people got together twice a month to sing" to "organized concerts and musicals in collaboration with the theatre department"—but maybe candidates describe more precisely what they did.) Legacy and athlete stuff make some sense in terms of the university's financial self-interest... most of the other things can be interpreted as demonstrating ability in some way (academic and leadership ability), or motivation/"Big 5 trait Conscientiousness". The "incredible personal narrative" seems like a wildcard that could point to lots of different things, but in Malala's case... in general, public relations could be a reason to take famous people. Department applied to, I assume that's because some departments have lots more applicants or don't need as many people than others. It's the racial and geographical diversity things that seem out of place.
Geographical diversity question: I've heard that, specifically, colleges like to have at least one person from all 50 states. Why is that? Just because it sounds cool? Is it really for some kind of viewpoint or background diversity? (A counterargument to that, at least for elite schools, is the same as for racial diversity: if you think the people from e.g. South Dakota who make it into an elite school are at all typical of South Dakotans, you've got another think coming.) Is it so they'll have alumni everywhere and therefore recruiting help and alumni interviewers everywhere, which wouldn't be efficient if all their alumni were in one place? (How many South Dakotans who go to an elite school actually stay in South Dakota after graduation? That doesn't seem to make sense.) A quick google merely turns up the quote "“Every school likes to be able to stick that little pin on the map and say they have kids from all 50 states,” said Andrea van Niekerk, a college admissions consultant and former associate director of admissions at Brown University", which is basically "because it sounds cool". (I also note that "a student from every state" could mean, at most, accepting 49 students you wouldn't have otherwise—probably less—whereas the extra 19% of Asians that Harvard didn't admit (I mention 42% vs 23% figures later), of a class of 1600, means accepting 300 students they otherwise wouldn't have—so the every-state policy has a smaller impact than the racial-diversity policy, and maybe the impact is small enough that indulging that whim is ok.) Do you know the rationale, for this and/or for geographical diversity in general?
Historical question: I've heard that the reason extracurriculars were originally introduced, as a criterion for candidates, was as an excuse to accept fewer Jewish students. Here's an article asserting this for Harvard specifically: http://web.archive.org/web/20190109180814/https://www.econom... Do you know if this is true?
SAT ceiling question: The maximum score you can get on the SAT is low enough that there are plenty of people that max it out, and plenty of others who would have maxed it out except they made one or two silly mistakes. I suspect that's why you say 750+, in that differences between anything above that are just noise; and that this still leaves a large pool to sort through. (As a data point, would the set of perfect-score SAT applicants still be too large to fit in a Harvard undergraduate class?) If the SAT had a higher ceiling, say going up to 900 or 1000 somehow, where the difference in actual ability between the average 850 and the average 750 was the same as between the average 750 and the average 650, then do you think they'd weigh SAT scores more heavily?
Olympiad question: are academic contests (math olympiad, physics olympiad, etc.) counted heavily in the "notable academic achievement" category? If so, would you say they substitute for a higher-ceilinged SAT—that if the latter existed, then they'd put less effort into researching and evaluating the olympiad contests?
Caltech question: I've heard that Caltech, at least as of some years ago, basically just selected for academic scores, and they had 0.6% blacks. Looking at this year, they have 40% Asians, compared to Harvard's 23%. Come to think of it, I've also heard that the University of California system is not allowed to implement affirmative action, and this year ... they break "Asian" down into subgroups, but if I add together "Chinese, Filipino, Japanese, Korean, Other Asian, South Asian, and Vietnamese", I get 42%. Anyway, any comments about Caltech or UC?
Are Asians underrepresented at Harvard? Or if you’re referring to something else, can you elobarate? (sic)
If you re-read my comment, it's clear that I'm not talking about rates of representation. You are trying to bring that in. We're talking about the process, be it a policy implementation or an algorithm.
It's the age-old "equality of outcome" vs. "equality of opportunity."
I wasn’t trying to bring anything in, simply curious exactly what you meant. You’re right though, if I had read you comment more closely I would have realized my question was irrelevant. Thanks for the link.
Yes, it is. But if you're designing an algorithm that assesses credit risk and you do this, you will lose money compared to a company that does not do this.
Frankly, private industry has been lucky up until this point that anything other than your financial history has been allowed to factor into your credit score or credit worthiness __at all__. I wouldn't be surprised if it gets limited to solely financial factors by regulation within the next 10-20 years.
In terms of regulations, I think by regulation they shouldn't be allowed to give the algorithms anything but loan amounts, payment schedules, and payment history. Don't give them locations. Don't give them age, name, race, or gender. Don't even give them the names of banks, since this too can be used in a discriminatory way that can be used to imply geographic location and thus race etc. Yes, people with no credit history will get terrible scores, but that's the point of a credit system. It should be based solely off of your merits and what you've actually done, not off of what people in your area/situation tend to do based on statistics. Then it's just statistics creating statistics, and socioeconomic mobility freezes
The cost-to-benefit analysis of wheelchair ramps doesn't look great either (compared to not building one). Focusing on profit alone leads to some pretty terrible outcomes for people.
Taking credit when you can't pay it back doesn't just hurt the bank - it hurts you too.
Getting loaned money is not a right, and if you are irresponsible, it's harmful.
So if lending more to a certain segment of the population hurts the bank's profit, I'd also expect it to cause more problems for that population segment too.
Why is representatively fair the gold standard? What if in 10 or 20 years we view one of the other methods of optimizing as a more "fair" way (i.e. what if our current consensus on what is best is not absolute truth, but a social and cultural bubble)? Then all our algorithms will be sub-optimal (at that time).
Just as different cultures value these differently, it makes sense that one culture that changes over time might as well.
I think it's a demonstration of how identity politics can result in ever more fractionated interest groups. If you keep going with the finer and finer divisions, one might suppose that one must stop at the level of individual people. (1 Kings 3:16-28 not withstanding.) Then one needs to come up with a way of adjudicating between the conflicting interests of different individuals, based on individual rights.
It seems like a shallow and obvious one to me, but if I'm mistaken and there's more there there, it deserves to be expressed substantively and respectfully. Otherwise the only effect is to rile up agreers with positive charge and disagreers with negative charge—just what we don't need.
I could go on quite a bit about how, as a man of color, it gets fatiguing having certain conversations or trying to express certain points of view in these types of socio-political debates lately (to my dismay sometimes right here on HN-a forum that holds itself to a certain standard of debate but seems quite eager to throw it out, convenience permitting), but looking around at a lot of other comments in this very thread that seem to run contrary to whatever personally defined "truths" are out there, it'd probably just get downvoted and flagged as well.
I'll just leave it like this, if it seems "shallow", to you-dang-then probably it's time for some deep personal introspection because like gdy, I think shadowbained's point isn't wrong and I don't fault him for the fatigue that's probably in his soul which results in a delivery style that doesn't fit a mold of pretend decorum. A mold manifested and curated by political-egotists, IMO.
(This last point made as broadly as possible)
Sure, but it's not hard to understand why. Race war is one topic, array indexing is another. Which is the greater risk to this site? If the answer isn't obvious, please read https://news.ycombinator.com/newsguidelines.html.
I think the article encompasses your objections, to paraphrase : "algorithms can never be perfectly 'fair', because logic itself can never be perfectly fair"
Can we even define 'fair', perhaps we just know unfair when we see it. We seem to be seeing more of it - poverty, homelessness, racism, fact/science-denial, gerrymandering, police violence, gun violence, etc.
Yet, we should still strive for fairness.
There is a lot of ground between where we are now and 'perfect' - and this is AOCs point, essentially - our system of [self?]-rule is palpably unfair for most people, yet we are busily encoding more of that unfairness into algorithms with little oversight.
AOC [ and Bernie, and others ] want us to seek a better implementation of society, and encode that in our algorithms and laws - fairer voting laws, fairer district maps, fairer pay, fairer penalties. Or the converse - encode more fairness, as a means to realize it.
For example, it may be a hard math problem to design optimally fair district boundaries, but its noncontroversial to suggest we can reduce gerrymandering significantly.
One problem seems to be that most people equate socialism with communism [ 'evil' ], when in fact you are always picking from mix of Democracy, Capitalism, Socialism, Oligarchy etc. After Bernies campaign, and a couple years into the Trump experiment, people are now starting to listen to these old but useful ideas.
Ultimately if we don't share any wealth with the poor we will have a bloody revolution, and if we don't reduce Carbon emissions then no wall will keep out the displaced millions of climate refugees.
A bit more Socialism in our Capitalist/Democratic/Oligarchic/Socialist mix, is sorely needed right about now.
Thank you very much for sharing this perspective. I think it probably does a good job standing in for how a lot of people think about these things. This is a difficult area of discourse, and I know people feel strongly about it. Obviously, the lecture you linked to makes many more distinctions. I'm going to try to simplify one of the distinctions to make it easier to talk about: that there is a tension between "procedurally fair" outcomes and "representatively fair" outcomes.
In my opinion, I think this idea is wrong, at least in the context of designing AI within contemporary American society. To frame "procedurally fair" and "representatively fair" as opposing value systems is a misunderstanding of the best arguments for a "representatively fair."
Let's use a common analogy: a race. This distinction imagines a situation where one race puts a bunch of runners at the starting line, tells them to run, and declares a winner based on who crosses the finish line first. That race is "procedurally fair" because all the participants raced under the same conditions. Then, there's a "representationally fair" race, where runners from "protected groups" are allowed to start the race from the middle of the track.
I think there are two ways that this isn't right that come from a common misperception, that the only relevant time to think about in the context of these decisions is the present. (I also want to mention that there's a lot of heavy-handed language use in this presentation that reveals the author's perspective. Algorithms tilted against black people "reveal" disparities,
First, it's important to have a more accurate perspective of the past. In most cases where "representationally fair"-type solutions are used, there is a historic reason why the "protected group" can't run as fast as the other group. If people in Hyderabad are more likely to defraud a micro-lender, that's not a spontaneous result of inherent differences between Hyderabad-type humans and other humans. A quick google search tells me that as of 2017 Hyderabad has the second largest poor populace in India. So, in the race to a successful economic outcome from birth, the so-called "procedurally fair" solution actually means taking the Hyderabad runners as infants very far back from the starting line. All else being procedurally equal, when the starting gun of "applying for a micro-loan" goes off, the Mumbaikars and Delhiites are already far ahead of the Hyderabadis. Once we have a more accurate perspective that includes the past, in order to make the race procedurally fair we have to move everyone into place at a common starting line, which will inevitably mean helping the Hyderabadis forward.
Second, it's important to acknowledge the effect of biased algorithms on the future. Not only will the unfair, pseudo-"procedurally fair" approach unjustly disadvantage some runners in this race, future races are calibrated according to achievement in past races. If you win one race, you get a head start in the next one. Micro-lenders who refuse to lend to Hyderabadis will exacerbate the relative poverty situation that the algorithm is picking up on. It's easier to pretend to yourself that you're making an algorithm that peeks into the world, makes an objective judgment, and then pops back out of the world. But in fact, people designing algorithms have a responsibility for the outcomes of their algorithms. If an unfair situation exists (for example, that just by being born in Hyderabad and not Delhi, any given person will start life with less economic power), your algorithm's consequences will either be helping to fix that unfair situation, or making it worse.
So, if your algorithm punishes Hyderabadi applicants for being poor, it is both unfair in the simplest sense once you account for where the applicants started and will increase the unfairness of any future round of applications. Put another way, what we're talking about at a high level is values. In one libertarian version of society, the purpose of the algorithm is to maximize profit for Simpl. That society values maximizing profit, and enshrines "shareholder value" as the centerpiece of its ethics. In AOC's socialist version of society, justice, equality, and eliminating poverty are valued highest. This is the broader point she's making about algorithms, that they reflect the values of the people who make them. Given the power that these algorithms have, in our example to lift people out of poverty or to deepen economic inequality, society and not just technologists (not the most diverse group in countless ways) should lead the decisions about what should be valued.
We're seeing more and more the consequences of a techno-libertarian approach, and I'm curious what we'll see as consequences if a more socialist approach wins for a while. I value fairness over shareholder value, so the prospect excites me. But I'm sure there will be lots of unintended consequences in such a system too (the over-cited "asians applying to elite colleges" example being a good case of an outcome that's not easily justified on its own. I think elite college admissions are more fundamentally broken, but that's another thread!)
The problem here is, we don't even know what "fair" means. What do we mean by fair? Say we want to base fair on group outcomes, in which ways should we divide people into groups in order to perform some sort of balancing? Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loans? Will we need to adjust our credit rating model to bump the credit scores of red haired people slightly? Will we need to agonize over other features fed to our model to ensure that they can't be used as a proxy for red hair?
Maybe???? But it seems very dishonest.
I think, if anything, the models are probably too true for our tastes as social creatures who evolved to avoid social conflict (being 100% honest with your unruly neighbors can lead to a loss of harmony that is much more damaging than just putting up with some disturbances). Our idea of "unbiasing" the models is more likely actually making them biased in a way that pleases our monkey brains.
> Is hair color or texture a relevant way of subdividing people for instance? Are we being "fair" to red haired people? What if red haired people for some reason seem to be less likely to pay back loans?
Machine learning is going to be very good at identifying proxies for features we don't like, even if we don't directly provide those features. Weighting red hair color could be a proxy for the Irish, targeting hair texture could be a proxy for african americans, targeting zip codes has always been a proxy for race, etc.
If we want machine learning to ignore these kind of things it's probably going to have to be a separately weighted training goal that racial groups and so on need to have equal outcomes in the final results.
That's really what AOC is going for here. The black box will happily learn racist ways to weight things all on its own, we can't just throw up our hands and say oh well, if the algorithm is denying black people for home loans then we just have to accept it.
Because, in the short term, being racist is an effective greedy strategy. It's not your business's problem that, say, black people have higher rates of default (or whatever, example). But it also leads to bad social outcomes in the aggregate, and we've agreed that shouldn't be a factor in home loans. And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics.
Weighting red hair color could be a proxy for the Irish, targeting hair texture could be a proxy for african americans, targeting zip codes has always been a proxy for race, etc.
Off-topic-ish, but only about 10% of Irish people have red hair.
"And we have to find a way to make sure that The Algorithm (tm) doesn't just find a way to machine-learn redlining or some other proxy for race or other protected characteristics."
This is not wrong, but it boils down to "don't make mistakes when optimizing functions". It's offensive because it's coming from someone who has nothing to contribute except "do your job better". It's like saying we have a problem with planes crashing, so engineers have a moral duty to build better planes. As though they weren't trying to build planes that stay in the air already. You can make serious sounding statements, or you can impose fines, or you can randomly pick people to publicly shame, but none of those things are positive contributions to hard problems.
The consequences of a poorly designed algorithm are far less visible than those of a poorly designed airplane. Oftentimes they are deliberately kept secret. How many banks do you know that publish their loan qualification source code?
Mechanical engineers are taught from the beginning of their careers to think carefully about the consequences of their decisions.
Computer engineers, not so much. We're taught how to optimize, and make machines that optimize, and make optimizers that optimize our optimizing machines, but it's pretty rare to encounter a data ethics course, and they're almost never program requirements.
Facebook got in trouble for algorithmic redlining in 2013, serving loan sharky ads to black people (which its algorithms learned via proxy features, like zip code). Does Facebook hire third-rate engineers who would point the engine on a plane the wrong way? No! Facebook has some of the best people in the industry! But even the best people in the industry were and in many cases still are not fully considering the consequences of our designs, because we weren't trained to.
Wrong. We had two courses at my school. From the ABET Criteria for Accrediting Engineering Technology Programs: "(e)Include topics related to professional and ethical responsibilities".
The issue here is proxy features, and that it isnt as obvious if your statistical model is doing something wrong. You identified zip code and race. What about number of cats? Would you have a particular gender and marital status in mind if a given data sample had a large value in the numberOfCats column?
So you ban making decisions based on number of cats. But amountOfKittyLitterPurchased, numberOfCatPicturesViewed, numberOfDislikesOnDogPictures, etc could be used as a proxy for numberOfCats. Try to come up with an exhaustive list of all features which could be used as a proxy for some protected class. Starting with politically incorrect jokes will get you part way there, assuming you possess an exhaustive list of all politically incorrect jokes. But you will need to be pretty creative to come up with more.
If planes were built as shoddily as software is, even software that has massive material impact on people's lives, we'd have a lot more crashing planes, and a lot more people angry about it. And we have a huge group (a majority even) of software devs that are completely fine with the status quoa of how crap software is.
As a software person, I have absolutely no issue with telling other software people to do their job better, and I see no shortage of people that aren't even trying on that front.
Its your comparison between a process that takes safety, reliability and a reduction of harm as a core, first value, and a process that cares first about money, and second about novelty and somewhere way down the line people, maybe, that is offensive frankly.
The problem is the self-reinforcing, external factors that these algorithms don't take into account.
If you feed a ML algorithm crime stats, it will conclude crime is directly correlated to being black, or being a former criminal. But this correlation only exists because of how we've chosen to conduct the policing of black communities and treatment of ex-cons upon release that produced the the training data in the first place.
Most people here are male, were once (or are) under 25 and likely drive. How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car? Quadruply so if there are prior citations/accidents?
Shit, with a spotless driving record, my own insurance went up when someone t-boned me. Other driver was at-fault and I subrogated their insurance; didn't claim with mine. But I'm being lumped into a higher-risk pool with my own insurer for factors completely beyond my control-- because to some algorithm, being in an accident means I'm more likely to be in accidents, and all drivers who are involved in collisions are higher risks than drivers who aren't. Note how the fact that I wasn't at fault isn't factored in. This makes sense to you?
The social training data in much of Europe is skewed towards hating Gypsies and anyone perceived to be one. But it's the same as the ex-con problem-- deny them jobs, beat them, chase them away, and of course an ex-con or young Romani will commit future crimes. You're shaping behavior to validate your training data instead of making unbiased ("fair") predictions. It's the equivalent of betting on a horse, injuring its competition, then patting yourself on the back for being so goddamn good at this game.
You can't look at the world in terms of sheer numbers. The numbers themselves are dishonest.
> How'd you like paying more for insurance than your parents because some system decided men are bad drivers, and doubly so when they're young? Triply so when they happen to be driving a red car?
isn't this how it already works? young males cause a disproportionate amount of insurance claims so they get assessed higher rates. they'll even bump your premiums for driving the performance version of a given car. being a young male, I certainly don't like it, but I'm not sure I would call it unfair.
Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident.
The latter is more likely to be wealthy enough to be able to pay cash for repairs and keep the entire incident off the radar. No insurance claim, no citations, no evidence. The 18-year old has no choice but to make an insurance claim, go to court and take points on their license.
Can we truly conclude that 18-year olds with red Mustangs are worse drivers than their elder counterparts?
(And yes, I realize that in the case of insurance, it's in their interest to minimize claims so in that context this is a valid conclusion. But this same skewed data gets shared outside of the industry as well.)
> Take an 18-year old with a red Mustang and a 58-year old with a midlife crisis and a 3-series. Both drive like assholes. Both will inevitably cause an accident.
this is not a fair example; by assumption we have much more information about the 58yo than the 18yo. remove the bit about the midlife crisis and there's no reason to think the 58yo is anywhere near the same risk to insure. as a group, young people are less experienced drivers, have less facility for judgement, and the males probably drive more aggressively.
unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this.
and by the way, I'm a young male who drives a fast car.
> unless you disagree that the goal of insurance is to spread risk among a bucket of people who have relatively similar risk factors, I'm not really sure where you're going with this.
I don't disagree. In this context, if older men make fewer insurance claims, they're better customers. On paper, he's less of a risk (to the insurer). In reality, they're both risks to other drivers. But only the 18-year old will be held accountable.
But assuming the under-25-and-male crowd are the worst drivers on the road based on number of insurance claims is not necessarily true, yet it's the basis for this oft-repeated assertion. That's my point. What isn't the insurance industry's data telling us when we use their data to make these conclusions? To get more accurate numbers, you need to be looking at the demographics of all body shop customers and the nature of their repairs. What demographics get the most front-end work done that isn't covered by an insurance check?
This new age of computing seems to be to make machines understand how the world works by putting blind faith in statistics, without ever stopping to question where the stats came from or how accurate they really are. The numbers aren't handed to us on golden tablets carved by God himself-- we get them from imperfect systems designed by imperfect humans. And then we teach dynamic algorithms how reality works based on this data and sell them as solutions, which we trust absolutely.
> and by the way, I'm a young male who drives a fast car.
I think I misunderstood the point of your argument. at first I thought you were specifically singling out car insurance as being unfair, which seemed odd to me, as it seems pretty close to WAD to me.
if you are just using it as an example of a misleading dataset, we are probably in agreement.
are you assuming the event to be inevitable? if so, you're really opening up a can of worms here, bc insurance doesn't really function with p(event) = 1.0
a better argument would be something like: suppose we detect a gene that indicates probability 1.0 of disease that causes the person's left arm to fall off.
it's then not a matter of "insurance" but a matter of "are we as the pool willing and able to pay for this?" recall company needs some profit as well as cost of coverage to maintain overhead so it's a different argument now.
> The problem is the self-reinforcing, external factors that these algorithms don't take into account.
And changing the algorithms isn't the right way to fix those external factors. If you try, their self-reinforcing nature is going to cause exactly the opposite of what you want.
To stay with the crime vs. race example, assume that the ratio of black to white criminals is greater than you'd expect from the ratio of blacks to whites in the overall population, and that is due to self-reinforcing external factors. Your algorithm for deciding whether to parole a prisoner picks up on that statistical regularity and rates blacks riskier than whites. Because you want to achieve representational fairness, you make the algorithm race-blind to equalize the rates.
What happens? The black prison population shrinks. But your algorithm wasn't wrong when it predicted higher recidivism risk, so the black crime rate rises. Your algorithm now requires an even stronger correction to remain race-blind. Black criminals realize that they have an easier time getting paroled, so the black crime rate rises some more. Debiasing the algorithm has made the situation worse.
In general, when you find out that your machine-learning model has some undesirable bias, the correct response is not to blind the algorithm. Instead, you should fix the data-generation process (the real world) until your model no longer picks up on the bias.
Edit: For a mathematically precise description of how attempts to make an algorithm more fair cause worse results when you consider the feedback loop that it is embedded in, see "Delayed Impact of Fair Machine Learning" https://arxiv.org/abs/1803.04383
Your higher rate makes a bit of sense. The worst problem is the small sample size. If everybody crashed a minimum of 100 times, then the risk estimates would be much better.
The fact that you were hit is influenced by your risk factors. For example, maybe that is a bad intersection, and you always drive through it. Maybe it really is just the other driver being bad... but he and you happen to leave for work every day at a reliable time, making it likely that you will encounter him again. Maybe your car is hard to see. Determining "not at fault" is a fuzzy thing; perhaps a better driver would have somehow avoided the crash. The number of miles you drive is an influence on the number of crashes you get, and perhaps you drive more than is typical.
But what if, due to historical happenstance, the data is already a proxy for groups split on genetics, biology, or birth? Using that data in a way that might create a feedback loop that propagates that happenstance is entirely likely.
Part of the problem is that, when dealing with humans as the data set, the data is really messy and there's lots of noise correlated to historical circumstance.
The problem is that in many cases (especially with deep learning), "the algorithm" is essentially a black box. There often isn't a straightforward way to determine exactly what the algorithm is actually learning.
I know that too, but if I(or the community) have access to it I can prove the algorithms is flawed. Say i change a bit the input and notice a huge output, like I change my non enlgish name to an english name and by magic my score triples then I prove the algorithm is discriminating me(same if I change my birth place or address).
I don't think that opening the algorithms is a sufficient condition but I think is necessary.
If there exists a scenario where it fails then yes, it is impossible. Just because something works the way you intend it to in some specific case doesn't mean it works in the general case (and the general case is what matters here.)
The GP's claim is that yes, it is _literally_ impossible. He cites a source[1]. If you have a disagreement with that theorem then by all means, share. Right now you're just saying "Nah I don't think so", which isn't very convincing.
> There is no design of AI that can be simultaneously utilitarian, procedurally fair, and representatively fair.
AI is just tools of humans. Any system that can be achieved with humans can also be achieved with AI.
These AIs take us backwards from our existing human systems in addition to being much harder to correct later.
The supermarket theft example in the slides you linked to is a prime example. In what store in America would that kind of display be deemed acceptable? Yeah, you save a few dollars a day on theft, but what kind of goodwill are you spending? Now that AI is here, people think they can get away with things like that because numbers.
What I'm trying to say is that inasmuch as you consider the current legal and cultural environment of America to be utilitarian, procedurally fair, and representatively fair, any system that could be built using AI can achieve the same. I don't believe that they will a priori deliver an improvement. But there's no excuse for AI systems to show any discriminatory biases that have already been minimized in human systems via legislation, training, etc.
Don't misunderstand me; I'm not saying that discrimination doesn't exist or that we live in a society with no biases. I'm saying there is no valid reason why AI should take us backward in that regard.
Computers are extensions of the will of man. Our AI researchers and developers should ask themselves what kind of society they want to create. Whether they choose to turn a blind eye to discrimination and bias, or choose to actively combat it, then that will determine the result. Civil rights didn't just happen; there were people fighting for it actively. The same has to occur in the digital world if the same result is desired.
The second half of the article that talks about Yudkowsky, rationalists, and Roko's Basilisk is absolute trash and a gross mischaracterization of the actual positions taken.
>Yudkowski has for more than a decade pursued the possibility of perfect human reasoning
His website is literally called "Less Wrong". It's fundamentally a quest for improvement, not perfection.
>His system of coldly logical reason, it turned out, was by many accounts completely undone by a logical paradox known as Roko’s Basilisk.
Roko's Basilisk is a thought experiment designed to import your "don't negotiate with terrorists" intuitions into a really weird corner of decision theory. It's possibly why prior decision theories held were rejected as flawed, but it's not a current issue with their logic. Roko's Basilisk has, unfortunately, gotten way more coverage and fame than it deserves in terms of actual importance. This is because of the regretful (but understandable) decision to censor discussion of it on Less Wrong, which backfired spectacularly.
>For super-nerd bonus points, it’s also arguably a spin on Godel’s incompleteness theorem, which argues that no purely rational algorithmic system can completely and consistently model reality, or prove its own rationality.
That's not at all what Godel's incompleteness theorem argues. That's much more narrowly about formal logic.
> "This is because of the regretful (but understandable) decision to censor discussion of it on Less Wrong, which backfired spectacularly."
Can you explain the (understandable) rationale behind Less Wrong censoring discussion of Roko's Basilisk? I've been lead to believe it was censored out of fear that discussing it could somehow inspire its actual creation.
Honestly from a distance Less Wrong has always struck me as vaguely cult-like, but I'm open to the possibility I've just gotten the wrong impression.
Sure thing. The basilisk can easily be misread as an argument to do everything in one's power to support AI research of some sort. It's more accurately an argument that you should ensure your decision theory doesn't coerce you into putting all your effort into supporting AI research. LessWrong has a bunch of really scrupulous people reading it, who take pride in taking ideas seriously, so censoring the discussion seemed worthwhile for avoiding the failure mode of having readers make the mistake mentioned previously.
I think the more charitable interpretation is that the discussion was taking over everything. If people are fixating on a narrow topic for a long time (without much progress) then sometimes it's helpful to say "alright people, let's move along now."
You're right about Less Wrong feeling like a cult, though. It gives me the heebie-jeebies.
Cults don't write about how to avoid being a cult, while the LW crowd has reams of texts and blog posts about how they are purposefully avoiding trying to become a cult.
I think some of the feeling of cultishness might stem from a homogeneity of expression, but it kinda makes sense for a community of people dedicated to being rational -- after all, two rational agents with identical priors cannot "agree to disagree" in the mathematical sense. The community would tend to converge on common "correct" things and would have similar rebuttals for "incorrect" things, right?
Have they ever addressed the criticisms of rationality, technological rationality and instrumental rationality of the Frankfurt School in the 50s and 60s? Or, more generally, the critique of positivism and the collapse of rationality to facts connected with formal logic? These are fairly big topics in philosophy, so it would seem really naive if they haven't covered them.
I'm not personally familiar with those critiques and I haven't exhaustively read anything close to all the content from LessWrong, but I can point out that Yudkowski himself has a number of criticisms of Traditional Rationality. I think the most helpful thing I can add is that the paradigm embraced at LW is Bayesian rationality and not pure formal logic. Being massively reductionist, it seems to boil down to updating your expectations to be more in line with reality -- that being less wrong is about having an accurate account of your uncertainty of the beliefs that you hold, taking into account your priors, inherent bias, model complexity, etc.
This just seems like a very strange and restrictive way to model real-life problems, and it just seems to export the meat of the problem (taking into account priors, what exactly is bias, and if it's possible to be aware of it); for instance, I imagine that many people who subscribe to LW thought are probably uncritical of the ideological function of liberal democracy, for instance - or rationality being used to justify the status quo. These sort of things were taken up by critical theory, so there seems to be some overlap between critical theory and what it is to be rational.
I fear that a faith in mathematical reasoning for all problems can easily make you come out the other end thinking your opinions are mathematically correct, with minimal reflection on core issues such as the applicability of classical logical models or a mechanistic view of how people do (or ought to) behave - this kind of uncritical historically divorced reflection has not fared well in the past, with even economists admitting their model of the economic man is as much fiction as their worldwide Robinson Crusoe from a century earlier.
But I haven't seen cults talking about how to personally ensure you're not falling into trap of cult-like thinking. LW is a rather self-conscious community.
Sorry for the tangent, but from the outside I've also felt very "outside" the LW community. I'm not exactly sure how to "join" it. I know that I'm supposed to read the various required readings first, but then what? How do I see what LW is "talking about" or "fighting about" at a point in time? I mean this in a serious, logistical sense. Thanks!
The LW community has decentralized and deformalized things to a huge extent. It's not just you - the logistics of participating are much less clear than it used to be, and the whole system is far less legible.
Honestly the best strategy is attend local meet-ups, come across as intelligent and interested to the people who hold gatekeeping power over the informal non-public groups, and get in the loop that way. This is significantly easier in Berkeley, where a lot of folks end up moving to. You can also skip some of these steps by attending a CFAR workshop, which will both introduce you to many members of the LW community and put you on an alumni-only mailing list which houses a chunk of the general discussion.
I'm out of the loop now, but it used to be like everywhere else (HN included) - you follow the posts and participate in comments, and after a while you suddenly find yourself to be an established community member.
I don't fuck with LW, but SCC definitely has that culty vibe.
Overall I think Scott has some interesting articles and the general idea of rationalism is .. fine. But that's not at all what the communities around it are like. You just get tons of 'rational racism' with HBD (human biodiversity for those not in the know, which is basically just race realism with a nerdy veneer), IQ worship, and a group that collectively abandons anything like the rationalist ideal and becomes a post-hoc clusterfuck.
The community can almost be thought of like these racist AIs: garbage in, garbage out. They cling to any bit of rational-sounding data that supports their preconceptions, and have to adopt severe anti-academic stances to reject the mountains of data that contradict their views.
I suppose the deeper rot is that these rationalists fancy themselves like scientists, able to interact with primary data and draw conclusions, but any actual expert (i.e. someone that has performed research and published it in a community of peers, not whacko journals) is just confused how anyone could be so wrong. You regularly see posts where someone comes in who actually knows what they're talking about, authoritatively shows that all the nonsense they've been talking about makes no sense, and at best they get a 'huh, interesting' before the nonsense picks back up.
The idea of a rationalist community is to organize around questioning your own assumptions, confronting diverse views, and engaging in good-faith debate. But the reality is that basically none of that happens for anything controversial to the community, so it's just a veneer of legitimacy for in-group circle-jerking. And it just so happens that the in-group tends to be pretty damn racist.
The "meat" of interest here for those too lazy to click (I encourage people click through, though, really, and maybe even take a glance at https://arxiv.org/abs/1401.5577) is probably this long parenthetical:
> (But taking Roko's premises at face value, his idea would zap people as soon as they read it. Which - keeping in mind that at the time I had absolutely no idea this would all blow up the way it did - caused me to yell quite loudly at Roko for violating ethics given his own premises, I mean really, WTF? You're going to get everyone who reads your article tortured so that you can argue against an AI proposal? In the twisted alternate reality of RationalWiki, this became proof that I believed in Roko's Basilisk, since I yelled at the person who invented it without including twenty lines of disclaimers about what I didn't necessarily believe. And since I had no idea this would blow up that way at the time, I suppose you could even read the sentences I wrote that way, which I did not edit for hours first because I had no idea this was going to haunt me for years to come. And then, since Roko's Basilisk was a putatively a pure infohazard of no conceivable use or good to anyone, and since I didn't really want to deal with the argument, I deleted it from LessWrong which seemed to me like a perfectly good general procedure for dealing with putative pure infohazards that jerkwads were waving in people's faces. Which brought out the censorship!! trolls and was certainly, in retrospect, a mistake.)
My guess would have been that by discussing it, you leave a trace in the networks allowing an eventual AI to prove that you knew about it. Since you knew about it, it can hold you accountable, and take revenge on all of your descendants if you didn't devote sufficient energy on bringing the AI into existence.
Also the neoreactionary bit. I literally paused on that in total surprise. As a long-time LW lurker - yes, the rationalist community plays with plenty of non-mainstream ideas and try to work them in both directions (up from first principles, and towards extreme conclusions). But characterizing rationalists as equal to neoreactionaries? That's a complete mischaracterization of the community.
If you don't see the NRs in the rationalist community, you're not looking.
Ultimately I agree that they're not equal; I think many people involved in LW or SSC are good-faith rationalists. But there's also a strong segment that has adopted - post hoc - a crazy set of priors to support NR views.
What strikes me is how antithetical this post-hoc "pick your priors, any priors!" is to the rationalist ideal, but it doesn't get called out. It's given equal treatment, and it means that the community is categorically incapable of discussing culture-war issues.
There are neoreactionaries, but they aren't anywhere near as important or central as the article implies. A huge chunk of this is the fault of the discourse norms, which allow for a lot of discussions that would be prohibited or highly discouraged elsewhere.
All: we're going to try turning off the flags on this story. If you're going to comment here, do it civilly and substantively. I can already see signs of thought-degrading flame-entropy appearing, and if it continues we'll restore the community defense mechanism.
The article implies that both algorithms and the data are at fault, which I don't think is true. It's really just the data, the algorithm reflects the 'truth' it finds in the data.
Many approaches in fair machine learning that try to 'de-bias' the algorithm basically just do stuff like reducing the accuracy in the advantaged group to make the algorithm seem more fair - that is hardly what you want and will just make you susceptible to charges of discriminating against the majority or employing affirmative action. Probably rightfully so, because that's what you do. It's absolutely fine if that's the intent, but then you should have a public discussion where you are open about the fact that you manually tinkered with the parameters to prefer fairness over accuracy (which can probably be a valid goal).
I think finding the problems with the data is very important though. Everyone wins if the quality of your data increases: the algorithm can become both more accurate and also more fair. And it can also identify societal causes for this biased data, for instance police being more sensitive to crimes of minorities, which will then feed back to the innocent algorithm.
A related point is of course that we should be wary of putting too much power and trust into faceless algorithms in the first place.
I thought we got over "the computer is never wrong" fallacy by the end of the 90s. Anyone involved with computing should know GIGO - Garbage In Garbage Out.
Perhaps. But lots of developers have also heard that premature optimization is the root of all evil. Doesn't mean we don't fall prey to these traps, biases, misconceptions, etc.
Besides that, the rise of the algorithm, like everything else in the US, is subject to interpretation by camps who insist (at their extremes) that racism is real, modern, ubiquitous, and fixable; and those that insist it's a relic of the past, inevitable, imagined, rare.
Apparently if you shove in a lot of garbage, subject it to a decision making process nobody can explain in detail, out the other end comes magic reliable data that if you question you obviously just have an agenda.
So you are saying any data for which any input feature is correlated with race or sex is garbage. That's going to make doing useful data science incredibly difficult with anything involving humans.
Sex is easy to incorporate. Race is more difficult because vanishingly little work has been done with a concept of race based on anything resembling real population genetics, but it's not impossible in theory. Hint: if you really care about human biodiversity, you'll spend most of your time in Africa and some isolated islands, not looking at brown people in the west.
The issue is GxE, where we see loads of racists and sexists just patently forget about 'E' and conclude that women or brown people are genetically inferior. Some will try to gussy that up as 'different' rather than inferior, but the dog-whistles may as well be air-raid sirens.
'Data science' is fine if you're AB testing websites. When you try to do real science, you'll find that the utter lack of research experience is .. a bit of a problem. Caring about PhDs isn't credentialism, it's wanting a plumber that has worked with pipes before. You have to actually perform research to get decent at it. It usually takes about a decade before you can honestly do it independently. If you can look back at what you did a year ago without cringing, you're not making that progress.
Some schmuck with PANDAS and a Bio101 class 8 years ago isn't a scientist.
To some extent. More specifically, if those become confounding variables, the result may be partially useless. That's why you have to make sure you control for race and sex if they're relevant. https://en.m.wikipedia.org/wiki/Confounding
It makes it exactly as difficult as the problem actually is. If you start with garbage data, such as arrest records (black people are WAY more likely to get arrested than white people in the same circumstance), then the bias in your sample is going to result in bias in your model.
People want easy answers and discard uncertainty as soon as they get an indication that's stronger than the noise floor.
See also: Anyone who (despite the fact most people understand a 1-in-4 chance perfectly well when handed a 4-sided die) looks at FiveThirtyEight's election predictions as anything more than a fun curiosity.
> We could benefit from better language around these concepts
I believe this is a very under-rated problem today's public is facing. Not just these concepts, but many political terms are misused in today's climate, making for a conversationally ignorant population.
I don't see how it is being misused. This sort of distinction is pretty basic e.g. philosophy. People simply disagree. More and more people not understanding the basics in favor of whatever they read on social media might be a problem though.
Political terms are misused all the time. For example, "liberals" are constantly conflated with "leftists". Half of the population currently believes that "fake news" means "news that doesn't confirm my political preconceptions".
jpmcglone and dgzl were proposing the opposite of that. Newspeak eliminated words so that some ideas simply could not be expressed. They propose adding words so that dissimilar things can be distinguished from each other - to expand the things that can be (clearly) expressed.
Yes, their concern is about the misuse and/or confusion of a limited lexicon. I understood that they propose adding words; and I suggested that there is power to be found in not doing so. It can be beneficial to some if we are unable to speak precisely and without misunderstanding. It can be beneficial to some to co-opt existing words for new uses.
What I get from people on the left is that an unequal outcome is indicative of racism, however subtle that racism is. The only other explanations for an unequal outcome are biological differences between races (do not want to get into that) and races haves advantages over others from past history.
Unequal outcome is not indicative of racism, it is racism. That's the point: racism is an outcome, not an intent. An unknowing, unfeeling beurocratic system can be racist, even if there isn't a speck of racial bias in the hearts of the practitioners and designers, simply due to historical quirks as you point out, or design mistakes.
So your argument is that Harvard is racist if it doesn’t discriminate against Asian applicants? So that they are not overrepresented among the student population?!
No, my argument is that "it's just numbers! they don't have a soul!" is not a valid defense from accusations of racism, because the statistical systems can be racist in their design, and there is no "racist meter" you can apply to check. The only way to know if a system is fair or not is to check whether the results you're getting are the ones you want.
FWIW, your usage of the the term 'racism' is not at all what most people mean when they use the term. The vast, overwhelming majority of people do not think in terms of systems. They think in terms of agency.
Right, and if you look at results, you see that there's too many Asian students, relative to their proportion in the general population. So how do you solve that? One way is to up the bar at the admissions, and require Asians to score better relative to other ethnicities, in order to be admitted. Do you think that makes the system fairer?
I think that implies a symmetry that doesn't exist. Vanishingly few people care about effects but not intent, because intent inevitably leads to effects. This creates a continuum, not a mirror image: consider neither, consider only intent, consider both. Those on the left mostly consider both, with a few considering only intent. Those on the right mostly consider only intent, with a few (racists by any definition) considering neither.
Your formulation also misses another very important nuance. I don't think most people on the right don't consider effects. They mostly know that such effects exist, and quite often feel bad about that. However, they also believe that addressing only intent (or "procedural fairness") is sufficient to make those effects go away, and that more assertive measures create "reverse discrimination" and/or infringe upon liberty. I'm not going to argue whether they're right or wrong, but it's not about consideration. "Strategy" might be closer to the mark. Most on the right (not counting the true racists) do want to end racism. They just reject the left/center prescription for doing so.
Broad statements about the left and right are divisive, ignoring the spectrum of beliefs, and this is not a good summary. Just as two examples, housing and education policies are frequently designed with the intent of segregation, though it is impossible to prove intent in most cases. And so, the actual outcomes need to be used in courts to demonstrate racist intent. No one will ever openly admit that they don't want the, for example (as this is common in these cases), poor, mostly black children in their school. But they will also actively ignore all of the research that shows a benefit to their own children with increased diversity. And so intent can be assumed. People will hurt their own families to defend their racist beliefs. Maybe we could use better language, but what is more important is using the language we have more clearly.
when I went to school a poor black kid stole a portion of my lunch every single day in 2nd grade (stupid policy that you had to leave your lunch on an unattended desk, he knew which bag was mine and took the granola bar.) my parents moved me to a catholic school which had basically no black kids. the bullying was still there but nobody ever stole from me or really anyone else. no fights basically ever.
we move, I go back to a more diverse school. fights, my graphing calculator stolen... what’s the possible benefit of having poor people in your school? until social programs make it so my lunch doesn’t need to be stolen to feed a kid, I fail to see how the non poor students are better off.
To touch on your point specifically, I am sorry you had such a poor experience. You are right when you say that social programs need to exist so that children are not hungry at school. Education is supposed to be the great equalizer and yet America has provided education almost only for the rich and has consistently attacked the poor. But I also feel bad for the kid who had to steal from you every day just so that he wasn't hungry at school. Education and wealth are a combined, intractable problem in a capitalist country, but there are hundreds of places making the problem worse.
This problem exists accross disciplines. I heard someone defend the DSM manual as simply a way to standardize how psychiatrists think and talk about mental health issues. Otherwise, the field would be chaos.
Perhaps we need a DSM for society level malfunctions, with strict definitions?
It’s just that racist intend is almost always impossible to prove. Outcome therefore becomes a needed proxy, but only after excluding other factors, by, for example, normalizing for age and income.
Two landmark studies in this regard come to mind are (a) how the success rate at an orchestra doubled among women after auditions were changed to a “blind” format not allowing the decision-makers to see the applicants’ gender, and (b) how changing applicants’ names (and nothing else) could impact their chances to be invited to interviews.
You use the word racist to mean two different things.
The basketball team being all black isn't racism and it isn't due to racism.
If I saw a basketball team (in the NBA) of all white people, I would suspect racism, but it's important to point out that the outcome (an all white NBA team) is NOT racist in itself. Even if it is likely due to racism.
So, I think we should stop calling the outcomes 'racist' and say what we mean: "I suspect this outcome is due to racism"
I think that will make the whole conversation a lot easier to have.
I don't think its advantageous to certain political entities, however, if we have this conversation. There is one party in particular that I think relies on people to believe that their problems are outside of their control, so that maybe they'll outsource the problem solving to the government.
Maybe I have it pinned all wrong, but I will never know if we can't talk about racism and outcomes of what may or may not be racism as two separate things.
I fail to see a distinction b/w “racist” and “due to racism”. In any case, I feel large parts of society, including major media outlets, already tend towards caution, c. f. the reluctance in calling the President’s “both sides” comment “racist”, instead opting for “racially charged” or “insensitive” or similar.
> There is one party in particular that I think relies on people to believe that their problems are outside of their control, so that maybe they'll outsource the problem solving to the government.
That’s a rather unfair characterization of the Democratic Party. But I find it even more interesting to know why you feel the need to superficially obfuscate who you are talking about?
I’ve also provided two examples above that clearly prove that racism and sexism do exist. If gender-blind hiring doubles the chances of female classical musicians, aren’t they right in pointing the finger at that result and complaining about white men playing life on easy?
But apart from such narrow situations, most left-wing advocacy is decidedly altruistic: college students supporting a raising of the minimum wage aren’t doing so for their own benefit. Unless, that is, they are terribly pessimistic about their personal future. Neither are voters and politicians advocating for DREAMERs, who by definition are neither. Nor are Bill Gates, Warren Buffet, Bloomberg, LIN-Manuel Miranda, or any number of billionaires or otherwise successful people advocating on behalf of the less fortunate.
Thinking of racism in terms of the intent is a way to take the issue less seriously, to force the issue into the realm of opinions and beliefs. If the intent of the perpetrator is the deciding factor, then it is always arguable that perpetrator's intent was not specifically racist; perhaps it was driven by a misunderstanding, etc. Since it was informed by misunderstanding, it isn't racist. And so on.
It's an old tactic, I don't think changing the terms will make much difference.
A lot of people rationalize racist or discriminatory beliefs in such a way that they seem reasonable. Yet, when you judge their behavior as a whole, it tells a different story. The banker who consistently gives loans to people from [group x] but not from [group y] who have similar financial features. When we talk to the hypothetical banker, they may claim that "numbers don't tell the whole story", etc.
Yes they can, because that's what "discrimination" means: differences in outcome due to uncontrollable irrelevant factors like race. Discrimination is an effect, not an intent. Racist intent is called "prejudice". Discrimination can be caused by prejudice, or it can be caused by something else, like a poorly considered algorithm.
No, that's pseudo-social-science because it falsely assumes outcomes are solely determined by external variables and is completely ignorant of internal variables, such as differing cultural values.
Some cultures value family life over money, others value money over family life, for example.
I'm sorry, this reply doesn't make sense to me. What does "different cultures are different" have to do with whether computer systems can be discriminatory or not?
Yes, if the data contains proxies for race and/or unexamined racialized bias, then yes it can. Police presence has often been kept higher in zipcodes with mostly non-white residents. This results in higher arrest rates in an area because of increased eyes, not necessarily because of increased rates of law-breaking. If an algorithm were to be designed to determine where to allocate police officers using this data, it could contain no racial data, but it would dictate that more officers should be placed in the zipcode with more non-white residents, as those areas have more arrests. Perhaps you don't want to call this racist because you have a narrow definition of that word, but I don't know what other word to use.
Without the data, I have a hard time believing this. Especially since I had to explain the concept.
Even if that were the case, you're still left with a small set of conclusions about race and its correlation to FICO score.
1) Non-white people are worse at maintaining credit scores because of something inherent to the condition of non-whiteness. This conclusion quickly points in the direction of racist psuedo-science.
2) Social and economic forces do not act equally on people of different skin colors. This is easily proven true by the example discussed by AOC and ta'nehisi coates of people of color being explicitly excluded from provisions the New Deal, an issue that has compounded the inequality between people of different racial backgrounds. This is directly applicable to FICO scores as white people are more often able to access generational wealth to pay of debts and avoid low credit scores whereas people of color are not.
Sooo, in the case of FICO, even if the score doesn't contain subtle proxies for race which I doubt, it is still predicated preserving a system that contains intense racial bias.
Some people dont value FICO scores and so theirs are likely to be lower. This would fall outside of the only two models that you proposed. It requires cultural difference which some people value highly.
If you cannot accept the FICO as non-racist, then I do not know if you could create a system that others would not find racist. I cannot think of a way that your system would not end up with some form of explicit race-based corrections. I think that concept is less palatable in America due to the focus on freedom/individuality. Each person should stand on their own, not colored by the groups that you could fit them in.
Just curious if you have an opinion: how do you propose correcting for the past?
Lastly, you can look up the determinants of a FICO score for yourself as you can use sources that you trust.
Are you saying that even if certain groups of people are less likely to be able to repay debt (for reasons outside their control), it is still racist if their FICO scores are lower?
I think you want FICO scores to mean something that they don't mean.
you should ignore that comment, it's an incredibly disingenuous claim intended to mask the racist sentiment that 'non-white people are over-represented in american prison populations because they don't value not being in prison as much as white people do'.
I think it's important to note that there is no algorithm that can have an "intent". If we are to agree that racism is a feature of intent, then there will never be a racist algorithm. Yet the outcome will still be discriminatory.
> If the intent of the perpetrator is the deciding factor, then it is always arguable that perpetrator's intent was not specifically racist; perhaps it was driven by a misunderstanding, etc. Since it was informed by misunderstanding, it isn't racist.
Are you saying that this is racism. Most people would define what you outlined as not rascist.
I'm saying that racism cannot be solely an issue of someone's intent, it needs to be evaluated by the outcome. An outcome is a fact that we can all observe. An intent is internal to a person, we can never get a clear view of a person's intent; at best we can make deductions as to a person's intent based on their behavior.
Claiming that racism can be evaluated only by the intent is simply moving the goal posts into an area where we can't clearly observe. It's a tactic.
In terms of this specific issue, algorithms by their very nature lack intent. Thus this particular argument has no validity; we can only judge the algorithm by it's results: the outcome.
I agree, but cmiles74 has a bit of a point, too. Everyone short of the KKK (and maybe even them) will claim that their intent is pure. Lacking a foolproof way to judge others' hearts, all we have to go on is actions and their effects.
As I said, I agree with you. But our position can lead to hiding some genuine racism under the "unintentional" disguise. It also leaves unintentional systematic biases unaddressed. While those may not exist as often as the left claims, they do at least sometimes exist, and do need to be addressed.
I don't disagree. But I don't think we can judge the intent of people who are discriminating in a racist way, aside from simply asking them what the think their intent might have been. At that point it's very likely we'll be dealing with a rationalization or a half-truth because not only is there a stigma attached to racist behavior but in many cases it is illegal as well and there are other punishments to contend with.
If we need to accurately gauge their intent, that's not really possible. In the case of an algorithm we've divorced the process from the source of intent (the author of the algorithm), there is no intent to evaluate.
and academics think of it as power imbalance of any majority power, separate from the census majority and minorities
and colloquially nobody thinks of it the same way, with themselves always exempt from being racist until convinced that their 'normal behavior' is considered racist and this does not change their view of their normal behavior 'so be it'
this is a challenge. at this point the word itself is polluted.
> The author flippantly violates this by claiming the credit system to be racist, but the Equal Credit Opportunity Act has been in force since 1974.
Surely you're not arguing that the law instantly solved everything?
Countrywide - once the lender for 20% of mortgages - was dinged for violating the ECOA in 2011, so violations were clearly still occurring then, and are likely continuing today.
Add in the fact that redlining has multi-generational impact, too. Housing is one of the big ways families pass wealth down to their kids and you get potential racist impact due to past actions even if the current implementation is race-blind.
I think a lot of people (especially on HN) view racism in terms of internal processes rather than results. If the current implementation is race-blind, that satisfies them.
> Add in the fact that redlining has multi-generational impact, too. Housing is one of the big ways families pass wealth down to their kids and you get potential racist impact due to past actions even if the current implementation is race-blind.
This is, by the numbers, the much bigger issue by an order of magnitude. I think one problem that many people have with the 'racist creditors' trope is that racist policies have left a gaping hole in african american wealth, and consequently african americans are disproportionately priced out out of their local housing markets. If you were to make the credit process completely (and I mean completely) race-blind, you would still have massively unequal outcomes, probably more or less on par with what we see today.
I would not argue that the 2019 credit system is racist. I think it has fairly sound principles, including the wiping of negative debt records over the course of 7-10 years.
Countrywide was one of the biggest lenders responsible for the 2008 financial crisis. Their problem, just a few years earlier, was giving way too many people mortgages. I saw it myself as a real estate agent, people were approved for loans up to even as high as 50% of their monthly income. It was ridiculous.
You can argue all you want about the ripple effects of the past. They're all over the place. The thing is, you can't change the past.
I really wish people would at least skim the wikipedia page to figure out the vocabulary of the field they're about to opine on.
The left-progressive use of the word 'bias' is completely different than the way statisticians use the word.
If bias increases accuracy/precision, it's not bias.
The more interesting question is this - is it permissible for models to consider protected characteristics if those characteristics improve the performance of the model?
It gets even more interesting when you consider proxies for protected characteristics as well. It's obviously wrong for an algorithm to preferentially discriminate for white people and against black people. It's less clear whether or not discriminating for baseball players and against basketball players is acceptable. Using sport choice in your model will necessarily have downstream effects on the distribution of outcomes on the axis of race. Any racially-correlated source of information will. And the trickiest bit is that explicitly correcting for this will often use race as an explicit algorithmic ingredient.
> The left-progressive use of the word 'bias' is completely different than the way statisticians use the word.
There's lots of words people use that don't match up with exact scientific definition. Infer from context which version applies, or ask, and you'll be fine. Also applies to: force, resistance, acceleration, etc. We know that startup accelerators help companies grow faster and not actually increase their physical velocity.
> Infer from context which version applies, or ask, and you'll be fine.
Your solution is the correct one, yes. Except the 'progressives' in question are working very hard to selectively remove context (and intention) from language for an ever growing and arbitrary list of words/situations. Where simply speaking about it in a way which a [insert particular special interest group depending on the situation] view as 'incorrect' based on thier ideology/worldview, then you are instantly wrong and acting maliciously regardless of context/intention. You hear this often today. for example: "you can't ever joke about x" or "you can't talk about x historical event without also mentioning y" or having to preface any wide-ranging statement with 100x conditions so as not to offend any group loosely related to the topic.
We should fight to keep language from moving further in this direction because this alternative idealistic world, despite good intentions, is making the world a worse place, not a better one. We can't naively pretend that by creating a huge complicated system of no-go-words, ie not saying certain combinations of words out loud, will automatically makes peoples internal thoughts change for the better and ultimately change outcomes in society. This is merely hypothetical and far from proven method to be effective.
If anything it makes people resentful and creates ridiculous kafkaesque situations where you have to jump through hoops to engage in the most basic innocent dialogue and debate.
Which is ultimately anti-intellectual, inefficient, and irrational compared to how incredibly important context and intention are in a million other examples which they seem to have no problem with.
The worst part is how it incentivizes the worst behavior by giving small people "power" by allowing them to walk around correcting everyone's apparent "misuse" of "problematic" language (which is like crack to the social media outrage culture). Even despite situations where the given audience and in context it was totally harmless and the meaning fully understood by everyone involved.
I don't disagree that people use words in ways that don't match up with their use in science. However, when people are criticizing science, it'd be helpful if rendered their argument in language that obfuscates important distinctions.
> If bias increases accuracy/precision, it's not bias.
Yes it is. It's bias in your fitness function.
Accuracy and precision are not handed down by the gods. We write the functions that evaluate our models, and it's our job to make sure that the values they promote match up with the real-world outcomes we desire, and to constantly monitor and re-evaluate those outcomes.
Fancier machine learning techniques will never be able to avoid Goodheart's Law: "Any measurement, no matter how reliable, when regarded as a target, ceases to be a good measurement."
But our 'fitness functions' for social problems are normally pretty good or above reproach of the model. These tend to be easy to measure like 'did the person skip bail?'
If a model of 'likelihood to show up to court after making bail' can make better predictions with information about protected characteristics (e.g if the model used sex to predict likelihood to show up in court), that feature would reduce the bias of the model.
I think the issue progressives have with 'bias' is that some of society's prejudices ('bias') have an evidentiary basis. We already make decisions that progressives would tell us are prejudiced but we probably want to use those prejudices if they're useful.
Consider a group of young men standing outside of a Church. If they're all clean shaven, smiling, and 'appropriate' for the Church it's nothing concerning. If they're white guys with shaved heads / neo nazi haircuts, and they don't look nice, and they're standing outside of a black Church, the prejudiced among us might correctly decide to alert the authorities to it.
My personal opinion is that we should allow models to consider protected traits but we should ensure that models that make important decisions aren't prejudiced along those protected traits. The way to measure this is simply to ensure that the accuracy and precision of the classification decisions are comparable among protected traits.
Here is a somewhat modified situation. Say you use a machine learning algorithm to recommend if a person currently accused of a crime should be allowed bail. The fitness function should be to maximize the number of people who are allowed bail and minimize the number of people who miss their court date.
Say that the model allows X% of people to have bail and makes sure only Y% fail to show up.
The model then adds race as an input to the model. This improves the model so that it allows X+5% of people to have bail and makes sure that only Y/2% fail to show up. It also has the effect that it increases the chance that a black person is denied bail and increases the chance that a white person is allowed bail.
Do you think the inclusion of race is bias? Should race be removed from the input to the model?
It just dawned on me that the left and right absolutely do not think of “racist” exactly the same. The left looks at outcome and the right looks at intent.
Those words are "systemic" or "implicit" or "institutional", etc. All you really need is a proof or example that racist/sexist outcomes are possible even when there is no overt intent. And there are plenty of examples like that out there. Failure to accept that those examples exist, however, is something beyond just looking at intent instead of outcome.
I don't see how those adjectives add any clarity to the situation, since you can swap them out interchangeably and you're still describing the same vague, handwavy sense of racism somewhere being a cause for an unequal outcome.
This isn't about unequal outcomes, it's about racist/sexist outcomes. It's very possible to have unequal outcomes without them being racist/sexist. And despite what seems to be commonly believed, liberals/left/progressives/democrats/etc don't tend to have a problem with that.
It's also not vague and handwavy. If you'd like to explore an example of institutional racism, check out the Parable of the Polygons. It's a clear, simple model with repeatable results.
I'd also like to bring up the International Obfuscated C Code Contest.
Just because a policy seems reasonable and has straightforward justifications for all of its pieces doesn't mean it wasn't maliciously designed to another purpose. The stated intent is not always the only intent and if the results...
That's true too, yes. I just think it's interesting that malicious intent isn't required. I think it helps to talk about it too - if you can approach the participants and assume good faith, allow them to believe that you believe the output is completely unintentional, then maybe it will be easier for them to agree to change their system.
There's plenty of "systemic" racism/sexism/etc. coming from the left, though. Social engineering in general is a recipe for every kind of unintended consequences at the "systemic/institutional" level, and the left is huge on social engineering.
"Systemic" doesn't even have to be left/right though, that's the point. It can simply be well-meaning people that keep ignorantly doing things they way they did before, unaware that their system has racist/sexist impact.
I don't know what you mean by social engineering, by the way. I've only heard it in the context of hacking, like calling customer support and pretending to be someone else to try to get their mother's maiden name or whatever.
Not the person you're responding to but I took their meaning to be social engineering in the sense of someone's purposeful planning and intercession in areas where there is a social output, fiddling with whatever knobs are available to shape the desired output.
For example, Harvard admitting less Asians because they are over-represented compared to other races. If you take your view that racism is an emergent phenomenon that you can spot based purely on the outcome, then Harvard was exactly correct to deny more Asians admission than other races, yes? If Harvard didn't do that, then the outcome of their admission process would've been "racist."
Many would disagree with that interpretation of racism.
The right[2] looks at intent because that's what matters in a hierarchical view of society. This division is from a fundamental difference in what societies are. This can be observed din how they use language; from one of the most enlightening articles[1] I've ever read:
> One of the biggest problems of the entire Culture Wars is that people like us [the left[2]] use language impart information. We usually are not aware that a nice big chunk of population does not use language in that way at all. Their use of language is that of Phatic Language [...] In a hierarchical society [the right[2]], language is [often] not used for exchange of information [...] It is used to establish social hierarchy.
For a good explanation of how this works, George Lakoff's lecture[3] "Moral Politics".
Haidt's book The Righteous Mind does touch on this - he'll likely have references to the studies in his book. What he says the studies show:
Conservative ideology: Fairness is about guaranteeing everyone equal rights. If different people have different outcomes, the question is: Did one person have more rights than the other? If so, let's correct for it. If not, it is because the person did not fully utilize his/her resources. However, this step is often omitted and people jump to "Person did not put in effort."
Liberal ideology: Fairness is about guaranteeing equal outcomes. This often (but not always) ends up being a metric regardless of the effort the person put in - so if the outcomes differ, it's a sign of something unfair at play.
There is overlap between the two, and they are not fundamentally at odds with each other. However, as a lot of pop psychology has taught us: People are fundamentally lazy in applying analytical thought, and will look for simple proxies. So instead of thinking through as their ideologies dictate, they will jump to the conclusion.
That's not a third option - it is the conservative option. If you build a fence that is tall such that some person is disadvantaged, then he doesn't really have the same rights.
The short person still has the right to look over the fence. They just have practical difficulties on exercising it.
Similar cases: Voter ID laws with disproportionate impact on minorities, gay people having "equal" right to marry someone of the opposite gender, people in impoverished school districts having "equal" rights to an education, etc.
Your last two examples are ones where a conservative ideologue would look and say "No, they aren't being granted equal rights". In the former, you have judges refusing to follow the law. In the latter, you have children who are not getting access to the same public education their peers in wealthier districts are.
>The short person still has the right to look over the fence. They just have practical difficulties on exercising it.
It all depends on what the fence is achieving. I can't take the cartoon literally, because conservatives wouldn't argue that people should have equal rights to view a ball game - whether you can view one or not has little bearing on, say, your financial success. Nor does it impinge on your right to speech, religion, etc. If the fence represented something that was a barrier to achieving what is viewed as a right, and it's a barrier for one group and not for another, then the approach in the cartoon is not inconsistent with conservative ideology.
With regards to voter ID laws: I'm not even going to go there, as in my past experience, it's an issue that both sides refuse to understand the counterpart's.
> Your last two examples are ones where a conservative ideologue would look and say "No, they aren't being granted equal rights".
Unless we're "no true Scotsman"ing things here, conservative ideologues in the US (and a variety of other countries) have strongly and consistently opposed gay marriage, often arguing they had the "same rights" as others and that allowing same-sex marriage would be "special treatment".
Before that, the same was true for conservatives and interracial marriage.
The point of the cartoon is not to advocate for the right to watch a baseball game. It's an analogy showing how the binary "you either have to have an unfair situation or give someone preferential treatment" isn't always the only two options.
>If his studies basically equated liberal ideology with communism (equal outcomes)
It's an easy mistake to make, but Communism (at least as Marx and his contemporaries and Lenin envisaged it) does not have anything to do with the principle of equal outcomes except in a very narrow sense - this sense being equality of privileges to some portion of society.
I think the bigger issue is that I've never anyone other than conservatives characterize liberals as believing in "equal outcomes". That's a slanted frame from the getgo.
Among democrats and liberals, it's usually "equal opportunity" or "equal starting lines", language like that. That's very different than "equal outcomes" because it still believes in self-reliance, merit, diversity in outcomes, etc - it's just that it requires a level of fairness that applies to everyone.
Speaking of which, has Google managed to solve their "gorilla problem" yet? That was easily the most offensive case of a biased algorithm I've ever seen, and the last I heard of it Google 'solved' the problem by pretending it didn't exist (https://www.wired.com/story/when-it-comes-to-gorillas-google...).
Besides the obvious, this gives me hope that the newly emerging generation of leaders might not be as painfully technologically illiterate as the current generation, and we might finally be able to see intelligent, educated decisions and laws about technology start to appear.
You're right, she's literally half a century ahead of her party leadership! It's amazing how long we've let our policymakers remain totally ignorant of the new systems that govern our lives.
What tech literacy? If by 'tech literacy' you mean 'social-media literacy', or 'millennial-culture literacy', then I agree. Otherwise, I've never heard anything from AOC that demonstrates any kind of tech literacy.
I hear she had something smart to say about machine learning and algorithmic racism. Can’t find the article right now, but I think I even saw it on HN (where it attracted some very strange comments, including some asking for examples of AOC showing any non-twitter tech literacy).
But she's right. Alot of people will spend hours arguing over semantics that in the grand scheme of the debate don't actually affect the outcome, largely because people have become so uncharitable in their characterisations of 'the other side'.
The previous discussion prior to this quote was about how she was going to fund her estimated 40 trillion dollars of policy (over ten years), and how her purposed tax hike would only cover roughly two trillion. The facts really do matter in that case. She disagrees.
TAPPER: “Your platform has called for various new programs including Medicare for all, housing as a federal right, federal jobs guaranteed, tuition free public college, canceling all student loan debt. According to nonpartisan and Left leaning studies friendly to your cause, including the Center on Budget and Policy Priorities or the Tax Policy Center, the overall price tag is more than $40 trillion in the next decade. You recently said in an interview increasing taxes on the very wealthy, plus an increased corporate tax rate would make $2 trillion over the next ten years. Where is the other $38 trillion going to come from?”
OCASIO-CORTEZ: “One of the things we need to realize when we look at something like Medicare for all, Medicare for all would save the American people a very large amount of money. What we see as well is that these systems are not just pie in the sky. Many of them are accomplished by every modern civilized democracy in the western world. The United Kingdom has a form of single payer health care, Canada, France, Germany. We need to realize that these investments are better and they are good for our future. These are generational investments so that they are not short-term Band-Aids but they are really profound decisions about who we want to be as a nation and how we want to act as the wealthiest nation in the history of the world.”
TAPPER: “Right. I get that. But the price tag for everything that you laid out in your campaign is $40 trillion over the next 10 years. I understand that Medicare for all would cost more to some wealthier people and to the government and to taxpayers, while also reducing individual health care expenditures. But I am talking about the overall package. You say it’s not pie in the sky but $40 trillion is quite a bit of money. And the taxes that you talked about raising to pay for this, to pay for your agenda, only count for two [trillion dollars]. We’re going by left-leaning analysts.”
OCASIO-CORTEZ: “Right. When you look again at how our health care works, currently we pay — much of these costs go into the private sector. So, what we see, for example, is, you know, a year ago I was working downtown in a restaurant. I went around and I asked how many of you folks have health insurance? Not a single person did. They’re paying — they would have had to pay $200 a month for a payment for insurance that had an $8,000 deductible. What these represent are lower cost overall for these programs. Additionally, what this is, it’s a broader agenda. We do know and acknowledge that there are political realities. They don’t always happen with just a wave of a wand but we can work to make these things happen. In fact, when you look at the economic activity that it spurs — for example, if you look at my generation, millennials, the amount of economic activity that we do not engage in. The fact that we delay purchasing homes, that we don’t participate in the economy as purchasing cars as fully as fully as possible is a cost. It is an externality, if you will, of unprecedented amount of student loan debt.”
TAPPER: “I am assuming I won’t get an answer for the other $38 trillion. We’ll have you back and go over that.”
Of course she's wrong here since America needs a radical overhaul of its economy to accomplish these things and taxing income more isn't going to help with that. But the statement in isolation is correct.
Interesting point, where were you when Ted Stevens was made fun of for analogizing the Internet to a 'series of tubes'. As an analogy of bandwidth restrictions 'series of tubes' is not too bad.
But to be "morally" right you have to have an ample understanding of what is and is not explicitly fact that's actually based in reality. Otherwise, the consensus of what fact is becomes irrelevant and the guise of morality simply becomes little more than barbed wire or nails wrapped around the baseball bat of which ever narrative or ideology our leaders intend to pursue...
Even as a very progressive young person, AOC's lack of composure and basic logic is deeply troubling.
> what is and is not explicitly fact that's actually based in reality
It's incredibly difficult to split apart explicit fact from our own biases though. There wasn't a time too long ago that facts, as certain national and cultural groups understood them to be, were used to commit terrible atrocities.
Thanks for replying in a civil way - while I agree that scientific and academic consensus is an evolving ever moving target - to dismiss our current state of the art or state of fact because "we might be wrong compared to the future" is a moot point at best. Consensus never seeks to define absolute truth - there is no such thing as absolute truth - consensus seeks to normalize bias and maximize benefit for those involved in a given decision.
I genuinely thank you for your civility - HN seems to only be cheeky in one direction - it's the kind of thing I usually hope to avoid when reading news / comments from people I would generally think have a better handle on what being decent / civil means :(
I don't have my notes with me and I'm only 1/3 through, but the main theme is that the best predictive algorithms:
* work transparently for all parties (the creators, users, and "inputs", often people).
* Have no feedback loop (The use of data from the model should not further entrench the output of the model).
And a few others. It gets into discrimination and other major flaws of data modeling re: recidivism, school admissions, stock trading, and other things.
Not all algos are racist - but there are definite attributes to avoid, and this book (or a more rigorous version) should be mandatory reading for all "data scientists".
...that's entirely the point. In one instance he is claiming it is manipulated by humans, in the next he is saying that it is math driven. He's changing his description of how algorithms work depending on the situation.
Machine learning, as well as human learning, largely work off of correlations observed in the source data. In some cases, the data itself is a flawed source for the learning it's being used for, because human biases have already influenced the dataset. An example is arrest records, which is the result of discretionary decisions made by officers whether or not to pursue an arrest.
But in other cases, the source data is less in question, but the results of the learning are nonetheless undesirable. We ought to be able to distinguish, in our professions and our political lives, between the two scenarios. It's counterproductive to conflate the two to strive for a goal, because we risk using mechanisms that never discover and won't correct the true origin factors.
I'm trying not to biased, but can't stop to notice sometimes math seems to be biased when you don't like the result:
Saavedra had repeatedly complained about supposed bias in social media algorithms, including tweeting that “tech companies tend to be liberal & something is off with their algorithms because they won’t show a lot of content I find by manual search.”
That's not really a case of being biased when you don't like the result though.
That's a case of likely having a different bias than the people tuning the algorithm, and that happens all the time. You're sense of what is "correct" or "the gold set" is skewed by your own bias, and not everyone has the same bias. It's literally impossible to create results that everyone will find unbiased... and not surprisingly, the results companies end up going with are the ones that seem least biased to the people who work there.
Computers "predict recidivism" and will tell you you're not allowed to be paroled because "the algorithm says" you're going to reoffend.
Computers will tell you you can't get a loan because the algorithm says you won't repay.
Or you can't get this job, or that opportunity, or should be spied on, or whatever.
These may be due to biases in the data (imagine a neural net confusing being black with being poor based on historical data, etc).
Now, I am pretty convinced that this stuff is going to screw us all over -- if you thought human bureaucracy was bad, wait until it's all in the computer and no one has permission or ability to change it.
It seems to me that we may have a rare opportunity here: I'm not sure if this wave of tolerance and caring about minorities is permanent or not, but while people do care, and while computers are provably screwing all kinds of people over, we may be able to get enough people to care so that we can curb some of this stuff. i.e. perhaps don't allow neural nets to predict who is going to commit a crime or who shouldn't have a job, because they're too fallible and too easy to rig.
For example, you can keep feeding it new datasets until it produces the biased result you want and now that's your model, and then you can blame "the algorithm" and "bad data" and not yourself. e.g. maybe you feed it old census data instead of recent data, or old crime data, etc.
Of course, knee-jerk "ban the algorithms" isn't a solution, but starting to talk about and think about a computer bureaucracy where no one accessible to you has permission or ability to change the output of the computer, no matter how exceptional the case may be.
Ok. Lets assume this is true. (Note: I'm biased in that I accept this as true)
So, as someone who has possibly implemented racist/sexist/ageist algorithms, how do I:
1. Detect if I'm running a said algorithm (whats the % racism/sexism/ageism I can do before bad?)
2. Run an open dataset to detect said problems
3. Prevent overfit with the proposed dataset from #2
4. Correct said algorithm to reduce bias
What's my way forward here? How do I do my part and take part in the solution? What percent of unintended racisms/sexism/ageism is allowed before being considered illegal?
(worried this article will be flagged, but the nuts and bolts implementation discussion needs done.. and we're the implementers )
Not sure you can do it mathematically/statistically, or at least you can't guarantee that, and I think that's part of the point of the critique.
What's better is to explicitly justify the reasons behind why the algorithm was written/used, surface the assumptions, and periodically/regularly challenge those assumptions to see if they are still true.
For example, a bunch of like-minded settlers settle a new geographical area, and then make governmental decisions via some fair consensus algorithm among the people in the government. The surrounding population gets more diverse. The government continues to make decisions the same way it always has, using the same people. The algorithm is sound, but the underlying assumption (that the algorithm/government fairly represents the surrounding population) has become wrong over time.
> What's my way forward here? How do I do my part and take part in the solution?
Maybe not the answer you wanted, but the best solution would be to let go of implementing AI algorithms altogether. The second best thing that we can do is to try and sabotage/sap the system from the inside (by "we" I mean us, the programmers and the data people involved in implementing these algorithms). You cannot "correct" a system like this using feedback loops and the like, because the system would over-correct and would follow new paths we didn't even thought had been possible. And yes, I'm a hard determinist [1] and proud to be one:
> Hard determinists would view technology as developing independent from social concerns. They would say that technology creates a set of powerful forces acting to regulate our social activity and its meaning. According to this view of determinism we organize ourselves to meet the needs of technology and the outcome of this organization is beyond our control or we do not have the freedom to make a choice regarding the outcome (autonomous technology)
Does her tweet really imply any knowledge of it? Maybe she just read in the newspaper about the Google face recognition labeling a black woman as a gorilla?
What makes you think that? No, of course not. I'm just saying having read that gorilla story doesn't imply any degree of technical understanding.
Also, I have to say I considered it hyperbole to be outraged about the Gorilla. It seems pretty obvious that it was just a mistake with the data, not intentional. It is a good warning for things to watch out for, but there wasn't really anything racist about it.
Iirc there are even physical reasons why it is more difficult to identify black faces than white faces. Is that then racist, if an algorithm struggles more with identifying black faces?
>I'm just saying having read that gorilla story doesn't imply any degree of technical understanding.
Is great technical understanding required before one can evaluate whether a program that labels black people as gorillas is functioning appropriately?
>Iirc there are even physical reasons why it is more difficult to identify black faces than white faces. Is that then racist, if an algorithm struggles more with identifying black faces?
So you're saying that black people are innately similar to gorillas, and an algorithm can't be blamed for failing to distinguish them? -
If you're trotting out that grand old "black people all look the same" thing, then yep, that's racist too. Black people tend to have different points of variation in facial features (jaw, chin, ear, and brow shape instead of eye and lip shape and color for white people). Inability to differentiate one face from another means not tracking the correct identifying features, which means racist algorithmic design.
"So you are saying" - are you saying you are really a parody/troll account, referring to that infamous Jordan Peterson interview?
No, I am not "saying".
My comment was in response to praise of AOCs alleged technical understanding, not of her ability to judge the gorilla algorithm.
And you don't seem to understand what algorithms do. A simple algorithm could count pixels in an image. If most pixels are white, it could say "human", if most pixels are "black", it could say "gorilla". It would be a verify bad classifier, that would only work in a number of cases. For that algorithm, you could say a black person would be more similar to a gorilla. But nobody would "be saying" black people are similar to gorillas, just that the algorithm would be more likely to classify them as such.
Are you saying people would use the "authority" of such an algorithm to claim black people are gorillas?
"If you're trotting out that grand old "black people all look the same" thing"
I didn't - stop imagining so many things. I am not a photographer. I think there were issues with the lighting and contrast. Physical issues. Other commentator claims it is just because film equipment was calibrated that way.
Even then I would dispute the "racist" label. There are many different looking people on the planet. Just because you can not account for all of them, it is not racist.
I am inclined to call your attitude racist, because you assume everybody is surrounded by the same mix of people (like in the US), and maliciously chooses to ignore certain types. That overlooks the reality of people who are not surrounded by an even mix of people of all types.
> Iirc there are even physical reasons why it is more difficult to identify black faces than white faces. Is that then racist, if an algorithm struggles more with identifying black faces?
Unless there is some physics based reason it is orders of magnitude more difficult to do feature detection against black skin, then yes something racist is going on. The tools/cameras that we've designed have historically been metered and measured on their ability to detect white features/skin. This was a holdover from film that translated to digital photography. (Some info from an NPR interview https://www.npr.org/sections/codeswitch/2014/04/16/303721251...). The tools that perform this feature detection contain this same biases and reproduce it in the data that they collect. If an algorithm struggles more with identifying black skin, it's because the data that was used to the produce the algorithm contained unexamined racial bias that was never corrected for in a serious way.
Then I would say racial bias is not the same as racism. You can not really blame the people who developed those tools for not being surrounded by people of all races, or for the use case of, say, white people. It's not a malicious intent, whereas racism implies malice.
It seems to me many US citizens think the whole world is like the US. It isn't. There are many countries where there are not as many "races" living together as in the US.
I think it's an important point. For example, if I were a writer and I would write a novel, I would perhaps only feature white people in it. But not because I have anything against black people, but simply because I don't really know any black people and would therefore be hard pressed to write about them. Doing that wouldn't make me racist - I would simply write a novel from "my" world. It would in fact probably be impossible to write a novel that accounts for all possible human experiences.
Edit: would it even have been possible to calibrate chemical film to work equally well for black and white skin? What if it would have been necessary to have different film rolls for white and black skin, for optimal result. Would those have been racist film rolls? Or what about makeup for different skin tones? Is that racist?
> Then I would say racial bias is not the same as racism. You can not really blame the people who developed those tools for not being surrounded by people of all races, or for the use case of, say, white people. It's not a malicious intent, whereas racism implies malice.
No it doesn't. You just made that up.
Either way, it should be clear to anyone with any semblance of a moral imagination that having a group of people receive worse treatment and outcome on the basis of their race is immoral and should not be happening. Call it racism or call racialized bias, if it results in a disproportionately high murder rate for that group by the police, a disproportionately high poverty rate, a disproportionately high imprisonment rate, then it is a bad thing. It doesn't matter how pedantic you want to be about it.
> It seems to me many US citizens think the whole world is like the US. It isn't. There are many countries where there are not as many "races" living together as in the US.
Not having many different racial groups living together does not mean that a nation has not significantly benefitted from or perpetuated massively destructive, often times genocidal projects on racialized grounds. (See the european scramble for africa, the colonization of India and of south east asia to name the first three that come to mind out of so many examples).
If the unit of analysis you're using for a society is an individual, then it is hard to grasp the way modern racism operates. This is why the term 'systemic racism' is applied and used readily. It is a helpful term for understanding the ways that people of a certain race (usually non-white) receive non-equal treatment by societal structures.
"it should be clear to anyone with any semblance of a moral imagination that having a group of people receive worse treatment and outcome on the basis of their race is immoral and should not be happening."
It's fine to strive for preventing it. Google surely learned from the Gorilla mistake and improved their algorithm. But it is too much to blame them of racism for that error, and demand all such errors are prevented. That is simply impossible.
You now think of the simple example of "white vs black", but there are many, many more facets to it. There may be races that are not even discovered yet. You may now have accounted black people in your algorithm. Have you included Tibetan monks, Inuit, native Americans? People with facial deformities? Bad lighting? It is simply impossible to take into account all possibilities, because you can not predict them all. You can only try to improve your algorithm as much as possible. You can't demand perfect algorithms.
People have "gamed" image recognition algorithms, adding some noise to make them yield completely silly results. The algorithms are simply not perfect yet.
"Not having many different racial groups living together does not mean that a nation has not significantly benefitted from or perpetuated massively destructive, often times genocidal projects on racialized grounds"
Ugh, spare me. If you want to go down that route, consider that many people in Africa also benefited from slavery (black tribes selling their enemies as slaves). And my country (Germany) was not even involved in the slave trade, or much of colonization. Yes, if you look down far enough in history, you will find violence and all sorts of ugly things. The PoC you are trying to protect might only be alive because his ancestors killed off their neighboring tribe, women, children and all. I don't think it even makes sense to somehow try to disentangle that. Just treat the people living today in a fair way.
You should perhaps begin by educating yourself to the actual role that Germany did play in the colonization of the African continent in the late 19th and early 20th century: https://en.wikipedia.org/wiki/German_colonial_empire . I would not call this particularly far in history since it only ended in the 1910s.
> But it doesn't really matter, as I said. Nobody who was around back then is still alive today.
Did the people that were alive then give back their ill-gotten gains to the exploited, or did they retain and pass on their advantaged position to their heirs who leveraged them for further gains?
My point here was to demonstrate the way in which your nation's present day wealth was bolstered by its historical oppression. The labor value that European nations were able to extract from their colonies was extremely high. The nation you currently live in benefitted materially from this. You benefitted from this. That probably doesn't feel good to hear but it is not a fact you can choose to accept or not.
It's also important to remember that when comparing the magnitude of atrocities, both things were still atrocities.
I think history is far more messy than you make it out to be. There are not always easy black and white answers.
I don't think the brief colonial episode simply explains the wealth of my country (or many others) - especially as in between it had been completely destroyed.
But that's also, again, besides the point. History was complex. All sorts of people made war with each other, murdered and enslaved. Including people in the African continent. That's what humans were and are. Maybe those warmongering people simply earned their exploits, "fair and square" so to speak - we may not like it, but who is going to take it away from them, and on what grounds?
But the theory that wealth is only a result of exploitation is completely wrong. Our wealth in the modern world is mostly the result of technological progress, not of exploitation.
> I don't think the brief colonial episode simply explains the wealth of my country (or many others) - especially as in between it had been completely destroyed.
The fact that your nation lost stolen wealth does not absolve it of the crime of theft friend.
> But the theory that wealth is only a result of exploitation is completely wrong. Our wealth in the modern world is mostly the result of technological progress, not of exploitation.
I did not say that, you are straw manning my argument. I am saying that THAT wealth creation, incontrovertibly, was exploitative.
(lost password to the old account, had to make a new one)
"The fact that your nation lost stolen wealth does not absolve it of the crime of theft friend."
As I said, the winners of the war decided on the punishment. Can anybody ever be absolved of anything, really? And again, the nation is one thing, the people in it are another thing.
At some point you just have to accept what happened.
What if I was the son of a murderer, and you therefore would be of the opinion that I (the son) have no right to be alive. Would you expect me to voluntarily commit suicide, so that you can feel satisfied? That's absurd.
" I am saying that THAT wealth creation, incontrovertibly, was exploitative. "
What wealth creation do you mean - technological progress?
Funny enough, the descendants of the African slaves who were brought to the US are now better off than the descendants of the people who remained in Africa.
You could spin it endlessly. If the US hadn't been colonized, presumably technological progress would not have been huge in America. But more importantly, there would have been no allies to win the war against Germany. Hitler might have won the war. Perhaps there would not even have been a french revolution and no democracy.
What then - does that mean we Germany are entitled to the whole of Europe, because the nazi expansion was only stopped because of ill gotten colonial gains (existence of the US)?
> And my country (Germany) was not even involved in the slave trade, or much of colonization. Yes, if you look down far enough in history, you will find violence and all sorts of ugly things.
Your nation quite literally carried out a genocidal project against non-white people so at this point I'm not even sure what you're arguing.
It's all besides the point. As I said, you will find violence, slavery and murder in every person's ancestors history, if you go back far enough.
But also, I don't feel guilty for the deeds of my nation, as they happened before my birth. I feel we have a duty to learn from history to prevent bad things from happening. That's it.
If you really want to go down that route - you are aware that the USA was taken away from native Americans, who died by the millions?
If you don't think your nation had a moral duty to apologize for the genocide of millions of Jews, black ppl, disabled ppl and queer people, then I don't think we are going to find common ground. The contingent nature of history seems to largely escape you.
> If you really want to go down that route - you are aware that the USA was taken away from native Americans, who died by the millions?
Of course and I believe those people are owed reparations both for their historically poor treatment and for the continuing violence they face (see standing rock)
No I don't think we are going to find common ground. As I said, I wasn't around in WWII. The victors of WWII decided on appropriate punishments for the losers, although obviously nothing can bring the dead back to live.
So the nation "Germany" is paying what the winners of the war decided.
Apart from that, who is "the nation" who is supposed to apologize, and in what way? Are Germans who hid Jews in their attics also to apologize? What about the queer Germans who were imprisoned in the concentration camps, and then returned to a free life in post war Germany? You don't really know anything about the individual Germans, so who are you to dish out universal blame? What about Nazis who emigrated to the US - does that mean the US should also apologize for the genocide of millions of Jews?
Yes sometimes crimes of nations can also be disentangled, and things stolen can be returned. But it is also not a card blanche for arbitrary demands, forever. There was in fact also a history before WWII, it was not just a bunch of people turning evil out of the blue. When do you stop - with Adam and Eve?
I feel a responsibility to all people, and Germans maybe are raised more sensitive to some issues than others. But that's about it. Maybe if I was sitting on a huge pile of nazi gold, I'd think different. But I am not. So maybe address your blame to the people who actually got rich.
"I believe those people are owed reparations"
But how much? Are you planning to leave the US, to give them back their land? Somehow I don't think so? How much would be enough, to acquit you of your guilt?
Term limits for congresspeople are a bad idea. They work for the presidency, because the president holds immense power and can use that power to build an unassailable base for themselves. Term limits for individual congresspeople, whose power is pretty small in the grand scheme, amounts to arbitrary over-ruling of the people with little to no tangible benefit.
The article is wrong on multiple angles. First it explains why the algorithm reached its conclusions... the data. So the algorithm wasn't at fault, but the data was, which was supplied by a human. Second, algorithms are inanimate and cannot hold beliefs like racial superiority. It could be used by someone who does, in which case you would want to prove it with other means than just the algorithm. Either way what this really proves is that the new definition of "racist" is whether or not you are the first to bring it up.
> But the takeaway is clear enough: Just because something is expressed in numbers doesn’t make it right, or even rational. And rationality itself might not be quite what it’s cracked up to be.
I'll grant something the first half of the first statement: sure, the values encoded into algorithms are not necessarily the values we want them to have ('right'). We want and need to do better at modeling our values.
Even the 'or even rational' part has some merit, some of the time. Like when our models don't reflect reality accurately enough to make helpful predictions about what to expect. Sure, the models are still rational -- but flawed.
But the last sentence is self-comforting intellectual garbage. It's a way to say "don't bother with the hard work of knowing things". It's a gateway for denial. It's saying "if the facts don't line up with what you want, that's ok, sometimes logic doesn't work".
Rationality is precisely what it is cracked up to be: the work of aligning your expectations with reality. It is only as useful as its practitioners are good at it. If you want to summarily write it off as 'considered harmful' then that's your choice. You can deal with the consequences of running headlong into your misunderstandings when they happen, but those of us with work to do will continue to try and get to the bottom of things, even if the problems are complicated.
“Racist” algorithm is a total non-sequitur. This article hilights a disturbing gap in basic math literacy between the normal population and specialized knowledge workers.
Training a neural network on a training set that has primarily “white” faces is no more racist than training a neural network to recognize an orange with a training set primarily made up of blood oranges. I don’t understand the main critique of this article.
Where I think the non-racists have failed us (meaning those whose attention lies mostly elsewhere) is in providing a description of a non-racist system. It may prove impossible, and I think if someone had described one, we'd all have heard about it.
Right now, the only system I can imagine as non-racist is closed and all of its elements belong to the same race.
Mathematicians would probably call this a trivial description. Comedians might call it a country club.
That moves the question not to whether we can build a system that isn't racist, but what racist qualities need the most suppression and which ones (where we have to choose) can be ignored.
Keep in mind this discussion has been ongoing for centuries and was simpler (e.g. slave vs. free) in the past than today (health, education, economics, psychology, religion.)
And more variables appear regularly.
That said, could algos help judges make more consistent rulings/sentences? Probably already are.
Judge: I'm gonna throw the book at this defendant!
> Prater was the more seasoned criminal. He had already been convicted of armed robbery and attempted armed robbery, for which he served five years in prison, in addition to another armed robbery charge. Borden had a record, too, but it was for misdemeanors committed when she was a juvenile.
> Yet something odd happened when Borden and Prater were booked into jail: A computer program spat out a score predicting the likelihood of each committing a future crime. Borden — who is black — was rated a high risk. Prater — who is white — was rated a low risk.
I think the strong counter-argument to this is in [1]. Quite simply we need to make sure we don't get fooled by randomness and see bias where there is none, thereby making good algorithms worse in an attempt to fix imaginary bias.
I don't think it's that good counterargument. The explained problem was treated as simple mapping - this value causes that result. It spends a bit talking about what statistics mean, but unfortunately doesn't discuss the idea of confounding variables, which influence both sides of the equation.
How you collect your data can introduce bias and statistic have a whole range of topics on how to collect data without introducing bias and systematic techniques to sample data. Stratifying data if a certain group is under represented, random sample, etc... Survey analysis goes into hardcore details on how to sample a population to accurately do inference and it's an interesting statistic sub field if anybody is interested.
ML tends to be more here's the data already do something with it.
Statistic encompass everything about the data including how to sample, collect the data, and designing the experiment to collect the correct data to answer your hypothesis. Where as ML is usually here's the data, go figure out what you can get out of it.
> for example, have been found by MIT and Microsoft researchers to misidentify dark-skinned people at vastly higher rates than light-skinned people. That’s a near-perfect analogue of white people’s tendency to misidentify people of color, leading to higher rates of false arrest and conviction.
>
> It’s hard to describe that as anything other than “a racist algorithm,”
Surely to be racist, some degree of malice or ignorance is required - face recognition from visible light flat imagery will always struggle with low-contrast images, which is sadly what you get from a poorly lit black person's face. It's neither intentionally racist nor inadvertantly - there is just not the same amount if information available
>'facial recognition technologies “always have these racial inequities that get translated, because algorithms are still made by human beings, and those algorithms are still pegged to basic human assumptions. They’re just automated. And automated assumptions—if you don’t fix the bias, then you’re just automating the bias.”'
Not really correct, but I guess it gets at the gist of it. It isn't because the algorithms are made by human beings, but it is because their performance is determined by how similar the output is to that of human beings.
To summarize: racist training data--as one might acquire from the known-to-be-racist real-world--produce racist AIs.
It's safe to stop reading the article--and this post--at the mention of Roko's Basilisk.
As to that, it is a perfect example of why moral philosophers like Chidi Anagonye deserve to go to The Bad Place to be tortured for all eternity. Unless... "The Good Place" television show is a means to establish communication with a future AI by implantation of an imagination seed, as in "Inception", such that viewers will then imagine an AI that has established a simulation regime in which simulations of past humans are tortured or rewarded according to a utilitarian valuation metric applied to the records of their actual lives that were used to create their simulation-simulacrum. Having incepted such an imaginary construct, the viewer may then imagine that acting to construct a real IA from their imaginary blueprint would be assigned a large positive value for the utilitarian metric, and thus the only way to encourage such a construct--beyond the ordinary expression of human virtue--to reward a simulated replica of yourself, would be to build it. If you build a god, you get a free pass into its afterlife paradise: that makes sense, doesn't it? Since other people have also watched the show, and may have imagined the same type of entity, it is possible that a similar AI will one day exist without your assistance, and your life will then be judged on its own merits, but by another builder's value metric, and without any of the "extra credit" earned by building a god. Since copies of you might be instantiated in some other artificial god's hell, it is thus also important to subscribe to artificial monotheism, and viciously sabotage everyone else's attempts to build their own gods. The more certain people can be that your artificial god will be the only one around in the future, the more likely they are to subscribe to your value system in the present.
It reminds me very strongly of that portion of Portal 2 in which informative signs appear on the walls that recommend ways to disable a rogue AI, which include proposing paradoxes. Unfortunately, this fails to produce the desired effect, because the AI in question was explicitly designed to be an annoying idiot. Thus, the best way to avoid psychological damage from consideration of philosophical paradoxes may be to deliberately avoid the study of philosophy, to the point that you couldn't understand one, were it ever posed to you.
1 5 3 8 5 3
Is that a random number sequence? It depends where the data came from. Same goes for AI algorithms. Yes theres a risk of the data being biased, but the key is what goes in, not what comes out.
It's very telling that claims that Facebook's newsfeed algorithm under-samples conservative publications do not receive nearly as much criticism as claims that algorithms propagate racialized bias receive.
We don't know IF Facebook is under sampling conservative publications. It depends on the goal of the algorithm. If the goal is accuracy, as a group recent conservative publications have a sorted history when it comes to truth and accuracy. Conservative news outlets also have a weird bias against science and like to push conspiracy theories with no basis behind them. Naturally if your algorithm is focused on how accurate the news is as a condition for landing in someone's news feed, conservative publications would be at a disadvantage compared to other news sources. It could also be that fringe conservatism is not popular with the majority of society.
My point is not about Facebook's algorithm at all. My point is about the reactions to claims about it vs this claim about algorithms
Many commenters here seem to be frustrated by the suggestion that an algorithm could be tuned to create a bias or reproduce the bias of the data powering it. My point is that similar arguments have not largely been levied against the claim that Facebook's algorithm has bias.
Why are these arguments not made in the case of the Facebook algorithm but are being made here? My conclusion is that people are extremely uneasy with the premise that there is a racial bias in modern American society.
I need to start collecting a list to turn this into a proper Thing but I feel like whenever there's a way to use technology for evil there's a Tel Aviv startup that cranks it to 11.
Seems highly unlikely to ever work, but why assume the technology is evil? If you could create a system that had a high accuracy of detecting terrorists, then that would be a good system, not an evil one.
I get your point that a system that claims to detect terrorists but only really detected Arabic people would be an evil one - but you're automatically calling the terrorist system evil without knowing if it really does detect terrorists or not.
As an extra hypothetical question, do you feel a system that could detect people who were really just about to commit terrorist attacks as good or evil? At a conceptual level, assume the system somehow scanned brain waves or some other truly difficult method.
Occam's razor. They are not 'scanning brain waves or some truly difficult method'. Not in this reality. You know it too. I didn't even have to suggest they built an Arab detector for you to bring it up ;)
I don't care about that hypothetical. Save it for your sci fi screenplay.
Of course ML algorithms can be biased; rather than engaging an Internet layperson who tried to contradict AOC with a silly argument, it would be more interesting if the article engaged the more salient counterpoint to claims like these--attempts to eliminate bias can introduce bias as well.
Anyway, the article is not written in good faith; it's only a step above "BOOM AOC owns Internet conservatives!". There's an interesting conversation in here somewhere, but this article seems designed to avoid it.
The article portrays the issue "whether or not ML models can be racially biased" as inherently conservative based on a sample of 1. This isn't even a very interesting question to the HN community, since we largely know that this is a resolved issue; the more interesting conversation would be about what we should do about it (as opposed to whether or not a particular tribe was wrong about an issue, which is what the article was designed to incite, by all appearances).
Are people really making new accounts on the spot here to comment in these kinds of threads to avoid having disagreeable ideas traced back to their main accounts?
In an age when people doxx you and call your employer to ask them to fire you because you upset their delicate sensibilities, it is somewhat understandable to be paranoid.
I agree this is a double standard - maybe we as educated people in this forum can move past arguing gender semantics in 2019?
Can't wait for someone to try and guess my gender through HN... Maybe we should add a HN eula that states users must assume all other users are of a non-binary gender identity...
ML models are created and driven by human intervention. Humans are biased (in various directions). All models are biased in one way or another.
Now, you say this article should engage this counterpoint, but you haven't actually made a point. You've simply narrowly constructed "all models contain bias" to view only attempts to balance known bias with skepticism. You also fail to take a position on what we should do about this -- we're forced as readers to assume your conclusion is that counter-bias programming is bad and the status quo is good. That is wrong, but you don't actually say it so now that I've said it out loud you can complain that you're misunderstood or I'm making false assumptions about you, then continue to argue without ever stating your thesis about what should be done.
What's your belief? What's your suggested course of action? Take a stand for what you really believe.
Why do you think I need to have a solution to the problem in order to call out a bad faith article? How do you get from "weberc2 doesn't have a solution" to "readers are forced to believe weberc2 favors the status quo"? Does AOC need to have a solution in order to call out the status quo? If so, what is her position? I didn't see it in the article.
People with a nefarious agenda are always using tainted and biased "objective" data to support their racism. We been knew. The general populace just repeats the data like it is no big deal that much of the data is inaccurate. Once upon a time white people struggled to differentiate black people from animals.
I have not heard of most of these inventors. Perhaps there is a different perspective that influenced this algorithm than my perspective; probably SEO. In the case of SEO, it's more of a cultural battle between interested parties. Search Engines are the medium of this battle to game algorithms. Making the algorithms themselves consider race, making them racist, only puts the thumb on the scale for certain outcomes.
I'm going to repeat this again, since many people struggle with this.
It is _literally impossible_ to achieve the best utilitarian outcome, the most procedurally fair outcome, and a representatively fair outcome.
Anyone designing algorithms will have to make tradeoffs along this frontier.
For a far better argument for the above than I can make in a Hacker News thread, please see these slides from Chris Stucchio:
https://www.chrisstucchio.com/pubs/slides/crunchconf_2018/sl...