Daniel Kahneman talks about an idea called substitution in his book Thinking Fast and Slow that I think really applies here. Here's the jist: When your brain is faced with doing a task that's going to require a lot of glucose it will look for shortcuts to save you energy. One of those shortcuts is your mind will look for an available heuristic, swap out the energy hungry analysis for the heuristic, and then signal your conscious mind that you did all the analytical hard work.
I think the truth of that matter is, most of us (myself included), don't know how to interview people well. Interviewing is really hard. Rather than doing the hard work by researching the subject and testing ideas, most of us try to imitate successful companies much in the same way the Melanesian cargo cults imitated the construction of airfields and air traffic control towers to lure back that wonderful cargo.
I suspect we like to tell ourselves that we're more analytical because our work can demand rigorous precision. More often than not, I find we developers tend to select heuristics that indirectly test a person on how similar they are to ourselves or people we aspire to be like. Then again, I'm probably making a broad generalization.
I think you're right about the mental shortcuts, but the main effect I see at play in the article is about misaligned incentives.
It's in the organization's interest to hire the underrated, but it's not in the HR recruiter's interest to do that. If the recruiter refers a candidate who looks good on paper but turns out to be a bad hire, no one's going to blame the recruiter. But every time recruiters have the engineers interview a candidate that's not from a top school or a top company, recruiters risk losing credibility with the engineers if the candidate is rejected or ends up being a bad hire. So recruiters focus on the candidates who are the most defensible, not necessarily those who would make the company the most successful.
I liken this to a government contract seller - nobody will ever get fired for choosing IBM/Microsoft/BigCo as a vendor if it fails, but you certainly can for choosing that small shop that can usually get stuff done faster.
Cover Your Ass isn't a good incentive pattern when you're trying to hire engineers, which is hard enough already. At the same time, its really difficult to balance the time. My trick (and I think this is common): interview anyone that comes in on a referral within reason.
Exactly. Recruiting is ripe with principal agent problems.
Recruiters don't get blamed for the undervalued candidates they failed to hire. Nor do they get fired for hiring Stanford grads. (The engineering equivalent of IBM.)
The frustrating part is that even when this is brought up and openly described and discussed explicitly in the context of working towards a better and healthier hiring practice, manager and HR types still insist on the cargo cult nonsense.
It goes further than just an inefficient heuristic or biases. It becomes codified, even venerated, standard practice, and gains an air of certified approval up the corporate ladder, and eventually becomes something that, politically, you're not allowed to disagree with and typically you must even display boisterous and enthusiastic approval of it.
It exacerbates the crab mentality that affects the programming world -- engineers basically start to feel that if they had to put up with stupid, inefficient interview hazing, and if it's all just going to be a political status game anyway, then they're going to drag the candidates down into the muck with them and give themselves over to designing their own hazing interview questions.
The spirit becomes focused on cutting people down, and only hiring people if some gauntlet was unable to cut them down. Without ever stopping to think: what kind of human beings are you actually helping to succeed? And what kind of human beings is this process systematically harming? Nobody cares.
"Good hiring" is fundamentally an altruistic ideal with all the attendant difficulties of idealistic practices in the modern workplace.
This may seem ridiculous given competition for talent and cutthroat markets involved in startups and tech companies...
But from the perspective of low-level hiring, where someone you hire will possibly replace or eclipse you, or where you may empower a team with more resources, or there is budgetary envy (or kill-or-be-killed cuts to funding), or easily masked discriminations aplenty (dotNET vs Java, Stanford vs Berkeley, Indian vs Pakistani, Georgia vs Alabama), conflicting management directives (we will hire the best but only pay for mediocre) or just sheer resistance to change... there are many reasons to resist transparency, hire mediocre, submarine good candidates, backstab rivals, all from obscure chaos of middle management and its wannabe Machiavellis.
Management can put their smiley faces as they impose conflicting priorities and try to parse data that is fundamentally enshrouded by the multiple agendas from actors with highly variant priorities in a complex game theory all they want, but idealism is hard in cutthroat capitalism.
Oh, did I mention that all candidates are lying to some degree of mendaciousness? Oh yeah, that.
I can see HR's desire to follow some sort of procedure though. If you let any manager hire anyway they so desire, it could open the company up to discriminatory hiring practices. By having a codified (if ineffective ) "procedure", they can use this as a defense in a lawsuit and say "look, see ? Everyone gets hired the same way at this company, and the plaintiff was subjected to the exact same scrutiny as everyone else."
I agree that some standards are helpful. However, I think for the sake of the discussion on this thread, we are talking about all sorts of unnecessary, buzzwordy things that are absolutely obviously not necessary, and in fact are even harmful and in some cases may even increase the chances of discrimination lawsuits, and that this is openly understood even by the HR managers who set such policies, and that they are still not changed or even re-evaluated under some framework providing even a tiny consideration for their human impact.
One of the modern classics is ageism in hiring, which is baked right into the whole process in a lot of ways that dangerously straddle the boundary of legality. HR types place a high emphasis on this because hiring younger engineers means they can pay lower wages and those younger engineers have less experience about how employers treat people, so they are less likely to expect basic, dignity-preserving job features, like private working conditions, respect for work/life balance, etc.
Of course, they can't come right out and say they are trying to hire cheap dummies who don't know they are being swindled. So instead they invent code words like "thrives in a dynamic environment" and "handles vague and conflicting business needs well" which are just short-hand for "this worker will not enact the obstinate, incredulous frustration that they rightfully should enact upon learning how we plan to actually treat them" -- which often screens out more experienced candidates who know what shit companies try to pull.
This is how a lot of the nonsense bullet points in a job ad get there. It's also how a lot of nonsense company handbook policies get there too. The bits comprising those characters didn't just get flipped by cosmic rays and randomly appear in the job ad or the company handbook. HR and legal staff placed them there, with intention and forethought -- which, if you're really thinking clearly, means that most job ads are frightening windows into how the company conceives of its workers.
The doesn't actually work if your hiring procedure is actually discriminatory.
There is a concept called disparate impact in US employment law. That means your employment process can't have a disproportionate adverse impact on a protected class, unless there is an actual business requirement that causes the disparate impact.
An example would be: if your job requirements is "must be able to lift 70lbs" it probably will have a disparate impact on women and the disabled. This is fine as long as the job actually requires heavy lifting (such as a mover). But if you require candidates to be able to lift 70 lbs for an office job - then it's illegal discrimination even if everyone who is hired meets that criteria.
I think most people in middle management, especially HR, are looking for "plausible deniability". They want their actions to appear defensive enough to keep their job. So don't hire self-taught, always demand a degree. Follow "industry standard" hiring practices. etc.
They don't understand tech and they don't need to. If something goes wrong they need to be able to demonstrate it's not their fault.
Kahneman also writes in the same book about his experience with interviews as a young psychologist with the Israeli army. His recommendation? A simple (but well thought out) rubric.
Identify 5 or 6 qualities that are essential to the success of your team. Tailor your questions around evaluating for those. Interview a bunch of people, score them on a scale of 1 to 5 for each of those categories. Resist your gut feeling. Then hire the ones with the highest score.
He acknowledges that it's crude but it was more successful than their previous process and makes as much sense as anything else I've read on the subject of hiring.
Amusingly, when he got pushback from the interviewing team for not allowing them to follow their well-honed instincts, he agreed to add "Gut Feeling" as one of the 5 or 6 parameters on which candidates would be scored. That detail, for me, sums up the man's brilliance.
This sounds quite similar to the competency model, which basically entails listing required competencies for a position and then developing a set of specific questions to evaluate each question.
Probably the only good thing I did in my first management job was to create a specific competency matrix for each position and to evaluate candidates against it. We only hired a handful of people, but I was very happy with every person I hired.
The amount of self-delusion among my fellow developers is truly breathtaking at times. Just the stuff I catch myself doing - even while this pattern is a huge pet peeve that I obsess over - is pretty bad.
We need to feel smart, even though half the stuff we do is pretty damned stupid, and we keep doing it over and over again for decades before catching on.
We still have watercooler conversations about issues that were identified in books that are now 30 years old. How depressing is that? Just how clever are we?
Unless one is interviewing for a really specific set of capabilities for a particular job, it's best to try to gauge aptitude for programming positions instead. I believe that it's possible, with the right questions, to check for basic competence in a few areas and to avoid some bad hires. I have a few small problems in my repertoire that can be solved by any good programmer in 5-10 minutes, but can't be solved in even 45 by a candidate who lacks basic understanding of how to analyze a problem, deal with basic abstractions like indirection, etc.
I.e., I've given up on trying to distinguish the great candidates from the good ones, and now I just try to identify the people who just have no idea what they're doing. The error bars around the results of a 45-minute technical interview are just too damn wide, even with research and training, to do anything except try to prevent obviously bad hires.
The hiring manager has a principal-agent problem going on. The manager's incentives are more around avoiding blame for bad hires than getting the best expected value for a hire at the best price.
I would also say that interviewing is hard because we have limited time to decide upon each person and what we judge is (apart from some technical qualities) not what kind of person really is but rather what we believe the person is (based on our limited human perception). And with handful of techniques a person can be really cool from at the first sight (trendy, rock star, etc.) but might be a bad choice in the long run.
Limited time makes it immensely difficult. However, I've found my prejudices against people not like myself have gone way past the interview. I had one intern for 2-3 months before I realized I was really lucky to have him, and it was only because someone else pointed it out.
This person didn't have the right education (classes, not pedigree) nor did he project a passion for coding on his sleeve, but it turned out he was disciplined, a fast autodidact and had a very good sense of priorities. I couldn't have been more wrong about his potential.
It's not just that. I think we too often discount people's ability to grow. My suspicion is we probably place too much emphasis on keeping the wrong people out and not enough on developing the people we have.
>When your brain is faced with doing a task that's going to require a lot of glucose it will look for shortcuts to save you energy. One of those shortcuts is your mind will look for an available heuristic, swap out the energy hungry analysis for the heuristic, and then signal your conscious mind that you did all the analytical hard work.
Sorry I didn't read the book but I am curious if there is any scientific evidence to back this up.
Thinking Fast and Slow isn't a textbook, but Kahneman is a Nobel laureate and most of the book reports experimental findings with 30+ pages of citations in the endnotes. It's a good read.
I have heard there are MRI studies that back this up. Basically people make instinctual decisions and the rational parts of their brains light up afterwards to rationalize the decision they have already made. I am on mobile but that should be searchable.
Humans are very good at pattern recognition and we're optimized for it. Real calculation takes time and energy and might make us less able to survive.
> Real calculation takes time and energy and might make us less able to survive.
Sometimes that grass moving strangely is just the wind..sometimes it's the tiger, the ancestors who sat down to have a good think about it got eaten.
When I was younger and played chess my teacher drilled into me that when you find a good move that's the time to look for a better one, we instinctively play the first 'good' move we see, in fact manoeuvring your opponent into a trap by giving them an obvious 'good' move is effective against people who don't play a lot and very ineffective against people who do (and even then vastly stronger players than I ever was still fall for it occasionally).
Just to be clear, this article is talking about ego depletion which is the theory that willpower is linked to glucose consumption implying exercising willpower in one area will deplete willpower for another area.
What Kahneman is talking about is a tendency for the brain to swap out expensive slower analytical thinking with cheaper and faster heuristic thinking. Both ideas are only tenuously related because they both discuss the role of the brain's glucose metabolism.
https://erikreads.files.wordpress.com/2014/04/thinking-fast-...
I think the truth of that matter is, most of us (myself included), don't know how to interview people well. Interviewing is really hard. Rather than doing the hard work by researching the subject and testing ideas, most of us try to imitate successful companies much in the same way the Melanesian cargo cults imitated the construction of airfields and air traffic control towers to lure back that wonderful cargo.
I suspect we like to tell ourselves that we're more analytical because our work can demand rigorous precision. More often than not, I find we developers tend to select heuristics that indirectly test a person on how similar they are to ourselves or people we aspire to be like. Then again, I'm probably making a broad generalization.