> Most engineering interviewees fail to advance because of cultural/personality/communication issues and not technical competence.
Not my experience. At least when hiring for programming positions, the typical fatal issue is that the candidate's coding is weak.
In my first role as a hiring manager, I didn't stress coding tests for candidates with long work history on their resume. Since then, I learned better.
I've seen candidates with anything between 1 and 15 years of full time coding work on their resumes fail to answer basic fizz-buzz level coding questions.
At least for the candidates I've seen, the biggest single reason why they get rejected is failure to perform the one main function their job requires: coding.
They can't code because you are looking figuratively over their shoulder---the clock is ticking next to a hundred grand in a suitcase.
A single question is pulled from subjects vast enough to fill up years of college and there is no time to research, as one would in the real world. Binary pass/fail with your pride/livelyhood/family on the line.
Please state your last exam conducted under such conditions?
Oh and your questions probably stink. I've gotten plenty of them with recursion that might take ten minutes in seclusion but simply not possible under the gun.
> They can't code because you are looking figuratively over their shoulder---the clock is ticking next to a hundred grand in a suitcase.
There is a difference between me asking you how to optimize a graph algorithm (good luck--I probably couldn't do that on the fly with references in front of me) vs giving me a basic FizzBuzz.
If I let you pick your language for your FizzBuzz, you need to demonstrate at least some of the following things:
I/O of any flavor (I'll even accept Logging if you don't do standard I/O or console I/O)
To be totally honest, I always find these stats about some x% > 50% failure to program at all totally unbelievable. What do these people do all day, if not design and build software? Yet they're apparently completely unable to write code? How is that possible? I understand that there is always a certain amount of slack and underperformance but any workforce would be completely paralyzed by 70% completely incompetent workers. And yet companies everywhere still produce software.
> What do these people do all day, if not design and build software?
Indeed this is a fascinating question, and I explored it a few times.
Some folks who have multiple years of "software engineering" on their resume worked in a place where they did some form of "paint by numbers" programming, producing some work by modifying templates with limited (at best) understanding beyond the specific blanks they were filling.
An example is folks who work with large frameworks that are designed to be extended with little to no coding, such as WordPress and Drupal.
Then they want to advance to software engineering, so they recast their experience - of essentially configuring and deploying software, i.e. sysadmin - as "programming", and get those 5 years represented as "a full time programming position".
There's variants of that, basically "developers" doing very limited ant-work with large frameworks they don't understand and couldn't rewrite from scratch if their lives depended on it.
There's also a bunch of languages and frameworks invented for the explicit purpose that someone with little to no CS knowledge of programming talent could produce useful work.
Most of us here live in a bubble of startups where we often create products from scratch using lean (or no) frameworks. In reality, a scary amount of the "IT work" out there appears to be what I described above. Possibly a substantial majority.
Go through it cover to cover and do all the exercises. All of them. Don't skip a single page or exercise, not even if you think "I already know this", and certainly not because it's too tough.
Once you're done, you should be able to start applying for jobs in whatever language that book taught you. If you're still not passing interviews, take on a serious - but manageable - programming project, such as some simple application for yourself or someone you know. If you're a web developer, a simple dynamic database-driven website is an excellent choice. If you want to do front-end, a simple Javascript game will teach you a lot.
You don't always have to apply for new jobs so early. Lots of tech shops have some teams where actual programming happens, even if most of the rest of the company is "paint by numbers" pseudo-programming. Often you can start to take on more responsibilities that involve actual coding if you show the necessary abilities. You won't have to change companies, or even change teams.
Good/excellent programmers rarely need to interview except as a formality. Consequently, you are, by definition, interviewing weaker programmers.
2) Tool use masquerades as programming
Visual Basic 4/5/6 was one of the original sins in this genre, but it continues now with other tools and frameworks. Being able to point-and-click a tool and plumb it together isn't programming--but people will claim it is. Then when someone asks for an actual program--those people fail.
This does not necessarily mean that those people are not productive. However, it does mean that they can't actually function in a position where they are expected to program.
It's because some large percentage of your time at work is spent in meetings, writing emails, discussing things, etc. where actual ability to do the core task isn't necessary.
And then some large percentage of "enterprise software development" is just hooking up buttons and text fields to a database, and doesn't require any more algorithmic complexity than a few if() statements.
And then some large percentage of the remaining work can be performed adequately by just picking some appropriate ready-made solution off Stack Overflow.
Put them all together and chances are, you can skate by for years (and actually perform acceptably) without ever actually needing to write FizzBuzz-level code for yourself.
It's because current tech interviews are a complete failure and none will admit it. In ten years you'll hear a new fashion and today's will be decried a failure.
The set of interviewees for a software engineering position is not really a representative sample of workers in the industry. Particularly at a company that doesn't have big name recognition, you're going to end up disproportionately getting job applications from the bottom sections of tech talent.
We always get these kinds of responses when interview testing is mentioned. This isn't some high-level algorithm analysis on a whiteboard with a panel of examiners grilling you. This is a simple test of basic competence. It's like interviewing a candidate Formula 1 driver and asking them "so which bit's the brake pedal?" If they can't tell you that instantly then they shouldn't be applying for the position in the first place.
> Please state your last exam conducted under such conditions?
University entrance exams.
University graduation exams.
Any other test that you do which you need to pass to enter or continue your career.
I don’t think I’ve ever been asked a “which is the brake pedal” level question in an onsite interview. Usually it’s more like I’m a mechanic being asked “how do you replace the water pump on a 1999 Saturn Ion, from memory, please.”
That's a benefit of the coding interview, actually. We don't ask you about that particular feature of some library from 10 years ago, which is basically a luck question: you either happen to know or you don't.
We ask you to code a simple task that you should be able to do in your favorite programming language.
Not entirely sure how the discussion of coding interviews veered towards obscure knowledge interviews - the former was expressly designed to supersede and eliminate the latter as one of its core goals!
That’s just it: replacing a water pump on a common used car is a task any competent mechanic should be able to do. But, possibly not without the repair manual. And, I certainly wouldn’t think they’d be able to describe step by step how to do it without the car or manual in front of them!
Likewise, many interview programming tasks can be are ones a competent programmer should be able to do with the tools of the trade accessible (namely Google and time to think).
Some simply are not. I was once expected to write, debug, and modify a concurrent chat server in an hour long interview. I do think that’s a reasonable programming task, but not one to be completed in an hour, under scrutiny. It’s the equivalent of being asked to rebuild a motor in an interview : definitely doable under real world conditions, but not a realistic test to determine whether to hire someone.
Sounds awful. I'd be interested to know what startup it was. But it's no secret that some of the well-known names do tend to drop the ball on hiring and various other aspects of operations.
We ask people to implement a relatively simple algorithm, that’s particularly practical, in the language of their choice. Then we change the parameters and ask them to make it more efficient.
It’s centered around solving a reasonably common place problem, and doesn’t require anything fancy to solve. It’s more of a test in how they programmatically solve problems over anything else. If somebody struggles with it, then they haven’t been bamboozled by some acedemic algorithm problem, they’ve just shown that they’re not going to be able to write clean code unsupervised.
Exactly. I don't think they are accounting for how some people react to interviews. I've had candidates who have been explicitly nervous and stuttering, and we've consciously had to decide not to hold that against them. They turned out to be a great hire.
I think I can do okay in coding interviews, but only if I prep for that style of work first. It's definitely different. I'm perfectly (or at least acceptably) competent in my day to day.
If we’d ever had somebody crippled by interview stage freight, I make be able to take that seriously. The coding skills you need to pass our test is writing a for loop that mutates a dict. We’ve never had anybody forget how to do that in our interview, but we have had people come up with remarkably inefficient solutions to the problem.
I don’t think it’s reasonable to say that people’s problem solving skills (which is what we’re actually testing) become exponentially less efficient during an interview. But even if it was, the test is still doing it’s job, because what would we expect to happen when we give a person like that a deadline, or ask them to review a PR?
Don't need to be "crippled." Simply a shot of adrenaline is enough to shut down most higher-level thinking. At that point the candidate is just pulling things from memory at random hoping they fit.
> remarkably inefficient solutions
The first one usually is. As the problem gets clearer, better solutions become obvious. That takes relaxation and calm thought however, e.g. Archimedes in the bathtub.
> I don’t think…exponentially less efficient
Doesn't need to be a large difference. With multiple candidates, even 10% is decisive.
There's also the psychological factor of knowing something makes the "knower" think it's easy while the others believe it's hard.
I do lots of interviews as a contractor, and must say they have little to no bearing on my ability. In the real world I've solved every problem encountered, given some time. Interviews almost none, the few successes depend on whether I wrote the same algorithm in the last two weeks by chance.
A random throw of the dice at the craps table would be just as accurate and less stressful for all involved.
We're talking about "basic fizz-buzz level coding questions". It sounds like you had a bad experience with something else that we weren't talking about.
(As for 'topic known in advance', I'd suggest that if you don't know what topics that you might be asked about in a job interview, maybe you shouldn't be interviewing for that job.)
I had to twitch my feet to all three imaginary pedals to figure out the brake is the middle one. Even in this metaphor doing and knowing how to do are distinct skills.
You would fail anyway, because the interviewer thought it's pretty clear that automatic transmission was implied, not manual, since the job is about baking doughnuts.
One possible way to address this issue is by asking candidates to talk about some existing code you give them, rather than trying to cook something up off the bat while under scrutiny and not having access to their familiar tooling.
”I've seen candidates with anything between 1 and 15 years of full time coding work on their resumes fail to answer basic fizz-buzz level coding questions.”
I haven’t seen this. I have, however, seen lots of junior interviewers ask idiotic questions and wash candidates out for bad reasons, and then turn around and claim the candidate ”failed at fizzbuzz”, just to cover their own ass (or because they think TopCoder medium questions are “fizzbuzz”. Equally stupid.)
Me personally? I’ve never interviewed a senior person who couldn’t code at all. Not after the phone screen, anyway.
> I have, however, seen lots of junior interviewers ask idiotic questions and wash candidates out for bad reasons
Why are "juniors" interviewing candidates? Let alone juniors asking "idiotic questions", and having decision-making capacity?
> then turn around and claim the candidate ”failed at fizzbuzz”, just to cover their own ass
Coding questions are among the most quantifiable and objective questions there are. If you know of something even more objective, let me know because I'd love to try it.
They are repeatable and have well-defined success criteria. Ideally you have two interviewers in the same room, and it's very clear if the candidate did or did not solve the problem. You can't really fudge it, even as a single interviewer, unless you're willing to outright lie that the candidate didn't write a working solution when he did.
> I’ve never interviewed a senior person who couldn’t code at all. Not after the phone screen, anyway.
Yes, if you do a coding phone screen, then obviously you're screening out those who can't code. I'm sure you saw some impressive resumes fail that screen, though.
Note that "junior interviewers" are not necessarily junior, just junior as interviewers. They may be good coders, but they don't know what is reasonable to expect for the role, or how to test for it.
Furthermore coding questions stop being quantifiable and objective as soon as you add in, "What feedback do you give, when, and how?" Two interviewers asking the same question of equivalent candidates can get very different results based on how stressed they make the candidate, what hints they offer, what followup questions they ask, how well they telegraph whether the candidate is on the right path, and so on.
> Note that "junior interviewers" are not necessarily junior, just junior as interviewers.
Sure, but these are highly correlated. If you've been working in startups for a few years, you will have some experience in both coding and interviewing coders.
Even most of the successful large companies are pretty good at cultivating interviewing skills among their employees.
> Furthermore coding questions stop being quantifiable and objective as soon as you add in, "What feedback do you give, when, and how?"
I would moderate that statement as "become less quantifiable and objective". They're still more objective than "tell me about the project you are most proud of" or (worst) "what is your biggest weakness?" (perfectionism, duh!).
You can also control many of these factors by standardizing interviewer behavior. For example, "no feedback of any kind until the candidate writes a working solution", or working off a predefined set of hints.
Sure, but these are highly correlated. If you've been working in startups for a few years, you will have some experience in both coding and interviewing coders.
Even most of the successful large companies are pretty good at cultivating interviewing skills among their employees.
I would put it the other way around.
My experience is that most startups just put you in front of people and tell you to figure it out. People who have done that for a bit wind up with really bad interview habits because they never get corrected. My experience with large companies (Google and Amazon) has them giving interview training. I learned a lot more about interviewing from that than the informal practice that I got in startups.
> My experience is that most startups just put you in front of people and tell you to figure it out.
Some do, sure. And that's the difference between a startup and a regular company right there: that "figure it out" bit.
You'll mess up and fumble and come up with crazy ideas, 95% of which are worse than what the established companies are doing.
But you'll learn a ton along the way.
It's messy and inefficient but that's how startups are.
I personally learned a ton from hacking interviews at startups and all that "figuring it out".
Also, guess what? All these sacred "FAANG interviews" that seem to have come down from the sky, etched on tablets? They were developed by Microsoft back when it was a startup, then evolved by various SV companies at their startup stage, too.
The next great idea in interviewing also won't come from the thousands of engineers obediently marching to the tune of the same principles everyone is using, but from some startup trying something crazy and ambitious and miraculously getting it to work.
I don’t know if you’ve noticed this, but the average age of programmers these days is just slightly above “right out of undergrad”.
If you wait for the senior people to interview everyone, you’re going to be waiting a while.
”Coding questions are among the most quantifiable and objective questions there are.”
Bullshit. Coding questions feel objective, and programmers love to pretend they’re objective, but like any other human evaluation process, they’re subjective. It’s not like you automatically get the job if you get the answer right.
Unless you’re verifying that your team is generating consistent, reproducible outcomes, I can pretty much guarantee that they’re not.
OK, I suppose lack of seniors is a legitimate constraint.
Still, interviewing is so important, that I typically give it a high priority. Even when I had only one other senior engineer, I made sure each candidate spent a long time with each of us.
Typically, I'd go in with a junior developer, and he'd go in with some other junior developer. I'd even see the same candidate twice rather than have two juniors interview him.
I've been on the other side of that kind of interview and I know how badly it can get messed up.
> Coding questions feel objective, and programmers love to pretend they’re objective, but like any other human evaluation process, they’re subjective.
I don't see a reason to swear, as this is a friendly discussion. Either way, we'll have to agree to disagree.
Coding questions ideally yield an answer that can compile and run and return correct output for at least some valid sets of input. That's about as objective as it gets.
> Coding questions ideally yield an answer that can compile and run and return correct output for at least some valid sets of input. That's about as objective as it gets.
Except for coding style (incl. naming convention), code organisation, test coverage, arch style, level of abstraction..plus a myriad of other things. I don't know anybody who treat the code in a coding question as a black box. It gets about as subjective as you can get when reviewing, with the (objectively) correct answer weighted the least.
Shit code that gets the right answer rates worse than code which ticks all the above subjective boxes but gets the answer wrong.
This topic was pretty lively last time I commented about my interviewing practices. Coding is a creative effort -- yes. It's not objective.
But I see no other way to test if the person is competent than sitting them down with a problem to solve -- if only to have them explain what they do and don't understand, and what their process is.
This is a key point. I see a lot of criticism of the coding interview. But what's the viable alternative?
My goal is to figure out the candidate can do their job - write software - and that myself and the rest of my team would enjoy working closely with them on their daily tasks.
How else do I figure that out, if not by a coding exercise?
I’m not saying that you shouldn’t do a coding exercise. I’m saying that coding exercises are overrated and far too difficult.
Programmers today think that TopCoder questions are “FizzBuzz” tests. That’s idiotic. Joel Sposky wrote that blog post to say that you should do an absolutely minimal check that someone can write code, then move on to more important things. It’s just bizarre how people have twisted that over the years.
But to answer your question: focus on what matters. Communication, writing skill, the ability to read code (this is easily 10x more important than algorithmic wizardry IRL), good habits (e.g. “does this person insist on putting all of their code in a single file?” - an actual thing I have seen!!), opinions that match your team, etc.
All of these things can be answered in interviews, but nobody cares to try.
Communication is obviously being tested in both phone screens and onsites. It's not like a candidate in a coding interview can simply start coding silently, finish, and then pass to the next stage because their code works. A big part of a coding interview is working with the candidate.
> the ability to read code
How do you test for that in isolation?
There are exercises that do that: you show the candidate a piece of code, and ask them to find the bug. They have all the worst issues that folks here are complaining about, amplified: stressful, awkward, etc. In my experience these types of exercises are also more random: i.e. depend more on luck. Sometimes great candidates fail them, and poor candidates get them right on a fluke. So they are overall less indicative.
Ultimately, though, it's back to the first point: how do I test if the candidate can do their job, i.e. code?
A candidate may communicate really well, say all the right things in best-practice questions like the one you mentioned, write well in English, but still unable to code their way out of a matchbox.
Again, nobody said “don’t test if a candidate can code”. You should do a minimal test, then move on to more important things, rather than beating the same horse.
”In my experience these types of exercises are also more random”
Well, “your experience” also says that you regularly encounter senior candidates who can’t code at all. I doubt the statistical validity of your experience.
> Joel Sposky wrote that blog post to say that you should do an absolutely minimal check that someone can write code, then move on to more important things.
Interesting, and I won't look up the article again before writing this comment.
I remember it as saying he was having a hard time finding people that can code at all, that a FizzBuzz is weeding out a large percentage of applicants. I don't recall it as a gateway to move on after.
> Coding questions feel objective, and programmers love to pretend they’re objective, but like any other human evaluation process, they’re subjective. It’s not like you automatically get the job if you get the answer right.
> Unless you’re verifying that your team is generating consistent, reproducible outcomes, I can pretty much guarantee that they’re not.
It seems as though there are a fair number of things like this, where people sort-of play act science but don’t bother with the rigor. I guess it’s a type of cargo culting.
> Coding questions are among the most quantifiable and objective questions there are.
Only in the most trivial cases (for statements, not necessarily solution). If you want to assess engineering ability, specific coding questions are a small, tiny sample of the space, and generally the more difficult the problem the less useful it is.
I think that's pretty common. At least, I hope it is. I have been that person, and completely flamed out. Probably came across like I knew exactly nothing. I do well under routine job stress, tight deadlines, etc, and I have a great track record producing real results, but something about the interview process puts me on edge and stresses me in unique ways.
Fortunately I don't find myself in that style of interview anymore. The guy interviewing me already knows I can code, so all we are really trying to see if there is a culture fit. My preferred interview location is over beer.
If it's any comfort: everyone performs worse under the stress of an interview. This is widely recognized by hiring employers, and accounted for in the interview evaluation.
For example, if I test-run a new question, and it takes my engineers 20 minutes to solve it, I will add 30-50% handicap for stress. I.E. if a candidate manages to solve it within 30 minutes, that's roughly equivalent.
Unfortunately, there's no way to account for someone having a complete blackout such as you describe.
Don't ask people to do something they've maybe never done before and will never have to do on the job (write code with a marker on a whiteboard as seconds tick down). If you want a reasonable coding assessment, sit them down at a terminal and give them an hour (and an appropriately difficult problem). But literal whiteboard tests are a recipe for complete blackouts.
> If you want a reasonable coding assessment, sit them down at a terminal and give them an hour (and an appropriately difficult problem).
I am doing just that.
> But literal whiteboard tests are a recipe for complete blackouts.
Not always. I've done a lot of whiteboard interviews too. It's a different way to discuss a problem, and one that people do use at work.
Most design sessions happen with a colleague or two in front of a whiteboard, not sitting at a workstation.
I disagree that blacking out is so common, though I understand it does happen occasionally. Also, we typically do at least 5 interviews in every onsite, one blackout wouldn't destroy the chances of an otherwise strong candidate.
It affects some people much more than others, it's not a uniform random occurrence, so a person with one blackout is more likely than most to have more than one. Some people are very anxious during interviews, and their frontal lobe basically shuts down, others love interviews. So it's very likely that many of the people you've interviewed who you thought "couldn't code" actually just couldn't code while you were watching them.
I personally don't care that much on how well a candidate can code, as measuring that is itself quite hard (in my opinion). What I like to interview for is someone who really likes to code. I look for things that demonstrate recent personal growth, ability to explain & teach something they do know, a tendency to avoid claiming they are good at anything, open mindedness on trying new things, etc. In my experience, someone who really wants to code and shows a minimum viable capacity for doing so is the person I want to hire.
> In my experience, someone who really wants to code and shows a minimum viable capacity for doing so is the person I want to hire.
My comment referred specifically to candidates with at least one year of full-time hands-on development experience under their belt. The typical candidate I interviewed had 3+ years of experience.
If you want to code, then you should have ample opportunity to do so during a year or more of full-time employment.
If you were working in a full-time programming position for over 3 years yet you can't write even a trivial amount of working code during the interview, then clearly you either lack the capacity or the will to develop your programming skills.
Programming nowadays is easy - everyone has a powerful laptop and internet access. It's not like you need privileged access to some $100k time sharing machine using punchcards, and learn machine-language from obscure mail-ordered manuals.
What excuse does anyone claiming to "want to code" have for not coding already?
I'd be quite suspicious of anyone waxing poetic about their genuine heartfelt love for coding who somehow couldn't demonstrate any coding ability.
Would you believe someone who swears they "love long-distance running", but collapses after 50 yards and says they just didn't have the opportunity to put this great love to practice?
> If you were working in a full-time programming position for over 3 years yet you can't write even a trivial amount of working code during the interview, then clearly you either lack the capacity or the will to develop your programming skills.
Genuine question... What kind of question(s) would I ask to get someone to demonstrate they can code? If my questions are effective, I'm fine. But if my questions stink, it'd be easy to overlook that fact that many excellent people wouldn't be able to answer them well.
There are different approaches, but you can come up with any simple programming task. Doubts are fine, so just run them through some of your existing engineers, especially those you think are really good. If people you consider excellent engineers fail them, then it's back to the drawing board.
That's an exercise I do for hiring in general. For example, every time a hiring process change is proposed, I run it through my existing engineers, and make sure I would have hired them by this process.
For example, in a place that was starting to get a lot of great candidates, someone proposed tightening the education requirements of the preliminary screen. I quickly observed that would have disqualified one of my best performers, who didn't have a CS degree at all. The proposition was therefore struck down.
> There are different approaches, but you can come up with any simple programming task.
My challenge internalizing this phrase is that I have a hard time latching onto the idea of simple. It's a highly subjective measurement. For example, if I ask an to implement FizzBuzz & they can't do it, to my satisfaction, are they a not worth hiring? I suppose it depends on what satisfies me. If my criteria for hire/no-hire is whether they wrote concurrent & streaming FizzBuzz, maybe my expectations are unrealistic. If I'm expecting someone to write FizzBuzz in as few characters as possible, am I going to flunk anyone who implements a verbose concurrent & streaming FizzBuzz? Is there a case where it makes sense to hire someone who implements Enterprise FizzBuzz? I may struggle myself with keeping things simple, so there's that to consider :)
Why ask any questions? Sit them down to a simplified version of a real world class your company is working on with corresponding unit tests and tell them to make the tests pass. That was one of my favorite type of interviews and I've used it before.
For simple questions to verify a person has basic programming competency, I'd keep it very simple. Any simple problem which requires you to use if / else if / else or switch case, or something involving simple for / while loop. This is something any programmer should be able to demonstrate.
> I've seen candidates with anything between 1 and 15 years of full time coding work on their resumes fail to answer basic fizz-buzz level coding questions.
Like, how? I used to hire undergraduate student sysadmins / coders, and we had zero candidates fail fizz buzz. None! The worst we saw were people who maybe had an awkward solution to the problem, typically because of unfamiliarity with the modulus operator.
I should stress I didn't ask fizz-buzz specifically, but similarly basic questions.
I should also stress (again) that back then I brought candidates directly to onsites when their recruiters sent an impressive resume with 3+ years of "full time programming experience".
I learned the hard way that most people who unknown recruiters are eager to push through your hiring funnel are unemployed for a reason.
I also learned this not long after I started conducting coding interviews. People with resumes from the best schools and the best companies would fail in multiple interviews in a row. Nowadays I don't even look at the resume, though to be fair I am only tasked with evaluating coding ability, and not things like culture fit or pay scale.
True, but give them something they can do at a computer, in an hour or a day or a weekend. Coding on a whiteboard with someone staring at you counting the seconds is not natural. It's like asking someone to do their job backwards and underwater. You'll get a lot of false negatives.
Not my experience. At least when hiring for programming positions, the typical fatal issue is that the candidate's coding is weak.
In my first role as a hiring manager, I didn't stress coding tests for candidates with long work history on their resume. Since then, I learned better.
I've seen candidates with anything between 1 and 15 years of full time coding work on their resumes fail to answer basic fizz-buzz level coding questions.
At least for the candidates I've seen, the biggest single reason why they get rejected is failure to perform the one main function their job requires: coding.