After having been through this crap and now being on the other side of the desk, I've come to the conclusion that this is simply standardized testing for adults, with all of the same myriad problems. It doesn't identify what you're actually looking for, gives you more false signals, and alienates the talent you actually want.
The way we do these at my current job is extremely productive for us. We look for two things, apart from basic competency: problem solving, and asking for help.
It's structured as a two 90 minute pair programming sessions. Interviewee shares their screen, and we work through the problems together. Obviously it's pretty hands-off, but we guide and nudge where appropriate. Here and there, when they use something that relates to a deeper topic, I'll ask questions to gauge how deep their knowledge goes. Like asking if they know how C#'s foreach works under the hood. Not as a selection criteria, but simply to get a sense of how much they know.
Use of a search engine is openly encouraged. A lot of the time, we don't even care if the program actually runs. If they struggle with syntax or the correct function overload, we'll help them out after giving them a little time to find the solution.
I also throw in a problem designed to get them stuck, and ask questions I expect they can't answer. A good programmer asks for help and admits when they don't know. A bad one bullshits their way through.
We want to hire programmers who can do a real job in the real world. Implementing red/black trees on a whiteboard blindfolded isn't a job skill, it's a party trick. That's not something a programmer will ever need to do.
In the real world, real people use google and stack overflow. They don't have encyclopedic knowledge of the entire language's syntax. They ask their coworkers for help or opinions.
Our interview process is designed to show us how a person will function in a scenario as close to the job as possible. Because that's what we're hiring them for. We look for their ability to work through a problem with the resources that everyone always has. We look for how they work with others and how much they lean on coworkers.
This has worked out extremely well for us. We've hired some very talented individuals, and have totally avoided the archetypal shitty dev. The people we hire immediately mesh with the team, and learn and grow the way all programmers do.
Granted, we are a small company and we have the time to have our own programmers giving interviews. We also have a much higher need to be so selective. But every single person who has made it to the technical interview has remarked unprompted that it's the best interview they've ever had. And I mean 100%.
It's because we treat candidates the way we'd treat our own employees. They get to know what the job is like, and we get to know how they'll do the job.
The way we do these at my current job is extremely productive for us. We look for two things, apart from basic competency: problem solving, and asking for help.
It's structured as a two 90 minute pair programming sessions. Interviewee shares their screen, and we work through the problems together. Obviously it's pretty hands-off, but we guide and nudge where appropriate. Here and there, when they use something that relates to a deeper topic, I'll ask questions to gauge how deep their knowledge goes. Like asking if they know how C#'s foreach works under the hood. Not as a selection criteria, but simply to get a sense of how much they know.
Use of a search engine is openly encouraged. A lot of the time, we don't even care if the program actually runs. If they struggle with syntax or the correct function overload, we'll help them out after giving them a little time to find the solution.
I also throw in a problem designed to get them stuck, and ask questions I expect they can't answer. A good programmer asks for help and admits when they don't know. A bad one bullshits their way through.
We want to hire programmers who can do a real job in the real world. Implementing red/black trees on a whiteboard blindfolded isn't a job skill, it's a party trick. That's not something a programmer will ever need to do.
In the real world, real people use google and stack overflow. They don't have encyclopedic knowledge of the entire language's syntax. They ask their coworkers for help or opinions.
Our interview process is designed to show us how a person will function in a scenario as close to the job as possible. Because that's what we're hiring them for. We look for their ability to work through a problem with the resources that everyone always has. We look for how they work with others and how much they lean on coworkers.
This has worked out extremely well for us. We've hired some very talented individuals, and have totally avoided the archetypal shitty dev. The people we hire immediately mesh with the team, and learn and grow the way all programmers do.
Granted, we are a small company and we have the time to have our own programmers giving interviews. We also have a much higher need to be so selective. But every single person who has made it to the technical interview has remarked unprompted that it's the best interview they've ever had. And I mean 100%.
It's because we treat candidates the way we'd treat our own employees. They get to know what the job is like, and we get to know how they'll do the job.