It shouldn't be on top of that! It wasn't for our process. Of course: we did in-person interviews. And in-person interviews are disruptive no matter what you do. But:
* Our in-person interviews were shorter than typical in-person interviews because they weren't tasked with fully teching candidates out.
* Because they didn't try to tech candidates out, they weren't as stressful, and so were less draining and unpleasant.
* Because our in-person interviews were entirely scripted (the interviewers had very little latitude with what they could ask and how they could ask it --- which they hated, by the way), they were easy for everyone to deliver.
* Before candidates arrived to their interview, they already knew (because we told them!) that they were likely to receive an offer from us based on their performance on the work-sample tests.
I understand why people resist the idea of coding challenges, given:
* They're not going to get feedback for weeks, months, or maybe ever
* The challenge is going to be graded pass-fail, or whimsically, without rigor or consistency
* They're going to have to do the exact same grueling bullshit draining dehumanizing programmer interview anyways
* They're not going to see the challenges coming, or, for that matter, know exactly what happens next after the challenges are done
That process is super common, even at companies that ostensibly do challenges. It sucks. I agree, no company can really get away with this in the long term.
But those are also chickenshit problems. Switching from unstructured interviews to work-samples is hard: you have to change your mindset on how to qualify candidates, and there's a leap of faith involved. But getting candidates feedback, telling them what to expect with your process, keeping a schedule, and having processes in place to be consistent aren't hard problems. They are table-stakes common sense business execution, and if your team can't handle that right now, your team is mismanaged.
Can you share an example of challenge given to candidates? In the past, I've used "script a file sharing tool that handles encryption", which worked well and took candidates ~4/8 hours to complete, but isn't something we used directly in our projects.
Sure: we wrote a very simple retail trading system as a one-file Ruby app. It used a custom binary protocol. We had candidates reverse the binary protocol, implement enough of it to test the system, and find vulnerabilities in it.
We had a grading rubric for the challenge that was based on the kinds of things people found in the application. We evaluated performance on the challenge on a 1-5 score; depending on how you did on other challenges, a 3 would keep you in the game, and a 4-5 probably assured an in-person interview.
we were going to get together in a room anyways.