This. A thousand times this. Watson absolutely CRUSHED the human players on pretty much every question that was basic facts. I know Watson can probably generate answers faster than humans on simple search stuff, but it seemed so bad at some points that I wondered: is Watson not wired in with some sort of delay that mimics the delay that humans have between deciding to buzz in and actually buzzing in? A lack of such a system would seem to skew the results somewhat.
I was at a watching party with a couple of IBMers who worked on Watson, and one thing they said is that it's not a question of speed, but of timing. Players time their pressing to an estimate of when Alex will finish the question, and Brad Rutter in particular has been clocked at under 2ms with shocking regularity. The advantages Watson has are consistency and the emotional perturbations in its opponents. You could see them getting frustrated, and that likely only served to harm their ability to hit that window between the end of the question and Watson's button press.
You're right: consistently being 6x faster on the buzzer than the common case for your opponent is going to let you destroy them. Their only hope is that you can't come up with a response before Alex finishes reading the question.
I arrived at the 6x approximation by googling around for avg. ethernet latencies. I'm consistently seeing numbers of .3 - .35 ms for an ethernet ping/pong. I think it's fair to assume that with the money IBM has invested in this, Watson is on at least ethernet quality connections.
When a question appears on the screen, Trebek reads it. A human decides when he's 'done' reading it and pushes a button, which makes a light appear. Contestants aren't allowed to buzz in until they see the light, and are penalized if they buzz in too quickly. Watson is also notified of this metaphorical gunshot to start the race, and won't try to start buzzing in before that.
My point is that a human nervous system is MUCH slower than Watson's equivalent. It is probable (don't know, but it seems likely) that it is more variable, as well. As such, under the current rules, Watson has an overwhelming advantage. If all the contestants know the answer before the question is "read", then Watson will consistently beat the humans to the buzzer. The only hope for the humans is that Watson hasn't decided an answer by the time the question is finished being "read".
Without some way to account for the fact that the human nervous system CANNOT beat Watson to the buzzer on any sort of consistent basis, the game is far less compelling than it should be.
He seems to be arguing that you shouldn't handicap the computer by removing one of its advantages, but they've already handicapped the humans by removing lots of their advantages: at least in the parts I've seen, there have been no audio or visual clues, and it's my understanding that Watson gets the text of the question over the wire, not even having to OCR the same text that the humans get to read while waiting for Alex to finish reading the question.
I'm an IBMer and I think Watson has been extremely impressive, but as a Jeopardy fan it gets tiresome to see Watson win the race to the buzzer this often.
Finally, I'm hoping that tonight features more of the wordplay-heavy clues that I hear were present on the first night (why oh why was that on Valentine's Day???), because that is the element that excited me most when I heard about the challenge.
