On January 20 2011 15:05 SpoR wrote: Also kinda curious how the thing rings the buzzer. There has to be some inherent delay a human can actually squeeze the button that a computer can beat 100% of the time if they both think the answer and the exact same second. Also, can the computer buzz in before the question is over?
Umm...I'm not 100% sure about this, but I think in Jeopardy you're not allowed to buzz in before the answer is read completely. If you try to buzz in beforehand then your buzzer is locked for some short amount of time. This makes it important to buzz at the exact moment the answer is no longer read.
I wonder if they programmed Watson to include the time you have to respond. Your time to respond is a lot of time for a computer. It'd be cool if they could get it to have an ETA for the correct response with a certain confidence level. That way it could consider that if it buzzed in without knowing the correct response completely then it might arrive at the question in the time given.
On January 20 2011 15:05 SpoR wrote: Also kinda curious how the thing rings the buzzer. There has to be some inherent delay a human can actually squeeze the button that a computer can beat 100% of the time if they both think the answer and the exact same second. Also, can the computer buzz in before the question is over?
Umm...I'm not 100% sure about this, but I think in Jeopardy you're not allowed to buzz in before the answer is read completely. If you try to buzz in beforehand then your buzzer is locked for some short amount of time. This makes it important to buzz at the exact moment the answer is no longer read.
Yup, if you watch the video (or any episode of Jeopardy) you'll see other people trying to buzz in but not get called.
On January 20 2011 15:27 mikeymoo wrote: Very smart move by IBM. If it was head to head vs Jennings, Watson would be less favoured than in a three-way format because when it can't answer a question, there wouldn't be two contestants to split the responses. Also, I'm wondering what the algorithm is for Daily Doubles and Final Jeopardy. SHould be interesting.
I would load it up with all the game data from the other two contestants for final jeopardy and form aggregate data then run risk assessment between their score and itself to make the bet based of that data and the success rate of Watson. For double jeopardy i would guess it would just get score vs score and the success rate for topics etc. AI is very fun to think about but i imagine a bitch to implement.
On January 20 2011 15:42 BatTheMan wrote: Can it detect sarcasm or humor?
It has to in a sense thing's like puns which are often done in jeopardy and it has to understand that to narrow down the searches.
It looks like the computer beats ken to the punch alot watching that video.
The buzzer rules in Jeopardy ensure that man vs computer will pretty much be a 1-player game. You can't buzz in until the question has been fully read, and the computer gets the entire text of the question at once rather than having to process the audio then think. Given these rules, if the computer doesn't win its matches by a landslide then IBM screwed up bigtime.
The computer has a guaranteed first shot at answering the question so it just needs to be right a reasonable amount of the time. A more interesting game would be to change the rules of jeopardy to give both human and computer the full text at the start and allow buzzing at any time, but this would both make the game awkward for live TV and change the dynamics to be about buzzing in immediately and hoping you can think of an answer in the 5 seconds after buzzing you are given to state your response.
On January 20 2011 16:24 professorjoak wrote: The buzzer rules in Jeopardy ensure that man vs computer will pretty much be a 1-player game. You can't buzz in until the question has been fully read, and the computer gets the entire text of the question at once rather than having to process the audio then think. Given these rules, if the computer doesn't win its matches by a landslide then IBM screwed up bigtime.
The computer has a guaranteed first shot at answering the question so it just needs to be right a reasonable amount of the time. A more interesting game would be to change the rules of jeopardy to give both human and computer the full text at the start and allow buzzing at any time, but this would both make the game awkward for live TV and change the dynamics to be about buzzing in immediately and hoping you can think of an answer in the 5 seconds after buzzing you are given to state your response.
On January 20 2011 16:24 professorjoak wrote: The buzzer rules in Jeopardy ensure that man vs computer will pretty much be a 1-player game. You can't buzz in until the question has been fully read, and the computer gets the entire text of the question at once rather than having to process the audio then think. Given these rules, if the computer doesn't win its matches by a landslide then IBM screwed up bigtime.
The computer has a guaranteed first shot at answering the question so it just needs to be right a reasonable amount of the time. A more interesting game would be to change the rules of jeopardy to give both human and computer the full text at the start and allow buzzing at any time, but this would both make the game awkward for live TV and change the dynamics to be about buzzing in immediately and hoping you can think of an answer in the 5 seconds after buzzing you are given to state your response.
From my understanding Watson "listens" they wouldn't talk about speech analysis in the videos if Watson could just read the question before hand, they also talk about it learning answers, and if it couldn't understand the answer being said by the host or players it couldn't learn from it. Unless it had a person typing in everything, which is not what's happening.
On January 20 2011 16:24 professorjoak wrote: The buzzer rules in Jeopardy ensure that man vs computer will pretty much be a 1-player game. You can't buzz in until the question has been fully read, and the computer gets the entire text of the question at once rather than having to process the audio then think. Given these rules, if the computer doesn't win its matches by a landslide then IBM screwed up bigtime.
The computer has a guaranteed first shot at answering the question so it just needs to be right a reasonable amount of the time. A more interesting game would be to change the rules of jeopardy to give both human and computer the full text at the start and allow buzzing at any time, but this would both make the game awkward for live TV and change the dynamics to be about buzzing in immediately and hoping you can think of an answer in the 5 seconds after buzzing you are given to state your response.
On jeopardy the questions are given in both speech and text instantly. Watson has exactly the same amount of time to process the information as the human contestants.
you say
The computer has a guaranteed first shot at answering the question so it just needs to be right a reasonable amount of the time.
well obviously. Thats the entire point of the project. I don't even get what you are trying to say...The computer will buzz first if it figures out the question in "a reasonable amount of time"? Thats not really saying anything.
yeah, the computer can hit the actual buzzer faster than a human, but its the processing time that really makes a difference, not the tenth of a second racing the button.
On January 20 2011 16:24 professorjoak wrote: The buzzer rules in Jeopardy ensure that man vs computer will pretty much be a 1-player game. You can't buzz in until the question has been fully read, and the computer gets the entire text of the question at once rather than having to process the audio then think. Given these rules, if the computer doesn't win its matches by a landslide then IBM screwed up bigtime.
The computer has a guaranteed first shot at answering the question so it just needs to be right a reasonable amount of the time. A more interesting game would be to change the rules of jeopardy to give both human and computer the full text at the start and allow buzzing at any time, but this would both make the game awkward for live TV and change the dynamics to be about buzzing in immediately and hoping you can think of an answer in the 5 seconds after buzzing you are given to state your response.
On jeopardy the questions are given in both speech and text instantly. Watson has exactly the same amount of time to process the information as the human contestants.
The computer has a guaranteed first shot at answering the question so it just needs to be right a reasonable amount of the time.
well obviously. Thats the entire point of the project. I don't even get what you are trying to say...The computer will buzz first if it figures out the question in "a reasonable amount of time"? Thats not really saying anything.
yeah, the computer can hit the actual buzzer faster than a human, but its the processing time that really makes a difference, not the tenth of a second racing the button.
However its highly likely that both the players and the computer will know the answer to a question before the question is finished being read. Watch a gameshow where they allow people to buzz in before the question is finished and the host rarely completes a full question before a buzz. Therefore the computer's ability to beat a person in a buzzer race is the factor which is going to give it an advantage.
I've never watched jepordy so I assumed that people could press their buzzer whenever and that it would select whoever pressed it first after the question had finished being read. That would be a more fair system.
Of course fairness isnt that important here, this is a tech demo, not someone trying to make money by cheating a gameshow.
80% of Jeopardy questions just come straight out of high school lit class. It wouldn't be particularly hard to load most of the lit/historical data into the computer to answer the majority of questions.
The more difficult part sounds like programming the computer to discern what the questions are looking for specifically. As the article mentioned, Jeopardy questions are often framed really obscurely, or rely on extra information (like the question's category) to be answered.
On January 21 2011 00:21 Offhand wrote: 80% of Jeopardy questions just come straight out of high school lit class. It wouldn't be particularly hard to load most of the lit/historical data into the computer to answer the majority of questions.
The more difficult part sounds like programming the computer to discern what the questions are looking for specifically. As the article mentioned, Jeopardy questions are often framed really obscurely, or rely on extra information (like the question's category) to be answered.
Yeah.. as long as they can program it for the computer to process the question and know what is being asked I'd be more surprised if the computer didn't win.. Even a crappy desktop computer would destroy the world champion in a spelling bee.
EPT
