- Email: [email protected]
Better than human? What's next for Jeopardy! computer
THIS WEEK
What’s next for Watson? A bright future awaits IBM’s supercomputer after taking on humans in the TV quiz show Jeopardy! involve puns and clues-withinclues, and cover a vast range of topics, from science to pop culture. The show demands an encyclopedic knowledge, and Watson is up against the two most successful players ever to appear on Jeopardy!.
A MEDICAL robot; a Google-killer; a financial adviser; a tool for trawling legal documents; an aide for the intelligence services. These are just some of the careers that could be in store for Watson, a supercomputer “Watson has a huge created by IBM which this week challenged human knowledge and memory, loaded with intelligence in a three-part special 200 million pages of text from encyclopaedias” edition of the US quiz Jeopardy!. As New Scientist went to press, Instead of creating new the outcome of the pre-recorded algorithms to allow Watson to final was still a carefully guarded understand Jeopardy!’s complex secret. But however it performs, questions, IBM’s team threw the fact that a machine is able everything they already had at the to compete on Jeopardy! “is a problem. The result is a system remarkable achievement”, says Boris Katz, an artificial intelligence that runs all the best existing algorithms for understanding researcher at the Massachusetts and answering questions. Institute of Technology. Led by David Ferrucci, the When IBM first suggested a team also gave their machine Jeopardy! machine four years a huge memory, which they ago, many AI researchers were loaded with 200 million pages doubtful the company would of text from encyclopedias and succeed. Computers are great at newspapers, alongside customfollowing clearly defined rules, made databases of facts. but Jeopardy! questions can
ben hider/getty
Jim Giles
To answer Jeopardy!’s questions, Watson first guesses the subject. It knows, for example, that the word “this” often precedes the subject. So if a question begins “This 19th-century novelist…”, Watson primes itself to search for authors.
Once it has worked out what it is looking for, Watson searches its databases for the answer. Its processor array is 2000 times more powerful than a desktop computer, allowing it to find thousands of possible answers in less than 3 seconds.
Silicon superiority
There is a long history of human-versus-machine match-ups and the results make for depressing reading if you are human. It started in the 18th century, when a chess-playing robot toured Europe, beating many in its path. It turned out to be a hoax: the “robot” was operated by a concealed human. The reprieve was short-lived, however
1979 BKG 9.8
World backgammon champion Luigi Villa suffers a humiliating 7-1 defeat at the hands of BKG 9.8
1994 CHINOOK 1986 MAVEN
MAVEN beats top Scrabble players. Its creator, Brian Sheppard, improves the software, and by 2002 it can think many moves ahead, something human players find difficult
6 | NewScientist | 19 February 2011
Chinook wins world checkers (draughts) championship. Thirteen years later, its creators say they have simulated every realistic game of checkers and developed a system that means Chinook can never lose a match
“Quote to go in here over four lines range left like this Quote to go in here like this xxxxx”
1997 DEEP BLUE
World chess champion Garry Kasparov loses to IBM’s Deep Blue. Computers are now considered superior to even the strongest human chess players
In this section n Andean settlers’ cancer-fighting secret, page 9 n Periodic table of shapes, page 12 n Augmented reality helps police track suspects, page 26
To work out which of these is the correct answer, Watson uses 100 different tests to rate its confidence in each answer. Some tests check for wordplay. Faced with a question about an “arresting landscape painter”, for example, Watson looks up
2008 POLARIS
Polaris narrowly defeats a line-up of big-name poker players in a series of one-on-one games in Las Vegas. For now, however, humans still have the upper hand: Polaris struggles in games that involve multiple players
at Columbia University in New could “learn” by searching for new York on a medical application. documents to add to its database, If Watson’s memory were filled and so would automatically with databases of medical update its search results as new literature, and combined with facts emerge. Prem Natarajan voice-recognition technology, it of Raytheon BBN Technologies could listen in on patient-doctor in Boston, one of the companies conversations, then search its involved in the DARPA project, databases and suggest possible compares this application diagnoses, each with its own to an oracle. confidence level. “Such systems could augment “Even Watson can make silly mistakes. We’re not healthcare,” says Amar Das of done yet. We’ve not even Stanford University in California, scratched the surface” who is helping a team at the nearby PARC research labs develop an AI system able to The Watson team have also answer doctors’ questions about spoken to lawyers about a possible HIV treatments. These systems use for the machine. Lawyers have “great potential”, he says, often have to sift through but medical language could pose mountains of documents when problems. Doctors in different preparing a case – something fields can use the same term to Watson could make easier. It mean slightly different things, could even improve the service for example. Watson would of staff at banks or insurance have to be trained to put terms companies, by helping them track in context and extract the down the answer to a question correct meaning. more rapidly. Intelligence services might also Thankfully, perhaps, Watson have a use for Watson. Members isn’t infallible: it can find simple of the IBM team are part of a questions baffling. This is in part machine-reading research because the software relies –Not just a game player– programme funded by the US heavily on finding text that looks Defense Advanced Research like the right answer to a question. meanings of “arresting” and Projects Agency (DARPA), based As a result, it misses out on checks for any connection with in Arlington, Virginia. information that is deemed by the names of landscape painters. DARPA is interested in humans to be too obvious to write Linked answers – in this case, machine reading in part because down. Not only that, but Watson “Constable” – get a higher intelligence analysts are faced sometimes thinks that fictional confidence score. Watson settles with far more reports than they characters are real, says Gondek: on the highest-rated answer and can synthesise, says Eric Nyberg, it once named the first woman in tries to buzz in. of Carnegie Mellon University in space as “Wonder Woman”. None of the tests represents Pittsburgh, Pennsylvania, who is To get a grip on such commona breakthrough in language working with the Watson team. sense issues, Watson will need processing, but by combining Instead of having to study every to improve its understanding of them all Watson has leapt report, analysts could use Watson the questions it is asked, rather ahead of rival question-andto extract information on, say, than just searching for answers answer systems. how individuals discussed in the that look plausible – a much Watson is not designed just documents are linked. bigger challenge. to play game shows, however. Watson’s potential also extends “Watson is an awesome It could help with any task that to everyday tasks. It is not currently machine,” says Katz. “But involves quickly processing connected to the internet, but if it sometimes it makes amazingly information in a large number were, it could improve its search silly mistakes. That should tell of text documents. results. “Watson could replace researchers that we’re not done. Take the medical profession. Google for some kinds of We’ve not even scratched Although IBM won’t reveal details searches,” says Nyberg. the surface.” n See how Watson fared in the of its plans to commercialise Like a human brain, a Watson final at our technology blog Watson’s software, it has said search engine would continually that it is working with researchers add to its knowledge. The software newscientist.com/onepercent 19 February 2011 | NewScientist | 7
What’s next for Watson? A bright future awaits IBM’s supercomputer after taking on humans in the TV quiz show Jeopardy! involve puns and clues-withinclues, and cover a vast range of topics, from science to pop culture. The show demands an encyclopedic knowledge, and Watson is up against the two most successful players ever to appear on Jeopardy!.
A MEDICAL robot; a Google-killer; a financial adviser; a tool for trawling legal documents; an aide for the intelligence services. These are just some of the careers that could be in store for Watson, a supercomputer “Watson has a huge created by IBM which this week challenged human knowledge and memory, loaded with intelligence in a three-part special 200 million pages of text from encyclopaedias” edition of the US quiz Jeopardy!. As New Scientist went to press, Instead of creating new the outcome of the pre-recorded algorithms to allow Watson to final was still a carefully guarded understand Jeopardy!’s complex secret. But however it performs, questions, IBM’s team threw the fact that a machine is able everything they already had at the to compete on Jeopardy! “is a problem. The result is a system remarkable achievement”, says Boris Katz, an artificial intelligence that runs all the best existing algorithms for understanding researcher at the Massachusetts and answering questions. Institute of Technology. Led by David Ferrucci, the When IBM first suggested a team also gave their machine Jeopardy! machine four years a huge memory, which they ago, many AI researchers were loaded with 200 million pages doubtful the company would of text from encyclopedias and succeed. Computers are great at newspapers, alongside customfollowing clearly defined rules, made databases of facts. but Jeopardy! questions can
ben hider/getty
Jim Giles
To answer Jeopardy!’s questions, Watson first guesses the subject. It knows, for example, that the word “this” often precedes the subject. So if a question begins “This 19th-century novelist…”, Watson primes itself to search for authors.
Once it has worked out what it is looking for, Watson searches its databases for the answer. Its processor array is 2000 times more powerful than a desktop computer, allowing it to find thousands of possible answers in less than 3 seconds.
Silicon superiority
There is a long history of human-versus-machine match-ups and the results make for depressing reading if you are human. It started in the 18th century, when a chess-playing robot toured Europe, beating many in its path. It turned out to be a hoax: the “robot” was operated by a concealed human. The reprieve was short-lived, however
1979 BKG 9.8
World backgammon champion Luigi Villa suffers a humiliating 7-1 defeat at the hands of BKG 9.8
1994 CHINOOK 1986 MAVEN
MAVEN beats top Scrabble players. Its creator, Brian Sheppard, improves the software, and by 2002 it can think many moves ahead, something human players find difficult
6 | NewScientist | 19 February 2011
Chinook wins world checkers (draughts) championship. Thirteen years later, its creators say they have simulated every realistic game of checkers and developed a system that means Chinook can never lose a match
“Quote to go in here over four lines range left like this Quote to go in here like this xxxxx”
1997 DEEP BLUE
World chess champion Garry Kasparov loses to IBM’s Deep Blue. Computers are now considered superior to even the strongest human chess players
In this section n Andean settlers’ cancer-fighting secret, page 9 n Periodic table of shapes, page 12 n Augmented reality helps police track suspects, page 26
To work out which of these is the correct answer, Watson uses 100 different tests to rate its confidence in each answer. Some tests check for wordplay. Faced with a question about an “arresting landscape painter”, for example, Watson looks up
2008 POLARIS
Polaris narrowly defeats a line-up of big-name poker players in a series of one-on-one games in Las Vegas. For now, however, humans still have the upper hand: Polaris struggles in games that involve multiple players
at Columbia University in New could “learn” by searching for new York on a medical application. documents to add to its database, If Watson’s memory were filled and so would automatically with databases of medical update its search results as new literature, and combined with facts emerge. Prem Natarajan voice-recognition technology, it of Raytheon BBN Technologies could listen in on patient-doctor in Boston, one of the companies conversations, then search its involved in the DARPA project, databases and suggest possible compares this application diagnoses, each with its own to an oracle. confidence level. “Such systems could augment “Even Watson can make silly mistakes. We’re not healthcare,” says Amar Das of done yet. We’ve not even Stanford University in California, scratched the surface” who is helping a team at the nearby PARC research labs develop an AI system able to The Watson team have also answer doctors’ questions about spoken to lawyers about a possible HIV treatments. These systems use for the machine. Lawyers have “great potential”, he says, often have to sift through but medical language could pose mountains of documents when problems. Doctors in different preparing a case – something fields can use the same term to Watson could make easier. It mean slightly different things, could even improve the service for example. Watson would of staff at banks or insurance have to be trained to put terms companies, by helping them track in context and extract the down the answer to a question correct meaning. more rapidly. Intelligence services might also Thankfully, perhaps, Watson have a use for Watson. Members isn’t infallible: it can find simple of the IBM team are part of a questions baffling. This is in part machine-reading research because the software relies –Not just a game player– programme funded by the US heavily on finding text that looks Defense Advanced Research like the right answer to a question. meanings of “arresting” and Projects Agency (DARPA), based As a result, it misses out on checks for any connection with in Arlington, Virginia. information that is deemed by the names of landscape painters. DARPA is interested in humans to be too obvious to write Linked answers – in this case, machine reading in part because down. Not only that, but Watson “Constable” – get a higher intelligence analysts are faced sometimes thinks that fictional confidence score. Watson settles with far more reports than they characters are real, says Gondek: on the highest-rated answer and can synthesise, says Eric Nyberg, it once named the first woman in tries to buzz in. of Carnegie Mellon University in space as “Wonder Woman”. None of the tests represents Pittsburgh, Pennsylvania, who is To get a grip on such commona breakthrough in language working with the Watson team. sense issues, Watson will need processing, but by combining Instead of having to study every to improve its understanding of them all Watson has leapt report, analysts could use Watson the questions it is asked, rather ahead of rival question-andto extract information on, say, than just searching for answers answer systems. how individuals discussed in the that look plausible – a much Watson is not designed just documents are linked. bigger challenge. to play game shows, however. Watson’s potential also extends “Watson is an awesome It could help with any task that to everyday tasks. It is not currently machine,” says Katz. “But involves quickly processing connected to the internet, but if it sometimes it makes amazingly information in a large number were, it could improve its search silly mistakes. That should tell of text documents. results. “Watson could replace researchers that we’re not done. Take the medical profession. Google for some kinds of We’ve not even scratched Although IBM won’t reveal details searches,” says Nyberg. the surface.” n See how Watson fared in the of its plans to commercialise Like a human brain, a Watson final at our technology blog Watson’s software, it has said search engine would continually that it is working with researchers add to its knowledge. The software newscientist.com/onepercent 19 February 2011 | NewScientist | 7