Another game just fell to the machines.
Yesterday, after 20 days of play at a casino in Pittsburgh, an AI built by two Carnegie Mellon researchers officially defeated four top players at no-limit Texas Hold 'Em—a particularly complex form of poker that relies heavily on longterm betting strategies and game theory. Over the past twenty years, machines have topped the best humans at checkers, chess, Scrabble, Jeopardy!, and even the ancient game of Go. But no AI had ever beaten the best at such an extreme game of "imperfect information," a game where certain elements, such as the cards on the table, are hidden. Among humans, no-limit Hold 'Em requires a certain degree of intuition, not to mention luck.
Carnegie Mellon professor Tuomas Sandholm and grad student Noam Brown designed the AI, which they call Libratus, Latin for "balance." Almost two years ago, the pair challenged some top human players with a similar AI and lost. But this time, they won handily: Across 20 days of play, Libratus topped its four human competitors by more than $1.7 million, and all four humans finished with a negative number of chips.
Yes, poker is just a game. But the game theory exhibited by Libratus could help with everything from financial trading to political negotiations to auctions, says University of Michigan professor Michael Wellman, who specializes in game theory and closely follows the world of AI poker. In no-limit Hold 'Em, players aren't necessarily trying to win each small hand. They're trying to win the most money, and that means developing betting strategies that play out over dozens of hands. A machine that masters no-limit Texas Hold 'Em mimics the kind of human intuition these strategies require.
According to the human players that lost out to the machine, Libratus is aptly named. It does a little bit of everything well: knowing when to bluff and when to bet low with very good cards, as well as when to change its bets just to thrown off the competition. "It splits its bets into three, four, five different sizes," says Daniel McAulay, 26, one of the players bested by the machine. "No human has the ability to do that."
So far, Sandholm has been coy about the particulars of how Libratus operates, but he has promised to share details in the days to come. The human players---who along with McAulay include Dong Kim, Jason Les, and Jimmy Chou---believe that the machine's play changed from day to day. If they ever felt they'd found a hole in its strategy, the hole would close. "It seemed to learn what we were doing and exploit it," McAuley said. Sandholm and Brown may have worked to change the machine's behavior from day to day, as they did when their earlier AI, Claudiro, went up against human players nearly two years ago. But the machine may also have learned from the match as it played out.
If it seems unfair that the Carnegie Mellon researchers may have altered the machine between rounds, consider that the human players also used every tactic at their disposal. Though the game was heads-up Hold 'Em---meaning each player was playing his own game against the machine---they would share strategies in the evenings. "We spend a couple of hours conferring every night," McAuley said. "We're playing against each other. But we're also trying to win for the humans."
