Brains vs AI - Humans Book Decisive Victory Over Claudico

The writing was on the wall days before the match was completed and the final moments only confirmed what could already be established with 99.9% certainty. No Limit Hold'em bot Claudico suffered a severe defeat from the four flesh-and-blood heads-up specialists led by Doug Polk.

The match, which was live streamed on Twitch for its entire duration, was meant to be an ultimate test of artificial intelligence advancements in the area on heads-up No Limit Hold'em poker - the game mathematically so complex and hard to solve that even Claudico's creators were not that optimistic going into the match.

However, after eleven years of development and fully aware of truly huge breakthroughs in the field of computer science, it would be unwise to discard the possibility of Claudico actually being good at poker. As it turned out, it isn't that good.

After every player played 20k hands against the AI, the final results were known and together, humans took the machine for $723,713. Bjorn Li was the biggest individual winner, with the advantage of $529k, while Jason Les was the only one who finished in the read, down $80,482.

Despite of the result, according to Carnegie Mellone University article, the difference of over 70 buy-ins was not big enough to be statistically significant and thus, although Claudico lost, it played well enough to get a "statistical draw".

After the match was over, Tuomas Sandholm, CMU professor and leading man in the development of Claudico, explained that beating humans in poker isn't their end-goal, but rather a stepping stone towards creating an AI that can help people with making decisions in situations where the information is incomplete.

Talking purely from poker perspective, Doug Polk noted that the AI had some very unusual patterns and behaviors that he just failed to understand, like severely under-betting or over-betting a pot. On the other hand, unlike human opponents, Claudico never lost its cool regardless of the results.

Despite of the actual loss (and "statistical tie"), Sandholm seems quite confident that the AI will be able to match and outplay top professionals very soon, especially with the help of 80,000 actual hands played against top pros.

Results aside, Claudico's very strange tendencies do show that it is capable of "thinking" on its own and doesn't follow any set-in-stone parameters. While this process clearly needs some tweaking, it does represent an absolutely necessary quality if it is ever to defeat human players. These tweaks, combined with ability to adjust faster, could potentially make this AI into a serious threat at the table, but we will have to wait and see what the development team will come up with in the future.

Photo: cmu.edu