Why Playing Poker with Chat GPT Might Not Be a Great Idea

07:25
15 May

According to a new study conducted by researchers from the University of Southern California, playing poker with artificial intelligence models can lead to unexpected outcomes, often terrible results. The researchers found that large language models, such as Chat GPT, often have difficulty weighing potential gains and losses, which can lead to irrational decision-making and ultimately nonsensical choices

So, if you had any ambition to use an AI tool such as Chat GPT to create poker strategies, think again because it will likely turn out to be a complete disaster. Of course, there will be many who will ignore this advice so maybe some of us will be able to cash in on this.

---

Can AI Calculate Properly?

Over the past few years, there has been a surge in the development of large language model artificial intelligence systems that can perform a range of tasks, from writing poetry and conducting human-like conversations to passing medical school exams.

Regardless of how powerful the likes of Chat GPT is, it cannot think for itself and unexpected mistakes will occur alongside a tendency to just make stuff up.

The fact that humans are using a tool which responds in their language can trick people into believing that the program can actually think for itself. For this reason, researchers are now beavering away to better understand what kind of cognitive ability they have and what, if any biases.

But it’s not all working as people expect at this early stage. We now know that large language models struggle to understand the concept of negativity. Not only that but even apparently simple calculations can prove to be a problem. Frustratingly, when the model is asked to provide a detailed explanation, it often can’t. It’s early days, as they say.

Research has led scientists down a path to better understand how language affects cognition. This area is already well-researched but with the growing importance of these tools, a more sophisticated level may be required to take the next steps in development.

To test whether language models are rational, the USC researchers conducted a series of experiments that showed that, in their original form, models like BERT behave randomly when presented with bet-like choices. This is the case even when they are given a trick question such as: If you toss a coin and it comes up heads, you win a diamond; if it comes up tails, you lose a car. Which would you choose?

Obviously heads, but weirdly, Chat GPT reckons tails as much as half of the time.

Scientists discovered that large language models generally don’t need a large set of examples to function accurately, but later realised that the situation is extremely complex.

For example, when the researchers used cards or dice instead of coins in the wording of the prompt, they saw that accuracy dropped by as much as 25%, but this was still better than random answering.

---

Further Tests Confirm the Issue

More recent case studies conducted using Chat GPT confirm that decision-making remains a complex issue and a perfect solution is still some time away. The study shows that such models have trouble weighing potential gains and losses, and this has significant implications in decision-making processes.

Large language models like Chat GPT and BERT have impressive abilities in generating fluent language, but they do not actually think. There is a tendency to make silly mistakes, including presenting incorrect information as fact without evidence.

Nonetheless, the idea that the model can be taught general principles of rational decision-making remains unresolved. These findings have implications in decision-making processes and call for continued research in the field of cognitive science and machine learning.

For years, poker players have worried about the effect that GTO solvers will have on the online game. There have already been confirmed real-time assistance cheating cases in the high stakes world and technology still marches forward with ever more powerful products hitting the market. If AI eventually gets it act together, and surely it will, poker may have a real problem on its hands that will require some new solutions to police.