How We Stopped Our AI Poker Coach From Hallucinating Hands

An AI poker coach has one job it absolutely cannot get wrong: knowing what hand you actually have.

Ours got it wrong anyway, early on, in ways that would make any poker player close the app. It would look at a board and misname the made hand. It would call four to a straight a flush draw. It would then build three paragraphs of confident coaching on top of the misread.

The fix was not a better prompt.

Why "prompt more carefully" fails

The obvious first move is to write stricter instructions: double-check the board, enumerate the draws, be careful about suits. It helps on the hands you test and fails on the hands you did not. The failure is not a motivation problem, it is a capability problem: evaluating a seven-card poker situation is exact combinatorics, and a language model is a probabilistic text machine. At scale, some fraction of evaluations will be wrong, and in a coaching product every single one is a credibility bomb.

A player who catches the coach misreading one hand stops trusting every sentence the coach has ever produced. Reasonably.

The fix: the model never gets to assert a poker fact

PokerInk already had the piece that does this correctly: the same deterministic engine that settles pots in the journal. So we moved the job. For every postflop street in a coaching request, code computes a ground-truth block before the model is ever called:

• the exact made hand on that street,
• the exact draws, including the distinctions that matter: nut flush draw versus plain flush draw, open-ender versus gutshot,
• whether the hero's cards even contribute to the made hand.

That block is injected into the payload, and the system prompt forces the model to defer to the computed labels verbatim. The model's job shrinks to the thing it is actually good at: reasoning about ranges, lines, and sizing in plain language, on top of facts it is not allowed to invent.

The principle we wrote down in the runbook: make these errors structurally impossible, don't prompt more carefully.

One ground truth, shared everywhere

A subtle second bug class is waiting here: the hand-level coach and the session-level coach drifting apart, one calling a hand a missed draw while the other calls it a pair. The cause would be two code paths computing "the facts" separately.

So there is exactly one ground-truth constant, and both the single-hand prompt and the whole-session prompt consume it. If the evaluator changes, both coaches change together. Two analyses of the same hand can disagree about strategy, which is poker, but never about what the hand was, which is arithmetic.

Where else the same rule applies

Once you adopt "the AI never asserts a poker fact," it spreads. Our training drills are AI-generated, but every question is built from evaluator-computed truth, and deterministic gates reject any generated hand whose claims do not check out. That pipeline got its own write-up.

The pattern generalizes beyond poker. If your domain has a computable source of truth, compute it, inject it, and demote the model from witness to commentator. Models are excellent commentators.

Get coaching grounded in your real hands

PokerInk is free to start on iPhone, Android, and the web. See pricing.