AI Generates, AI Verifies, Code Decides: Building a Poker Drill Bank You Can Trust
An LLM can write a plausible poker question in two seconds. Plausible is the problem. The pipeline that grows a reviewed drill bank of thousands of hands.
Aug 17, 2026 · 8 min read · PokerInk Team
We needed a thousand poker training questions, each with an answer that could survive verification.
An LLM can write a plausible poker question in two seconds. Plausible is the problem. A question that looks right and marks the wrong answer as correct is worse than no question at all: it actively teaches the mistake, with confidence, forever.
Hand-writing a thousand good questions does not scale for a small team. Raw LLM generation ships wrong answers. The pipeline that squares that circle is the most reused idea in PokerInk.
Generate, verify, gate
The bank grows in stages, and no stage trusts the one before it.
- A strong model generates candidate hands: the spot, the action history, the decision point, the options, the reasoning.
- A second model verifies, reading each candidate cold and challenging the claims. Generator and verifier are never the same pass: the model that wrote the work does not get to approve it.
- Deterministic gates decide. Code re-derives the facts and rejects candidates that claim a draw that is not there, put actions in an illegal order, deal the same card twice, or leak the answer into the prompt. A poker evaluator computes the equity and the made hands; if the question's claims do not match the computation, the question dies.
- A post-hoc audit re-checks everything, including a cross-hand pass that resolves contradictions, so the same spot is never graded two different ways in different questions.
- Only reviewed hands are served. The bank that reaches users contains only reviewed hands, thousands and growing with every batch.
This is the same principle our coaching runs on, applied upstream: the model never asserts a poker fact. The evaluator computes ground truth, and the question is constructed from it, not the other way around.
The war story: all spade flush draws
Verification catches wrong answers. It does not catch boring ones.
A routine audit of the equity drills found that the generator had quietly collapsed into a rut: only four distinct hole-card combos across a whole batch, every one of them a spade flush draw. Each individual question was correct. The batch was useless, because a player drilling it was learning one spot over and over.
The fix was to rotate diverse hero hands, seats, and textures into every generation prompt, and to make distribution itself something the audit measures. Correctness gates were necessary but not sufficient; variety needed its own gate.
Content-addressed questions
Every question's ID is a hash of its content. Editing a question mints a new ID.
That sounds like a detail until the first time you fix a live question. With content-addressed IDs there is no such thing as silently changing a question that thousands of attempts already reference: the fixed version is a new entity, the old one is retired, and historical ratings stay attached to exactly the question that was actually answered. Prod fixes become safe by construction.
Why go to this much trouble
Because the bank feeds an Elo system. Poker IQ treats every drill as a rated match between player and question, which only means something if the marked answer is actually correct. A rating computed against wrong answers is not a rating, it is noise with a leaderboard.
The general lesson we would give any team generating content with LLMs: separate the doing from the checking, give the final veto to deterministic code wherever your domain allows it, and audit for distribution, not just correctness. Generation is cheap. Trust is the expensive part, and it is built one gate at a time.
Train on a bank built to be trusted
PokerInk is free to start on iPhone, Android, and the web. See pricing.
