Learned pattern values

Can lookup tables of small board patterns, learned from self-play, replace the hand-written leaf and save a ply of search?

The decision at the table

Every search ends by asking a leaf evaluator how good a position is. The tunable search prices points, cards, production, and build targets with a dozen hand-written terms. Those terms cannot know that a city on an eight next to a six is worth more than its pips suggest, that a settlement hemmed in by two opponents' roads will never expand, or that a hand of five ore and no grain is a trap. A search can discover some of this by looking further ahead, at four to six times the cost per ply.

The mechanism to test

Read the position through small windows and look each one up in a table of learned numbers, as the lookup-table evaluators of 2048 and the Drop7 n-tuple tables do. Three families of windows share tables across the whole board, so one number serves every hex and every intersection:

• Hex windows. A hex's resource and pips, whether the robber sits there, and its six corners, each empty, own settlement, own city, other settlement, or other city, canonicalized over the hex's twelve symmetries.
• Intersection windows. The three hexes that meet at a corner as a sorted triple, the building or blocked status there, whether an own road or a foreign road reaches it, and the port.
• Portfolio tables. Production pips per resource, the hand and buildings left, own and opponent scores with the turn, development and award status, and an opponent aggregate.

The value is the sum of about eighty looked-up numbers from twelve million entries, in win-probability units. Every window reads only what the seat can see in a sampled world: the board, its own hand and cards, and public counts. No opponent hand, seed, or phase reaches a table, so the same tables serve the search's leaf and the training policy.

The numbers are learned by self-play. Every seat plans its turn with the tunable search at depth 1 with the tables as the leaf, and at game end each seat's chain of end-of-turn positions is nudged toward 1 for the winner and 0 for everyone else, with a small reward per point gained and temporal-coherence step sizes per entry. Blank tables cannot finish a game, so the first twenty thousand games are played by the fast builder while the tables watch.

What has been built

settlers_expectimax::ntuple holds the tables, the pattern extraction, and the file format. SearchConfig.leaf.tables names a table file and swaps the leaf inside the unchanged search; the arena and the remote protocol seat load it once per process. The server's train binary plays self-play games on shared tables with every thread of the machine and validates the frozen tables as the depth-2 leaf against the hand-written leaf on fixed seeds at each checkpoint. Tables stay server-side; the browser worker keeps the hand-written leaf.

Proposed experiment

A pilot trains six layouts for eighteen minutes each: the full set, each pair of families, portfolio tables alone as the mechanism ablation, and the full set without the point reward. The layout with the largest final validation margin trains again from scratch for several hours with a plateau rule, and its best validation point is frozen.

The frozen tables then play registered deterministic cohorts on fresh seeds, 256 games each, one contrast per cohort: the tables versus the hand-written leaf at depth 2 in the same search, the tables played one ply against the depth-2 search, and the tables at depth 2 against the hand-written leaf at depth 3. Hypothesis: the tables beat the hand-written leaf at equal depth, and match it one ply deeper. A protocol cohort against expectimax-v2-plan follows if the engine screen passes.

First results

On 64 fresh deterministic boards the frozen hex plus portfolio tables, as the leaf of the depth-2 search, won 130 of 256 slot-games against 74 for the hand-written leaf in the same search (paired contrast +0.219, 95% interval +0.124 to +0.314). Played one ply with no search they beat the two-turn hand-written search (+0.164), at depth 2 they beat the hand-written leaf at depth 3 (+0.156), and at depth 3 against depth 3 the margin is +0.242 (+0.140 to +0.345). Through the authoritative server the same tables inside the baseline's search won 46 of 80 games against 25 for expectimax-v2-plan (paired contrast +0.263, 95% interval +0.094 to +0.431), so ntuple-leaf is the protocol baseline. The report has the training curve, the decoded hex table, and every run ID, and the protocol report the confirmation. One lineup with two fixed builders; nothing is established against negotiating opponents or humans.

Third round: blending the two leaves, and the endgame switches under the blend

The search can add the hand-written terms on top of the tables (leaf.hand) and set how many hand-written points one win is worth (leaf.scale). Measured as swapped pairs through development, confirmation, population, and extension stages, a quarter of the hand-written terms improves the learned leaf by about +0.07 wins per game in both lineups (pooled +0.066, +0.024 to +0.107, over 192 L1 seeds) at about 40 percent more decision time, half strength is noise, and full strength loses to the tables alone. Halving the scale is refuted and doubling it changes trading without changing wins, so the default scale stands. The endgame race and hidden-points leaves, which read +0.018 and +0.006 under the hand-written leaf, do nothing under the quarter-strength blend either (race slightly negative, hidden points zero), so they stay off under both leaves. The blend report has the tables; the quarter-strength blend is the one candidate here waiting for a protocol cohort against ntuple-leaf.

What could disprove it

Self-play against copies of itself can learn a style that beats itself and loses to the builders; the validation lineup keeps two builders at the table for that reason. Tables trained on depth-1 play may not transfer to the deeper search that uses them. A margin on the fixed validation seeds chooses the candidate and cannot confirm it; only the fresh registered cohorts count.

All approaches · Player’s guide · Experiment program

Tracked investigations