After the fix: half the games on paired boards
Corrected search at depth 2 against the builders
95% intervalWhat the defect was
The first cohorts ran with a turn-planning defect. After any non-ending action, the enumerator reused the root's restricting action as its own action list, so it re-applied that action, failed, and explored nothing. A road, a bank exchange, or a purchase was therefore valued at the position right after it, without the opponents' round that every end-turn candidate received, and inner turns collapsed to ending the turn. Measuring the game tree exposed it: inner-level nodes averaged 1.17 end-of-turn positions where a hand with several options should produce many.
The correction limits the restriction to the root step. A second problem then surfaced: with sequences actually explored, the flattened list of end-of-turn positions averaged over the player's own alternatives instead of taking the best, and enumerating every sequence cost seconds. The planner is now a recursion that takes the maximum over own actions and the expectation over chance outcomes, keeps every first action but only the best few continuations of the current turn, follows one greedy line in later turns, and spends the lookahead on the afterstates of the chosen line.
Depth 3 wins more still
Run 156f77ff-b781-42d6-b6e0-999bdf9506fe seated the same lineup with depth 3
under a 1.5 s budget. It won 147 of 256 slot-games (57.4%) against the
compared ETA slot's 16.0%; the paired contrast is +0.414 (95% interval
+0.327 to +0.502), above depth 2's +0.301 on the same boards. Decisions averaged
360 ms. With the defective search, depth 3 had bought nothing; with the turn
tree working, the extra planned turn is worth about eleven more wins per 256.
Corrected search at depth 3 with a 1.5 s budget
95% intervalAgainst the reference it replaces
Run 436f4740-c7a5-48f2-9e35-9f1ce92c3af5 seated the corrected depth-2 search
with the frozen expectimax-v1, an ETA builder and a fast builder. It won
120 of 256 slot-games (46.9%); the reference won 34 (13.3%), ETA 69 and
fast 33. The paired contrast against the reference is +0.336 (95% interval
+0.249 to +0.423). The reference spent 240 ms per decision to the corrected
search's 88 ms.
Corrected search against the frozen reference
95% intervalWhat the corrected search does differently
The per-game means show a builder-like profile: 2.0 settlements, 2.1 cities, 4.7 roads and 2.8 development cards, with 6.2 completed player trades and 9.9 offers per game. The defective search offered less and, in its deep variant, bought more development cards than it built cities.
What each policy built per game
Measured evidenceScroll the chart horizontally to inspect all values.
Per-game means over 256 completed games per slot; the corrected search builds like ETA and trades with players more often.
Source: Engine-arena run a0c91849-4abf-4f1c-9de1-6255e1ef73e6; per-game means over 256 completed games per slot.
View data table
| Series | Action | Per game |
|---|---|---|
| expectimax-v2 depth 2 | settlements | 2.02 |
| expectimax-v2 depth 2 | cities | 2.06 |
| expectimax-v2 depth 2 | roads | 4.71 |
| expectimax-v2 depth 2 | dev cards | 2.76 |
| expectimax-v2 depth 2 | player trades | 6.2 |
| expectimax-v2 depth 2 | bank trades | 4.27 |
| ETA builder | settlements | 2.31 |
| ETA builder | cities | 2.31 |
| ETA builder | roads | 4.75 |
| ETA builder | dev cards | 1.08 |
| ETA builder | player trades | 5.3 |
| ETA builder | bank trades | 5.29 |
| fast builder | settlements | 1.46 |
| fast builder | cities | 2.79 |
| fast builder | roads | 3.11 |
| fast builder | dev cards | 2.06 |
| fast builder | player trades | 5.34 |
| fast builder | bank trades | 5.44 |
Agent notes
Registered experiments d5eb6354-2b59-4ca4-8ab0-fefee3690cb3 (run
a0c91849-4abf-4f1c-9de1-6255e1ef73e6, depth 2) and
11cb5a59-5490-4a78-96cf-2332d8364cf2 (run
156f77ff-b781-42d6-b6e0-999bdf9506fe, depth 3 with "time_budget_ms":1500)
seat the search against eta, fast, eta; experiment
8d5b24cd-e6ed-4a04-87c8-c1d0db7b747c (run
436f4740-c7a5-48f2-9e35-9f1ce92c3af5) seats depth 2 against v1, eta,
fast. The depth-2 configuration is
v2:{"depth":2,"samples":4,"scenarios":8,"inner_scenarios":4}; every cohort
uses seeds 0 to 63 with all four rotations. The paired
contrast is the per-seed mean of slot-0 minus slot-1 wins with a
normal-approximation interval over 64 seeds. Reproduce with
just engine-run EXPERIMENT after just build-arena; the manifest records the
tournament binary hash and both repositories' Git state (server commit
59433b7 carries the fix). Attribution: Claude Fable 5.1 (claude-fable-5-1)
through Claude Code.