A draft-aware opening planner, calibrated to the leaf
What changed
The tunable search used to rank setup sites by the static leaf, keep six, and
value each through a two-turn lookahead that includes the first rolls. That
ranking is dominated by raw production, so the pairs it chose covered the
island badly: eleven brick pips and no grain on one traced board, no ore on two
others. The planner in crates/expectimax/src/v2/opening.rs replaces it. It
scores a pair of sites as a portfolio with seven terms: production per
resource with a scarcity markup, coverage of distinct dice numbers, a penalty
for resources left thin, the time to afford a settlement with a road and a
city with surplus converting at bank and port ratios, ports, expansion within
two or three roads, and the value lost to the draft. A first placement is
valued by the best pair it still leaves after the seats ahead in the snake
draft have picked, predicted with the same scoring; the last seat plans both
placements at once; roads head for the best site that survives the remaining
picks. Every candidate carries its terms, the strategy that values it, and the
expected partner site, so a report explains a placement rather than only
choosing it.
The calibration
Each row is one unregistered screen of 256 games (boards 0 to 63, four rotations) with the planner-decided seat against the previous placement at depth 2, plus an ETA and a fast builder.
| Setting | Planner | Search | Contrast |
|---|---|---|---|
| pip weights, expansion 0.5 (first draft) | 23.4% | 53.5% | −0.301 |
| leaf weights, specialize 0.5, scarcity 0.5, expansion 0.15 | 28.5% | 44.9% | −0.164 |
| calibrated, specialize 0, scarcity 0 | 47.3% | 37.9% | +0.094 |
| calibrated, specialize 1, scarcity 0 | 31.6% | 47.7% | −0.161 |
| calibrated, specialize 0, scarcity 1 | 46.9% | 35.2% | +0.117 |
| calibrated, specialize 0.5, scarcity 0.5 | 42.2% | 42.6% | −0.004 |
| calibrated, coverage 0.4, balance 0.5 | 46.1% | 37.1% | +0.090 |
Two lessons. First, the leaf's preference for ore and grain, which the search
had learned to win with against builders, is right in this arena, and a
valuation that rewards spreading production across all five resources is
wrong by a wide margin. Second, the five discrete strategies (coverage,
cities, roads, ports, cards), blended in by specialize, hurt at every
strength because taking the best specialized valuation inflates concentrated
pairs of the wrong kind. The strategies remain in the code as a research knob
with the default at zero.
Registered cohorts
Experiment d1bdca03-d27f-475c-b1ee-a8caf70937fe, run
08fdd9a7-6e0a-4e2a-a1cc-8124294feae2, boards 64 to 127, four rotations, all
256 games complete: planner in slot 0, 110 wins (43.0%); previous placement in
slot 1, 94 (36.7%); paired contrast +0.062 (−0.036 to +0.161). The registered
rule asked for an interval excluding zero and was not met.
The same day the bargaining arms found that two identical searches in slots 0 and 1 differ by seating alone (arms report), so the comparison was registered again as swapped pairs on the same boards, planner in slot 0 and then the previous placement in slot 0, with the effect taken as half the difference of the two contrasts and the seating term as half their sum.
| Boards | Planner in slot 0 | Previous placement in slot 0 | Planner effect | Seating term |
|---|---|---|---|---|
| 0 to 63 | +0.051 | +0.062 | −0.006 (−0.087 to +0.075) | +0.057 |
| 64 to 127 | +0.043 | +0.098 | −0.027 (−0.111 to +0.057) | +0.070 |
| pooled, 128 boards | −0.017 (−0.075 to +0.042) | +0.063 (+0.008 to +0.119) |
Every single-seating number above, including the calibration sweep, carried that seating term. The calibration ordering still stands (the arms were run in the same seating, and the pip-only draft lost by far more than any seating term), but the claim that the calibrated planner beats the search does not.
What could still be wrong
Every game here is against fixed builders that never block and rarely trade, which is exactly the setting where an ore-and-grain start is safest. Against opponents who contest ground the coverage and expansion terms may deserve more weight than this calibration gives them; the knobs exist for that study. The draft prediction assumes opponents place by the same planner, which is exact for the builders and the search but not for a person.
Agent notes
Screens were run with cargo run --release -p settlers-arena --bin tournament -- --seat 'v2:{"depth":2,"opening_weights":{...}}' --seat 'v2:{"depth":2,"opening":"search"}' --seat eta --seat fast --seeds 0-63 --rotations 4 --deterministic and are not filed as records. The confirmation
is filed under records/runs/08fdd9a7-6e0a-4e2a-a1cc-8124294feae2.md; the
swapped pairs are runs b698064e-39e8-4d73-9844-ac3d5afefce6,
da0b1591-e74e-4e56-96b1-1ece9b37900f, 5e3c4985-251d-441f-bea8-512dfe3224b4,
and d89ea4de-913b-4255-af42-ba2f18309e87 under the same study. The
planner's terms and settings are documented in the server's
docs/expectimax.md under Opening placement.