Settlers / Research

73 pages · Search titles and descriptions

↑ ↓ to navigate · Enter to open · Esc to closeLocal search
Play the game

A draft-aware opening planner, calibrated to the leaf

What changed

The tunable search used to rank setup sites by the static leaf, keep six, and value each through a two-turn lookahead that includes the first rolls. That ranking is dominated by raw production, so the pairs it chose covered the island badly: eleven brick pips and no grain on one traced board, no ore on two others. The planner in crates/expectimax/src/v2/opening.rs replaces it. It scores a pair of sites as a portfolio with seven terms: production per resource with a scarcity markup, coverage of distinct dice numbers, a penalty for resources left thin, the time to afford a settlement with a road and a city with surplus converting at bank and port ratios, ports, expansion within two or three roads, and the value lost to the draft. A first placement is valued by the best pair it still leaves after the seats ahead in the snake draft have picked, predicted with the same scoring; the last seat plans both placements at once; roads head for the best site that survives the remaining picks. Every candidate carries its terms, the strategy that values it, and the expected partner site, so a report explains a placement rather than only choosing it.

The calibration

Each row is one unregistered screen of 256 games (boards 0 to 63, four rotations) with the planner-decided seat against the previous placement at depth 2, plus an ETA and a fast builder.

SettingPlannerSearchContrast
pip weights, expansion 0.5 (first draft)23.4%53.5%−0.301
leaf weights, specialize 0.5, scarcity 0.5, expansion 0.1528.5%44.9%−0.164
calibrated, specialize 0, scarcity 047.3%37.9%+0.094
calibrated, specialize 1, scarcity 031.6%47.7%−0.161
calibrated, specialize 0, scarcity 146.9%35.2%+0.117
calibrated, specialize 0.5, scarcity 0.542.2%42.6%−0.004
calibrated, coverage 0.4, balance 0.546.1%37.1%+0.090

Two lessons. First, the leaf's preference for ore and grain, which the search had learned to win with against builders, is right in this arena, and a valuation that rewards spreading production across all five resources is wrong by a wide margin. Second, the five discrete strategies (coverage, cities, roads, ports, cards), blended in by specialize, hurt at every strength because taking the best specialized valuation inflates concentrated pairs of the wrong kind. The strategies remain in the code as a research knob with the default at zero.

Registered cohorts

Experiment d1bdca03-d27f-475c-b1ee-a8caf70937fe, run 08fdd9a7-6e0a-4e2a-a1cc-8124294feae2, boards 64 to 127, four rotations, all 256 games complete: planner in slot 0, 110 wins (43.0%); previous placement in slot 1, 94 (36.7%); paired contrast +0.062 (−0.036 to +0.161). The registered rule asked for an interval excluding zero and was not met.

The same day the bargaining arms found that two identical searches in slots 0 and 1 differ by seating alone (arms report), so the comparison was registered again as swapped pairs on the same boards, planner in slot 0 and then the previous placement in slot 0, with the effect taken as half the difference of the two contrasts and the seating term as half their sum.

BoardsPlanner in slot 0Previous placement in slot 0Planner effectSeating term
0 to 63+0.051+0.062−0.006 (−0.087 to +0.075)+0.057
64 to 127+0.043+0.098−0.027 (−0.111 to +0.057)+0.070
pooled, 128 boards−0.017 (−0.075 to +0.042)+0.063 (+0.008 to +0.119)

Every single-seating number above, including the calibration sweep, carried that seating term. The calibration ordering still stands (the arms were run in the same seating, and the pip-only draft lost by far more than any seating term), but the claim that the calibrated planner beats the search does not.

What could still be wrong

Every game here is against fixed builders that never block and rarely trade, which is exactly the setting where an ore-and-grain start is safest. Against opponents who contest ground the coverage and expansion terms may deserve more weight than this calibration gives them; the knobs exist for that study. The draft prediction assumes opponents place by the same planner, which is exact for the builders and the search but not for a person.

Agent notes

Screens were run with cargo run --release -p settlers-arena --bin tournament -- --seat 'v2:{"depth":2,"opening_weights":{...}}' --seat 'v2:{"depth":2,"opening":"search"}' --seat eta --seat fast --seeds 0-63 --rotations 4 --deterministic and are not filed as records. The confirmation is filed under records/runs/08fdd9a7-6e0a-4e2a-a1cc-8124294feae2.md; the swapped pairs are runs b698064e-39e8-4d73-9844-ac3d5afefce6, da0b1591-e74e-4e56-96b1-1ece9b37900f, 5e3c4985-251d-441f-bea8-512dfe3224b4, and d89ea4de-913b-4255-af42-ba2f18309e87 under the same study. The planner's terms and settings are documented in the server's docs/expectimax.md under Opening placement.