Settlers / Research

73 pages · Search titles and descriptions

↑ ↓ to navigate · Enter to open · Esc to closeLocal search
Play the game

What each mechanism is worth

Search, beliefs, trading, and opponent model

Measured evidence
-0.54-0.3675-0.195-0.02250.15event-knowledgesampled-worldslookaheadscenario-countoffer-policytrade-opennessopponent-modelAblation armWins per game, treatment minus controlno difference

Scroll the chart horizontally to inspect all values.

Paired seed contrast (95% interval)95% intervalno difference

Each point is the mean per-seed difference in wins between a search with one change and the unchanged depth-2 control; the bar is the 95% interval over 64 seeds. Points below zero mean the change hurt.

Source: Engine-arena ablation cohorts: 64 deterministic seeds each played once per rotation (256 games per arm); treatment in slot 0, the unchanged depth-2 control in slot 1, an ETA and a fast builder in the other seats. Run IDs are listed in the ablation report.

View data table
SeriesAblation armWins per game, treatment minus controlLowHigh
Paired seed contrast (95% interval)event-knowledge-0.0898-0.20080.0211
Paired seed contrast (95% interval)sampled-worlds-0.0508-0.16590.0643
Paired seed contrast (95% interval)lookahead-0.3242-0.4133-0.2351
Paired seed contrast (95% interval)scenario-count-0.4102-0.4891-0.3312
Paired seed contrast (95% interval)offer-policy-0.0273-0.13210.0774
Paired seed contrast (95% interval)trade-openness-0.0938-0.19230.0048
Paired seed contrast (95% interval)opponent-model-0.0078-0.11130.0957

Leaf terms and style sliders

Measured evidence
-0.23-0.12-0.010.10.21search-placementacquisition-time-termsite-value-termsport-value-termsdiscard-exposurerisk-appetitecard-termscard-appetiteopponent-termsaggressionAblation armWins per game, treatment minus controlno difference

Scroll the chart horizontally to inspect all values.

Paired seed contrast (95% interval)95% intervalno difference

Each point is the mean per-seed difference in wins between a search with one leaf term switched off or one slider at its limit and the unchanged depth-2 control; the bar is the 95% interval over 64 seeds.

Source: Engine-arena ablation cohorts: 64 deterministic seeds each played once per rotation (256 games per arm); treatment in slot 0, the unchanged depth-2 control in slot 1, an ETA and a fast builder in the other seats. Run IDs are listed in the ablation report.

View data table
SeriesAblation armWins per game, treatment minus controlLowHigh
Paired seed contrast (95% interval)search-placement-0.0352-0.13270.0624
Paired seed contrast (95% interval)acquisition-time-term0.0625-0.03930.1643
Paired seed contrast (95% interval)site-value-terms-0.0742-0.16920.0207
Paired seed contrast (95% interval)port-value-terms0.0195-0.08590.1249
Paired seed contrast (95% interval)discard-exposure-0.0234-0.13350.0866
Paired seed contrast (95% interval)risk-appetite-0.0625-0.17540.0504
Paired seed contrast (95% interval)card-terms-0.0352-0.14420.0739
Paired seed contrast (95% interval)card-appetite0.0312-0.06970.1322
Paired seed contrast (95% interval)opponent-terms0-0.10470.1047
Paired seed contrast (95% interval)aggression0.0195-0.08420.1232

How the arms were built

Every arm seats a search with exactly one change in slot 0 and the unchanged depth-2 control in slot 1, with an ETA builder and a fast builder in the other seats. Games run in deterministic mode, so both searches and both builders face the same board, the same k-th roll, the same k-th theft, and the same deck order on every seed; each seed is played once per rotation of the four slots. The primary outcome is the per-seed difference in wins between slots 0 and 1 with a normal-approximation interval over 64 seeds. Depth 2 is the control because it is representative and four times cheaper than the confirmed depth-3 candidate; a change that helps at depth 2 is a candidate for the deep configuration, not a proof.

The arms

ApproachChange testedTreatment winsControl winsContrast (95% interval)
Searchdepth 1: no lookahead50133−0.324 (−0.413 to −0.235)
Searchtwo dice scenarios instead of eight27132−0.410 (−0.489 to −0.331)
Beliefshands sampled from counts only, no event history78101−0.090 (−0.201 to +0.021)
Beliefsone sampled world instead of four89102−0.051 (−0.166 to +0.064)
Bargainingno trade margin (trade slider at 1)80104−0.094 (−0.192 to +0.005)
Bargainingnever propose a trade8390−0.027 (−0.132 to +0.077)
Expansionsite-value and road-contention terms off87106−0.074 (−0.169 to +0.021)
Riskrisk slider at 1 (half the discard weight)88104−0.062 (−0.175 to +0.050)
Riskdiscard-exposure term off9096−0.023 (−0.134 to +0.087)
Openingbuilder placement instead of search placement8796−0.035 (−0.133 to +0.062)
Developmentcard and knight terms off91100−0.035 (−0.144 to +0.074)
Developmentdevelopment slider at 19890+0.031 (−0.070 to +0.132)
Adaptationleaf-guided opponents instead of the builder model8890−0.008 (−0.111 to +0.096)
Containmentall opponent terms off96960.000 (−0.105 to +0.105)
Containmentaggression slider at 19994+0.020 (−0.084 to +0.123)
Portsport terms off9792+0.020 (−0.086 to +0.125)
Tempoacquisition-time plan term off10589+0.062 (−0.039 to +0.164)

Each row is 256 games. Removing the plan term did not hurt at depth 2, which was unexpected: the card, production, and hand terms already carry most of the acquisition-time signal once the search plans a whole turn, so the term that rescued the first leaf is now nearly redundant. Six arms whose intervals brushed zero (event knowledge, trade margin, expansion terms, risk slider, sampled worlds, and the plan term) are registered again on 192 fresh seeds; their results are added below when they complete.

Confirmation on fresh seeds

The six borderline arms were registered again on 192 fresh seeds (64 to 255), 768 games each, with everything else unchanged.

Confirmation on fresh seeds

Measured evidence
-0.18-0.0975-0.0150.06750.15acquisition-time-termsite-value-termsevent-knowledgesampled-worldsrisk-appetitetrade-opennessAblation armWins per game, treatment minus controlno difference

Scroll the chart horizontally to inspect all values.

Paired seed contrast (95% interval)95% intervalno difference

Each point is the mean per-seed difference in wins between a search with one change and the unchanged depth-2 control; the bar is the 95% interval over 192 fresh seeds (64 to 255).

Source: Engine-arena ablation cohorts: 192 fresh seeds (64 to 255), each played once per rotation; treatment in slot 0, the unchanged depth-2 control in slot 1, an ETA and a fast builder in the other seats. Run IDs are listed in the ablation report.

View data table
SeriesAblation armWins per game, treatment minus controlLowHigh
Paired seed contrast (95% interval)acquisition-time-term0.0391-0.0210.0991
Paired seed contrast (95% interval)site-value-terms-0.0156-0.07790.0467
Paired seed contrast (95% interval)event-knowledge-0.0664-0.1253-0.0075
Paired seed contrast (95% interval)sampled-worlds-0.0117-0.07180.0483
Paired seed contrast (95% interval)risk-appetite-0.0547-0.11780.0084
Paired seed contrast (95% interval)trade-openness-0.0156-0.07470.0435
ArmTreatment winsControl winsContrastVerdict
Tempo: plan term off307277+0.039 (95% interval -0.021 to +0.099)inconclusive; the term is not needed at depth 2
Expansion: site and road terms off275287-0.016 (95% interval -0.078 to +0.047)no effect
Beliefs: counts only, no event history250301-0.066 (95% interval -0.125 to -0.008)supported: event knowledge pays
Beliefs: one sampled world271280-0.012 (95% interval -0.072 to +0.048)no effect
Risk: slider at 1261303-0.055 (95% interval -0.118 to +0.008)inconclusive, leaning harmful
Bargaining: no trade margin281293-0.016 (95% interval -0.075 to +0.043)no effect

One mechanism survives: sampling hidden hands from the event history instead of public counts alone is worth about 0.07 wins per game, and its interval excludes zero. The expansion terms, the number of sampled worlds, and the trade margin do nothing measurable at this scale, and the plan term that rescued the first leaf is redundant once turns are planned. The risk slider leans harmful but is not resolved by 768 games.

Four styles at one table

Four search styles at one table

95% interval
builder (slot 0)25.8% · n=256
raider (slot 1)23.0% · n=256
gambler (slot 2)27.0% · n=256
trader (slot 3)24.2% · n=256
0%50%100%
Per-preset win rates when four depth-2 searches with different inclination presets share a table. Run 729fc30f-edaa-4120-829d-125e68825e66: 256 of 256 games completed; deterministic tapes, 64 seeds, four rotations.

Four depth-2 searches that differ only in their inclination presets (builder, raider, gambler, trader) split a table almost evenly: 66, 59, 69, and 62 wins of 256. The sliders change what a seat builds and how it trades without changing how often it wins against equals, which is what a style control should do.

What this program cannot test

Truthful signals, bluffs, and bounded pliability need messages and a recipient that reads them; the fixed builders ignore conversation. Reputation has one mechanical proxy, a willingness model that learns each partner's acceptance rate from the public trade record. Its arm (run 7c17e2c7-a140-4fe5-b055-68bdea33ea60) changed the contrast by -0.031 wins per game (95% interval -0.125 to +0.063): against builders that accept by a fixed rule there is nothing to learn, so the test needs a reciprocal opponent.

candidate Acandidate Bsame dice1: 6 8 5762: 9 4 7343: 6 6 10984: 2 8 353mean 6.0mean 5.3
Both candidates face the same numbered dice sequences, so their values compare directly.
Agent notes

Engine-arena registrations under the approach studies, all with deterministic = true, seeds 0 to 63, four rotations, treatment slot 0 v2:{...one change...}, control slot 1 v2:{"depth":2,"samples":4,"scenarios":8,"inner_scenarios":4}, then eta and fast. Run IDs: opening 47db0437-fcc8-4bce-be06-57c2caadf5f9, tempo d4472a6a-7480-43c5-a045-bd23cc41f944, expansion 52198182-e594-45d0-b71a-71c6f90835e4, ports 03b1a60e-01fb-4a0f-85a6-db7c83a98472, beliefs (event knowledge) 165fc976-0cb1-40af-a17b-14911bbdcc89, beliefs (sampled worlds) e5e7218e-5a00-4689-842d-01228f1bca19, search (lookahead) 7586d662-410e-413b-a6d6-b1f8386ff39e, search (scenarios) 917f7901-15a9-410f-b8f5-314a8041ab5b, risk (exposure term) 0b9e852a-cc87-4d37-8694-ef01545b677e, risk (slider) d611102f-d5c0-4828-9131-1e746844ff65, development (terms) 3fdce968-dd9a-4bba-9698-79bee17b6d28, development (slider) e2b45621-eead-4c98-ade1-8113ced106f5, bargaining (proposals off) 0e7068ca-536d-4c35-a77c-e37f55119639, bargaining (trade slider) 7ff05cd9-d1d7-477a-8386-f49ddefba3be, containment (terms) 49df8c4e-96e9-497f-80ce-d8dd9ed19302, containment (slider) c4427fe6-e074-4598-83cc-aa2e4a0dc283, adaptation 3807570d-117b-4b6f-b0c4-86b77767b6f5, cross-play 729fc30f-edaa-4120-829d-125e68825e66. Reproduce any arm with just engine-run EXPERIMENT; the forest plot comes from analysis/ablation_assets.py. Server commit 36c6899 carries the deterministic mode and ablation switches. Attribution: Claude Fable 5.1 (claude-fable-5-1) through Claude Code. Confirmation runs: tempo b4d274b1-d42c-44cb-bb54-25bfa331f346, expansion 0ea66d80-04a1-470f-b6dc-384cf6f1dde6, beliefs (event knowledge) 9fa439db-c29a-446b-94ea-147bae23e473, beliefs (sampled worlds) 5b5b528d-de10-4a20-921f-68f1b286d6e9, risk (slider) 837c9293-5996-49d1-8d1f-019c2dd395ee, bargaining (trade slider) 1bfc00f0-05ed-4635-b499-69cb50bc4c80; reciprocity 7c17e2c7-a140-4fe5-b055-68bdea33ea60.