Settlers / Research

81 pages · Search titles and descriptions

↑ ↓ to navigate · Enter to open · Esc to closeLocal search
Play the game

The quarter blend through the protocol arena

The quarter blend against ntuple-leaf through the protocol arena

Measured evidence
-0.25-0.12500.1250.25Half A (blend first)Half B (blend second)Seating-corrected effectSeating termContrastWins per gameNo difference

Scroll the chart horizontally to inspect all values.

20 seeds, both seatings95% intervalNo difference

Each seating alone carries the turn-order term; the corrected effect is the middle point.

View data table

Source: Source: analysis/blend_protocol.py over runs run record (half A) and run record (half B); seeds 2000 to 2019, four rotations per seed per half, 80 and 80 completed games. Contrasts are candidate minus control in wins per game; the effect is half the per-seed difference of the two halves' slot-0-minus-slot-1 contrasts and the seating term half their sum; 95% normal intervals over seeds.

SeriesContrastWins per gameLowHigh
20 seeds, both seatingsHalf A (blend first)0.013-0.1560.181
20 seeds, both seatingsHalf B (blend second)0.025-0.1450.195
20 seeds, both seatingsSeating-corrected effect0.019-0.0870.124
20 seeds, both seatingsSeating term-0.006-0.1390.126

What was registered

The blend is ntuple-leaf's search (depth 3, 1.5 s budget, 8 dice scenarios, 4 sampled worlds, 4 inner scenarios) with leaf.hand 0.25 added on top of the same frozen tables, registered in the policy registry as ntuple-leaf-blend. The engine retest had read it at +0.066 wins per game against builders and +0.064 at a table of searches, at 1.4 times the decision time, and the current evidence asked for a protocol cohort before any default changes.

Two halves were registered under patterns/blend-protocol before either ran, each naming the other: half A seats the blend in slot 0 and ntuple-leaf in slot 1, half B swaps them; eta and fast fill slots 2 and 3 in both. Each half plays deterministic seeds 2000 to 2019 once per rotation of the four slots, 80 games, with the same points, match, decision, and turn caps as the cohort that made ntuple-leaf the baseline. The rule: all 160 games valid with no server timeout moves; support if the seating-corrected effect's interval lies above zero and the estimate reaches +0.03 wins per game; refutation if the interval lies below zero; otherwise inconclusive.

The two halves

Half A and half B each completed all 80 games with no timeout moves, twelve matches in flight, 72 and 76 turns per game on average. The two searches ran as separate remote processes against the server and saw only their own seated view and event history; both loaded the frozen tables (SHA-256 f8e61e66…) and reported the tables evaluator on every decision.

Half A, blend in slot 0:

Slot / policyWinsCompletedWin rate95% Wilson intervalMean points
0 / ntuple-leaf-blend348042.5%32.3% to 53.4%7.79
1 / ntuple-leaf338041.2%31.1% to 52.2%7.33
2 / eta88010.0%5.2% to 18.5%6.28
3 / fast5806.2%2.7% to 13.8%6.55

Half B, blend in slot 1:

Slot / policyWinsCompletedWin rate95% Wilson intervalMean points
0 / ntuple-leaf328040.0%30.0% to 51.0%7.24
1 / ntuple-leaf-blend348042.5%32.3% to 53.4%7.81
2 / eta5806.2%2.7% to 13.8%6.88
3 / fast98011.2%6.0% to 20.0%6.96

Win rate by competitor, half A

95% interval
ntuple-leaf-blend · slot 042.5% · n=80
ntuple-leaf · slot 141.3% · n=80
eta · slot 210.0% · n=80
fast · slot 36.3% · n=80
0%50%100%
run record. 80 completed of 80 planned games. 95% Wilson intervals; descriptive evidence only.

Win rate by competitor, half B

95% interval
ntuple-leaf · slot 040.0% · n=80
ntuple-leaf-blend · slot 142.5% · n=80
eta · slot 26.3% · n=80
fast · slot 311.3% · n=80
0%50%100%
run record. 80 completed of 80 planned games. 95% Wilson intervals; descriptive evidence only.

The seating-corrected effect

ContrastWins per game (95% interval over 20 seeds)
Half A, slot 0 minus slot 1 (blend minus ntuple-leaf)+0.013 (−0.156 to +0.181)
Half B, slot 0 minus slot 1 (ntuple-leaf minus blend)−0.025 (−0.195 to +0.145)
Seating-corrected effect of the blend+0.019 (−0.087 to +0.124)
Seating term−0.006 (−0.139 to +0.126)

The blend won one more game than the tables alone in half A and two more in half B, and the two seatings agree that there is nothing large to find: the corrected estimate is +0.019 wins per game, below the +0.03 bar, with an interval that includes zero, so the registered rule reads inconclusive. It is not a refutation. Twenty boards played in both seatings resolve about ±0.11 wins per game, and the engine estimate of +0.066 sits comfortably inside the interval; this cohort could not separate an effect of that size from nothing. The seating term is invisible here (−0.006), unlike the engine arena's +0.05 to +0.18, which says nothing about the arena difference on twenty boards.

What the blend changes at the table is the same as in the engine: over both halves it built more cities per game (1.43 and 1.38 against 1.09 and 1.14), traded more with players (2.30 and 2.46 against 1.96 and 2.14), and finished with about half a point more (7.79 and 7.81 against 7.33 and 7.24), and those points did not become wins.

Decision time and completed depth

The remote runner's search seat now writes an optional per-decision trace (SETTLERS_SEARCH_TRACE, kept in each seat's private policy log), so this is the first protocol cohort that can say how much of the depth-3 search the 1.5 s clock actually delivered. The machine was shared with five other agents throughout (load average 60 to 80 on 32 cores), and both seats in every game shared that load.

HalfPolicyContested decisionsMean msMedian msCompleted depth 3Depth 2Depth 1Stopped by the budget
Antuple-leaf-blend3,26793288249.9%42.8%7.3%26.0%
Antuple-leaf3,07382773261.2%35.3%3.5%20.2%
Bntuple-leaf3,26783474960.9%35.4%3.7%21.8%
Bntuple-leaf-blend3,42291185351.8%42.1%6.2%25.3%

Contested decisions are main-phase decisions with at least two root actions; a single candidate never deepens past the first depth. Over all decisions, including the trivial ones, the blend's median was 516 and 478 ms against 427 and 456 ms. The blend spent about 1.1 times the control's time per contested decision, less than the engine's 1.4 because the clock caps the difference, and paid for it in depth: it completed depth 3 in half of its contested decisions where the tables alone completed it in three fifths, and it was stopped by the budget in a quarter of them against a fifth. Both seats fell back to the first legal action on a search error in about 0.2% of decisions (22 and 19 for the blend, 19 and 27 for the tables alone, out of 9,000 to 10,000 decisions per seat per half), a pre-existing path of the search when it returns no candidate.

One game, viewed from above

The first match of the cohort, seed 2000 in its first rotation, which the blend's seat won in 49 turns with ten points against two, five, and eight. It is a real recorded game chosen for its position in the schedule; it does not by itself establish which policy is stronger.

ntuple-leaf-blend slot 0’s turn

Turn 0 · Place a settlement · 10 points to win

Grain8Grain, rolls 8, 5 of 36 dice outcomesLumber4Lumber, rolls 4, 3 of 36 dice outcomesOre11Ore, rolls 11, 2 of 36 dice outcomesDesertDesert, blocked by the robberOre3Ore, rolls 3, 2 of 36 dice outcomesOre6Ore, rolls 6, 5 of 36 dice outcomesLumber12Lumber, rolls 12, 1 of 36 dice outcomesGrain5Grain, rolls 5, 4 of 36 dice outcomesLumber10Lumber, rolls 10, 3 of 36 dice outcomesBrick11Brick, rolls 11, 2 of 36 dice outcomesWool5Wool, rolls 5, 4 of 36 dice outcomesBrick9Brick, rolls 9, 4 of 36 dice outcomesGrain2Grain, rolls 2, 1 of 36 dice outcomesGrain9Grain, rolls 9, 4 of 36 dice outcomesWool4Wool, rolls 4, 3 of 36 dice outcomesBrick10Brick, rolls 10, 3 of 36 dice outcomesLumber6Lumber, rolls 6, 5 of 36 dice outcomesWool3Wool, rolls 3, 2 of 36 dice outcomesWool8Wool, rolls 8, 5 of 36 dice outcomes2:1Lumber 2:1 port2:1Ore 2:1 port2:1Wool 2:1 port2:1Brick 2:1 port3:1Any resource 3:1 port3:1Any resource 3:1 port2:1Grain 2:1 port3:1Any resource 3:1 port3:1Any resource 3:1 port
Settlement City Robber Roll frequency
Bank19Brick19Lumber19Wool19Grain19Ore25 development cards left

ntuple-leaf-blend slot 0

Playing

0 VP
0 resource cards0 development cards0 settlements0 cities0 roads0 knights played
Longest route: 0

ntuple-leaf slot 1

0 VP
0 resource cards0 development cards0 settlements0 cities0 roads0 knights played
Longest route: 0

eta slot 2

0 VP
0 resource cards0 development cards0 settlements0 cities0 roads0 knights played
Longest route: 0

fast slot 3

0 VP
0 resource cards0 development cards0 settlements0 cities0 roads0 knights played
Longest route: 0

Actions & conversation

1 events · since the previous position

  1. Table

    The game started.

Game 1 in the scheduled cohort.

120 of 159 recorded positions. Board changes between positions are grouped.

Replay sources

Game b61c9297-11d1-400a-87e6-340c7d29ae90 · version 17

Frames: d02fab140bfb654d4f7629c111a8c89875998f52ba0643f56974f7f721c328b0

Public events: 72f7ae3368602daf0c7e1b656a11e19f38001bf5d827ef3fd957ff9cbe2b678d

Limitations

One lineup, one rotation scheme, twenty boards: a swapped pair on twenty seeds resolves about ±0.11 wins per game, so only a large effect could clear the +0.03 bar here, and the engine estimate of +0.066 sits inside what this cohort cannot separate from zero. The two searches are time-budgeted, and the cohort ran on a machine shared with five other agents, so both seats completed depth 3 less often than the unloaded 2026-09-09 cohort would have, and the blend lost about ten points of depth-3 share to its cost. The contrast is fair within a game, but it measures the two evaluators as the machine could run them, not at full depth. A first attempt at half A stopped on a seat-join conflict at game creation (abandoned attempt, 18 valid games, not pooled) and the harness's join step was made to retry on conflict before the pair was played; the daily log has the details. Nothing here speaks to negotiating opponents, humans, or the browser's hand-written leaf.

What the browser would need

The browser worker runs the hand-written leaf in WASM and cannot load the tables, so leaf.hand is not a browser switch. Using the blend there would need the 14.3 MB tables shipped to the worker, which the WASM build rejects by decision, or a server-side seat; nothing in web changes with this result.

Half A record · Half B record · Learned pattern values · Log 2026-09-25

Detailed result and cohort context

Public frame source SHA-256 of the replayed game: d02fab140bfb654d4f7629c111a8c89875998f52ba0643f56974f7f721c328b0.

Through the authoritative server, on 20 deterministic seeds (2000 to 2019) each played once per rotation of the four slots in each of two seatings, ntuple-leaf-blend (the ntuple-leaf search at depth 3 under 1.5 s with leaf.hand 0.25 over the frozen hex plus portfolio tables) won 34 of 80 games in half A (blend in slot 0) against 33 for ntuple-leaf, and 34 of 80 in half B (blend in slot 1) against 32. The slot-0-minus- slot-1 contrasts are +0.013 (−0.156 to +0.181) and −0.025 (−0.195 to +0.145); the seating-corrected effect of the blend is +0.019 wins per game (95% interval −0.087 to +0.124) and the seating term −0.006 (−0.139 to +0.126). All 160 games completed validly with no timeout move. By the registered rule the result is inconclusive: the interval includes zero and the estimate is below +0.03. The protocol reference remains ntuple-leaf; the server default is unchanged.

Reproduction: SETTLERS_SEARCH_TRACE=1 just run EXPERIMENT --parallel 12 from the research checkout with SETTLERS_SERVER_DIR pointing at a server worktree at or after the trace commit on dev, then python3 analysis/blend_protocol.py cfee89ba-3e06-4549-8cae-c31652464985 2047d413-be8e-4a80-bcea-4c53fbf78619 --assets content/assets. The half A run belongs to experiment 0d174884-6e7e-4d88-8f1c-dd51cb21ba29, the half B run to 23b1085d-20a9-473f-a31d-daa907181449; the abandoned first attempt at half A is experiment 7de5ddb4-c007-4e66-b4e6-7c08e0fa07c1, run 99bfec6f-10ac-4536-99bf-460d62d0fea5, and the four-game smoke that preceded them is run 692397a2-d408-421c-b7b6-ac518031aba4. Half A's chart and replay come from just publish cfee89ba-… --match 0 --slug blend-protocol.