Settlers / Research

73 pages · Search titles and descriptions

↑ ↓ to navigate · Enter to open · Esc to closeLocal search
Play the game

Development-card timing under the learned leaf

Design

Each arm is one switch on the tables baseline seat (v2:{"depth":2,"leaf":{"tables":...}}) measured as a swapped pair: half A seats the candidate in slot 0 and the control in slot 1, half B swaps them, both on the same 64 deterministic boards played once per rotation of the four slots (512 games per pair). The effect is the per-seed half-difference of the two slot-0-minus-slot-1 contrasts and the seating term their half-sum, which cancels the arena's turn-order term. Stages, in order: development on boards 0 to 63 with an ETA and a fast builder beside the searches (the lineup of the hand-leaf cohorts, so the leaves compare on the same boards), confirmation on fresh boards 800 to 863 in the same lineup, and population on boards 800 to 863 with a third tables search in slot 2 and an ETA in slot 3, so purchases and knight plays are answered by opponents that trade and block.

buy_tempo and hold_knight are candidate rules and act under the tables alone. knight_play_value shapes a hand-written leaf term that the tables alone never evaluate, so its control and candidate both carry the hand-written terms on top of the tables. A blend screen run in parallel measured the full blend (hand weight 1.0) at −0.207 wins per game against the tables alone and the quarter blend at +0.074, so the arm's primary evidence is on the quarter blend; the development-stage pair on the full blend was already running when the screen landed and is reported as what it is. One defect had to be fixed first: the blended leaf dropped the knight_play_value flag, which would have made the switch inert under any blend; the blend now evaluates the hand terms with the flag, pinned by a test in the server's development suite.

Development-card timing switches under the learned tables leaf

Measured evidence
-0.15-0.0650.020.1050.19buy tempo, development, boards 0-63buy tempo, confirmation, boards 800-863buy tempo, population, boards 800-863hold knight, development, boards 0-63hold knight, confirmation, boards 800-863hold knight, population, boards 800-863knight value (quarter blend), development, boards 0-63knight value (quarter blend), confirmation, boards 800-863knight value (quarter blend), population, boards 800-863knight value (full blend), development, boards 0-63CohortSeating-corrected effect, wins per game

Scroll the chart horizontally to inspect all values.

Seating-corrected effect (swapped pair)95% interval

Each point is one swapped pair’s seating-corrected effect in wins per game with its 95% interval over boards; the two candidate-rule arms play under the tables alone, the knight-value arm under the named blend.

Source: Engine-arena swapped pairs of 512 games each on deterministic boards played once per rotation, the candidate and the control in slots 0 and 1 in both seatings; the builders lineup (L1) seats an ETA and a fast builder beside the searches, the population lineup (L2) a third tables search and an ETA. Each point is the per-seed half-difference of the two slot-0-minus-slot-1 contrasts of a pair with its 95% interval over 64 boards. Runs: f97d8683, 176f12e2; 3292674f, a86a5e49; ea42e55d, 64bbd775; c5ffb6cf, 4a18f4f2; 30180b9b, e6c5fbd4; 873ab9ef, 08636046; d493c5b4, 8a8b0ffb; b41de3cf, 41c8da11; 8263d242, e3fc4a9c; a6afd7a8, 5046ec2d.

View data table
SeriesCohortSeating-corrected effect, wins per gameLowHigh
Seating-corrected effect (swapped pair)buy tempo, development, boards 0-63-0.0605-0.1151-0.006
Seating-corrected effect (swapped pair)buy tempo, confirmation, boards 800-863-0.0059-0.07010.0583
Seating-corrected effect (swapped pair)buy tempo, population, boards 800-863-0.0527-0.10990.0044
Seating-corrected effect (swapped pair)hold knight, development, boards 0-63-0.0293-0.07110.0125
Seating-corrected effect (swapped pair)hold knight, confirmation, boards 800-863-0.0254-0.07630.0256
Seating-corrected effect (swapped pair)hold knight, population, boards 800-863-0.0508-0.0976-0.0039
Seating-corrected effect (swapped pair)knight value (quarter blend), development, boards 0-63-0.002-0.03110.0272
Seating-corrected effect (swapped pair)knight value (quarter blend), confirmation, boards 800-8630-0.02840.0284
Seating-corrected effect (swapped pair)knight value (quarter blend), population, boards 800-8630.0039-0.02650.0343
Seating-corrected effect (swapped pair)knight value (full blend), development, boards 0-630.10940.05470.1641

The pairs

Every half completed 256 of 256 games. The contrast columns give each half's slot-0-minus-slot-1 wins per game; the effect column is the seating-corrected combination with its 95% interval over the 64 boards. Negative effects mean the switch loses.

SwitchStageHalf A (run)Half B (run)Effect (95% interval)
buy_tempodevelopment+0.016 (−0.078 to +0.109), f97d8683-79fd-4109-a1ef-7b106f8b3956+0.137 (+0.050 to +0.223), 176f12e2-d391-4402-9b7a-7fa5cc470570−0.061 (−0.115 to −0.006)
buy_tempoconfirmation+0.070 (−0.022 to +0.163), 3292674f-249b-44d7-967c-5481ac3045f1+0.082 (−0.024 to +0.188), a86a5e49-0165-4c3d-ac6a-1a301a4cc496−0.006 (−0.070 to +0.058)
buy_tempopopulation−0.094 (−0.178 to −0.010), ea42e55d-da88-4794-a3b7-08c580473ee8+0.012 (−0.072 to +0.095), 64bbd775-5639-4bfe-bd5f-b0bcb0cef057−0.053 (−0.110 to +0.004)
hold_knightdevelopment+0.062 (−0.023 to +0.148), c5ffb6cf-45d1-48a1-a2d8-3b2fa3be7666+0.121 (+0.027 to +0.216), 4a18f4f2-f9cf-4f33-97e6-26e471bf6654−0.029 (−0.071 to +0.013)
hold_knightconfirmation+0.035 (−0.066 to +0.136), 30180b9b-3de5-46d8-ae0e-30e259e21c4d+0.086 (−0.024 to +0.196), e6c5fbd4-7f3d-4639-a41e-caac6e6c9022−0.025 (−0.076 to +0.026)
hold_knightpopulation−0.086 (−0.171 to −0.001), 873ab9ef-b922-498b-84d4-17f28380a5fa+0.016 (−0.070 to +0.101), 08636046-1590-4736-be1a-7b1fe0b5e018−0.051 (−0.098 to −0.004)
knight_play_value, quarter blenddevelopment+0.133 (+0.035 to +0.231), d493c5b4-e8fa-4e07-84f3-83d995d73d53+0.137 (+0.040 to +0.234), 8a8b0ffb-2e38-4252-913f-06496c939ff0−0.002 (−0.031 to +0.027)
knight_play_value, quarter blendconfirmation+0.105 (−0.004 to +0.214), 8263d242-7450-4e05-a2c3-7483c2d527a2+0.105 (−0.009 to +0.220), e3fc4a9c-0e3b-4700-9dfe-1bd58c317c26+0.000 (−0.028 to +0.028)
knight_play_value, quarter blendpopulation+0.043 (−0.038 to +0.124), a6afd7a8-248a-4e0f-a4db-b2ca75d8ee31+0.035 (−0.052 to +0.122), 5046ec2d-3835-42f5-9fc0-3ea829bbafeb+0.004 (−0.026 to +0.034)
knight_play_value, full blenddevelopment+0.125 (+0.013 to +0.237), b41de3cf-c42a-4b0a-b37d-065d7c1e1c1b−0.094 (−0.207 to +0.019), 41c8da11-f31a-440a-a1ee-61737f4439c7+0.109 (+0.055 to +0.164)

Half B's contrast is slot 0 (control) minus slot 1 (candidate), so a real effect makes the two halves point in opposite directions. Both buy_tempo halves point the same way on the development and confirmation boards: the candidate loses more in slot 1 than the seating term explains, and the corrected sign is negative in all three stages. The seating terms themselves (+0.08 to +0.14 on L1, near zero on L2 with a third search seated between) match the earlier null pairs on these boards.

What the switches did

Per-game means for the candidate against the control across each pair. The mechanisms all fire; the first two simply fire against the tables' own timing.

Arm (stage)Cards boughtKnights playedArmy awardsCities builtDecision ms
buy_tempo (development)3.19 / 3.981.67 / 2.060.27 / 0.381.39 / 1.20458 / 673
buy_tempo (confirmation)3.18 / 3.921.67 / 1.910.28 / 0.341.45 / 1.22508 / 749
buy_tempo (population)3.35 / 4.181.74 / 2.060.26 / 0.321.27 / 1.24414 / 581
hold_knight (development)3.86 / 3.941.84 / 2.050.31 / 0.381.20 / 1.24705 / 717
hold_knight (confirmation)3.83 / 3.901.78 / 1.960.30 / 0.351.25 / 1.24722 / 739
hold_knight (population)3.95 / 4.191.79 / 2.120.24 / 0.371.23 / 1.22590 / 596
knight_play_value (quarter blend, all stages pooled means)4.06 / 4.002.06 / 2.000.35 / 0.351.50 / 1.49950 / 946
knight_play_value (full blend, development)4.00 / 3.922.08 / 1.990.36 / 0.301.97 / 1.96938 / 933

Under the hand-written leaf the tempo rule halved purchases (1.54 against 2.55 per game) and advanced the first city by a turn without winning. Under the tables it cuts only a fifth of the purchases (about 3.2 against 3.9 to 4.2), still builds the extra cities (1.4 against 1.2), and now loses. The tables already spend on cards when the board justifies it, so a rigid "no build within a turn" gate mostly removes good buys. Holding knights forgoes about 0.2 plays and a fifth of the army awards per game under the tables, half the bite it had under the hand leaf, and loses more at the population table, where a rival search punishes the held robber. The knight schedule changes almost no count on the quarter blend: the tables' own knight valuation leaves the shaped schedule nothing to correct, which is why its intervals are the tightest of the three arms. On the full blend the same schedule is worth +0.109, which reads as the hand-written flat knight weight being wrong at full strength, not as a gain available to any configuration worth running.

Decision cost moves only where the candidate list changes: buy_tempo prunes the purchase branch and decides about 30% faster than the control, while hold_knight and knight_play_value cost the same as their controls. The blends cost about 900 to 1000 ms per decision at depth 2 under load, against 450 to 750 ms for the tables alone and about 200 to 250 ms for the hand-written leaf, because the hand terms are evaluated at every leaf on top of the tables.

Comparison with the hand-leaf results

Under the hand-written leaf (the earlier study, same development boards) the single-seating contrasts were +0.043 (buy_tempo, boards 0-63) and +0.098 and +0.066 (hold_knight), all inside the seating term, and the knight_play_value swapped pair corrected to +0.004 (−0.061 to +0.069). The sign pattern flips under the tables: what was harmless under a leaf that under-bought cards is harmful under a leaf that buys them well. The tables learned card timing from self-play, and the hand-written timing rules, tuned by intuition, now fight the leaf instead of patching it.

What could still be wrong

All evidence here is engine-arena development tier at depth 2 on deterministic boards, two lineups, and one frozen table file; nothing is protocol-tier. The population lineup still seats an ETA builder, so "opponents that trade and block" means one more search, not three humans. The buy_tempo estimate mixes a refuted development stage with two inconclusive ones; the pooled interval excludes zero but pooling across stages is after the fact. The quarter blend itself is a screen reading (+0.074, interval +0.011 to +0.138, unregistered) chosen by a parallel study; if a different blend becomes the default, the knight_play_value reading scales with the hand weight and should be re-read at that weight. The full-blend pair is one stage on one lineup, so +0.109 is the thinnest reading here. Decision times come from a machine shared at high load and compare candidate with control within each run, which cancels the load but not the noise.

What this means for the browser

The browser build runs the hand-written leaf at depth 3 under a one-second budget, and none of the three switches earns a place there. The bar is a pooled effect of at least +0.03 wins per game with the interval excluding zero and decision time within 25% of the control's; no arm comes close on either leaf. buy_tempo was zero under the hand leaf and loses under the tables (−0.043 pooled). It does decide about 30% faster than its control in both leaves, but a cheaper search is only useful when the strength holds: here the strength is absent under the hand leaf and negative under the tables, so the saving buys nothing. hold_knight loses under the tables (−0.035 pooled) and showed nothing under the hand leaf. knight_play_value is zero under the hand leaf and under the quarter blend; its only positive reading sits on the full blend, a configuration too weak to ship. All three switches stay off by default, in the engine and in the browser.

Agent notes

Candidate seats add one switch to the tables baseline v2:{"depth":2,"leaf":{"tables":"/home/keshav/settlers/research/artifacts/ntuple/hex-portfolio-main.bin"}}; the knight_play_value arm and its control replace the leaf object with {"tables": ..., "hand": 0.25} (quarter blend, primary evidence) or "hand": 1.0 (full blend, development stage only). Registrations: b6b12fdd/7385ffd6, 750b7ebc/be7df6ce, afb09868/5e75db43 (buy_tempo); b7467b5e/75774a6d, 4d8b27cc/56cb999e, 6fe05d29/b21f1bdb (hold_knight); 80e5bceb/858e2a4d, c9a94c3d/941be1a7, 8e35acce/274d88a0 (knight_play_value, quarter blend); 89dd235d/c9ba1dac (knight_play_value, full blend, development). The full-blend registrations for the confirmation and population stages (00d8c63d, e5715fe9, e4ddbc81, 2d22c801) were withdrawn before running when the blend screen measured hand 1.0 at −0.207 against the tables alone. Every pair ran on server branch research/timing-tables, which fixes the blended leaf to honor knight_play_value (test in crates/expectimax/tests/development.rs); the two candidate rules are unchanged engine behavior. The extension stage (boards 864-927) was registered for no arm because no pooled L1 effect was positive. All twenty runs are archived under artifacts/timing-tables/ with receipts in records/artifacts/. python3 analysis/timing_tables.py recomputes every number, the pooled estimates, and the plot asset from the retained runs; analysis/seating_pair.py gives the same effect for any one pair from its two games.jsonl files. Mean decision times compare candidate and control within each run; absolute values carry the shared machine's load.