I built a bot that collects draws

This is a postmortem of a finished competition, the second in a series that records three losses in sequence.

My final result in the NYPC 2026 Master preliminary was 447th out of 937 teams. The top twenty advanced to the finals, and the cutoff was around a performance rating of 2300. That is the top 2%. My final rating oscillated between 1830 and 1910.

Looking only at the numbers, it appears that I simply lacked skill. The trajectory shows something else.

I was at 1930 on July 1, and between 1830 and 1910 on the deadline eight days later. I built twenty-six bots during those eight days. The net gain was effectively zero. My rank fell from 45th to 447th not because I got worse, but because the field grew from 539 to 937 teams while I stood still.

This is a record of stagnation. The three causes of that stagnation had all been decided on the first day of the competition.

Cause 1: An easy metric hijacked the real objective

This competition had two kinds of scoring. The eight sample problems returned results immediately and without limit whenever I submitted. The performance rating ran only three rounds per day, through automatic matches against other entrants.

I got a perfect score on all eight sample problems. 8.0 / 8. But advancement to the finals was decided by performance rating, not the samples.

I already knew this on the first night. I wrote in my notes, “The real objective function is participant rank.” Then I spent six of the ten days grinding samples.

In retrospect, the reason is obvious. Samples provide immediate, unlimited feedback. The performance rating runs three times a day and swings by ±100 depending on the opponent. People are drawn to metrics that respond quickly, even when those metrics are not the objective.

Goodhart's law is commonly quoted as “When a measure becomes a target, it ceases to be a good measure.” My case was different. The metric did not break. I did not create a broken metric; I was drawn to a perfectly sound metric for the wrong reason. This was not a measurement problem. It was an attention problem.

As a rule, I ask myself once each cycle: “If I conquer this metric, have I achieved the objective?” If the answer is no, the metric is a calibration tool, not the objective. A tool belongs in its place.

Cause 2: I built a bot that would not lose, and that was the problem

The bot architecture evolved like this.

economy-turtle  →  turn losses into draws  →  never lose  →  no-regression gate

Every stage was a reasonable improvement on the last. I designed a resource-saving strategy, turned losing games into draws, made avoiding losses the goal, then added a gate that rejected new versions if they were worse than the current champion.

The result was a draw collector. The later versions went 7 wins, 7 draws, and 6 losses.

I also dissected replays from the leading bots. Bots rated 2150 or higher seized the board as first player between turns 25 and 70, then finished the game before the first turning point arrived. Their average was 50.7 turns. My bot was built by stacking reactive layers that watched the opponent's moves and answered them. That structure cannot decide a game before turn 50. To react, the opponent has to move first.

I started with a core with a low ceiling, then spent eight days adding twenty-six layers on top of it.

Cause 3: I optimized a step function as though it were continuous

This was the largest misjudgment. It is now something I verify on the first day of every new competition.

The reward in this competition is a step function. The top twenty teams receive everything, and 21st receives zero. There is no difference in reward between 21st and 447th. Both receive zero.

Yet every gate in my bot optimized for rank preservation: turning losses into draws, rejecting anything worse than the current champion, and preventing regressions. All of them allocated resources to rules for “not losing my current position.”

That is correct when I am defending a place above the cutoff. Below the cutoff, the opposite is true. Below the cutoff, variance is an asset, not a cost.

In expected-value terms:

hold position:   100% × 0        = 0
rebuild:          15% × everything > 0

My 1900 score could not reach the 2300 cutoff. If I successfully maintained that state safely, I would receive a guaranteed zero. A redesign with a 15% chance of success was better. The arithmetic is this simple, yet I spent eight days reducing variance.

After the competition, someone told me, “If you were going to finish outside 400th, you should at least have gambled.” That was accurate. There is another point to understand here: no one decided to be conservative. Conservatism was embedded in the gate machine I had built. The rule to discard anything worse than the champion automatically killed all exploration.

A rebuild from scratch did exist. I executed it after two days

On the eighth day, I abandoned the reactive structure and built a bot from scratch. It had a completely different core built to control tempo.

That bot received two iterations before I discarded it. The basis for the decision was its 0 wins, 2 draws, and 38 losses in head-to-head games against the current champion.

That judgment was wrong for two reasons.

First, the evaluation axis was wrong. A bot rebuilt from scratch will of course lose against a mature champion. The proper way to evaluate it was to see how closely it matched the tempo curves in leading replays. I needed to measure distance from the target corpus, but I measured distance from my current champion.

Second, the budget was wrong. At the pace that produced twenty-six bots in eight days, splitting off on day six would have allowed eight to twelve iterations for the rebuilt bot. Two did not even meet the minimum needed for a judgment.

So I do not write in the retrospective that “a rebuild was impossible.” I write, “I did not choose it.” Feasibility and choice are different questions. Mixing them makes me repeat the same judgment next time.

Four rules I added for the next competition

1. Classify the reward function on the first day. I establish from the rules whether it is a step function or continuous. If it is a step function and I am below the cutoff, I prohibit rank-preservation strategies.
2. Exploration lanes are exempt from gates. Champion-protection gates apply only to the champion lane. A rebuild lane assumes early regressions. I evaluate it by alignment with the target corpus rather than matches against its own champion, and I guarantee a minimum number of iterations before judging it.
3. Put a midpoint checkpoint on the calendar at 50% of the competition period. I calculate the gap between the current architecture's expected endpoint and the cutoff, then explicitly decide “layers or rebuild.”
4. Do not measure deltas smaller than the feedback bandwidth. In an environment where matching noise is ±100, repeating a change with an expected gain of ±20 twenty-six times makes all twenty-six measurements meaningless.

The question that stays

I worked hard for eight days. Twenty-six bots were not the product of laziness. Every day I built something, connected it, and measured it. Yet the net gain from those eight days was zero.

Diligence is an asset only when the direction is right. The first signal that the direction is wrong usually appears in the ratio of “results to effort.” I watched that ratio every day and still did the same thing for several more days.

This is what I need to ask next. How much did the past three days of work move the target metric? If that value is smaller than measurement noise, am I working, or merely busy?