You don't invent the answer. You converge on it.

Notes from building and running my own personal AI assistant runtime. Part 4.

When I could not see a direction, I used to think harder. I would sit down, list options, weigh their pros and cons, and choose one.

I do something different now. When direction is unclear, the problem is not insufficient thought. It is insufficient research. I research, decide from the results whether and where to dig further, then dig again on the basis of that decision. Keep repeating that, and the answer usually converges on one direction.

I formalized this as a workflow. This post covers its design and what happened when I ran it in practice for the first time.

The structure of one round

A round runs through four stages. Fan-out → synthesis → red-team → judgment.

Fan-out: open the lanes at once

I open four kinds of lanes in the same round. This is not a choice between going broad and going deep.

There is a reason the perspective lane is separate. Breadth comes from the number of perspectives, not the number of queries. Stanford’s STORM research reports that perspective-based question generation improved coverage by 10 percentage points and organization by 25 percentage points. Establishing five perspectives on the same topic works better than firing twenty queries at it.

I isolate context for each lane. LangChain measured this while building Open Deep Research: when several topics are placed in one context, each is explored less deeply. So I run lanes in separate sessions or subagents.

And lanes write their outputs directly to files. Passing summaries verbally turns research into a telephone game and evidence leaks away.

Scale breadth to query value

This matters because depth is a budget, not an act of will. Anthropic’s multi-agent research report found that token usage alone explained 80% of the variance in BrowseComp performance. Fan-out, by contrast, consumes roughly 15 times the tokens of chat. Opening ten lanes to verify a single deadline is a failure.

Synthesis: classify into four boxes

I separate findings into facts / patterns / hypotheses / unknowns. A finding needs two conditions to become an insight: at least two independent sources, and a real effect on our decision.

The second condition matters. A discovery that is interesting but changes no decision is trivia, not insight.

Red-team: at least three in every round

This is not decoration added at the end of a round. It is a lane opened from the start.

• Run disconfirmation queries separately. Confirmation queries return confirmation.
• Give the opposing position the strongest possible defense in one paragraph. A straw man does not count.
• Check for an echo chamber. Are sources copying one another? Are they reheating the same press release? Agreement is not convergence.
• Killer questions: Who has already done this, and why did they fail? If this were easy, why has no one done it yet? Is what we think gives us an advantage actually something everyone already knows?

A formal one-liner such as “Risk: lack of time” is not red-teaming. A counterargument has to bring facts.

Judgment: the axis of this loop

Before closing a round, answer explicitly. Should I dig further or stop? If I dig further, where? What is the provisional direction now, and did this round change or reinforce it?

If I move on to another search without recording a judgment, rounds accumulate and direction never appears. That is the difference between collecting materials and doing research.

Rules for handling evidence

Source tiers. Primary sources are original announcements, application screens, actual data, repository code, laws, and official API documentation. They can support a decision. Secondary sources are articles, reviews, and summary sites. They point toward primary sources and cannot stand alone as evidence. Tertiary sources are search snippets, my own memory, and common knowledge. These are not sources. They are things to verify.

When a summary and an original conflict, the original wins. I have seen a weighting table absent from a summary announcement appear in the original PDF, with its highest-scoring item not even mentioned in the summary.

Assessments have four values. confirmed / likely / unverified / disproved. Two sentences belong here. Finding a candidate is not confirmation. And the absence of search results is not evidence of absence.

Verification economics. I do not verify every claim to the same depth. I allocate depth by the cost of an error multiplied by the cost of verification. A fact whose error would invalidate the submission itself gets two checks against primary originals. A fact such as a color gets one check, with only the residual risk recorded.

When to stop

There are three stopping conditions, and I stop only when all three are met.

1. A new round does not change the decision. The direction remains the same even when new sources appear.
2. The red-team lane’s strongest objection has a fact-based response. “We will manage it as a risk” is evasion, not a response.
3. The remaining unknowns are things research cannot eliminate and only execution can resolve.

The number of rounds is not a plan. It is an output of judgment. I do not decide in advance to “run three rounds.”

There is also failure in the opposite direction. When I encounter a non-convergence signal, I change the question instead of adding rounds. If the direction flips every round, the problem definition is wrong. If sources increase but no new facts appear, it is an echo chamber. If the conclusion is unchanged but all its support is secondary, I have not yet looked at primary sources.

There is a hard stop too. I do not set a number of rounds, but I set a time budget. For deadline-driven work, I recommend one-third of the available time. When that is exceeded, I record the best hypothesis and unresolved risks, then begin. Research is not an alibi for postponing the start.

The trap of parallelism

Fan-out is not free. Cognition measured that parallel agents cannot see one another’s implicit decisions, so they break at the assembly stage.

That produces one rule. Fan-out ends with investigation. Writing and final judgment stay single-threaded. If report sections are written in parallel, the result fragments. LangChain encountered the same problem and withdrew that design.

What happened when I ran it in practice for the first time

I ran this loop with five lanes for the first time. The topic was “a writing method for turning my work records into long-form Korean content.”

Two things differed from what I expected.

First, one lane quietly died. I commissioned five, but only four files appeared. There was no failure message. Had I not noticed the death, that gap would have remained permanently unfilled. So I added a rule. Before synthesis, verify in the file system that every commissioned file actually exists. Recommission a dead lane in a recovery wave, but include a list of what has already converged in that brief to prevent duplicate research.

Second, and more importantly, the original-text analysis lane disproved its own sample.

Its assignment was to “collect Korean long-form pieces that had actually spread widely and analyze their structure.” It read and analyzed eight pieces, then wrote this at the end: public indicators confirmed that only two of the eight had spread widely; the remaining pieces had 33 to 43 public views. So it would use them as writing samples, but would not call them success cases.

That self-disconfirmation was the round’s largest discovery. Actual metrics from eight well-structured pieces demonstrated that good structure does not spread without distribution.

That led to another rule. Assess sample eligibility before analysis. For a task that says “collect N examples of X and extract patterns,” first determine whether the collected examples really are X. Otherwise, I end up calling the common traits of unrelated things a pattern.

Then the loop attacked a premise of my goal

The red-team lane attacked not the conclusion, but the goal.

The goal I received was “publish as much content as possible.” The red-team lane brought back three pieces of evidence: Ahrefs analyzed roughly 14 billion pages in its index and found that 96.55% had zero search traffic; Google explicitly identifies mass production as a warning sign in its quality self-assessment; and a personal retrospective described how a goal of publishing once a week led to declining quality and abandonment.

There are two possible responses here. Lower the goal, or redefine the unit of the goal.

Lowering the goal is not the answer. The goal should remain a constant. Instead, I changed the unit of “more.”

• ❌ A publication-count quota, filled by inventing subjects
• ✅ Recover every piece of work that already has measured evidence into writing

My repository contains 17 days of campaign logs, three measured rankings, submission histories, and spending analyses that remain unwritten. This is not mass production. It is inventory recovery. And once I add a gate that says I do not write posts without measured numbers, inventing a post when subjects run out becomes physically impossible.

I also record the redefined objective in the judgment box. I keep the original instruction next to it verbatim so I can compare them. That makes it possible to verify later that I did not quietly substitute my own goal.

The question that stays

The hardest part of designing this loop was the stopping condition. Research can continue forever, and it always feels slightly unsafe to stop because more work will always produce something.

So I made the stopping condition not “the feeling that I know enough,” but “a new round does not change the decision.” Feelings cannot be measured. Decision changes can.

The question to ask is this. Did the last round change my decision? If it did not, one of two things is true: the research converged, or the question was wrong. When sources keep increasing but the decision does not change, I should suspect the latter first.