The Update Protocol

The structured method for moving belief when disconfirmation arrives. Pre-committed conditions, written down before pressure, so that they can still hold when motivated reasoning would otherwise win.

Mechanism

Knowing you should change your mind and actually changing it are separated by everything in your psychology. Inside Revising Beliefs Under Evidence, the Update Protocol is the structured method that closes that gap: pre-committing to specific conditions under which you will revise a belief, recording those conditions before encountering the evidence, and following through when the conditions are met.

Belief revision happens under pressure, in real time, with identity and social cost attached. Knowing about cognitive bias and wanting truth do not, on their own, produce revision. These reactions feel like clarity from the inside while the position stays exactly where it was. The protocol exists because the gap between understanding and behavior does not close on willpower alone.

Three mechanisms make the protocol more than common sense.

First, the pre-commitment. You decide what would count against your belief before you see the result, while your judgment is not yet under pressure. Once evidence threatens a position you care about, reasons to dismiss it come easily: the evidence can feel weaker than it is, the methodology can seem flawed, the source can appear biased. These reactions are fast and convincing, and from the inside they feel like judgment. The written conditions give you a fixed point to measure them against. When a condition you set has been met, the question is no longer only "Is this evidence strong enough to change my mind?" It becomes "Would I accept this objection if the result had gone my way?"

Second, the written record. Writing it down is the mechanism, not a nice-to-have. Verbal commitments to openness evaporate under pressure; memory edits itself to protect existing beliefs. A written statement ("I will revise my position on X if I observe Y") creates a fixed point you cannot quietly rewrite, and creates accountability to yourself — and, if shared, to others.

Third, graduated updating. The protocol is not binary. It asks: what evidence would move you from high confidence to moderate confidence? From moderate to uncertain? From uncertain to "I was probably wrong"? The false choice between total commitment and total abandonment is one of the reasons people resist updating in the first place. If the only options are "I was right all along" and "Everything I believed was wrong," most people will choose the first regardless of the evidence. The protocol creates a middle path: honest, incremental adjustment.

One more distinction worth making. "Being open-minded" is a self-description; running the Update Protocol is a procedure. Procedures operate on structure rather than character, which means they can still work when willpower fails.

Practice

The diagnostic question is: "What would change my mind about this? Can I write it down, specifically, right now?"

If you cannot answer that question for a belief you hold with conviction, treat it as a warning. The belief may have stopped answering to evidence and become part of who you are. It may also be too vague to test yet, or rest on so many converging reasons that only a pattern of evidence, not a single observation, would move it. Find out which it is before you rely on the belief.

Three practices build the protocol into a working method.

The hardest part is following through when the conditions are met. The evidence will often feel weaker than it looked when you wrote down your conditions, the methodology will seem to need revision, and the threshold will seem to have been set too aggressively. Sometimes one of those objections is right. The test may have been flawed, or evidence may have arrived that you did not anticipate. Then amend the test in the open: keep the original conditions, date the amendment and write down why the change is justified, so that you, and anyone you share the record with, can tell a prediction made in advance from an explanation found afterwards. What the protocol rules out is changing the test quietly. It does not eliminate the resistance; it makes the resistance visible. You wrote down what would change your mind, the conditions were met, and now you either follow through or write down an objection you would also have accepted if the result had gone your way.

In the Wild

A startup team believed their new feature would increase user retention. Before launch, the product manager wrote down a commitment: "If, after eight weeks, 30-day retention among users with the feature is not at least 5 percentage points above the comparison group without it, we kill the feature and move the engineers to other work." Eight weeks later, the gap was 2 points. The team spent three meetings arguing that the sample was too small and that the timing coincided with a holiday. The product manager checked both objections. The sample was large enough to tell a 5-point gain from a 2-point one, and the holiday had fallen on both groups alike. With the objections answered, the commitment held, and they killed the feature. The protocol cost them a project they loved. The result did not show that the feature did nothing. It showed that its gain fell short of the one they had decided in advance was worth the engineers' time.

A person who held strong views on a specific policy question asked themselves: "What would I need to see to change my mind about this?" They struggled to answer. They could not name a single piece of evidence that would shift their position. That inability was the diagnostic. It did not mean the position was wrong. When they looked closer, they found that the position had stopped being a conclusion from evidence and had become a piece of identity. They did not update that day. But they noticed something about how they were holding the belief, and that noticing changed what happened the next time evidence arrived.

A researcher hypothesized that a compound would show a meaningful effect in a specific assay. She defined meaningful in advance: an effect size of at least 0.3, the size that would justify the next round of development. Before running the experiment, she wrote down the analysis, a sample size large enough to tell an effect of that size from a much smaller one, and her decision: "If the estimate comes back clearly below 0.3, I stop developing this compound for this use and move to the alternative." The estimate came back at 0.22, and even the top of its uncertainty range fell short of 0.3. Her collaborator wanted to lower the bar, run a subgroup analysis, try a different assay. She pointed to the pre-registration and kept the decision. The collaborator's ideas went into the record as new questions, marked as ideas that came after the result. The result did not prove the compound had no effect, and it did not prove the alternative right. It showed that the effect was smaller than the one worth building on.

The next time you find yourself certain about a belief you are relying on to act, try this: write down what would change your mind. Be specific enough that a stranger could check whether your conditions were met. Then keep that piece of paper where you will see it again. When the evidence arrives, and it will, you will discover whether you were practicing honesty or just describing yourself as honest.

Lineage

The Codex did not invent the Update Protocol. It assembled and framed a practice that others built over decades.

Karl Popper's falsificationism, developed in Logik der Forschung (1934; in English as The Logic of Scientific Discovery, 1959), proposed falsifiability as the mark of a scientific theory: a theory belongs to science only if it says what observations would contradict it. Popper offered this as a way to separate science from other kinds of claims, not as a test of whether a claim has meaning. A belief that no evidence could contradict is unfalsifiable, not strong. The Update Protocol takes this out of the philosophy of science and applies it to personal belief: if you cannot say what would change your mind about an empirical claim, ask whether the belief still answers to evidence at all. Popper remains the starting point for the epistemology underneath the protocol.

Eliezer Yudkowsky and the rationalist community on Less Wrong (beginning around 2007) translated Popper's principle into personal practice. Yudkowsky's framing of "making beliefs pay rent" captured the operational point: beliefs should generate predictions, and when predictions fail, the beliefs should update. The Center for Applied Rationality (CFAR), founded in 2012, developed "Trigger Action Plans," a formalized structure for pre-committing to specific responses triggered by specific conditions. The Update Protocol adapts this format directly. Yudkowsky's Rationality: From AI to Zombies and CFAR's workshop materials are the primary sources for the practical rationalist framework.

Pre-registration in scientific methodology formalized the same logic at the institutional level. Researchers declare their hypotheses and methods before collecting data, so that a planned test can be told apart from an analysis chosen after the results were known. A pre-registered study can still change course for good reasons, and those reasons go on the record. The Update Protocol applies pre-registration to personal belief. The replication crisis, which intensified through the 2010s, showed at institutional scale what happens when that separation is missing. Flexible analysis, small samples and the pressure to publish positive results let findings into the literature that failed when other researchers repeated the studies, and correcting them was slow, in part because revision carried real costs in careers and publishing.

Philip Tetlock's research, published in Expert Political Judgment (2005) and Superforecasting (2015), and the Good Judgment Project he led with Barbara Mellers and colleagues, supply much of the evidence here. In those forecasting tournaments, accuracy came from several sources together: ability, knowledge, training, teamwork, and the habit of updating often and carefully. The best forecasters treated new information as an occasion to adjust; the worst treated their initial judgment as something to defend. Tetlock's work is the place to go for evidence that update discipline improves forecasting. The Update Protocol, as the Codex assembled it from this lineage, has not itself been tested, and no study shows that it removes bias.

Cross-references

Within the category. Steelmanning can supply a challenge worth assessing: the strongest version of the opposing position, constructed before you respond to it. The Update Protocol keeps your response to that challenge answerable to what you wrote down in advance. Revision can also begin with observations, failed predictions or re-examined reasoning, and it can end with good reason to keep the belief.

Within the Foundation. Scout Mindset is the orientation that helps the protocol's pre-commitments survive contact with motivated reasoning; without the disposition to want accuracy over defense, the conditions written down at calm time tend to get re-interpreted at pressure time. Noticing is the in-moment practice that catches the disconfirmation arrival — the gap that lets the protocol's conditions register as having been met rather than dismissed. Confirmation Bias names the filter the protocol is engineered against: pre-commitment counters the in-moment filtering by setting the test before the filtering starts. Identity Decoupling helps keep the protocol from collapsing when the belief is fused with the self; if updating feels like self-attack, even a strong pre-commitment can crumble. Calibrating Confidence to Evidence carries the static side of evidence accountability that the Update Protocol's dynamic side complements.

Two limitations worth naming. The protocol is strongest for empirical questions where evidence can be specified in advance — well-suited to "What would change my mind about whether this drug is effective?" and ill-suited to "What would change my mind about whether this painting is beautiful?" The domain still covers more territory than most people assume, but it is not universal. The protocol also requires honest initial commitment. Setting deliberately impossible conditions ("I would change my mind if every physicist in the world signed a letter") is performing the protocol without practicing it. If your conditions would embarrass you when read by someone reasonable, they are probably not honest.