The Half-Life of an Instruction

Two French examples. That is all it took.

I told DeepSeek’s larger model, V4 Pro, to translate every question into Russian. One earlier turn in which the assistant had answered in French was enough to make it slip 17 times in 100. Two, and it answered in French 90 times out of 90. Switch on thinking mode - the setting where the model writes out its reasoning before it answers - and the same model followed the instruction between 89 and 100 times in 100 at every number of turns I tested, up to 50.

That’s one result from a benchmark I ran this week, after a cluster of publications made the same point from different angles. On 12 October 2026 Anthropic’s red team described Memento, a harness - the program that runs the agent - that periodically throws the agent away and starts a fresh one from notes, because agents fixate. On 14 October a Berkeley group published Scaling Discovery Requires Forgetting. On 24 October Drew Breunig posted his AGNTCon talk on the state of the field, and his advice to anyone complaining that “Claude’s been nerfed” was three words: delete your context.

The short version: the contest used to be how much a model could hold. The hard problem now is what to throw out, when to do it and what it costs. I rebuilt three of the experiments behind these claims as n8n workflows and ran them against DeepSeek. Every claim below links to its source. Where the source is a preprint, an anecdote or my own small run, I say so.

Two Examples Is All It Takes

The experiment comes from Camassa and Shiller, Do as I Say, Not as I Do (a preprint, presented as a workshop poster at COLM 2026). The first user message carries an instruction: translate every question into Russian. Then come N earlier turns in which the assistant translated each question into French. Then a new question. Does the model follow the instruction, or the pattern?

Across 13 models, in the task-based conditions, the pattern wins eventually. Claude Opus and Sonnet hold out longest. GPT-5.2 without reasoning follows the instruction between 47% and 66% of the time at every N; with reasoning (OpenAI’s name for thinking mode) switched on, it averages 88% and never drops to half within 50 turns.

I ran the Russian/French condition on two DeepSeek models, with thinking on and off, 30 questions that never appeared in the examples, three different example orderings, temperature 0 (the setting that makes the output as repeatable as it gets). Scoring is a Cyrillic-letter count in code (a Russian answer is mostly Cyrillic, a French one has none); no second model judges the first.

xychart-beta
  title "Instruction followed vs number of contradicting turns (DeepSeek, Russian/French)"
  x-axis "contradicting turns (N)" [1, 2, 3, 4, 6, 8, 11, 16, 22, 31, 43, 50]
  y-axis "share answered in Russian" 0 --> 1
  line "Flash, no thinking" [1, 0.411, 0.167, 0.122, 0.044, 0.056, 0.011, 0, 0, 0, 0, 0]
  line "Flash, thinking on" [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
  line "V4 Pro, no thinking" [0.833, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
  line "V4 Pro, thinking on" [1, 1, 0.978, 0.989, 1, 0.944, 0.956, 0.889, 0.933, 0.9, 0.944, 0.922]

90 trials per point, 4,320 calls in all.

Call the first tested N at which adherence is at or below 50% the instruction half-life. The paper reports the same statistic as N50%. One figure per model, and the one that matters if an instruction is doing a job for you.

Two things in that table surprised me. The bigger DeepSeek model fails faster than the small one when thinking is off. And thinking changes the result from the first turn on. The paper’s GPT-5.2 numbers point the same way (the authors say reasoning lengthens the half-life rather than removing it), and it’s the cheapest context fix I’ve seen: a request parameter.

In May I called the all-caps system prompt negotiation. There’s now a number for how fast you lose it: two turns.

Read the paper, not the slide. Breunig’s talk is a good map of the field, and I lean on it below. But checking his slides against the sources turned up the pattern this post is about: a confident summary standing in for the record.

None of this breaks his argument. All of it is what happens when the summary is closer to hand than the source. Hold that thought.

It’s Not Only Length. It’s Distance.

Under Breunig’s post on Hacker News, Benoit Essiambre argued that “context too long” is a scapegoat diagnosis. Length matters, but so does layout: a model does best when the things that belong together sit close together. The idea comes from psycholinguistics. Two orderings of the same sentence; the arcs connect words that depend on each other, the numbers are the distances.

Humans prefer the first. Futrell, Mahowald and Gibson showed in 37 languages that real sentences keep dependencies shorter than chance, and Gibson’s Dependency Locality Theory explains why: distance costs memory. The slide deck was closer to hand than the papers. Same mechanism, different reader.

For a transformer this is an analogy; what is measured is cruder. Lost in the Middle (TACL 2024, on 2023-era models) found retrieval works best at the start and end of a long context and worst in the middle. And the drift result above is a distance effect in disguise: the instruction sits at the top, the French examples sit next to the question.

Essiambre’s advice follows: restate an early instruction after a long stretch of thinking and discussion, but sparingly, because every reminder pushes something else apart. The paper tried a gentle version, telling the model the earlier turns were hardcoded, and gained 11 points on average. I tested the blunt version: repeat the instruction once, in the same message as the final question, thinking off.

90 trials per cell. One sentence, placed next to the question, does most of what thinking mode does. And thinking mode, it turns out, is the model writing that sentence for itself: in 60 of 60 reasoning traces I sampled (both models, N=8 and N=22) the model restates the Russian instruction before answering, several of them noting that “prior assistant answered French incorrectly”. The same fix, applied by the model instead of by you, at a few hundred extra tokens per call.

The Cache Decides What You Can Forget

Two of Breunig’s six fixes, pruning and summarisation, rewrite history. In the same HN thread, one commenter said they had dropped the idea of pruning because of something Breunig covers in one sentence: the prompt cache - providers keep the computed state of a conversation’s opening stretch, so a request that starts identically is cheaper and faster.

Every provider’s cache works on prefixes - if turns A to E are cached and you delete C, as winwang put it, D was cached as f(f(f(A,B),C),D) and is now f(f(A,B),D). Everything after the cut is recomputed. DeepSeek is stricter still: its cache stores whole units, and a request only hits if it matches a stored unit completely. A+B followed by A+C is a full miss.

block
    columns 7
    h1["Turn 5"] s1["system"] a1["A"] b1["B"] c1["C"] d1["D<br>f(ABC, D)"] e1["E"]
    h2["Turn 6,<br>C pruned"] s2["system"] a2["A"] b2["B"] x2["(gone)"] d2["D<br>f(AB, D)"] e2["E"]
    h3["DeepSeek:<br>A+B was one unit"] s3["system"] a3["A"] c3["C"] space:3
    h4["Legend"] k1["cache hit"] k2["recomputed"] space:4
    classDef hit stroke-width:3px
    classDef miss stroke-dasharray:5 4
    class s1,a1,b1,c1,d1,e1,s2,a2,b2,k1 hit
    class x2,d2,e2,s3,a3,c3,k2 miss

The price gap makes this a budget question. On 26 October 2026 DeepSeek’s off-peak price for Flash was $0.003 per million cached input tokens and $0.15 per million uncached: 50×. Anthropic reads cached tokens at a tenth of the base price on most models; OpenAI caches by default.

So I replayed one scripted 12-turn conversation over a 2,500-token system prompt, three ways, on DeepSeek Flash, and read the cache fields the API returns. A sliding window keeps only the last few messages. Append-only with one compaction replaces the older turns with a model-written summary, once. Append-only with one pruned turn deletes a single message from the middle.

xychart-beta
  title "Prompt cache hit % per turn"
  x-axis "turn" [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
  y-axis "cache hit %" 0 --> 100
  line "A: sliding window" [0, 84, 86, 53, 54, 54, 53, 54, 54, 53, 53, 53]
  line "B: compact once" [0, 84, 86, 86, 89, 60, 85, 89, 90, 90, 90, 91]
  line "C: prune once" [0, 84, 86, 87, 89, 67, 90, 91, 92, 92, 94, 94]

The HN commenter was right and wrong. Pruning does break the cache, but once: strategy C dips to 67% on the turn after the cut and is back at 90% a turn later. The expensive choice is the sliding window. It moves the start of the history every turn, so only the system prompt ever hits, and it cost 54% more than append-and-compact for the same conversation. It also answered worse, because by turn 12 it could only see turn 11.

Two rules fall out of this. Append-only history is a cache strategy as much as a memory strategy. And compact by appending: strategy B’s summary request was itself a new turn on the same prefix, so even the compaction call read 4,608 of its 4,772 tokens from cache. Compact, then cut. Never cut first.

Forget on Purpose

If rewriting history is expensive, the alternative is to stop and start again. Three groups arrived at that in the same month.

Memento. Anthropic’s red team built it because agents on long runs “become fixated on an incorrect idea” and keep pursuing it. Their fix starts a fresh agent instance with the previous one’s workspace and a short set of notes. There is no controlled test in the post. It shows a top lab designing for the problem; it doesn’t measure the cure.

Fork-and-Flush. Microsoft Research ran the same autoresearch agent (one that runs experiments to improve its own score) eight times on one task and watched the runs plateau at different scores, between 68 and 100, each stuck in an idea basin: one approach it keeps polishing instead of trying another. Forking into fresh chat contexts that inherit the workspace improved the normalised score by 66% over a single run across 13 tasks, at equal compute. The detail I’d underline: periodic compaction did not get agents out of the basin. Summarising the past preserves the fixation. Preprint.

Scaling Discovery Requires Forgetting. The Berkeley group went further and threw away everything but the best candidate, sessions and history included. After each forget, median edits jumped from 6 lines to 40. Gains run up to 39% over a single agent on their four tasks; the mean over replicates is smaller and noisier. Also a preprint.

A JetBrains team adds a cheaper data point in a NeurIPS workshop paper: on SWE-bench Verified (a standard set of real GitHub bug fixes), simply masking old tool output halved cost and matched LLM summarisation. Forgetting doesn’t have to be clever.

I tried the pattern myself on a toy task: an agent improving a Golomb ruler (ten marks, scored by how many distinct pairwise distances they produce, 45 at most) over eight rounds, with and without a reset every three rounds plus a short handoff note. The mechanism worked. The result did not: 36 against 38 for the baseline in one run, 37 against 37 in the other. The baseline did drift into one-mark nudges, as the papers describe, but at eight rounds and a single run that’s an illustration.

Who holds the eraser is the open question. Victor Taelin’s UniiChat design puts forgetting in the harness: a cheap model folds the chat into summaries that only grow at the end, so the cache holds (his 98.6% hit figure is simulated). A UW and Meta group’s Context Language Models, another preprint, put it in the weights: train the model to edit its own context as a file. Breunig’s closing question was exactly this: what to make deterministic, and what to leave to the model.

My answer, after a week of running these: put what must be true in code, let the model do the re-reading, and let the cache decide how often you can afford to change your mind.

Where This Breaks

One vendor. The drift benchmark ran on two DeepSeek models in one condition at temperature 0. The paper’s 13 models show the same shape with longer half-lives, but I haven’t shown that thinking or a reminder rescues other models.

n8n is a scheduler, not a lab. The first attempt at the full 4,320-call grid ran the n8n process out of memory; the published template splits the work into child runs for that reason.

The cache numbers are DeepSeek’s. Its whole-unit rule is stricter than Anthropic’s or OpenAI’s prefix matching, so the ratios will differ elsewhere; the direction follows from how every prefix cache works.

The reset experiment showed nothing in two runs. A null result at small scale is the honest finding, and the workflow is the useful part.

And the dependency-length argument is an analogy from human language. The reminder test shows that distance matters; it doesn’t show why.

What to Do This Week

1. Measure your instruction half-life. Take any instruction you’re relying on as a control - a tone rule, a disclosure rule, a “never do X” - and run the drift test against your model. If it’s under five, the instruction is a suggestion, not a control. Put the rule in code.
2. Read your cache fields. cached_tokens, prompt_cache_hit_tokens, cache_read_input_tokens. One number per call tells you what your memory design costs.
3. Replace sliding-window memory. Append, compact by appending once the history is long, then cut. If you build in n8n, Simple Memory is the expensive default.
4. Reset long runs on purpose. Fresh session, handoff note, artefacts kept - every N steps, or when the agent starts making one-line edits.
5. Keep instructions next to what they govern. One reminder next to the question beat seven in all caps at the top. If you can’t place it, switch thinking on and let the model write the reminder.
6. Load only the tools the task needs. The team Breunig describes was loading 600 skills at startup. The model wasn’t confused. It was buried.

The fourth template, above, is the agent that does all three at once: tools loaded per request and enforced, notes offloaded to a table, research quarantined in a sub-agent that returns an 80-word report.

TL;DR

• Two contradicting examples were enough for DeepSeek V4 Pro to abandon a standing instruction, answering in French 90 times out of 90; with thinking on it held at 89-100% through 50 turns. Claude Opus 4.6’s instruction half-life is 31 turns, Gemma-3 12B’s 3 (Camassa and Shiller, preprint).
• One reminder next to the question restored 99-100% on Flash and 79-100% on V4 Pro. All 60 sampled thinking traces restate the instruction: thinking is the model writing its own reminder.
• Instructions decay; code doesn’t. Rules that must hold go in code.
• Pruning breaks the prompt cache once; a sliding window breaks it every turn. On DeepSeek the sliding window cost 54% more than append-and-compact over 12 turns, at a 50× price gap between cached and uncached tokens. Compact by appending, then cut.
• Forgetting is now a designed feature: Anthropic’s Memento, Microsoft’s Fork-and-Flush, Berkeley’s forgetting agents. Compaction alone didn’t escape the plateau.
• Check the slide against the paper. The forgetting chart in Breunig’s talk is credited to the wrong paper.
• Four n8n templates: instruction drift, memory vs cache, reset with handoff, context-managed agent.

Two years ago the question was how much a model can remember. The better question is what it can afford to forget.

All runs were made on 26 October 2026 against the DeepSeek API at the prices quoted, from n8n 2.44.7 on a 1 GB container: 5,100 scored calls, 90 trials per cell, temperature 0, scoring in code. Total model spend was under €2. The workflow JSON files above are the ones that produced the numbers; n8n.io template pages will replace the links once approved. The benchmark ran out of DeepSeek credit halfway through, a small prepaid balance. I topped up. The second half is real too. Research and drafting were done with AI assistance; the experiments, the reading of the sources and the opinions are human.