{ "name": "Memory Strategy vs Prompt Cache Cost – sliding window vs append-only", "nodes": [ { "id": "c48c397d-2411-4b73-a7c3-04b1b745e813", "name": "Start", "type": "n8n-nodes-base.manualTrigger", "typeVersion": 1, "position": [ 0, 300 ], "parameters": {} }, { "id": "78fb9f3f-f0f6-4f33-96a3-dd44ebeafd2e", "name": "Script", "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ 220, 300 ], "parameters": { "jsCode": "// Fixed script: the same 12 questions are replayed under every memory strategy.\n// Edit the knobs below; everything else is derived from them.\nconst reference = \"FIELD GUIDE: CONTEXT ENGINEERING FOR SMALL AGENT TEAMS (internal handbook, edition 3)\\n\\n1. Purpose of this guide\\nThis handbook collects the working rules our team uses when building assistants and agents on top of large language models. A model only knows what is in its weights and what is in its context window. Everything we control at run time is the context: the system prompt, the conversation history, tool definitions, tool results and retrieved documents. Context engineering is the discipline of deciding what goes into that window, in what order, and for how long it stays there. Prompt wording still matters, but most production failures we have seen came from the wrong material in the window, not from a badly phrased instruction.\\n\\n2. The four ways context goes wrong\\nWe use four labels when we review a failing session. Context poisoning: an early mistake or hallucination is written into the history and the model keeps building on it. Context distraction: the history grows so long that the model starts repeating earlier actions instead of reasoning about the current step. Context confusion: irrelevant material, such as forty tool definitions when three are needed, pulls the model toward the wrong action. Context clash: two parts of the context disagree, for example an early guess and a later tool result, and the model averages them instead of trusting the newer evidence. Each label has its own fix, so name the failure before you change anything.\\n\\n3. Context length budget\\nOur default budget for an interactive assistant is 32,000 tokens of working context, even when the model advertises a window of one million tokens. Quality on retrieval and multi-step reasoning tasks drops long before the advertised limit; we call this context rot. When a session passes 24,000 tokens, the harness raises a warning. When it passes 32,000 tokens, the harness must compact. Long-document analysis jobs may request a larger budget, but they must say so in the job definition and they run on a separate queue.\\n\\n4. Tool loadout\\nAn agent should see between three and seven tools for any given step. If a task needs more, we route first: a small classifier picks the relevant tool group, and only that group is placed in the context. Every tool description is limited to 60 words and must state when NOT to use the tool. Tool results longer than 2,000 tokens are written to the scratch store and replaced in the history by a short pointer plus a three-line summary.\\n\\n5. The scratch store\\nThe scratch store is a key-value area outside the context window. The agent can write notes, plans and large tool outputs to it and read them back on demand. Keys use the pattern session-id/step-number/label. Entries expire after 72 hours. The scratch store is the main defence against context distraction: the plan lives in the store, and only the current step of the plan is in the window.\\n\\n6. Compaction policy\\nCompaction replaces old turns with a summary. Our rules: compact at most once per 32,000 tokens of growth; always keep the last two exchanges verbatim; the summary must list open questions, decisions made, and identifiers (file names, ticket numbers, customer ids) exactly as written. The summary is produced by the same model, with the existing conversation as the prefix and the summary request appended at the end, so that the compaction call itself reads from the prompt cache. A compaction summary may not exceed 400 tokens.\\n\\n7. Prefix caching and cost\\nProviders cache the processed prefix of a request. If the next request starts with exactly the same tokens, those tokens are billed at the cache-hit price, which on our main provider is about one fiftieth of the miss price. The cache works on the prefix only: changing a single token near the start, such as a timestamp in the system prompt, invalidates everything after it. Rules that follow from this: keep the system prompt byte-for-byte stable; never put the current time or a request id at the top; append new material at the end; and treat any edit to the middle of the history as a full re-read of everything after the edit.\\n\\n8. Sliding windows\\nA sliding window keeps only the last N messages. It is simple and it bounds the context size, but it moves the start of the history on every turn. With prefix caching, the first message after the system prompt changes every turn, so the cached prefix is cut right after the system prompt or earlier. We only use sliding windows for stateless helpers where each turn is close to independent, and we set N to at least six messages.\\n\\n9. Pruning\\nPruning removes a specific old turn, for example a failed tool call or an off-topic question. Pruning improves focus, but it has a cost: the cache for everything after the pruned turn is lost for one request. If you prune, prune rarely and prune in batches, ideally at the same moment as a compaction, so that you pay for one cache rebuild instead of several.\\n\\n10. Retrieval\\nRetrieved passages go at the end of the context, directly before the user question, never into the system prompt. Each passage carries its source id and date. We retrieve at most eight passages of up to 300 tokens each. If two passages disagree, the newer one wins and the answer must say that a conflict existed.\\n\\n11. Evaluation\\nEvery change to a prompt, tool description or memory policy is tested against the regression set of 120 recorded sessions before release. We track four numbers per change: task success rate, average tokens per session, cache hit ratio, and cost per successful task. A change that raises success by less than two percentage points but raises cost per successful task by more than 20 percent is rejected by default.\\n\\n12. Incident log, short version\\nIncident 14: an agent copied a wrong customer id from turn 2 into every later ticket; fixed by adding identifiers to the compaction summary checklist. Incident 19: a timestamp was added to the top of the system prompt for debugging, the cache hit ratio fell from 87 percent to 4 percent and the monthly bill tripled within a week. Incident 23: an agent with 46 tools picked the deprecated export tool in 30 percent of runs; fixed by tool routing with a maximum of seven tools. Incident 27: a sliding window of four messages dropped the user's original constraints, and the agent produced a report in the wrong currency.\\n\\n13. Glossary\\nContext window: the tokens the model sees in one request. Prefix cache: provider-side storage of the processed beginning of a request. Cache hit ratio: cached prompt tokens divided by all prompt tokens. Compaction: replacing old turns with a summary. Pruning: deleting selected turns. Tool loadout: the set of tools visible to the model at one step. Context rot: quality loss as the context grows.\\n\\n14. Long-term memory\\nLong-term memory is stored outside the model and loaded on demand. We store three kinds of records: user preferences (language, currency, tone), project facts (repository names, deployment targets, owners), and resolved incidents. Each record has an owner, a source and a review date. Memory records are never loaded wholesale; the harness retrieves at most five records that match the current task and places them after the system prompt as a single block that changes only when the task changes. Records older than 180 days without a review are archived automatically.\\n\\n15. Model routing\\nWe run three model tiers. The small tier handles classification, tool routing and summaries; it is fast and costs about one tenth of the large tier. The medium tier handles ordinary chat and single-step tool use. The large tier is reserved for planning, code changes over 200 lines and any task the medium tier failed twice. Routing decisions are logged with the reason, so we can check every month whether the large tier is used for tasks the medium tier could have handled. A session may move up a tier, but it never moves down in the middle of a task, because a switch between models also discards the prompt cache.\\n\\n16. Multi-agent handoffs\\nWhen one agent hands work to another, it does not forward its whole history. It writes a handoff note of at most 250 tokens with four fields: goal, current state, open questions, and artefacts (with scratch store keys). The receiving agent starts with a fresh context made of its own system prompt and the handoff note. Sub-agents return a result of at most 500 tokens to the parent; long outputs go to the scratch store. This keeps each context small and keeps parent histories append-only.\\n\\n17. Cost reference table (per million tokens, standard hours)\\nSmall tier: cached input 0.003 dollars, uncached input 0.15 dollars, output 0.60 dollars. Medium tier: cached input 0.02 dollars, uncached input 0.66 dollars, output 1.98 dollars. Large tier: cached input 0.30 dollars, uncached input 3.00 dollars, output 15.00 dollars. Off-peak discounts apply to the small and medium tiers only. A session of 20 turns over a stable 3,000-token system prompt costs roughly six times more when the cache is broken on every turn than when every turn hits the cache.\\n\\n18. On-call checklist for a cost spike\\nFirst, compare the cache hit ratio for the last 24 hours with the previous week. Second, diff the system prompt and tool definitions against the last release, looking for anything that changes per request (dates, ids, random ordering of tools). Third, check whether a memory policy was changed from append-only to a sliding window or to frequent pruning. Fourth, check the tier routing log for unexpected use of the large tier. Fifth, write the incident into the log in section 12 with a one-line cause and a one-line fix.\";\n\nconst strategies = {\n A: 'Sliding window (last 4 messages)',\n B: 'Append-only + compact once',\n C: 'Prune one middle turn',\n};\n\n// One fresh nonce per strategy run. It is the very first line of the system prompt,\n// so no strategy (and no earlier run of this workflow) can reuse another one's cache units.\nconst runId = Date.now().toString(36);\nconst nonces = {};\nfor (const s of Object.keys(strategies)) nonces[s] = `${runId}-${s}-${Math.random().toString(36).slice(2, 8)}`;\n\nconst questions = [\n \"What is our default working-context budget, and at what size does the harness warn?\",\n \"List the four ways context goes wrong, one short phrase each.\",\n \"I am building a support bot with 46 tools. What does the guide tell me to do?\",\n \"How long may a tool description be, and what must it state?\",\n \"Why does a sliding window hurt the prompt cache?\",\n \"What happened in incident 19?\",\n \"What are the compaction rules?\",\n \"Where do retrieved passages go, and how many may we use?\",\n \"Compare pruning with compaction in terms of cache cost.\",\n \"What goes into a multi-agent handoff note?\",\n \"On the medium tier, what does 1M input tokens cost uncached versus cached?\",\n \"Recap: which of my earlier questions in this chat were about cost or caching? Give their numbers.\"\n];\n\n// Each user turn carries a pasted log excerpt (~300 tokens), like a tool result in an agent loop.\n// Deterministic (seeded by turn), so all three strategies see byte-identical turns.\nconst attachments = questions.map((_, t) => {\n let x = 7919 * (t + 1);\n const rnd = n => (x = (x * 48271) % 2147483647) % n;\n const lines = [];\n for (let i = 0; i < 12; i++) {\n const prompt = 2000 + rnd(30000), hit = Math.min(prompt, rnd(32000));\n lines.push(`2026-10-${10 + t} ${String(8 + rnd(10)).padStart(2, '0')}:${String(rnd(60)).padStart(2, '0')} ` +\n `session=s${100 + t}-${i} tier=${['small', 'medium', 'large'][rnd(3)]} prompt=${prompt} cache_hit=${hit} ` +\n `tools=${1 + rnd(46)} status=${['ok', 'ok', 'ok', 'retry', 'fail'][rnd(5)]}`);\n }\n return lines.join('\\n');\n});\n\n// Steps: every turn is one call. Strategy B also gets a \"compact?\" check after each turn\n// (it only calls the model when the history is over the threshold, and only once).\nconst steps = [];\nfor (const s of Object.keys(strategies)) {\n questions.forEach((q, i) => {\n steps.push({ strategy: s, kind: 'turn', turn: i + 1 });\n if (s === 'B' && i < questions.length - 1) steps.push({ strategy: s, kind: 'compact', turn: i + 1 });\n });\n}\nsteps.forEach((st, i) => { st.idx = i; }); // position in the run; Build checks it against its run index\n\nreturn [{ json: {\n model: 'deepseek-flash',\n maxTokens: 220, // answers of 3-4 sentences, so the history grows like a real chat\n summaryMaxTokens: 200,\n windowK: 4, // A: messages kept (= Simple Memory \"context window length\" 2)\n compactAfterTokens: 2000, // B: compact when history > 2000 tokens (measured from usage)\n pruneAtTurn: 6, // C: at turn 6 ...\n pruneTurn: 3, // ... delete turn 3 (question + answer) from the history\n // USD per 1M tokens, OFF-PEAK, keyed by model (DeepSeek list prices, 2026-10). Record doubles them\n // during DeepSeek's peak hours. Check https://api-docs.deepseek.com/quick_start/pricing before relying on them.\n price: {\n 'deepseek-flash': { hit: 0.003, miss: 0.15, out: 0.60 },\n 'deepseek-v4-pro': { hit: 0.022, miss: 0.66, out: 1.98 },\n },\n systemPrompt: 'You are the team assistant. Answer from the handbook below in three or four sentences and name the section you used. ' +\n 'Questions in this chat are numbered from 1 in the order asked. Ignore attached logs unless the question needs them.\\n\\n' + reference,\n compactPrompt: 'Summarize our conversation so far in at most 120 words: list each question number with a ' +\n 'one-line answer, keeping numbers and identifiers exactly. Reply with the summary only.',\n strategies, nonces, questions, attachments, steps,\n} }];\n" } }, { "id": "8a006134-8044-4cb5-96ae-7c3d0ef873f9", "name": "Steps", "type": "n8n-nodes-base.splitOut", "typeVersion": 1.1, "position": [ 440, 300 ], "parameters": { "fieldToSplitOut": "steps", "options": {} } }, { "id": "6c4b6118-90b3-4215-abfa-f5413b4d1599", "name": "Loop", "type": "n8n-nodes-base.splitInBatches", "typeVersion": 3, "position": [ 660, 300 ], "parameters": { "batchSize": 1, "options": {} } }, { "id": "03c179d2-c86a-4eef-bcb5-3f3d40256924", "name": "Build", "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ 880, 300 ], "parameters": { "jsCode": "// Builds the request for one step.\n// RUN THE WHOLE WORKFLOW (Execute workflow). The history is rebuilt from the earlier runs of\n// Record, matched by run index, so \"Execute step\" on a single node or a re-run after a\n// mid-run failure cannot rebuild it. The idx checks below stop the run instead of mixing histories.\nconst cfg = $('Script').first().json;\nconst step = $input.first().json;\nif (step.idx !== $runIndex) throw new Error(`Build run ${$runIndex} got step ${step.idx}: execute the whole workflow from Start`);\n\n// Replay this strategy's earlier rows (question text comes from Script, answers from Record).\nconst ask = t => `Question ${t}: ${cfg.questions[t - 1]}\\n\\nAttached log excerpt:\\n${cfg.attachments[t - 1]}`;\nlet history = [], compacted = false, baseline = null, lastPrompt = 0;\nfor (let i = 0; i < $runIndex; i++) {\n const r = $('Record').all(0, i)[0]?.json;\n if (!r || r.idx !== i) throw new Error(`Record run ${i} is missing or out of order: execute the whole workflow from Start`);\n if (r.strategy !== step.strategy || !r.called || r.error) continue; // failed calls add nothing\n if (r.kind === 'turn') {\n history.push({ turn: r.turn, role: 'user', content: ask(r.turn) }, { turn: r.turn, role: 'assistant', content: r.answer });\n lastPrompt = r.prompt_tokens;\n if (baseline === null) baseline = r.prompt_tokens; // turn 1 = system prompt + one question\n } else {\n // B: the summary replaces old turns once; the last exchange stays verbatim. Append-only afterwards.\n history = [{ turn: 0, role: 'user', content: 'Summary of our conversation so far:\\n' + r.answer },\n { turn: 0, role: 'assistant', content: 'Noted.' }, ...history.slice(-2)];\n compacted = true;\n }\n}\n// C: from the prune turn on, turn 3 (question + answer) is gone from the history.\nlet note = '';\nif (step.strategy === 'C' && step.turn >= cfg.pruneAtTurn) {\n history = history.filter(m => m.turn !== cfg.pruneTurn);\n if (step.turn === cfg.pruneAtTurn && step.kind === 'turn') note = `pruned turn ${cfg.pruneTurn} from the middle`;\n}\n\n// The nonce is the very first token run of the prompt: it isolates this strategy's cache.\nconst sys = { role: 'system', content: `Session ${cfg.nonces[step.strategy]}\\n\\n${cfg.systemPrompt}` };\nconst msgs = h => h.map(({ role, content }) => ({ role, content }));\nconst body = (messages, max) => ({ model: cfg.model, thinking: { type: 'disabled' },\n temperature: 0, max_tokens: max, messages });\n\nif (step.kind === 'turn') {\n const past = step.strategy === 'A' ? history.slice(-cfg.windowK) : history;\n if (step.strategy === 'A' && history.length > cfg.windowK) note = `window: last ${cfg.windowK} msgs`;\n return [{ json: { ...step, call: true, note,\n body: body([sys, ...msgs(past), { role: 'user', content: ask(step.turn) }], cfg.maxTokens) } }];\n}\n\n// kind === 'compact' (strategy B only): compact once, when the history is over budget.\nconst historyTokens = lastPrompt - baseline;\nif (!compacted && baseline !== null && historyTokens > cfg.compactAfterTokens) {\n // Same prefix as the conversation + the request at the END, so the summary call reads from cache.\n return [{ json: { ...step, call: true, note: `history ~${historyTokens} tok > ${cfg.compactAfterTokens}: compact`,\n body: body([sys, ...msgs(history), { role: 'user', content: cfg.compactPrompt }], cfg.summaryMaxTokens) } }];\n}\nreturn [{ json: { ...step, call: false } }];\n" } }, { "id": "62a8e331-1765-43f9-ba0e-26c55c2bcac8", "name": "Call?", "type": "n8n-nodes-base.if", "typeVersion": 2.2, "position": [ 1100, 300 ], "parameters": { "conditions": { "options": { "caseSensitive": true, "leftValue": "", "typeValidation": "strict", "version": 2 }, "conditions": [ { "id": "c1", "leftValue": "={{ $json.call }}", "rightValue": "", "operator": { "type": "boolean", "operation": "true", "singleValue": true } } ], "combinator": "and" }, "options": {} } }, { "id": "19fc8014-f522-40f1-a983-14e32c3d2ecd", "name": "DeepSeek", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.5, "position": [ 1320, 300 ], "parameters": { "method": "POST", "url": "https://api.deepseek.com/chat/completions", "authentication": "predefinedCredentialType", "nodeCredentialType": "deepSeekApi", "sendBody": true, "specifyBody": "json", "jsonBody": "={{ JSON.stringify($json.body) }}", "options": { "timeout": 60000 } }, "credentials": { "deepSeekApi": { "name": "DeepSeek account" } }, "retryOnFail": true, "maxTries": 3, "waitBetweenTries": 5000, "onError": "continueRegularOutput" }, { "id": "7ccb6ba2-8097-4512-baaf-1707afeee78d", "name": "Wait 5s", "type": "n8n-nodes-base.wait", "typeVersion": 1.1, "position": [ 1540, 300 ], "parameters": { "amount": 5, "unit": "seconds" }, "webhookId": "f3cbf789-3529-42ee-acaf-80f15cad97f0" }, { "id": "20d71c7c-9df5-49cd-be7b-48c6609f1523", "name": "Record", "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ 1760, 300 ], "parameters": { "jsCode": "// Reads usage from the DeepSeek response and prices it. One small row per step;\n// Build replays these rows to rebuild the history, so no state object is carried around.\nconst cfg = $('Script').first().json;\nconst b = $('Build').first().json;\nconst base = { idx: b.idx, strategy: b.strategy, turn: b.turn, kind: b.kind };\nif (!b.call) return [{ json: { ...base, called: false } }];\n\nconst r = $input.first().json;\nconst u = r.usage;\nconst text = r.choices?.[0]?.message?.content;\nif (!u || typeof text !== 'string') {\n // DeepSeek failed after its retries (the node continues on error): log it, leave the history alone.\n const msg = r.error?.description ?? r.error?.message ?? r.error ?? 'response without usage';\n return [{ json: { ...base, called: true, error: true, note: String(typeof msg === 'string' ? msg : JSON.stringify(msg)).slice(0, 200) } }];\n}\nconst hit = u.prompt_cache_hit_tokens ?? 0, miss = u.prompt_cache_miss_tokens ?? 0, out = u.completion_tokens ?? 0;\n\n// DeepSeek peak hours (weekdays 01-04 and 06-10 UTC) cost 2x. Window and multiplier are DeepSeek's\n// as of 2026-10; check their pricing page.\nconst p = cfg.price[cfg.model];\nif (!p) throw new Error(`No price for model \"${cfg.model}\": add it to price in Script`);\nconst d = new Date((r.created ?? Date.now() / 1000) * 1000);\nconst h = d.getUTCHours(), wd = d.getUTCDay();\nconst peak = wd >= 1 && wd <= 5 && ((h >= 1 && h < 4) || (h >= 6 && h < 10));\nconst cost = (hit * p.hit + miss * p.miss + out * p.out) * (peak ? 2 : 1) / 1e6;\n\nreturn [{ json: { ...base, called: true, error: false, note: b.note,\n messages_sent: b.body.messages.length, prompt_tokens: u.prompt_tokens, hit, miss, out,\n hit_ratio: u.prompt_tokens ? +(hit / u.prompt_tokens).toFixed(3) : 0,\n cost_usd: +cost.toFixed(7), peak, answer: text } }];\n" } }, { "id": "8ef14d23-cbf7-47ed-92d7-034c422ef70f", "name": "Report", "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ 880, 520 ], "parameters": { "jsCode": "// Per-call rows, per-strategy totals, a markdown table and two Pele/Mermaid charts.\nconst cfg = $('Script').first().json;\nconst all = $input.all().map(i => i.json);\nconst rows = all.filter(r => r.called && !r.error); // failed calls are counted, not scored\nconst errors = all.filter(r => r.error);\nconst S = Object.keys(cfg.strategies);\nconst turns = cfg.questions.length;\nconst fmt = (x, n = 4) => Number(x).toFixed(n);\n\nconst totals = S.map(s => {\n const rs = rows.filter(r => r.strategy === s);\n const sum = k => rs.reduce((a, r) => a + (r[k] ?? 0), 0);\n const prompt = sum('prompt_tokens');\n return { strategy: s, name: cfg.strategies[s], calls: rs.length,\n errors: errors.filter(r => r.strategy === s).length, prompt_tokens: prompt,\n hit: sum('hit'), miss: sum('miss'), out: sum('out'),\n hit_ratio: prompt ? +(sum('hit') / prompt).toFixed(3) : 0, cost_usd: +sum('cost_usd').toFixed(6) };\n});\n\n// Per turn: hit ratio of the turn's call; cumulative cost includes B's compaction call.\nconst perTurn = S.map(s => {\n let cum = 0; const ratio = [], cost = [];\n for (let t = 1; t <= turns; t++) {\n const rs = rows.filter(r => r.strategy === s && r.turn === t);\n const turn = rs.find(r => r.kind === 'turn');\n ratio.push(turn ? Math.round(100 * turn.hit_ratio) : 0);\n cum += rs.reduce((a, r) => a + r.cost_usd, 0);\n cost.push(+(cum * 1000).toFixed(3)); // in 1/1000 USD (milli-dollars)\n }\n return { s, ratio, cost };\n});\n\nconst axis = ` x-axis \"turn\" [${Array.from({ length: turns }, (_, i) => i + 1).join(', ')}]`;\nconst label = s => `\"${s} ${cfg.strategies[s]}\"`;\nconst hitChart = ['xychart-beta', ' title \"Prompt cache hit % per turn\"', axis, ' y-axis \"cache hit %\" 0 --> 100',\n ...perTurn.map(p => ` line ${label(p.s)} [${p.ratio.join(', ')}]`)].join('\\n');\nconst maxCost = Math.ceil(Math.max(...perTurn.flatMap(p => p.cost)) * 1.1 * 100) / 100;\nconst costChart = ['xychart-beta', ' title \"Cumulative cost (milli-USD)\"', axis, ` y-axis \"milli-USD\" 0 --> ${maxCost}`,\n ...perTurn.map(p => ` line ${label(p.s)} [${p.cost.join(', ')}]`)].join('\\n');\n\nconst md = [\n '| Strategy | Calls | Failed | Prompt tok | Cache hit | Cache miss | Hit ratio | Cost USD |',\n '|---|---:|---:|---:|---:|---:|---:|---:|',\n ...totals.map(t => `| ${t.strategy} ${t.name} | ${t.calls} | ${t.errors} | ${t.prompt_tokens} | ${t.hit} | ${t.miss} | ${Math.round(100 * t.hit_ratio)}% | ${fmt(t.cost_usd, 5)} |`),\n].join('\\n');\nconst mdTurns = [\n '| Strategy | Turn | Kind | Prompt | Hit | Miss | Out | Cost USD | Note |',\n '|---|---:|---|---:|---:|---:|---:|---:|---|',\n ...rows.map(r => `| ${r.strategy} | ${r.turn} | ${r.kind} | ${r.prompt_tokens} | ${r.hit} | ${r.miss} | ${r.out} | ${fmt(r.cost_usd, 6)} | ${r.note ?? ''} |`),\n].join('\\n');\n\nreturn [{ json: {\n totals, markdown: md, markdownPerCall: mdTurns,\n mermaidHitRatio: '```mermaid\\n' + hitChart + '\\n```',\n mermaidCost: '```mermaid\\n' + costChart + '\\n```',\n rows: rows.map(({ answer, ...r }) => r),\n errors,\n finalAnswersTurn12: Object.fromEntries(S.map(s => [s, rows.find(r => r.strategy === s && r.turn === turns && r.kind === 'turn')?.answer])),\n} }];\n" } }, { "id": "7b667f8a-0bbf-4f10-b625-6f2b3f7e30c4", "name": "Note 1", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ -640, -300 ], "parameters": { "content": "## Memory strategy vs prompt cache\nOne scripted 12-question chat over a ~2k-token handbook, replayed with three memory strategies. Every turn is a real DeepSeek call, and the workflow records how many prompt tokens came from the provider's **prefix cache** and what each turn cost.\n\n**A** sliding window (what Simple Memory does) · **B** append-only, compact once · **C** prune one middle turn.\n\n### Setup\n1. Create a **DeepSeek** credential (API key) and select it in the *DeepSeek* node.\n2. No data table needed. Results come out of the *Report* node.\n3. Click **Execute workflow** (the whole workflow, not single steps). A run takes about 4 minutes (37 calls, 5 s apart).\n\nFailed calls are retried 3 times, then skipped and counted in Report's *Failed* column.\n\nBackground: [Drew Breunig, *The State of Context Engineering*](https://www.dbreunig.com/2026/10/24/the-state-of-context-engineering.html)", "width": 560, "height": 620, "color": 4 } }, { "id": "92a4dc87-b168-4ae3-8337-3c047d4eab32", "name": "Note 2", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ -40, -300 ], "parameters": { "content": "## Each strategy gets its own cache\nDeepSeek reuses the longest cached prefix of a request. In our runs the reused part was everything before the first changed token, rounded down to a 128-token block, and the last partial block was not reused. It is billed at the hit price (1/50 of a miss).\n\n*Script* puts a fresh **nonce on the first line of each strategy's system prompt**. Without it, strategy B would read the blocks strategy A already paid for, and a second run would read the first one's.\n\n[DeepSeek: context caching](https://api-docs.deepseek.com/guides/kv_cache)", "width": 640, "height": 460, "color": 6 } }, { "id": "c1c75f46-bd83-40be-91b8-8754b02c4e3a", "name": "Note 3", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ 620, -300 ], "parameters": { "content": "## Three ways to keep a chat short\nEach turn carries a ~300-token log excerpt, like a tool result in an agent loop.\n\n**A** sends only the last 4 messages. The start of the history moves every turn, so from turn 4 only (most of) the system prompt hits.\n\n**B** appends. Once the history passes 2,000 tokens it asks for a summary, with the request *appended* so the summary call is itself cached, then continues append-only.\n\n**C** deletes turn 3 at turn 6: the [A,B,C,D,E] case from [this HN thread](https://news.ycombinator.com/item?id=50236737). Everything after the cut is re-read once.", "width": 580, "height": 460, "color": 5 } }, { "id": "f66202d2-d39b-402c-acae-571f01ab9504", "name": "Note 4", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ 1220, -300 ], "parameters": { "content": "## Why raw HTTP?\nn8n's AI Agent with the DeepSeek Chat Model reports only prompt and completion tokens, not cache hits and misses, and always runs with thinking on.\n\nCalling `/chat/completions` directly returns `prompt_cache_hit_tokens` and `prompt_cache_miss_tokens`, and lets us turn thinking off.", "width": 300, "height": 460, "color": 3 } }, { "id": "5c57871a-27c1-49ef-ab78-3b618b17e38a", "name": "Note 5", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ 1540, -300 ], "parameters": { "content": "## Why wait 5 s?\nDeepSeek says building a cache entry takes seconds. If the next call goes out too soon, it can miss a prefix that would hit a moment later, and the strategy would be blamed for a timing effect.\n\nIn our probe, an identical request sent right after the answer already hit, and waiting 25 s gained nothing. The wait costs run time, not money.", "width": 380, "height": 460, "color": 7 } }, { "id": "68993c38-b6cf-4ede-aaac-c680e09a1e59", "name": "Note 6", "type": "n8n-nodes-base.stickyNote", "typeVersion": 1, "position": [ 1100, 560 ], "parameters": { "content": "## Example result (our run, peak prices)\n| | prompt tok | hit ratio | cost |\n|---|---:|---:|---:|\n| A sliding window | 41,552 | 55% | $0.0070 |\n| B compact once | 56,713 | 84% | $0.0045 |\n| C prune turn 3 | 61,083 | 85% | $0.0042 |\n\nC sent the most tokens and was still the cheapest. The sliding window was the most expensive, and at turn 12 it could no longer list the earlier cost questions. Your numbers will differ a little.\n\n**Cost:** this run cost **$0.016** at DeepSeek peak prices (weekdays 01-04 and 06-10 UTC), about half that off-peak. `deepseek-flash`, thinking off. Prices per model are in *Script*.\n\n*Report* returns markdown tables and two `xychart-beta` blocks (Mermaid/Pele).", "width": 700, "height": 440, "color": 2 } } ], "connections": { "Start": { "main": [ [ { "node": "Script", "type": "main", "index": 0 } ] ] }, "Script": { "main": [ [ { "node": "Steps", "type": "main", "index": 0 } ] ] }, "Steps": { "main": [ [ { "node": "Loop", "type": "main", "index": 0 } ] ] }, "Loop": { "main": [ [ { "node": "Report", "type": "main", "index": 0 } ], [ { "node": "Build", "type": "main", "index": 0 } ] ] }, "Build": { "main": [ [ { "node": "Call?", "type": "main", "index": 0 } ] ] }, "Call?": { "main": [ [ { "node": "DeepSeek", "type": "main", "index": 0 } ], [ { "node": "Record", "type": "main", "index": 0 } ] ] }, "DeepSeek": { "main": [ [ { "node": "Wait 5s", "type": "main", "index": 0 } ] ] }, "Wait 5s": { "main": [ [ { "node": "Record", "type": "main", "index": 0 } ] ] }, "Record": { "main": [ [ { "node": "Loop", "type": "main", "index": 0 } ] ] } }, "settings": { "executionOrder": "v1" }, "pinData": {}, "meta": { "templateCredsSetupCompleted": false } }