6 · Context Management ★

This is the part of the system with the highest engineering density. The problem it solves fits in one sentence: an agent’s context grows on its own, and the context window has a hard ceiling.

6.1 The five-tier pipeline

Before every model call, the message history passes through this pipeline. The tiers are ordered from cheapest to most expensive:

// Inside the query.ts main loop, at the top of each iteration
let messagesForQuery = [...getMessagesAfterCompactBoundary(messages)]
//   First take the messages "after the last compact boundary"; everything before has been replaced by a summary

// ① Tool result budget: a single result / the per-message aggregate over the limit → spill to disk, replace with a reference
messagesForQuery = await applyToolResultBudget(
  messagesForQuery,
  toolUseContext.contentReplacementState,
  persistReplacements ? records => void recordContentReplacement(...) : undefined,
  new Set(toolUseContext.options.tools
    .filter(t => !Number.isFinite(t.maxResultSizeChars))    // exclude tools that declare Infinity
    .map(t => t.name)),
)

// ② Snip: remove zombie messages and stale markers
let snipTokensFreed = 0
if (feature('HISTORY_SNIP')) {
  const snipResult = snipModule!.snipCompactIfNeeded(messagesForQuery)
  messagesForQuery = snipResult.messages
  snipTokensFreed = snipResult.tokensFreed
  if (snipResult.boundaryMessage) yield snipResult.boundaryMessage
}

// ③ Micro-compaction: precisely delete old results by tool-call id
const microcompactResult = await deps.microcompact(messagesForQuery, toolUseContext, querySource)
messagesForQuery = microcompactResult.messages
const pendingCacheEdits = feature('CACHED_MICROCOMPACT')
  ? microcompactResult.compactionInfo?.pendingCacheEdits : undefined

// ④ Context collapse: projection-style, replayable
if (feature('CONTEXT_COLLAPSE') && contextCollapse) {
  const collapseResult = await contextCollapse.applyCollapsesIfNeeded(
    messagesForQuery, toolUseContext, querySource)
  messagesForQuery = collapseResult.messages
}

// ⑤ Auto summary compaction: one full model call
const { compactionResult, consecutiveFailures } = await deps.autocompact(
  messagesForQuery, toolUseContext, { systemPrompt, userContext, systemContext,
  toolUseContext, forkContextMessages: messagesForQuery }, querySource, tracking, snipTokensFreed)

// ⑥ Hard blocking check (only in effect when auto-compaction is turned off)
if (!compactionResult && querySource !== 'compact' && ... ) {
  const { isAtBlockingLimit } = calculateTokenWarningState(
    tokenCountWithEstimation(messagesForQuery) - snipTokensFreed,
    toolUseContext.options.mainLoopModel)
  if (isAtBlockingLimit) {
    yield createAssistantAPIErrorMessage({ content: PROMPT_TOO_LONG_ERROR_MESSAGE, ... })
    return { reason: 'blocking_limit' }
  }
}

The reasoning behind the order

On why tier ④ must come before tier ⑤, the source has one precise sentence:

The counterintuitive design of tier ⑥

Note the trigger condition for the hard block: it is in effect only when auto-compaction is turned off. The comment explains:

6.2 Tier ①: the tool result budget

Spilling a single result to disk

export const PREVIEW_SIZE_BYTES = 2000                     // how many bytes to preview
export const PERSISTED_OUTPUT_TAG = '<persisted-output>'    // wrapper tag
export const TOOL_RESULTS_SUBDIR = 'tool-results'          // subdirectory on disk
export const TOOL_RESULT_CLEARED_MESSAGE = '[Old tool result content cleared]'

claude-code/src/utils/toolResultStorage.ts

The flow: a result exceeds the tool’s declared maxResultSizeChars → the full content is written to the tool-results/ directory → the model receives “a preview of the first 2000 bytes + the file path” → when it needs the full text, the model goes and Reads that file itself.

The Infinity exception

The per-message aggregate budget

Beyond the single-result limit, there is also a ContentReplacementState:

export type ContentReplacementState = {
  seenIds: ...        // result ids that have already passed the budget check
  ...
}
export function createContentReplacementState(): ContentReplacementState
export function cloneContentReplacementState(...): ContentReplacementState

What it guards against: the model fires off 20 search calls in parallel, each result within the single-result limit, but the total blows up.

Its lifecycle is also carefully managed; the comment describes three cases:

6.3 Tier ③: micro-compaction and cache edits

The dilemma

The goal is modest: delete old tool results that are no longer useful. But deleting them outright has a fatal side effect:

The fix: have the server delete inside the cache

/**
 * Cached microcompact path - uses cache editing API to remove tool results
 * without invalidating the cached prefix.
 *
 * - Does NOT modify local message content
 *   (cache_reference and cache_edits are added at API layer)
 * - Uses count-based trigger/keep thresholds from GrowthBook config
 * - Takes precedence over regular microcompact (no disk persistence)
 */
async function cachedMicrocompactPath(messages, querySource) {
  const mod    = await getCachedMCModule()
  const state  = ensureCachedMCState()
  const config = mod.getCachedMCConfig()

  // 1. Scan for the call ids of every "compactable tool"
  const compactableToolIds = new Set(collectCompactableToolIds(messages))

  // 2. Group by "user message" and register these tool results
  for (const message of messages) {
    if (message.type === 'user' && Array.isArray(message.message.content)) {
      const groupIds: string[] = []
      for (const block of message.message.content) {
        if (block.type === 'tool_result' &&
            compactableToolIds.has(block.tool_use_id) &&
            !state.registeredTools.has(block.tool_use_id)) {
          mod.registerToolResult(state, block.tool_use_id)
          groupIds.push(block.tool_use_id)
        }
      }
      mod.registerToolMessage(state, groupIds)
    }
  }

  // 3. Ask the state machine: which ones should be deleted? (keep the most recent N)
  const toolsToDelete = mod.getToolResultsToDelete(state)

  if (toolsToDelete.length > 0) {
    // 4. Generate a cache_edits instruction block and queue it for the API layer
    const cacheEdits = mod.createCacheEditsBlock(state, toolsToDelete)
    if (cacheEdits) pendingCacheEdits = cacheEdits
    ...
    // 5. ★ Return the messages unchanged — not a single byte was modified locally
    return { messages, compactionInfo: { pendingCacheEdits: {...} } }
  }
  return { messages }
}

claude-code/src/services/compact/microCompact.ts

Only 8 kinds of tool have deletable results

const COMPACTABLE_TOOLS = new Set<string>([
  FILE_READ_TOOL_NAME,      // read a file
  ...SHELL_TOOL_NAMES,      // Bash / PowerShell
  GREP_TOOL_NAME,           // content search
  GLOB_TOOL_NAME,           // filename search
  WEB_SEARCH_TOOL_NAME,     // web search
  WEB_FETCH_TOOL_NAME,      // fetch a web page
  FILE_EDIT_TOOL_NAME,      // edit a file
  FILE_WRITE_TOOL_NAME,     // write a file
])

What these 8 have in common: the result is a one-time observation — read a file, search for a word, run a command; once the model has digested it, the original text is no longer needed.

Other tools’ results (for instance TodoWrite, which maintains the to-do list) represent ongoing, still-valid state; deleting them would cause amnesia.

How many tokens were deleted? You have to wait for the server to tell you

// query.ts, after the streaming response ends
if (feature('CACHED_MICROCOMPACT') && pendingCacheEdits) {
  const lastAssistant = assistantMessages.at(-1)
  const usage = lastAssistant?.message.usage
  // ★ This field is "cumulative/sticky" (the total since the session began), not this request's delta
  const cumulativeDeleted = usage
    ? ((usage as unknown as Record<string, number>).cache_deleted_input_tokens ?? 0) : 0
  const deletedTokens = Math.max(0,
    cumulativeDeleted - pendingCacheEdits.baselineCacheDeletedTokens)   // subtract the pre-request baseline
  if (deletedTokens > 0) {
    yield createMicrocompactBoundaryMessage(
      pendingCacheEdits.trigger, 0, deletedTokens, pendingCacheEdits.deletedToolIds, [])
  }
}

Because nothing was changed locally, the client does not know how much was actually saved, so the “compacted” notification message is deferred until after the API response.

Side effect: there is a brief “perception gap” window — during the few seconds between “deciding to delete” and “the response coming back,” the client’s estimate of the context size runs high. This is also why the auto-compaction threshold checks later on are full of manual compensation terms (such as - snipTokensFreed).

Three guards against state contamination

// Guard 1: run cache editing only for the main thread
if (mod.isCachedMicrocompactEnabled() &&
    mod.isModelSupportedForCacheEditing(model) &&
    isMainThreadSource(querySource)) {
  return await cachedMicrocompactPath(messages, querySource)
}

Comment: “Only run cached MC for the main thread to prevent forked agents (session_memory, prompt_suggestion, etc.) from registering their tool_results in the global cachedMCState, which would cause the main thread to try deleting tools that don't exist in its own conversation.”

In other words: run cached micro-compaction only for the main thread, to keep forked agents (session memory, prompt suggestions, etc.) from registering their tool results in the global state, which would make the main thread try to delete tools that do not exist in its own conversation at all.

The root of the problem: cachedMCState is a module-level global singleton, holding its own copy of the truth alongside the message array. This is the biggest source of complexity in the whole mechanism.

The path in the other direction: the time-based trigger

export function evaluateTimeBasedTrigger(messages, querySource) {
  const config = getTimeBasedMCConfig()
  if (!config.enabled || !querySource || !isMainThreadSource(querySource)) return null
  const lastAssistant = messages.findLast(m => m.type === 'assistant')
  if (!lastAssistant) return null
  const gapMinutes = (Date.now() - new Date(lastAssistant.timestamp).getTime()) / 60_000
  if (!Number.isFinite(gapMinutes) || gapMinutes < config.gapThresholdMinutes) return null
  return { gapMinutes, config }
}

The comment lays out the logic of this path:

The Math.max(1, ...) boundary trap

// Floor at 1: slice(-0) returns the full array (paradoxically keeps everything),
// and clearing ALL results leaves the model with zero working context.
// Neither degenerate is sensible — always keep at least the last.
const keepRecent = Math.max(1, config.keepRecent)
const keepSet  = new Set(compactableIds.slice(-keepRecent))
const clearSet = new Set(compactableIds.filter(id => !keepSet.has(id)))

In other words: floor it at 1: because slice(-0) returns the entire array (paradoxically deleting nothing), while clearing every result would leave the model with no working context at all. Neither degenerate case makes sense.

(In JavaScript, slice(-N) means “take the last N.” But -0 is numerically equal to 0, and slice(0) means “take everything from the start.” So configuring “keep the most recent 0” actually behaves as “keep everything” — a classic boundary-value trap.)

After clearing, reset the state and notify monitoring

suppressCompactWarning()
// The global cached-micro-compaction state holds tool ids registered in earlier turns. We just cleared some
// of their content, and by changing the prompt we invalidated the server cache. If the next turn's cached
// micro-compaction ran with stale state, it would try to cache_edit entries the server no longer has. So reset it.
resetMicrocompactState()

// We just changed the prompt content — the next response's cache reads will be low, but we caused that
// ourselves; it isn't a cache break. Tell the detector to expect a dip.
if (feature('PROMPT_CACHE_BREAK_DETECTION') && querySource) {
  notifyCacheDeletion(querySource)
}

That final notifyCacheDeletion points to the cache-break detection system covered in Chapter 12 — it watches for “a sudden drop in cache hit rate” and alerts. This one is a “legitimate drop,” so it has to be told in advance not to raise a false alarm.

6.4 Tier ⑤: auto summary compaction

Thresholds

export const WARNING_THRESHOLD_BUFFER_TOKENS = 20_000   // buffer below the warning line
export const ERROR_THRESHOLD_BUFFER_TOKENS   = 20_000   // buffer below the error line

const threshold        = getAutoCompactThreshold(model)
const warningThreshold = threshold - WARNING_THRESHOLD_BUFFER_TOKENS
const errorThreshold   = threshold - ERROR_THRESHOLD_BUFFER_TOKENS

Compaction is a “forked subagent” call

Compaction itself calls the model to produce a summary. Claude Code runs it in a forked subagent, and lets the fork inherit the parent’s full tool set — not because summarization needs tools, but so the cache key matches and the cache prefix the parent already built gets reused.

That decision led to a real production issue, and the source comment even preserved the numbers:

const NO_TOOLS_PREAMBLE = `CRITICAL: Respond with TEXT ONLY. Do NOT call any tools.

- Do NOT use Read, Bash, Grep, Glob, Edit, Write, or ANY other tool.
- You already have all the context you need in the conversation above.
- Tool calls will be REJECTED and will waste your only turn — you will fail the task.
- Your entire response must be plain text: an <analysis> block followed by a <summary> block.

`

claude-code/src/services/compact/prompt.ts

Two-part output: the scratchpad pattern

const DETAILED_ANALYSIS_INSTRUCTION_BASE = `Before providing your final summary,
wrap your analysis in <analysis> tags to organize your thoughts and ensure
you've covered all necessary points. In your analysis process:

1. Chronologically analyze each message and section of the conversation.
   For each section thoroughly identify:
   - The user's explicit requests and intents
   - Your approach to addressing the user's requests
   - Key decisions, technical concepts and code patterns
   - Specific details like:
     - file names
     - full code snippets
     - function signatures
...`

And the processing function formatCompactSummary() strips the entire analysis block, placing only the summary block into the context. Source comment: “The <analysis> block is a drafting scratchpad that formatCompactSummary() strips before the summary reaches context.”

Telemetry after a successful compaction

logEvent('tengu_auto_compact_succeeded', {
  originalMessageCount:  messages.length,
  compactedMessageCount: compactionResult.summaryMessages.length +
                         compactionResult.attachments.length +
                         compactionResult.hookResults.length,
  preCompactTokenCount, postCompactTokenCount, truePostCompactTokenCount,
  compactionInputTokens:  compactionUsage?.input_tokens,
  compactionOutputTokens: compactionUsage?.output_tokens,
  compactionCacheReadTokens:     compactionUsage?.cache_read_input_tokens ?? 0,
  compactionCacheCreationTokens: compactionUsage?.cache_creation_input_tokens ?? 0,
  compactionTotalTokens: ...,
  queryChainId: ..., queryDepth: ...,
})

Note that there are three “post-compaction token counts”: postCompactTokenCount and truePostCompactTokenCount. The difference between them is exactly the “perception gap” described earlier — one is the client’s estimate, the other the true value returned by the server. Reporting both the estimate and the true value lets them continuously monitor the estimator’s error.

6.5 When the context really does overflow: the three-tier recovery cascade

Why step ③ must not run the stop hooks

Take a moment to appreciate the shape of this loop. The “pre-finish quality check” feature is perfectly reasonable on its own, but when the cause of failure is “the context no longer fits,” it will:

1. See a failed response
2. Judge it inadequate and generate feedback along the lines of “your answer has the following problems…”
3. Inject that feedback into the context — the context gets even longer
4. Retry → even further over → fails again → back to step 1