2 · The Session Layer: QueryEngine

This chapter covers how “a conversation” is represented and managed inside the program.

2.1 The problem it solves

A large language model has no memory — every call has to resend the entire history. So something must hold all the state of the conversation and stay alive between one user question and the next.

That something is QueryEngine. The class comment in the source puts it plainly:

2.2 What state it holds

export class QueryEngine {
  private config: QueryEngineConfig       // immutable configuration (tools, commands, model, etc.)
  private mutableMessages: Message[]      // ★ the full message history; grows forever
  private abortController: AbortController // the abort switch, threaded through the entire call chain
  private permissionDenials: SDKPermissionDenial[]  // record of operations that were denied
  private totalUsage: NonNullableUsage    // cumulative token usage
  private hasHandledOrphanedPermission = false
  private readFileState: FileStateCache   // ★ which files have been read, and which version

  // The two below are "per-turn tracking," cleared at the start of each turn
  private discoveredSkillNames = new Set<string>()   // skills discovered this turn
  private loadedNestedMemoryPaths = new Set<string>() // memory files loaded this turn
}

claude-code/src/QueryEngine.ts

Two of these fields deserve a closer look

readFileState (the file-read state cache) records “which files the model has read, and which version.” It serves three purposes:

1. Preventing duplicate injection. When the memory system prefetches a file, and the model has already read it on its own, it is not injected again as “memory.”
2. Detecting external modification. The model read a.ts; later the user changed it in their editor; now the content the model is holding is stale. The system detects this and injects a notice.
3. A safety check before editing. When the model tries to modify a file it has never read, the tool refuses and tells it to read first — because editing blind is a recipe for mistakes.

abortController is the “cancel switch.” It is passed into every tool call and every network request. When the user presses Ctrl+C, it is pulled, and the whole chain can feel it. Chapter 3 covers how to handle it correctly.

2.3 The full flow of one submitMessage

submitMessage() is the core method of this class. It is an async generator — meaning it does not “finish computing, then return”; it emits messages while it works, and the caller can consume them in real time.

async *submitMessage(
  prompt: string | ContentBlockParam[],
  options?: { uuid?: string; isMeta?: boolean },
): AsyncGenerator<SDKMessage, void, unknown>

(async * is JavaScript’s async generator syntax. To yield a value means “hand this out first; once the caller has it, I continue.” This is the foundation that lets a streaming interface update live.)

The full flow:

2.4 Why the user message must hit disk first

Step ⑤ carries a long comment describing a real production issue:

In plain English: if the program is killed during the few seconds between the user sending a message and the model replying, that conversation is gone for good. Because the write to disk happened only after the model’s reply.

The fix is to move the write forward to the moment “the user message is accepted.” But that surfaces a performance trade-off:

if (persistSession && messagesFromUserInput.length > 0) {
  const transcriptPromise = recordTranscript(messages)
  if (isBareMode()) {
    void transcriptPromise          // ★ bare mode: fire and forget; don't wait for the write
  } else {
    await transcriptPromise         // normal mode: wait for the write before continuing
    ...
  }
}

The comment explains why bare mode gets special treatment:

2.5 Consuming the main loop’s output: one big switch

Once inside the main loop, QueryEngine uses a for await loop to consume each message the main loop emits, handling each type differently:

Why model messages are fire-and-forget while user messages wait synchronously

There is a subtle mechanism in here that is worth unpacking:

The progress message’s “write immediately” also has a dedicated comment:

2.6 The compact boundary: proactively releasing memory

When the main loop emits a “compact boundary” message, QueryEngine does something important:

if (message.subtype === 'compact_boundary' && message.compactMetadata) {
  // Messages before the boundary have been replaced by a summary; release them to the garbage collector
  const mutableBoundaryIdx = this.mutableMessages.length - 1
  if (mutableBoundaryIdx > 0) {
    this.mutableMessages.splice(0, mutableBoundaryIdx)   // ★ delete them from the array outright
  }
  const localBoundaryIdx = messages.length - 1
  if (localBoundaryIdx > 0) {
    messages.splice(0, localBoundaryIdx)
  }
  yield { type:'system', subtype:'compact_boundary', ... }
}

Comment: “Release pre-compaction messages for GC. query.ts already uses getMessagesAfterCompactBoundary() internally, so only post-boundary messages are needed going forward.” (Release the pre-compaction messages to the garbage collector. The main loop already only uses messages after the boundary internally, so only those need to be kept from here on.)

Why go out of the way to do this? Because the message history of a long session can run to hundreds of MB. After compaction, the earlier messages are logically useless, but as long as the array still references them, the garbage collector will not reclaim them — memory keeps climbing until the OS kills the process.

But the disk write must finish before the release

And the order cannot be wrong. Right before the deletion, there is a dedicated piece of disk-write logic:

if (persistSession && message.type === 'system' &&
    message.subtype === 'compact_boundary') {
  const tailUuid = message.compactMetadata?.preservedSegment?.tailUuid
  if (tailUuid) {
    const tailIdx = this.mutableMessages.findLastIndex(m => m.uuid === tailUuid)
    if (tailIdx !== -1) {
      await recordTranscript(this.mutableMessages.slice(0, tailIdx + 1))
    }
  }
}

The comment explains the consequence of not doing this: “If the SDK subprocess restarts before then (claude-desktop kills between turns), tailUuid points to a never-written message → applyPreservedSegmentRelinks fails its tail→head walk → returns without pruning → resume loads full pre-compact history.”

In other words: if the subprocess restarts before then (the desktop app kills the process between turns), the tail node of the preserved segment points to a message that was never written → the relinking function’s tail-to-head walk fails → it returns without pruning → and resume loads the full pre-compaction history.

The symptom: a session the user had compacted comes back after resume looking the way it did before compaction, and the context blows up immediately.

2.7 Three kinds of exit result

submitMessage ends by emitting a result message indicating how the turn finished:

That last error type carries a dedicated diagnostic prefix

errors: (() => {
  const all = getInMemoryErrors()
  const start = errorLogWatermark ? all.lastIndexOf(errorLogWatermark) + 1 : 0
  return [
    // ★ Diagnostic prefix: states outright what the three "failure verdict" conditions each evaluated to
    `[ede_diagnostic] result_type=${edeResultType} ` +
    `last_content_type=${edeLastContentType} stop_reason=${lastStopReason}`,
    ...all.slice(start).map(_ => _.error),
  ]
})()

And the error list is sliced to the range of this turn — using a “watermark”:

// Use a reference, not an index, as the watermark, so the errors array for error_during_execution
// is scoped to this turn. A length index breaks when the 100-entry ring buffer shifts — the index slides away.
// If the marker itself has been rotated out, lastIndexOf returns -1 and we include everything (safe fallback).
const errorLogWatermark = getInMemoryErrors().at(-1)

2.8 ask(): a convenience wrapper for one-shot calls

The end of the file also exports an ask() function, a one-shot wrapper around QueryEngine — create an instance, run one round, hand the file cache back to the caller:

export async function* ask({...}) {
  const engine = new QueryEngine({
    ...,
    readFileCache: cloneFileStateCache(getReadFileCache()),   // ★ what gets passed in is a clone
  })
  try {
    yield* engine.submitMessage(prompt, { uuid: promptUuid, isMeta })
  } finally {
    setReadFileCache(engine.getReadFileState())   // ★ hand it back no matter what
  }
}

Two details:

• The file cache passed in is a clone. So this call’s changes to the cache do not affect the outside world right away — not until it is explicitly handed back at the end.
• The finally block guarantees the hand-back. Even if an exception is thrown or the call is interrupted midway, the knowledge of “which files the model has read” is not lost. Losing it would cause the next turn to inject memory twice or misjudge file freshness.