12 · The Observability System

This chapter covers something that almost never appears in architecture discussions but decides whether a product can keep evolving over the long run: how the system knows what's happening to itself.

12.1 Instrumentation Density

The numbers first:

12.2 Event Naming

Every event starts with tengu_ (tengu is an internal codename). The event names that appear in the main loop show the naming pattern:

Two patterns in the naming

Pattern one: paired instrumentation. xxx_used / xxx_not_used, xxx_before / xxx_after — that's what lets you compute ratios and deltas, not just absolute counts.

The pair tengu_streaming_tool_execution_used/not_used, for instance, gives you “the streaming executor's enablement rate” directly. If that ratio suddenly drops after a release, some code path is unexpectedly bypassing it.

Pattern two: carry the decision's “why.” tengu_auto_mode_decision records not just “allowed or denied” but also fastPath: 'acceptEdits' | 'allowlist' — which fast path was taken. That's what lets you evaluate “what percentage of classifier calls did the fast paths intercept,” in other words, whether the optimization is worth it at all.

12.3 Query Chain Tracing

Nearly every instrumentation call carries two fields:

queryChainId: queryChainIdForAnalytics,   // unique ID for this user request
queryDepth:   queryTracking.depth,        // agent nesting depth (main 0, child 1, grandchild 2)

The generation logic was covered in section 3.9:

const queryTracking = toolUseContext.queryTracking
  ? { chainId: toolUseContext.queryTracking.chainId,          // inherit the parent's chain ID
      depth:   toolUseContext.queryTracking.depth + 1 }       // depth +1
  : { chainId: deps.uuid(), depth: 0 }                        // top level: create a new one

12.4 Privacy Protection at the Type Level

The instrumentation code is full of an odd type cast:

toolName: sanitizeToolNameForAnalytics(tool.name),
errorDetails: errorContent.slice(0, 2000)
  as AnalyticsMetadata_I_VERIFIED_THIS_IS_NOT_CODE_OR_FILEPATHS,
queryChainId: queryTracking.chainId
  as AnalyticsMetadata_I_VERIFIED_THIS_IS_NOT_CODE_OR_FILEPATHS,

Read the type name as plain words: analytics metadata: I verified this is not code or file paths.

There's a companion sanitizer too: sanitizeToolNameForAnalytics() — because MCP tool names include the server name, and server names can be user-defined and contain sensitive information.

12.5 Cache-Break Detection

A dedicated subsystem monitors the health of the prompt cache: services/api/promptCacheBreakDetection.ts, controlled by the compile-time flag PROMPT_CACHE_BREAK_DETECTION.

What it monitors

Normally, a session's cache-read volume should grow turn over turn (the history keeps getting longer, so more and more of it hits the cache). If it suddenly plunges on some turn, the cache was broken — some code changed the context prefix.

But some drops are legitimate

So the system provides an interface for proactive notification:

// microCompact.ts, after a cache edit runs
// Notify cache break detection that cache reads will legitimately drop
if (feature('PROMPT_CACHE_BREAK_DETECTION')) {
  notifyCacheDeletion(querySource ?? 'repl_main_thread')
}

// after time-based microcompaction runs
// We just changed the prompt content — the next response's cache read will
// be low, but that's us, not a break. Tell the detector to expect a drop.
if (feature('PROMPT_CACHE_BREAK_DETECTION') && querySource) {
  notifyCacheDeletion(querySource)
}

Put plainly: we just changed the prompt content — the next response's cache read will be low, but that's us, not a break. Tell the detector to expect a drop.

12.6 Profiling Checkpoints

Two sets of checkpoints are scattered through the code:

// startup path
profileCheckpoint('cli_entry')
profileCheckpoint('cli_dump_system_prompt_path')
profileCheckpoint('cli_bridge_path')
...

// latency tracking in headless mode
headlessProfilerCheckpoint('before_getSystemPrompt')
headlessProfilerCheckpoint('after_getSystemPrompt')
headlessProfilerCheckpoint('before_skills_plugins')
headlessProfilerCheckpoint('after_skills_plugins')
headlessProfilerCheckpoint('system_message_yielded')
headlessProfilerCheckpoint('query_started')

// inside the main loop
queryCheckpoint('query_fn_entry')
queryCheckpoint('query_snip_start') / queryCheckpoint('query_snip_end')
queryCheckpoint('query_microcompact_start') / ('query_microcompact_end')
queryCheckpoint('query_autocompact_start') / ('query_autocompact_end')
queryCheckpoint('query_setup_start') / ('query_setup_end')
queryCheckpoint('query_api_loop_start')
queryCheckpoint('query_api_streaming_start') / ('query_api_streaming_end')
queryCheckpoint('query_tool_execution_start') / ('query_tool_execution_end')
queryCheckpoint('query_recursive_call')

Note how these checkpoints are distributed: they map almost exactly onto each stage of the five-stage pipeline from chapter 6. That's no coincidence — only by measuring each stage's time on its own can you judge “whether optimizing this stage is worth it.”

There's a heavier option too: the compile-time flag PERFETTO_TRACING. Perfetto is Google's performance tracing tool, which produces a visual timeline.

12.7 Slow-Operation Logging

The compile-time flags include SLOW_OPERATION_LOGGING, with a matching module utils/slowOperations.ts. It exports a function called jsonStringify — meaning even “turn an object into a JSON string” has been wrapped on its own and brought under slow-operation monitoring.

Why? Because in this system, the object being serialized may be hundreds of MB of message history. At that scale, JSON.stringify blocks the main thread for hundreds of milliseconds — and the main thread is simultaneously rendering the streaming UI, so the stutter is visible to the user immediately.

12.8 The In-Memory Error Buffer

const errorLogWatermark = getInMemoryErrors().at(-1)
...
errors: (() => {
  const all = getInMemoryErrors()
  const start = errorLogWatermark ? all.lastIndexOf(errorLogWatermark) + 1 : 0
  return [ `[ede_diagnostic] ...`, ...all.slice(start).map(_ => _.error) ]
})()

The system keeps a ring buffer holding only the most recent 100 entries of error logs in memory. When an execution fails, the errors “within this turn's range” get packaged into the result together.

Section 2.7 covered the watermark trick here: remember a reference to the element, not an array index, because the ring buffer shifts and indices slide away.

12.9 Loud Logging for Internal Errors

// To help track down bugs, log loudly for ants
logAntError('Query error', error)

The system distinguishes two kinds of error logging:

12.10 Cost-Awareness in Instrumentation

One last point worth noting: instrumentation has costs of its own, and Claude Code is aware of it.

// query.ts
const dumpPromptsFetch = config.gates.isAnt
  ? createDumpPromptsFetch(toolUseContext.agentId ?? config.sessionId)
  : undefined

The comment explains why this object is created only once: