The harness sends messages and tool definitions as JSON, as shown in Agent loop anatomy. This is the API’s request format, not the model’s input format.
In the local setup used here, the server parses that request and applies a model-specific chat template. The template arranges the messages and tool definitions into text with role and turn markers. That text is then converted into token IDs, which are what the model processes.
The stateless API used by the loop requires the conversation history to be included with each request. The input grows as messages and tool results accumulate. Prefix caching can reuse earlier computation, so sending the history again does not necessarily mean evaluating every token again.
Context sources and lifecycle
The model input comes from five sources, not just the agent loop.
Fig. 1The writers' deskOne stack, five writers. Time runs down the page.
Harness code
Instruction files + skills index
User messages
The model
Tool results
At session start, code and disk write the top of the string. Code supplies the tool schemas, the core prompt and whatever it knows about the environment. Disk supplies your instruction files and the list of available skills. Nothing from the conversation is in there yet.
Ordinary turns extend the tail. You write a message. The model responds, perhaps with thinking and a tool call. A tool writes its result. Harnesses can also append reminders when files change, rather than rewriting the earlier context.
Compaction interrupts that growth: the harness replaces older history in the next request with a summary, usually keeping recent messages intact. The full history can remain on disk even when it is no longer sent to the model.
Context structure: head and tail
That relatively stable setup is the head. The accumulated conversation below it is the tail.
Fig. 2The context barWire order, to scale. The double rule between environment and summary is the useful line.
Harness code
Instruction files + skills index
User messages
The model
Tool results
Top: one Pi session of mine at its last turn; the 272K window is inferred from where it compacted and Pi's 16,384 reserve; hatched thinking is what the session stored, what is resent varies by provider. Bottom: sizes from pinned source revisions in the Qwen tokenizer, not billing counts, November 2026. Codex measured with its own prompt dump on an empty directory. Hatching marks a range or an uncertain size; Claude Code captures disagree; the tick is one session of mine, from its own /context estimate.
The head is what people fight over. But look at the top strip, which is one of my real sessions at its last turn: the head is a sliver.
The table gives each region a writer, a lifetime and a format. The head usually persists across turns. A summary carries older context until the next compaction. A reminder becomes part of the history and can later be summarized too.
From session records to model tokens
Take a small conversation: the user asks which port the dev server uses, the assistant calls read_file on package.json, and the tool returns its contents. The figure shows four representations of that exchange, grouped by where they belong.
Fig. 3aFrom session record to model inputClient-side formats on the left; inference-side representations on the right.
Composite example, not a captured Claude Code-to-Qwen pipeline. The first column uses Claude Code transcript keys with invented values; the dashed link compares formats. The local JSON request, Qwen3.6 prompt and token IDs were reproduced through llama.cpp b12110 on 2026-11-09. Excerpts omit reasoning and other prompt content.
The left pair compares client-side formats: an illustrative Claude Code session record and the JSON request sent to llama.cpp for a separate local test. They are not a captured Claude Code-to-Qwen pipeline. A harness can store transcript metadata without including it in the API request.
The right pair follows that local request through llama.cpp. The server parses the JSON and applies the model’s chat template to construct a prompt string in memory. The XML-like text means “call read_file with path set to package.json.” Here it represents an earlier assistant call reconstructed as part of the history. The tokenizer encodes the prompt, including its role and turn markers, as token IDs.
The next figure compares that same intended call across five model templates. These are alternative encodings, not further stages in the pipeline.
Fig. 3bTool-call syntax by model templateEach row represents the same call: read_file(path="package.json").
Excerpts of chat-template output, not API request formats or different harnesses. Qwen was rendered by local llama.cpp b12110; the other examples were rendered with sandboxed Jinja from templates pinned at cd959bae in llama.cpp. Line wrapping and whitespace are simplified. These are not captures of hosted API prompts.
Chat templates usually ship with an open-weight model’s tokenizer files or model package; in this local GGUF, the template is stored as tokenizer.chat_template metadata and applied by llama.cpp. Hosted services generally keep this formatting server-side, so clients see structured messages rather than the rendered prompt.
Fig. 4 shows input prepared for the next model call, not a generated response. The conversation is constructed: the earlier assistant reasoning and tool call are included as history.
Tokenization happens behind the API. Here llama.cpp tokenizes the rendered prompt using the vocabulary and tokenizer metadata bundled in the Qwen GGUF model file. The tokenizer itself is not part of the prompt. The harness normally sends messages and tools, not token IDs, although it may count tokens to manage its context budget. On the return path, the server decodes generated token IDs and parses tool calls into the API response.
Fig. 4Tokenized input to QwenConstructed conversation history, prepared for the next model call.
377 input tokens, in prompt order.
Show 272 setup tokensHide 272 setup tokensTool definition, tool-use instructions and opening markers
Hover or focus a tile to see its token ID and position in the input.
The same input, as regionstokens
Tool setup272 tokens
System prompt11 tokens
User message13 tokens
Assistant history50 tokens
Tool result26 tokens
Next assistant prefix5 tokens
Top to bottom: prompt order. Counts include markers; bands are not to scale.
Harness code
User messages
The model
Tool results
Qwen3.6-35B-A3B tokenizer from the UD-Q4_K_XL GGUF, run by llama.cpp b12110; measured 2026-11-09. ⏎ is a newline, · is a space; an outline marks a reserved token.
Tool definitions and tool-use instructions are prompt content too, not overhead introduced by the tokenizer. The collapsed setup preserves those tokens without making you read through all of them.
Context assembly across harnesses
Harnesses agree on the regions and disagree on the details. The lower half of Fig. 2 shows the two details that matter most.
The first is how much is in the head.
Pi starts with 1.3K tokens and four tools.
Gemini CLI starts around 10K to 12K.
Codex measured 13K on my machine, with its own prompt dump, in an empty directory.
For Claude Code I can’t give you one number: published captures range from 18K to 65K depending on version and MCP setup.
The source-derived estimates use one tokenizer as a ruler, not as a billing count. Claude Code’s published captures use its own accounting, so the comparison is approximate.
The second is where your instruction files land. The dots in the figure show it. Pi and OpenCode put them in the system region. Codex and Claude Code put them in an early user-role message. Gemini CLI does both: global memory in the system instruction, project memory in the first user message.
Claude Code will show you this split itself. Type /context:
Fig. 5/context, read as regionsClaude Code's own estimate of one session of mine.
Claude Code with Opus 5.5, November 2026, the tool's own token estimate. Head is everything above Messages: 3.6K + 19.8K + 0.7K + 1.7K + 6.5K = 32.3K. Deferred tool definitions (about 27K if all were loaded) are not part of the 278.1K total.
That’s the session that helped write this post, so its tail is large. Its head is 32.3K tokens. Look at the line it doesn’t count, too: another 27K tokens of tool definitions are deferred, and load only when needed.
Lazy loading reduces the initial context size. Skills keep only an index in the head: a name and a description for each. In one repository of mine, 19 skills cost 1,136 tokens as an index, against 29,257 tokens if the bodies were loaded. When the model decides it needs one, it reads the file. So a loaded skill is literally a tool result, sitting in the tail like any other.
Context composition in 264 Pi sessions
These measurements reflect my own workflow. Alongside coding, I work extensively with file-based knowledge bases, or “second brains”.
Here are three of my own Pi sessions as drill cores, to one scale. Time runs upward. Each message lays down one layer in the colour of whoever wrote it.
Fig. 6The core sampleThree real sessions, to one scale. Each message lays down a layer.
Harness code
Instruction files + skills index
User messages
The model
Tool results
Pi sessions of mine, chosen because they are typical for their length, not at random. Layer sizes: characters converted with calibrated per-type ratios, then scaled so the live stack matches the provider-reported total at the last turn (scale factors 0.903, 0.805, 0.846). Numbers only, measured 2026-11-09.
The median session is mostly read and bash output in ochre. Violet marks the model’s thinking and tool calls; pink marks my messages. Green is the instruction files and skills index loaded at session start. Reading a note from a second brain is a tool result, so it appears in ochre, not green.
Across my 264 sessions up to 2026-11-08 (15,468 assistant turns, mixed providers), read and bash produced 92% of the tool result characters.
Who wrote the end-of-session context, on average:
Fig. 7Who wrote it, and where the head wentLeft: share of context at session end. Right: median context by turn.
My Pi sessions, 2026-08-28 to 2026-11-08, mixed providers: composition is the mean over 185 sessions with 5+ turns; columns are p50 of provider-reported context over 264 sessions. Later columns only cover sessions that lived that long, and compaction resets are included.
On average, tool results make up 65.5% of my end-of-session context. My own messages account for under 10%.
On the right, the head becomes a smaller share of a growing context. Vanilla Pi’s 1.3K head is the thin line. My real first-turn median, with extensions and skills loaded, is 7.6K. By turn 10 the median context is 45K, so even that bigger head is a sixth of it. By turn 100 it’s 4%, and only 40 of my 264 sessions get there.
So if you want a smaller context, the system prompt is rarely where to look. Look at what your tools return.
A larger context is not necessarily a more useful one. Long-context evaluations show that models can become less reliable as input grows, and that the position of relevant information matters. But these session measurements describe what occupies the context, not how well the model uses it. Tool output dominating the history is a reason to examine its relevance, not proof that it makes the agent less capable.
Compaction and information loss
Compaction makes room for new work by replacing older history with a shorter account of it. The growing tail forces a choice: what must the next request still contain?
In Pi, the harness triggers this when its context estimate crosses a configured threshold below the model’s window limit, or when the user requests it. This manages capacity; it does not detect declining model quality.
By default, Pi keeps recent messages unchanged and makes a separate model call to summarize older history. The next request contains the head, the summary and the retained recent history, rather than the full conversation.
In my 264 sessions up to 2026-11-08, compaction happened 62 times across 36 sessions. Most ended without it. The long core in Fig. 6 shows older layers ghosted below each compaction boundary, with a thin summary seam above. That session used pi-vcc to extract its summaries without a model call.
Missing from context does not mean deleted from storage. Pi keeps the original session entries on disk, but omitted details must be retrieved to reach the model again. A summary can preserve a decision while dropping its rationale, or omit a constraint entirely. These measurements do not tell us how often that caused mistakes.
Tool results fill most of my history, but volume alone does not tell us what to preserve. Code and notes can often be read again. Requirements stated only in conversation need to survive in the retained context or a record the agent knows to retrieve.
Context processing costs
What remains in context also has a processing cost. The following comparison estimates that cost for a median session: 23 turns on a cloud model, priced at list.
Fig. 8The receiptA median 23-turn session, and the same work on my own GPU.
Left: hypothetical repricing of a median 23-turn session at Opus 5.5 list prices of 2026-11-08 with every cache hit, not what was paid. Region shares from the session-end composition; items are rounded so they don't sum exactly. Right: Qwen3.6-35B-A3B on an RTX 3060, llama.cpp b12110, 144 requests. The next post is about why these two receipts disagree.
In the cloud estimate, the four written-once lines together cost more than re-reading the history. That depends on the assumed cache hits; it is hypothetical repricing, not my actual bill. The local measurements express the cost in reserved memory and processing time instead.
Cloud pricing and local inference reflect different costs. The next post examines the role of caching.