When you switch models mid-session, you bring your baggage with you.

Earlier today in Policy-as-Code for Your Coding Agent’s Wallet, we built an automatic gearbox: a CEL rule engine that shifts from an expensive frontier model to a fast utility model when cost, context, or turn limits are crossed. But switching the model is only half the battle.

When the new model receives its first prompt, the coding agent forwards the entire historical transcript across the wire. That transcript is packed with multi-kilobyte bash command outputs, huge grep listings, and foreign tool schemas executed by the previous model.

This causes three distinct points of friction:

  1. Paying for dead weight: You pay full input token prices for historical tool results that the new model will never re-read or act on.
  2. Cross-provider schema friction: Different model families handle tool signatures and parallel calls differently (for example, Gemini thought signatures vs. Anthropic tool blocks). Sending foreign tool calls across providers risks gateway rejections or confused reasoning.
  3. Cache invalidation traps: If you naively prune historical results using a rolling sliding window, you move the boundary every turn and bust the prompt cache on every subsequent prompt.

Here is how we built model-switch-prune-omp, an extension for oh-my-pi that drops foreign tool overhead without touching the session history on disk or breaking prompt caching.

Why Disk Compaction Is the Wrong Mechanism

The obvious first impulse when switching models is to run compaction: summarize the session or invoke OMP’s internal /shake mechanism.

That impulse has two major problems:

Approach Where It Operates Drawback
Compaction / Summarization Persisted session branch Destructive: rewrites history, burns LLM summarization tokens, and loses raw command precision.
Native /shake Persisted session branch Modifies session storage on disk and is currently internal to the host agent (not exposed as a programmable hook for extensions).
Wire-Only Context Pruning In-memory context event Non-destructive: modifies only the outgoing wire array. The on-disk .jsonl transcript stays 100% complete.

The key architectural primitive in OMP is the context lifecycle event. When context fires, the extension receives the array of messages assembled for serialization. Returning a modified { messages } array changes what travels over HTTP to the provider, but leaves the underlying session log untouched.

If you later switch back to the original model (for instance, returning from an AFK downshift), that model’s messages are native again and the original verbatim context is restored instantly.

The Prompt Cache Trap: Provenance vs. Sliding Windows

In our initial design draft, the instinct was: “Keep the last 2 turns intact, and prune everything older.”

That sounds clean until you think about prompt caching. Modern LLMs cache prompt prefixes. In a normal session, every new turn appends to the existing conversation, and the entire prefix hits the cache.

If your pruning rule says “protect the last 4 messages and elide everything earlier,” your elision boundary shifts forward on every single prompt:

Because message 7 changed from verbatim to elided, the prompt prefix changed. Every subsequent prompt is a 100% cache miss.

The Fix: Anchor by Provenance

Instead of a sliding turn window, prune based on message provenance:

function isForeignAssistantToolTurn(message: AgentMessage, currentModel: ModelProvenance): boolean {
  return (
    message.role === "assistant" &&
    message.content.some((part) => part.type === "toolCall") &&
    (message.api !== currentModel.api || 
     message.provider !== currentModel.provider || 
     message.model !== currentModel.id)
  );
}

Every message recorded in OMP carries the api, provider, and model that generated it.

When you switch from Claude to Gemini or GPT:

  1. All tool calls generated by Claude become “foreign”.
  2. The extension strips those foreign tool call blocks from assistant turns, leaving the assistant’s prose and explanations intact.
  3. The linked toolResult messages are collapsed into readable narrative text blocks, with older results (>6,000 characters) trimmed to a 1,500-character head and a 500-character tail.
  4. The two most recent foreign results are kept complete for immediate continuity.

Because the set of foreign messages is frozen the moment the switch occurs, the pruned prefix remains byte-for-byte identical on turn 2, turn 3, and turn 20 under the new model. The first prompt to the new model is cold (as any model switch must be), but every prompt after that hits the cache cleanly.

Wire Transformation in Action

Foreign tool calls are completely stripped from assistant messages so foreign schema markers never reach the new provider. Foreign tool results are transformed into user narration parts:

[result of grep]
12|   export function collectShakeRegions(...)
15|   export function applyShakeRegion(...)
[elided 8140 chars of a foreign tool result — this turn is narration only and is no longer replayable as a live call]
98|   export default function shake(...)

The model gets the context it needs—the command that was run, the start of the output, and the tail—without paying the token tax for thousands of irrelevant middle lines.

Configuration and Mechanics

The extension lives in packages/model-switch-prune-omp/ in the omp-extensions repository.

It reads ~/.omp/agent/model-switch-prune.json:

{
  "mode": "elide",
  "debug": false
}

When combined with model-shift-omp, the handoff is seamless:

  1. model-shift-omp evaluates CEL rules at agent_end and changes the active session model.
  2. On the very next prompt, model-switch-prune-omp intercepts the context event, identifies the previous model’s turns as foreign, and yields a lightweight, cache-stable payload.

Source