Why AI Agents Waste Tokens (And How Context Optimization Fixes It)

When you chat with an LLM in ChatGPT or Claude, context consumption is small. A human types a couple of sentences, the model replies with a paragraph, and conversation history grows linearly: O(N).

Autonomous agents are completely different. In an agent execution loop, token consumption grows quadratically: O(N²).

Here is why:

At step 1, the prompt contains only the system instructions and the task. But at step 2, it sends step 1's thought, tool action, and tool output. By step 10, the prompt contains the entire accumulated history of every tool execution:

• A 500-line git diff from step 3.
• A 200-line stack trace from step 5.
• An entire 400-line file inspected at step 7.
• Directory listing results from step 8.

Every previous observation is sent back to the model on every single turn. By step 15, an agent performing a routine code edit can easily send 80,000 to 120,000 tokens per API call. At scale, this burns through API budgets fast and slows down response times.

The Math Behind the Quadratic Blowup

Let's break down the actual arithmetic.

On each step k, the input tokens sent to the model equal:

InputTokens(k) = SystemPrompt + TaskDescription + Sum(Previous Steps 1 to k-1)

If your average tool output (compiler log, file read, diff) is M tokens (typically 1,500 to 4,000 tokens):

Step 1:  S
Step 2:  S + M
Step 3:  S + 2M
...
Step N:  S + (N - 1)M

The cumulative tokens billed across an N-step task is the sum:

Total Tokens = N * S + M * (N * (N - 1) / 2)  --> O(N²)

Notice the difference: at 30 steps, an unoptimized agent consumes over 1.1 million tokens for a single task.

Four Strategies to Tame the Context Curve

Here are four production-proven strategies to bring token usage back under control.

1. Ephemeral Tool Observations (Clear on Next Step)

When an agent reads a 300-line file to understand a function, it needs that content for that specific turn. It rarely needs that entire 300-line raw dump 10 steps later.

Replace large historical file dumps with concise reference stubs once the step completes:

interface AgentStep {
  stepIndex: number;
  tool: string;
  args: any;
  result: string;
  isEphemeral?: boolean;
}

export function compactAgentHistory(history: AgentStep[], currentStep: number): any[] {
  return history.map((step) => {
    // If the tool was an exploratory read and happened more than 2 turns ago:
    if (step.tool === "read_file" && step.stepIndex < currentStep - 1) {
      return {
        role: "tool",
        tool_call_id: `call_${step.stepIndex}`,
        // Replace 400 lines of file content with an indexed 1-line reference
        content: `[Read file: ${step.args.filePath} (${step.result.split('\n').length} lines) - content cleared from active context]`
      };
    }
    return {
      role: "tool",
      tool_call_id: `call_${step.stepIndex}`,
      content: step.result
    };
  });
}

2. Truncate Command Logs (Head + Tail Windowing)

When an agent runs a build command or test suite, the compiler output might be 1,000 lines. The model only needs the first few lines to see what ran, and the last 30 lines where the actual error or failure happened:

export function smartTruncateLog(rawLog: string, maxLines = 60): string {
  const lines = rawLog.split("\n");
  if (lines.length <= maxLines) return rawLog;

  const head = lines.slice(0, 15).join("\n");
  const tail = lines.slice(-35).join("\n");
  const omitted = lines.length - 50;

  return `${head}\n\n[... ${omitted} lines omitted. Use specific search tools to inspect full output ...]\n\n${tail}`;
}

3. AST Pruning Before Injection

Instead of passing entire source files to the LLM, parse the file into an Abstract Syntax Tree (AST) and pass only the relevant class signatures, method declarations, and type definitions. Pass the full function implementation only for the specific method the agent is modifying.

4. Rolling Checkpoints (Summarization Boundaries)

Every 10 steps, summarize the past 10 steps into a structured checkpoint:

• What was accomplished: e.g. "Identified bug in auth controller; modified JWT expiry calculation."
• What failed: e.g. "Test suite failed on test_refresh_token; 401 Unauthorized."
• Current hypothesis: e.g. "Need to update the refresh token rotation handler in auth.service.ts."

Wipe the raw history of the previous 10 steps and replace them with this single checkpoint. This caps maximum context size at a fixed ceiling, turning quadratic growth back into linear O(N).

Production Results

Applying ephemeral observation pruning and head/tail log truncation directly changes the economics of running agents:

• Context payload per step: Drops from 90,000+ tokens to an average of ~18,000 tokens.
• Latency per turn: Decreases significantly because the model generates completions against a compact context window.
• Task completion rate: Actually improves, because the model avoids distraction from obsolete compiler warnings generated early in the run.

Managing context isn't just an optimization—it is what separates a toy demo from an agent system that can run reliably in production.