Reduce LLM Agent Token Usage: 7 Proven Cost Optimization Techniques

Token cost for an AI agent isn't dominated by the user's question — it's dominated by everything the agent re-sends on every turn: system prompts, tool definitions, and accumulated tool output. Here's where the actual savings are.

Step 1: Cache the Static Parts of Every Prompt

System instructions, tool schemas, and long reference documents rarely change between calls, but without caching they're billed as fresh input tokens on every single turn.

python

Cached input tokens are typically discounted around 90% versus fresh input tokens on the next call within the cache TTL window (usually 5 minutes, extendable to 1 hour). For an agent loop that calls the same system prompt and tool definitions dozens of times per task, this is the single highest-leverage change available.

Step 2: Truncate Tool Output Before It Re-Enters Context

A file read or search result can return thousands of tokens, and once it's in the conversation history, it gets re-sent on every subsequent turn — not just the turn it was fetched on.

python

A 20-turn agent run that never truncates a 5,000-token search result effectively pays for that same result 20 times over. Truncating or summarizing tool output immediately after use, rather than keeping the raw payload in context indefinitely, is usually a bigger win than any prompt-level optimization.

Step 3: Summarize Conversation History Past a Threshold

Instead of letting context grow unbounded, compact older turns into a summary once the transcript crosses a fixed size.

python

This keeps token usage roughly flat across a long-running agent session instead of scaling linearly with turn count.

Step 4: Route Subtasks to Smaller Models

Not every step in an agent's workflow needs the most capable (and most expensive) model. Classification, extraction, and formatting subtasks are usually handled correctly by a smaller model at a fraction of the cost.

python

Per-token pricing between model tiers commonly differs by 5-10x, so routing high-volume, low-ambiguity subtasks to a cheaper model while reserving the largest model for planning and judgment calls has an outsized effect on blended cost.

Step 5: Trim Tool Schemas and System Prompts

Every tool definition and every line of the system prompt is paid for on every single agent turn — including ones the current task never uses. Removing unused tools from the active toolset for a given task, and trimming verbose instructions that don't change model behavior, reduces this fixed per-call overhead.

python

Step 6: Measure Before and After

python

Tracking cache hit rate and input token count per turn over time is what tells you whether an optimization actually worked, rather than assuming it did — a change that looks correct in code can fail to reduce cost if, for example, cache keys change on every call due to a dynamic timestamp in the system prompt.

Key Takeaways

Prompt caching and tool-output truncation are the two highest-leverage token optimizations for most agents because they target the parts of context that repeat or accumulate across every turn, model routing cuts blended cost by reserving expensive models for genuinely hard subtasks, and measuring actual token usage per turn is the only way to confirm an optimization is working rather than assuming it.

Frequently Asked Questions

What's the fastest way to cut LLM agent token costs without changing behavior?

Enable prompt caching on the static parts of your prompt — system instructions, tool definitions, and any long reference document that doesn't change between calls. Anthropic's cache_control and OpenAI's automatic caching both discount cached input tokens heavily (often 90% off), and since these tokens are identical on every turn, caching is pure savings with zero behavior change.

Why do tool outputs consume more tokens than the actual conversation?

A single file read, search result, or API response can easily return thousands of tokens of raw text, all of which gets re-sent as input on every subsequent turn once it's in context. A 20-turn agent loop that never truncates old tool output effectively re-pays for that same data 20 times, which is why truncation and summarization of tool results matter more than trimming the prompt itself.

Should every agent task use the largest, most capable model?

No — model routing (using a smaller model for classification, extraction, or formatting subtasks, and reserving the largest model for multi-step planning and ambiguous judgment calls) is one of the highest-leverage cost optimizations available, since per-token cost differences between model tiers are often 5-10x.

How do I know if my context is growing unnecessarily during a long agent run?

Log the token count of the full context sent on every turn. If it grows roughly linearly with the number of turns rather than plateauing, your agent is accumulating stale tool output or conversation history that should be summarized or dropped once it's no longer relevant to the current step.