Skip to content

Client-side agent optimization

Choosing the right model for your agents can save 100x money and time

AgentOpt evaluates model combinations across your full agent pipeline, measures accuracy, cost, and latency, and recommends a configuration or a set of accuracy–cost tradeoffs.

Get started View on GitHub

Exponential
combo space
Smart search
on best model combination
0
code changes
any
agent framework

The Problem

Your agent has multiple steps — planning, reasoning, tool use, synthesis. Each step could use a different model. With 5 candidate models across 3 steps, that's 125 combinations. Testing them manually is impractical. Picking blindly leaves performance (and money) on the table.

The Solution

Give AgentOpt your agent and a small evaluation dataset (~100 samples). Gittins selection searches the model combination space and returns one recommended configuration by default. Set objective_mode="multi" to inspect accuracy/cost alternatives and choose a deployment preference.

from agentopt import ModelSelector

selector = ModelSelector(
    agent=MyAgent,
    models={
        "planner": ["gpt-4o", "gpt-4o-mini", "gpt-4.1-nano"],
        "solver":  ["gpt-4o", "gpt-4o-mini", "gpt-4.1-nano"],
    },
    eval_fn=eval_fn,
    dataset=dataset,
)

results = selector.select_best(parallel=True)
results.print_summary()

    Model Selection Results
    --------------------------------------------------------------------------
    Rank  Model                                       Accuracy  Latency    Price
    --------------------------------------------------------------------------
>>>  1  planner=gpt-4.1-nano + solver=gpt-4.1-nano   100.00%    0.85s  $0.000420
     2  planner=gpt-4o-mini + solver=gpt-4o-mini      100.00%    1.20s  $0.002372
     3  planner=gpt-4o + solver=gpt-4o                 100.00%    2.70s  $0.014355
    ...

Why AgentOpt

  • Non-Intrusive

    Define your agent as a class. No framework adapters, no SDK wrappers, no code changes to your agent internals.

  • Framework-Agnostic

    Works with OpenAI, LangChain, LangGraph, CrewAI, LlamaIndex, AG2 — any framework that calls LLMs over HTTP.

  • Smart Search

    Gittins selection by default, with random, UCB, and Bayesian baselines. Allocate an evaluation budget across model configurations and questions.

  • Automatic Tracking

    Transparently intercepts all LLM calls to measure tokens, latency, and cost. No manual instrumentation needed.

  • Response Caching

    Identical LLM calls are cached in-memory and on disk (SQLite). Re-running experiments is instant and free.

  • Parallel Evaluation

    Evaluate model combinations concurrently with configurable concurrency limits. Get results faster.

How It Works

graph LR
    A["Your Agent"] --> B["httpx layer"]
    B --> C["LLM API"]
    B --> D["AgentOpt Interceptor"]
    D --> E["Track tokens, latency, cost"]
    D --> F["Cache responses"]
    E --> G["Evaluate & Rank"]
    G --> H["Winner or accuracy/cost recommendations"]

AgentOpt patches httpx at the transport level — the same HTTP library used by every major LLM SDK. Your agent code stays untouched. AgentOpt silently records every LLM call, caches responses, and aggregates metrics per model combination.

Learn more about the architecture

Selection Algorithms

Method Objective Strategy
Auto / Gittins Single or multi Cost-aware allocation with posterior recommendations
Random Single Random configuration order
UCB Single Configuration mean plus an exploration bonus
Bayesian Single Gaussian-process surrogate with expected improvement
Random configurations Multi Complete configurations in random order
Random questions Multi Shared random question order across all configurations

Compare algorithms in detail

Get Started