Selectors

All selectors share a common constructor interface and the select_best() method.

Common Parameters

Parameter	Type	Description
agent	type	Agent class with __init__(self, models) and run(self, input_data) methods
models	Dict[str, List]	Maps node names to candidate model lists
eval_fn	Callable	(expected, actual) -> float score in [0, 1]
dataset	List[Tuple]	[(input_data, expected_answer), ...]
model_prices	Dict, optional	Custom pricing: {"model": {"input_price": x, "output_price": y}}
tracker	LLMTracker, optional	Custom tracker instance (e.g., with disk cache)

select_best()

results = selector.select_best(
    parallel=False,       # Use async parallel evaluation
    max_concurrent=20,    # Max concurrent API calls per combination
)

Returns a SelectionResults object.

Automatic cleanup

select_best() automatically calls tracker.stop() when it returns (or raises), flushing any cached data to disk.

---

Selector Classes

agentopt.model_selection.brute_force.BruteForceModelSelector

Selects the best model combination by evaluating all combinations.

Supports sequential and async-parallel evaluation via select_best().

agentopt.model_selection.random_search.RandomSearchModelSelector

Selects the best model combination from a random subset of candidates.

agentopt.model_selection.hill_climbing.HillClimbingModelSelector

Select models via stochastic hill climbing with random restarts.

agentopt.model_selection.arm_elimination.ArmEliminationModelSelector

Select models via successive arm elimination.

agentopt.model_selection.epsilon_lucb.EpsilonLUCBModelSelector

Select models via epsilon-optimal LUCB.

agentopt.model_selection.threshold_successive_elimination.ThresholdBanditSEModelSelector

Select models via threshold-based successive elimination.

agentopt.model_selection.lm_proposal.LMProposalModelSelector

Model selector where an LLM proposes the single best combination.

agentopt.model_selection.bayesian_optimization.BayesianOptimizationModelSelector

Select models via Bayesian optimization.