Cost Management
Running AI workers costs money — every token processed incurs a charge. WRM's cost management features give you control over spending through budgets, visibility through cost tracking, and resilience through provider failover.
Budget hierarchy
Budgets in WRM follow a hierarchical override pattern that mirrors how worker configuration works:
| Level | Scope |
|---|---|
| Role | Applies to all workers filling a given role |
| Worker definition | Applies to all instances of a specific worker definition |
| Worker instance | Applies to a single worker instance |
Budget settings are optional at every level. The system checks from the most specific level upward — if a worker instance has a budget set, that budget applies. If not, the system checks the worker definition, then the role. The first non-empty budget wins.
This means you can set a sensible default at the role level and override it where needed. For example, a backend role might have a $50/day budget, but a specific worker definition optimised for large refactoring tasks might have a higher $100/day limit.
Budget fields
Each level in the hierarchy can specify:
| Field | Purpose |
|---|---|
| Budget (USD) | The maximum spend allowed per budget period |
| Budget period | The time window over which spending is measured |
| Max tokens | An optional cap on tokens per run (independent of cost) |
Budget periods
The budget period controls how spending is aggregated:
| Period | How it works |
|---|---|
| DAILY | Spending resets at the start of each day |
| WEEKLY | Spending resets at the start of each week (Monday) |
| MONTHLY | Spending resets at the start of each month |
| PER_RUN | Each individual run is checked against the budget independently |
Before each dispatch, the system calculates how much has been spent in the current period and compares it against the budget. If the budget has been reached, the dispatch is blocked with a BUDGET_EXCEEDED status.
Budget checks happen before dispatch — the system prevents a run from starting if the budget is already exhausted. It does not terminate a run mid-execution if costs exceed the budget during the run. The max_tokens field provides a per-run safety net by capping how many tokens a single run can consume.
Provider failover
AI providers occasionally experience outages, rate limits, or degraded performance. WRM handles this through provider failover — if the preferred provider fails, the system automatically tries the next provider in the worker's model preference list.
How failover works
Worker definitions specify model preferences in priority order (see Providers, Models, and Credentials). When a dispatch fails with a provider error, the system:
- Records the failure in the agent run steps
- Checks the next model preference in the priority list
- Attempts the dispatch with the alternative provider
- Records the actual model and provider used in the agent run
If all providers in the preference list fail, the run fails with the original error.
Circuit breaker
To avoid repeatedly hitting a provider that is known to be failing, WRM implements a circuit breaker pattern. Each provider can have a failover policy that defines:
| Setting | What it controls |
|---|---|
| Max failures | How many recent failures trigger the circuit breaker |
| Window (minutes) | The time window over which failures are counted |
| Error patterns | Which error types count as failures (e.g. rate limits, 503 errors, timeouts) |
Before attempting a provider, the system queries recent agent runs to count how many matching failures occurred within the window. If the failure count meets or exceeds the threshold, the provider is skipped without attempting a call.
The circuit breaker uses the agent run history itself as its data source — there is no separate state to manage. Recent failures in the agent_runs table are queried directly, which means the circuit breaker automatically recovers as failed runs age out of the time window.
Configure failover policies based on your experience with each provider. A provider with frequent rate limits might use a lower max_failures threshold and shorter window, while a generally reliable provider might use higher thresholds to avoid premature skipping.
Cost tracking and reporting
Every agent run records its cost (total USD), token counts (input and output), and duration. This data accumulates over time and can be queried to answer operational questions:
By worker or role
See how much each worker or role is spending:
- Total cost per role over a time period
- Average cost per run by worker definition
- Cost trends over time — is spending increasing or decreasing?
By provider
Understand your provider spend:
- Total cost per provider
- Cost per token by provider (useful for comparing pricing)
- Failure rate by provider — are you paying for runs that fail?
By work item
Track cost at the project level:
- How much did a specific epic or feature cost in agent runs?
- What is the average cost per work item type (bug fix vs. feature vs. documentation)?
Connecting cost to quality
When combined with eval scorer results, cost data becomes more meaningful:
- Cost per successful CI pass — how much do you spend per unit of passing work?
- Issue adherence score vs. cost — are more expensive runs producing better results?
- Model comparison — which model gives the best quality per dollar?
Concurrency control
WRM integrates with the Foundation (FND) job system to control how many workers can run simultaneously. Each dispatch creates an FND job record, and the job system enforces a configurable maximum on concurrent processes.
This prevents resource exhaustion — without concurrency limits, a large batch of work items could trigger dozens of simultaneous dispatches, overwhelming API rate limits and local compute resources.
The concurrency limit is configured through the FND worker manager, not through WRM directly. See the Foundation job scheduling documentation for details on job scheduling and concurrency.