Skip to content

ADR-0012: Cost as framework primitive

  • Status: Accepted
  • Date: 2026-04-12

Context

Agent systems spend money on every invocation — LLM tokens, external API calls, DB queries. Costs are volatile, hard to attribute, and easy to accidentally amplify (one agent calling another in a loop). Without framework-level support:

  • Cost is observed post-hoc, in billing dashboards, after damage is done.
  • Budgets can only be enforced at tenant level, not per-capability / per-principal.
  • Cost-aware agent planning is impossible (no introspection on how expensive a capability is).
  • Anomaly detection is manual, reactive.

Options:

  1. No framework cost support. Devs instrument their own. Ignores the agent-system reality; most apps underinstrument until it hurts.
  2. Optional middleware library. Available but not default. Better; still ignored by most apps.
  3. Optional-but-strongly-suggested. Framework includes it; opt-in per capability.
  4. Always-on primitive. Every capability has a cost envelope whether it wants to or not.

Decision

Cost is a framework primitive. Every capability invocation opens a cost envelope and closes it with actuals.

  • @capability(cost_estimate={"tokens": n, "usd": x, "latency_ms": y}) is declared per capability (defaults to zero if not specified).
  • Pre-handler: CostAccountant.open(cap, principal, est) reserves budget. Budget-exhausted → 429.
  • Handler runs and may update envelope mid-flight (e.g., after an LLM call).
  • Post-handler: CostAccountant.close(env, actual) records final cost and emits receipt.
  • Budgets enforced at three levels: per-capability, per-principal-per-capability, per-principal-global.
  • Cost receipts attached to response envelope + PROV-O graph.
  • Anomaly hook: if actual > estimate * threshold (default 3×), fire configurable alert.

Cost envelopes are NOT optional. Apps that want to ignore costs set estimates to zero; the envelope still opens and closes but records zero.

Consequences

Positive

  • Agent planning can use costs. Capability manifests expose estimates; planners prefer cheaper tools.
  • Budget safety. Runaway loops blocked at framework boundary, not at credit-card boundary.
  • Observability out of the box. Every app gets cost metrics without extra code.
  • Honest design conversations. Developers declaring cost estimates forces thinking about it.
  • Aligns with author's cost-discipline preference (documented in memory).

Negative

  • Per-invocation overhead. Two small kernel calls per invocation. Benchmarked at < 0.5 ms; within NFR-Perf1 budget.
  • Requires dev effort to populate estimates accurately. Mitigated by:
  • Defaulting to zero (safe, shows no cost).
  • trails doctor flags capabilities whose actuals diverge > 3× from estimates for ≥ 100 invocations.
  • trails sim can populate estimates empirically.
  • Budget misconfiguration can block legitimate work. Mitigated by sensible defaults in trails.toml and clear error messages on budget-exceeded (429 + BudgetStatus).

Non-consequences

  • Cost envelopes don't require a payment system integration — they're accounting primitives, not billing.
  • Apps that bill users can layer their billing on top of envelopes (Stripe-backed CostAccountant implementation in v2).

Revisit conditions

  • If framework overhead on cost is measured at > 1 ms consistently, optimize kernel-side (batched writes).
  • If application feedback indicates cost primitives are consistently ignored (all zero estimates), re-evaluate: the primitive exists but isn't serving its purpose; find the UX defect.