Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Budget-aware multi-model routing with per-task cost tracking

未关闭
  • 哪里不对？
#598 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
28/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
停滞
技术栈
rust
领域
ai, api, backend

调研方向

Start in the affected terraphim_service, terraphim_types, terraphim_multi_agent, and terraphim_mcp_server crates, then review the event-sourcing dependency in #597 and the existing terraphim-llm-proxy token headers. Done means task budgets meter costs, enforce limits, fall back through model chains, persist cost events, and expose the current and historical MCP budget resources.

由索引模型根据 Issue 内容生成。

描述

architecture enhancement

Summary

Add per-task cost tracking with fallback chains and hard spending limits to LLM routing. Each agent operation gets a budget allocation; if the primary model is too expensive or unavailable, fall back to cheaper alternatives automatically.

Motivation

Inspired by Ouroboros per-round cost tracking with per-worker budget allocation. Current state:

  • terraphim-llm-proxy routes between multiple models but has no per-task budget tracking
  • genai (terraphim fork) handles LLM calls but does not meter usage or cost
  • The multi-model judge pattern (quick -> deep -> tiebreaker) needs budget awareness: only escalate to expensive models when cheaper ones disagree
  • No spending limits exist -- a runaway agent loop could consume unbounded API credits

Implementation Plan

1. Budget Struct
pub struct TaskBudget {
    pub max_cost_usd: f64,
    pub model_chain: Vec<ModelConfig>,  // ordered by preference (cheapest first)
    pub spent_usd: f64,
    pub request_count: u32,
    pub created_at: jiff::Timestamp,
}

pub struct ModelConfig {
    pub model_id: String,
    pub cost_per_1k_input: f64,
    pub cost_per_1k_output: f64,
    pub max_latency_ms: u64,
    pub fallback_on: Vec<FallbackReason>,  // Timeout, RateLimit, BudgetExceeded, EmptyResponse
}
2. Cost Tracking in terraphim_service
  • After each LLM call, compute cost from token counts + model pricing
  • Accumulate in TaskBudget.spent_usd
  • If spent_usd >= max_cost_usd, reject further calls for this task
  • Log cost events to event store (#597)
3. Fallback Chain Logic
  • Try models in model_chain order
  • On failure (timeout, rate limit, empty response, budget exceeded for this model tier), try next model
  • If all models exhausted, return error with cost summary
4. Integration with terraphim-llm-proxy
  • Proxy already reports token counts in responses
  • Add cost metadata to proxy response headers (cost_usd, model_used, tokens_in, tokens_out)
  • Agent-side budget tracking consumes these headers
5. MCP Resource Exposure
  • Expose /budget/current as MCP resource showing active task budgets
  • Expose /budget/history for cost analysis over time

Affected Crates

  • terraphim_service (primary -- add budget tracking to LLM routing)
  • terraphim_types (add TaskBudget, ModelConfig types)
  • terraphim_multi_agent (wire budget into agent task execution)
  • terraphim_mcp_server (expose budget resources)

Dependencies

  • #597 Event sourcing (for cost event persistence)

Estimated Effort

~4 hours (mostly plumbing between terraphim_service and proxy response headers)

Part of

Epic #595

主要语言
Rust
星标
64
派生
5
平均合并
1 小时 17 分钟
30 天内合并 PR
2

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

terraphim/terraphim-ai 的其他 Issue

查看 terraphim/terraphim-ai 的全部 Issue

相似的 Issue

更多 Rust Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。