Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Question: should the Model Pool's judge go free, or be skipped, while the OpenRouter balance is low?

未关闭
  • 哪里不对？
#1,493 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
go
领域
ai-infra-agents, cli, documentation

调研方向

Read internal/credits/credits.go for LowCreditsUSD and Reading.Low, then inspect Candidates in internal/pool/judge/judge.go and the judge loop in cmd/codeaf/poolrecord.go. Review the low-credit guidance in internal/manual/chat/openrouter-credits.md and models-and-costs.md. Done requires a maintainer decision among the three options, matching behavior, and documentation that explains what happens to judges when credit is low.

由索引模型根据 Issue 内容生成。

描述

area:provider needs a ruling

A question for a ruling, not a defect. Nothing here behaves differently from how it was written. #1440 made a new situation possible, and nobody has decided how the Model Pool's judge should behave in it.

What is true today

  • #1440 (merged into dev as f101ad2e7) reads the OpenRouter balance. When the account is known to be low (credits.LowCreditsUSD, $0.50 or less remaining), every crew tier the person has not written a model for goes to a free model, so a near-zero account keeps working on free models.
  • After a task lands, the Model Pool records it and asks a judge model to score the seats that ran it (cmd/codeaf/poolrecord.go, the loop that calls judge.Judge). It asks up to judgeTries candidates in turn.
  • The judge is never free. judge.Candidates in internal/pool/judge/judge.go leaves out every row whose prompt and completion prices sum to zero, and every id ending in :free. Its comment gives the reason: a free row is rate-limited and answers with a 429 when a judge most needs an answer.

Together: on an account codeaf itself knows is almost out of credit, the task's own work runs on free models, but the judge pass after it still picks the cheapest paid model that qualifies. That call spends from the balance #1440 is trying to protect. When the balance is nearly empty, the call is also likely to fail with a 402 that nobody sees, and the pool then records a failed judge attempt instead of scores.

The options

  1. A free judge while the balance is low. Drop the free-row exclusion when the reading says Low, and accept 429s. A judge that fails costs nothing, and the pool already records a failed judge as such.
  2. Skip the judge pass while the balance is low. The landing is recorded unjudged, with a reason such as balance low. Nothing is spent, and the pool loses those scores.
  3. Keep it paid. The scores are worth the cents. If so, the manual should say that a judge call is paid even when the crew has gone free, and the low-credit notice should not imply that everything is now free.

Whichever is chosen, the words a person reads about low credit (internal/manual/chat/openrouter-credits.md and models-and-cost.md) should say what the judge does.

Where

  • internal/pool/judge/judge.go: Candidates. Search for A free row is a rate-limited.
  • cmd/codeaf/poolrecord.go: the judge loop. Search for judge.Candidates(models(), seats, judge.DefaultFloor).
  • internal/credits/credits.go: LowCreditsUSD and Reading.Low.

Raised while merging #1440 into santos/dev2 for #1410.

主要语言
Go
星标
115
派生
14
平均合并
9 小时 37 分钟
30 天内合并 PR
755

环境准备

我们还没有检查这个项目的环境配置文件。先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

Agent-Field/CodeAF 的其他 Issue

查看 Agent-Field/CodeAF 的全部 Issue

相似的 Issue

更多 Go Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。