Request permission to use InferenceX benchmark results in ConfigIQ
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
- 难度
- 2/5
- 预计耗时
- 1 小时以内
- 新手友好度
- 18/100
- Issue 类型
- 文档
- 描述清晰度
- 基本清楚
- 活跃度
- 活跃
- 领域
- data, documentation
调研方向
This is a licensing and permission question rather than a code change: the issue asks whether SemiAnalysis approves private storage, public attributed display, and weekly API refresh of benchmark results, and it points to the API docs and Section 7 of the general terms. A newcomer would read the linked terms and API documentation and check for any stated license, attribution or rate-limit rules. Done means a maintainer has answered the permission question in the thread.
由索引模型根据 Issue 内容生成。
描述
Hi InferenceX team,
The ConfigIQ team is developing a Compare page that shows measured inference performance alongside AISimulators predictions across concurrency levels.
We would like to add InferenceX as a separately labeled measurement source on ConfigIQ’s comparison page. If approved, we propose to:
- Fetch selected published single-turn 8192-input / 1024-output-token results using
/api/v1/availability,/api/v1/views/options, and/api/v1/benchmarks. - Normalize the selected results and store them in a private Hugging Face dataset so ConfigIQ can load configuration options.
- Display selected measurements on ConfigIQ’s public comparison charts alongside our measurements and AISimulators predictions. We would clearly attribute the data to InferenceX/SemiAnalysis and link to the original benchmark runs. The private dataset would not be offered as a public download.
- Refresh the private dataset from your API approximately once a week.
The API documentation describes accessing and exporting data for analysis, but we could not determine whether it permits this specific workflow. Section 7 of SemiAnalysis’s general terms also raises a question about scheduled API collection.
Would SemiAnalysis approve these uses: private storage of a normalized subset, public display of selected measurements with attribution, and a weekly API refresh? Are there license, attribution, or request-frequency requirements we should follow? Thanks.
- 主要语言
- Python
- 星标
- 2k
- 派生
- 354
- 平均合并
- 2 天 40 分钟
- 30 天内合并 PR
- 431
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
SemiAnalysisAI/InferenceX 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 68/100
SemiAnalysisAI/InferenceX#2125 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 68/100
SemiAnalysisAI/InferenceX#1587 ·
维护者通常 1 天内回复
-
[experimental chore/agentx-v0.2-aiperf-testing branch] agentic build_replay_cmd: missing --tokenizer-trust-remote-code breaks Kimi-K2.5 (and other custom-code tokenizer models) / agentic build_replay_cmd 缺少 --tokenizer-trust-remote-code 导致 Kimi-K2.5 等自定义 tokenizer 模型失败可能重新可做 @sahil-bhatti-ai 于 173 天前认领，目前没有进行中的 PR。 未关闭
难度 1/5 1 小时以内 新手友好度 78/100
SemiAnalysisAI/InferenceX#1369 · 3 条评论 ·
维护者通常 1 天内回复
-
难度 1/5 1-3 小时 新手友好度 76/100
SemiAnalysisAI/InferenceX#1359 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 47/100
SemiAnalysisAI/InferenceX#4533 ·
维护者通常 1 天内回复
查看 SemiAnalysisAI/InferenceX 的全部 Issue
相似的 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 66/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
yt-project/yt#5624 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
-
bug
难度 2/5 1-3 小时 新手友好度 82/100
NousResearch/hermes-agent#148376 ·
维护者通常 1 天内回复
-
Analysis
难度 2/5 1-3 小时 新手友好度 72/100
SACGF/variantgrid#2615 ·
维护者通常 1 天内回复