Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Operator limits: managed model tiers as operator configuration

Closed
  • Is something wrong?
#280 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
8/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
typescript
Domain
api, backend

Research direction

This depends on #269, #275, #276 and #279, so read those and the parent #268 first; the capability module from #269 and the derived capabilities from #279 are used here. The new pieces are the managed_model_tiers instance setting, the GET|PUT /api/operator/v1/model-tiers route, the modelTier review setting, and the claim-time substitution for managed jobs. Done means test cases T01 to T11 pass, including the check that no tier model id appears in any workspace-facing answer.

Written by the indexing model from the issue text.

Description

area: app type: feature

Part of #268. Depends on #269, #275, #276 and #279.

Problem

A workspace on the operator's key should not have to pick among hundreds of models, and the operator should be free to change the models behind a choice without every workspace re-choosing. The operator wants three named levels of quality, Low, Medium and High, whose models it configures and may change, and which workspace members choose without ever seeing a model name.

Behavior

  • New instance setting managed_model_tiers: for each of low, medium, high, an optional diff model and an optional dynamic model (OpenRouter ids, validated with the model-id rule of the review settings). Written and read only through the operator API: GET|PUT /api/operator/v1/model-tiers. The instance admin portal shows only whether each tier is configured, not the ids.
  • A dynamic model is validated on write with the capability module of #269: unsupported or unknown is refused (DYNAMIC_MODEL_TOOLS_UNSUPPORTED, DYNAMIC_MODEL_UNKNOWN); here an unverifiable model is refused too (DYNAMIC_MODEL_UNVERIFIED), because this is operator configuration, not a user's choice.
  • New review setting modelTier (low, medium, high; default medium) at workspace default and project level, with the same layering as the other settings. It only has an effect when the workspace's key mode is managed.
  • At claim, for a managed job: the tier of the project's effective settings is mapped to the configured model of the job's review kind; that model replaces model or dynamicModel of the settings, and the capabilities of #279 are derived for it. A tier without a model for the kind fails the job closed with managed_tier_unavailable (no silent move to another tier). For an own or unlimited workspace nothing changes: model and dynamicModel apply, modelTier is ignored.
  • The token budget (maxTokens) is clamped at claim to the tier model's context window when known; the settings page does not validate maxTokens against a model for a managed workspace (the model is not known to the user).
  • The usage record (#272) stores the tier name in model_tier.
  • The model id of a tier never leaves the server, except to the operator API and the runner (which needs it to call the provider): it is not in any workspace-facing API answer, page, Runs view, usage view, pull request comment, log line that a workspace member can see, or error copy. The settings API/pages for a managed workspace return the tier, not the model.

Scope

Instance setting + operator API route + contract, review settings field and resolution, claim, usage record column fill, docs, tests. The settings form change is #283.

Out of scope

Which models back which tier (operator data, not in the repository), per-tier limits, the UI (#283).

Test cases

ID kind level case expected
T01 behavior integration PUT/GET model tiers with the operator token stored; GET returns what was PUT; instance admin portal shows only configured/not
T02 negative integration PUT a Dynamic model without tool choice refused DYNAMIC_MODEL_TOOLS_UNSUPPORTED, nothing stored
T03 negative integration PUT a Dynamic model whose metadata is unreachable refused DYNAMIC_MODEL_UNVERIFIED
T04 behavior integration managed job, project tier high, high.diff configured the claim's model is high.diff; usage record has tier high
T05 behavior integration managed Dynamic job claim carries the tier's dynamic model and its capabilities
T06 negative integration managed job whose tier has no model for the kind job fails managed_tier_unavailable; no fallback to another tier
T07 regression integration own or unlimited workspace with modelTier set model/dynamicModel used, tier ignored
T08 security integration a member's reads of settings, runs, usage, job comments no model id of any tier appears (search of all workspace-facing answers)
T09 behavior unit maxTokens above the tier model's context window clamped at claim
T10 behavior integration default tier medium when nothing is set; project overrides workspace default
T11 security integration operator API absent (token unset) the route is a missing path
Dominant language
TypeScript
Stars
0
Forks
1
Avg merge
1h 15m
Merged PRs (30d)
217

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ExpidetGames/code-reviewer

All issues in ExpidetGames/code-reviewer

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.