[OHE] Per-User LLM Gateway Attribution in Bifrost
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- python
- Domain
- api, authentication, backend
Research direction
Start by tracing LLM Profile extra_headers and the existing forward_client_headers_to_llm_api mechanism, then identify where authenticated user and organization context is available for interactive and automated requests. Done means supported variables are validated and resolved per request, forwarded through the gateway, and covered for the stated attribution and unresolved-variable requirements.
Written by the indexing model from the issue text.
Description
Problem
Enterprise customers often route OpenHands inference through an existing LLM gateway like Bifrost for centralized governance, routing, cost management, and access control.
Today, OpenHands typically authenticates to the external gateway using a single shared virtual key configured by an administrator. As a result, the downstream gateway sees all OpenHands inference as originating from the same identity.
This creates two problems:
- Limited attribution and governance
- Gateway administrators cannot easily attribute token usage or cost to individual OpenHands users.
- Existing per-user budgets, policies, rate limits, and reporting configured in the gateway cannot be applied.
- Limited gateway personalization
- Some gateways use the incoming identity or virtual key to determine which resources a user can access, including models, routing policies, or MCP servers.
- When all OpenHands traffic shares the same identity, these user-specific policies cannot be preserved.
OpenHands already supports forwarding arbitrary extra_headers from an LLM Profile through its built-in LiteLLM gateway. When header forwarding is enabled, those headers reach the downstream gateway.
However, these headers are currently static values defined on an LLM Profile. Supporting per-user attribution therefore requires creating and maintaining a separate profile for each user.
Goal of this ticket
GOAL: Generate solution options for supporting Per-user Usage and Budget attribution in Bifrost
This should allow an external LLM gateway to apply its existing:
- Usage and cost attribution
- User-level budgets and rate limits
- Routing and access policies
- MCP or tool entitlements
- Audit and reporting policies
Solution Options
Option 1: Dynamic extra_headers added to LiteLLM and passed into Bifrost
Extend LLM Profile extra_headers to support OpenHands context variables that are resolved dynamically for each inference request (e.g., {{user.id}})
Bifrost extra headers: https://docs.getbifrost.ai/providers/request-options#extra-headers-x-bf-eh
Example configuration:
{
"x-bf-eh-user-id": "{{user.id}}"
}
Pros:
- Provides per-user usage attribution in Bifrost (against a shared Virtual Key)
Cons:
- Does not enforce user-level budgets given all usage is still against a shared Virtual Key
Option 2: Support an Org-setting that allows users to bring their own LLM Key
Pros:
- Provides a way for both usage tracking and budget enforcement to occur on the user's specific LLM Key
Cons:
- Hard to scale — every user needs to copy/paste their own Virtual Key manually given to them from an Admin
- LLM Profiles and Org Settings have historically been brittle
- Dominant language
- Python
- Stars
- 9
- Forks
- 17
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 396
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenHands/enterprise
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
OpenHands/enterprise#1269 ·
Maintainers usually reply within 1 day
-
type: test
Difficulty 1/5 Under an hour Newbie friendliness 84/100
OpenHands/enterprise#816 ·
Maintainers usually reply within 1 day
-
type: test
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
OpenHands/enterprise#814 ·
Maintainers usually reply within 1 day
-
type: test
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
OpenHands/enterprise#810 ·
Maintainers usually reply within 1 day
-
type: test
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
OpenHands/enterprise#811 ·
Maintainers usually reply within 1 day
All issues in OpenHands/enterprise
Similar issues
-
[Bug]: reme CLI crashes when the running server used a relative config pathPossibly taken @nurlanmnn claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
agentscope-ai/ReMe#770 ·
Maintainers usually reply within 1 day
-
Pi: paused model calls are reported as "not signed in" when the model's provider comes from an extensionPossibly taken @balcsida claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
microsoft/TokenEconomizer#288 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 67/100
sickn33/agentic-awesome-skills#2437 ·
Maintainers usually reply within 1 day
-
site-bug triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ethereum/consensus-specs#5846 ·
Maintainers usually reply within 1 day