Add a dedicated Chutes sidecar route without changing existing /v1/chat/completions behavior
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 55/100
Research direction
Start by tracing the existing POST /v1/chat/completions handler and its E2EE parse/decrypt and encrypt logic. Review how configuration and errors are handled, then add the explicitly gated Chutes routes and verify stream and non-stream behavior, replay protection, 503 misconfiguration responses, unchanged existing behavior, and absence of API-key leakage in logs.
Written by the indexing model from the issue text.
Description
Summary
We want to integrate Chutes models into vllm-proxy while preserving the current production-stable behavior for existing users.
Instead of modifying the current /v1/chat/completions path, add a dedicated Chutes route:
POST /v1/chutes/chat/completionsGET /v1/chutes/models
This lets upstream gateways (e.g., Redpill) continue receiving standard OpenAI-style /v1/chat/completions requests and selectively forward Chutes-bound traffic to the new sidecar route.
Goals
- Keep existing
/v1/chat/completionsbehavior unchanged - Minimize coupling/risk to current production path
- Add Chutes integration behind explicit route + config flag
Proposed behavior
Existing route (unchanged)
POST /v1/chat/completions
- continues to forward to local vLLM backend as today
New Chutes route
POST /v1/chutes/chat/completions
- reuses existing E2EE parse/decrypt logic on ingress
- forwards plaintext request to Chutes OpenAI-compatible endpoint over TLS:
${CHUTES_BASE_URL}/v1/chat/completions- with
Authorization: Bearer ${CHUTES_API_KEY}
- reuses existing E2EE encrypt logic on egress
New Chutes models route
GET /v1/chutes/models
- proxies
${CHUTES_BASE_URL}/v1/modelswith Chutes auth header
Config
CHUTES_ENABLED(default false)CHUTES_BASE_URL(defaulthttps://llm.chutes.ai)CHUTES_API_KEY(required if enabled)
Security note
This is not pure client-to-model cryptographic E2EE.
It is a practical TEE-mediated segmented model:
- Client ↔ Proxy: existing E2EE
- Proxy ↔ Chutes: TLS
- Plaintext is only visible inside trusted runtime boundaries
Acceptance criteria
- Existing route behavior remains unchanged
- Chutes route supports stream + non-stream
- Existing E2EE nonce/replay checks remain in effect
- Misconfig returns explicit 503 errors
- No API-key leakage in logs
- Dominant language
- Python
- Stars
- 0
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 1/5 Under an hour Newbie friendliness 62/100
-
Difficulty 1/5 Under an hour Newbie friendliness 70/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
topoteretes/cognee#5773 ·
Maintainers usually reply within 1 day
-
MiKTeX: a failed build reports no errors because texify --quiet switches the log to file:line: styleOpen
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
SublimeText/LaTeXTools#1718 ·