docs: document model name-based routing for LLMInferenceService
Maintainers usually reply within 4 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 68/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Quiet
- Domain
- documentation
Research direction
Start by reading the related PR #5521 and the existing Envoy AI Gateway integration documentation. Document generic X-Gateway-Model-Name routing, configurable header names, LoRA adapter routing, shared-gateway deployments, llm-d BBR or gateway-native integrations, and -model-routing status addresses; done means each listed behavior is covered.
Written by the indexing model from the issue text.
Description
Summary
KServe v0.19 added model name-based routing for LLMInferenceService, enabling shared-gateway deployments. While the Envoy AI Gateway integration documents x-ai-eg-model header routing, the generic X-Gateway-Model-Name header routing is not documented.
What needs to be documented
- How model name-based routing works with the
X-Gateway-Model-Nameheader - Configurable header name
- How LoRA adapter requests are automatically routed through the same backend by dynamically expanding header matches
- Shared-gateway deployment pattern where a single gateway dispatches to multiple models
- Integration with components like llm-d BBR (IPP) or gateway-specific native features
-model-routingsuffix address types in status
Related PR
Since
KServe v0.19
- Dominant language
- MDX
- Stars
- 113
- Forks
- 200
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 9
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from kserve/website
-
Difficulty 2/5 1-2 days Newbie friendliness 72/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
kserve/website#725 · 1 comment ·
Maintainers usually reply within 4 days
Similar issues
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
buckai-observatory/Automated-Cloud-Removal-and-Seamless-Satellite-Image-Mosaicking-Pipeline#52 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 82/100
run-llama/llama_index#24163 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
btclib-org/btclib#3090 ·
Maintainers usually reply within 1 day
-
Translate the READMEOpendocumentation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
asasemahmed/openboard#68 ·
Maintainers usually reply within 1 day