docs: document latency predictor sidecar and LoRA affinity scorer auto-injection
Maintainers usually reply within 4 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 68/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Quiet
- Domain
- documentation
Research direction
Start with the existing scheduler and LoRA adapter documentation, then review related PRs #5678 and #5655 for the latency predictor sidecar and LoRA affinity scorer behavior. Document the listed enablement, injection, override, and scheduling details, including a minimal scheduler configuration example. Done means both features and the existing LoRA documentation clearly explain their automatic behavior.
Written by the indexing model from the issue text.
Description
Summary
KServe v0.20 added two auto-injection features for LLMInferenceService that are not documented:
-
Latency predictor sidecar: When the
predicted-latency-producerplugin is detected in the EPP config, the controller auto-injects training-server and prediction-server sidecar containers via a well-knownLLMInferenceServiceConfig. -
LoRA affinity scorer: When LoRA adapters are configured, the
lora-affinity-scorerplugin is automatically injected into the default scheduling profile, scoring endpoints by whether the requested adapter is already active on each pod.
What needs to be documented
Latency predictor sidecar
- How to enable: add
predicted-latency-producerplugin to EPP inline config - Auto-injection of
kserve-config-llm-scheduler-latency-predictorwell-known config - What gets injected: training-server and prediction-server sidecar containers
- How to override defaults via custom
LLMInferenceServiceConfigandbaseRefs - Example minimal scheduler config
LoRA affinity scorer
- Automatic injection when
spec.model.lora.adaptersis configured - What it does: scores endpoints by active adapter availability
- Injection details: added to
pluginslist,pluginRefwithweight: 4in default scheduling profile - No manual configuration required — it's fully automatic
- Update existing LoRA adapter documentation to mention this behavior
Related PRs
- https://github.com/kserve/kserve/pull/5678 (latency predictor sidecar)
- https://github.com/kserve/kserve/pull/5655 (LoRA affinity scorer)
Since
KServe v0.20
- Dominant language
- MDX
- Stars
- 113
- Forks
- 200
- Avg merge
- 5d 13h
- Merged PRs (30d)
- 9
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from kserve/website
-
Difficulty 2/5 1-2 days Newbie friendliness 72/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
kserve/website#725 · 1 comment ·
Maintainers usually reply within 4 days
Similar issues
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
buckai-observatory/Automated-Cloud-Removal-and-Seamless-Satellite-Image-Mosaicking-Pipeline#52 ·
Maintainers usually reply within 1 day
-
documentation
Difficulty 1/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 82/100
run-llama/llama_index#24163 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
btclib-org/btclib#3090 ·
Maintainers usually reply within 1 day
-
Translate the READMEOpendocumentation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
asasemahmed/openboard#68 ·
Maintainers usually reply within 1 day