Internal server error for role orders in LLM inference

Open

#113 opened on Apr 30, 2024 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 35/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Domain: ai, api

Research direction

Start by tracing the validation that produces the quoted role-order and last-message errors, then reproduce a request with continuation or agent-observation messages. Done means supported non-alternating role sequences no longer return HTTP 500 while the intended validation behavior remains covered.

Written by the indexing model from the issue text.

Description

Hi,

I am getting 500s with description

chat messages must alternate roles between 'user' and 'assistant'.  Message may have a leading 'system' role message

and

"Internal Server Error\",\"status\":500,\"detail\":\"Last message role should be 'user'

I think this order validation is unnecessary. Most of the models (e.g. mixtrals, llama3, gemma etc) are perfectly fine with any order of roles. This claim can be validated on groq's playground: https://console.groq.com/playground

This issue currently breaks some existing patterns, like continuations (without the user explicitly saying "continue") or in some cases running agents with observations etc.

Dominant language: Jupyter Notebook
Stars: 4.2k
Forks: 1.1k
Avg merge: 10h 15m
Merged PRs (30d): 1

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

[Bug]: PrevNextNodePostprocessor loses retrieved scores when neighbor windows overlap Open

Difficulty 2/5 1-3 hours Newbie friendliness 76/100

run-llama/llama_index#23199 · Opened Sep 23, 2026
`OpenAIChatModel` streaming keeps only the last chunk's `logprobs` Open

p:2-high pydanty:bug

Difficulty 2/5 1-3 hours Newbie friendliness 90/100

pydantic/pydantic-ai#8642 · 2 comments · Opened Sep 22, 2026
[BUG] Unified agent on Bedrock Converse with no tools is unusable: an empty `toolConfig` is emitted and Bedrock rejects the request Open

bug untriaged

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

opensearch-project/ml-commons#5094 · Opened Sep 22, 2026
[langchain-groq] bind_tools ignores a known tool_calling=False model profile and sends a request that Groq rejects Open

bug external groq

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

langchain-ai/langchain#40771 · 1 comment · Opened Sep 22, 2026
bug report: ai observability - dry-running a paused evaluation emails the whole org Open

ai-observability bug team/ai-observability

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

PostHog/posthog#104687 · Opened Sep 22, 2026

More AI Infra & Agents issues