[bug] Chat completions sometimes do not follow requested template

Open
  • Is something wrong?
#11 0 comments 0 reactions 1 assignee View on GitHub

@kelsey-wong is already working on this.

Since Dec 18, 2025.

Assessment

This issue has not been assessed yet.

Description

bug

Sometimes, LiteLLM's response will omit the closing </answer> tag (which means that the <answer> tag will show up in the parsed response). Debug how to fix this and/or consider handling this through the parser.

Example:

Generated ObservedConsistencyQACompletionTemplate completion for messages:
    [
  {
    "role": "user",
    "content": "Explain the concept of machine learning in simple terms.\n\n\n## Additional instructions for formatting your response\n\nPlease strictly use the following template to format your response:\n\n<think>\n[think carefully step by step (no more than 375 words) before providing your answer]\n</think>\n\n<answer>\n[provide your final answer]\n</answer>"
  }
]

    Content:
    <answer>
Machine learning is like teaching a computer to learn from examples, just like a person learns from experience. Instead of giving the computer exact instructions for every task, we show it lots of information or data. The computer looks for patterns in this data and uses what it has learned to make decisions or predictions on new information. For example, showing a computer many pictures of cats and dogs helps it learn how to tell the difference between them. Over time, the computer gets better at recognizing things or solving problems on its own without needing step-by-step instructions from a person.
    Explanation:
    To explain machine learning in simple terms, I first need to identify the core idea behind it without using technical jargon. Machine learning is about computers learning from data and improving over time without being explicitly programmed for every specific task. 

Step-by-step:

1. Begin by comparing machine learning to something familiar, like learning for humans.
2. Explain that instead of writing instructions line-by-line, we let the computer figure out patterns by itself.
3. Mention that the computer uses data as examples to learn from.
4. Highlight the outcome, which is that after learning, the computer can make decisions or predictions.
5. Keep the explanation concise and easy to understand, avoiding complex terms like algorithms or neural networks.

This approach will help a beginner grasp the basic idea of machine learning clearly.
    Response fields:
    {
  "answer": "<answer>\nMachine learning is like teaching a computer to learn from examples, just like a person learns from experience. Instead of giving the computer exact instructions for every task, we show it lots of information or data. The computer looks for patterns in this data and uses what it has learned to make decisions or predictions on new information. For example, showing a computer many pictures of cats and dogs helps it learn how to tell the difference between them. Over time, the computer gets better at recognizing things or solving problems on its own without needing step-by-step instructions from a person."
}
Dominant language
Python
Stars
9
Forks
6
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from cleanlab/tlm

All issues in cleanlab/tlm

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.