Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Hugging Face Chat Template Missing `<|constrain|>` Token for Tool Calls

Open
  • Is something wrong?
#91 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
huggingface, python
Domain
ai

Research direction

Start with the chat_template.jinja embedded in the tokenizer_config.json used by the Hugging Face gpt-oss tokenizer, then reproduce the output with AutoTokenizer.apply_chat_template using the supplied script. Compare tool-call rendering with the Harmony expectation; done means the generated history includes <|constrain|>json between commentary and the message token.

Written by the indexing model from the issue text.

Description

Background

While using the gpt-oss model series, I discovered an inconsistency between the official OpenAI Harmony rendering logic and the chat_template.jinja provided in the Hugging Face repository.

According to the OpenAI Harmony specification (as referenced in Harmony Issue #58), structural outputs and tool calls require a specific content_type set to <|constrain|>json. However, the current Jinja template in the HF tokenizer_config.json does not correctly render this token, even when tools are provided to the template.

The Problem

  • Inconsistent Inference: The model is trained to expect and produce <|constrain|>json for tool arguments. When the chat template omits this token during few-shot prompting or conversation history reconstruction, it breaks the model's structural consistency and may degrade tool-calling performance.

  • Template Implementation Gap: The current Jinja implementation appears to hardcode json or omit the <|constrain|> modifier entirely, which is inconsistent with the Harmony framework's requirements for proper tool execution formatting.

Steps to Reproduce

Run the following script with openai/gpt-oss-20b (or gpt-oss-120b):

from transformers import AutoTokenizer
import json

# Using the official GPT-OSS tokenizer
tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b", use_fast=False)

obj = {
    "tools": [
        {
            "function": {
                "name": "python_executor",
                "description": "Executes Python code and returns the output or error message.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "code": {"type": "string", "description": "Python code to execute"}
                    },
                    "required": ["code"]
                }
            }
        }
    ],
    "messages": [
        {"role": "system", "content": "You are Qwen, a helpful AI assistant that can execute Python code using available tools."},
        {"role": "user", "content": "Please run this code: print(2 + 3)"},
        {
            "role": "assistant",
            "tool_calls": [
                {
                    "function": {
                        "name": "python_executor",
                        "arguments": {"code": "print(2 + 3)"}
                    }
                }
            ]
        },
        {"role": "tool", "content": "{\"stdout\": \"5\\n\", \"stderr\": \"\"}"}
    ]
}

text = tokenizer.apply_chat_template(
    obj["messages"],
    tokenize=False,
    add_generation_prompt=True,
    tools=obj["tools"]
)
print(text)

Actual vs Expected Output

Actual Output (Current Template)
...<|start|>assistant|>functions.python_executor<|channel|>commentary json<|message|>{"code": "print(2 + 3)"}<|call|>...
Expected Output (Per Harmony Spec)
...<|start|>assistant|>functions.python_executor<|channel|>commentary <|constrain|>json<|message|>{"code": "print(2 + 3)"}<|call|>...
Key Difference

Actual: <|channel|>commentary json
Expected: <|channel|>commentary <|constrain|>json

The <|constrain|> special token is missing between commentary and json.

Impact

This discrepancy means:

  1. Tool-calling examples in conversation history are not properly formatted
  2. Few-shot prompting with tools may not work as intended
  3. The model may not correctly recognize tool call formatting during inference
  4. Behavior diverges from the official OpenAI Harmony implementation
Dominant language
Rust
Stars
4.5k
Forks
315
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openai/harmony

All issues in openai/harmony

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.