HfArgumentParser cannot parse dict-typed dataclass fields from the CLI (registers them with type=dict)

Open Beginner friendly
#48,030 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
cli, tooling

Research direction

Start with HfArgumentParser._parse_dataclass_field and compare the existing list-field handling with the generic dict branch. Reproduce the shown dataclass example, then add coverage for optional, bare, and parameterized dict annotations. Done means JSON such as {"a": false} parses successfully from the CLI without breaking existing field parsing.

Written by the indexing model from the issue text.

Description

bug
System Info
  • transformers version: 5.15.0
  • Platform: macOS-26.3.1-arm64-arm-64bit-Mach-O
  • Python version: 3.13.5

(also reproduced on transformers 5.14.1; the parsing logic is platform-independent)

Who can help?

No response

Information
  • My own modified scripts
Tasks
  • My own task or dataset (give details below)
Reproduction
from dataclasses import dataclass, field
from transformers import HfArgumentParser

@dataclass
class Cfg:
    kw: dict | None = field(default=None)

parser = HfArgumentParser((Cfg,))
parser.parse_args_into_dataclasses(["--kw", '{"a": false}'])
# argparse error: argument --kw: invalid dict value: '{"a": false}'

Tried JSON (with and without spaces), k=v and k: v formats: all fail. The cause is in _parse_dataclass_field: list fields have a special case (type=<element>, nargs='+'), but dict fields fall through to the generic branch, which registers the argument with type=dict, and dict("<string>") raises.

The same happens for bare dict and dict[str, Any] annotations.

Expected behavior

dict fields parse from the CLI (e.g. as JSON via type=json.loads), mirroring the existing special case for list fields. There is no backward-compatibility concern: passing a dict-typed argument from the CLI always errors today, so the change is purely additive.

Real-world impact: downstream configs expose dict fields that currently cannot be set from example-script flags at all. For example TRL's GRPOConfig.chat_template_kwargs (needed to disable Qwen3's thinking mode) fails with --chat_template_kwargs '{"enable_thinking": false}' because TrlParser inherits from HfArgumentParser.

I am happy to send a PR (small special case in _parse_dataclass_field + test) if this is welcome.

Dominant language
Python
Stars
166k
Forks
34.6k
Avg merge
2d 21h
Merged PRs (30d)
247

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/transformers

All issues in huggingface/transformers

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.