Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Per-task YAML configuration for the reference training scripts

Open
  • Is something wrong?
#2,149 1 comment 0 reactions 1 assignee View on GitHub

Maintainers usually reply within 1 day

@felixdittrich92 is already working on this.

Since Sep 29, 2026.

Assessment

This issue has not been assessed yet.

Description

ext: references topic: documentation type: new feature
🚀 The feature

Let each references/*/train.py read its full training configuration from a YAML file (one per task, with a shipped default), keeping the CLI only for overriding individual keys:

python references/detection/train.py --config references/detection/configs/default.yaml \
    --set data.train_path=/data/train data.val_path=/data/val training.epochs=20

A first sketch of the schema, mirroring what the scripts already accept (the six training scripts currently declare 27 to 36 add_argument calls each, most of them shared):

model:
  arch: db_resnet50
  pretrained: true
  input_size: 1024
  assume_straight_pages: true          # `--rotation` today
data:
  train_path: null                     # or built-in datasets: train_datasets: [FUNSD, SVHN]
  val_path: null
  workers: null
  labels_name: labels.json
augmentations:                         # per task defaults, all switchable
  horizontal_flip: 0.15
  random_crop: {p: 0.25, scale: [0.75, 1.0], ratio: [0.85, 1.15]}
  random_resize: {p: 0.25, scale_range: [0.4, 0.9]}
  rotation: {enabled: false, max_angle: 90, expand: true}
  perspective: {p: 0.0, distortion: 0.2}        # RandomPerspective (#2143)
  photometric: {glare: 0.0, lighting: 0.0, shadow: 0.3, blur: 0.3, noise: 0.1, grayscale: 0.15, color_inversion: 0.3}
training:
  epochs: 10
  batch_size: 2
  optimizer: {name: adam, lr: 1e-3, weight_decay: 0, backbone_lr: null}
  scheduler: poly
  amp: {enabled: false, dtype: float16}
  freeze_backbone: false
  early_stopping: {enabled: false, patience: 5, min_delta: 0.01}
  device: null                         # cuda index, cuda:N, mps, cpu; null = auto
  backend: nccl
runtime:
  output_dir: .
  name: null
  save_interval_epoch: false
  resume: null
  test_only: false
  logging: {wandb: false, clearml: false, push_to_hub: false}

Implementation idea, to keep the diff reviewable:

  1. references/common/config.py: load YAML → nested dict → SimpleNamespace/dataclass, apply --set key.path=value overrides (typed from the default), and validate unknown keys.
  2. Each script keeps its main(args) body; only parse_args() changes to build args from the config (a thin compatibility layer can keep the current flat attribute names, e.g. args.lr, so the training loops do not move in the same PR).
  3. The resolved config is written next to every checkpoint (<name>.yaml) together with the run metadata, which also covers the "save useful metadata besides the checkpoints" point discussed in #2142.
  4. Ship configs/default.yaml per task; the README shows the YAML instead of a long command line.
Motivation, pitch

Discussed in #2142 (review by @felixdittrich92): the training scripts keep accumulating CLI flags (--rotation, --eval-straight, --early-stop-*, --amp, now --amp-dtype, --no-hflip, ...), each one needing documentation, and the augmentation pipeline is hard-coded so any change (disable the horizontal flip for semantic region classes, add the new perspective / photo transforms, tune the crop scale) means editing the script. A per-task YAML:

  • makes a training run reproducible and shareable (the config is the experiment, and it is saved with the checkpoint),
  • lets users adjust augmentations without patching the scripts,
  • keeps the six scripts aligned (same keys, same defaults), which is also what the device / AMP unification asked in #2142 needs,
  • avoids --no-hflip-style flags altogether.
Alternatives
  • Keep adding CLI flags (status quo): simplest, but the argument lists are already 27-36 entries per script and augmentation choices stay hard-coded.
  • Hydra / OmegaConf: powerful, but adds a dependency and its own conventions; a ~100-line loader with --set overrides covers this use case.
  • Python config modules (config.py per experiment): flexible but not serialisable next to a checkpoint.
Additional context

Happy to implement it in steps once the shape is agreed: (1) loader + detection script, (2) the other five scripts, (3) documentation. The device / AMP unification and the checkpoint metadata from #2142 would then be built on top of the config rather than as more flags.

Dominant language
Python
Stars
6.4k
Forks
684
Avg merge
20h 28m
Merged PRs (30d)
28

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from mindee/doctr

All issues in mindee/doctr

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.