facebookresearch/fairseq

Use symbolic link for saving best/last checkpoints

Open

#5,490 opened on Apr 23, 2024

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Python (6,224 forks)batch import
enhancementhelp wantedneeds triage

Repository metrics

Stars
 (29,107 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

🚀 Feature Request

When preparing checkpoint_best.pt and checkpoint_last.pt, create symbolic link instead of making exact copy of the checkpoint

Motivation

I'm running on a machine that somehow has bad io performance, writing the same file 3 times seems extremely inefficient.

Seems that there is plan to implement a asynchronous copying but i believe a symlink is much more efficient https://github.com/facebookresearch/fairseq/blob/bedb259bf34a9fc22073c13a1cee23192fa70ef3/fairseq/checkpoint_utils.py#L128-L141

Pitch

Alternatives

Additional context

Contributor guide

Research direction
Inspect the checkpoint saving logic in fairseq's checkpoint utils.py lines 128-141. Understand how checkpoint best.pt and checkpoint last.pt are currently copied. Replace the copy operations with os.symlink to create symbolic links. Consider cross platform support: on Windows, symlinks may require special handling. Test by running the training pipeline and verifying that links point to correct files.
Tech stack
python
Domain
backend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GitPythonOperating system file operations
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75