Hacktoberfest 2026：維護者為十月標記出來的 issue，仍然開放、適合新手。 瀏覽 Hacktoberfest issue

`HyperparameterTuner` drops `ModelTrainer` checkpoint and retry configuration

未關閉 適合新手
  • 哪裡不對？
#6,500 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 2 天內回覆

@mohamedzeidan2021 已經在處理了。

開始於 2026年10月20日。

  • #6511 來自 @mohamedzeidan2021 —— 未關閉

評估

難度
2/5
預估耗時
1-3 小時
新手友好度
85/100
Issue 類型
缺陷
描述清晰度
描述清楚
活躍度
活躍
技術堆疊
aws, python
領域
backend-api-design, machine-learning

研究方向

從HyperparameterTuner._build_training_job_definition()開始，檢查它如何根據ModelTrainer建構definition_kwargs。將這些欄位與HyperParameterTrainingJobDefinition以及現有的 tuner 定義測試進行比較。完成標準是：提供檢查點和重試設定時予以保留，未提供時則妥善省略，託管式 spot 訓練也包括在內。

由索引模型根據 Issue 內容生成。

描述

PySDK Version

  • PySDK V2 (2.x)
  • PySDK V3 (3.x)

Describe the bug

HyperparameterTuner builds a
HyperParameterTrainingJobDefinition from its ModelTrainer.

The generated shape supports checkpoint_config and retry_strategy.
However, _build_training_job_definition() does not copy either value from
the trainer.

As a result, tuned training jobs lose:

  • The checkpoint S3 location and local checkpoint path.
  • The training retry strategy.

This behavior is especially risky for managed spot tuning. An interrupted job
cannot resume from the configured checkpoint, and transient failures do not use
the configured retry count.

This defect is present in SageMaker Python SDK v3.25.0.

To reproduce

Run this script. It does not call a live AWS service.

from unittest.mock import patch

import boto3
from sagemaker.core.parameter import IntegerParameter
from sagemaker.core.training.configs import (
    CheckpointConfig,
    Compute,
    OutputDataConfig,
    RetryStrategy,
)
from sagemaker.core.workflow.pipeline_context import PipelineSession
from sagemaker.train import ModelTrainer
from sagemaker.train.tuner import HyperparameterTuner

role = "arn:aws:iam::123456789012:role/example"
boto_session = boto3.Session(
    aws_access_key_id="test",
    aws_secret_access_key="test",
    region_name="us-east-2",
)
pipeline_session = PipelineSession(boto_session=boto_session)

with patch(
    "sagemaker.train.defaults.resolve_and_validate_role",
    return_value=role,
):
    trainer = ModelTrainer(
        training_image="123456789012.dkr.ecr.us-east-2.amazonaws.com/example:latest",
        role=role,
        sagemaker_session=pipeline_session,
        compute=Compute(
            instance_type="ml.m5.large",
            instance_count=1,
            enable_managed_spot_training=True,
        ),
        checkpoint_config=CheckpointConfig(
            s3_uri="s3://example-bucket/checkpoints/",
            local_path="/opt/ml/checkpoints",
        ),
        output_data_config=OutputDataConfig(
            s3_output_path="s3://example-bucket/output/",
        ),
    ).with_retry_strategy(
        RetryStrategy(maximum_retry_attempts=10)
    )

tuner = HyperparameterTuner(
    model_trainer=trainer,
    objective_metric_name="validation:accuracy",
    objective_type="Maximize",
    hyperparameter_ranges={
        "epochs": IntegerParameter(1, 5),
    },
    max_jobs=2,
    max_parallel_jobs=1,
)

definition = tuner._build_training_job_definition([])

print("Trainer checkpoint:", trainer.checkpoint_config)
print("Tuning checkpoint:", definition.checkpoint_config)
print("Trainer retry:", trainer.retry_strategy)
print("Tuning retry:", definition.retry_strategy)

assert definition.checkpoint_config == trainer.checkpoint_config
assert definition.retry_strategy == trainer.retry_strategy

Both assertions fail because the tuning definition fields remain unassigned.

Expected behavior

The tuning job definition must preserve the trainer's checkpoint and retry
configuration.

Screenshots or logs

The generated shape accepts both fields:

HyperParameterTrainingJobDefinition(
    # ...
    checkpoint_config=...,
    retry_strategy=...,
)

HyperparameterTuner._build_training_job_definition() does not pass them in
definition_kwargs.

System information

  • SageMaker Python SDK version: 3.25.0
  • Framework name (eg. PyTorch) or algorithm (eg. KMeans): Framework-independent
  • Framework version: Not applicable
  • Python version: 3.12
  • CPU or GPU: CPU
  • Custom Docker image (Y/N): Reproduces with either

Additional context

A possible fix is to add the assigned trainer values to
definition_kwargs:

if model_trainer.checkpoint_config is not None:
    definition_kwargs["checkpoint_config"] = model_trainer.checkpoint_config

if model_trainer.retry_strategy is not None:
    definition_kwargs["retry_strategy"] = model_trainer.retry_strategy

Tests must cover normal and managed spot trainers, explicit values, and omitted
values.

主要語言
Python
星號
2.3k
分支
1.3k
平均合併
2 天 16 小時
30 天內合併 PR
88

環境準備

  • 沒有 Dockerfile 或 Docker Compose 檔案
  • 沒有 Pull Request 範本
  • 閱讀貢獻指南

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

aws/sagemaker-python-sdk 的其他 Issue

查看 aws/sagemaker-python-sdk 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。