Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Bug] ModelTrainer drops TrainingJobName for PipelineSession, breaking use_custom_job_prefix on TrainingStep

Aperta Adatta ai principianti
  • Qualcosa non va?
#5,776 1 commento 1 reazione 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@mohamedzeidan2021 ci sta già lavorando.

Dal 24/9/2026.

  • #6323 di @mohamedzeidan2021 — aperta

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
72/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Tranquilla
Stack tecnologico
aws, python
Ambito
cloud, machine-learning

Direzione di ricerca

Inizia in sagemaker/train/model_trainer.py, in ModelTrainer._create_training_job_args, e riproduci l’asserzione TrainingJobName mostrata con PipelineSession. Controlla TrainingStep.arguments e trim_request_dict; è completato quando la richiesta serializzata conserva TrainingJobName per l’uso con use_custom_job_prefix=True, mentre il caso false continua a rimuoverlo.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

component: pipelines type: bug

PySDK Version

  • PySDK V2 (2.x)
  • PySDK V3 (3.x)

Describe the bug
When a ModelTrainer is executed under a PipelineSession (i.e. produces a TrainingStep), ModelTrainer._create_training_job_args explicitly removes training_job_name from the request before serializing it to PascalCase:

# sagemaker/train/model_trainer.py (sagemaker-train 1.8.0)
if boto3 or isinstance(self.sagemaker_session, PipelineSession):
    if isinstance(self.sagemaker_session, PipelineSession):
        training_request.pop("training_job_name", None)
    # Convert snake_case to PascalCase for AWS API
    pipeline_request = {to_pascal_case(k): v for k, v in training_request.items()}
    serialized_request = serialize(pipeline_request)
    return serialized_request

Because the key is popped, the resulting request dict has no TrainingJobName. Downstream, TrainingStep.arguments (with PipelineDefinitionConfig(use_custom_job_prefix=True)) relies on TrainingJobName being present in the request so the prefix is preserved (and trim_request_dict removes it when use_custom_job_prefix=False).

The net effect is that use_custom_job_prefix=True is silently ignored for TrainingStep when the step is built from a ModelTrainer: every pipeline execution produces a random auto-generated training job name instead of the configured base_job_name prefix.

This is the same class of bug as #3991 and #4590 (which were about TransformStep), but for the new V3 ModelTrainer → TrainingStep path.

To reproduce

from sagemaker.core.workflow.pipeline import Pipeline
from sagemaker.core.workflow.pipeline_context import PipelineSession
from sagemaker.core.workflow.pipeline_definition_config import PipelineDefinitionConfig
from sagemaker.train.model_trainer import ModelTrainer
# ... build a ModelTrainer `trainer` with base_job_name=\"my-prefix\" ...

pipeline_session = PipelineSession()
trainer.sagemaker_session = pipeline_session

step_args = trainer._create_training_job_args()
assert \"TrainingJobName\" in step_args, step_args  # FAILS — key was popped

pipeline = Pipeline(
    name=\"repro\",
    steps=[...],  # TrainingStep built from trainer
    sagemaker_session=pipeline_session,
    pipeline_definition_config=PipelineDefinitionConfig(use_custom_job_prefix=True),
)
# Pipeline executions will NOT use \"my-prefix-...\" as the training job name.

Expected behavior
TrainingJobName should remain in the request dict so that PipelineDefinitionConfig(use_custom_job_prefix=True) produces training jobs named with the configured prefix. When use_custom_job_prefix=False, TrainingStep.arguments/trim_request_dict will strip the key as usual.

A minimal fix is to stop popping the key:

if boto3 or isinstance(self.sagemaker_session, PipelineSession):
    pipeline_request = {to_pascal_case(k): v for k, v in training_request.items()}
    serialized_request = serialize(pipeline_request)
    return serialized_request

As a workaround we currently monkey-patch _create_training_job_args to re-insert TrainingJobName = _get_unique_name(self.base_job_name) when the session is a PipelineSession.

Screenshots or logs
N/A — silent misbehavior; the pipeline executes but job names use the default random name instead of the configured prefix.

System information

  • SageMaker Python SDK version: sagemaker-train 1.8.0, sagemaker-core 2.8.0, sagemaker-mlops 1.8.0, sagemaker-serve 1.8.0 (also reproduces on 1.7.1 / 2.7.1)
  • Framework name or algorithm: custom (source_code via ModelTrainer)
  • Framework version: N/A
  • Python version: 3.13
  • CPU or GPU: CPU (irrelevant, bug is SDK-side)
  • Custom Docker image (Y/N): Y

Additional context
Related closed issues for other step types: #3991 (TransformStep), #4590 (TransformStep/ProcessingStep).

Lingua principale
Python
Stelle
2.3k
Fork
1.3k
Merge medio
3g 2h
PR unite (30g)
70

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di aws/sagemaker-python-sdk

Tutte le issue di aws/sagemaker-python-sdk

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.