`QualityCheckStep` misplaces the output KMS key and omits tags
Maintainers usually reply within 2 days
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- aws, python
- Domain
- cloud, machine-learning
Research direction
Start in the QualityCheckStep arguments method in sagemaker.mlops.workflow.quality_check_step, where the processing-job request dict is built. Move KmsKeyId from the S3 output dict up to ProcessingOutputConfig, and add Tags from check_job_config to the request. Run the reproduction script from the issue with print(step.arguments); done when KmsKeyId sits on ProcessingOutputConfig and Tags appears in the output.
Written by the indexing model from the issue text.
Description
PySDK Version
- PySDK V2 (2.x)
- PySDK V3 (3.x)
Describe the bug
QualityCheckStep.arguments builds a processing-job request manually.
The method puts KmsKeyId inside ProcessingS3Output. The SageMaker API
requires this field on ProcessingOutputConfig.
The method also omits the tags from CheckJobConfig.
The generated request has this incorrect structure:
{
"ProcessingOutputConfig": {
"Outputs": [
{
"S3Output": {
"KmsKeyId": "arn:aws:kms:..."
}
}
]
}
}
This behavior is present in SageMaker Python SDK v3.25.0.
To reproduce
Set these environment variables:
export AWS_REGION="us-east-2"
export SAGEMAKER_ROLE_ARN="arn:aws:iam::<account-id>:role/<sagemaker-execution-role>"
export KMS_KEY_ARN="arn:aws:kms:us-east-2:<account-id>:key/<key-id>"
Run this script:
import os
import boto3
from sagemaker.core.workflow.pipeline_context import PipelineSession
from sagemaker.mlops.workflow.check_job_config import CheckJobConfig
from sagemaker.mlops.workflow.quality_check_step import (
DataQualityCheckConfig,
QualityCheckStep,
)
session = PipelineSession(
boto_session=boto3.Session(region_name=os.environ["AWS_REGION"])
)
job_config = CheckJobConfig(
role=os.environ["SAGEMAKER_ROLE_ARN"],
instance_type="ml.m5.xlarge",
instance_count=1,
output_kms_key=os.environ["KMS_KEY_ARN"],
tags=[{"Key": "test", "Value": "quality-check"}],
sagemaker_session=session,
)
quality_config = DataQualityCheckConfig(
baseline_dataset="s3://example-bucket/baseline.csv",
dataset_format={"csv": {"header": True}},
output_s3_uri="s3://example-bucket/baseline-output/",
)
step = QualityCheckStep(
name="quality-check-reproduction",
quality_check_config=quality_config,
check_job_config=job_config,
register_new_baseline=True,
skip_check=True,
)
print(step.arguments)
The output contains ProcessingOutputConfig.Outputs[0].S3Output.KmsKeyId.
The output does not contain Tags.
Expected behavior
The request must put the KMS key on ProcessingOutputConfig:
{
"ProcessingOutputConfig": {
"KmsKeyId": "arn:aws:kms:...",
"Outputs": []
},
"Tags": [
{"Key": "test", "Value": "quality-check"}
]
}
Screenshots or logs
The v3.25 implementation contains:
if self.check_job_config.output_kms_key:
s3_output_dict["KmsKeyId"] = self.check_job_config.output_kms_key
The request builder does not read self.check_job_config.tags.
System information
- SageMaker Python SDK version: 3.25.0
- Framework name (eg. PyTorch) or algorithm (eg. KMeans): SageMaker Model Monitor
- Framework version: Not applicable
- Python version: 3.12
- CPU or GPU: CPU
- Custom Docker image (Y/N): No
Additional context
A temporary QualityCheckStep subclass can correct the request. The workaround
moves KmsKeyId to ProcessingOutputConfig and adds the configured tags.
A possible fix is:
processing_output_config = {"Outputs": processing_outputs}
if self.check_job_config.output_kms_key:
processing_output_config["KmsKeyId"] = self.check_job_config.output_kms_key
request_dict["ProcessingOutputConfig"] = processing_output_config
if self.check_job_config.tags:
request_dict["Tags"] = self.check_job_config.tags
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 1.3k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 88
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from aws/sagemaker-python-sdk
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
aws/sagemaker-python-sdk#6500 ·
Maintainers usually reply within 2 days
-
[Bug] ModelTrainer with no input channels emits InputDataConfig: [], which CreatePipeline rejects (min=1) — v2 omitted the keyPossibly taken @sagemaker-bot claimed this 20 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
aws/sagemaker-python-sdk#6156 · 2 comments ·
Maintainers usually reply within 2 days
-
sagemaker-train should depend on mlflow-skinny, following sagemaker-mlflow 0.5.0Possibly taken @mohamedzeidan2021 claimed this 21 days ago. Open
Difficulty 2/5 Half a day Newbie friendliness 72/100
aws/sagemaker-python-sdk#6152 ·
Maintainers usually reply within 2 days
-
[Bug] ModelTrainer drops TrainingJobName for PipelineSession, breaking use_custom_job_prefix on TrainingStepPossibly taken @mohamedzeidan2021 claimed this 25 days ago. Opencomponent: pipelines type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
aws/sagemaker-python-sdk#5776 · 1 comment · 1 reaction ·
Maintainers usually reply within 2 days
-
`ModelTrainer` bug with method to load hyperparameters from file for Amazon Nova RecipePossibly taken @sagemaker-bot claimed this 20 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
aws/sagemaker-python-sdk#5770 ·
Maintainers usually reply within 2 days
All issues in aws/sagemaker-python-sdk
Similar issues
-
good first issue tools
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
bug deferred domain:ci P1
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
docling-project/docling-mcp#147 ·
-
issue: A pipe model's Valves button ignores the Allow Chat Valves permissionPossibly taken A pull request linked to this issue is open or already merged. Openbug confirmed issue
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
open-webui/open-webui#32771 · 1 comment ·
Maintainers usually reply within 1 day
-
Scheduled Security Scan is failing on mainPossibly taken @pkupt claimed this today. Opensecurity-scan-failure
Difficulty 1/5 1-3 hours Newbie friendliness 74/100
semantica-agi/semantica#2045 ·
Maintainers usually reply within 2 days