[Bug]: Merge CI, test_bagel_img2img_shared_memory_connector, AssertionError: Pixel mismatch at (100, 100): expected (156, 172, 217), got (193, 161, 164)

Your current environment

CI env

Your code version

0.26.0

cbab1d4d8f814e48ee691b69ac4cee9d0419f218

🐛 Describe the bug

https://buildkite.com/vllm/vllm-omni/builds/12782/canvas?sid=019fa43b-ab67-469c-ad28-ce3e66ac4a2f&tab=output

=========================================================================== FAILURES ===========================================================================
Actions for L1038
1038
__________________________________________________________ test_bagel_img2img_shared_memory_connector __________________________________________________________
Actions for L1039
1039

Actions for L1040
1040
run_level = 'advanced_model'
Actions for L1041
1041

Actions for L1042
1042    
@pytest.mark.advanced_model
Actions for L1043
1043    
@pytest.mark.diffusion
Actions for L1044
1044    
@hardware_test(res={"cuda": "H100", "rocm": "MI325"})
Actions for L1045
1045    
def test_bagel_img2img_shared_memory_connector(run_level):
Actions for L1046
1046        
"""Test Bagel img2img with shared memory connector."""
Actions for L1047
1047        
input_image = _load_input_image()
Actions for L1048
1048        
config_path = _resolve_deploy_config(BAGEL_CI_DEPLOY, run_level)
Actions for L1049
1049        
with OmniRunner(
Actions for L1050
1050            
"ByteDance-Seed/BAGEL-7B-MoT",
Actions for L1051
1051            
stage_configs_path=config_path,
Actions for L1052
1052        
) as runner:
Actions for L1053
1053            
generated_image = _generate_bagel_img2img(runner.omni, input_image)
Actions for L1054
1054            
if run_level == "advanced_model":
Actions for L1055
1055
>               _validate_pixels(generated_image)
Actions for L1056
1056

Actions for L1057
1057
tests/distributed/omni_connectors/test_bagel_shared_memory_connector.py:243:
Actions for L1058
1058
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Actions for L1059
1059

Actions for L1060
1060
image = <PIL.Image.Image image mode=RGB size=1024x672 at 0x7C79357C88C0>
Actions for L1061
1061
reference_pixels = [{'position': (100, 100), 'rgb': (156, 172, 217)}, {'position': (400, 50), 'rgb': (105, 144, 217)}, {'position': (700,... (180, 22, 52)}, {'position': (512, 336), 'rgb': (221, 211, 194)}, {'position': (700, 400), 'rgb': (192, 10, 46)}, ...]
Actions for L1062
1062
tolerance = 10
Actions for L1063
1063

Actions for L1064
1064    
def _validate_pixels(
Actions for L1065
1065        
image: Image.Image,
Actions for L1066
1066        
reference_pixels: list[dict[str, Any]] = REFERENCE_PIXELS,
Actions for L1067
1067        
tolerance: int = PIXEL_TOLERANCE,
Actions for L1068
1068    
) -> None:
Actions for L1069
1069        
"""Validate that image pixels match expected reference values.
Actions for L1070
1070

Actions for L1071
1071    
    Args:
Actions for L1072
1072    
        image: The PIL Image to validate.
Actions for L1073
1073    
        reference_pixels: List of dicts with 'position' (x, y) and 'rgb' (R, G, B).
Actions for L1074
1074    
        tolerance: Maximum allowed difference per color channel.
Actions for L1075
1075

Actions for L1076
1076    
    Raises:
Actions for L1077
1077    
        AssertionError: If any pixel differs beyond tolerance.
Actions for L1078
1078    
    """
Actions for L1079
1079        
for ref in reference_pixels:
Actions for L1080
1080            
x, y = ref["position"]
Actions for L1081
1081            
expected = ref["rgb"]
Actions for L1082
1082            
actual = image.getpixel((x, y))[:3]
Actions for L1083
1083
>           assert all(abs(a - e) <= tolerance for a, e in zip(actual, expected)), (
Actions for L1084
1084                
f"Pixel mismatch at ({x}, {y}): expected {expected}, got {actual}"
Actions for L1085
1085            
)
Actions for L1086
1086
E           AssertionError: Pixel mismatch at (100, 100): expected (156, 172, 217), got (193, 161, 164)
Actions for L1087
1087
E           assert False
Actions for L1088
1088
E            +  where False = all(<generator object _validate_pixels.<locals>.<genexpr> at 0x7c78cdb5c040>)
Actions for L1089
1089

Actions for L1090
1090
tests/distributed/omni_connectors/test_bagel_shared_memory_connector.py:148: AssertionError

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.