Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

[BUG] pyservicemaker: element errors never reach on_message, and a fatal error leaves wait() blocked forever

Đang mở
  • Có gì không ổn?
#98 2 bình luận 0 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@nv-support đang làm issue này rồi.

Từ ngày 26/10/2026.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
50/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
cpp, python
Lĩnh vực
api, backend

Hướng nghiên cứu

Read src/service-maker/sources/core/src/pipeline.cpp around bus_call and src/service-maker/sources/python/src/bindings/pipeline.h around on_pipeline_message; the issue identifies both entry points and the missing error message handling. Trace how the existing message types reach Python, then check the C++ and Python pipeline tests for callback and wait behavior. Done means errors and warnings reach on_message and fatal errors let wait() return, with a failed transition to PLAYING reported as described.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Describe the bug

Pipeline.start(on_message) delivers only StateTransitionMessage, DynamicSourceMessage and EOSMessage. A GST_MESSAGE_ERROR from an element is printed to stderr and dropped, so a Python application cannot see that its pipeline failed or why.

It gets worse when the error comes while the pipeline is playing. The error does not stop the main loop, so Pipeline.wait() never returns and the process hangs with no output to Python. The only exceptions are the two error strings bus_call matches ("Reconnection attempts exceeded..." and "Output window was closed").

When the error comes during the change to PLAYING, start() returns normally, wait() returns at once, and the callback gets no message at all. Python cannot tell this failed start from a normal end.

The cause is in bus_call in src/service-maker/sources/core/src/pipeline.cpp: the GST_MESSAGE_ERROR case only calls g_printerr, and there is no Pipeline::Message subclass for errors. on_pipeline_message in src/service-maker/sources/python/src/bindings/pipeline.h casts only the three message types above, so the C++ API and the Python API both lose the error.

Steps/Code to reproduce bug

Save as repro.py:

import sys
import time

from pyservicemaker import Pipeline

CASES = {
    # filesrc fails during the change to PLAYING: the file does not exist.
    "missing-file": [("filesrc", "src", {"location": "/nonexistent.mp4"})],
    # qtdemux fails while PLAYING: the file is not an MP4.
    "bad-mp4": [("filesrc", "src", {"location": "/etc/os-release"}), ("qtdemux", "demux", {})],
}

case = sys.argv[1]
messages = []
pipeline = Pipeline(case)
for kind, name, props in CASES[case]:
    pipeline.add(kind, name, props)
pipeline.add("fakesink", "sink", {})
pipeline.link(*[name for _, name, _ in CASES[case]], "sink")

started = time.monotonic()
pipeline.start(lambda message: messages.append(type(message).__name__))
pipeline.wait()
print(f"wait() returned after {time.monotonic() - started:.1f} s")
print(f"messages seen by Python: {sorted(set(messages))}")

Run it in the stock container (pyyaml is installed first because import pyservicemaker needs it and the image does not ship it):

docker run --rm --gpus all -v "$PWD":/repro --entrypoint bash nvcr.io/nvidia/deepstream:9.1-triton-multiarch \
  -c "pip install -q --break-system-packages pyyaml && python3 /repro/repro.py bad-mp4"

bad-mp4 prints these lines to stderr and then hangs until the container is killed. wait() never returns, and the callback never sees an error:

Main Loop Running...
ERROR from demux: This file is invalid and cannot be played.
ERROR from demux: Internal data stream error.
ERROR from demux: This file contains no playable streams.

missing-file prints:

Unable to set the pipeline to the playing state.
Pipeline missing-file stopped
wait() returned after 0.0 s
messages seen by Python: []

Expected behavior

  1. A GST_MESSAGE_ERROR reaches the on_message callback as a message type of its own (for example ErrorMessage with origin, message and debug_info), in C++ and in Python. The same for GST_MESSAGE_WARNING.
  2. After a fatal error, wait() returns, as it does after end of stream.
  3. A failed change to PLAYING raises from start(), or reaches the callback as an error message.

Environment details

  • Environment location: Docker on bare-metal (Ubuntu 26.04.1 LTS host, RTX A4000 Laptop GPU, driver 595.99.02)
  • Method of DeepStream install: Docker, nvcr.io/nvidia/deepstream:9.1-triton-multiarch (digest sha256:fd31f5b44ababdbdee8cd397a375e888191b49e402ac237254a4cdc239130f5b), DeepStream 9.1.0, pyservicemaker 0.0.1, TensorRT 10.16.0.72, CUDA 13.2
  • Docker command: see the steps above
  • scripts/print_env.sh output, run inside the same container:
Click here to see environment details
 **git***
 Not inside a git repository
 
 ***OS Information***
 DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=24.04
 DISTRIB_CODENAME=noble
 DISTRIB_DESCRIPTION="Ubuntu 24.04.4 LTS"
 PRETTY_NAME="Ubuntu 24.04.4 LTS"
 NAME="Ubuntu"
 VERSION_ID="24.04"
 VERSION="24.04.4 LTS (Noble Numbat)"
 VERSION_CODENAME=noble
 ID=ubuntu
 ID_LIKE=debian
 HOME_URL="https://www.ubuntu.com/"
 SUPPORT_URL="https://help.ubuntu.com/"
 BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
 PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
 UBUNTU_CODENAME=noble
 LOGO=ubuntu-logo
 Linux b5ade298bb5a 7.0.0-38-generic #38-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep  4 09:10:14 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
 
 ***GPU Information***
 Thu Oct 22 14:52:51 2026
 +-----------------------------------------------------------------------------------------+
 | NVIDIA-SMI 595.99.02              Driver Version: 595.99.02      CUDA Version: 13.2     |
 +-----------------------------------------+------------------------+----------------------+
 | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
 | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
 |                                         |                        |               MIG M. |
 |=========================================+========================+======================|
 |   0  NVIDIA RTX A4000 Laptop GPU    Off |   00000000:01:00.0 Off |                  N/A |
 | N/A   63C    P0             25W /   80W |     161MiB /   8192MiB |      0%      Default |
 |                                         |                        |                  N/A |
 +-----------------------------------------+------------------------+----------------------+
 
 +-----------------------------------------------------------------------------------------+
 | Processes:                                                                              |
 |  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
 |        ID   ID                                                               Usage      |
 |=========================================================================================|
 |  No running processes found                                                             |
 +-----------------------------------------------------------------------------------------+
 
 ***CPU***
 Architecture:                            x86_64
 CPU op-mode(s):                          32-bit, 64-bit
 Address sizes:                           39 bits physical, 48 bits virtual
 Byte Order:                              Little Endian
 CPU(s):                                  16
 On-line CPU(s) list:                     0-15
 Vendor ID:                               GenuineIntel
 Model name:                              11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz
 CPU family:                              6
 Model:                                   141
 Thread(s) per core:                      2
 Core(s) per socket:                      8
 Socket(s):                               1
 Stepping:                                1
 CPU(s) scaling MHz:                      42%
 CPU max MHz:                             5000.0000
 CPU min MHz:                             800.0000
 BogoMIPS:                                5222.40
 Flags:                                   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l2 cdp_l2 ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves split_lock_detect user_shstk dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect md_clear ibt flush_l1d arch_capabilities
 Virtualization:                          VT-x
 L1d cache:                               384 KiB (8 instances)
 L1i cache:                               256 KiB (8 instances)
 L2 cache:                                10 MiB (8 instances)
 L3 cache:                                24 MiB (1 instance)
 NUMA node(s):                            1
 NUMA node0 CPU(s):                       0-15
 Vulnerability Gather data sampling:      Vulnerable
 Vulnerability Ghostwrite:                Not affected
 Vulnerability Indirect target selection: Mitigation; Aligned branch/return thunks
 Vulnerability Itlb multihit:             Not affected
 Vulnerability L1tf:                      Not affected
 Vulnerability Mds:                       Not affected
 Vulnerability Meltdown:                  Not affected
 Vulnerability Mmio stale data:           Not affected
 Vulnerability Old microcode:             Not affected
 Vulnerability Reg file data sampling:    Not affected
 Vulnerability Retbleed:                  Not affected
 Vulnerability Spec rstack overflow:      Not affected
 Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
 Vulnerability Spectre v1:                Mitigation; usercopy/swapgs barriers and __user pointer sanitization
 Vulnerability Spectre v2:                Mitigation; Enhanced / Automatic IBRS; IBPB conditional; PBRSB-eIBRS SW sequence; BHI SW loop, KVM SW loop
 Vulnerability Srbds:                     Not affected
 Vulnerability Tsa:                       Not affected
 Vulnerability Tsx async abort:           Not affected
 Vulnerability Vmscape:                   Not affected
 
 ***CMake***
 /usr/bin/cmake
 cmake version 3.28.3
 
 CMake suite maintained and supported by Kitware (kitware.com/cmake).
 
 ***g++***
 /usr/bin/g++
 g++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
 Copyright (C) 2023 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
 
 ***nvcc***
 /usr/local/cuda/bin/nvcc
 nvcc: NVIDIA (R) Cuda compiler driver
 Copyright (c) 2005-2026 NVIDIA Corporation
 Built on Mon_Mar_02_09:52:23_PM_PST_2026
 Cuda compilation tools, release 13.2, V13.2.51
 Build cuda_13.2.r13.2/compiler.37434383_0
 
 ***Python***
 
 ***Environment Variables***
 PATH                            : /opt/proto/bin:/opt/tritonserver/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/mpi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin:/opt/amazon/efa/bin:/.local/bin
 LD_LIBRARY_PATH                 : /opt/tritonserver/lib:/opt/tritonclient/lib:/usr/src/tensorrt/lib:/opt/riva/lib:/usr/local/cuda-13/lib64:/usr/lib/x86_64-linux-gnu:/opt/hpcx/ucc/lib/:/opt/hpcx/ucx/lib/:/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/.local/lib
 NUMBAPRO_NVVM                   :
 NUMBAPRO_LIBDEVICE              :
 CONDA_PREFIX                    :
 PYTHON_PATH                     :
 
 conda not found
 ***pip packages***
 /usr/bin/pip
 Package             Version
 ------------------- ---------------
 annotated-doc       0.0.4
 annotated-types     0.7.0
 anyio               4.12.1
 apparmor            4.0.1
 certifi             2026.2.25
 cupy-cuda13x        13.6.0
 distlib             0.4.0
 distro              1.9.0
 fastapi             0.138.1
 fastrlock           0.8.3
 filelock            3.25.2
 h11                 0.16.0
 httpcore            1.0.9
 httpx               0.27.2
 idna                3.11
 iniconfig           2.3.0
 jaraco.context      6.1.0
 jiter               0.13.0
 LibAppArmor         4.0.1
 numpy               1.26.4
 openai              1.107.3
 packaging           24.0
 partial-json-parser 0.2.1.1.post7
 pip                 24.0
 platformdirs        4.9.4
 pluggy              1.6.0
 pydantic            2.10.6
 pydantic_core       2.27.2
 Pygments            2.19.2
 pyservicemaker      0.0.1
 pytest              9.0.2
 python-apt          2.7.7+ubuntu5.2
 python-discovery    1.1.3
 scipy               1.16.3
 setuptools          78.1.1
 sniffio             1.3.1
 starlette           0.52.1
 tqdm                4.67.3
 tritonfrontend      2.67.0
 tritonserver        0.0.0
 typing_extensions   4.15.0
 typing-inspection   0.4.2
 virtualenv          21.2.0
 wheel               0.42.0

Additional context

A real pipeline hits this on any element error: a TensorRT engine that does not deserialize, or a message broker that cannot reach Kafka. Today the only way a Python service can notice is a watchdog on the frame rate, and the reason is only in stderr. A pyds application could read errors from the GStreamer bus, and with pyds deprecated pyservicemaker has no way to do this.

Separately, the stock image cannot import pyservicemaker until pyyaml is installed (source_config.py imports yaml).

Ngôn ngữ chính
C++
Star
283
Fork
62
Merge trung bình
2 giờ 24 phút
Pull request đã merge (30 ngày)
20

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của NVIDIA/DeepStream

Tất cả issue của NVIDIA/DeepStream

Issue tương tự

Thêm issue về C++

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.