NVIDIA/DALI

conditional execution error while dali model is loaded by tritonserver

Open

#4,750 创建于 2023年3月28日

在 GitHub 查看
 (9 评论) (1 反应) (1 负责人)C++ (670 fork)auto 404
conditional executionhelp wanted

仓库指标

Star
 (5,722 star)
PR 合并指标
 (PR 指标待抓取)

描述

I'm building a face keypoints detect system which the model is served by tritonserver. I use dali to do some pre-process steps for face keypoints step. The triton accept origin image and face boundingbox which is the output of face detect model, then crop the face roi image from origin image. Up to this #4735 , I must use conditional execution to crop multiple face from each origin image. My GPU info is below: 微信图片_20230328173011

My tritonserver container version is 21.11 and I reinstall dali by pip install https://developer.download.nvidia.com/compute/redist/nightly/nvidia-dali-nightly-cuda110/nvidia_dali_nightly_cuda110-1.25.0.dev20230327-7720921-py3-none-manylinux2014_x86_64.whl

Below is my code called dali.py.

import nvidia.dali as dali
import nvidia.dali.types as types
from nvidia.dali.pipeline.experimental import pipeline_def

BATCH_SIZE = 8
INPUT_SIZE = 192
FILL_VALUE = 0

def parse_args():
    import argparse
    parser = argparse.ArgumentParser(description="Serialize the pipeline and save it to a file")
    parser.add_argument('file_path', type=str, help='The path where to save the serialized pipeline')
    return parser.parse_args()


def get_warp_matrix(bboxes_size, bboxes_center, bboxes_index):
    bbox_size = bboxes_size[bboxes_index]
    bbox_center = bboxes_center[bboxes_index]
    bbox_start = bbox_center - bbox_size / 2.0
    bbox_end = bbox_center + bbox_size / 2.0
    mt = dali.fn.transforms.crop(from_start=bbox_start, from_end=bbox_end, to_start=[0, 0],
    to_end=[INPUT_SIZE, INPUT_SIZE])
    return mt


def crop_images(mt, images):
    cropped_images = dali.fn.warp_affine(images, size=[INPUT_SIZE, INPUT_SIZE], matrix=mt, fill_value=FILL_VALUE,
    inverse_map=False)
    cropped_images = dali.fn.color_space_conversion(cropped_images, image_type=types.RGB, output_type=types.BGR)
    return cropped_images


@pipeline_def(batch_size=BATCH_SIZE, num_threads=1, device_id=0, enable_conditionals=True)
def simple_pipeline():
    image = dali.fn.external_source(device="cpu", name="DALI_INPUT_0")
    face_bboxes = dali.fn.external_source(device="cpu", name="DALI_INPUT_1")
    bboxes_h = face_bboxes[:, 3] - face_bboxes[:, 1]
    bboxes_h = bboxes_h[:, dali.newaxis]
    bboxes_w = face_bboxes[:, 2] - face_bboxes[:, 0]
    bboxes_w = bboxes_w[:, dali.newaxis]
    bboxes_size = dali.fn.cat(bboxes_h, bboxes_w, axis=1)
    bboxes_size = dali.fn.cast(bboxes_size, dtype=types.FLOAT)
    center_x = (face_bboxes[:, 0] + face_bboxes[:, 2]) / 2
    center_x = center_x[:, dali.newaxis]
    center_y = (face_bboxes[:, 1] + face_bboxes[:, 3]) / 2
    center_y = center_y[:, dali.newaxis]
    center = dali.fn.cat(center_x, center_y, axis=1)
    center = dali.fn.cast(center, dtype=types.FLOAT)
    scale = INPUT_SIZE / (dali.fn.reductions.max(bboxes_size, axes=[1]) * 1.5)
    scale = scale[:, dali.newaxis]
    bboxes_size = dali.fn.cast(bboxes_size, dtype=types.FLOAT) * scale
    face_bounding_num = dali.fn.shapes(face_bboxes)[0]

    # if has one face bounding box
    if face_bounding_num == 1:
        mt = get_warp_matrix(bboxes_size, center, 0)
        cropped_image = crop_images(mt, image)
        out = dali.fn.reshape(cropped_image, src_dims=[-1, 0, 1, 2])

    # if has two face bounding box
    elif face_bounding_num == 2:
        mt_1 = get_warp_matrix(bboxes_size, center, 0)
        cropped_image_1 = crop_images(mt_1, image)
        mt_2 = get_warp_matrix(bboxes_size, center, 1)
        cropped_image_2 = crop_images(mt_2, image)
        out = dali.fn.stack(cropped_image_1, cropped_image_2)

    # if has three face bounding box
    elif face_bounding_num == 3:
        mt_1 = get_warp_matrix(bboxes_size, center, 0)
        cropped_image_1 = crop_images(mt_1, image)
        mt_2 = get_warp_matrix(bboxes_size, center, 1)
        cropped_image_2 = crop_images(mt_2, image)
        mt_3 = get_warp_matrix(bboxes_size, center, 2)
        cropped_image_3 = crop_images(mt_3, image)
        out = dali.fn.stack(cropped_image_1, cropped_image_2, cropped_image_3)

    # if has more than three bounding box
    else:
        mt_1 = get_warp_matrix(bboxes_size, center, 0)
        cropped_image_1 = crop_images(mt_1, image)
        mt_2 = get_warp_matrix(bboxes_size, center, 1)
        cropped_image_2 = crop_images(mt_2, image)
        mt_3 = get_warp_matrix(bboxes_size, center, 2)
        cropped_image_3 = crop_images(mt_3, image)
        mt_4 = get_warp_matrix(bboxes_size, center, 3)
        cropped_image_4 = crop_images(mt_4, image)
        out = dali.fn.stack(cropped_image_1, cropped_image_2, cropped_image_3, cropped_image_4)
    out = dali.fn.transpose(out, perm=[0, 3, 1, 2])
    return out


def main(filename):
    simple_pipeline().serialize(filename=filename)


if __name__ == '__main__':
    args = parse_args()
    main(args.file_path)

I serialized this to format that can by load by triton by pythhon3 dali.py model.dali. This step produce no error,but when I load it by trion. It produce some errors, below is the detailed log 微信图片_20230328171151 the error message: Unknown: DALI Backend error: [/opt/dali/dali/pipeline/operator/op_spec.h:87] Assert on "schema_ != nullptr" failed: No schema found for operator "_conditional__Split" seems like it has something to do with conditional execution, anyone can tell me how can I correct it.

贡献者指南