NVIDIA/DALI

fn.random_bbox_crop() to get the same absolute w and h

Open

#3,303 opened on Aug 31, 2021

View on GitHub
 (4 comments) (0 reactions) (0 assignees)C++ (670 forks)auto 404
help wanted

Repository metrics

Stars
 (5,722 stars)
PR merge metrics
 (PR metrics pending)

Description

Hello, I get errors when I use function fn.random_bbox_crop. For Example: input_image: shape(1280, 768) or other, I want that the crop image has the same absolute width and height, but when I set aspect_ratio=[1., 1.], I can only get the same relative width and height. The crop image has the same aspect_ratio as input image. How can I get crop image with the same absolute width and height, please.

I try to set

fixed_aspect_ratio = src_w / src/h fn.random_bbox_crop(aspect_ratio = fixed_aspect_ratio)

But I meet errors: TypeError: float() argument must be a string or a number, not 'DataNode'. It seems that I cannot fixed set aspect_ratio

Code: DALI/docs/examples/use_cases/detection_pipeline.ipynb

def coco_reader_def():
    inputs, bboxes, labels, polygons, vertices = fn.readers.coco(
        file_root=file_root,
        annotations_file=annotations_file,
        polygon_masks=True, # Load segmentation mask data as polygons 
        ratio=True,         # Bounding box and mask polygons to be expressed in relative coordinates
        ltrb=True,          # Bounding boxes to be expressed as left, top, right, bottom coordinates
    )
    return inputs, bboxes, labels, polygons, vertices

def random_bbox_crop_def(bboxes, labels, polygons, vertices, aspect_ratio):
    # RandomBBoxCrop works with relative coordinates
    # The arguments have been selected to produce a significantly visible crop
    # To learn about all the available options, see the documentation
    anchor_rel, shape_rel, bboxes, labels, bbox_indices = fn.random_bbox_crop(
        bboxes,
        labels,
        aspect_ratio=[aspect_ratio, aspect_ratio],     # Range of aspect ratios
        thresholds=[0.0],          # No minimum intersection-over-union, for demo purposes
        allow_no_crop=True,       # No-crop is disallowed, for demo purposes 
        scaling=[0.3, 1.],        # Scale range of the crop with respect to the image shape
        seed=12345,                # Fixed random seed for deterministic results
        bbox_layout="xyXY",        # left, top, right, back
        output_bbox_indices=True,  # Output indices of the filtered bounding boxes
    )
    
    # Select mask polygons of those bounding boxes that remained in the image
    polygons, vertices = fn.segmentation.select_masks(
        bbox_indices, polygons, vertices
    )
    
    return anchor_rel, shape_rel, bboxes, labels, polygons, vertices

pipe = Pipeline(batch_size=batch_size, num_threads=num_threads, device_id=device_id)
with pipe:
    inputs, bboxes, labels, polygons, vertices = coco_reader_def()

    input_shape = fn.peek_image_shape(inputs)
    w = fn.slice(input_shape, 0, 1, axes = [0], dtype=types.FLOAT)
    h = fn.slice(input_shape, 1, 1, axes = [0], dtype=types.FLOAT)

    anchor_rel, shape_rel, bboxes, labels, polygons, vertices = \
        random_bbox_crop_def(bboxes, labels, polygons, vertices, w/h)
    
    # Partial decoding of the image
    images = fn.decoders.image_slice(
        inputs, anchor_rel, shape_rel, normalized_anchor=True, normalized_shape=True, device='cpu'
    )
    # Cropped image dimensions
    crop_shape = fn.shapes(images, dtype=types.FLOAT)
    crop_h = fn.slice(crop_shape, 0, 1, axes=[0])
    crop_w = fn.slice(crop_shape, 1, 1, axes=[0])

    images = images.gpu()

    # Adjust masks coordinates to the coordinate space of the cropped image, while also converting
    # relative to absolute coordinates by mapping the top-left corner (anchor_rel_x, anchor_rel_y), to (0, 0) 
    # and the bottom-right corner (anchor_rel_x+shape_rel_x, anchor_rel_y+shape_rel_y) to (crop_w, crop_h)
    MT_vertices = fn.transforms.crop(
        from_start=anchor_rel, from_end=(anchor_rel + shape_rel),
        to_start=(0.0, 0.0), to_end=fn.cat(crop_w, crop_h)
    )    
    vertices = fn.coord_transform(vertices, MT=MT_vertices)
    
    # Convert bounding boxes to absolute coordinates
    MT_bboxes = fn.transforms.crop(
        to_start=(0.0, 0.0, 0.0, 0.0), to_end=fn.cat(crop_w, crop_h, crop_w, crop_h)
    )
    bboxes = fn.coord_transform(bboxes, MT=MT_bboxes)
    
    pipe.set_outputs(images, bboxes, labels, polygons, vertices)

pipe.build()
outputs = pipe.run()
show(outputs)

Contributor guide