D-FINE engine with fp16 has a degraded performance with TensorRT > 10.4

Open
  • Is something wrong?
#7 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Quiet
Tech stack
python
Domain
machine-learning, performance

Research direction

Start by reproducing the D-FINE fp16 export with TensorRT 10.4 and 10.6 using the shown build_engine function. Inspect the NORMALIZATION precision handling, STRICT_TYPES, and precision flags, then compare engine precision and performance. Done means identifying the TensorRT-version regression and documenting or implementing a validated fix.

Written by the indexing model from the issue text.

Description

Hi,

Thanks for the great library.
I wanted to test DEIMv2 and Edgecrafter, but I need to use tensorrt >= 10.6.
So, I installed this repo with tensorrt==10.6, but D-FINE precision drops significantly.
Do you have any idea why this is happening? I read that I had to export the layernorms on FP32 and the other layers in FP16 to make D-FINE work, but it didn't work.

Here is my exportation code


from importlib.metadata import version
from packaging.version import Version

required_version = Version('10.4')
installed_version = Version(version('tensorrt'))

def build_engine(model_path, engine_path, use_fp16=False):
    logger = trt.Logger([trt.Logger.INFO](http://trt.logger.info/))
    builder = trt.Builder(logger)

    EXPLICIT_BATCH = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
    network = builder.create_network(EXPLICIT_BATCH)


    config = builder.create_builder_config()
    if use_fp16:
        config.set_flag(trt.BuilderFlag.FP16)
        # MY ADDITION
        if installed_version > required_version:
            config.clear_flag(trt.BuilderFlag.OBEY_PRECISION_CONSTRAINTS)
            config.set_flag(trt.BuilderFlag.PREFER_PRECISION_CONSTRAINTS)
            for layer_idx in range(network.num_layers):
                layer = network[layer_idx]
                if layer.type == trt.LayerType.NORMALIZATION:
                    layer.precision = trt.float32
                    layer.set_output_type(0, trt.float32)

    parser = trt.OnnxParser(network, logger)
    config.set_flag(trt.BuilderFlag.STRICT_TYPES)

    with open(model_path, "rb") as f:
        model_data = f.read()
   
    if not parser.parse(model_data):
        print("Failed to parse ONNX model")
        for error in range(parser.num_errors):
            print(parser.get_error(error))
        return None

    # Create optimization profile to fix dynamic batch dimensions
    profile = builder.create_optimization_profile()
   
    # Handle dynamic input shapes - fix batch size to 1
    for i in range(network.num_inputs):
        input_tensor = network.get_input(i)
        input_shape = input_tensor.shape
        print(f"Input {i} ({[input_tensor.name](http://input_tensor.name/)}): {input_shape}")
       
        # Check if batch dimension is dynamic (typically -1)
        if input_shape[0] == -1:
            # Fix batch size to 1
            fixed_shape = (1,) + tuple(input_shape[1:])
            print(f"  Setting fixed batch shape: {fixed_shape}")
           
            # Set min, optimal, and max shapes all to batch size 1
            profile.set_shape([input_tensor.name](http://input_tensor.name/), fixed_shape, fixed_shape, fixed_shape)

    # Add the optimization profile to the configuration
    config.add_optimization_profile(profile)
    config.set_flag(trt.BuilderFlag.STRICT_TYPES)

    print(f"Building engine from {model_path} to {engine_path}")
    engine = builder.build_serialized_network(network, config)
   
    if engine is None:
        print("Failed to build engine")
        return None
       
    print(f"Engine built successfully")

    with open(engine_path, "wb") as f:
        f.write(engine)

    return engine
Dominant language
Python
Stars
45
Forks
3
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from roboflow/single_artifact_benchmarking

All issues in roboflow/single_artifact_benchmarking

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.