openvinotoolkit/openvino
Voir sur GitHub[Bug]: inconsistent inference results between OpenVINO and ONNX
Open
#20 872 ouverte le 6 nov. 2023
bugcategory: ONNX FEgood first issue
Métriques du dépôt
- Stars
- (10 286 stars)
- Métriques de merge PR
- (Merge moyen 14j 3h) (305 PRs mergées en 30 j)
Description
OpenVINO Version
openvino-nightly 2023.2.0.dev20231101
Operating System
Ubuntu 18.04 (LTS)
Device used for inference
CPU
Framework
ONNX
Model used
https://github.com/jikechao/onnx_models/blob/main/ReverseSequence.onnx
Issue description
For the Given model, OpenVINO and ONNX gave different inference results.
Step-by-step reproduction
Step 1:
Download the model and put it into the same directory with the following test script.
Step 2:
Run the test script:
import onnxruntime as ort
import openvino as ov
import numpy as np
onnx_model_path = './ReverseSequence.onnx'
session = ort.InferenceSession(onnx_model_path)
input_x = np.random.random([4, 4]).astype(np.float32)
input_seq = np.random.randint(0, 2, [4], dtype=np.int64)
input_data = {"x": input_x, "sequence_lens": input_seq}
output_name = session.get_outputs()[0].name
onnx_output = session.run([output_name], input_data)[0]
ov_model = ov.convert_model(onnx_model_path)
ir_path = f"temp_OVIR.xml"
ov.save_model(ov_model, ir_path, compress_to_fp16=False)
core = ov.Core()
model = core.read_model(ir_path)
compiled_model = core.compile_model(model=model, device_name="CPU")
# show the model structure
# input_key = compiled_model.input(0)
output_key = compiled_model.output(0)
# network_input_shape = input_key.shape
ov_result = compiled_model(input_data)[output_key]
np.testing.assert_allclose(onnx_output, ov_result, atol=1e-3)
Relevant log output
AssertionError:
Not equal to tolerance rtol=1e-07, atol=0.001
Mismatched elements: 8 / 16 (50%)
Max absolute difference: 0.885355
Max relative difference: 1.
x: array([[0.046832, 0.807565, 0. , 0. ],
[0.717256, 0.383817, 0. , 0. ],
[0.798318, 0.265362, 0. , 0. ],
[0.88803 , 0.997446, 0. , 0. ]], dtype=float32)
y: array([[0.046832, 0.807565, 0.164403, 0.664873],
[0.717256, 0.383817, 0.127707, 0.20239 ],
[0.798318, 0.265362, 0.824921, 0.604113],
[0.88803 , 0.997446, 0.803242, 0.885355]], dtype=float32)
Process finished with exit code 1
Issue submission checklist
- I'm reporting an issue. It's not a question.
- I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
- There is reproducer code and related data files such as images, videos, models, etc.