[BUG] Slow read performance for multichannel EXRs in oiiotool

オープン
  • 何か問題がありますか？
#4,707 コメント 5 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 2 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
45/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
静か
技術スタック
cpp
領域
performance

調査の方向性

提供された EXR と oiiotool -v -i:ch=R,G,B,A input.exr -o output.png で問題を再現し、リンク先のテスト実行と所要時間を比較します。マルチチャンネル EXR の読み取りパスと、チャンネルフィルタリングの影響を調査します。RGBA のみへの変換で報告されている過剰な読み取り時間が発生しなくなり、関連する所要時間が記録されていれば完了です。

索引モデルが issue の本文から書いたものです。

説明

performance

Describe the bug

I'm getting very slow read performance on multi-channel EXRs where I just need to parse the RGBA channel and write out a .png file from it.

From 19s on my own machine (which is crazy for a 55 MB exr?) to 5s on the fastest machine I have around.
On my own slow machine, taking the same EXR (on no cold/hot cache) through Fusion to a PNG takes 1.3s

The 19 seconds (but 1.3s through fusion) is on:

  • Windows 10
  • 2x Intel(R) Xeon(R) CPU E5-2680 0 @2.70Ghz
    • Dual Xeon in HP workstation
  • 128 GB RAM
  • 4 GB GPU

The 5 seconds is on:

  • Windows 11
  • AMD Ryzen 9 9900X 12-Core Processor (4.40 Ghz)
  • 128 GB RAM
  • 12 GB GPU

OpenImageIO version and dependencies

See test runs here.

OIIO 3.1.1.0dev | unknown arch?
    Build compiler: MSVS 1943 | C++17/199711
    HW features enabled at build: sse2
    No CUDA support (disabled / unavailable at build time)
Dependencies: DCMTK NONE, expat 2.6.3, FFmpeg NONE, fmt 10.2.1, Freetype 2.13.2, GIF NONE, Imath 3.2.0, JPEG 80,
    JXL NONE, libdeflate 1.23, Libheif NONE, libjpeg-turbo 3.0.4, LibRaw NONE, libuhdr 1.2.0, minizip-ng 4.0.7,
    OpenColorIO 2.4.1, OpenCV NONE, OpenEXR 3.4.0, OpenJPEG NONE, OpenVDB NONE, PNG 1.6.47, Ptex NONE, Ptex NONE,
    pystring 1.1.4, Robinmap 1.4.0, TBB 2022.2.0, TIFF 4.6.0, WebP 1.4.0, yaml-cpp 0.8.0, ZLIB 1.3.1, ZLIB 1.3.1

To Reproduce

Steps to reproduce the behavior:

  1. oiiotool -v -i:ch=R,G,B,A input.exr -o output.png

I'm explicitly filtering -i:ch=R,G,B,A to avoid reading the rest of the file. Also adding :now=1 doesn't make much of a difference.
Other options, like also passing -ch flag are included in the linked test runs with their timings. Using i:ch=... gave the fastest results, but still at 19s seems awful.

Here's the file I'm testing with: https://we.tl/t-Fjm4XVFw8z
It'll only remain available for about four weeks.

Evidence

See test runs here.

Additional Context

This also came up on ASWF slack with a lengthy thread here

Some notable comments from there:

Build options didn't make much of a difference?

So, I've been testing some build variants.
When enabling more hardware features like SSSE3, SSE4.1 SSE4.2, AVX etc. makes it so that the executable doesn't work on my machine - likely due to some AVX (it does work on newer machines but there was no real measurable performance difference on this simple read/write test cases) so I opted going back to just the sse2 default but adding OpenTBB which does run. I've added the performance on my slow machine to the gist (see link below)
There is no noticable speed difference. It remains around 18/19 seconds on my machine

For what it's worth, using --frames 1001-1006 and --parallel-frames also takes about 20 seconds on my machine (so almost no time increase).
Using --frames 1001-1030 with --parallel-frames takes about 53 seconds on my machine.

So I can get good gains with allowing it to read many files. However, it's still the majority the read performance taking a long time it seems.

主要言語
C++
スター
2.4k
フォーク
718
平均マージ
2日 18時間
マージ済み PR（30日）
85

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

AcademySoftwareFoundation/OpenImageIO のほかの issue

AcademySoftwareFoundation/OpenImageIO の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。