USD decoder and MJCF plugin copy colors between linear USD and sRGB rgba without conversion

オープン
#3,930 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 6 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
58/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
cpp, python

調査の方向性

Read plugin/usd_decoder/material_parsing.cc at ParsePreviewSurface, plugin/usd_decoder/usd_decoder.cc at ParseDisplayColorAndOpacity, and src/experimental/usd/plugins/mjcf/mujoco_to_usd.cc at the two color-writing paths. Compare decoded and exported values for both material diffuse color and display color, including a round trip; done means the decoder converts linear USD values to MuJoCo's expected color space and the plugin writes linear USD values. The payload does not name tests, so locate or add coverage for these paths.

索引モデルが issue の本文から書いたものです。

説明

My setup

MuJoCo 4.0.1 with mujoco-usd 4.0.1 from PyPI (usd-core 25.11), Python 3.12, Linux x86_64. The same code is on main.

What's happening? What did you expect?

The USD decoder copies UsdPreviewSurface inputs:diffuseColor into mjsMaterial.rgba and primvars:displayColor into mjsGeom.rgba without a color space conversion. USD authors both as linear values, while MuJoCo interprets rgba as sRGB (the Filament renderer defaults to rgb_color_type = mjCOLORSPACE_SRGB, and the classic renderer displays rgba without encoding). Decoded models therefore render darker than the source. An MJCF grey of 0.3, written to USD as linear 0.0732, decodes as rgba 0.0732.

mujoco-usd-converter converts MJCF rgba from sRGB to linear when it writes USD, so every converted asset with non-white or non-black colors is affected. For example, MuJoCo Menagerie's Berkeley Humanoid material metal (rgba=".3 .3 .3 1") renders nearly black after decoding.

The MJCF file format plugin has the mirror behavior: it writes rgba into inputs:diffuseColor and primvars:displayColor without converting to linear. The plugin and the decoder agree with each other, so MJCF -> plugin -> decoder round trips look correct, but USD authored by other tools decodes too dark, and plugin output renders too light in other USD applications.

I expected the decoder to convert linear USD colors to sRGB for rgba, and the MJCF plugin to convert rgba from sRGB to linear, so both match USD's linear convention. Texture color spaces are already handled through inputs:sourceColorSpace.

The relevant code at 4.0.1:

Steps for reproduction
  1. pip install mujoco-usd==4.0.1
  2. Run the code below. It builds a stage with a linear grey of 0.0732 (sRGB 0.3) as a UsdPreviewSurface diffuseColor on one cube and as displayColor on another, then decodes it. It also opens an MJCF with rgba 0.3 through the MJCF file format plugin.
  3. The decoder keeps the linear value in rgba for both cubes, and the plugin writes the sRGB value into the linear USD attributes.
USD linear color 0.0732 (sRGB 0.3)
  decoded material rgba: [0.0732 0.0732 0.0732]
  decoded displayColor rgba: [0.0732 0.0732 0.0732]
MJCF file format plugin, rgba 0.3:
  /MuJoCo_Model/Materials/grey/PreviewSurface inputs:diffuseColor = (0.3, 0.3, 0.3)
  /MuJoCo_Model/with_rgba primvars:displayColor = [(0.3, 0.3, 0.3)]
Minimal model for reproduction

The USD stage is built in memory by the code below. The MJCF opened through the file format plugin:

minimal XML
<mujoco>
  <asset><material name="grey" rgba="0.3 0.3 0.3 1"/></asset>
  <worldbody>
    <geom name="with_material" type="box" size="0.1 0.1 0.1" material="grey"/>
    <geom name="with_rgba" type="box" size="0.1 0.1 0.1" pos="0 0 1" rgba="0.3 0.3 0.3 1"/>
  </worldbody>
</mujoco>
Code required for reproduction
import tempfile
from pathlib import Path

import mujoco
from pxr import Gf, Sdf, Usd, UsdGeom, UsdPhysics, UsdShade


def srgb_to_linear(c: float) -> float:
    return c / 12.92 if c <= 0.04045 else ((c + 0.055) / 1.055) ** 2.4


# MJCF rgba 0.3 grey, written to USD as linear color the way UsdPreviewSurface and displayColor expect.
SRGB = 0.3
LINEAR = srgb_to_linear(SRGB)

stage = Usd.Stage.CreateInMemory()
UsdGeom.SetStageMetersPerUnit(stage, 1.0)
UsdGeom.SetStageUpAxis(stage, UsdGeom.Tokens.z)
UsdPhysics.Scene.Define(stage, "/PhysicsScene")

material = UsdShade.Material.Define(stage, "/World/Looks/grey")
shader = UsdShade.Shader.Define(stage, "/World/Looks/grey/PreviewSurface")
shader.CreateIdAttr("UsdPreviewSurface")
shader.CreateInput("diffuseColor", Sdf.ValueTypeNames.Color3f).Set(Gf.Vec3f(LINEAR))
shader.CreateInput("metallic", Sdf.ValueTypeNames.Float).Set(0.0)
shader.CreateInput("roughness", Sdf.ValueTypeNames.Float).Set(0.5)
material.CreateSurfaceOutput().ConnectToSource(shader.ConnectableAPI(), "surface")

body = UsdGeom.Xform.Define(stage, "/World/body").GetPrim()
UsdPhysics.RigidBodyAPI.Apply(body)
with_material = UsdGeom.Cube.Define(stage, "/World/body/with_material")
UsdShade.MaterialBindingAPI.Apply(with_material.GetPrim()).Bind(material)
UsdPhysics.CollisionAPI.Apply(with_material.GetPrim())
with_display_color = UsdGeom.Cube.Define(stage, "/World/body/with_display_color")
with_display_color.CreateDisplayColorAttr([Gf.Vec3f(LINEAR)])
with_display_color.AddTranslateOp().Set((0.0, 0.0, 3.0))

model = mujoco.MjSpec.from_file(stage.GetRootLayer().identifier).compile()
print(f"USD linear color {LINEAR:.4f} (sRGB {SRGB})")
geom = model.geom("/World/body/with_material")
print(f"  decoded material rgba: {model.mat_rgba[geom.matid[0]][:3].round(4)}")
print(f"  decoded displayColor rgba: {model.geom('/World/body/with_display_color').rgba[:3].round(4)}")

# The MJCF file format plugin writes rgba into USD without converting to linear.
with tempfile.TemporaryDirectory() as tmp:
    xml = Path(tmp) / "grey.xml"
    xml.write_text(
        f"""<mujoco>
  <asset><material name="grey" rgba="{SRGB} {SRGB} {SRGB} 1"/></asset>
  <worldbody>
    <geom name="with_material" type="box" size="0.1 0.1 0.1" material="grey"/>
    <geom name="with_rgba" type="box" size="0.1 0.1 0.1" pos="0 0 1" rgba="{SRGB} {SRGB} {SRGB} 1"/>
  </worldbody>
</mujoco>"""
    )
    mjcf_stage = Usd.Stage.Open(str(xml))
    print(f"MJCF file format plugin, rgba {SRGB}:")
    for prim in mjcf_stage.Traverse():
        if prim.IsA(UsdShade.Shader) and prim.GetAttribute("inputs:diffuseColor"):
            print(f"  {prim.GetPath()} inputs:diffuseColor = {prim.GetAttribute('inputs:diffuseColor').Get()}")
        color = prim.GetAttribute("primvars:displayColor")
        if color and color.HasAuthoredValue():
            print(f"  {prim.GetPath()} primvars:displayColor = {color.Get()}")
Confirmations
主要言語
C++
スター
18.3k
フォーク
2.2k
平均マージ
9日 11時間
マージ済み PR(30日)
5

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

google-deepmind/mujoco のほかの issue

google-deepmind/mujoco の issue をすべて見る

似ている issue

C++ の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。