USD decoder and MJCF plugin copy colors between linear USD and sRGB rgba without conversion

Abierto
#3,930 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 6 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
58/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
cpp, python

Línea de trabajo

Read plugin/usd_decoder/material_parsing.cc at ParsePreviewSurface, plugin/usd_decoder/usd_decoder.cc at ParseDisplayColorAndOpacity, and src/experimental/usd/plugins/mjcf/mujoco_to_usd.cc at the two color-writing paths. Compare decoded and exported values for both material diffuse color and display color, including a round trip; done means the decoder converts linear USD values to MuJoCo's expected color space and the plugin writes linear USD values. The payload does not name tests, so locate or add coverage for these paths.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

My setup

MuJoCo 4.0.1 with mujoco-usd 4.0.1 from PyPI (usd-core 25.11), Python 3.12, Linux x86_64. The same code is on main.

What's happening? What did you expect?

The USD decoder copies UsdPreviewSurface inputs:diffuseColor into mjsMaterial.rgba and primvars:displayColor into mjsGeom.rgba without a color space conversion. USD authors both as linear values, while MuJoCo interprets rgba as sRGB (the Filament renderer defaults to rgb_color_type = mjCOLORSPACE_SRGB, and the classic renderer displays rgba without encoding). Decoded models therefore render darker than the source. An MJCF grey of 0.3, written to USD as linear 0.0732, decodes as rgba 0.0732.

mujoco-usd-converter converts MJCF rgba from sRGB to linear when it writes USD, so every converted asset with non-white or non-black colors is affected. For example, MuJoCo Menagerie's Berkeley Humanoid material metal (rgba=".3 .3 .3 1") renders nearly black after decoding.

The MJCF file format plugin has the mirror behavior: it writes rgba into inputs:diffuseColor and primvars:displayColor without converting to linear. The plugin and the decoder agree with each other, so MJCF -> plugin -> decoder round trips look correct, but USD authored by other tools decodes too dark, and plugin output renders too light in other USD applications.

I expected the decoder to convert linear USD colors to sRGB for rgba, and the MJCF plugin to convert rgba from sRGB to linear, so both match USD's linear convention. Texture color spaces are already handled through inputs:sourceColorSpace.

The relevant code at 4.0.1:

Steps for reproduction
  1. pip install mujoco-usd==4.0.1
  2. Run the code below. It builds a stage with a linear grey of 0.0732 (sRGB 0.3) as a UsdPreviewSurface diffuseColor on one cube and as displayColor on another, then decodes it. It also opens an MJCF with rgba 0.3 through the MJCF file format plugin.
  3. The decoder keeps the linear value in rgba for both cubes, and the plugin writes the sRGB value into the linear USD attributes.
USD linear color 0.0732 (sRGB 0.3)
  decoded material rgba: [0.0732 0.0732 0.0732]
  decoded displayColor rgba: [0.0732 0.0732 0.0732]
MJCF file format plugin, rgba 0.3:
  /MuJoCo_Model/Materials/grey/PreviewSurface inputs:diffuseColor = (0.3, 0.3, 0.3)
  /MuJoCo_Model/with_rgba primvars:displayColor = [(0.3, 0.3, 0.3)]
Minimal model for reproduction

The USD stage is built in memory by the code below. The MJCF opened through the file format plugin:

minimal XML
<mujoco>
  <asset><material name="grey" rgba="0.3 0.3 0.3 1"/></asset>
  <worldbody>
    <geom name="with_material" type="box" size="0.1 0.1 0.1" material="grey"/>
    <geom name="with_rgba" type="box" size="0.1 0.1 0.1" pos="0 0 1" rgba="0.3 0.3 0.3 1"/>
  </worldbody>
</mujoco>
Code required for reproduction
import tempfile
from pathlib import Path

import mujoco
from pxr import Gf, Sdf, Usd, UsdGeom, UsdPhysics, UsdShade


def srgb_to_linear(c: float) -> float:
    return c / 12.92 if c <= 0.04045 else ((c + 0.055) / 1.055) ** 2.4


# MJCF rgba 0.3 grey, written to USD as linear color the way UsdPreviewSurface and displayColor expect.
SRGB = 0.3
LINEAR = srgb_to_linear(SRGB)

stage = Usd.Stage.CreateInMemory()
UsdGeom.SetStageMetersPerUnit(stage, 1.0)
UsdGeom.SetStageUpAxis(stage, UsdGeom.Tokens.z)
UsdPhysics.Scene.Define(stage, "/PhysicsScene")

material = UsdShade.Material.Define(stage, "/World/Looks/grey")
shader = UsdShade.Shader.Define(stage, "/World/Looks/grey/PreviewSurface")
shader.CreateIdAttr("UsdPreviewSurface")
shader.CreateInput("diffuseColor", Sdf.ValueTypeNames.Color3f).Set(Gf.Vec3f(LINEAR))
shader.CreateInput("metallic", Sdf.ValueTypeNames.Float).Set(0.0)
shader.CreateInput("roughness", Sdf.ValueTypeNames.Float).Set(0.5)
material.CreateSurfaceOutput().ConnectToSource(shader.ConnectableAPI(), "surface")

body = UsdGeom.Xform.Define(stage, "/World/body").GetPrim()
UsdPhysics.RigidBodyAPI.Apply(body)
with_material = UsdGeom.Cube.Define(stage, "/World/body/with_material")
UsdShade.MaterialBindingAPI.Apply(with_material.GetPrim()).Bind(material)
UsdPhysics.CollisionAPI.Apply(with_material.GetPrim())
with_display_color = UsdGeom.Cube.Define(stage, "/World/body/with_display_color")
with_display_color.CreateDisplayColorAttr([Gf.Vec3f(LINEAR)])
with_display_color.AddTranslateOp().Set((0.0, 0.0, 3.0))

model = mujoco.MjSpec.from_file(stage.GetRootLayer().identifier).compile()
print(f"USD linear color {LINEAR:.4f} (sRGB {SRGB})")
geom = model.geom("/World/body/with_material")
print(f"  decoded material rgba: {model.mat_rgba[geom.matid[0]][:3].round(4)}")
print(f"  decoded displayColor rgba: {model.geom('/World/body/with_display_color').rgba[:3].round(4)}")

# The MJCF file format plugin writes rgba into USD without converting to linear.
with tempfile.TemporaryDirectory() as tmp:
    xml = Path(tmp) / "grey.xml"
    xml.write_text(
        f"""<mujoco>
  <asset><material name="grey" rgba="{SRGB} {SRGB} {SRGB} 1"/></asset>
  <worldbody>
    <geom name="with_material" type="box" size="0.1 0.1 0.1" material="grey"/>
    <geom name="with_rgba" type="box" size="0.1 0.1 0.1" pos="0 0 1" rgba="{SRGB} {SRGB} {SRGB} 1"/>
  </worldbody>
</mujoco>"""
    )
    mjcf_stage = Usd.Stage.Open(str(xml))
    print(f"MJCF file format plugin, rgba {SRGB}:")
    for prim in mjcf_stage.Traverse():
        if prim.IsA(UsdShade.Shader) and prim.GetAttribute("inputs:diffuseColor"):
            print(f"  {prim.GetPath()} inputs:diffuseColor = {prim.GetAttribute('inputs:diffuseColor').Get()}")
        color = prim.GetAttribute("primvars:displayColor")
        if color and color.HasAuthoredValue():
            print(f"  {prim.GetPath()} primvars:displayColor = {color.Get()}")
Confirmations
Lenguaje dominante
C++
Estrellas
18.3k
Forks
2.2k
Merge medio
9 d 11 h
PR fusionados (30 d)
5

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de google-deepmind/mujoco

Todos los issues de google-deepmind/mujoco

Issues similares

Más issues de C++

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.