Online adaptation advances observation history multiple times per policy query
Los mantenedores suelen responder en 1 día
@sylvesterkaczmarek ya está trabajando en esto.
Desde el 30/10/2026.
- #67 de @sylvesterkaczmarek — abierto
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 52/100
- Tipo de issue
- Error
- Claridad
- Bien especificado
- Estado de actividad
- Activo
- Área
- machine-learning
Línea de trabajo
Start with _prepare, embed_for_buffer, propose_noise, act, and invert in rho/online/adaptation.py, then trace the call sequence in environments/metaworld/online_adapt.py’s run_episode. Reproduce the two-frame example and inspect how history and padding change across helper calls. Done when history advances once per query, inversion uses that query’s snapshot, episode resets clear history, and regression coverage exercises the listed rollout cases.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Problem
A single policy query in environments/metaworld/online_adapt.py calls embed_for_buffer(chunk_obs), optionally propose_noise(chunk_obs), and then act(chunk_obs, noise). Each path calls PolicyInterface.process_observation(), which appends the observation to its temporal queues. Finalizing a corrected action chunk calls invert(chunk_obs, ...), which appends the same starting observation again.
Consequently, observation history represents helper calls rather than distinct incoming observations. The conditioning used to collect a noise-policy training example can differ from the conditioning used to generate or invert its action chunk. Padding masks can also claim that an incomplete history is fully populated.
Affected code at 148bb1ae75b92a1162172ea36d8e5ceb830ceebb:
rho/online/adaptation.py:_prepare,embed_for_buffer,propose_noise,act, andinvertenvironments/metaworld/online_adapt.py:run_episode
Reproduction
Use a policy with observation deltas [-2, -1, 0], identity input transforms, and successive single-frame observations with values 1 and 2. For each query, follow the rollout's sequence:
embedding = adapter.embed_for_buffer(obs)
noise = adapter.propose_noise(obs)
actions = adapter.act(obs, noise)
On the second query, the model should see history [1, 1, 2] and padding mask [True, False, False]. It instead sees [2, 2, 2] with [False, False, False]. The first query's padding is already wrong after the repeated insertions.
This was reproduced through the actual run_episode() and PolicyInterface using a deterministic local environment and a lightweight policy, without model downloads or physical hardware. The Gaussian-noise path is also affected: it still calls both embed_for_buffer() and act(). Inversion of corrected chunks adds another history update.
The rollout also resets the environment without resetting PolicyInterface, so multi-step history can cross episode boundaries. Single-frame policies are unaffected by this particular history-ordering failure. This report does not establish an error in any published benchmark result.
Proposed fix
Prepare one explicit observation snapshot per query, advancing the history once. Reuse its transformed inputs for embedding, noise proposal, and action generation. Retain its raw, remapped history separately so executed physical actions can be normalized against the original query state during inversion, without advancing the queues or consulting a later query's history.
Reset the interface at episode boundaries. Add regression coverage for real rollout call sequences, Gaussian and learned noise, corrected and uncorrected chunks, history padding, repeated helper calls, consecutive multi-frame observations, delayed inversion, and reused environment buffers.
- Lenguaje dominante
- Python
- Estrellas
- 38
- Forks
- 6
- Merge medio
- 2 d 9 h
- PR fusionados (30 d)
- 20
Preparar el entorno
Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de microsoft/rhobotics
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 1 día
-
Dificultad 4/5 3-5 días Aptitud para principiantes 48/100
Los mantenedores suelen responder en 1 día
-
Dificultad 3/5 1-2 días Aptitud para principiantes 45/100
Los mantenedores suelen responder en 1 día
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 25/100
Los mantenedores suelen responder en 1 día
Todos los issues de microsoft/rhobotics
Issues similares
-
Dificultad 1/5 1-3 horas Aptitud para principiantes 76/100
samuelclay/NewsBlur#2214 ·
Los mantenedores suelen responder en 1 día
-
MODEL_NOT_FOUND gives no suggestion for a model listed only as model:profilePosiblemente ocupada @ShaiShmuel la tomó hoy. Abiertoenhancement rust
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
superlinked/sie#985 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
readthedocs/readthedocs-builder#73 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
reflex-dev/reflex#7915 · 1 comentario ·
Los mantenedores suelen responder en 1 día