Baseline schedule policy emits actions in the wrong order for SB1

Open Beginner friendly
  • Is something wrong?
#196 1 comment 0 reactions 0 assignees View on GitHub

@Sidharth1743 is already working on this.

Since Oct 9, 2026.

  • #197 by @Sidharth1743 — closed without merging
  • #200 by @Sidharth1743 — open

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
75/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
machine-learning

Research direction

Find create_baseline_schedule_policy and trace how action_sequence is used to construct the policy output; compare it with the order in env.action_normalizers. Start by locating the policy and the environment action setup, then check callers such as populate_starter_buffer.py. Done means each scheduled setpoint maps to the environment action with the matching name and the policy output follows environment action order.

Written by the indexing model from the issue text.

Description

Description

create_baseline_schedule_policy hard-codes action_sequence as
[supply_air_heating_temperature_setpoint, supply_water_setpoint].

The environment builds its action vector from building.devices (boiler,
then air handler), so on SB1 the expected order is
[supply_water_setpoint, supply_air_heating_temperature_setpoint].

populate_starter_buffer.py (and any other caller) feeds the policy output
to the environment unchanged, so each scheduled value is applied to the
other setpoint.

Impact

On a weekday daytime schedule the intended setpoints are:

  • supply water: 350 K
  • air heating: 292 K

With the swapped order (and SB1 action normalizer bounds), the environment
instead receives roughly:

  • supply water: ≈ 331 K
  • air heating: ≈ 298 K

Most of the reward gap shows up between 07:00 and 12:00 local time.
This corrupts starter replay-buffer data and baseline benchmarking.

Reward impact (SB1, 64 days)
Mapping Episode return
Intended (each scheduled setpoint on its matching action) −30.77 ± 0.05 (95% CI)
As-run (policy output fed unchanged / swapped) −43.51 ± 0.07

The swapped order costs about 13 return points per episode versus theschedule the policy is meant to apply.

Expected behavior

Each scheduled setpoint should land on the environment action with the
matching name, in the environment's action order.

Proposed fix

Derive action_sequence from env.action_normalizers (same order as the
action vector) by matching setpoint name suffixes, instead of hard-coding
[AC, HWS].

Dominant language
Python
Stars
132
Forks
64
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from google/sbsim

All issues in google/sbsim

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.