[Bug] RGB_TOG wont work until on right half of split keyboard until audio is cycled

Describe the Bug

On my snap I have a standard configuration with with a rotary encoder on the right and left sides (and each with its own MCU of course). I am using the bit-c pro rp2040.

My current code/keymap/etc is here: https://git.qoto.org/freemo/qmk-firmware/-/tree/freemo/keyboards/nullbitsco/snap/rp2040/keymaps/freemo

The problem only happens when I enable the audio in rules as I do on my branch. If my keyboard comes up with the LED underglow lit, and the audio active (and only then) then pressing my right rotary encoder, which is mapped to RGB_TOG. Now strangly enough if I turn the audio off then on again using the left rotary encoder, pressing it, then I can use the right rotary encoder just fine.

Another thing that is closely related, I notice if I hit the right encoder once, it does nothing. But when I turn the audio off it immediately turns off the LED, so it does change the internal state (so the button press is registered). But for some reason the firmware doesnt update the LED state. I think this is a bug?

Anyway I can probably call an internal function to force update the leds... ill come back with a work around if I find one...

Keyboard Used

Nullbits snap

Link to product page (if applicable)

https://nullbits.co/snap/

Operating System

NixOS - Linux

qmk doctor Output

❯ qmk doctor                   
Ψ QMK Doctor is checking your environment.
Ψ CLI version: 1.1.2
Ψ QMK home: /home/freemo/source/qmk_firmware
Ψ Detected Linux (NixOS 23.11 (Tapir)).
⚠ Missing or outdated udev rules for 'atmel-dfu' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'kiibohd' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'stm32-dfu' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'apm32-dfu' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'gd32v-dfu' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'wb32-dfu' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'bootloadhid' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'usbasploader' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'usbtinyisp' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'md-boot' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Detected ModemManager without the necessary udev rules. Please either disable it or set the appropriate udev rules if you are using a Pro Micro.
⚠ Missing or outdated udev rules for 'caterina' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
⚠ Missing or outdated udev rules for 'hid-bootloader' boards. Run 'sudo cp /home/freemo/source/qmk_firmware/util/udev/50-qmk.rules /etc/udev/rules.d/'.
Ψ Git branch: freemo
Ψ Repo version: 0.22.14
⚠ Git has unstashed/uncommitted changes.
Ψ - Latest freemo: 2023-11-11 15:28:27 -0500 (1a680e0db3) -- Fixed bug where usb on slave side caused oled to render improperly
Ψ - Latest upstream/master: 2023-11-09 04:03:07 +0200 (daabe2d8c5) -- Update layer state of Lily58 from 'Default' to 'Base' (#22397)
Ψ - Latest upstream/develop: None
Ψ - Common ancestor with upstream/master: 2023-11-09 04:03:07 +0200 (daabe2d8c5) -- Update layer state of Lily58 from 'Default' to 'Base' (#22397)
Ψ - Common ancestor with upstream/develop: None
Ψ All dependencies are installed.
Ψ Found arm-none-eabi-gcc version 12.2.1
Ψ Found avr-gcc version 8.5.0
Ψ Found avrdude version 7.2
Ψ Found dfu-programmer version 0.7.2
Ψ Found dfu-util version 0.11
Ψ Submodules are up to date.
Ψ Submodule status:
Ψ - lib/chibios: 2023-04-15 13:48:04 +0000 --  (11edb1610)
Ψ - lib/chibios-contrib: 2023-07-17 11:39:05 +0200 --  (da78eb37)
Ψ - lib/googletest: 2021-06-11 06:37:43 -0700 --  (e2239ee6)
Ψ - lib/lufa: 2022-08-26 12:09:55 +1000 --  (549b97320)
Ψ - lib/vusb: 2022-06-13 09:18:17 +1000 --  (819dbc1)
Ψ - lib/printf: 2022-06-29 23:59:58 +0300 --  (c2e3b4e)
Ψ - lib/pico-sdk: 2023-02-12 20:19:37 +0100 --  (a3398d8)
Ψ - lib/lvgl: 2022-04-11 04:44:53 -0600 --  (e19410f8)
Ψ QMK is ready to go, but minor problems were found

Is AutoHotKey / Karabiner installed

• AutoHotKey (Windows)
• Karabiner (macOS)

Other keyboard-related software installed

No response

Additional Context

No response