[Bug] Ploopy Classic scrolls erratically and erroneously during mouse movement after reconnect

Describe the Bug

I'm using the Ploopy Classic trackball (running QMK) with a USB switch plugged into two computers. When I press a button on the switch to change to the other computer, moving the mouse will randomly start scrolling a little bit. It's noticeable due to things like seeking in mpv getting activated when I just try to move the mouse. It took me a while to realize what was going on. I thought maybe I was bumping the scroll wheel, but that's not it. Usually what I'll do is find somewhere I can scrolll and then move the mouse wheel a bit, then go into drag scroll and scroll up and down, then turn off drag scroll again and it doesn't have issues beyond that point. Not sure how much of that process is necessary. I think of it a bit like an analog stick being stuck going in a direction because it was being pushed that way when the controller was plugged in, or like a shift key getting stuck until you press it again.

System Information

Keyboard: Ploopy Classic Revision (if applicable): Ordered quite recently (this year), going to say it's the latest revision of the mouse probably. Operating system: Guix System GNU/Linux and Windows 10 Enterprise (happens on both) qmk doctor output: Not sure if this is relevant, and I usually compile and flash on different machines + use dfu-programmer for the flashing... if it matters a lot I can get it from whichever machine(s) later

(Paste output here)

Any keyboard related software installed?

• AutoHotKey (Windows) (yes on the Windows machine, no on the GNU/Linux machine, so likely irrelevant)
• Karabiner (macOS)
• Other:

Additional Context

USB Switch used: https://www.amazon.com/dp/B01N6GD9JO/

I had the switch for a little while before I had the Ploopy and I did not notice any similar issues with my Logitech G400.

I can get my QMK files off the other machine I do the build on if needed. I basically combined the VIA profile (although I have never used VIA and don't plan to) and drag_scroll profile and then lowered the DPI presets a lot. Everything works great besides this weird scrolling issue.