SMP phase 3a: queue locks, remote wake IPI, migration and the policy-swap rendezvous

Start in Core/Scheduler/SchedulerManager.cs and PerCpuState.cs to see how _lock, s_globalLock and the policy hooks are used today, then read SpinLock.cs for the IRQ-safe lock. The work spans interrupt controllers and the rendezvous in Core/CPU/CpuStop.cs, so it needs kernel-level familiarity before any edit. Done means the listed tests (Mutex_CrossCpuHandoff, InterruptEvent_WakesThreadOnOtherCpu, SetScheduler_SwapsWhileOtherCpusRun) pass under smp4 on both architectures. A linked pull request (#3371) is already open, so coordinate with it first.

Part of #3336. Proposal: section 3.3.

Goal

The scheduler mechanism is correct with several CPUs touching it at once. It keeps its shape and gains locks, a remote wake and a migration primitive. Placement still puts everything on CPU 0 in this phase (that changes in phase 5), so behavior does not move yet.

Work

• PerCpuState._lock, which today guards only the dead Schedule path and SetPriority, becomes the queue lock: an IRQ-safe spinlock around every policy hook that touches that CPU's run structure, around CurrentThread and around _needReschedule. The owning CPU takes it from its tick and its interrupt exits; a remote CPU takes it to wake a thread onto this CPU or to pull one off it.
• A registry lock around s_allThreads, the counts and the thread id; s_globalLock (SchedulerManager.cs:24), which today guards only SetScheduler, becomes it. RegisterThread runs under it (valentinbreiz/nativeaot-patcher#444 is worth fixing in the same pass). The interrupt masks in RegisterThread, CreateThread, ReadyThread, BlockThread, MarkSleeping, ExitThread and GetBusyCpuTimeNs stop being the only exclusion.
• Lock order: queue locks in ascending CPU index, then the registry lock, then the heap lock; two queue locks are held at once only by a migration. The rule that nothing allocates while holding a spinlock is written into scheduler.md; phase 4 explains why it is what keeps a stop-the-world collection from deadlocking.
• State, _needReschedule, the online mask and every other field read on one CPU and written on another go through Volatile, and every handshake is an acquire-release pair. Free on x64; on ARM64 the difference between a kernel that works and one that hangs rarely.
• ReadyThread asks the policy where the thread goes (SelectCpu, under affinity once phase 3b lands), takes that CPU's queue lock, inserts, sets its _needReschedule, and if that CPU is not the caller's sends it the reschedule interrupt: x64 vector 0xF0 by fixed delivery to its APIC id (the ICR high and low writes under an interrupt mask after waiting for the delivery-status bit, since a handler on the same CPU may be sending too; LocalApic gains a targeted send beside SendSelfIpi), ARM64 SGI 8 with the target's affinity in ICC_SGI1R_EL1. This is what Mutex.cs:267, ConditionVariable.cs:172 and InterruptEvent.cs:359 have been missing when they ready a thread on another CPU. The park protocol does not change.
• Migration primitive: both queue locks, remove from one structure, rewrite CpuId, OnThreadMigrate so the policy rebases its virtual time, insert into the other. A running thread whose affinity no longer allows its CPU is marked for reschedule and its CPU is sent the interrupt; the switch-out re-queue moves it.
• SetScheduler (SchedulerManager.cs:137) rebuilds every CPU's structure and allocates while doing it (InitializeCpu, OnThreadCreate), under s_globalLock, which the rule above forbids under any lock a tick can spin on. It rendezvous instead, and the stopped world plus the caller's own interrupt mask is the whole exclusion: no spinlock is held across the swap, so the swap may allocate. The stop lives in Core/CPU/CpuStop.cs (new, shared with phase 4): an owner word claimed by compare-exchange, the stop interrupt (x64 vector 0xF1, ARM64 SGI 9), and each CPU's acknowledgement in the per-CPU block slot phase 1 names StoppedForGc, renamed Stopped since it serves both. The sequence: claim the owner word, spinning with interrupts enabled while another CPU owns the stop so that CPU can stop this one too; mask interrupts; send the stop interrupt to every other online CPU; wait for every acknowledgement; swap; set every CPU's _needReschedule; release. Each stopped CPU reschedules on the stop handler's own exit, which is the reschedule on the next interrupt exit the swap already relies on. The swap accepts only the handler's acknowledgement, never the spin-loop safepoint phase 4 adds to SpinLock: a CPU stopped in the handler had interrupts enabled, so it holds no IRQ-safe lock, while a CPU acknowledging from a spin loop may hold a queue lock halfway through a migration, and the swap would rebuild a structure that CPU is still writing. Nothing spins on the swapper, which holds no lock, so every spinner finishes its section, unmasks and reaches the handler. A collection the swap's allocations trigger finds the stop already owned by its CPU, stops nothing again, and scans the stopped CPUs from the frames the handler saved. Phase 4 extends the handler with the stack capture and adds the safepoint acknowledgement for the collector's own stop.
• One locking rule for the whole seam: a hook runs under its CPU's queue lock and never takes it. The manager takes the lock around every hook, the four that mask interrupts themselves today included (SetPriority, GetPriority and the two run-queue diagnostics), so Stride and the Threading suite's RoundRobin drop their MaskInterrupts (Stride's RemoveThreadFromQueue too). The lock stays a plain IRQ-safe spinlock, not a reentrant one: SpinLock is not reentrant, and a hook taking it itself would deadlock the manager's own SetPriority, which already holds _lock around the hook, and every caller that holds the lock across a count-then-index walk, which the contract tells it to do. GetPriority receives no PerCpuState, so SchedulerManager.GetPriority locks the CPU the thread's CpuId names and re-reads CpuId under the lock, since a migration can move the thread between the two. SchedulerManager.LockCpu(state) becomes public, a scope that masks and locks, for the callers that walk a run queue or reach a hook without the manager: SchedulerDiagnostics (GetRunQueueCount, TryGetRunQueueThread, the RequestKill walk), the Threading suite's RunQueueHolds, and any entry a policy adds of its own (a tuning setter, a statistics read), which locks because nobody else does. MaskInterrupts stays for what only needs this CPU's tick kept out. SelectCpu reads other CPUs' loads without their locks; a stale read is the heuristic's to tolerate. IScheduler, scheduler.md, spinlocks-and-masking.md and the plugging guide's kernel constraints are rewritten to the rule. Stride's s_onTickLogCount moves into StrideCpuData. Any structure a policy adds must be pre-sized like StrideCpuData.RunQueue already is.

Proves itself with

Mutex_CrossCpuHandoff and InterruptEvent_WakesThreadOnOtherCpu, with the thread placed on another CPU through the phase 3b API or, if 3a lands first, a test-only placement hook; SetScheduler_SwapsWhileOtherCpusRun (a spinner on every other CPU across a swap in each direction, every counter still advancing afterwards); every suite green under smp4 on both architectures.

Files

Core/Scheduler/SchedulerManager.cs, PerCpuState.cs, SchedulerThread.cs, SpinLock.cs, Mutex.cs, ConditionVariable.cs, InterruptEvent.cs, Stride/StrideScheduler.cs, Stride/StrideCpuData.cs, Core/CPU/CpuStop.cs, Core.X64/Cpu/LocalApic.cs, Core.X64/Cpu/X64InterruptController.cs, Core.ARM64/Cpu/ARM64InterruptController.cs, Core.ARM64/Cpu/GICv3.cs, Core.ARM64/Cpu/GICv2.cs, the native IRQ stubs for the stop vector, System/Diagnostics/SchedulerDiagnostics.cs, tests/Kernels/Cosmos.Kernel.Tests.Threading/Kernel.cs, tests/Kernels/Cosmos.Kernel.Tests.Threading/RoundRobinScheduler.cs, docs/articles/dev/scheduler.md, docs/articles/dev/sched-concepts/spinlocks-and-masking.md, docs/articles/user/scheduler-plugging.md.