# Threads

Guest threads, their synchronization and how the host schedules them. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom.

## Thread creation and cores

### Core mask changes move the ideal core into the new mask

- **Symptom:** per-core job queues never drained.
- **Cause:** a `SetThreadCoreMask` that pins a thread to other cores (ideal core −3 = keep) left the ideal core outside the new mask, so `GetCurrentProcessorNumber` reported a core the thread cannot run on.
- **Fix:** the host moves the ideal core into the new mask, as Horizon does, so `GetCurrentProcessorNumber` reports a core the thread can run on.

### Ideal core −2 resolves to the process default core

- **Symptom:** crash indexing a per-core table with a huge offset (index −2).
- **Cause:** threads created with ideal core −2 (*use process default*) were recorded as core −2, and `GetCurrentProcessorNumber` returned −2.
- **Fix:** −2 resolves to the process default core (0), and a negative core is never returned (`hle_thread.c`).
- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md), in its font engine (`main+0x107dbc0`).

### Free thread count (`svcGetInfo` type 24)

- **Symptom:** the main thread waits forever on an event right after `nnMain` starts, with only three or four threads created; `DM_THR=1` shows no `CreateThread` for the thread it waits on.
- **Cause:** Unity's thread start (`main+0x3e1a50`) creates a thread only when `nn::os::GetFreeThreadCount()` is at least 1. That is `svcGetInfo` type 24 (FreeThreadCount, Horizon 11 and later), which the host answered 0 like every type it did not know, so the thread was never asked for. The kernel answers the thread resource limit less the threads alive (Atmosphère `kern_svc_info.cpp`); `pm` gives an application 96 threads (`pm_spec.cpp`, 256 with a boost).
- **Fix:** the host answers 96 less the game's threads that have not exited (`thr_limit_used`; guest threads the host spawns for itself are left out).
- **Found on:** [Monster Train 2](../../dm/games/010051701FB46000/README.md) (Unity 2022.3.54f1 on SDK 18.3.5: `main+0x3e7288` starts a thread and waits for it to signal `main+0x539cf80`). With the fix the game creates 40 threads and reaches its title.
- **Note:** of the 20 games extracted on this machine, only this `main` imports the function.

### Pausing threads and reading their context for garbage collectors

- **Symptom:** in garbage-collected titles (Unity / IL2CPP), live objects freed, with their finalizers releasing native resources.
- **Cause:** the Boehm collector pauses each thread, reads its registers for roots and scans its stack. `SetThreadActivity` (0x32) and `GetThreadContext3` (0x33) were placeholders that only reported success: threads kept running and the context was empty, so roots held only in registers were missed.
- **Fix:** both really pause and report registers (`hle_thread.c`). A thread inside an SVC counts as paused at once, with the gate frame as its context (`thr_svc_enter` / `thr_svc_leave` around every gate SVC; it waits at the SVC exit while paused). A thread in guest code gets SIGUSR2, and the handler saves the ucontext and waits there. A thread in host code is retried every 100 µs. The context is Horizon's 0x320-byte `ThreadContext`.
- **Found on:** Unity / IL2CPP titles in general.

### Same-core wake handoffs

- **Symptom:** crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`.
- **Cause:** Horizon schedules the highest-priority runnable thread on a core ([Mesosphere's scheduler](https://github.com/Atmosphere-NX/Atmosphere/blob/master/libraries/libmesosphere/source/kern_k_scheduler.cpp)). A lower-priority thread that wakes a higher-priority thread pinned to its core cannot continue until that work blocks. The host's pthreads run in parallel. The earlier approximation also confused host dispatch state with guest readiness: `wake()` cleared the guest wait before the pthread resumed, but `thread_info` could still report that pthread waiting. The caller then continued too soon. Its 5 ms timeout opened the same hole under a longer host delay; the recorded `yield_pending` flag was unused unless the profile enabled checks after every thread SVC.
- **Fix:** every game's thread SVC now honours `yield_pending`. Before returning, it waits for the higher-priority peers pinned to the caller's core to block, sleep, exit, be suspended, or enter explicit host GPU work (`svc_thread` / `dm_yield_to_core_peers` in `hle_thread.c`). Guest runnable state is authoritative, including the wake-to-pthread-resume interval; a host dispatch snapshot cannot release the handoff, and there is no timeout that lets unfinished work through. Host-created helpers and threads that can run on another core are excluded.
- **Check:** `make -C dm/host/tests GAME=010093801237C000 preempt` delays a woken pthread's dispatch by 1 ms and 20 ms through real event, mutex and condition-variable SVCs. The worker's payload and exit must precede the signaller's return. The original implementation failed both delays; changing readiness alone fixed 1 ms but still failed 20 ms. Equal-priority, other-core, migratable, host-helper, suspended and host-GPU-work peers do not block the caller.
- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md), whose per-build movie request-ring gates and `core_preempt on` setting are now removed.
- **Scope:** these are cooperative thread-SVC handoffs, not a complete asynchronous guest scheduler. The optional `core_preempt` profile key (`DM_PREEMPT=0` / `1` overrides it) additionally checks already-runnable peers after every thread SVC; it defaults off. Historically that broader policy moved [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s first frame from 0.6 s to 2.9 s by waiting for busy decompression threads. Wake-triggered handoffs apply with either setting.

### Simultaneous crashes of several threads

- **Symptom:** several threads crash at once and the dumps interleave, then the process hangs instead of exiting.
- **Cause:** Unity's job threads tend to crash together.
- **Fix:** `crash.c` lets only the first crashing thread print (`crash_gate`); the others stop in `pause()`, and a 15 s alarm ends the process if the dump itself stalls.
- **Found on:** Unity titles in general.

## Synchronization and address arbitration

### `SignalToAddress` type 2 with count ≤ 0

- **Symptom:** none seen; found while investigating [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md)'s demo crash, which it did not change (that was the address-space shortage, [Size-aligned regions exhaust the alias address space](memory.md#size-aligned-regions-exhaust-the-alias-address-space)).
- **Cause:** for `SignalToAddress` type 2 (`SignalAndModifyByWaitingCountIfEqual`) with count ≤ 0 and waiters present, the new value is `value − 2`, not `value − 1` (mesosphere `KAddressArbiter`; Ryujinx and yuzu agree). The host had −1.
- **Fix:** the host writes `value − 2`, the specified behaviour.

### `WaitForAddress` type 3: 64-bit wait if equal

- **Symptom:** a 64-bit wait could return at once or block when it should not (not met in a game).
- **Cause:** Horizon 18 added a 64-bit "wait if equal" arbitration type, which SDK 19 exports as `nn::os::WaitForAddress(const long*, long)`; the host used to fall into its 32-bit "wait if less than" test.
- **Fix:** type 3 compares the 64-bit value (`hle_thread.c`).
- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (SDK 19.3.11): Z-A does not call it, but the SDK has it.

### Late PhysX task destructor after the unlock

- **Symptom:** a PhysX worker crashes in a task's `removeReference` with a null task manager, right as the game switches to 30 fps (`pc=main+0x10ccf28`, `lr=main+0x13c236c`); on the iPhone and in 2 of 5 Mac runs, in every Mac run with 12 busy loops loading the CPU.
- **Cause:** a destructor that runs a moment too late. The game rebuilds each physics world's completion task in place before every `simulate` (`main+0x15730`, main thread). A worker runs the task (`fetchResults`, sets the world's done flag, notifies, unlocks) and then calls its `release`, which calls the task's own destructor (vtable back to the `PxLightCpuTask` base, `mTm = 0`). At 30 fps the game steps twice per frame, so the unlock wakes a main thread that steps again at once. On the Switch the `release` follows the unlock within nanoseconds; on Apple devices the worker can be descheduled between them, the main thread rebuilds the task, and the late destructor wipes the new one. The next simulation's `NpSceneCompletion::release` then dereferences `mTm`.
- **Fix:** a per-build cut makes that one `release` a `ret` (its `patches.py`); the destructor frees nothing and the step rewrites every field.
- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md). How it was found is in [05](../agents/05-debugging.md#racy-crashes-load-trace-gates).

## Yields and spin waits

The yield back-off (`DM_YIELD_BACKOFF`) and the rewriter's spin waits (`patch_nso.py` "Spin waits", `dm_spin_wait` in `hle_thread.c`), with their measurements, are in [development.md](../development.md#power) ("Yield loops", "Spin loops").

### Timer slack in guest sleeps (`sleep_precise`)

- **Symptom:** a 30 Hz game thread runs at 27 Hz, and whatever it paces comes out slow (cutscenes at 36 ms per frame instead of 33.3; gameplay was fine at 60).
- **Cause:** macOS timer slack: `nanosleep` and `mach_wait_until` overshoot by about 10% at any QoS (measured: 33 ms → 36.4 ms, 5 ms → 6.1 ms, 1 ms → 1.26 ms). Dread's sound thread (`main+0xf95b0`) reads a millisecond clock, updates, then sleeps "33 ms minus this round's work", so on the host its period was 36.9 ms. The cutscene loop renders a frame whenever the audio play position, which that thread refreshes, moves (`main+0x119a3d0`; the position is played samples ÷ sample rate of the cutscene's stream), so cutscenes ran at about 27 fps.
- **Fix:** `SleepThread` sleeps three quarters of the remaining time per step until within 20 µs (`sleep_precise` in `hle_thread.c`).
- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md): cutscenes at 33.3 ms, the sound thread at 33.2 ms. [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s frame-time p95 also went from 19.7 to 18.4 ms.
- **Note:** timed waits (`WaitSynchronization`, condition variables) still use the host's own timeouts.

### Sleeps under 20 µs take the yield path

- **Symptom:** a guest thread keeps a core at 100% with billions of `SleepThread` calls.
- **Cause:** `sleep_precise` returns at once for anything under 20 µs (its nanosleep would overshoot), so a loop of such sleeps never sleeps.
- **Fix:** sleeps under 20 µs take the yield path and its back-off (plain yields for the first 2 ms of a run, then short sleeps).
- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md): Hagi polls with `SleepThread(100 ns)` (`main+0x517c`), 1.4 billion calls in 200 s, and the render thread nearly as busy. The two threads went from 199 s and 171 s of CPU per 200 s to 5 s and 19 s per 120 s, at the same frame rate.

### Yields that make progress return at once (`thr_yield`)

- **Symptom:** a loading screen sits for a minute with the CPU nearly idle; one thread yields thousands of times a second, every yield a back-off sleep (the power line's top yielder, with as many back-off sleeps).
- **Cause:** the thread was working, not waiting: its loop calls `nn::os::YieldThread` once per item, and the back-off made each yield a 250 µs sleep.
- **Fix:** a yield whose caller's x19–x28 changed since the last one is progress (`thr_yield` in `hle_thread.c`; the SDK reaches the SVC through tail branches, so those registers are the loop's own): it returns at once with no host call, and the run starts again. Polls keep those registers (Three Houses' limiter, Galaxy 2's Hagi sleep poll at `main+0x11ec`); counters move them.
- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md): a black screen with the plane icon for 77 s after the opening movie, `SaveProc` at about 2,600 yields a second, its grid walks over the new island (`main+0x157d020`) yielding once per cell; also about 40 s of the same at boot. The island now shows about 10 s after the movie, as with `DM_YIELD_BACKOFF=0`. Polls that keep their registers: [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md), [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md).
- **Check:** check such a change on [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) outdoors: its `PlacementMgr` and `RadarMgr` turned out to yield per item too, and a `sched_yield` on each of their 200,000 yields a second cost a third of a core more than the back-off had; with no host call, CPU energy fell instead (numbers in [development.md](../development.md) "Yield loops").
- **Note:** a temporary log of which of x19–x28 move at each yield, per thread, is what told the two kinds apart.

## Events and notifications

### One auto-clear event shared by a pool's workers

- **Symptom:** the main thread waits forever on a job's done event at boot while every worker sleeps, two of them on one event (`main+0x1426c` waiting on `Job::m_doneEvent`); 4 of 6 Mac runs.
- **Cause:** the engine's job pools wake one worker through the pool's single auto-clear event. A job goes into worker k's own queue (`main+0x154c0`), and if k's state reads asleep the caller signals the event (`main+0x12554`); the kernel wakes the waiter that slept first, which is k only while the workers sleep in index order. A worker woken for another's job runs only its own queue and sleeps again, so with two workers in one pool the signal is lost: at boot when worker 1 went to sleep before worker 0, and later as a job left in worker 0's queue for 12 s. The thread dump's `wait #` (the order condition waiters arrived in, which `SignalProcessWideKey` wakes them in at equal priority) showed the order in every hung run.
- **Fix:** the game's own wait wrapper (`main+0x12500`, `{event, pending}`) signals the event once more when it takes a nonzero pending word after waking, and nothing set it: a build cut makes the signal set it first (its `patches.py`), so a worker woken in another's place passes the wake on; 6 of 6 runs boot.
- **Found on:** [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md).
- **Note:** a host change that made `StartThread` return only once the new thread runs (as a free Switch core takes it within microseconds) brought the hangs to 1 in 6 and was dropped.

### The oe notification event and its queue

- **Symptom:** abort in `os::WaitSystemEvent` under `oe::PopNotificationMessage` (The Legend of Zelda: Link's Awakening); one guest thread at about 1000 ms/s on the iPhone only, the phone hot, CPU energy 5–7 W (Monster Hunter Rise's `MessageSuperviseThread`, priority 32, in the `[nvn] threads` lines).
- **Cause:** the host keeps the notifications in a queue of its own. The blocking pop waits on the system event `oe::Initialize` would have created, and the host's `Initialize` is a no-op. Monster Hunter Rise's thread waits on `oe::GetNotificationMessageEvent()` with `os::WaitAny`, which does not clear an event, then pops notifications until the queue is empty. The host only ever signalled its notification event, so after the first focus change (iOS changes focus at launch; the Mac does not) it stayed set and the loop never blocked again.
- **Fix:** the host now implements the blocking pop on its notification queue (a condition variable, then the `TryPop` path), and the event follows the queue as the am service's does: set while a notification is queued, cleared when the last one is popped (`hle_oe.c` `mq_ev_sync`, `hle_clear_system_event`).
- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0, the blocking pop); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (the event).
- **Check:** Mac repro `DM_OE_SCRIPT=focus=2@5,focus=1@6` (the script also starts at the game's first `GetNotificationMessageEvent`, since such a game pops only once the event fires): before 995 ms/s, CPU 109%, 6.97 W; after, the thread is idle and CPU energy is 0.05 W.