Same-core handoffs wait only for the threads their wake made runnable (#247, #259 section 6)

A wake's handoff waited until every runnable higher-priority thread pinned to the caller's core was blocked, not only the ones the wake made runnable. The host runs a lower-priority thread beside such threads at all other times, and a core's busy loaders, workers and pollers are seldom all blocked at once, so the wait had no bound. Tears of the Kingdom 1.4.3 (issue #247) stayed on an empty scene after the boot logos: on 82ab25b, 84 functions / 771 NVN calls a frame, no packaged texture uploaded, 250,008 ms of handoff waits in a 180 s run; a temporary log showed BootupBackgroundWorkThread (priority 58) waiting 200-290 ms at a time for ARes Loading 1, Presentation Thread and BaseProcExecutor. Donkey Kong Country: Tropical Freeze's priority-58 job thread waited the same way for pollers (issue #259, section 6).

A wake now records the woken thread in the waker's current handoff and in those the waker belongs to (note_runnable, at most four), so Metroid Dread's main thread still waits for its movie dispatcher and for the worker the dispatcher wakes. Blocking, a sleep, YieldToAnyThread (SleepThread -2, which on Horizon takes the thread off its core so that lower-priority threads run there), exit, suspension and host GPU work end a thread's hold; a handoff ends when its caller's thread SVC returns (hand_epoch). There is still no timeout, and core_preempt keeps checking every runnable peer.

Checked on the Mac, without shader libraries for chain 8393d85a4eae (draws skipped): TotK on the issue #70 route for 180 s, 1,221 ms of handoff waits and the same per-frame NVN call counts and texture first uploads as 9feac2f (before the unbounded handoff). Dread, fresh saves, DM_KEYS=4:A*20/0.7, 60 s: a control host with the handoff off crashed in 1 of 3 runs at sdk+0x347eb4 from main+0x225338; the limited handoff passed 12 of 12 runs, each through three decoders and two teardowns with about 12,400 handoffs. t_preempt adds three handoff-scope cases: a busy peer nobody woke (main's code never returns), a worker woken by the woken dispatcher (fails without the recorded chain), a woken thread yielding to any thread (never returns without that rule). Host tests, docs tests and an iOS syntax check of hle_thread.c pass. Gameplay with shaders, iOS and DKC itself are not checked here.

Co-authored-by: HougeLangley <1161594+HougeLangley@users.noreply.github.com>
Co-authored-by: aurlight <120410928+aurlight@users.noreply.github.com>

dm/games/010093801237C000/README.md

@@ -93,6 +93,7 @@ General host changes first needed here; each applies to every game:
 - **Synchronization review gameplay validation:** [Final gameplay validation](#final-gameplay-validation-2026-10-26-current) records the #158/#213 comparison and its limits; the general fixes and synthetic regressions are linked under the earlier smoke routes.
 - **Patch areas before the module base** for modules with a giant bss (negative `PATCH_OFF`, `FILE_IMAGE_LEN`, `IMAGE_SIZE`): [Patch area before the module base for a large bss](../../../docs/reference/rewriter.md#patch-area-before-the-module-base-for-a-large-bss).
 - **Used memory counts the bss of front-patch modules** (`big_bss_bytes`): [Heap sized from the application pool less used memory](../../../docs/reference/memory.md#heap-sized-from-the-application-pool-less-used-memory).
+- **Recheck after handoffs were limited to the threads a wake makes runnable (issue #247, 2026-10-26, Mac):** fresh saves, `DM_KEYS=4:A*20/0.7`, 60 s each, no shader libraries for chain `8393d85a4eae` (draws skipped; the movie threads run as usual). A control host with the handoff turned off crashed in 1 of 3 runs at `sdk+0x347eb4` called from `main+0x225338`, the lifetime crash above. The limited handoff passed 12 of 12 runs, each with three decoders and two teardowns, about 12,400 handoffs and 1.2–1.6 s of handoff waits. Runs are in the NSP work folder's `bug06/`.
 - **`sleep_precise`** against macOS timer slack, **same-core wake handoffs** (`svc_thread` / `dm_yield_to_core_peers`): [Timer slack in guest sleeps (`sleep_precise`)](../../../docs/reference/threads.md#timer-slack-in-guest-sleeps-sleep_precise), [Same-core wake handoffs](../../../docs/reference/threads.md#same-core-wake-handoffs).
 - **Native-texture movies** (`nvn_host_write_texture`, `nvn_host_signal_sync`), the `D1` destructor, `md_event`, the SDK 11 names, `FinishStartupLogo`, `SendVibrationValues` and lexical `..` (`norm_rest`): [Decoding into the client's NVN texture](../../../docs/reference/movies.md#decoding-into-the-clients-nvn-texture), [The complete-object destructor](../../../docs/reference/movies.md#the-complete-object-destructor), [Client events read through the registered pointer](../../../docs/reference/movies.md#client-events-read-through-the-registered-pointer), [Older SDK names for the same calls](../../../docs/reference/services.md#older-sdk-names-for-the-same-calls), [The startup logo](../../../docs/reference/services.md#the-startup-logo), [Vibration values flushed by the SDK's vibration graph](../../../docs/reference/services.md#vibration-values-flushed-by-the-sdks-vibration-graph), [Paths with `.` and `..` components](../../../docs/reference/services.md#paths-with--and--components).
 - **Per-stage program binding** (`bind_stages`, composite programs, `Z` lines in pass dumps), **in-place `WriteTexels`** (`write_region_bl`) and the **MSHD reader**: [Programs bound per stage](../../../docs/reference/rendering.md#programs-bound-per-stage), [`WriteTexels` regions written in place by block-linear address](../../../docs/reference/nvn-objects.md#writetexels-regions-written-in-place-by-block-linear-address) and [MSHD in MercurySteam's bshdat files](../../../docs/reference/shaders.md#mshd-in-mercurysteams-bshdat-files).

dm/games/0100F2C0115B6000/README.md

@@ -84,6 +84,7 @@ General host changes first needed here; each applies to every game:
 - **Conditional rendering** evaluated at replay (`OP_RENDER_COND`, profile key `render_cond`) and **CPU-written textures with no flush** (`tex_dirty_upload`, `ft_add`): [Conditional rendering evaluated at replay](../../../docs/reference/rendering.md#conditional-rendering-evaluated-at-replay), [CPU writes with no flush, caught by page protection](../../../docs/reference/nvn-objects.md#cpu-writes-with-no-flush-caught-by-page-protection-tex_dirty_upload).
 - **Mid-pass, partial-mask and partial-scissor clears** drawn as a triangle (`clear_color_tri` / `clear_ds_tri`), **8-bit indices widened on the GPU** when written in the same submission (`wide_run`), **`ReadTexels` from guest memory on any thread** with `nvn_ft_poll` marking textures written after first use, **attachment-less passes** (`bare_dims`): same section.
 - **Rewriter:** out-of-line atomic retry blocks (`retry_block` / `site_retry_window`, 8 sites) and jump tables relative to an `adr` (`jump_adr_offsets`, `main+0x393bb0`): [Out-of-line retry blocks of atomic updates](../../../docs/reference/rewriter.md#out-of-line-retry-blocks-of-atomic-updates), [Offset tables relative to an adr](../../../docs/reference/rewriter.md#offset-tables-relative-to-an-adr).
+- **Resolved: black screen at boot since `9a56ecf` (issue #247), handoffs that waited for the whole core:** [Same-core wake handoffs](../../../docs/reference/threads.md#same-core-wake-handoffs). Each wake's handoff waited for every runnable higher-priority thread pinned to the caller's core. `BootupBackgroundWorkThread` (priority 58) then waited 200–290 ms at a time for `ARes Loading 1`, `Presentation Thread` and `BaseProcExecutor`, threads its wake had not made runnable. Checked on the Mac with update 1.4.3 and no shader libraries for chain `8393d85a4eae` (draws skipped), the issue #70 route for 180 s. On `82ab25b` the scene stayed empty (`84 functions 771 calls` a frame, no texture uploaded), and handoffs waited 250,008 ms in total. With handoffs limited to the threads the wake made runnable, they waited 1,221 ms, and the run matched `9feac2f` (before the change) frame by frame: 11 first uploads in both, 14,500–20,500 calls a frame. Gameplay with shaders has not been rechecked.
 - **Slotted command lists** (`OP_RESERVE`, `t_nvn` "slotted lists"): [Prerecorded lists in slots of their measured size](../../../docs/reference/nvn-objects.md#prerecorded-lists-in-slots-of-their-measured-size).
 
 ## Open

dm/games/README.md

@@ -132,7 +132,7 @@ One row per profile; the notes are each profile's `README.md`.
 | `pso_mtl4` | `on` / `off` (the default): on iOS, render pipelines through MTL4Compiler by default, with the pipeline archive, the prepare step, the hybrid pipelines and the prebuild (on by default on the Mac; `DM_PSO_MTL4=1` / `0` overrides). A setting the game was tested with on an iPhone. It also gives the app the [pipeline archive made offline](../../docs/development.md#the-pipeline-archive-made-offline) (`build_ipa.sh` builds it for `g18p` unless `DM_PSO_GPUS` says otherwise; without the key, none) and the pulling libraries (`_bp`). Breath of the Wild sets it on |
 | `fps_target` | Frame rate cap of the rewriter's frame step, for builds that have one (`FRAME_STEP` in the profile's `patches.py`: Tears of the Kingdom 1.4.3, Breath of the Wild 1.9.0, Hyrule Warriors: Age of Calamity 1.3.0, Paper Mario: The Thousand-Year Door 1.0.1, ASTRAL CHAIN 1.0.1), or the engine's own frame limit the host writes into game memory (`hle_fpslock.c`: Monster Hunter Rise 16.0.2, which also needs `present_interval display`; Octopath Traveler 0 1.0.5, Unreal's `t.MaxFPS` and `rhi.SyncInterval`, so the game asks for present interval 1 itself; Luigi's Mansion 3 1.4.0, its frame limiter's milliseconds per frame, with `present_interval display`). `0` (the default): the game's own step. `30` / `60`: the host decides each frame's step in whole vsyncs and keeps the frame on screen that long (`hle_nvn.c` `fstep_tick`, `nvn_fstep.h`): 1 at a 60 cap, 2 when 60 does not hold or at a 30 cap (and while a prerendered movie plays), plus one vsync while the game is behind real time after a late frame. Presents follow that hold instead of the interval the game sets. `DM_FPS_TARGET` overrides it, also from `Documents/env.txt`; the log line `frame step: game time … of real time` checks the game's speed |
 | `fps_hold` | `on` (the default): with `fps_target`, three late frames in a row raise the hold above the cap's vsyncs (30 or 20 fps), and steady frames that fit try the lower hold again. `off`: frames are never held longer than the cap's vsyncs; each is shown as soon as it is ready. For a game whose updates do not wait for the presents and step by the cap whatever the hold (Age of Calamity): there the hold only paces the screen. `DM_FPS_HOLD=on` / `off` overrides it, also from `Documents/env.txt` Paper Mario: The Thousand-Year Door sets it off too: its frame waits for three render worker threads, and holding frames left them idle, so a frame cost more the longer the hold (the hold stuck at 3 vsyncs, 20 fps, on a loaded Mac; 32–58 fps without holds) |
-| `core_preempt` | `on` / `off` (the default): additionally check higher-priority runnable threads pinned to the caller's core after every thread SVC (`hle_thread.c`). `DM_PREEMPT=0` / `1` overrides it. Wake-triggered same-core handoffs apply to every game with either setting and wait until the peer blocks, with no wall-clock cap. Metroid Dread no longer needs this key or its movie request-ring gates; see [Same-core wake handoffs](../../docs/reference/threads.md#same-core-wake-handoffs) |
+| `core_preempt` | `on` / `off` (the default): additionally check higher-priority runnable threads pinned to the caller's core after every thread SVC (`hle_thread.c`). `DM_PREEMPT=0` / `1` overrides it. Wake-triggered same-core handoffs apply to every game with either setting and wait until the threads the wake made runnable block, with no wall-clock cap. Metroid Dread no longer needs this key or its movie request-ring gates; see [Same-core wake handoffs](../../docs/reference/threads.md#same-core-wake-handoffs) |
 | `alloc_big_align` | `on` / `off` (the default): `nn::mem::StandardAllocator::Allocate` aligns blocks of 64 KB or more to 64 KB (`hle_os.c`). Vanillaware's engine (13 Sentinels: Aegis Rim, Unicorn Overlord) takes a memory pool's storage from malloc and cuts ring heaps out of it at 64 KB alignment relative to the pool's GPU address. On the Switch that address is big-page aligned; the host's is the CPU address, only 4 KB aligned, so the last ring heap does not fit and keeps base 0, and the game writes through a null pointer (13 Sentinels entering its first battle, Unicorn Overlord past the title screen) |
 | `shared_font` | Optional, decided by the game files without it (the SDK version the host reads from the `sdk` module, `loader.c` `dm_sdk_major`, and `main`'s imports, `dm_main_imports`): what the shared-font address points at. `bfttf` (SDK 11 and later, when `main` imports `ScalableFontEngineHelper::Decode`) = the 8-byte BFTTF header (with the "already decoded" marker) followed by the TTF, which Mario passes to `Decode`; `ttf` = the TTF itself, which SDK 10 and older games hand straight to `ScalableFontEngine::LoadFont` (Super Mario 3D World + Bowser's Fury: with the header the engine fails with error 403 and the game crashes looking up a glyph; Odyssey, Link's Awakening, Age of Calamity and Three Houses alike) and Unity's player to FreeType (Patrick's Parabox: with the header its Japanese, Korean and Chinese text drew nothing). No profile sets it |
 | `device_api` | Optional, decided by the SDK version without it: the NVN API version `major.minor` that `nvnDeviceGetInteger` reports, 55.15 for SDK 11 and later (what Mario requires) and 53.313 for SDK 10 and older (`hle_nvn.c` `nvn_api`). Games check it and skip creating their device on a mismatch: Super Mario 3D World + Bowser's Fury (SDK 10) requires major 53 and minor ≥ 313. No profile sets it. `DM_NVN_API` overrides. Not part of the shader chain version (only `shader_*` keys are) |

dm/host/hle_thread.c

@@ -57,6 +57,8 @@ typedef struct thr {
     int wkind, woke; uint64_t waddr, wkey; uint32_t wtag; int64_t wres;
     _Atomic int sleeping;      // inside SleepThread's timed sleep (not runnable; used by dm_yield_to_core_peers)
     int yield_pending;         // this caller woke a higher-priority thread pinned to its core: hand off before returning to guest code
+    _Atomic uint64_t hand_epoch;   // this thread's current handoff: threads it makes runnable belong to it until its next thread SVC returns (svc_thread)
+    struct { struct thr *by; uint64_t epoch; } hand[4]; int nhand;   // the handoffs this thread was made runnable for (its waker's, then those the waker belonged to), until it blocks (note_runnable / hand_leave)
     _Atomic int *host_work;    // points at the thread's dm_host_work (registered when the thread starts)
     int host_owned;            // a guest thread the host created itself (dm_guest_thread_spawn): it waits in host code and is not one of the game's threads
     uint64_t wseq;             // Sequence number of wait registration: at equal priority first come, first woken (same order as the Horizon kernel's KThread wait tree)
@@ -126,14 +128,26 @@ void dm_thread_mark_host(void) { thr_t *t = pthread_getspecific(g_keythr); if (t
 static int thr_runnable(const thr_t *t) {
     return !t->wkind && !t->sleeping && !t->pause_req && !(t->host_work && *t->host_work);
 }
-int dm_yield_to_core_peers(void) {
+// Holding G. Whether t was made runnable for self's current handoff (by self, or by a thread that was) and has not blocked since
+static int thr_in_handoff(const thr_t *t, const thr_t *self) {
+    for (int i = 0; i < t->nhand; i++) if (t->hand[i].by == self && t->hand[i].epoch == self->hand_epoch) return 1;
+    return 0;
+}
+// Holding G. t blocked (a guest wait, a timed sleep, YieldToAnyThread): it gave its core back, so it no longer holds the threads whose handoff made it runnable
+static void hand_leave(thr_t *t) { t->nhand = 0; }
+// The peers a handoff waits for are the ones it made runnable, not every runnable higher-priority thread of the core. The host runs a
+// lower-priority thread beside busy higher-priority threads of its core all the time (it has no per-core scheduler; core_preempt aside),
+// and waiting for those threads at a wake had no bound: Tears of the Kingdom's BootupBackgroundWorkThread (priority 58) waited at each
+// wake for its core's busy loaders and workers (issue #247: black screen, no texture loaded), and Donkey Kong Country: Tropical Freeze's
+// priority-58 job thread for threads that poll with short sleeps (issue #259). all = 1 (core_preempt): every runnable higher-priority peer
+int dm_yield_to_core_peers(int all) {
     thr_t *self = pthread_getspecific(g_keythr); if (!self) return 0;
     uint64_t bit = 1ull << (self->core & 63); int waited = 0;
     for (;;) {
         int busy = 0;
         pthread_mutex_lock(&G);
         for (thr_t *t = g_all; t && !busy; t = t->all)
-            if (t != self && t->started && !t->exited && !t->host_owned && t->mask == bit && t->prio < self->prio && thr_runnable(t)) busy = 1;
+            if (t != self && t->started && !t->exited && !t->host_owned && t->mask == bit && t->prio < self->prio && thr_runnable(t) && (all || thr_in_handoff(t, self))) busy = 1;
         pthread_mutex_unlock(&G);
         if (!busy) return waited;
         { struct timespec ts = {0, 50000}; nanosleep(&ts, 0); } waited = 1;
@@ -157,9 +171,16 @@ static thr_t *pick(int kind, uint64_t a) {
 }
 // Holding G. Starting a thread makes it runnable just as waking a waiter does;
 // neither operation promises that an equal-priority or other-core peer has run.
+// t joins the waker's current handoff and those the waker is part of: Metroid Dread's main thread (priority 44) waits for its movie dispatcher (30),
+// and for the worker (30) the dispatcher wakes in turn, though the dispatcher's wake of an equal-priority thread is no handoff of its own
 static void note_runnable(thr_t *t) {
     thr_t *self = pthread_getspecific(g_keythr);
-    if (self && self != t && !t->host_owned && t->prio < self->prio && t->mask == 1ull << (self->core & 63)) self->yield_pending = 1;
+    t->nhand = 0;
+    if (!self || self == t || t->host_owned) return;
+    t->hand[t->nhand].by = self; t->hand[t->nhand++].epoch = self->hand_epoch;
+    for (int i = 0; i < self->nhand && t->nhand < 4; i++)
+        if (self->hand[i].by != t && self->hand[i].by->hand_epoch == self->hand[i].epoch) t->hand[t->nhand++] = self->hand[i];   // only handoffs still in progress
+    if (t->prio < self->prio && t->mask == 1ull << (self->core & 63)) self->yield_pending = 1;
 }
 static void wake(thr_t *t, int64_t res) {
     wait_unlink(t); t->wkind = 0; t->woke = 1; t->wres = res; pthread_cond_signal(&t->cv);
@@ -169,6 +190,7 @@ static void wake(thr_t *t, int64_t res) {
 static int64_t block(thr_t *t, int kind, int64_t ns) {
     t->wkind = kind; t->woke = 0; t->next = g_wait; g_wait = t; t->wseq = ++g_wseq;
     if (ns == 0) { wait_unlink(t); t->wkind = 0; return R_TIMEOUT; }
+    hand_leave(t);
     struct timespec dl;
     int det_wait = ns > 0 && dm_detclock_enabled();
     uint64_t guest_deadline = det_wait ? guest_ns() + (uint64_t)ns : 0;
@@ -254,6 +276,8 @@ static void *thread_main(void *p) {
 }
 static void thr_free(thr_t *t) {   // Holding G; unlink from the all-threads chain and free
     for (thr_t **p = &g_all; *p; p = &(*p)->all) if (*p == t) { *p = t->all; break; }
+    for (thr_t *u = g_all; u; u = u->all)   // no handoff entry may point at the freed object (note_runnable reads the waker's epoch)
+        for (int i = 0; i < u->nhand; ) if (u->hand[i].by == t) u->hand[i] = u->hand[--u->nhand]; else i++;
     pthread_cond_destroy(&t->cv); pthread_cond_destroy(&t->pcv); free(t);
 }
 static void wake_sync_waiters(const void *obj);
@@ -757,10 +781,10 @@ int svc_thread(int num, uint64_t *regs) {
     // also checks peers that were already runnable, without serializing every
     // game's SVCs behind unrelated busy workers by default.
     if (me && (g_core_preempt || me->yield_pending)) {
-        uint64_t t0 = now_ns(); int w = dm_yield_to_core_peers();
+        uint64_t t0 = now_ns(); int w = dm_yield_to_core_peers(g_core_preempt);
         if (w) { g_preempt_n++; g_preempt_ns += now_ns() - t0; }
     }
-    if (me) me->yield_pending = 0;
+    if (me) { me->yield_pending = 0; me->hand_epoch++; }   // the handoff is complete: threads this one made runnable no longer hold it
     return r;
 }
 static int svc_thread_impl(int num, uint64_t *regs) {
@@ -793,6 +817,9 @@ static int svc_thread_impl(int num, uint64_t *regs) {
     case 0x0A: thread_exit(cur()); return 1;
     case 0x0B: {  // SleepThread(x0 ns); 0/-1/-2 = the various kinds of yield
         int64_t ns = (int64_t)regs[0];
+        // A sleep, and YieldToAnyThread (-2), which takes the thread off its core so that lower-priority threads run there, end its part in a handoff;
+        // the other yields (0, -1) pass the core only to threads of its own priority or higher
+        if (ns > 0 || ns == -2) { thr_t *me = pthread_getspecific(g_keythr); if (me && me->nhand) { pthread_mutex_lock(&G); hand_leave(me); pthread_mutex_unlock(&G); } }
         if ((ns > 0 && dm_detclock_enabled()) || ns >= 20000) dm_guest_sleep_ns(ns);
         // Yield (0 / -1 / -2), and sleeps shorter than 20 µs, which sleep_precise would return from at once: Super Mario Galaxy's Hagi polls with SleepThread(100 ns) (main+0x517c),
         // which kept a core at 100% (1.4 billion calls in 200 s); as yields they get the back-off below.

dm/host/tests/t_preempt.c

@@ -152,13 +152,79 @@ static int delayed_wake(unsigned delay, int kind, int exclusion) {
     printf("%s: wait kind %d, dispatch delay %u us, exclusion %d, payload before signaller returns %d\n", ok ? "PASS" : "FAIL", kind, delay, exclusion, ordered);
     return ok;
 }
+// The scope of a handoff: the threads the signaller's wake made runnable, and those they wake in turn, each until it blocks.
+// A busy higher-priority peer of the core that the signaller never woke must not hold it (issue #247: Tears of the Kingdom's
+// boot thread waited for its core's busy loaders at every wake; issue #259: Tropical Freeze's job thread for pollers).
+static thr_t scope_peer[3];
+static uint32_t scope_ev[2][2];   // [event][write, read]
+static _Atomic int scope_payload, scope_release;
+static thr_t *scope_new(thr_t *t, int prio) {
+    memset(t, 0, sizeof *t);
+    pthread_cond_init(&t->cv, NULL); pthread_cond_init(&t->pcv, NULL);
+    t->started = 1; t->prio = prio; t->core = 0; t->mask = 1;
+    pthread_mutex_lock(&G); t->handle = h_new(O_THREAD, t); t->all = g_all; g_all = t; pthread_mutex_unlock(&G);
+    return t;
+}
+static void scope_free(thr_t *t) {
+    pthread_mutex_lock(&G);
+    for (thr_t **p = &g_all; *p; p = &(*p)->all) if (*p == t) { *p = t->all; break; }
+    g_h[t->handle - 0x1000].type = O_FREE; g_h[t->handle - 0x1000].obj = NULL;
+    pthread_mutex_unlock(&G);
+    pthread_cond_destroy(&t->cv); pthread_cond_destroy(&t->pcv);
+}
+static void scope_wait(int ev) { uint32_t h = scope_ev[ev][1]; uint64_t r[32] = {0, (uint64_t)&h, 1, (uint64_t)-1}; svc_thread(0x18, r); assert(r[0] == 0); }
+static void scope_signal(int ev) { uint64_t r[32] = {scope_ev[ev][0]}; svc_thread(0x11, r); assert(r[0] == 0); }
+static void *scope_worker(void *arg) {   // waits on event 1, then finishes its payload (1 ms of host work) and exits
+    thr_t *t = arg; pthread_setspecific(g_keythr, t);
+    scope_wait(1);
+    atomic_store(&scope_payload, 1); usleep(1000); atomic_store(&scope_payload, 2);
+    thread_exit(t); return NULL;
+}
+static void *scope_dispatcher(void *arg) {   // waits on event 0, wakes the worker (its equal priority: no handoff of its own) and exits
+    thr_t *t = arg; pthread_setspecific(g_keythr, t);
+    scope_wait(0); scope_signal(1);
+    thread_exit(t); return NULL;
+}
+static void *scope_any_yield(void *arg) {   // waits on event 1, then yields to any thread (-2) until the signaller has returned
+    thr_t *t = arg; pthread_setspecific(g_keythr, t);
+    scope_wait(1);
+    atomic_store(&scope_payload, 1);
+    while (!atomic_load(&scope_release)) { uint64_t r[32] = {(uint64_t)-2}; svc_thread(0x0B, r); }
+    thread_exit(t); return NULL;
+}
+static void scope_until_waiting(thr_t *t) {
+    for (;;) { pthread_mutex_lock(&G); int w = t->wkind == 4; pthread_mutex_unlock(&G); if (w) break; usleep(100); }
+}
+// kind 0: a busy unrelated peer and one woken worker; 1: a dispatcher that wakes the worker; 2: a woken thread that yields to any thread
+static int handoff_scope(int kind) {
+    for (int e = 0; e < 2; e++) { uint64_t r[32] = {0}; svc_thread(0x45, r); assert(r[0] == 0); scope_ev[e][0] = r[1]; scope_ev[e][1] = r[2]; }
+    atomic_store(&scope_payload, 0); atomic_store(&scope_release, 0);
+    thr_t *busy = scope_new(&scope_peer[0], 30);   // runnable in guest code the whole time, woken by nobody
+    thr_t *w = scope_new(&scope_peer[1], 30), *d = kind == 1 ? scope_new(&scope_peer[2], 30) : NULL;
+    pthread_t pw, pd;
+    assert(!pthread_create(&pw, NULL, kind == 2 ? scope_any_yield : scope_worker, w)); scope_until_waiting(w);
+    if (d) { assert(!pthread_create(&pd, NULL, scope_dispatcher, d)); scope_until_waiting(d); }
+    scope_signal(d ? 0 : 1);
+    pthread_mutex_lock(&G); int payload = atomic_load(&scope_payload), done = w->exited; pthread_mutex_unlock(&G);
+    atomic_store(&scope_release, 1);
+    pthread_join(pw, NULL); if (d) pthread_join(pd, NULL);
+    if (d) scope_free(d);
+    scope_free(w); scope_free(busy);
+    for (int e = 0; e < 2; e++) for (int i = 0; i < 2; i++) { uint64_t r[32] = {scope_ev[e][i]}; svc_thread(0x16, r); }
+    int ok = kind == 2 ? payload == 1 : payload == 2 && done;
+    static const char *what[] = {"woken worker beside a busy unrelated peer", "worker woken by the woken dispatcher", "woken thread yielding to any thread"};
+    printf("%s: handoff scope, %s: signaller returned, payload %d, worker exited %d\n", ok ? "PASS" : "FAIL", what[kind], payload, done);
+    return ok;
+}
 int main(int argc, char **argv) {
     (void)argv;
     alarm(10);
     setenv("DM_PREEMPT", argc > 1 ? "1" : "0", 1);
+    setenv("DM_YIELD_BACKOFF", "0", 1);   // plain yields only: a back-off sleep would end the yielding thread's handoff on its own
     main_handle = svc_thread_main_init();
     condition_order();
     int ok = 1;
+    if (argc == 1) for (int kind = 0; kind <= 2; kind++) ok &= handoff_scope(kind);   // with every runnable peer checked (core_preempt), the busy peer holds the signaller by design
     for (int kind = 0; kind <= 4; kind++) if (kind != 3) {
         ok &= delayed_wake(1000, kind, 0);
         ok &= delayed_wake(20000, kind, 0);

docs/agents/05-debugging.md

@@ -102,7 +102,7 @@ Patterns that worked: print every call of one function with its arguments; stop
   - **Logs:** `sites.tsv` lists each loop as `<head> S <back-edges>`. In the power line's top yielders, a thread's spin passes show as `spin passes N, last at main+0x…`; the site is the loop head.
   - **Checking a loop:** to tell whether one is a real wait, disassemble from its head to its last back-edge. It must reload what it compares on every pass.
 - **Who runs during a hitch.** Guest threads carry their guest names as host thread names (`0x1001 main thread`, `0x1008 Worker`), so `sample <pid>` and `xcrun xctrace record --template 'Time Profiler' --attach <pid>` (no root needed) name them; the export (`xctrace export --xpath '/trace-toc/run[@number="1"]/data/table[@schema="time-profile"]'`) gives each sample's time, thread and backtrace, and guest frames map to `main+…` with the module bases at the top of the log. The trace starts at the TOC's `start-date`; the TSV's `@call_ms` counts from the process start (`kinfo_proc` `p_starttime`), which lines the two up. `DM_HITCH_DUMP=<ms>[:<s>]` logs every guest thread once whenever no frame has been presented for `<ms>` (from `<s>` seconds after the start on, at most 24 times): what it waits on, its last SVC, and where it is, with x0–x2 and x19–x25 when it runs guest code. Breath of the Wild's CPU hitches showed up there as the main thread and two workers in one guest loop with no SVC, and the registers told which thread the loop waited for (its item index in x0; [docs/development.md](../development.md), Spin loops).
-- `DM_PREEMPT=0` / `1` disables / enables the additional check of already-runnable same-core peers after every thread SVC, overriding the profile's `core_preempt` (`hle_thread.c`). Wake-triggered handoffs apply with either setting. The thread dump reports `same-core handoffs N, total T ms`; these waits have no timeout that releases unfinished guest work. See [Same-core wake handoffs](../reference/threads.md#same-core-wake-handoffs) for the runnable-state rule and regression test.
+- `DM_PREEMPT=0` / `1` disables / enables the additional check of already-runnable same-core peers after every thread SVC, overriding the profile's `core_preempt` (`hle_thread.c`). Wake-triggered handoffs apply with either setting; they wait only for the threads the wake made runnable (and those these wake in turn) until each blocks. The thread dump reports `same-core handoffs N, total T ms`; these waits have no timeout that releases unfinished guest work, and a total near or above the run's length means a handoff waits for threads it should not. See [Same-core wake handoffs](../reference/threads.md#same-core-wake-handoffs) for the runnable-state rule and regression test.
 
 ## Racy crashes: load, trace gates
 

docs/agents/symptoms.md

@@ -76,7 +76,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
 - crash indexing a per-core table with a huge offset (index −2). → [Ideal core −2 resolves to the process default core](../reference/threads.md#ideal-core-2-resolves-to-the-process-default-core)
 - the main thread waits forever on an event right after `nnMain` starts, with only three or four threads created; `DM_THR=1` shows no `CreateThread` for the thread it waits on. → [Free thread count (`svcGetInfo` type 24)](../reference/threads.md#free-thread-count-svcgetinfo-type-24)
 - in garbage-collected titles (Unity / IL2CPP), live objects freed, with their finalizers releasing native resources. → [Pausing threads and reading their context for garbage collectors](../reference/threads.md#pausing-threads-and-reading-their-context-for-garbage-collectors)
-- crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`. → [Same-core wake handoffs](../reference/threads.md#same-core-wake-handoffs)
+- crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`. The other way round, a handoff that waits too widely: a black screen at boot that never loads a texture (`packaged textures: first uploads 0`), or a load that never ends behind an animated loading screen, with the thread dump's `same-core handoffs N, total T ms` larger than the run itself. → [Same-core wake handoffs](../reference/threads.md#same-core-wake-handoffs)
 - several threads crash at once and the dumps interleave, then the process hangs instead of exiting. → [Simultaneous crashes of several threads](../reference/threads.md#simultaneous-crashes-of-several-threads)
 - none seen; found while investigating Fire Emblem Engage's demo crash, which it did not change (that was the address-space shortage, Size-aligned regions exhaust the alias address space). → [`SignalToAddress` type 2 with count ≤ 0](../reference/threads.md#signaltoaddress-type-2-with-count--0)
 - a 64-bit wait could return at once or block when it should not (not met in a game). → [`WaitForAddress` type 3: 64-bit wait if equal](../reference/threads.md#waitforaddress-type-3-64-bit-wait-if-equal)

docs/reference/threads.md

@@ -42,11 +42,11 @@ Guest threads, their synchronization and how the host schedules them. Each entry
 
 ### Same-core wake handoffs
 
-- **Symptom:** crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`.
-- **Cause:** Horizon schedules the highest-priority runnable thread on a core ([Mesosphere's scheduler](https://github.com/Atmosphere-NX/Atmosphere/blob/master/libraries/libmesosphere/source/kern_k_scheduler.cpp)). A lower-priority thread that wakes a higher-priority thread pinned to its core cannot continue until that work blocks. The host's pthreads run in parallel. The earlier approximation also confused host dispatch state with guest readiness: `wake()` cleared the guest wait before the pthread resumed, but `thread_info` could still report that pthread waiting. The caller then continued too soon. Its 5 ms timeout opened the same hole under a longer host delay; the recorded `yield_pending` flag was unused unless the profile enabled checks after every thread SVC.
-- **Fix:** every game's thread SVC now honours `yield_pending`. Before returning, it waits for the higher-priority peers pinned to the caller's core to block, sleep, exit, be suspended, or enter explicit host GPU work (`svc_thread` / `dm_yield_to_core_peers` in `hle_thread.c`). Guest runnable state is authoritative, including the wake-to-pthread-resume interval; a host dispatch snapshot cannot release the handoff, and there is no timeout that lets unfinished work through. Host-created helpers and threads that can run on another core are excluded.
-- **Check:** `make -C dm/host/tests GAME=010093801237C000 preempt` delays a woken pthread's dispatch by 1 ms and 20 ms through real event, mutex and condition-variable SVCs. The worker's payload and exit must precede the signaller's return. The original implementation failed both delays; changing readiness alone fixed 1 ms but still failed 20 ms. Equal-priority, other-core, migratable, host-helper, suspended and host-GPU-work peers do not block the caller.
-- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md), whose per-build movie request-ring gates and `core_preempt on` setting are now removed.
+- **Symptom:** crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`. The other way round, a handoff that waits too widely: a black screen at boot that never loads a texture (`packaged textures: first uploads 0`), or a load that never ends behind an animated loading screen, with the thread dump's `same-core handoffs N, total T ms` larger than the run itself.
+- **Cause:** Horizon schedules the highest-priority runnable thread on a core ([Mesosphere's scheduler](https://github.com/Atmosphere-NX/Atmosphere/blob/master/libraries/libmesosphere/source/kern_k_scheduler.cpp)). A lower-priority thread that wakes a higher-priority thread pinned to its core cannot continue until that work blocks. The host's pthreads run in parallel. The earlier approximation also confused host dispatch state with guest readiness: `wake()` cleared the guest wait before the pthread resumed, but `thread_info` could still report that pthread waiting. The caller then continued too soon. Its 5 ms timeout opened the same hole under a longer host delay; the recorded `yield_pending` flag was unused unless the profile enabled checks after every thread SVC. The first unbounded handoff then waited for every runnable higher-priority thread pinned to the caller's core, not only the ones its wake made runnable. The host runs a lower-priority thread beside such threads at all other times, and a core's busy loaders, job workers and pollers are seldom all blocked at the same moment, so a wake could hold its caller for minutes.
+- **Fix:** every game's thread SVC honours `yield_pending`. Before returning, it waits for the threads its wakes made runnable that are higher priority and pinned to the caller's core, and for those such a thread makes runnable in turn while the handoff lasts (Dread's dispatcher wakes the worker), until each blocks, sleeps, yields to any thread (`SleepThread(-2)`, which takes it off its core so that lower-priority threads run there), exits, is suspended, or enters explicit host GPU work (`svc_thread` / `dm_yield_to_core_peers` in `hle_thread.c`). A wake records the thread in the waker's current handoff and in those the waker belongs to (`note_runnable`, at most four); blocking ends that (`hand_leave`), and a handoff ends when its caller's thread SVC returns (`hand_epoch`). Guest runnable state is authoritative, including the wake-to-pthread-resume interval; a host dispatch snapshot cannot release the handoff, and there is no timeout that lets unfinished work through. A higher-priority peer the wake did not make runnable does not hold the caller, as at every other point on the host. Host-created helpers and threads that can run on another core are excluded.
+- **Check:** `make -C dm/host/tests GAME=010093801237C000 preempt` delays a woken pthread's dispatch by 1 ms and 20 ms through real event, mutex and condition-variable SVCs. The worker's payload and exit must precede the signaller's return. The original implementation failed both delays; changing readiness alone fixed 1 ms but still failed 20 ms. Equal-priority, other-core, migratable, host-helper, suspended and host-GPU-work peers do not block the caller. Its handoff-scope cases add a busy higher-priority peer that nobody woke: the woken worker must still finish first and the signaller must return (the whole-core wait never returns), a worker woken by the woken dispatcher must finish before the signaller returns (fails without the recorded chain), and a woken thread that yields to any thread releases the signaller (never returns without that rule).
+- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md), whose per-build movie request-ring gates and `core_preempt on` setting are now removed; the whole-core wait on [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) ([issue #247](https://github.com/seedds/nxnative/issues/247)) and Donkey Kong Country: Tropical Freeze ([issue #259](https://github.com/seedds/nxnative/issues/259), section 6; no profile here).
 - **Scope:** these are cooperative thread-SVC handoffs, not a complete asynchronous guest scheduler. The optional `core_preempt` profile key (`DM_PREEMPT=0` / `1` overrides it) additionally checks already-runnable peers after every thread SVC; it defaults off. Historically that broader policy moved [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s first frame from 0.6 s to 2.9 s by waiting for busy decompression threads. Wake-triggered handoffs apply with either setting.
 
 ### Simultaneous crashes of several threads