Spin waits that take a spin lock (general, from PR #134, found on Breath of the Wild; fixes #129). In steady play on the Mac, Breath of the Wild had frames with 30-210 ms of CPU work every 1-10 s: the main thread and the two workers spinning in one job queue poll (main+0xef62dc). The first item of some of its jobs is for the thread of core 0; the others wait until it is taken, taking and releasing the queue's lock (an exclusive test-and-set, then a compare-and-clear) and writing a status word on every pass, so the spin wait search (a store or an exclusive store ends a path) did not take it. Polled at full speed by the two workers, the lock went back and forth between their cores, and the main thread, whose item it was, lost nearly every exclusive store: in a 40 s Time Profiler trace it never held the lock while each worker held it in about 750 samples. patch_nso.py "Spin waits": a lock taken and released on the same address on every path back to the head (spin_lock 'acq' / 'rel', counted like the exclusive adds), a test loop on a word inside the region ('tst', whose branch back goes to the original loop and its own gate), and up to four stores whose value and address registers the region never writes ('stc', only in a region that loads something). Such stores write fixed values to fixed addresses, so a loop can change its own state for a few hundred passes at most, well inside the gate's 2 ms of polling, before its passes repeat. A store of a value a pass changes ends its path and the search runs again without it (the path that takes an item stores the new cursor); when the region with stores does not cover the old one's back-edges, the old one is used. Through the gate the workers leave the lock a moment between passes and the main thread takes its item at once. Zora's Domain on an M2 Max at 120 Hz (one host, 42 s of steady play per run, three runs each, before -> after): standing still, frames with more than 30 ms of work 2.8 / 6.3 / 3.5 -> 0 per 30 s, the frame's work p99 20-21 -> 8.3-8.4 ms; turning the camera, CPU hitches 11.9 / 13.3 / 6.3 -> 0, p99 21-25 -> 9-15 ms; process CPU per frame unchanged. Every module on the game drives rewritten with the old and the new search (206: 108 NSOs and Super Smash Bros. Ultimate's 98 NROs): 175 byte for byte as before (Hades, Super Smash Bros. Ultimate, Super Mario Odyssey among them), 7 fail alike under both (every rtld, New Super Mario Bros. U Deluxe), and 24 take new or wider loops, all but Breath of the Wild's queue through the store rule alone: Monster Hunter Rise's main 90 -> 166 (72 of them one inlined countdown whose reset makes the next pass leave), BALL x PIT 87 -> 94, Animal Crossing: New Horizons 162 -> 166, Balatro 9 -> 12, Octopath Traveler 0 75 -> 78, Pikmin 4 80 -> 83, Bayonetta 3 35 -> 37, Breath of the Wild 308 -> 310, one more each in Big Brain Academy, Mario Kart 8 Deluxe, Metroid Dread, Pokemon Legends: Z-A, Stardew Valley, Super Mario Bros. Wonder (both builds) and Tears of the Kingdom; the movie library's thread loop is wider in the subsdk0 of Breath of the Wild, Mario Kart 8 Deluxe, Animal Crossing, BALL x PIT, Monster Train 2 and Octopath Traveler 0, and one loop is wider in the main of AI: THE SOMNIUM FILES and Advance Wars 1+2. No loop of the old search is lost or loses a back-edge. Each distinct shape was read in its disassembly: one-pass resets and state setters (the gate is reached only when another thread changes the word), parser states that leave on the next pass, and real flag waits. Rewriting all 206 takes about 4,100 s of CPU instead of 3,250. Diagnostics: DM_HITCH_DUMP=<ms>[:<s>] logs every guest thread (its wait, last SVC, where it is, x0-x2 and x19-x25 in guest code) when no NVN frame was presented for that long, and each guest thread names its pthread '<handle> <name>' (0x1008 Worker) before it enters guest code, the main thread '0x1001 main thread', so sample and Instruments' Time Profiler show which guest thread a sample belongs to. The graphics tests that link hle_nvn.c without the host define dm_present_ns (t_host_stubs.c weak, t_nvn.c). New test dm/tools/tests/test_spin_waits.py. Checked: the host builds, the host unit tests, the 46 tool tests and the gfx run list pass; Breath of the Wild into Zora's Domain and Mario Kart 8 Deluxe for 85 s from boot with the new rewrite (the PR's author), and Monster Hunter Rise from a fresh save for 180 s headless with each rewrite on one host (e72a065's, the chain of its Mac libraries): 60 fps, frame p95 about 17.5 ms, the same red lines, no hang. Not checked: Paper Mario: The Thousand-Year Door, DARK SOULS: REMASTERED, FINAL FANTASY TACTICS and FINAL FANTASY RESONANCE (no files on the drives), the iPhone. docs/development.md (Spin loops, Breath of the Wild) and docs/agents/05-debugging.md say so
This commit is contained in:
seedds authored and MacMini committed
1 parent
24c38e849a
commit
6d34984488
8 files changed
+231
-12
No files matched your search
@@ -1660,6 +1660,7 @@ NVN(nvnQueuePresentTexture, void, NVNqueue *q, NVNwindow *w, int idx) {
|
||||
}
|
||||
gfx_enter();
|
||||
uint64_t t0 = now_ns();
|
||||
{ extern _Atomic uint64_t dm_present_ns; atomic_store(&dm_present_ns, t0); } // DM_HITCH_DUMP (hle_thread.c)
|
||||
if (idx >= 0 && idx < o->ntex && idx < 4) o->ticket[idx] = mtl_present(CTX(o->tex[idx]), present_period(o) / 1e9, o->crop); // enqueue only, return immediately
|
||||
gfx_leave();
|
||||
atomic_fetch_add(&g_nvn_ns, now_ns() - t0);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
// What the tests that link hle_nvn.c and hle_bigstack.c without the host need of it: weak definitions, so a test that defines its own keeps it
|
||||
#include <stdint.h>
|
||||
__attribute__((weak)) uint64_t g_t0_ns; // process start time (host main.c), read by hle_nvn's per-frame timing
|
||||
__attribute__((weak)) _Atomic uint64_t dm_present_ns; // when the last present was made, for DM_HITCH_DUMP (hle_thread.c)
|
||||
__attribute__((weak)) int dm_guest_as_contains(uintptr_t address) { (void)address; return 0; } // no guest address space (hle_bigstack.c)
|
||||
@@ -2,6 +2,7 @@
|
||||
// For build and run see dm/gfx/tests/Makefile (make -C dm/gfx/tests).
|
||||
#include "../nvn/hle_nvn.c"
|
||||
uint64_t g_t0_ns; // process start time from host main.c (used by hle_nvn's per-frame @t_s); unit test stub
|
||||
_Atomic uint64_t dm_present_ns; // when the last present was made (hle_thread.c, DM_HITCH_DUMP); unit test stub
|
||||
#include "../tex/nvn_deswizzle.h"
|
||||
#include <time.h>
|
||||
void *front_metal_layer(void) { return NULL; } // no window in unit tests: present only counts
|
||||
|
||||
+60
-1
@@ -238,6 +238,7 @@ static void qos_apply(thr_t *t) { // on the thread itself
|
||||
}
|
||||
// A host helper thread (audio notify / silent sink, vsync, movie decode) created from a guest thread inherits that thread's class; under DM_QOS_MAP it sets its own instead
|
||||
void dm_host_helper_qos(qos_class_t q) { if (g_qos_map) pthread_set_qos_class_self_np(q, 0); }
|
||||
static const char *thr_name(thr_t *t, char *buf, size_t n);
|
||||
static void *thread_main(void *p) {
|
||||
thr_t *t = p;
|
||||
pthread_setspecific(g_keythr, t);
|
||||
@@ -248,6 +249,7 @@ static void *thread_main(void *p) {
|
||||
crash_thread_init();
|
||||
guest_thread_init();
|
||||
pthread_mutex_lock(&G); while (!t->started) pthread_cond_wait(&t->cv, &G); pthread_mutex_unlock(&G);
|
||||
{ char nb[48], hn[64]; snprintf(hn, sizeof hn, "%#x %s", t->handle, thr_name(t, nb, sizeof nb)); pthread_setname_np(hn); } // the guest's name (set before StartThread) for sample / Instruments
|
||||
dm_thread_enter(t->entry, t->arg, t->stack_top); // Does not return: the guest must exit through ExitThread
|
||||
die("guest thread entry returned");
|
||||
}
|
||||
@@ -255,7 +257,6 @@ static void thr_free(thr_t *t) { // Holding G; unlink from the all-threads cha
|
||||
for (thr_t **p = &g_all; *p; p = &(*p)->all) if (*p == t) { *p = t->all; break; }
|
||||
pthread_cond_destroy(&t->cv); pthread_cond_destroy(&t->pcv); free(t);
|
||||
}
|
||||
static const char *thr_name(thr_t *t, char *buf, size_t n);
|
||||
static void wake_sync_waiters(const void *obj);
|
||||
static void thread_exit(thr_t *t) {
|
||||
t->exit_ns = now_ns(); t->exit_tick = dm_thread_tick_fn ? dm_thread_tick_fn() : 0;
|
||||
@@ -335,16 +336,74 @@ static void ctx_fill(const thr_t *t, hz_ctx_t *o) { // holding G, t is stopped
|
||||
}
|
||||
}
|
||||
|
||||
// DM_HITCH_DUMP=<ms>[:<s>] (diagnostic): when no frame has been presented for <ms>, from <s> seconds after the start on, log every guest thread once per stall: priority, core and
|
||||
// mask, what it waits on, its last SVC and call site, and where it is (pc, lr and the frame-pointer chain: up to 8 guest frames, the host frames above them; x0-x2 and x19-x25
|
||||
// when it runs guest code), read with the thread suspended for a moment. At most 24 stalls per run. It found Breath of the Wild's job queue poll (patch_nso.py "Spin waits")
|
||||
_Atomic uint64_t dm_present_ns; // hle_nvn.c: when the last present was made (host monotonic ns)
|
||||
static void hitch_where(thr_t *t, char *o, size_t n) {
|
||||
size_t q = 0; o[0] = 0;
|
||||
if (!t->mport || t->exited || thread_suspend(t->mport) != KERN_SUCCESS) return;
|
||||
arm_thread_state64_t st; mach_msg_type_number_t c = ARM_THREAD_STATE64_COUNT;
|
||||
if (thread_get_state(t->mport, ARM_THREAD_STATE64, (thread_state_t)&st, &c) == KERN_SUCCESS) {
|
||||
uint64_t pc = __darwin_arm_thread_state64_get_pc(st), fp = __darwin_arm_thread_state64_get_fp(st), lr = __darwin_arm_thread_state64_get_lr(st);
|
||||
if (module_of((void *)pc)) { q += (size_t)snprintf(o + q, n - q, " [x0=%#llx x1=%#llx x2=%#llx", (unsigned long long)st.__x[0], (unsigned long long)st.__x[1], (unsigned long long)st.__x[2]);
|
||||
for (int i = 19; i <= 25; i++) q += (size_t)snprintf(o + q, n - q, " x%d=%#llx", i, (unsigned long long)st.__x[i]);
|
||||
q += (size_t)snprintf(o + q, n - q, "]"); }
|
||||
for (int k = 0, g = 0; k < 24 && g < 8 && q < n - 80; k++) {
|
||||
uint64_t a = k == 0 ? pc : k == 1 ? lr : 0;
|
||||
if (k >= 2) { uint64_t fr[2]; vm_size_t got = 0; if (!fp || vm_read_overwrite(mach_task_self(), fp, 16, (vm_address_t)fr, &got) || got != 16) break; a = fr[1]; fp = fr[0]; }
|
||||
module_t *m = module_of((void *)a); Dl_info di;
|
||||
if (m) { q += (size_t)snprintf(o + q, n - q, " %s+%#llx", m->name, (unsigned long long)(a - (uint64_t)m->base)); g++; }
|
||||
else if (k > 5) continue;
|
||||
else if (dladdr((void *)a, &di) && di.dli_sname) q += (size_t)snprintf(o + q, n - q, " %s", di.dli_sname);
|
||||
else q += (size_t)snprintf(o + q, n - q, " %#llx", (unsigned long long)a);
|
||||
}
|
||||
}
|
||||
thread_resume(t->mport);
|
||||
}
|
||||
static void *hitch_main(void *arg) {
|
||||
pthread_setname_np("dm hitch dump");
|
||||
const char *e = arg; uint64_t lim = (uint64_t)atoi(e) * 1000000ull, seen = 0, skip = 0; int n = 0;
|
||||
if (strchr(e, ':')) skip = (uint64_t)atoi(strchr(e, ':') + 1) * 1000000000ull;
|
||||
extern uint64_t g_t0_ns;
|
||||
while (n < 24) {
|
||||
{ struct timespec ts = {0, 2000000}; nanosleep(&ts, 0); }
|
||||
uint64_t p = atomic_load(&dm_present_ns), now = now_ns();
|
||||
if (!p || p == seen || now - p < lim || now - g_t0_ns < skip) continue;
|
||||
seen = p; n++;
|
||||
static const char *kinds[] = {"run", "mutex", "condvar", "address", "sync"};
|
||||
LOG("hitch dump %d: no present for %.1f ms", n, (now - p) / 1e6);
|
||||
thr_t *ts[256]; int nt = 0;
|
||||
pthread_mutex_lock(&G); for (thr_t *t = g_all; t && nt < 256; t = t->all) if (t->started && !t->exited) ts[nt++] = t; pthread_mutex_unlock(&G); // not held while threads are suspended
|
||||
for (int k = 0; k < nt; k++) { thr_t *t = ts[k];
|
||||
char nb[32], w[900]; module_t *m = module_of((void *)t->last_lr);
|
||||
thread_basic_info_data_t bi; mach_msg_type_number_t c = THREAD_BASIC_INFO_COUNT; int rs = -1;
|
||||
if (t->mport && thread_info(t->mport, THREAD_BASIC_INFO, (thread_info_t)&bi, &c) == KERN_SUCCESS) rs = bi.run_state;
|
||||
hitch_where(t, w, sizeof w);
|
||||
LOG(" %#x '%s' prio %d core %d mask %#llx %s %#llx/%#llx%s%s state %d svc %#x lr=%s+%#llx at:%s", t->handle, thr_name(t, nb, sizeof nb), t->prio, t->core, (unsigned long long)t->mask,
|
||||
kinds[t->wkind & 7], (unsigned long long)t->waddr, (unsigned long long)t->wkey, t->sleeping ? " sleeping" : "", t->host_work && *t->host_work ? " host-work" : "", rs, t->last_svc, m ? m->name : "?",
|
||||
(unsigned long long)(m ? t->last_lr - (uint64_t)m->base : t->last_lr), w);
|
||||
}
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
static void hitch_init(void) {
|
||||
const char *e = getenv("DM_HITCH_DUMP"); if (!e || atoi(e) <= 0) return;
|
||||
pthread_t th; pthread_create(&th, NULL, hitch_main, (void *)e); pthread_detach(th);
|
||||
LOG("DM_HITCH_DUMP=%s: the guest threads are logged when no frame is presented for that many ms", e);
|
||||
}
|
||||
// Register the main thread and return its handle (passed to nnosInitialize as the main thread handle)
|
||||
uint32_t svc_thread_main_init(void) {
|
||||
pthread_key_create(&g_keythr, NULL);
|
||||
qos_map_init();
|
||||
pause_install();
|
||||
hitch_init();
|
||||
thr_t *t = calloc(1, sizeof *t);
|
||||
pthread_cond_init(&t->cv, NULL); pthread_cond_init(&t->pcv, NULL);
|
||||
t->started = 1; t->prio = 44; t->core = 0; t->mask = 7; t->pt = pthread_self(); t->mport = pthread_mach_thread_np(t->pt); t->host_work = &dm_host_work;
|
||||
pthread_mutex_lock(&G); t->handle = h_new(O_THREAD, t); t->id = g_next_id++; t->all = g_all; g_all = t; pthread_mutex_unlock(&G);
|
||||
pthread_setspecific(g_keythr, t);
|
||||
pthread_setname_np("0x1001 main thread");
|
||||
qos_class_t qc = QOS_CLASS_UNSPECIFIED; int rp = 0;
|
||||
pthread_get_qos_class_np(pthread_self(), &qc, &rp); t->qos = (uint32_t)qc;
|
||||
return t->handle;
|
||||
|
||||
+88
-10
@@ -910,11 +910,21 @@ if FS_STATE_OFF: # the build's stubs (FRAME_STEP of its patches.py)
|
||||
# A loop counts as a spin wait only when it provably cannot leave by itself. From its head T, every pass along the region R (the paths from T back to T, within SPIN_WIN bytes
|
||||
# after T) runs only loads, register and flag arithmetic, direct branches and barriers: no store, call, SVC, system register or event wait; and no register or flag that is
|
||||
# live at T is written in R (no counter, cursor or delay). One exception: an exclusive add of a constant (ld(a)xr; add/sub #k; st(l)xr; cbnz) that every path back to T
|
||||
# undoes on the same address (a worker that registers with a job, sees it finished and backs out). Such a pass leaves memory and registers as it found them, so only another
|
||||
# thread can end the loop, and a pause there changes only timing. R is copied into the patch area; the original back-edges jump to the copy (no pause on the first pass, so a
|
||||
# undoes on the same address (a worker that registers with a job, sees it finished and backs out). Three more let a poll that takes a spin lock to look at the state it waits
|
||||
# for count too (spin_lock, spin_store): a lock taken (an exclusive test-and-set retried until its store succeeds) and released (an exclusive compare-and-clear, or a store-release
|
||||
# of zero) on the same address on every path back to T; a test loop on a word (load, compare with a constant, branch back to the load) inside R; and stores whose value and
|
||||
# address registers R never writes, at most SPIN_STORES of them and only beside a load (see below). Every loop the search took without these still counts as before, the
|
||||
# same region over the same back-edges or a wider one over more. Breath of the Wild's job queue (main+0xef62dc) is such a poll: the first item of some jobs is for the
|
||||
# thread of core 0, and the others wait until it is taken, taking and releasing the queue's lock on every pass; polled at full speed by two workers, the lock went back and
|
||||
# forth between their cores, and the main thread, whose item it was, lost nearly every exclusive store to them (CPU frames of 30-200 ms with three cores spinning there).
|
||||
# Such a pass leaves memory and registers as it found them (a pass with stores, from its last state change on), so only another thread can end the loop, and a pause
|
||||
# there changes only timing. R is copied into the patch area; the original back-edges jump to the copy (no pause on the first pass, so a
|
||||
# single retry costs nothing), the copy's back-edges go through a spin gate, which the host takes as a yield with the yield back-off (hle_thread.c dm_spin_wait), and exits
|
||||
# from the copy go back to the original code. The gate keeps every register and the flags (entry.S)
|
||||
SPIN_PATHS, SPIN_STEPS = 256, 4096 # complete paths back to T; search steps (R itself lies within SPIN_WIN bytes)
|
||||
SPIN_PATHS, SPIN_STEPS, SPIN_STORES = 256, 4096, 4 # complete paths back to T; search steps (R itself lies within SPIN_WIN bytes)
|
||||
# Stores in a region: each writes a fixed value to a fixed address, so the words they write can hold only their first values or the stored ones, and the registers at T are the
|
||||
# same on every pass: with at most SPIN_STORES of them a loop can change its own state only a few hundred passes at most (each a cycle of a few instructions), then its passes
|
||||
# repeat, and only another thread ends it. The gate polls on for its first 2 ms, so a loop that ends by itself never sleeps there
|
||||
SPIN_LSE = ('ldadd', 'ldclr', 'ldeor', 'ldset', 'ldsmax', 'ldsmin', 'ldumax', 'ldumin')
|
||||
SPIN_BAD = {'bl', 'blr', 'br', 'ret', 'eret', 'svc', 'hvc', 'smc', 'brk', 'hlt', 'udf', 'mrs', 'msr', 'sys', 'sysl', 'wfe', 'wfi', 'sev', 'sevl', 'clrex', 'dc', 'ic', 'at',
|
||||
'tlbi', 'drps', 'dcps1', 'dcps2', 'dcps3', 'blraa', 'blrab', 'blraaz', 'blrabz', 'braa', 'brab', 'braaz', 'brabz', 'retaa', 'retab', 'eretaa', 'eretab'}
|
||||
@@ -966,6 +976,42 @@ def spin_rmw(a):
|
||||
if r(l[1], 1) != r(l[0], 0) or r(l[2], 1) != r(l[1], 0) or r(l[3], 0) != r(l[2], 0) or l[3].operands[1].imm != a: return None
|
||||
k = l[1].operands[2].imm * (1 if l[1].mnemonic == 'add' else -1)
|
||||
return spin_norm(l[0].reg_name(base.base)), k, {spin_norm(r(l[0], 0)), spin_norm(r(l[1], 0)), spin_norm(r(l[2], 0))} - {None}, l
|
||||
def spin_lock(a):
|
||||
"""An exclusive spin lock sequence or a test loop entered at a → (kind, xN, its instructions, how many of them come before a, how many from a on, registers read, registers written)
|
||||
'acq' ld(a)xr wA,[xN]; cbz wA,+16; clrex; b <the load>; st(l)xr wS,wK,[xN]; cbnz wS,<the load> test-and-set, retried until the store succeeds (wK not zero)
|
||||
'rel' clrex; ld(a)xr wA,[xN]; cmp wA,#k; b.ne <the clrex>; st(l)xr wS,wzr,[xN]; cbnz wS,<the load> compare-and-clear, entered at the load; or stlr wzr,[xN]
|
||||
'tst' ldr|ldrb|ldrh wA,[xN{,#imm}]; cmp wA,#k; b.eq|b.ne <the load> a test loop that only another thread ends"""
|
||||
l = [spin_ins(a + 4 * i) for i in range(6)]
|
||||
if l[0] is None: return None
|
||||
r = lambda x, i: x.reg_name(x.operands[i].reg)
|
||||
plain = lambda x, i: x.operands[i].type == ARM64_OP_MEM and not x.operands[i].mem.index and not x.operands[i].mem.disp
|
||||
m0 = l[0].mnemonic
|
||||
if m0 in ('ldxr', 'ldaxr') and None not in l and r(l[0], 0).startswith('w') and plain(l[0], 1):
|
||||
base = l[0].operands[1].mem.base; bn = spin_norm(l[0].reg_name(base))
|
||||
if (l[1].mnemonic == 'cbz' and r(l[1], 0) == r(l[0], 0) and l[1].operands[1].imm == a + 16 and l[2].mnemonic == 'clrex' and l[3].mnemonic == 'b' and l[3].operands[0].imm == a
|
||||
and l[4].mnemonic in ('stxr', 'stlxr') and plain(l[4], 2) and l[4].operands[2].mem.base == base and r(l[4], 1) not in ('wzr', 'xzr')
|
||||
and l[5].mnemonic == 'cbnz' and r(l[5], 0) == r(l[4], 0) and l[5].operands[1].imm == a):
|
||||
return ('acq', bn, l, 0, 6, {bn, spin_norm(r(l[4], 1))}, {spin_norm(r(l[0], 0)), spin_norm(r(l[4], 0))})
|
||||
p = spin_ins(a - 4)
|
||||
if (p is not None and p.mnemonic == 'clrex' and l[1].mnemonic == 'cmp' and r(l[1], 0) == r(l[0], 0) and l[1].operands[1].type == ARM64_OP_IMM
|
||||
and l[2].mnemonic == 'b.ne' and l[2].operands[0].imm == a - 4 and l[3].mnemonic in ('stxr', 'stlxr') and r(l[3], 1) == 'wzr'
|
||||
and plain(l[3], 2) and l[3].operands[2].mem.base == base and l[4].mnemonic == 'cbnz' and r(l[4], 0) == r(l[3], 0) and l[4].operands[1].imm == a):
|
||||
return ('rel', bn, [p] + l[:5], 1, 5, {bn}, {spin_norm(r(l[0], 0)), spin_norm(r(l[3], 0)), 'nzcv'})
|
||||
if m0 == 'stlr' and len(l[0].operands) == 2 and r(l[0], 0) in ('wzr', 'xzr') and plain(l[0], 1):
|
||||
bn = spin_norm(l[0].reg_name(l[0].operands[1].mem.base)); return ('rel', bn, l[:1], 0, 1, {bn}, set())
|
||||
if (m0 in ('ldr', 'ldrb', 'ldrh') and None not in l[:3] and len(l[0].operands) == 2 and r(l[0], 0).startswith('w') and l[0].operands[1].type == ARM64_OP_MEM
|
||||
and not l[0].operands[1].mem.index and not l[0].writeback and l[1].mnemonic == 'cmp' and r(l[1], 0) == r(l[0], 0) and l[1].operands[1].type == ARM64_OP_IMM
|
||||
and l[2].mnemonic in ('b.eq', 'b.ne') and l[2].operands[0].imm == a):
|
||||
bn = spin_norm(l[0].reg_name(l[0].operands[1].mem.base)); return ('tst', bn, l[:3], 0, 3, {bn}, {spin_norm(r(l[0], 0)), 'nzcv'})
|
||||
return None
|
||||
def spin_store(ins):
|
||||
"""str / strb / strh / stur / sturb / sturh of a w register or zr to [xN{,#imm}] (no writeback, no register offset) → (value register or None for zr, base register), or None"""
|
||||
if ins is None or ins.mnemonic not in ('str', 'strb', 'strh', 'stur', 'sturb', 'sturh') or ins.writeback or len(ins.operands) != 2: return None
|
||||
v, o = ins.operands
|
||||
if v.type != ARM64_OP_REG or o.type != ARM64_OP_MEM or o.mem.index: return None
|
||||
vn = ins.reg_name(v.reg)
|
||||
if not (vn.startswith('w') or vn in ('xzr',)): return None
|
||||
return (spin_norm(vn), spin_norm(ins.reg_name(o.mem.base)))
|
||||
def spin_rebranch(ins, pc, tgt):
|
||||
"""the direct branch ins re-encoded at pc with target tgt"""
|
||||
w = struct.unpack('<I', ins.bytes)[0]; off = tgt - pc
|
||||
@@ -991,14 +1037,24 @@ def spin_quick(T, hi):
|
||||
if 0x14 <= t <= 0x17: succ = (a + (((w & 0x3FFFFFF) ^ 0x2000000) - 0x2000000) * 4,)
|
||||
elif t == 0x54 or t & 0x7E == 0x34: succ = (a + ((((w >> 5) & 0x7FFFF) ^ 0x40000) - 0x40000) * 4, a + 4)
|
||||
elif t & 0x7E == 0x36: succ = (a + ((((w >> 5) & 0x3FFF) ^ 0x2000) - 0x2000) * 4, a + 4)
|
||||
elif any(w & m == v for m, v in SPIN_STOP): continue
|
||||
elif any(w & m == v for m, v in SPIN_STOP) and not (w & 0xBFC00000 in (0xB9000000, 0x39000000, 0x79000000) or w & 0xBFE00C00 in (0xB8000000, 0x38000000, 0x78000000)): continue # str / strb / strh (immediate, unscaled) may store the same on every pass
|
||||
else: succ = (a + 4,)
|
||||
if T in succ: return True
|
||||
todo.extend(succ)
|
||||
return False
|
||||
class SpinNo(Exception): pass
|
||||
def spin_region(T, bes):
|
||||
"""the spin wait region with head T (bes: the backward branches to T found by the scan) → dict, or None"""
|
||||
"""the spin wait region with head T (bes: the backward branches to T found by the scan) → dict, or None. A store whose value or address a pass changes is no part of a region:
|
||||
the paths through it are dropped and the region is found again without them (a poll that takes its item once the wait is over stores the new cursor there). When the paths
|
||||
through stores bring in a register that a pass changes, the region is looked for once more with every store ending its path, as before stores could be part of one"""
|
||||
ban = set()
|
||||
for _ in range(8):
|
||||
g = spin_region1(T, bes, ban, True)
|
||||
if not isinstance(g, set): break
|
||||
ban |= g
|
||||
old = spin_region1(T, bes, set(), False) # the region without stores, which the search took before: the wider one only when it covers its back-edges
|
||||
return g if isinstance(g, dict) and (not old or old['valid'] <= g['valid']) else old
|
||||
def spin_region1(T, bes, ban, stores):
|
||||
hi = max(bes)
|
||||
if any(s < hi + 4 and T < e for s, e in DATA_BLOCKS) or not spin_quick(T, hi): return None
|
||||
nodes, back, bad, inner = {}, set(), set(), set()
|
||||
@@ -1006,9 +1062,16 @@ def spin_region(T, bes):
|
||||
while todo:
|
||||
a = todo.pop()
|
||||
if a in nodes or a in bad: continue
|
||||
if a in ban: bad.add(a); continue
|
||||
rmw = spin_rmw(a) if a + 12 <= hi and struct.unpack_from('<I', IMG, a)[0] & 0x3FE00000 == 0x08400000 else None # ldxr / ldaxr
|
||||
lk = None if rmw else spin_lock(a)
|
||||
if lk and lk[0] == 'tst' and a == T: lk = None # the region's own test loop
|
||||
if rmw:
|
||||
n = {'kind': 'rmw', 'base': rmw[0], 'k': rmw[1], 'ins': rmw[3], 'uses': {rmw[0]}, 'defs': rmw[2], 'succ': [a + 16]}
|
||||
elif lk:
|
||||
n = {'kind': lk[0], 'base': lk[1], 'k': {'acq': 1, 'rel': -1, 'tst': 0}[lk[0]], 'ins': lk[2], 'pre': lk[3], 'uses': lk[5], 'defs': lk[6], 'succ': [a + 4 * lk[4]]}
|
||||
elif stores and spin_store(spin_ins(a)):
|
||||
x = spin_ins(a); st = spin_store(x); n = {'kind': 'stc', 'st': st, 'ins': [x], 'uses': {st[1]} | ({st[0]} - {None}), 'defs': set(), 'succ': [a + 4]}
|
||||
else:
|
||||
x = spin_ins(a); k = spin_kind(x)
|
||||
if k is None: bad.add(a); continue
|
||||
@@ -1017,9 +1080,11 @@ def spin_region(T, bes):
|
||||
succ = [t] if x.mnemonic == 'b' else [t, a + 4]
|
||||
else: succ = [a + 4]
|
||||
n = {'kind': k, 'ins': [x], 'succ': succ}
|
||||
if any(a + 4 * i in spin_touched for i in range(len(n['ins']))): bad.add(a); continue
|
||||
pre = n.get('pre', 0)
|
||||
if any(a + 4 * (i - pre) in spin_touched for i in range(len(n['ins']))): bad.add(a); continue
|
||||
nodes[a] = n
|
||||
if n['kind'] == 'rmw': inner |= {a + 4, a + 8, a + 12}
|
||||
if n['kind'] in ('acq', 'rel', 'tst'): inner |= {a + 4 * (i - pre) for i in range(len(n['ins']))} - {a}
|
||||
for s in n['succ']:
|
||||
if s == T: back.add(a)
|
||||
elif T < s <= hi: todo.append(s)
|
||||
@@ -1040,8 +1105,9 @@ def spin_region(T, bes):
|
||||
cnt[1] += 1
|
||||
if cnt[1] > SPIN_STEPS: raise SpinNo
|
||||
n = nodes[a]
|
||||
if n['kind'] == 'rmw':
|
||||
key = (n['base'], ver.get(n['base'], 0)); delta = dict(delta); delta[key] = delta.get(key, 0) + n['k']
|
||||
if n['kind'] in ('rmw', 'acq', 'rel'): # exclusive adds and lock / unlock pairs, each kind on its own
|
||||
key = (n['kind'] != 'rmw', n['base'], ver.get(n['base'], 0)); delta = dict(delta); delta[key] = delta.get(key, 0) + n['k']
|
||||
if n['kind'] != 'rmw' and not 0 <= delta[key] <= 1: raise SpinNo # a release before its acquire or a second acquire: not a lock this pass takes once
|
||||
if n['defs']: ver = dict(ver); ver.update({r: ver.get(r, 0) + 1 for r in n['defs']})
|
||||
for s in n['succ']:
|
||||
if s == T:
|
||||
@@ -1067,7 +1133,12 @@ def spin_region(T, bes):
|
||||
elif s in R: out |= live[s]
|
||||
new = n['uses'] | (out - n['defs'])
|
||||
if new != live[a]: live[a] = new; more = True
|
||||
if live[T] & set().union(*(nodes[a]['defs'] for a in R)): return None
|
||||
defs = set().union(*(nodes[a]['defs'] for a in R))
|
||||
if any(nodes[a]['kind'] == 'stc' for a in R) and not any(nodes[a]['kind'] in ('rmw', 'acq', 'rel', 'tst') or nodes[a]['ins'][0].mnemonic.startswith('ld') for a in R): return None # stores and nothing polled
|
||||
vary = {a for a in R if nodes[a]['kind'] == 'stc' and nodes[a]['uses'] & defs} # a store's value and address must come from registers no pass writes
|
||||
if vary: return vary
|
||||
if sum(nodes[a]['kind'] == 'stc' for a in R) > SPIN_STORES: return None
|
||||
if live[T] & defs: return None
|
||||
return {'T': T, 'R': sorted(R), 'nodes': nodes, 'valid': valid}
|
||||
def spin_emit(g):
|
||||
T, R, nodes, valid = g['T'], g['R'], g['nodes'], g['valid']
|
||||
@@ -1077,6 +1148,12 @@ def spin_emit(g):
|
||||
if n['kind'] == 'rmw':
|
||||
for y in n['ins'][:3]: emit(struct.unpack('<I', y.bytes)[0])
|
||||
fixes.append((len(patch), n['ins'][3], a)); emit(0) # retry: the copy's exclusive load
|
||||
elif n['kind'] in ('acq', 'rel'): # copied as they are: their branches stay inside the sequence
|
||||
cpy[a] = here() + 4 * n['pre']
|
||||
for y in n['ins']: emit(struct.unpack('<I', y.bytes)[0])
|
||||
elif n['kind'] == 'tst': # its branch back goes to the original loop, which passes its own spin gate when it is a spin wait itself
|
||||
for y in n['ins'][:2]: emit(struct.unpack('<I', y.bytes)[0])
|
||||
fixes.append((len(patch), n['ins'][2], ('orig', a))); emit(0)
|
||||
elif n['kind'] == 'pc':
|
||||
rd, tgt = num(spin_norm(x.reg_name(x.operands[0].reg))), x.operands[1].imm
|
||||
emit(ADRP(rd, here(), tgt))
|
||||
@@ -1084,7 +1161,7 @@ def spin_emit(g):
|
||||
elif n['kind'] == 'br': fixes.append((len(patch), x, None)); emit(0)
|
||||
else: emit(struct.unpack('<I', x.bytes)[0])
|
||||
if not (n['kind'] == 'br' and x.mnemonic == 'b'):
|
||||
ft = a + 4 * len(n['ins'])
|
||||
ft = a + 4 * (len(n['ins']) - n.get('pre', 0))
|
||||
if not (i + 1 < len(R) and R[i + 1] == ft): fixes.append((len(patch), None, ft)); emit(0) # the next instruction is not the next one copied
|
||||
pause = here()
|
||||
emit(STP_PRE(29, 30, 31, -16)); emit_gate_call(); emit(BLR(30)); ret = here(); emit(LDP_POST(29, 30, 31, 16))
|
||||
@@ -1095,6 +1172,7 @@ def spin_emit(g):
|
||||
for idx, x, info in fixes:
|
||||
pc = PATCH_OFF + idx
|
||||
if x is None: w = B(pc, cpy.get(info, info))
|
||||
elif isinstance(info, tuple): w = spin_rebranch(x, pc, out(info[1]))
|
||||
elif info is not None: w = spin_rebranch(x, pc, cpy[info])
|
||||
else:
|
||||
t, cond = x.operands[-1].imm, x.mnemonic != 'b'
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
"""The rewriter's spin wait search (patch_nso.py "Spin waits") on hand-assembled AArch64 loops, without game files."""
|
||||
import ast
|
||||
from pathlib import Path
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from capstone import Cs, CS_ARCH_ARM64, CS_MODE_ARM
|
||||
from capstone.arm64 import ARM64_OP_IMM, ARM64_OP_MEM, ARM64_OP_REG
|
||||
|
||||
TOOLS = Path(__file__).resolve().parents[1]
|
||||
# patch_nso.py is a command-line program whose top level loads and rewrites an image: compile only the spin wait search and its constants
|
||||
tree = ast.parse((TOOLS / 'patch_nso.py').read_text())
|
||||
FUNCS = {'spin_ins', 'spin_norm', 'spin_rw', 'spin_kind', 'spin_rmw', 'spin_lock', 'spin_store', 'spin_quick', 'spin_region', 'spin_region1'}
|
||||
CONSTS = {'SPIN_PATHS', 'SPIN_STEPS', 'SPIN_STORES', 'SPIN_LSE', 'SPIN_BAD', 'SPIN_STOP'}
|
||||
body = [n for n in tree.body if (isinstance(n, ast.FunctionDef) and n.name in FUNCS) or (isinstance(n, ast.ClassDef) and n.name == 'SpinNo')
|
||||
or (isinstance(n, ast.Assign) and any(isinstance(t, ast.Tuple) and any(getattr(e, 'id', None) in CONSTS for e in t.elts) or getattr(t, 'id', None) in CONSTS for t in n.targets))]
|
||||
|
||||
# A job queue poll (the shape of Breath of the Wild's main+0xef6354): take the queue's spin lock, look whether the wait is over, release the lock, again.
|
||||
# 0x00 T: cbz x20, X 0x2c str w12, [x20, #0x100] 0x50 ldr w13, [x21, #0x90] (the wait is over: take an item,
|
||||
# 0x04 str w11, [x20, #0x100] 0x30 ldrb w14, [x21, #0x98] 0x54 sub w8, w13, w25 store the new cursor, release
|
||||
# 0x08 L0: ldr w14, [x9] 0x34 cmp w0, #0 0x58 add w15, w8, w25 and leave)
|
||||
# 0x0c cmp w14, #1 0x38 ccmp w14, #0, #4, ne 0x5c str w15, [x21, #0x94]
|
||||
# 0x10 b.eq L0 0x3c ldr w25, [x21, #0x94] 0x60 cset w13, hs
|
||||
# 0x14 L1: ldaxr w14, [x9] 0x40 ccmp w25, #0, #0, ne 0x64 b U
|
||||
# 0x18 cbz w14, L2 0x44 cset w14, gt 0x68 C: clrex
|
||||
# 0x1c clrex 0x48 cmp w14, #1 0x6c U: ldxr w15, [x9] / cmp w15, #1 / b.ne C
|
||||
# 0x20 b L1 0x4c b.ne U 0x78 stlxr w15, wzr, [x9] / cbnz w15, U
|
||||
# 0x24 L2: stxr w14, w10, [x9] / cbnz w14, L1 0x80 cbz w14, T 0x84 X: ret
|
||||
POLL = [0xb4000434, 0xb901028b, 0xb940012e, 0x710005df, 0x54ffffc0, 0x885ffd2e, 0x3400006e, 0xd5033f5f, 0x17fffffd, 0x880e7d2a, 0x35ffff6e, 0xb901028c,
|
||||
0x394262ae, 0x7100001f, 0x7a4019c4, 0xb94096b9, 0x7a401b20, 0x1a9fd7ee, 0x710005df, 0x54000101, 0xb94092ad, 0x4b1901a8, 0x0b19010f, 0xb90096af,
|
||||
0x1a9f37ed, 0x14000002, 0xd5033f5f, 0x885f7d2f, 0x710005ff, 0x54ffffa1, 0x880ffd3f, 0x35ffff8f, 0x34fffc0e, 0xd65f03c0]
|
||||
|
||||
|
||||
def region(words, T, bes):
|
||||
md = Cs(CS_ARCH_ARM64, CS_MODE_ARM); md.detail = True
|
||||
scope = {'re': re, 'struct': struct, 'ARM64_OP_IMM': ARM64_OP_IMM, 'ARM64_OP_MEM': ARM64_OP_MEM, 'ARM64_OP_REG': ARM64_OP_REG,
|
||||
'IMG': struct.pack('<' + 'I' * len(words), *words) + bytes(64), 'md': md, 'spin_dec': {}, 'DATA_BLOCKS': [], 'spin_touched': set()}
|
||||
exec(compile(ast.Module(body=body, type_ignores=[]), str(TOOLS / 'patch_nso.py'), 'exec'), scope)
|
||||
return scope['spin_region'](T, bes)
|
||||
|
||||
|
||||
def patched(**at):
|
||||
w = list(POLL)
|
||||
for off, word in at.items(): w[int(off[1:], 16) // 4] = word
|
||||
return w
|
||||
|
||||
|
||||
class SpinWaits(unittest.TestCase):
|
||||
def test_lock_poll_is_a_wait(self):
|
||||
g = region(POLL, 0, [0x80])
|
||||
self.assertIsNotNone(g)
|
||||
self.assertEqual(g['valid'], {0x80})
|
||||
self.assertNotIn(0x5c, g['R']) # the path that takes an item stores a new cursor: not part of the wait
|
||||
self.assertEqual({g['nodes'][a]['kind'] for a in g['R']} >= {'acq', 'rel', 'tst', 'stc'}, True)
|
||||
|
||||
def test_store_of_a_changing_value(self): # the wait path stores w14, which the pass writes
|
||||
self.assertIsNone(region(patched(o2c=0xb901028e), 0, [0x80]))
|
||||
|
||||
def test_lock_not_released(self): # the wait path skips the release (b.ne U → b.ne to the back-edge)
|
||||
self.assertIsNone(region(patched(o4c=0x540001a1), 0, [0x80]))
|
||||
|
||||
def test_nothing_polled(self): # str xzr, [x8]; b back: stores of a constant and no load
|
||||
self.assertIsNone(region([0xf900011f, 0x17ffffff], 0, [4]))
|
||||
|
||||
def test_plain_test_loop(self): # ldr w8, [x0]; cbz w8, back
|
||||
g = region([0xb9400008, 0x34ffffe8], 0, [4])
|
||||
self.assertIsNotNone(g)
|
||||
self.assertEqual(g['valid'], {4})
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -101,6 +101,7 @@ Patterns that worked: print every call of one function with its arguments; stop
|
||||
- **Switches:** `DM_SPIN=0` makes the gate return at once, so the loops poll with no sleep; the gate itself stays. `DM_SPIN_POLL=<µs>` sets how long a wait polls before it yields: 2000 by default, which is where the yield back-off starts sleeping. `DM_SPIN_LIST=1` makes the rewriter print every loop it found.
|
||||
- **Logs:** `sites.tsv` lists each loop as `<head> S <back-edges>`. In the power line's top yielders, a thread's spin passes show as `spin passes N, last at main+0x…`; the site is the loop head.
|
||||
- **Checking a loop:** to tell whether one is a real wait, disassemble from its head to its last back-edge. It must reload what it compares on every pass.
|
||||
- **Who runs during a hitch.** Guest threads carry their guest names as host thread names (`0x1001 main thread`, `0x1008 Worker`), so `sample <pid>` and `xcrun xctrace record --template 'Time Profiler' --attach <pid>` (no root needed) name them; the export (`xctrace export --xpath '/trace-toc/run[@number="1"]/data/table[@schema="time-profile"]'`) gives each sample's time, thread and backtrace, and guest frames map to `main+…` with the module bases at the top of the log. The trace starts at the TOC's `start-date`; the TSV's `@call_ms` counts from the process start (`kinfo_proc` `p_starttime`), which lines the two up. `DM_HITCH_DUMP=<ms>[:<s>]` logs every guest thread once whenever no frame has been presented for `<ms>` (from `<s>` seconds after the start on, at most 24 times): what it waits on, its last SVC, and where it is, with x0–x2 and x19–x25 when it runs guest code. Breath of the Wild's CPU hitches showed up there as the main thread and two workers in one guest loop with no SVC, and the registers told which thread the loop waited for (its item index in x0; [docs/development.md](../development.md), Spin loops).
|
||||
- `DM_PREEMPT=0` / `1` turns the same-core preemption approximation off / on whatever the profile's `core_preempt` says (`hle_thread.c`). The thread dump at exit reports `same-core preemption yields N, total T ms, hit the cap K times` (yields, total wait, 5 ms cap hits). A higher-priority thread that keeps a core busy shows up in the first eight `same-core preemption yield hit its cap` lines, with its last SVC and call site.
|
||||
|
||||
## Racy crashes: load, trace gates
|
||||
|
||||
+5
-1
@@ -210,10 +210,13 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- **Spin loops.** A guest loop that polls memory with no SVC at all spins on a host core. On the Switch the spin fills only its own core, and a higher-priority thread pinned there preempts it. Paper Mario: The Thousand-Year Door's three job workers (`main+0xa8e680`) poll the job list this way while a batch is pending; in Rogueport on an iPhone they took 94% of a core each and the game ran at 24 fps. The rewriter finds such loops in every module (`patch_nso.py`, "Spin waits"). A loop counts only when it provably cannot leave by itself:
|
||||
- every path from its head back to it runs only loads, register and flag arithmetic, direct branches and barriers (no store, call, SVC, system register or event wait);
|
||||
- nothing live at the head is written on the way (no counter, cursor or delay);
|
||||
- an exclusive add that every path back undoes on the same address is allowed (a worker that registers with a job, sees it finished and backs out).
|
||||
- an exclusive add that every path back undoes on the same address is allowed (a worker that registers with a job, sees it finished and backs out);
|
||||
- so is a spin lock taken and released on the same address on every path back (an exclusive test-and-set retried until it stores, then an exclusive compare-and-clear or a store-release of zero), a test loop on a word inside the loop (load, compare with a constant, branch back to the load), and up to four stores whose value and address registers the loop never writes. Each such store writes a fixed value to a fixed address, so the words it writes can only hold their first value or a stored one: a loop like that changes its own state for a few hundred passes at most (a fraction of the gate's 2 ms of polling), then its passes repeat and only another thread can end it (a loop with such stores must also load something). A store of a value the pass changes ends its path, and the search runs again without it: a poll that takes its item once the wait is over stores the new cursor there;
|
||||
|
||||
Such a pass leaves memory and registers as they were, so only another thread can end the loop, and a pause there changes only timing. The loop is copied into the patch area. Its original back-edges jump to the copy, so the first pass costs nothing, and the copy's back-edges go through a spin gate (`dm_spin_wait`). A wait polls on through the gate for its first 2 ms and then sleeps as a run of yields does (`DM_SPIN_POLL`, `DM_YIELD_BACKOFF`); `DM_SPIN=0` never sleeps there. The gate alone slows the poll down, and on the Mac that was the whole gain: most waits are handoffs between workers, and a `sched_yield` on every pass raised frame p95 by 2–5 ms there.
|
||||
|
||||
Breath of the Wild's job queue (`main+0xef62dc`) needed the lock rule. The first item of some of its jobs is for the thread of core 0, and the threads of the other cores wait until it is taken (the queue's flag at +0x98, its cursor at +0x94), each taking and releasing the queue's lock (+0x1c) on every pass. The two workers (cores 1 and 2) polled at full speed, so the lock went back and forth between their cores, and the main thread (core 0), whose item it was, lost nearly every exclusive store to them: in 40 s of a Time Profiler trace it never once held the lock while the workers held it about 750 times each, and the three spun for 30–210 ms at a time, the CPU hitches in its notes below. Through the gate the workers leave the lock a moment between passes and the main thread takes its item at once. Zora's Domain on an M2 Max at 120 Hz, the same host, 42 s of steady play per run, three runs each, before → after: standing still, frames with more than 30 ms of CPU work 2.8 / 6.3 / 3.5 → 0 per 30 s, the frame's work p99 20–21 → 8.3–8.4 ms, frames late on screen 2.8 / 5.6 / 2.8 → 2.1 / 0.7 / 2.1 per 30 s (the rest are 120 Hz refresh slips); turning the camera, 11.9 / 13.3 / 6.3 → 0 CPU hitches, p99 21–25 → 9–15 ms, late frames 12.6 / 13.3 / 7.7 → 1.4 each. The search keeps every loop it found before and adds: Breath of the Wild's `main` 308 → 310 (this one, and a state setter at `main+0xf82230` that stores its state and returns on the second pass unless another thread changed it), its `subsdk0` one loop wider (`0x27804`, the movie library's thread loop, 6 → 23 instructions over the same back-edge and two more); Mario Kart 8 Deluxe's `main` 25 → 26 (a state setter like Breath of the Wild's, `main+0x715df8`) and its `subsdk0` one loop wider (the same movie library loop, `0x24118`). Of the 206 modules on the game drives (108 NSOs and Super Smash Bros. Ultimate's 98 NROs), 175 rewrite byte for byte as before (Hades, Super Smash Bros. Ultimate and Super Mario Odyssey among them) and 7 fail under both rewriters (every `rtld`, New Super Mario Bros. U Deluxe); the other 24 take new loops, all through the store rule alone (only Breath of the Wild's job queue has a lock): Monster Hunter Rise's `main` 90 → 166 (72 of them one inlined countdown, `ldr w9,[x21,#0x24]; subs w9,w9,#1; b.ge; str w22,[x21,#0x24]; ldr; cbz`, whose reset makes the next pass leave), BALL x PIT 87 → 94, Animal Crossing: New Horizons 162 → 166, Balatro 9 → 12, Octopath Traveler 0 75 → 78, Pikmin 4 80 → 83, Bayonetta 3 35 → 37, and one more each in Big Brain Academy, Metroid Dread, Pokémon Legends: Z-A, Stardew Valley, Super Mario Bros. Wonder (both builds) and Tears of the Kingdom; the movie library loop is wider in the `subsdk0` of Animal Crossing, BALL x PIT, Monster Train 2 and Octopath Traveler 0 too, and one loop is wider in the `main` of AI: THE SOMNIUM FILES and of Advance Wars 1+2. No loop of the old search is lost or loses a back-edge. Each distinct shape was read in its disassembly: one-pass resets and state setters (the gate is reached only when another thread changes the word), parser states that leave on the next pass (Octopath Traveler 0 `main+0x4ebfa54`, Balatro `main+0x1f4600`), and real flag waits (Big Brain Academy `main+0x1734e0`, the movie library). Monster Hunter Rise, the module with the most new loops, runs alike on the Mac with both rewrites from a fresh save (180 s headless from the boot each, the same host: 60 fps, frame p95 about 17.5 ms, the same red lines, no hang). Searching paths through stores costs time: Breath of the Wild's `main` takes 26 s of CPU to rewrite instead of 19, all 206 modules about 4,100 s instead of 3,250.
|
||||
|
||||
Results, on the Rogueport route on the Mac, side by side with the committed build under the same load: frame p95 29–37 ms → 18–22 ms, CPU 93–129% → 38–57% of a core, the workers 576–885 → 65–94 ms/s, CPU energy 22–32 → 11–17 mJ per frame. Loops found: TTYD 17 in `main`, 5 in `sdk`, 8 in `subsdk0`; Wonder's `main` 175, Breath of the Wild's 308, Tears of the Kingdom's 843. In Tears of the Kingdom, 701 of them are one inlined atomic decrement (a compare-and-swap retried after another thread changed the count), which yields only on a second retry in a row. The x18 scan now decodes in blocks without details, so the rewrite takes less time than before even with the search: TTYD `main` 20.5 → 11.7 s of CPU, Tears of the Kingdom's 101.5 → 64.2 s, Monster Hunter Rise's 507 → 342 s.
|
||||
- **Waits block instead of polling:** the in-process audio renderer, upload batches, `nvnQueueFinish`. A signalled event wakes only the threads waiting on it; leaving an SVC takes the global lock only while a pause (IL2CPP's garbage collector) is pending.
|
||||
- **Real-time policy.** The thread that acquires the window texture declares a Mach time-constraint policy, which tells the performance controller the frame deadline. `DM_RT=0` leaves it on its QoS, for measuring.
|
||||
@@ -244,6 +247,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- Real time lost to a late frame or a hitch is paid back one vsync per frame (at most 15; the rest of a long load is dropped), so game time = real time. The log line `frame step: game time 30.00 s in 30.00 s (1.000 of real time)` checks it every 1,800 presents, with the frames held 1 / 2 / 3 vsyncs, late frames, paid-back vsyncs and hold changes.
|
||||
- Scenes that fix the game's step themselves (the VFR's override: 2 at `main+0xc3b588` / `+0xc5ebf8`, 1 at `main+0xc609d8`) keep it, and the host holds frames that long (log line `the game fixes its own step`; not reached on the Mac yet).
|
||||
- Mac (M4 Pro), from the title into the save's scene: 1.000 of real time at 60; with 12 ms added to every frame (`DM_TEST_PRESENT_LAG=1,12`) it drops to 30 and stays at 1.000. `DM_FPS_TARGET=30` holds 30 (saves power).
|
||||
- **CPU hitches (Mac):** in steady play, frames with 30–210 ms of CPU work came every 1–10 s (Zora's Domain, M2 Max), the main thread and the two workers spinning on a job queue's lock that the main thread could not get. Since the spin wait rule takes that loop they are gone ([Power](#power), Spin loops).
|
||||
- **The game's own interval controller** (at `DM_FPS_TARGET=0`, and the interval the framework keeps while the VFR override holds): patches to it lift its floor from 2 to 1. The game's time step follows its frame interval, so the speed is right at a steady 60, 30 or 20. The game picks the interval from the frame's CPU time and its GPU time, which it reads from two GPU timestamps around the frame; the host gives each timestamp the GPU time of the work around it (profile key `gpu_timestamps`, which its dynamic resolution needs too). Before that it saw only the main thread's CPU time (about 3 ms a frame in the first shrine) and stayed at 60 in slow motion: at 40 fps, about two thirds of the normal speed (an iPhone 16 Pro Max in handheld mode, GitHub issue #9; [04](agents/04-graphics.md)). Under load it still ran at 0.93–0.98 of real time.
|
||||
- **Outdoors on the Mac (M4 Pro, the Great Plateau):** about 165 render passes a frame instead of 386 and GPU p50 about 6.5 ms instead of 9.3 since a partly updated uniform block no longer ends the pass (the game updates 64 bytes of a 64 KB block before each draw group), and the G-buffer clears ride on its first pass ([04](agents/04-graphics.md)). The frame still loads and stores about 800 MB of attachments, most of it where compute dispatches of the GPU particles split passes.
|
||||
- **Render scale:** `DM_RENDER_SCALE=0.5`–`1.0` (also from `Documents/env.txt`) fixes the 3D views at that fraction of 1600×900 docked (1280×720 handheld) and turns the game's own dynamic resolution off; 1.0, the default, keeps it. A stub sets the scale where the game applies its own (`main+0x1266880`), so every scene pass, the shader's resolution ratio and the LOD check follow it: at 0.5 the 1600×900 passes become 800×450 and the 800×450, 400×225, 200×112 and 100×56 chains halve too, while the shadow maps and the 1920×1080 output keep their size and the HUD stays sharp. It holds in handheld mode too (960×540 at 0.75). On the Mac (M4 Pro) the GPU time does not change (about 13 ms in the first shrine at 1.0 and at 0.5): a frame has about 650 passes, most of them tiny, whose fixed cost outweighs their pixels there. Whether it helps an iPhone, whose GPU loads and stores each pass's attachments, is what the setting is for.
|
||||
|
||||
Reference in new issue
Block a user