Files
b87a0cc6aacb4f60c661c1bced067cea4f1d0e3e
T
seedds
b87a0cc6aa
Upload batches flushed outside a submission are handed to the encode thread at once (general, from PR #141). With the encode thread on (the iOS default, DM_NVN_ENC_THREAD), an upload batch's commit is a record in the encode thread's current block (rec_commit_upload), which is handed over when the block fills or a submission ends. Outside a submission (no replay command buffer: a texture zeroed at creation, tex_zero_fill, or a packaged texture uploaded between submissions) up_flush meant to commit the batch on the spot, but the record stayed in the block until the next submission, and every wait for it waited for that submission: nvnQueueFinish (mtl_finish_gpu, on any thread) or up_begin on a busy slot. A game whose last submitting thread creates a texture and then calls nvnQueueFinish before it submits again waited for good. up_flush now hands the block over when it runs outside a submission (it runs on the submit thread only, up_owner_check). From PR #141 (LNPANDY, found analysing a Hades II save-load freeze on an iPhone 14 Pro): mtl_finish_gpu hands over the block before it waits when it runs on the submit thread, as up_begin does since 9cb7af3 and q_cb does; nvnQueueFinish cannot run inside a submission (one nvnQueueSubmitCommands call, which ends with rec_drain), so that line matters for the window above and for the diagnostics that wait mid-submission (td_dump, DM_NVN_T2T_TRACE). Checked: dmhost builds for Super Mario Bros. Wonder (NVN) and Hades (Vulkan), the host unit tests pass, the graphics run list passes, and with DM_NVN_ENC_THREAD=1 every test passes but t_rt_range, which stops on main alike (DmRecEnc has no stand-in for the copyFromTexture:...toBuffer:...options: form). Not checked: a game, the iPhone
Upload batches flushed outside a submission are handed to the encode thread at once (general, from PR #141). With the encode thread on (the iOS default, DM_NVN_ENC_THREAD), an upload batch's commit is a record in the encode thread's current block (rec_commit_upload), which is handed over when the block fills or a submission ends. Outside a submission (no replay command buffer: a texture zeroed at creation, tex_zero_fill, or a packaged texture uploaded between submissions) up_flush meant to commit the batch on the spot, but the record stayed in the block until the next submission, and every wait for it waited for that submission: nvnQueueFinish (mtl_finish_gpu, on any thread) or up_begin on a busy slot. A game whose last submitting thread creates a texture and then calls nvnQueueFinish before it submits again waited for good. up_flush now hands the block over when it runs outside a submission (it runs on the submit thread only, up_owner_check). From PR #141 (LNPANDY, found analysing a Hades II save-load freeze on an iPhone 14 Pro): mtl_finish_gpu hands over the block before it waits when it runs on the submit thread, as up_begin does since
9cb7af3 and q_cb does; nvnQueueFinish cannot run inside a submission (one nvnQueueSubmitCommands call, which ends with rec_drain), so that line matters for the window above and for the diagnostics that wait mid-submission (td_dump, DM_NVN_T2T_TRACE). Checked: dmhost builds for Super Mario Bros. Wonder (NVN) and Hades (Vulkan), the host unit tests pass, the graphics run list passes, and with DM_NVN_ENC_THREAD=1 every test passes but t_rt_range, which stops on main alike (DmRecEnc has no stand-in for the copyFromTexture:...toBuffer:...options: form). Not checked: a game, the iPhone
CI on GitHub's macOS runners (from PR #132): .github/workflows/tests.yml runs, for pull requests, pushes to main and manual dispatch, three jobs on macos-26 in parallel: the host unit tests and the graphics tests that need no GPU (t_dirty_slots, t_dirty_race, t_pace, t_fstep, t_hmap, t_cover, t_pso_skey, t_refl and t_rt_range_ro, the last two skipping their sample checks); every test_*.py of dm/tools/tests, dm/ios/tests and dm/gfx/tests on Python 3.14 with lz4, capstone and fonttools; and a build check: dmhost for Wonder (NVN), Hades (Vulkan) and Digimon Story Cyber Sleuth (GL), every graphics test compiled but not run, and the iOS host sources compiled for the iPhone SDK by a new helper, dm/ios/tools/compile_check.py (the host Makefile's file list and flags as gen_build.py takes them, the host's own command first so its header paths come before libvpx's; 133 files, no game files, signing or Xcode project). Read-only permissions, superseded runs cancelled, a timeout per job, logs uploaded only on failure, actions pinned to commit SHAs (checkout v7.0.1, setup-python v7.1.0, upload-artifact v7.0.2, each checked against its tag). The GPU tests stay local (the hosted runners are virtual Macs without Metal 4, issue #112), and so does profile_guess.py --check (it reads game extractions). test_xcode_workflow.py: the stand-in profile carries the registered profile's title_id, which report_profile_status reads since
ce17300 (the test had 3 errors on main). Checked by replaying the three jobs on this Mac on current main: all pass, the tool tests in a fresh Python 3.14 venv with only the three packages
Spin waits that take a spin lock (general, from PR #134, found on Breath of the Wild; fixes #129). In steady play on the Mac, Breath of the Wild had frames with 30-210 ms of CPU work every 1-10 s: the main thread and the two workers spinning in one job queue poll (main+0xef62dc). The first item of some of its jobs is for the thread of core 0; the others wait until it is taken, taking and releasing the queue's lock (an exclusive test-and-set, then a compare-and-clear) and writing a status word on every pass, so the spin wait search (a store or an exclusive store ends a path) did not take it. Polled at full speed by the two workers, the lock went back and forth between their cores, and the main thread, whose item it was, lost nearly every exclusive store: in a 40 s Time Profiler trace it never held the lock while each worker held it in about 750 samples. patch_nso.py "Spin waits": a lock taken and released on the same address on every path back to the head (spin_lock 'acq' / 'rel', counted like the exclusive adds), a test loop on a word inside the region ('tst', whose branch back goes to the original loop and its own gate), and up to four stores whose value and address registers the region never writes ('stc', only in a region that loads something). Such stores write fixed values to fixed addresses, so a loop can change its own state for a few hundred passes at most, well inside the gate's 2 ms of polling, before its passes repeat. A store of a value a pass changes ends its path and the search runs again without it (the path that takes an item stores the new cursor); when the region with stores does not cover the old one's back-edges, the old one is used. Through the gate the workers leave the lock a moment between passes and the main thread takes its item at once. Zora's Domain on an M2 Max at 120 Hz (one host, 42 s of steady play per run, three runs each, before -> after): standing still, frames with more than 30 ms of work 2.8 / 6.3 / 3.5 -> 0 per 30 s, the frame's work p99 20-21 -> 8.3-8.4 ms; turning the camera, CPU hitches 11.9 / 13.3 / 6.3 -> 0, p99 21-25 -> 9-15 ms; process CPU per frame unchanged. Every module on the game drives rewritten with the old and the new search (206: 108 NSOs and Super Smash Bros. Ultimate's 98 NROs): 175 byte for byte as before (Hades, Super Smash Bros. Ultimate, Super Mario Odyssey among them), 7 fail alike under both (every rtld, New Super Mario Bros. U Deluxe), and 24 take new or wider loops, all but Breath of the Wild's queue through the store rule alone: Monster Hunter Rise's main 90 -> 166 (72 of them one inlined countdown whose reset makes the next pass leave), BALL x PIT 87 -> 94, Animal Crossing: New Horizons 162 -> 166, Balatro 9 -> 12, Octopath Traveler 0 75 -> 78, Pikmin 4 80 -> 83, Bayonetta 3 35 -> 37, Breath of the Wild 308 -> 310, one more each in Big Brain Academy, Mario Kart 8 Deluxe, Metroid Dread, Pokemon Legends: Z-A, Stardew Valley, Super Mario Bros. Wonder (both builds) and Tears of the Kingdom; the movie library's thread loop is wider in the subsdk0 of Breath of the Wild, Mario Kart 8 Deluxe, Animal Crossing, BALL x PIT, Monster Train 2 and Octopath Traveler 0, and one loop is wider in the main of AI: THE SOMNIUM FILES and Advance Wars 1+2. No loop of the old search is lost or loses a back-edge. Each distinct shape was read in its disassembly: one-pass resets and state setters (the gate is reached only when another thread changes the word), parser states that leave on the next pass, and real flag waits. Rewriting all 206 takes about 4,100 s of CPU instead of 3,250. Diagnostics: DM_HITCH_DUMP=<ms>[:<s>] logs every guest thread (its wait, last SVC, where it is, x0-x2 and x19-x25 in guest code) when no NVN frame was presented for that long, and each guest thread names its pthread '<handle> <name>' (0x1008 Worker) before it enters guest code, the main thread '0x1001 main thread', so sample and Instruments' Time Profiler show which guest thread a sample belongs to. The graphics tests that link hle_nvn.c without the host define dm_present_ns (t_host_stubs.c weak, t_nvn.c). New test dm/tools/tests/test_spin_waits.py. Checked: the host builds, the host unit tests, the 46 tool tests and the gfx run list pass; Breath of the Wild into Zora's Domain and Mario Kart 8 Deluxe for 85 s from boot with the new rewrite (the PR's author), and Monster Hunter Rise from a fresh save for 180 s headless with each rewrite on one host (e72a065's, the chain of its Mac libraries): 60 fps, frame p95 about 17.5 ms, the same red lines, no hang. Not checked: Paper Mario: The Thousand-Year Door, DARK SOULS: REMASTERED, FINAL FANTASY TACTICS and FINAL FANTASY RESONANCE (no files on the drives), the iPhone. docs/development.md (Spin loops, Breath of the Wild) and docs/agents/05-debugging.md say so