Commit Graph
204 Commits
Author SHA1 Message Date
5632203274 Optimize CPU deswizzle copies and host symbol lookup (#261)
Hoist block-linear row and slice addressing out of segment copies, and
index HLE, forced-takeover and missing-import names while retaining
existing duplicate precedence and stub identity.

Add byte-address differential tests and direct collision, duplicate,
growth and reuse checks. Correct the ASTC-tail and loader-serialization
comments and document independent CPU microbenchmarks. Keep TotK
prewarm data in a separate change.

Validation: 5,334 texture checks; symbol-index and NRO lifecycle tests;
AddressSanitizer/UBSan; documentation checks; Mac host build and changed
iOS source compilation. No new gameplay validation.

Co-authored-by: draco-cong <71372970+draco-cong@users.noreply.github.com>
Co-authored-by: laobeibei <laobeibei@users.noreply.github.com>
2026-10-26 15:28:52 +08:00
f7345d9a00 Same-core handoffs wait only for the threads their wake made runnable (#247, #259 section 6)
A wake's handoff waited until every runnable higher-priority thread pinned to the caller's core was blocked, not only the ones the wake made runnable. The host runs a lower-priority thread beside such threads at all other times, and a core's busy loaders, workers and pollers are seldom all blocked at once, so the wait had no bound. Tears of the Kingdom 1.4.3 (issue #247) stayed on an empty scene after the boot logos: on 82ab25b, 84 functions / 771 NVN calls a frame, no packaged texture uploaded, 250,008 ms of handoff waits in a 180 s run; a temporary log showed BootupBackgroundWorkThread (priority 58) waiting 200-290 ms at a time for ARes Loading 1, Presentation Thread and BaseProcExecutor. Donkey Kong Country: Tropical Freeze's priority-58 job thread waited the same way for pollers (issue #259, section 6).

A wake now records the woken thread in the waker's current handoff and in those the waker belongs to (note_runnable, at most four), so Metroid Dread's main thread still waits for its movie dispatcher and for the worker the dispatcher wakes. Blocking, a sleep, YieldToAnyThread (SleepThread -2, which on Horizon takes the thread off its core so that lower-priority threads run there), exit, suspension and host GPU work end a thread's hold; a handoff ends when its caller's thread SVC returns (hand_epoch). There is still no timeout, and core_preempt keeps checking every runnable peer.

Checked on the Mac, without shader libraries for chain 8393d85a4eae (draws skipped): TotK on the issue #70 route for 180 s, 1,221 ms of handoff waits and the same per-frame NVN call counts and texture first uploads as 9feac2f (before the unbounded handoff). Dread, fresh saves, DM_KEYS=4:A*20/0.7, 60 s: a control host with the handoff off crashed in 1 of 3 runs at sdk+0x347eb4 from main+0x225338; the limited handoff passed 12 of 12 runs, each through three decoders and two teardowns with about 12,400 handoffs. t_preempt adds three handoff-scope cases: a busy peer nobody woke (main's code never returns), a worker woken by the woken dispatcher (fails without the recorded chain), a woken thread yielding to any thread (never returns without that rule). Host tests, docs tests and an iOS syntax check of hle_thread.c pass. Gameplay with shaders, iOS and DKC itself are not checked here.

Co-authored-by: HougeLangley <1161594+HougeLangley@users.noreply.github.com>
Co-authored-by: aurlight <120410928+aurlight@users.noreply.github.com>
2026-10-26 15:00:32 +08:00
seeddsandchengjie 2019ff866b Recognize framed pointer tail calls safely (#256)
Preserve the existing terminal-tail recognizer and accept a contiguous
framed method-pointer thunk followed by its separate null-error call.
Reject stores that overwrite saved FP/LR, unproven store addresses and
stack/frame/link-register clobbers, including WSP. Stores are limited to
fixed caller stack-argument slots above the saved frame.

Keep the fallback narrower than #188's proposed unframed block recognition.
Add synthetic AArch64 regressions and document the rule beside the other
tail-call findings; no game-specific IDs, fixed addresses or profile changes.

Validation: 179 tool tests run, 174 passed and 5 skipped; 10 tail-call tests
pass; the legacy terminal recognizer is AST-identical to main; git diff
--check passes. Real game modules were not rewritten for this integration.

Co-authored-by: chengjie <chengjie177@gmail.com>
2026-10-26 14:46:10 +08:00
seeddsandRAHUL REX 4acde72a6e Recover the frame step after movie and thermal caps (#253)
Keep the proven frame cost and GPU reference for each cap, and probe the
shorter hold when a movie or thermal cap ends. Otherwise a GPU-bound game
can fall straight back to 30 fps without a usable recovery reference.

Save an uninterrupted sustained run when a cap begins, even if no earlier
hitch populated cref. Keep the earlier reference through short runs and
update it after another sustained run. Count cap-release probes and give
them all 30 grace frames: the completed frame at the old cap consumes none.

Add movie and thermal recovery regressions, including first movies,
repeated caps, full grace and failed-probe backoff. Document the mechanism
and link it from both Zelda game notes.

Validated: 64/64 frame-step checks, including ASan/UBSan; six checks fail
against the original PR. Production thermal tests and ThreadSanitizer,
docs tests and diff checks pass; the BotW Mac host builds. Device clock
recovery remains modelled, without a new gameplay observation.

Co-authored-by: RAHUL REX <75500427+RahulRex0@users.noreply.github.com>
2026-10-26 14:22:03 +08:00
seeddsandRAHUL REX 82ab25b40b Reuse a present allocator only after its own present completes (#254)
present_now cycles three Metal 4 command allocators by ticket, and before resetting one it waited for g_pq_done to reach the ticket that last used it. g_pq_done is not GPU completion: every display-link callback sets it to ticket - 1 before presenting (the previous present was committed, not run), and a present that was not submitted sets it to its own ticket. So a display-link present never waited there: with three presents in flight, or two after a dropped frame, the allocator was reset and its command buffer and UV buffer encoded again under a present the GPU had not run. Each allocator now keeps the newest ticket its own commit feedback reported (g_present_alloc_done), and its reuse waits for that. The display-link callback can now block in present_now until the GPU has run that allocator's previous present; it still moves g_pq_done before it waits, so the game's swapchain acquire is unchanged. General: every game presents through this path, display-link mode is the default.

t_present_alloc holds the Metal 4 queue behind a shared event and calls the display-link callback by hand, each present with a drawable of its own offscreen layer: a dropped frame (tickets 1, 3, then 4 on allocator 1 again) and one frame per tick (5, 6, 7, then 8 on allocator 2 again). Checked on an Apple M6 at 6062518: without the production change 4 of 13 checks fail in 10 of 10 runs, the earlier drawable reads back 0 in all 64 pixels, and every run logs [RED] display command buffer error with kIOGPUCommandBufferCallbackErrorPageFault; with it 13 of 13 pass in 20 of 20 runs under MTL_DEBUG_LAYER=1 with no red line. The whole dm/gfx/tests run list passes, dmhost builds for Mario, docs tests pass. Not checked: a game run that hits it, the iPhone, the not-submitted path. Docs: the frame-rate reference entry (with the page-fault symptom) and the symptom index.

Co-Authored-By: RAHUL REX <75500427+RahulRex0@users.noreply.github.com>
2026-10-26 14:06:20 +08:00
seeddsandv0lumehi 60625182c4 Harden GPU fence wake and pass scheduling (#158, #213)
Integrate fast shared-event wakeups, per-segment publication, array-slice
pass merging, and independent compute dispatch hoisting from the PRs.
Serialize CPU completion effects before publishing completed fences and
validate recycled post records. Preserve submission order for early waits.

Guard scheduling against attachment aliases, first-draw and bindless
reads, indirect ranges, stencil writes, and explicit barriers. Add direct
Metal regression coverage and a gated CPU completion race test with TSAN.

Record final BotW and Metroid Dread gameplay validation with optimizations
on and off. BotW retains fewer render passes, but the measured 60 Hz M4
Pro runs do not establish a timing speedup. Keep the original higher-load
and iOS performance claims explicitly unverified.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 13:33:02 +08:00
seeddsandv0lumehi fb20bce70e Add GPU diagnostics and close periodic profiles per frame (#225)
Add periodic encoder profiles with attachment traffic and formats, alternating A/B measurement phases, bounded Metal trace capture, and gpuprof.py log summaries. Preserve main's offline pipeline-archive integration while adding the capture build setting.

Close periodic samples at frame boundaries through the last committed shared-event fence, so every:1 reports without mixing frames or reusing an unresolved heap. Add 28 CPU checks of production profiling code and include them in CI; document the mechanism and update the symptom index.

Validated with all 31 CI-style Python test files, the profiling regression, a Mac host build, and all 133 iOS source compiles. Profiling completion and heap ownership were checked with synthetic counter/event objects; no new real-GPU or gameplay validation.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 12:09:07 +08:00
seeddsandAmost d50ecc1f84 Support masked-predecessor jump tables and external patches (#237)
Recognize offset-table dispatches whose predecessor masks the index while
the dispatch checks a masked copy of the same source. Reject calls and
resolved table paths that bypass masking, undecodable instruction gaps or
targets, and writes that invalidate the table base. Keep the existing
recognizers ahead of this fallback.

Load build-gated patches from DM_GAMES_DIR and include them in rewrite reuse
keys. Preserve repository matching rules for derived profiles, select the
first external profile for provisional matching rules, and deduplicate
patch paths while retaining conflicts between distinct build claims.

Validation on current main: Python tool tests (156, 14 skipped), iOS tests
(6), graphics tests (44, 2 skipped), documentation index and diff checks.
Regression cases fail on the original implementation and pass with these
guards; both supported base-loading forms remain accepted. Real game
modules and gameplay were not revalidated.

Co-authored-by: Amost <6445689+Amost-@users.noreply.github.com>
2026-10-26 11:03:47 +08:00
seeddsandv0lumehi 92fc2f2711 Build and ship offline Metal pipeline archives (#226)
Compile eligible shader-pair bases on the Mac with metal-tt, package one
archive per selected iPhone GPU, and use a matching bundle before falling
back to on-device preparation. Carry the profile depth mode through packing
and include the pulling libraries required by the bundled bases.

Integrate PR #226 with four confirmed fixes: validate archive container
bounds before invoking Metal; require at least one actual successful bundle
probe; hash consumed library bytes into offline cache keys; and emit device
remainder lists for constant-only plans instead of aborting packaging.
Keep game measurements in the game note and document the general findings.

Validation on current main: 52 real Metal bundle checks, 57 archive checks,
79 prewarm checks on each pipeline path, 73 Python regressions covering
offline tools, packaging, documentation and chain checks, and compilation
of all 133 iOS source files. No new iPhone gameplay validation. The archive guard validates
container structure, not private Metal payloads.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 10:52:50 +08:00
seeddsandFernando Messias 62a640b037 Fix custom key loading for compressed game inputs (#204)
nsz 4.6.1 rejects --keys and loads keys.txt beside argv[0] during import.
Run the converter in its temporary output folder with the selected key
file linked there before import, using absolute input and output paths.
Remove the temporary link on both successful and failed conversions.

Integrate PR #204 with XCZ and relative-path regressions, rename the test
flag that shadowed unittest.TestCase.fail, and qualify the documentation
for the verified nsz release. Record the packing finding and provenance.

Validation: reproduced the exact parser failure using real nsz 4.6.1;
a synthetic NSZ conversion verifies selected-key lookup and link cleanup;
52 NSP tests, four documentation tests and git diff --check pass.
The synthetic payload is unencrypted and uses dummy keys. Real-game
decryption, actual XCZ conversion and a full IPA build remain unverified.

Co-authored-by: Fernando Messias <fernandomessias95@gmail.com>
2026-10-26 10:36:23 +08:00
seeddsandv0lumehi c4b74a62f9 Add thermal frame-rate cap with synchronized recovery (#220)
Hold the host frame step at 30 fps while iOS reports serious or critical
thermal pressure, and restore its configured cap after 30 calm seconds.
Keep the existing movie cap and DM_THERMAL_CAP=0 override.

Sample thermal state independently of movies and configured 30 fps targets
so heating resets recovery and cooling counts during playback. Serialize
the sample, cached cap and recovery timer across acquire/present readers.
The original PR failed five timing checks and ThreadSanitizer detected a
race through concurrent production cap getters.

Add controlled-clock production-path regressions and CI coverage, including
disabled/inactive caps and ThreadSanitizer. Record the shared mechanism
and validation limits in the frame-rate reference and BotW note.

Validated on latest main with the BotW profile: 24 thermal checks normally
and under ThreadSanitizer, 51 frame-step checks, 154 NVN checks, the
present-snapshot suite, 21 tool/docs checks, Mac host build, and iOS compile
check (133 sources). No new device gameplay or thermal recovery run.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 10:13:42 +08:00
seedds c76f679834 Remove alpha_from_red: Super Mario Odyssey's layout captures need only the nvnTextureCompare fix. The game creates each capture with its per-format default mapping (an R8 one reads (R, 0, 0, 1)), rebuilds it on the same storage with the capture's own, (1, 1, 1, R), and replaces the registered descriptor only on a difference (main+0x982320); while the host's comparison ignored the mapping the captures kept alpha 1 (white coin cards, a dark box behind hints), which the 26-shader alpha_from_red list papered over. Checked on the Mac from the "Look Around" hint with a probe host on the comparison (a c3a0880^ host for the profile's chain dec5ee141a9f libraries): 40 mapping-only rebuilds in 11,700 frames (R8 captures 640x80 down to 48x57 to (1, 1, 1, R), a 320x180 RGBA one to (R, G, B, A)); ignoring the mapping again brings back the box (about 12,800 pixels from frame 4,200), the old list hid it by converting the 378x80 capture, and with the fix the list converted nothing and changed no pixel of frames 4,200-6,600. The comparison and the views have no platform branch, so the iPhone gets the same rebuilt mapping (not run on a device). With no profile left using it, the key, its host code (tex_alpha_red, afr_want, DM_NVN_ALPHA_RED) and its docs go; treg_drop_ar becomes treg_drop_im (the image-slot view it also drops stays). Odyssey's note has the A/B runs. Checked: the host builds for Odyssey, the iOS compile check (133 files), the tool tests (111, 14 skipped) 2026-10-26 10:01:20 +08:00
seeddsandparsifal295 2db12f7efd Fix vibration handles before controller haptics routing (#246)
GetVibrationDeviceHandles used routed haptics availability to decide whether
handles existed. Games that cache this answer at startup could retain an empty
answer after a controller connected. Return handles for a connected npad style
when rumble is enabled and an output sink is installed; check routed outputs
when values are sent, keeping the existing permission and lifecycle behavior.

Include the PR's rumble diagnostics, count unroutable values individually in
mixed batches, and protect aggregate counters across game threads. Correct the
code comment to describe Unicorn Overlord's observed empty handles and zero
values without claiming restored physical rumble.

Add regressions for cached startup handles delivering a nonzero value after
routing, diagnostic totals and concurrent sends. Include HID in the existing
ThreadSanitizer target and record the mechanism in the service reference.

Checked with GAME=010069401ADB8000: full host suite, 73 HID checks, audio and HID
ThreadSanitizer checks, docs checks, Mac host build, and iOS compilation
(133 files). Physical rumble and iOS runtime remain untested. Broader tool
validation is incomplete because Capstone is missing and iconutil failed.

Co-authored-by: parsifal295 <130349765+parsifal295@users.noreply.github.com>
2026-10-26 09:54:43 +08:00
seedds 16061e0399 Rule: fix a bug at its root cause, not around it. AGENTS.md says to find why a bug happens (the faulting instruction and its decompilation, the bad value and what wrote it, the semantics the host or translator gets wrong) and fix it there; a change that only hides the symptom (stub, skipped call, clamp, retry, sleep or timeout, cache flush, profile key that turns a feature off) is a workaround, used only when the cause lies where it cannot be fixed (the game's own code, changed deliberately by a rewriter cut, or a platform limit) or the user asks for a stopgap, and then recorded as open with the cause or what is unknown. docs/agents/README.md has the same as a ground rule. Checked: dm/tools/tests/test_docs.py passes (AGENTS.md 1,419 words) 2026-10-26 09:42:57 +08:00
seeddsandv0lumehi c13ddcf531 Add opt-in MetalFX spatial upscaling and a single-screen launcher (#215)
Scale the valid presented image with MetalFX's spatial scaler on the shared
Metal 4 path, preserving crop, letterboxing, origin and gamma-encoded colours.
Keep Standard as the default and fall back when scaling is unsupported.

Use the same path for asynchronous whole-window captures and diagnostic
screenshots, retaining resources until GPU completion. Bound scaler recreation
during resizing and report presentation GPU time separately by scaling mode.

Put all launcher settings on one screen with Game, Controls and Performance
sections, inline menus and sliders, and a reset to tested defaults. Extract the
settings model for native persistence tests, validate saved ranges, share HUD
setup, and document the measured costs without claiming an iPhone baseline.

Validated with Metal presentation, NVN and snapshot regressions; tool and iOS
tests; the Mac host; and an unsigned iOS smoke build. No new device gameplay run.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 09:38:05 +08:00
seedds 3dd5d25a51 Add Echoes of Wisdom 1.1.0 port and correct cube fallback diagnostic 2026-10-26 09:14:43 +08:00
seeddsandv0lumehi 577df01c39 Fix iOS pulling shader packaging for profile defaults (#211)
Include pulling vertex libraries when the compiled game profile enables the
Metal 4 path, preserving explicit IOS_PULL overrides. Keep the pack options
array nonempty so inclusion also works with macOS Bash 3.2 and set -u.

Exercise the actual archive step with tiny shader fixtures for Breath of
the Wild and Wonder, checking both defaults and explicit overrides. Record
the packaging failures and fixes in the rendering reference and game note.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 08:56:50 +08:00
seeddsandAleksandr Svidchenkov 1a93086e2d Fix service-event and vsync worker lifetimes (#202)
Apply PR #202's host-thread skip and duplicate vsync suppression. Give
bindings unique lifetime IDs so a retired worker cannot signal another
service's event after guest storage and kernel handles are reused. Replace
the eight-entry worker table with allocated records so rapid replacement
does not abort while retired workers wait for their next tick.

Add deterministic regression cases through real event binding and SVCs,
and document the mechanism with an updated symptom index. Both the
capacity abort and stale signal reproduce against the original PR.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:35:19 +08:00
seedds 02e40ba753 Record threading audit and retain verified game workarounds 2026-10-26 08:22:50 +08:00
seedds c188131824 Honor same-core priority handoff when starting threads 2026-10-26 08:22:50 +08:00
seeddsandAleksandr Svidchenkov 4896c78d8a Fix packaged texture dimensions and linear storage sizing (#201)
Decode packaged byte 0 as separate block-height and block-depth nibbles,
including explicit zero dimensions, and size linear compressed textures
by pixel rows. Port the driver-derived rules from PR #201 and preserve
its attribution to digitalhivehub's PR #11 measurements.

Integrate the layout test with current main, correct its BC4 label, add
independently addressed CPU markers and packaged GPU mip coverage, and
record the mechanisms in the reference docs and symptom index.

Validation: 9 driver shapes, 189 CPU texture checks (also ASan/UBSan),
17 GPU deswizzle cases, 154 NVN checks, documentation checks, Mac host
build, and iOS compilation of 133 sources pass. No new gameplay or
iPhone runtime validation.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:19:08 +08:00
seeddsandAleksandr Svidchenkov c1b7eff61f Fix Metal buffer alias visibility and sparse copy ordering (#200)
Carry ResourceAlias visibility on copy encoders containing buffer copies,
inline updates or fills, using the contribution from PR #200. Operations
through sparse virtual mappings also need ordering against surrounding
operations: disjoint guest addresses can name the same physical bytes,
so guest-range dependency detection alone misses their hazards.

Add a regression using distinct no-copy Metal buffers and real sparse
mappings. The original host failed 37 of 101 checks on an M4 Pro and
PR #200 alone left 24 failures; the complete fix passes all 101 checks
with both direct and deferred encoding under Metal validation. Ordinary
independent buffer operations retain range-based batching.

Validation: t_nvn passes 154 checks; the Mac host builds; all 133 iOS
source compile checks and the documentation checks pass. Monster Hunter
Rise gameplay, iPhone execution and sparse-barrier performance were not
retested.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:28:09 +08:00
seedds 31b5944eb4 Port NieR Automata with 60 fps and geometry fixes 2026-10-26 07:13:22 +08:00
seeddsandAleksandr Svidchenkov a5513d4ae0 Fix Depth16 load-clears to match fragment depth writes (#199)
Apple GPUs truncate Depth16Unorm load-clears while rounding fragment depth,
so clearing to 0.5 stores 32767 instead of the fragment's 32768 and fails
an EQUAL comparison. Apply the format-specific correction to both
attachment and whole-texture clears, preserving other depth formats.

Exercise both production command paths against independent GPU fragment
writes for 1,042 float depths, including half-bin boundaries and adjacent
floats. Removing either correction produces 518 mismatches on M4 Pro;
with both corrections enabled, neither path differs. Record the mechanism
and the pending TOTK highlight and iPhone validation in the documentation.

Validated on M4 Pro with Metal validation: t_depth16_clear, t_clear_ds
(43 checks), t_clear_tex (49 checks), and the documentation tests pass.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:06:32 +08:00
seedds 9a56ecf0c1 Order same-core wake handoffs and remove Dread movie gates
Use guest runnable state after a higher-priority same-core wake instead of Mach dispatch state or a 5 ms cutoff. Honour wake handoffs for every game and remove Metroid Dread request-ring gates and its core_preempt setting.

Validate event, mutex and condition-variable handoffs with delayed host dispatch; 13 valid Dread movie-transition runs passed. Host tests, docs and shader-chain checks passed, along with an iOS syntax check. Record Wonder boot comparison and pending iOS runtime validation.
2026-10-26 02:27:03 +08:00
seeddsandAleksandr Svidchenkov 9feac2f1e7 Rescue stalled display links with bounded retries (#198)
Switch active-app presents to drawable acquisition after 500 ms of display-link
silence, then retry with exponential backoff capped at 60 seconds. Preserve the
requested pacing mode, synchronize rescue state, and prevent focus timestamp
underflow from bypassing the foreground grace period.

Add CPU regression coverage in CI, a real Metal layer ownership check, and the
mechanism reference with upstream provenance and validation limits.

Validated: 43 CPU rescue checks; 47 checks with real Metal drawable acquisition;
34 present/crop and 10 pacing checks; Wonder Mac host build; all 133 iOS source
compile checks; documentation tests. Windowed gameplay recovery and iPhone
validation remain outstanding.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 02:25:06 +08:00
seeddsandAleksandr Svidchenkov 5b0c11a4f3 Fix NVN dirty-page upload races (#197)
Take upload stamps before protection, initialize cached checks from those
stamps, and wait out pages another thread is unprotecting before retrying.

Publish page generations before advancing the clock again in both the fault
handler and batched host pre-write path. A shared stamp_page helper closes
the remaining batched publication race in the original PR.

Add deterministic publication regressions for both paths and document the
mechanism and the limits of the game-level evidence. The regressions fail
on main, and the batched case also fails with the original PR applied.

Validation: full graphics suite, Wonder Mac host build, all 133 iOS source
compile checks, and documentation tests pass. No fresh gameplay or iPhone
runtime validation.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 02:06:27 +08:00
seeddsandAleksandr Svidchenkov 1fa521958a Merge PR #196: deduplicate normalized NVN pipeline keys
Drop unused normal-pass colour-target blend and mask state from pipeline keys.
Normalize historical prewarm rows and deduplicate their parsed keys before
compilation, preserving merged passes' logical target state.

Include production-path GPU regression coverage and the reference/game notes.
Validated on the latest main: Wonder host build, key serialization tests,
48 pending-pipeline checks, 48 archive checks, 158 prewarm GPU checks across
MTLDevice and Metal 4, and documentation checks. The regression fails on the
original base and detects wrong pixels if the merged-pass guard is removed.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:54:19 +08:00
seeddsandAleksandr Svidchenkov 796d592aa8 Test and document NVN prewarm key normalization
Exercise historical manifest rows through the production warmer, shader loader,
pipeline builders and draw-time cache. Check normal and merged passes on both
MTLDevice and Metal 4, with GPU pixels compared against the original descriptors.
The regression fails on the PR base and when the merged-pass guard is removed.

Document the shared normalization rule and keep game measurements in their notes.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:52:46 +08:00
seedds a8e76bf601 Port Portal: Source engine NROs on NVN, shaders from game.zip
- profile 01007BB017812000 (1.0.3, SDK 13.3.0): plays on the Mac at 60 fps
- provisional profiles add the subsdks the NROs import (profile_guess nro_modules)
- an NRO importing nvnBootstrapLoader makes an NVN game; NROs calling glslcCompile get the run-time GLSLC capture
- shader_split reads zip archive members (Portal's nxcontent/nvsb GLSLC outputs)
- ro::LookupModuleSymbol for a name a loaded module lacks is not red (Horizon fails it too)
- AGENTS.md: a Mac gameplay test is enough to count a game as playable (65)
2026-10-26 01:28:35 +08:00
seeddsandAleksandr Svidchenkov 6d4202b562 Merge PR #195: fix NVN autorelease ownership and registry contention
Keep the latest upload-staging and compact-binding fixes from main. Add a real-Metal readback lifetime regression and reference documentation for all three general fixes.

Validation: nine Metal test runs pass, including direct and deferred upload-budget tests; Wonder host build, 133-file iOS compile check, and documentation checks pass.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:27:04 +08:00
seedds 2d344f050c Test NVN readback lifetime and document general Metal fixes 2026-10-26 01:25:40 +08:00
seedds caee62a853 Compact NVN buffer bindings and remove Wonder heap workaround 2026-10-26 01:21:10 +08:00
seedds 7dc250e2f1 Bound NVN upload staging and retire it on upload completion
Split GPU upload batches at 128 MiB and retire overflow and CPU staging when the upload completes, rather than retaining it until the main rendering fence. Scope weak-reference diagnostics to avoid temporary staging retention.

Add a real-Metal regression covering memory bounds, upload pixels and an open render pass in direct and deferred encoding modes. Both modes pass 261 checks; documentation checks and the TotK host build pass.

Refs #147. Reporter-device validation and the full-device reboot investigation remain open.
2026-10-26 01:05:44 +08:00
seedds be5b197640 Port Persona 5 Royal with Needle shaders and NVN texture fixes 2026-10-25 23:49:24 +08:00
seeddsandb4f8abbb24dbf56a7d0280f296e16eeb c3a0880304 Guard Metal image atomics and reject unsupported bindings
Apply the generalized image-atomic bounds and backing checks from PR #178.
Also reject unsupported descriptor formats before integer fallback and fail
unresolved bindless atomics instead of silently deleting them. Rebuild the
translator and record chain 467efdfd0b62, with regression tests and findings.

Validation: compiler/helper and sanitizer tests, 96 Metal compile/link checks,
ten GPU probes, all 28,603 Z-A shaders for Mac and iPhone, and a 240-second
Mac station-area run reaching 6,987 frames without a freeze or GPU fault.
Reported iPhone cases and the station-to-world transition remain unverified.

Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
2026-10-25 23:37:28 +08:00
seedds b5c78fbddb Match verified game patches across compatible updates 2026-10-25 23:21:17 +08:00
seedds ec4ea16da0 Enable ASTRAL CHAIN 60 fps with corrected game timing 2026-10-25 20:59:04 +08:00
seedds 1159ccc675 Add ASTRAL CHAIN profile and fix PlatinumGames startup compatibility
Support PKZ v1 archives and advertise the translator's GPU-code range independently of SDK version. Register the 1.0.1 profile, shader tables, regression checks and Mac bring-up findings. Retain 30 fps; document the unshipped 60 fps experiment and outstanding iPhone validation.
2026-10-25 19:36:05 +08:00
seeddsandv0lumehi a1886130a6 Add opt-in frame buffering with safe snapshot pacing and suspend
Integrate the present snapshots and frame-step work from #87 together with
the encode-thread support and wait ledger from #157. Keep snapshots off
by default on both platforms pending wider device validation.

Offer Frame buffering in the iOS launcher for NVN builds. The existing
launcher remembers the choice and preserves env.txt overrides. Pass the
packer's detected graphics API to the settings generator.

Wait for snapshot capacity at acquire; a present reaching a busy slot
falls back to its original texture ticket. Drain snapshot commits and GPU
completion during suspend even when display tickets were discarded.
Regression tests cover both waits, fallback tickets and copied pixels.

Validated on M4 Pro: full gfx run list, 38 snapshot lifecycle checks,
219 checks with encoding and snapshots enabled, Mac host build, 133-file
iOS compile check, and eight launcher/docs/packing tests. The host unit
suite cannot link t_heap_size because dm_ue_heap_owns is missing, reproduced
on unchanged main cbbb600. No iPhone runtime performance claim.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-25 18:39:57 +08:00
seedds 7d368c4249 Ghost Trick: Phantom Detective (010029B018432000, base game 1.0.0 = v0, Capcom's RE Engine on SDK 13.3.0, with the Bonus Content add-on): game profile, and finalized pool buffers no longer kept by a guest thread's autorelease pool (every game). Plays on the iPhone (IPA 3.0 GB, user)
The profile is the NSP script's provisional one with tex_flush_upload and tex_dirty_upload: RE Engine writes its UI textures (ASTC) into CPU-visible pools in place, as Monster Hunter Rise does, and without both keys the language screen's background and buttons were magenta (with tex_flush_upload alone they still were). 7,744 shaders from the PAK's SDF definitions and main; 7,739 build. The other 5 are compute shaders with atomics on a 3D image, which SPIRV-Cross indexes with an int3, as Octopath Traveler 0's 16; none was requested on the routes below. PR #138 (Nishikori-Yui) built them by flattening the coordinate in dmsh_build.py, a chain change that would rebuild every game's libraries, so it is left out until a run requests one (the game note). PR #138's profile is the same as this one (the same keys and shader tables).

Host fix: the memory line's "freed but alive" count only grew in most games (Animal Crossing: New Horizons 3,113 after 27,900 frames, Pokémon Sword 2,050, Super Smash Bros. Ultimate 1,279; Ghost Trick 125 to 146 buffers, 6.6 to 8.7 GB of zero-copy buffers, so GPU allocated read 9.9 to 11.7 GB against 2.2 GB resident). A temporary census of every host table that can hold a texture over a pool buffer found no holder: mtl_pool_del runs on the guest thread that finalizes the pool, whose implicit autorelease pool never drains, and adding the buffer to the weak table g_dead_pools left it autoreleased there. containsObject: / removeObject: on a weak NSHashTable do the same (a standalone test), and res_add / res_remove call them on guest threads too. mtl_pool_del, res_add_at and res_remove now have their own @autoreleasepool. Ghost Trick: 0 freed but alive, GPU allocated 1.7 GB instead of 8.3 GB 300 frames into the boot and 2.9 GB in Chapter 1, resident memory, frame times and pictures unchanged; Pokémon Sword: 8 to 0 at the title after 90 s, the same frame pacing.

On the Mac: the opening movies, the language screen, the first-boot notices, save creation, the Bonus Content notices and Chapter 1's opening in the junkyard (the Ghost World, the first trick), and a second boot that loads the save, at 60 fps (frame interval p99 about 17 to 19 ms), about 2.3 GB resident, no red line; the Wwise sound plays (a muted run with DM_AUDIO_DUMP). Checked: the tool tests (57, the docs check among them), t_nvn (124 checks), a boot of the rebased host. README (games gallery, a Mac picture of the junkyard), AGENTS.md's count (sixty-two: the gallery's cells), the game note, the games table, docs/reference/nvn-objects.md and the symptom index say so
2026-10-25 17:44:41 +08:00
seedds 16f4889192 OCTOPATH TRAVELER II (0100A3501946E000, update 1.1.1 = v131072, Unreal Engine 4.27 on SDK 12.3.2): game profile, inline Unreal shaders in version 11 paks, and five general host fixes. Plays on the iPhone (IPA 5.9 GB, user; that IPA predates the geometry and allocator changes below). On the Mac: the boot, the title, a new game, the traveler selection, Hikari's opening into his first battle (won) and the scene after it, and Castti's opening (the flashback, the ship, free play on the deck), at 60 fps (frame time p50 16.7 ms, p95 about 17 ms) with CRIWARE sound, about 2.8 GB resident. Shaders: the project ships no shader code archives (bShareMaterialShaderCode off), so each material's shaders are inline in its .uexp and the global ones in Engine/GlobalShaderCache-SF_GLSL_SWITCH.bin, each an entry of FShaderMapResourceCode (an LZ4 block, its raw size, its frequency): uepak.py inline_shaders sends a version 11 pak with cooked packages and no .ushaderbytecode to shader_split.py's inline reader (ue_legacy_jobs; Zlib and LZ4 pak blocks decoded in Python, Pak.read), ue_inline_shaders finds each LZ4 block by its first literals (LSLG), and profile_guess.py / the NSP script tell the two kinds apart; 13,980 shaders in 13 s. Profile: Unreal's FMallocBinned3 cut with an OBS_UE_MALLOC gate as DRAGON QUEST VII Reimagined's (GCreateMalloc main+0xbecbf0, a 20-slot vtable, hle_uemalloc.c), a 2 GB engine region (game_args) with the node-area cut on this SDK 12.3.2 build (sdk_patches.py, sdk+0x346190 / +0x34622c), fps_target 60 through t.MaxFPS / rhi.SyncInterval (hle_fpslock.c; the same route at 30 and 60 fps reaches the same scenes at the same times), pad_layout full (the battle menu picks weapons with the D-pad), a prewarm list of 843 lines. General fixes: (1) a shader may bind 96 resources instead of 64 (NVN_REFL_MAXB: the traveler selection's 3D map shader binds 66, and the map was black); (2) the parallel precompile look-ahead (pso_predict_rest) restores the scissor, which state_op keeps in R, not S: with pipelines built on demand the picture went bright green and white from Castti's flashback and Hikari's battlefield on (bisected with simultaneous runs; DM_NVN_PREDICT=0 turns the look-ahead off); (3) a geometry draw above the 256 MiB capture scratch bound is drawn in runs instead of skipped (nvn_geometry.inc geo_split: one instance at a time, then runs of at most 4,096 list primitives whose index range fits 64 MB, one read of the indices, one GPU wait), the index compaction after the geometry kernel runs as count / offset / write dispatches instead of one thread (each frame drawing Unreal's 129,032-triangle point light shadow took about 300 ms with it), and scratch arenas above 32 MB are released after 600 idle submissions (arena_new takes the smallest that fits); (4) MapMemory takes a thread stack from the host's Unreal heap (dm_ue_heap_owns: a range outside the guest block the host maps read-write, vm_region_64): with cae5ec2's stricter source check every Unreal game on the host heap aborted in MapAliasStack at nnMain (checked: this game and Pikmin 4 boot); (5) the host Makefile rebuilds on a change to dm/gfx/nvn/*.inc. Scripted input takes partly tilted sticks (L<x>/<y>): the traveler selection opens a card only while its quill cursor rests on a traveler, and full-tilt presses swept past them. Open: 2 of 87 Mac boots crashed at the title in the game's CRI cue sheet loader (main+0x1d8054, a record copied while one of its TArrays held count 1 and no data); the writer is not found yet (the game note). Checked: 57 tool tests, the gfx run list (t_geometry 95 checks) and the docs check pass; the NSP script packs the IPA from the NSPs (Mac and iOS). README (games gallery, a Mac picture of Hikari's first battle), the game note, the games table, docs/reference shaders / memory, docs/agents/04 and the symptom index say so 2026-10-25 15:43:48 +08:00
seedds 0df4dbe280 A short AGENTS.md: what the repository is (the native-execution model, the host and its three graphics layers, sixty games, each with its note in dm/games/<id>/README.md and a row in the games table), where to start for each kind of task (a crash, abort, hang, [RED] line or wrong picture: the symptom index, then the reference entry and its games; a game short of its frame loop: 03; rendering: 04 and the rendering, shaders and NVN objects references; one game or its profile: its note; another title: the NSP script and docs/agents; packing, building, prewarm, power: development.md), and where a new finding goes, once (a fact about one game in its note; a general mechanism as an entry in its reference, the symptom index regenerated and a line in the note's Found on this game; this file only for a rule or the count). The rules are as before. The game list of 4,500 words, one sentence per game, is gone: each sentence is in its game's note. New dm/tools/docs_symptoms.py writes docs/agents/symptoms.md from the references' Symptom bullets (--check reports whether it is current). The NSP script's survey points at docs/reference/nvn-objects.md for SDK object sizes and at the symptom index; Balatro's note names capture_shaders. AGENTS.md goes from 6,235 to 1,137 words. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 on main still appears in the repository's Markdown (besides the five file patterns, nn::hid::GetGestureStates and nn::oe::SetCpuOverclockEnabled appear without the nn:: prefix in Taiko Risshiden V DX's note), every relative link and anchor resolves, and the tool tests and test_xcode_workflow pass 2026-10-25 12:56:14 +08:00
seedds 50ad443cb1 Bayonetta and Bayonetta 2 in the new layout: their game notes (dm/games/010076F0049A2000/README.md, dm/games/01007960049A0000/README.md) and rows in the games table, from their Game notes entries in docs/development.md, their AGENTS.md sentences, their profiles' comments and their commits; the SDK 3.6.0 entry of 03's catalogue becomes services.md "SDK 3.6.0 answered as SDK 4", and 04's two Seen-in sections become shaders.md "BNSH files in PlatinumGames DAT archives inside CRI CPK archives" (cpk.py, cpk_jobs) and rendering.md "Clears of attachments larger than the render pass" (rp_begin_ex_) and "Downsample of a single-sample texture" (copy_t2t), each with the games it was found on. Mario Kart 8 Deluxe's note no longer calls SDK 4.4.0 the oldest. Checked: every profile with a game.tsv has a note and a table row (60), every relative link and anchor in the repository's Markdown resolves, and every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 on main (4,669 items) still appears (the same five file patterns as before) 2026-10-25 12:44:03 +08:00
seedds 3a993fa3fe Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in <game>" subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games/<id>/README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv) 2026-10-25 12:31:58 +08:00
seedds 846c54c63c Per-game notes beside each profile: dm/games/<id>/README.md for all 58 games, and the games table in dm/games/README.md. Each note says what the profile needs and why (a table of its game.tsv keys, patches.py cuts and frame-step stubs, sdk_patches.py cuts for its sdk build, manual shader rows), the game's own findings by topic (frame rate, memory, rendering, audio, movies, services, files and shaders, input, Mac routes), each marked current or resolved, a "Found on this game" list of the general host changes first needed there (one line each, linked to where they are explained), and what is still open, including stale profile comments and figures the sources disagree on. The notes gather what was spread over AGENTS.md, docs/development.md (Game notes and Power), docs/agents/01-05 and the profiles' own comments. The Game notes section of docs/development.md (523 lines, 53 games) is now a pointer to the table; development.md goes from 33,300 to 8,600 words. README.md, AGENTS.md and the Breath of the Wild link in 04-graphics.md point to the notes. A README.md in a profile folder changes no shader cache key (dmsh_chain.inputs hashes only the named tables and keys) and no profile listing (game.py lists folders with a game.tsv). Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (five file patterns now appear as the full names they stood for), and every relative link and heading anchor in the repository's Markdown resolves 2026-10-25 12:28:47 +08:00
seedds 1fbbd1486a Bayonetta (010076F0049A2000, update 1.1 = v65536, PlatinumGames' engine in its Wii U form on SDK 3.6.0, as Bayonetta 2): game profile, no host change. Plays on the iPhone (IPA 8.8 GB, user). Same sdk and subsdk0 builds as Bayonetta 2, so its profile is Bayonetta 2's with its own build row and tables, and the CPK reader and the two NVN fixes made for Bayonetta 2 cover it. On the Mac from a fresh save: the boot, the opening movie (USM; Plus skips it), the title and its attract demo, a new game (PLAY, NEW GAME, the auto-save notice, Normal), the prologue on the falling clock tower, the graveyard cutscene (skipped), the touch-controls tips, the practice tutorial (punch, kick, combos, evade, Witch Time), the graveyard fight of Verse 1 with Witch Time, its autosave and the next loading screen's practice, with CRI ADX2 sound through one 6-channel AudioOut and no red line: 60 fps in play (frame time p50 16.65 ms, p95 17.1-17.5 ms; GPU 2-4 ms a frame on an M6), 30 in its cutscenes, where the game sets present interval 2; about 2.45 GB resident and 1.54 GB of Metal allocations, flat over 15 minutes. No virtual address memory (a 3.11 GB heap in the 4 GB layout, mem_used 102 MB). Files: nine CRI CPK archives (data00.cpk ... data03.cpk, data10.cpk ... data14.cpk, 4.25 GB) of PlatinumGames DAT archives, beside the USM movies and the ADX2 sound. Shaders: 458 BNSH files of one stage each (236 pixel, 222 vertex), the .vso / .pso members of shader.dat in data00.cpk, all built for the Mac and iOS. Profile: the 3D pad layout (the camera is on the right stick; the touch controls are handheld only), a prewarm list recorded on the Mac over the route above (513 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); the Python tool, gfx and iOS tests pass (test_tbdr_budget with DM_GAME set). README (games gallery, a Mac picture of the graveyard fight), AGENTS.md (59 -> 60), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-25 11:03:12 +08:00
seedds 9c1e449470 Bayonetta 2 (01007960049A0000, update 1.2 = v131072, PlatinumGames' engine in its Wii U form on SDK 3.6.0, the oldest SDK so far): game profile, a CRI CPK reader for the shader split, and two general NVN fixes. Plays on the iPhone (IPA 12.5 GB, user). On the Mac from a fresh save: the boot, the PlatinumGames logo movie (USM), the title, a new game (slot 1, the auto-save notice, Story, Normal), the opening narration (which cannot be skipped on a first play), the prologue cutscene (skipped: Plus, Down, A, A), the tutorial (closed: Plus, Down x4, A), Verse 1 on the jet, Rodin's cutscene and the next fight with an Umbran Climax and a torture attack, with Wwise sound through two 6-channel AudioOuts: 60 fps in play (frame time p50 16.67 ms, p95 about 17.5 ms; GPU 2-3.5 ms a frame on an M6), 30 in its cutscenes as the game asks, about 1.6 GB resident and 1.75 GB of Metal allocations. No host change was needed for SDK 3.6.0 itself (NVN API 53); no virtual address memory (a 3.11 GB heap in the 4 GB layout, mem_used 102 MB). Files: six CPK archives (9.5 GB, 6,091 files, 12 GB of them CRILAYLA-compressed) of PlatinumGames DAT archives. New dm/gfx/tools/cpk.py: the @UTF tables (and the masked ones of Unicorn Overlord's CPKs, the same tables as before once unmasked), the TOC and CRILAYLA (a decoder over a bit string, about 29 MB/s); shader_split.py cpk_jobs splits the CPKs into jobs and scans each entry (a CPK it cannot read is scanned as one file, with a [RED] line). Shaders: 1,856 BNSH files of one stage each (.vso / .pso) in shader.dat, shader2.dat and shadereff.dat of data000.cpk, all built for the Mac and iOS. nvn_mtl.m, both general: a cleared attachment larger than the render pass (Metal's render area is the smallest attachment) is now cleared in a pass of its own and loaded by the pass, as NVN clears the whole target (rp_begin_ex_): the game's 32x32 and 256x256 passes cleared only that corner of its full-size depth target, so the opening narration drew in a corner of a black screen and the opening scenes were speckled; and Downsample of a single-sample texture is a filtered scaling copy of the whole level that may convert the format (copy_t2t, its bloom chain 320x176 -> 80x48 and its 1280x720 BGRA8 frame into an RGBA8 texture), which logged red lines before. build_ipa_from_nsp.py: a survey note for CPK archives. Profile: the 3D pad layout (the camera is on the right stick; the game's touch controls are handheld only), a prewarm list recorded on the Mac over the route above (974 lines). Checked: the graphics run list, the host unit tests and the 49 tool tests pass; Unicorn Overlord's CPKs split as before. Not checked: 13 Sentinels and Catherine: Full Body's CPKs (no files on the drives), the clear fix in other games beyond the unit tests. README (games gallery, a Mac picture of Verse 1), AGENTS.md (58 -> 59), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-25 08:56:02 +08:00
seedds 3f4d973ac5 Mario vs. Donkey Kong (0100B99019412000, update 1.0.1 over the cartridge base game, Nintendo's sead / agl / gsys engine on SDK 15.3.0): game profile plus three general host changes. Plays on the iPhone (IPA 1.6 GB, user). On the Mac: the opening movie with sound, the title (Press L+R), file select, the play style choice, the first cutscene, level 1-1 (both colour switches, the spring, a time-out and its restart), the pause menu, leaving the level, level select and the save it shows: 60 fps (frame interval p99 about 17 ms), sound, about 1 GB resident in the level and flat while it runs, CPU about 30% of a core, no red line on that route.
Movie decoder (hle_movie.c, every game), both found on the opening movie (Movie/Cutscene01.mp4, H.264 and AAC through movie::Extractor):
- AAC decoders offer at most 32 outputs (MD_NOUT_AAC; Vorbis keeps 64, video 16). The game's movie player keeps the outputs it holds in a ring of 32 entries per track (main+0xe93d4c: 0x200 bytes) and pushes with no bounds check (main+0xe94140), so with 64 the 33rd AAC output wrote its index (0x3f) and time over the next heap block, and the game crashed in sead::ExpHeap's free a second into the movie, at the same address every boot. The console's AAC decoder cannot offer more than 32, or the game would overflow there too; Monster Train 2 peaked at 30 held.
- An AAC decoder's GetOutputBufferSize is one frame (1024 samples per channel, 2048 with SBR), what each output holds. The player sizes its own AudioOut buffers by it, decodes into them and appends each whole, advancing its audio clock by size / 4 samples (main+0xe92a74); with the 128 KB Vorbis bound every 4 KB AAC frame played as 0.68 s, the clock crawled and the movie froze on its first second. Vorbis keeps the bound (its packets vary in length).

Advanced blend modes (hle_nvn.c, nvn_obj.h, nvn_mtl.m, every game): the blend state keeps the mode, the premultiplied-source flag and the overlap (with their getters), and the pipeline blends the Porter-Duff modes (ZERO, SRC, DST, SRC/DST_OVER, _IN, _OUT, _ATOP, XOR, the PLUS modes) with the fixed-function factors they amount to under uncorrelated overlap (adv_blend); the others keep the state's functions with a red line naming the mode. The game's levels set SRC_OVER on one blend state, which the host used to ignore.

DM_PROBE_POLL takes a value filter (=<value>): every change is logged, the threads are snapshotted and dumped only at changes to that value. That is how the ring overflow was caught: DM_PROBE_WATCH on the heap link made the movie's file reads into the same page fail.

Profile: the provisional profile's guesses held (no system resource, a 3.14 GB heap in the 4 GB layout, modules main subsdk0 sdk), plus render_cond on (the game skips clears under render conditions; without it every level was black behind its HUD). 12,691 shaders (BNSH / BAHS in the zstd RomFS and main), all translated. Pad only (main imports no touch screen), the default 2D layout. A prewarm list recorded on the Mac from a fresh save over the route above (550 lines; with it nothing was built on demand). Checked: the NSP script packs it from the NSPs with the profile in dm/games; the graphics run list and the 49 tool tests pass; after rebasing on 6d34984 (spin waits) the game boots into level 1-1 as before. README (gallery, a Mac picture of level 1-1; 2D layout), AGENTS.md (57 -> 58) and docs/development.md, docs/agents 03 / 04 / 05 say so
2026-10-25 01:19:47 +08:00
seedds 6d34984488 Spin waits that take a spin lock (general, from PR #134, found on Breath of the Wild; fixes #129). In steady play on the Mac, Breath of the Wild had frames with 30-210 ms of CPU work every 1-10 s: the main thread and the two workers spinning in one job queue poll (main+0xef62dc). The first item of some of its jobs is for the thread of core 0; the others wait until it is taken, taking and releasing the queue's lock (an exclusive test-and-set, then a compare-and-clear) and writing a status word on every pass, so the spin wait search (a store or an exclusive store ends a path) did not take it. Polled at full speed by the two workers, the lock went back and forth between their cores, and the main thread, whose item it was, lost nearly every exclusive store: in a 40 s Time Profiler trace it never held the lock while each worker held it in about 750 samples. patch_nso.py "Spin waits": a lock taken and released on the same address on every path back to the head (spin_lock 'acq' / 'rel', counted like the exclusive adds), a test loop on a word inside the region ('tst', whose branch back goes to the original loop and its own gate), and up to four stores whose value and address registers the region never writes ('stc', only in a region that loads something). Such stores write fixed values to fixed addresses, so a loop can change its own state for a few hundred passes at most, well inside the gate's 2 ms of polling, before its passes repeat. A store of a value a pass changes ends its path and the search runs again without it (the path that takes an item stores the new cursor); when the region with stores does not cover the old one's back-edges, the old one is used. Through the gate the workers leave the lock a moment between passes and the main thread takes its item at once. Zora's Domain on an M2 Max at 120 Hz (one host, 42 s of steady play per run, three runs each, before -> after): standing still, frames with more than 30 ms of work 2.8 / 6.3 / 3.5 -> 0 per 30 s, the frame's work p99 20-21 -> 8.3-8.4 ms; turning the camera, CPU hitches 11.9 / 13.3 / 6.3 -> 0, p99 21-25 -> 9-15 ms; process CPU per frame unchanged. Every module on the game drives rewritten with the old and the new search (206: 108 NSOs and Super Smash Bros. Ultimate's 98 NROs): 175 byte for byte as before (Hades, Super Smash Bros. Ultimate, Super Mario Odyssey among them), 7 fail alike under both (every rtld, New Super Mario Bros. U Deluxe), and 24 take new or wider loops, all but Breath of the Wild's queue through the store rule alone: Monster Hunter Rise's main 90 -> 166 (72 of them one inlined countdown whose reset makes the next pass leave), BALL x PIT 87 -> 94, Animal Crossing: New Horizons 162 -> 166, Balatro 9 -> 12, Octopath Traveler 0 75 -> 78, Pikmin 4 80 -> 83, Bayonetta 3 35 -> 37, Breath of the Wild 308 -> 310, one more each in Big Brain Academy, Mario Kart 8 Deluxe, Metroid Dread, Pokemon Legends: Z-A, Stardew Valley, Super Mario Bros. Wonder (both builds) and Tears of the Kingdom; the movie library's thread loop is wider in the subsdk0 of Breath of the Wild, Mario Kart 8 Deluxe, Animal Crossing, BALL x PIT, Monster Train 2 and Octopath Traveler 0, and one loop is wider in the main of AI: THE SOMNIUM FILES and Advance Wars 1+2. No loop of the old search is lost or loses a back-edge. Each distinct shape was read in its disassembly: one-pass resets and state setters (the gate is reached only when another thread changes the word), parser states that leave on the next pass, and real flag waits. Rewriting all 206 takes about 4,100 s of CPU instead of 3,250. Diagnostics: DM_HITCH_DUMP=<ms>[:<s>] logs every guest thread (its wait, last SVC, where it is, x0-x2 and x19-x25 in guest code) when no NVN frame was presented for that long, and each guest thread names its pthread '<handle> <name>' (0x1008 Worker) before it enters guest code, the main thread '0x1001 main thread', so sample and Instruments' Time Profiler show which guest thread a sample belongs to. The graphics tests that link hle_nvn.c without the host define dm_present_ns (t_host_stubs.c weak, t_nvn.c). New test dm/tools/tests/test_spin_waits.py. Checked: the host builds, the host unit tests, the 46 tool tests and the gfx run list pass; Breath of the Wild into Zora's Domain and Mario Kart 8 Deluxe for 85 s from boot with the new rewrite (the PR's author), and Monster Hunter Rise from a fresh save for 180 s headless with each rewrite on one host (e72a065's, the chain of its Mac libraries): 60 fps, frame p95 about 17.5 ms, the same red lines, no hang. Not checked: Paper Mario: The Thousand-Year Door, DARK SOULS: REMASTERED, FINAL FANTASY TACTICS and FINAL FANTASY RESONANCE (no files on the drives), the iPhone. docs/development.md (Spin loops, Breath of the Wild) and docs/agents/05-debugging.md say so 2026-10-25 00:48:24 +08:00