From 3a993fa3feaa1071cece5677b4c1b752973b956e Mon Sep 17 00:00:00 2001 From: seedds Date: Sun, 25 Oct 2026 12:24:24 +0800 Subject: [PATCH] Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in " subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games//README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv) --- dm/games/0100000000010000/README.md | 10 +- dm/games/010005501E68C000/README.md | 8 +- dm/games/010013C00E930000/README.md | 6 +- dm/games/010014E00DB56000/README.md | 6 +- dm/games/010015100B514000/README.md | 16 +- dm/games/0100152000022000/README.md | 6 +- dm/games/010020400E1C2000/README.md | 2 +- dm/games/010028600EBDA000/README.md | 8 +- dm/games/01002B00111A2000/README.md | 12 +- dm/games/01002B30028F6000/README.md | 2 +- dm/games/0100300012F2A000/README.md | 4 +- dm/games/0100346017304000/README.md | 2 +- dm/games/010036E00FB20000/README.md | 6 +- dm/games/010038B015560000/README.md | 8 +- dm/games/01003FC01670C000/README.md | 10 +- dm/games/01004A4010FEA000/README.md | 4 +- dm/games/01004AB00A260000/README.md | 10 +- dm/games/010051701FB46000/README.md | 8 +- dm/games/01005270232F2000/README.md | 12 +- dm/games/0100535012974000/README.md | 2 +- dm/games/010055D009F78000/README.md | 12 +- dm/games/0100606026F12000/README.md | 6 +- dm/games/0100620012D6E000/README.md | 12 +- dm/games/010069401ADB8000/README.md | 4 +- dm/games/01006A800016E000/README.md | 8 +- dm/games/01006BB00C6F0000/README.md | 8 +- dm/games/01006C300E9F0000/README.md | 14 +- dm/games/01006F8002326000/README.md | 8 +- dm/games/01007EF00011E000/README.md | 8 +- dm/games/010086A022444000/README.md | 4 +- dm/games/010089B00D09C000/README.md | 4 +- dm/games/01008BC024FDE000/README.md | 8 +- dm/games/010093801237C000/README.md | 12 +- dm/games/010099C022B96000/README.md | 8 +- dm/games/01009BF0072D4000/README.md | 2 +- dm/games/0100A00019DE0000/README.md | 2 +- dm/games/0100A5C00D162000/README.md | 6 +- dm/games/0100A6301214E000/README.md | 16 +- dm/games/0100A83022A22000/README.md | 18 +- dm/games/0100A9D01C446000/README.md | 12 +- dm/games/0100ABF008968000/README.md | 6 +- dm/games/0100B04011742000/README.md | 16 +- dm/games/0100B7C00933A000/README.md | 6 +- dm/games/0100B99019412000/README.md | 4 +- dm/games/0100BF00112C0000/README.md | 6 +- dm/games/0100C2A01A730000/README.md | 4 +- dm/games/0100CD801CE5E000/README.md | 4 +- dm/games/0100D9F01D474000/README.md | 6 +- dm/games/0100DCA0064A6000/README.md | 6 +- dm/games/0100E44016C22000/README.md | 6 +- dm/games/0100E65002BB8000/README.md | 2 +- dm/games/0100ECD018EBE000/README.md | 10 +- dm/games/0100F2C0115B6000/README.md | 10 +- dm/games/0100F43008C44000/README.md | 12 +- dm/games/0100F5D01DA3A000/README.md | 4 +- dm/games/0100FD8022DAA000/README.md | 10 +- dm/games/profile_guess.py | 2 +- dm/gfx/nvn/nvn_mtl.m | 2 +- dm/host/hle_audio.c | 2 +- docs/agents/02-survey.md | 2 +- docs/agents/03-bringup.md | 318 +------------ docs/agents/04-graphics.md | 374 +-------------- docs/agents/05-debugging.md | 8 +- docs/agents/README.md | 9 +- docs/agents/symptoms.md | 386 ++++++++++++++++ docs/reference/README.md | 19 + docs/reference/audio.md | 205 +++++++++ docs/reference/frame-rate.md | 99 ++++ docs/reference/game-state.md | 50 ++ docs/reference/memory.md | 264 +++++++++++ docs/reference/movies.md | 188 ++++++++ docs/reference/nvn-objects.md | 361 +++++++++++++++ docs/reference/rendering.md | 686 ++++++++++++++++++++++++++++ docs/reference/rewriter.md | 219 +++++++++ docs/reference/services.md | 595 ++++++++++++++++++++++++ docs/reference/shaders.md | 557 ++++++++++++++++++++++ docs/reference/threads.md | 118 +++++ 77 files changed, 3978 insertions(+), 912 deletions(-) create mode 100644 docs/agents/symptoms.md create mode 100644 docs/reference/README.md create mode 100644 docs/reference/audio.md create mode 100644 docs/reference/frame-rate.md create mode 100644 docs/reference/game-state.md create mode 100644 docs/reference/memory.md create mode 100644 docs/reference/movies.md create mode 100644 docs/reference/nvn-objects.md create mode 100644 docs/reference/rendering.md create mode 100644 docs/reference/rewriter.md create mode 100644 docs/reference/services.md create mode 100644 docs/reference/shaders.md create mode 100644 docs/reference/threads.md diff --git a/dm/games/0100000000010000/README.md b/dm/games/0100000000010000/README.md index 9ecf846..2cce0a1 100644 --- a/dm/games/0100000000010000/README.md +++ b/dm/games/0100000000010000/README.md @@ -38,7 +38,7 @@ No `patches.py`. `op_mode` stays docked: Odyssey only takes its motion moves in ## Audio -- It is silent for its first minute with either audio renderer update lock: the old SDK 16 offset and the one `update_lock` now reads off the SDK's own `Update` (`MutexType` at +0x1d8 on SDK 6.4–8.2) ([03, Audio](../../../docs/agents/03-bringup.md#audio)). +- It is silent for its first minute with either audio renderer update lock: the old SDK 16 offset and the one `update_lock` now reads off the SDK's own `Update` (`MutexType` at +0x1d8 on SDK 6.4–8.2) ([The renderer's update lock](../../../docs/reference/audio.md#the-renderers-update-lock-update_lock)). ## Files and shaders @@ -55,11 +55,11 @@ No `patches.py`. `op_mode` stays docked: Odyssey only takes its motion moves in General host changes first needed here; each applies to every game: -- **Largest point size at `c0 + 0x610`**: [04, section 3](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census) and [04, Seen in Odyssey](../../../docs/agents/04-graphics.md#seen-in-super-mario-odyssey). -- **Invariant positions** (`-fpreserve-invariance` in `dmsh_build.py` and `ios_metal.py`): [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). -- **Full controller and six-axis histories**, **six-axis sensors and the shake** (`DM_BTN_SHAKE`), **SDK 7 start-up services**: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Largest point size at `c0 + 0x610`**: [04, section 3](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census) and [Graphics `c0 + 0x610`: the largest point size](../../../docs/reference/shaders.md#graphics-c0--0x610-the-largest-point-size). +- **Invariant positions** (`-fpreserve-invariance` in `dmsh_build.py` and `ios_metal.py`): [Position invariance across shaders](../../../docs/reference/shaders.md#position-invariance-across-shaders). +- **Full controller and six-axis histories**, **six-axis sensors and the shake** (`DM_BTN_SHAKE`), **SDK 7 start-up services**: [History reads fill every requested entry](../../../docs/reference/services.md#history-reads-fill-every-requested-entry), [Six-axis sensors](../../../docs/reference/services.md#six-axis-sensors), [SDK calls that open a system service through sm](../../../docs/reference/services.md#sdk-calls-that-open-a-system-service-through-sm). - **H.264 movies through VideoToolbox** (the `avcc` record, a presentation-order queue): described under [Movies](#movies); `hle_movie.c`. -- **Small host segments for small grants** (`hostseg_size`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Small host segments for small grants** (`hostseg_size`): [Host-lent command memory segments](../../../docs/reference/nvn-objects.md#host-lent-command-memory-segments). ## Open diff --git a/dm/games/010005501E68C000/README.md b/dm/games/010005501E68C000/README.md index c0de899..3d8273d 100644 --- a/dm/games/010005501E68C000/README.md +++ b/dm/games/010005501E68C000/README.md @@ -48,10 +48,10 @@ The C# file API and the real-time clock reach the SDK's C library (nnmusl), not General host changes first needed here; each applies to every game: -- **Indirect tail calls through `x18`** (`tail_call` / `site_br_call`), tried only where the rewriter used to stop: [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **The SDK 17 C library's path checks** (`IsPathMounted` and the length-limit queries, `hle_fs.c`) and **the C library's real-time clock** (`hle_time.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Node area cut for a Unity 2022 game with virtual address memory** (`StandardAllocator::Initialize(nullptr, pool − used)`): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **Unity 2022 `Shader` objects** (`unity_code_blobs`, no local keyword list in `unityfs.subprogram`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **Indirect tail calls through `x18`** (`tail_call` / `site_br_call`), tried only where the rewriter used to stop: [Indirect tail calls through x18 in IL2CPP thunks](../../../docs/reference/rewriter.md#indirect-tail-calls-through-x18-in-il2cpp-thunks). +- **The SDK 17 C library's path checks** (`IsPathMounted` and the length-limit queries, `hle_fs.c`) and **the C library's real-time clock** (`hle_time.c`): [C library path checks ask the file system's limits](../../../docs/reference/services.md#c-library-path-checks-ask-the-file-systems-limits), [`clock_gettime(CLOCK_REALTIME)` through the time source pointer](../../../docs/reference/services.md#clock_gettimeclock_realtime-through-the-time-source-pointer). +- **Node area cut for a Unity 2022 game with virtual address memory** (`StandardAllocator::Initialize(nullptr, pool − used)`): [The SDK's address-region node area](../../../docs/reference/memory.md#the-sdks-address-region-node-area). +- **Unity 2022 `Shader` objects** (`unity_code_blobs`, no local keyword list in `unityfs.subprogram`): [Unity 2022 Shader objects with parameter blobs](../../../docs/reference/shaders.md#unity-2022-shader-objects-with-parameter-blobs). ## Open diff --git a/dm/games/010013C00E930000/README.md b/dm/games/010013C00E930000/README.md index acca672..a5e1eb7 100644 --- a/dm/games/010013C00E930000/README.md +++ b/dm/games/010013C00E930000/README.md @@ -47,9 +47,9 @@ Update 1.0.30000 (v655360) over the base game, SDK 20.5.8. Unity 6000.0.50f1 IL2 General host changes first needed here; each applies to every game: -- **`hid::GetDebugPadStates`**, the history form (`hle_hid.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Unity 6 memory with no StandardAllocator and no heap**, and the NSP script's 5 GB block for every game with an NPDM system resource (`profile_guess.py`): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space), [dm/games/README.md, Adding a game](../README.md#adding-a-game). -- **Unity 6 `Shader` objects** read from the bundles' own type trees: [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **`hid::GetDebugPadStates`**, the history form (`hle_hid.c`): [History reads fill every requested entry](../../../docs/reference/services.md#history-reads-fill-every-requested-entry). +- **Unity 6 memory with no StandardAllocator and no heap**, and the NSP script's 5 GB block for every game with an NPDM system resource (`profile_guess.py`): [Unity 6 takes its memory with no StandardAllocator and no heap](../../../docs/reference/memory.md#unity-6-takes-its-memory-with-no-standardallocator-and-no-heap), [dm/games/README.md, Adding a game](../README.md#adding-a-game). +- **Unity 6 `Shader` objects** read from the bundles' own type trees: [Unity 6 Shader objects](../../../docs/reference/shaders.md#unity-6-shader-objects). ## Open diff --git a/dm/games/010014E00DB56000/README.md b/dm/games/010014E00DB56000/README.md index cdec9d1..f44012b 100644 --- a/dm/games/010014E00DB56000/README.md +++ b/dm/games/010014E00DB56000/README.md @@ -13,7 +13,7 @@ On the Mac: the logos, the title, the privacy policy, the DigiLab story choice, | 4 GB layout: `mem_reserve 0x100000000`, `mem_alias 0x10000000`, `mem_stack 0x10000000`, `app_pool 0xCD500000` | `game.tsv` | the 4 GB layout of the games without virtual address memory | | `mem_used 0x4a00000` (74 MB) | `game.tsv` | the rewritten modules are larger than the default 64 MB of used memory | | No NVN shader tables | `game.tsv` `shader_note` | no NVN shaders: `gl_shaders.py` writes the GL library and its table, not the profile | -| The GL path | `main`'s imports | `profile_guess.py` gives `graphics gl` when `main` imports `eglCreateWindowSurface` and names `glShaderSource`; of the 22 games extracted ([04](../../../docs/agents/04-graphics.md#seen-in-digimon-story-cyber-sleuth-complete-edition-opengl) says), only Digimon does | +| The GL path | `main`'s imports | `profile_guess.py` gives `graphics gl` when `main` imports `eglCreateWindowSurface` and names `glShaderSource`; of the 22 games extracted ([rendering.md](../../../docs/reference/rendering.md#telling-a-gl-game-from-an-nvn-or-vulkan-one) says), only Digimon does | | glslang | packing | for a GL game the NSP script runs `gl_shaders.py` in place of the NVN shader split and chain build (Mac and iOS libraries in `_build/shader/gl` and `shader/ios/gl`); it needs `brew install glslang` | No `pad_layout`: the virtual gamepad starts in the 2D layout (the default). @@ -59,10 +59,10 @@ SDK 6.4.0 boots, one each: General host changes first needed here; each applies to every game: -- **The OpenGL layer** (`dm/gfx/gl/hle_gl.m`: EGL, GL on Metal, `mtl_set_origin_lower_left`, `mtl_ext_tex` presents) and **offline GLSL translation** (`dm/gfx/tools/gl_shaders.py`: glslang `-V -R`, SPIRV-Cross, varyings matched by name, `--fixup-clipspace`; `DM_GL_CAPTURE`): [04, Seen in Digimon Story Cyber Sleuth](../../../docs/agents/04-graphics.md#seen-in-digimon-story-cyber-sleuth-complete-edition-opengl). +- **The OpenGL layer** (`dm/gfx/gl/hle_gl.m`: EGL, GL on Metal, `mtl_set_origin_lower_left`, `mtl_ext_tex` presents) and **offline GLSL translation** (`dm/gfx/tools/gl_shaders.py`: glslang `-V -R`, SPIRV-Cross, varyings matched by name, `--fixup-clipspace`; `DM_GL_CAPTURE`): [Games that render with desktop OpenGL through EGL](../../../docs/reference/rendering.md#games-that-render-with-desktop-opengl-through-egl), [GLSL source translated offline and found by hash](../../../docs/reference/rendering.md#glsl-source-translated-offline-and-found-by-hash). - **MVGL archives** (`dm/gfx/tools/mvgl.py`). - **Loader census** (`dm/tools/loader_census.py`) and the `graphics gl` guess in `profile_guess.py`: [02, NVN surface](../../../docs/agents/02-survey.md#nvn-surface). -- **Add-on content queries on the big host stack** (`HLE_FORCE_BIG`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Add-on content queries on the big host stack** (`HLE_FORCE_BIG`): [Add-on content queries on a small guest stack](../../../docs/reference/services.md#add-on-content-queries-on-a-small-guest-stack). ## Open diff --git a/dm/games/010015100B514000/README.md b/dm/games/010015100B514000/README.md index b41afe2..c535225 100644 --- a/dm/games/010015100B514000/README.md +++ b/dm/games/010015100B514000/README.md @@ -68,7 +68,7 @@ Game-specific findings; the general host changes they led to are listed under [F ## Audio -- The `sdk` carries the audio renderer (SDK 16): the host switches it to CPU rendering with manual execution and drives it from a guest thread, no IPC ([03, Audio](../../../docs/agents/03-bringup.md#audio)). +- The `sdk` carries the audio renderer (SDK 16): the host switches it to CPU rendering with manual execution and drives it from a guest thread, no IPC ([Running the SDK's own renderer on the CPU](../../../docs/reference/audio.md#running-the-sdks-own-renderer-on-the-cpu)). - **Resolved:** Mario updates the renderer twice in a row at start-up; with a real audio device the first frame came after both, and the light limiter later crashed on a null state pointer (`ApplyLightLimiterEffect`, `sdk+0x228494`). The `AudioRenderSystem::Update` hook renders a catch-up frame first when nothing was rendered since the last update. ## Services and survey @@ -90,13 +90,13 @@ Game-specific findings; the general host changes they led to are listed under [F General host changes first needed here; each applies to every game: -- **NVN on Metal basics** in [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario): 8-bit index widening, texture swizzles (`nvn_fmt_to_mtl`, `img_fit`), format-changing render-target views (`attach`, `present_raw_src`), storage-shared transient targets, inline uniform updates as in-order copies snapshotted at encode (`mtl_sub_update`), CPU-written linear textures on the pool buffer (`tex_linear`), integer vertex inputs (`int_inputs`, `vfmt_for_ct`), intra-encoder barriers (`blit_buf`, `DM_NVN_BLIT_ORDER`), zero-filled new textures (`tex_zero_fill`), scissored zero clears (`clear_rect_zero`), vertex output padding from shader pairs (`vs_pad`, `shader_pairs.tsv`), virtual pools aliased onto one physical pool. -- **Binding layout and handles in uniform buffers** (`NVN_RB_TEXU` / `NVN_RB_SAMPU`): [04, section 3](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census); the compute code start: [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **The SDK's audio renderer in-process** and the catch-up frame between two updates: [03, Audio](../../../docs/agents/03-bringup.md#audio). -- **Resource heaps cut by build ID** (`CUTS` in `patches.py`), **memory partition** (`mem_init`), **virtual NVN pools**: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space) and [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Update imports diffed before the first run**, **fixed-address code gated by `module_t.build_id`**, **BFTTF shared font** (`hle_sdk16.c`), **the extension font**, **`GetCommandMemoryUsed` counting host segments**, **`NVNcommandBuffer` +0x10** (`nvn_flag10`), **`prog_t` size**: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk) and [03, NVN object sizes](../../../docs/agents/03-bringup.md#nvn-object-sizes-memory-corruption). -- **Exclusive windows** (fall-through exit first, Mario had 82): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). **Ideal core −2**: [03, Threads](../../../docs/agents/03-bringup.md#threads). -- **Present-interval holds** (`g_link_hold_until`, `nvn_game_fps`): [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). +- **NVN on Metal basics** in [rendering.md](../../../docs/reference/rendering.md): [8-bit index widening](../../../docs/reference/rendering.md#8-bit-index-buffers), [texture swizzles](../../../docs/reference/rendering.md#texture-swizzle-through-swizzled-metal-views) (`nvn_fmt_to_mtl`, `img_fit`), [format-changing render-target views](../../../docs/reference/rendering.md#render-target-views-that-change-the-format) (`attach`, `present_raw_src`), [storage-shared transient targets](../../../docs/reference/rendering.md#transient-render-targets-shared-by-storage), [inline uniform updates as in-order copies](../../../docs/reference/rendering.md#inline-uniform-updates-as-ordered-gpu-copies) snapshotted at encode (`mtl_sub_update`), [CPU-written linear textures](../../../docs/reference/nvn-objects.md#linear-textures-in-cpu-visible-pools) on the pool buffer (`tex_linear`), [integer vertex inputs](../../../docs/reference/rendering.md#integer-vertex-inputs) (`int_inputs`, `vfmt_for_ct`), [intra-encoder barriers](../../../docs/reference/rendering.md#metal-4-does-not-order-operations-inside-one-encoder) (`blit_buf`, `DM_NVN_BLIT_ORDER`), [zero-filled new textures](../../../docs/reference/rendering.md#new-textures-start-as-zero) (`tex_zero_fill`), [scissored zero clears](../../../docs/reference/rendering.md#scissored-colour-clears) (`clear_rect_zero`), [vertex output padding from shader pairs](../../../docs/reference/shaders.md#pixel-shader-inputs-the-vertex-shader-does-not-write) (`vs_pad`, `shader_pairs.tsv`), [virtual pools aliased onto one physical pool](../../../docs/reference/nvn-objects.md#virtual-memory-pools-mapped-onto-one-physical-pool). +- **Binding layout and handles in uniform buffers** (`NVN_RB_TEXU` / `NVN_RB_SAMPU`): [04, section 3](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census); the compute code start: [Compute code with and without a header](../../../docs/reference/shaders.md#compute-code-with-and-without-a-header). +- **The SDK's audio renderer in-process** and the catch-up frame between two updates: [Running the SDK's own renderer on the CPU](../../../docs/reference/audio.md#running-the-sdks-own-renderer-on-the-cpu), [At least one rendered frame between two updates](../../../docs/reference/audio.md#at-least-one-rendered-frame-between-two-updates). +- **Resource heaps cut by build ID** (`CUTS` in `patches.py`), **memory partition** (`mem_init`), **virtual NVN pools**: [Per-resource heaps sized from Nintendo's measured peaks](../../../docs/reference/memory.md#per-resource-heaps-sized-from-nintendos-measured-peaks), [Partitioning the guest block between alias region, stacks and heap](../../../docs/reference/memory.md#partitioning-the-guest-block-between-alias-region-stacks-and-heap), [Virtual memory pools mapped onto one physical pool](../../../docs/reference/nvn-objects.md#virtual-memory-pools-mapped-onto-one-physical-pool). +- **Update imports diffed before the first run**, **fixed-address code gated by `module_t.build_id`**, **BFTTF shared font** (`hle_sdk16.c`), **the extension font**, **`GetCommandMemoryUsed` counting host segments**, **`NVNcommandBuffer` +0x10** (`nvn_flag10`), **`prog_t` size**: [A game update imports SDK calls the base game did not](../../../docs/reference/services.md#a-game-update-imports-sdk-calls-the-base-game-did-not), [Code inherited from another title at fixed addresses](../../../docs/reference/services.md#code-inherited-from-another-title-at-fixed-addresses), [Shared font address: BFTTF header or bare TTF](../../../docs/reference/services.md#shared-font-address-bfttf-header-or-bare-ttf), [Nintendo-extension glyphs for button icons](../../../docs/reference/services.md#nintendo-extension-glyphs-for-button-icons), [What `GetCommandMemoryUsed` counts](../../../docs/reference/nvn-objects.md#what-getcommandmemoryused-counts), [Inlined SDK code writes into `NVNcommandBuffer` +0x10](../../../docs/reference/nvn-objects.md#inlined-sdk-code-writes-into-nvncommandbuffer-0x10), [Host records must fit the game's SDK object sizes](../../../docs/reference/nvn-objects.md#host-records-must-fit-the-games-sdk-object-sizes). +- **Exclusive windows** (fall-through exit first, Mario had 82): [Exit stub order in exclusive windows](../../../docs/reference/rewriter.md#exit-stub-order-in-exclusive-windows). **Ideal core −2**: [Ideal core −2 resolves to the process default core](../../../docs/reference/threads.md#ideal-core-2-resolves-to-the-process-default-core). +- **Present-interval holds** (`g_link_hold_until`, `nvn_game_fps`): [Frames of a dynamic present interval held on screen and at the acquire](../../../docs/reference/frame-rate.md#frames-of-a-dynamic-present-interval-held-on-screen-and-at-the-acquire). - **Glitch detector before bisection**: [05, Intermittent rendering glitches](../../../docs/agents/05-debugging.md#intermittent-rendering-glitches); **offline pipeline link check**: [05](../../../docs/agents/05-debugging.md#offline-pipeline-link-check). ## Open diff --git a/dm/games/0100152000022000/README.md b/dm/games/0100152000022000/README.md index 66a5e73..af6aa6f 100644 --- a/dm/games/0100152000022000/README.md +++ b/dm/games/0100152000022000/README.md @@ -59,9 +59,9 @@ The game builds Mii heads from the console's MiiModel system data, starting at b General host changes first needed here; each applies to every game: -- **The Mii database and the MiiModel stand-in** (`hle_mii.c`, `dm_mii_standin`, `MountSystemData`), **the add-on content cache sized as Horizon's RomFS working memory** (`romfs_work_size`), **device save data as a save of its own** for a game that mounts account save data too (`_device`) and **SDK 4's DeviceSink** (`g_sink4`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **The audio renderer's update lock read off each SDK's `Update`** (`update_lock`): [03, Audio](../../../docs/agents/03-bringup.md#audio). -- **The zero cube for a 2D array of fewer than six layers in a cube slot** (`tex_fit`): [04, Seen in Pokémon Legends: Z-A](../../../docs/agents/04-graphics.md#seen-in-pokémon-legends-z-a). +- **The Mii database and the MiiModel stand-in** (`hle_mii.c`, `dm_mii_standin`, `MountSystemData`), **the add-on content cache sized as Horizon's RomFS working memory** (`romfs_work_size`), **device save data as a save of its own** for a game that mounts account save data too (`_device`) and **SDK 4's DeviceSink** (`g_sink4`): [Mii database answered by the host](../../../docs/reference/services.md#mii-database-answered-by-the-host), [System data stand-in for the MiiModel archive](../../../docs/reference/services.md#system-data-stand-in-for-the-miimodel-archive), [Add-on content mount cache sized as RomFS working memory](../../../docs/reference/services.md#add-on-content-mount-cache-sized-as-romfs-working-memory), [Device and account save data as two saves](../../../docs/reference/services.md#device-and-account-save-data-as-two-saves), [SDK 4: DeviceSink is command 0xf of 0x130 bytes](../../../docs/reference/audio.md#sdk-4-devicesink-is-command-0xf-of-0x130-bytes). +- **The audio renderer's update lock read off each SDK's `Update`** (`update_lock`): [The renderer's update lock (`update_lock`)](../../../docs/reference/audio.md#the-renderers-update-lock-update_lock). +- **The zero cube for a 2D array of fewer than six layers in a cube slot** (`tex_fit`): [A 2D texture in a cube slot binds the zero cube](../../../docs/reference/rendering.md#a-2d-texture-in-a-cube-slot-binds-the-zero-cube). ## Open diff --git a/dm/games/010020400E1C2000/README.md b/dm/games/010020400E1C2000/README.md index c2fd1d3..ec34c27 100644 --- a/dm/games/010020400E1C2000/README.md +++ b/dm/games/010020400E1C2000/README.md @@ -36,7 +36,7 @@ Base game 1.0.0 (v0), SDK 7.3.2. Unity 2018.3.10f1 IL2CPP with FMOD on NVN. Modu ## Found on this game -- **Unity 2018.3's `Shader` layout written out** (`SHADER_2018_3`, from AssetRipper's TypeTreeDumps): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **Unity 2018.3's `Shader` layout written out** (`SHADER_2018_3`, from AssetRipper's TypeTreeDumps): [Unity Shader layouts written out for players without bundles](../../../docs/reference/shaders.md#unity-shader-layouts-written-out-for-players-without-bundles). - **The heap check keeps the hardware pool** for a Unity 2018.3 game that maps only inside StandardAllocator's region: [dm/games/README.md, Adding a game](../README.md#adding-a-game). ## Open diff --git a/dm/games/010028600EBDA000/README.md b/dm/games/010028600EBDA000/README.md index 5fa64da..41f4bd8 100644 --- a/dm/games/010028600EBDA000/README.md +++ b/dm/games/010028600EBDA000/README.md @@ -53,11 +53,11 @@ No `patches.py`, no `dm/tools/sdk_patches.py` cut for its `sdk` build (`204eaef8 General host changes first needed here; each applies to every game: -- **Compatible texture views for a mismatched declared type** (`tex_fit`), **1D textures as one-row 2D**, **no writes to unwritten colour outputs** (`color_known`), **same-pass feedback snapshots** (`fb_prepare`, `feedback_snap`, `DM_NVN_MAGSCAN`): [04, Seen in 3D World](../../../docs/agents/04-graphics.md#seen-in-super-mario-3d-world--bowsers-fury). -- **SDK 10 control section lengths** (`ctl_len`) and **compute code without a header**: [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **The NVN API version by SDK** (`nvn_api`, `device_api`), **plain-TTF shared fonts by SDK** (`font_skip`, `shared_font`), **the inline keyboard** (`hle_swkbd.c`), **file system hooks on the big stack** (`HLE_FORCE_BIG`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk) and [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). +- **Compatible texture views for a mismatched declared type** (`tex_fit`), **1D textures as one-row 2D**, **no writes to unwritten colour outputs** (`color_known`), **same-pass feedback snapshots** (`fb_prepare`, `feedback_snap`, `DM_NVN_MAGSCAN`): [Compatible views for a mismatched declared texture type](../../../docs/reference/rendering.md#compatible-views-for-a-mismatched-declared-texture-type), [1D textures as one-row 2D textures](../../../docs/reference/rendering.md#1d-textures-as-one-row-2d-textures), [No writes to colour outputs the shader does not write](../../../docs/reference/rendering.md#no-writes-to-colour-outputs-the-shader-does-not-write), [Same-pass feedback read through a snapshot](../../../docs/reference/rendering.md#same-pass-feedback-read-through-a-snapshot). +- **SDK 10 control section lengths** (`ctl_len`) and **compute code without a header**: [Control sections shorter than 0x880 bytes](../../../docs/reference/shaders.md#control-sections-shorter-than-0x880-bytes), [Compute code with and without a header](../../../docs/reference/shaders.md#compute-code-with-and-without-a-header). +- **The NVN API version by SDK** (`nvn_api`, `device_api`), **plain-TTF shared fonts by SDK** (`font_skip`, `shared_font`), **the inline keyboard** (`hle_swkbd.c`), **file system hooks on the big stack** (`HLE_FORCE_BIG`): [The device's API version](../../../docs/reference/nvn-objects.md#the-devices-api-version), [Shared font address: BFTTF header or bare TTF](../../../docs/reference/services.md#shared-font-address-bfttf-header-or-bare-ttf), [The inline keyboard as a resident library applet](../../../docs/reference/services.md#the-inline-keyboard-as-a-resident-library-applet), [Host calls on small guest stacks](../../../docs/reference/memory.md#host-calls-on-small-guest-stacks). - **Yaz0 SARC in `shader_split.py`**: [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). -- **`L+R` joins in `DM_KEYS`**: [03, Game state](../../../docs/agents/03-bringup.md#game-state). +- **`L+R` joins in `DM_KEYS`**: [Player join screens (`hid::StartLrAssignmentMode`)](../../../docs/reference/game-state.md#player-join-screens-hidstartlrassignmentmode). ## Open diff --git a/dm/games/01002B00111A2000/README.md b/dm/games/01002B00111A2000/README.md index 3700db5..78a9c05 100644 --- a/dm/games/01002B00111A2000/README.md +++ b/dm/games/01002B00111A2000/README.md @@ -73,7 +73,7 @@ Update 1.3.0 (v327680) over the base game, with four DLC packages (one is only a - Everything sits in RDB archives: `asset/*.rdb` is the index (`_DRK0000` header, one `IDRK0000` record per file); `asset/*.rdb.bin*` and `asset/data/*.file` hold `IDRK` entries, zlib in 16 KB chunks (flag 0x100000). The DLC folders add `.rdb.bin3_0` files of the same layout ([01](../../../docs/agents/01-extract.md#compressed-romfs-content)). `romfs_marker` is `asset/MaterialEditor.rdb.bin`. - Shaders: 40,726 GLSLC outputs (37,043 pixel, 3,440 vertex, 241 compute, 2 geometry): KTGL's G2S containers (`_S2G3000`, type KTID `0x7bcd279f`; 23,108 entries in `asset/system.rdb.bin*`, 2.3 GB inflated) and one `_DOK0000` container (type `0x20a6a0bb`, the engine's system resources: 151 bare GLSLC outputs, 21 of them drawn in the first battle), plus 117 in `main` (`shader_split.py` `idrk_shaders`). Many have 0x850-byte controls and the compute ones no code header. Mario's texture handle layout (`0x20 + 8 × slot`); pairs come from the single-stage cover (`separable_pairs`). -- `glslc_section` took 3 more outputs (GLSLC version 0x1d.0, whose control +0x20 holds odd values) that used to be dropped ([04, Seen in Dark Souls Remastered](../../../docs/agents/04-graphics.md#seen-in-dark-souls-remastered)). +- `glslc_section` took 3 more outputs (GLSLC version 0x1d.0, whose control +0x20 holds odd values) that used to be dropped ([GLSLC outputs read through their section headers](../../../docs/reference/shaders.md#glslc-outputs-read-through-their-section-headers)). ## Mac routes @@ -83,12 +83,12 @@ Update 1.3.0 (v327680) over the base game, with four DLC packages (one is only a General host changes first needed here; each applies to every game: -- **Rewriter:** byte jump tables indexed by x30 (`main+0x138216c`; capstone names x30 `lr` and x29 `fp`, mapped back): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **`mem_used`** for games whose modules exceed 64 MB: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **Copyright frame buffer no-ops** and the **hardware Opus multi-stream decoder** as the SDK's software one (`hle_oe.c`, `hle_codec.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Compute's own binding** (`S.cprog`, `g_pp_cprog`), **`nvnTexturePoolRegisterImage`**, **indirect draws and dispatches** (`ixc_run`, `DM_NVN_IXCLOG`), **dispatches in the copy encoder** (`dispatch_enc`, `DM_NVN_DISPATCH_OWN=1`) and **texture views cached by texture** (`vc_get`): [04, Seen in Hyrule Warriors: Age of Calamity](../../../docs/agents/04-graphics.md#seen-in-hyrule-warriors-age-of-calamity). +- **Rewriter:** byte jump tables indexed by x30 (`main+0x138216c`; capstone names x30 `lr` and x29 `fp`, mapped back): [Byte tables indexed by x30 (capstone register names)](../../../docs/reference/rewriter.md#byte-tables-indexed-by-x30-capstone-register-names). +- **`mem_used`** for games whose modules exceed 64 MB: [Heap sized from the application pool less used memory](../../../docs/reference/memory.md#heap-sized-from-the-application-pool-less-used-memory). +- **Copyright frame buffer no-ops** and the **hardware Opus multi-stream decoder** as the SDK's software one (`hle_oe.c`, `hle_codec.c`): [Copyright frame buffer overlay](../../../docs/reference/services.md#copyright-frame-buffer-overlay), [Hardware Opus decoders built on the SDK's software decoders](../../../docs/reference/audio.md#hardware-opus-decoders-built-on-the-sdks-software-decoders). +- **Compute's own binding** (`S.cprog`, `g_pp_cprog`), **`nvnTexturePoolRegisterImage`**, **indirect draws and dispatches** (`ixc_run`, `DM_NVN_IXCLOG`), **dispatches in the copy encoder** (`dispatch_enc`, `DM_NVN_DISPATCH_OWN=1`) and **texture views cached by texture** (`vc_get`): [Compute programs bound apart from graphics programs](../../../docs/reference/rendering.md#compute-programs-bound-apart-from-graphics-programs), [Images registered on their own](../../../docs/reference/nvn-objects.md#images-registered-on-their-own), [Indirect draws and dispatches read by the GPU](../../../docs/reference/rendering.md#indirect-draws-and-dispatches-read-by-the-gpu), [Compute dispatches share the copy encoder](../../../docs/reference/rendering.md#compute-dispatches-share-the-copy-encoder), [Texture views cached by texture and view parameters](../../../docs/reference/rendering.md#texture-views-cached-by-texture-and-view-parameters). - **CPU vertex conversion over the draw's vertices** (`vconv_range`, `vconv_bind`): [dm/games/README.md](../README.md). -- **Shadow vertex inputs** (`dmsh_vin.py`, `vin_ct` / `vin_on`) and **row wrap** for buffer textures read by 1D fetches (`fetch1d`, `tbuf_as_2d`, `nvn_tbuf_rows`): [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). +- **Shadow vertex inputs** (`dmsh_vin.py`, `vin_ct` / `vin_on`) and **row wrap** for buffer textures read by 1D fetches (`fetch1d`, `tbuf_as_2d`, `nvn_tbuf_rows`): [Shadow vertex inputs](../../../docs/reference/rendering.md#shadow-vertex-inputs), [Buffer textures read by 1D integer fetches (row wrap)](../../../docs/reference/rendering.md#buffer-textures-read-by-1d-integer-fetches-row-wrap). - **`fps_hold off`** for games whose updates do not wait for the presents: [dm/games/README.md](../README.md). ## Open diff --git a/dm/games/01002B30028F6000/README.md b/dm/games/01002B30028F6000/README.md index 5aadf9a..0e2c37c 100644 --- a/dm/games/01002B30028F6000/README.md +++ b/dm/games/01002B30028F6000/README.md @@ -36,7 +36,7 @@ No `pad_layout`: the virtual gamepad starts in the 2D layout (the default). General host changes first needed here; each applies to every game: -- **MonoGame effects** (MGFX) in XNB files and in `main` (`mgfx_effect` in `shader_split.py`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **MonoGame effects** (MGFX) in XNB files and in `main` (`mgfx_effect` in `shader_split.py`): [MGFX effects in MonoGame XNB files](../../../docs/reference/shaders.md#mgfx-effects-in-monogame-xnb-files). ## Open diff --git a/dm/games/0100300012F2A000/README.md b/dm/games/0100300012F2A000/README.md index ba58e80..00d99a7 100644 --- a/dm/games/0100300012F2A000/README.md +++ b/dm/games/0100300012F2A000/README.md @@ -48,7 +48,7 @@ On the Mac: the boot, the Bink opening movie, the title, Nell's introduction, a ## Found on this game -- **Shader control sections loose in `main`** (`shader_split.py` `raw_shaders`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **`sdk` node-area and 1 GB alignment cuts in the `orr` form** of the SDK 7 / 8 constructor: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). +- **Shader control sections loose in `main`** (`shader_split.py` `raw_shaders`): [Loose control sections and code areas in `main` (Bink 2)](../../../docs/reference/shaders.md#loose-control-sections-and-code-areas-in-main-bink-2). +- **`sdk` node-area and 1 GB alignment cuts in the `orr` form** of the SDK 7 / 8 constructor: [The SDK's address-region node area](../../../docs/reference/memory.md#the-sdks-address-region-node-area), [1 GB alignment of regions of 1 GB and more](../../../docs/reference/memory.md#1-gb-alignment-of-regions-of-1-gb-and-more). - **The heap check's 1 GB pool is too small here**: [dm/games/README.md, Adding a game](../README.md#adding-a-game). - The spin-wait search's store rule takes one wider loop in its `main`: [development.md, Power](../../../docs/development.md#power). diff --git a/dm/games/0100346017304000/README.md b/dm/games/0100346017304000/README.md index 7b2722b..8e8de63 100644 --- a/dm/games/0100346017304000/README.md +++ b/dm/games/0100346017304000/README.md @@ -47,4 +47,4 @@ Update 1.1.1 (v131072) over the base game, SDK 12.3.3 (no other game's `sdk` bui General host changes first needed here; each applies to every game: -- **`oe::SetCpuOverclockEnabled`** (`hle_oe.c`) and **gestures for `hid::GetGestureStates`** recognized from the touch points (`gesture_sample`: yuzu's rules plus an 8 px slop, `t_gesture`, `DM_TOUCH_LOG`, `DM_CTL` `drag`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **`oe::SetCpuOverclockEnabled`** (`hle_oe.c`) and **gestures for `hid::GetGestureStates`** recognized from the touch points (`gesture_sample`: yuzu's rules plus an 8 px slop, `t_gesture`, `DM_TOUCH_LOG`, `DM_CTL` `drag`): [CPU overclock calls without an apm session](../../../docs/reference/services.md#cpu-overclock-calls-without-an-apm-session), [Touch gestures recognized from the touch points](../../../docs/reference/services.md#touch-gestures-recognized-from-the-touch-points). diff --git a/dm/games/010036E00FB20000/README.md b/dm/games/010036E00FB20000/README.md index f8bc140..b5ebd0b 100644 --- a/dm/games/010036E00FB20000/README.md +++ b/dm/games/010036E00FB20000/README.md @@ -48,6 +48,6 @@ Base game 1.0.0 (v0), with its DLC `010036E00FB21001`. SDK 10.4.1. Capcom's MT F General host changes first needed here; each applies to every game: -- **CPU-written textures with no flush** (`tex_dirty_upload`): [04, Seen in The Great Ace Attorney Chronicles](../../../docs/agents/04-graphics.md#seen-in-the-great-ace-attorney-chronicles), key in [dm/games/README.md](../README.md). -- **The MT Framework ARC and SPK readers** (`mtarc_members`, `mtspk_shaders`): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **SDK 10 calls found from the import list**, `nvnDeviceSetIntermediateShaderCache` and `nvnCommandBufferDiscardColor`: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **CPU-written textures with no flush** (`tex_dirty_upload`): [CPU writes with no flush, caught by page protection](../../../docs/reference/nvn-objects.md#cpu-writes-with-no-flush-caught-by-page-protection-tex_dirty_upload), key in [dm/games/README.md](../README.md). +- **The MT Framework ARC and SPK readers** (`mtarc_members`, `mtspk_shaders`): [SPK shader package in an MT Framework ARC](../../../docs/reference/shaders.md#spk-shader-package-in-an-mt-framework-arc). +- **SDK 10 calls found from the import list**, `nvnDeviceSetIntermediateShaderCache` and `nvnCommandBufferDiscardColor`: [services.md](../../../docs/reference/services.md), [NVN calls that were red stubs](../../../docs/reference/nvn-objects.md#nvn-calls-that-were-red-stubs). diff --git a/dm/games/010038B015560000/README.md b/dm/games/010038B015560000/README.md index 009e2f6..0c6a36f 100644 --- a/dm/games/010038B015560000/README.md +++ b/dm/games/010038B015560000/README.md @@ -35,7 +35,7 @@ Mario's binding layout (combined textures at `0x20 + 8 × slot`, images at `0x12 - 306 shaders: 257 GLSLC outputs in the engine's technique files (`resources_nx/release/master/shader/*.tec`, magic `TEC\0`, each output whole after a table of parameter names) and 49 in `main`. `shader_split.py` reads GLSLC outputs in any file no other reader takes, and pairs each as a single-stage program. Groups `resources` and `resourcesnx` (`shader_groups`). - **Unbuilt:** one geometry shader (`bahamuteffect.tec`), as Metal has no geometry stage. -- On the outputs the old GLSLC rule keeps, this game's among them, the section-header reading added for SDK 4 (`glslc_section`) gives the same key ([04, Seen in Dark Souls Remastered](../../../docs/agents/04-graphics.md#seen-in-dark-souls-remastered)). +- On the outputs the old GLSLC rule keeps, this game's among them, the section-header reading added for SDK 4 (`glslc_section`) gives the same key ([GLSLC outputs read through their section headers](../../../docs/reference/shaders.md#glslc-outputs-read-through-their-section-headers)). - `romfs_marker`: `resources/release/master/system/ffto/fftpack.en.bin`. ## Mac routes @@ -46,9 +46,9 @@ Mario's binding layout (combined textures at `0x20 + 8 × slot`, images at `0x12 General host changes first needed here; each applies to every game: -- **GLSLC outputs in any RomFS file no other reader takes** (`shader_split.py`): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **CPU writes into uncached pools uploaded** (`tex_dirty_upload` with `tex_flush_upload`), **integer views of image slots** (`img_fit`), **fragment barriers that end a pass after shader writes** (`swrite_note`, `nvn_refl.m`), **whole clears by channel mask** (`mtl_pf_chmask`) and the **new NVN entry points**: [04, Seen in FINAL FANTASY TACTICS - The Ivalice Chronicles](../../../docs/agents/04-graphics.md#seen-in-final-fantasy-tactics---the-ivalice-chronicles). -- **A job wake lost between two workers on one auto-clear event** (thread dump `wait #`): [03, Threads](../../../docs/agents/03-bringup.md#threads). +- **GLSLC outputs in any RomFS file no other reader takes** (`shader_split.py`): [GLSLC outputs in `main` and in loose files](../../../docs/reference/shaders.md#glslc-outputs-in-main-and-in-loose-files). +- **CPU writes into uncached pools uploaded** (`tex_dirty_upload` with `tex_flush_upload`), **integer views of image slots** (`img_fit`), **fragment barriers that end a pass after shader writes** (`swrite_note`, `nvn_refl.m`), **whole clears by channel mask** (`mtl_pf_chmask`) and the **new NVN entry points**: [CPU writes with no flush, caught by page protection (`tex_dirty_upload`)](../../../docs/reference/nvn-objects.md#cpu-writes-with-no-flush-caught-by-page-protection-tex_dirty_upload), [Integer views for images a shader declares uint or int](../../../docs/reference/rendering.md#integer-views-for-images-a-shader-declares-uint-or-int), [Fragment-ordering barriers between draws that write images](../../../docs/reference/rendering.md#fragment-ordering-barriers-between-draws-that-write-images), [A channel mask that covers every channel is a whole clear](../../../docs/reference/rendering.md#a-channel-mask-that-covers-every-channel-is-a-whole-clear), [Plural texture binds, builder-registered samplers, strided texel writes, device item 0x67](../../../docs/reference/nvn-objects.md#plural-texture-binds-builder-registered-samplers-strided-texel-writes-device-item-0x67). +- **A job wake lost between two workers on one auto-clear event** (thread dump `wait #`): [One auto-clear event shared by a pool's workers](../../../docs/reference/threads.md#one-auto-clear-event-shared-by-a-pools-workers). ## Open diff --git a/dm/games/01003FC01670C000/README.md b/dm/games/01003FC01670C000/README.md index fba2098..2186e14 100644 --- a/dm/games/01003FC01670C000/README.md +++ b/dm/games/01003FC01670C000/README.md @@ -18,7 +18,7 @@ No `patches.py`, no `sdk_patches.py` cut for its `sdk` build (`2d369d99…`), no ## Services -- **Resolved:** SDK 11 calls that reach a service: `hid::GetMouseState` (the first boot aborted in it: the SDK's mouse resource was never initialized), `fs::CleanDirectoryRecursively`, `fs::EnableGlobalFileDataCache`, `oe::SetUserInactivityDetectionTimeExtendedUnsafe`, `time::ToPosixTime` (host `mktime`) and `audio::GetAudioOutName` ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)). +- **Resolved:** SDK 11 calls that reach a service: `hid::GetMouseState` (the first boot aborted in it: the SDK's mouse resource was never initialized), `fs::CleanDirectoryRecursively`, `fs::EnableGlobalFileDataCache`, `oe::SetUserInactivityDetectionTimeExtendedUnsafe`, `time::ToPosixTime` (host `mktime`) and `audio::GetAudioOutName` ([Mouse, keyboard and key code maps](../../../docs/reference/services.md#mouse-keyboard-and-key-code-maps), [Directory, file data cache and save-thumbnail calls](../../../docs/reference/services.md#directory-file-data-cache-and-save-thumbnail-calls), [oe calls answered without an applet proxy](../../../docs/reference/services.md#oe-calls-answered-without-an-applet-proxy), [Time zone rules, calendar conversion and the network clock](../../../docs/reference/services.md#time-zone-rules-calendar-conversion-and-the-network-clock), [Audio device output volume and names](../../../docs/reference/audio.md#audio-device-output-volume-and-names)). ## Files and shaders @@ -38,10 +38,10 @@ No `patches.py`, no `sdk_patches.py` cut for its `sdk` build (`2d369d99…`), no General host changes first needed here; each applies to every game: -- **64 KB-aligned large allocations** (`alloc_big_align`, `hle_os.c`; shared with Unicorn Overlord, [010069401ADB8000](../010069401ADB8000/README.md)): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space) and [the profile keys](../README.md#keys). -- **GLSLC outputs in `main`** (`shader_split.py` `glslc_shaders`, the control / code / length check): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **Single-stage programs paired with a cover** (`vs_pad`), **a shader cache key that covers the pairs** (`shader_key`, `pair_keyed`), **triangle fans and indexed quads** (a shared fan index buffer; CPU expansion at replay in `nvn_mtl.m`), **vertex streams 14 and 15 at slots 14 and 15** (`VB_SLOT`) and **packaged texture data copied at `TextureInitialize`**: [04, Seen in 13 Sentinels](../../../docs/agents/04-graphics.md#seen-in-13-sentinels). -- **Finding service calls before the first boot** (`main`'s `nn::` imports filtered against `HLE_FORCE` names): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **64 KB-aligned large allocations** (`alloc_big_align`, `hle_os.c`; shared with Unicorn Overlord, [010069401ADB8000](../010069401ADB8000/README.md)): [Ring heaps cut from a pool at 64 KB alignment](../../../docs/reference/memory.md#ring-heaps-cut-from-a-pool-at-64-kb-alignment) and [the profile keys](../README.md#keys). +- **GLSLC outputs in `main`** (`shader_split.py` `glslc_shaders`, the control / code / length check): [GLSLC outputs in `main` and in loose files](../../../docs/reference/shaders.md#glslc-outputs-in-main-and-in-loose-files). +- **Single-stage programs paired with a cover** (`vs_pad`), **a shader cache key that covers the pairs** (`shader_key`, `pair_keyed`), **triangle fans and indexed quads** (a shared fan index buffer; CPU expansion at replay in `nvn_mtl.m`), **vertex streams 14 and 15 at slots 14 and 15** (`VB_SLOT`) and **packaged texture data copied at `TextureInitialize`**: [Single-stage programs paired with a cover](../../../docs/reference/shaders.md#single-stage-programs-paired-with-a-cover), [Shader cache key covering the pairs](../../../docs/reference/shaders.md#shader-cache-key-covering-the-pairs), [Triangle fans and indexed quads](../../../docs/reference/rendering.md#triangle-fans-and-indexed-quads), [Vertex streams 14 and 15 at Metal slots 14 and 15](../../../docs/reference/rendering.md#vertex-streams-14-and-15-at-metal-slots-14-and-15), [Packaged texture data copied at `TextureInitialize`](../../../docs/reference/nvn-objects.md#packaged-texture-data-copied-at-textureinitialize). +- **Finding service calls before the first boot** (`main`'s `nn::` imports filtered against `HLE_FORCE` names): [Listing a game's unhooked imports before they are met](../../../docs/reference/services.md#listing-a-games-unhooked-imports-before-they-are-met). ## Open diff --git a/dm/games/01004A4010FEA000/README.md b/dm/games/01004A4010FEA000/README.md index e902d13..abe3d8d 100644 --- a/dm/games/01004A4010FEA000/README.md +++ b/dm/games/01004A4010FEA000/README.md @@ -42,9 +42,9 @@ Update 1.2.0 (v131072) over the base game, SDK 13.3.2 (no other game's `sdk` bui General host changes first needed here; each applies to every game: -- **The NVN layer's newest GPU code version follows the SDK** (1.15 from SDK 8, 1.16 from SDK 13, `hle_nvn.c`), `nvnWindowGetCrop` and `nvnSamplerBuilderGetLodClamp`: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **The NVN layer's newest GPU code version follows the SDK** (1.15 from SDK 8, 1.16 from SDK 13, `hle_nvn.c`), `nvnWindowGetCrop` and `nvnSamplerBuilderGetLodClamp`: [Newest GPU code version reported by the driver](../../../docs/reference/nvn-objects.md#newest-gpu-code-version-reported-by-the-driver), [NVN calls that were red stubs](../../../docs/reference/nvn-objects.md#nvn-calls-that-were-red-stubs). - **Oodle's streaming decoder in the Mac host's tool mode** (`hle_oodle.c`, `OodleLZDecoder_*`, `oodle_packs`) and the **PKZ / DAT readers**: [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). -- **PlatinumGames shader objects** (`pg_shader`): [04, Seen in Bayonetta 3](../../../docs/agents/04-graphics.md#seen-in-bayonetta-3). +- **PlatinumGames shader objects** (`pg_shader`): [PlatinumGames' shader objects](../../../docs/reference/shaders.md#platinumgames-shader-objects). ## Open diff --git a/dm/games/01004AB00A260000/README.md b/dm/games/01004AB00A260000/README.md index 0e36849..ec609a2 100644 --- a/dm/games/01004AB00A260000/README.md +++ b/dm/games/01004AB00A260000/README.md @@ -66,14 +66,14 @@ SDK 4.5.0 boots, one each: General host changes first needed here; each applies to every game: -- **SDK 4's GLSLC outputs read through their section headers** (`glslc_section`) and **uniform-buffer handles resolved against the bound pools** (`OP_SET_TEX_POOL` / `OP_SET_SAMP_POOL`, `ubo_tex_id` / `ubo_samp_id`, `cmd_t` `tex_base` / `samp_base`): [04, Seen in Dark Souls Remastered](../../../docs/agents/04-graphics.md#seen-in-dark-souls-remastered). -- **The dvdbnd / DCX / BND3 reader** (`bdt_shaders`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **SDK 4's movie decoder constructor and its buffer without a sync object** (`hle_movie.c`), **`friends::GetBlockedUserList` without a `Uid`**, **the UTF-16 profanity masking by name** (`hle_misc.c`), and **`Poll` / `Select` with nothing to wait on sleeping out their timeout** (`hle_net.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **The real-time declaration covering the window thread's peak frames** (`rt_comp_tick` in `hle_nvn.c`): [03, Frame rate](../../../docs/agents/03-bringup.md#frame-rate). +- **SDK 4's GLSLC outputs read through their section headers** (`glslc_section`) and **uniform-buffer handles resolved against the bound pools** (`OP_SET_TEX_POOL` / `OP_SET_SAMP_POOL`, `ubo_tex_id` / `ubo_samp_id`, `cmd_t` `tex_base` / `samp_base`): [GLSLC outputs read through their section headers](../../../docs/reference/shaders.md#glslc-outputs-read-through-their-section-headers), [Handles from uniform buffers take the bound pools' bases](../../../docs/reference/shaders.md#handles-from-uniform-buffers-take-the-bound-pools-bases), [Bases of bound handles kept in the command buffer](../../../docs/reference/shaders.md#bases-of-bound-handles-kept-in-the-command-buffer). +- **The dvdbnd / DCX / BND3 reader** (`bdt_shaders`): [GLSLC outputs in FromSoftware's BND3 and dvdbnd archives](../../../docs/reference/shaders.md#glslc-outputs-in-fromsoftwares-bnd3-and-dvdbnd-archives). +- **SDK 4's movie decoder constructor and its buffer without a sync object** (`hle_movie.c`), **`friends::GetBlockedUserList` without a `Uid`**, **the UTF-16 profanity masking by name** (`hle_misc.c`), and **`Poll` / `Select` with nothing to wait on sleeping out their timeout** (`hle_net.c`): [SDK 4's `movie::Decoder` constructor without a priority](../../../docs/reference/movies.md#sdk-4s-moviedecoder-constructor-without-a-priority), [SDK 4's `movie::Buffer` has no sync object](../../../docs/reference/movies.md#sdk-4s-moviebuffer-has-no-sync-object), [SDK 4's `friends::GetBlockedUserList` without a `Uid`](../../../docs/reference/services.md#sdk-4s-friendsgetblockeduserlist-without-a-uid), [`nn::ngc::ProfanityFilter` vtable and by-name overloads](../../../docs/reference/services.md#nnngcprofanityfilter-vtable-and-by-name-overloads), [Socket `Poll` and `Select` with nothing to wait on](../../../docs/reference/services.md#socket-poll-and-select-with-nothing-to-wait-on). +- **The real-time declaration covering the window thread's peak frames** (`rt_comp_tick` in `hle_nvn.c`): [Time-constraint computation declared below the thread's peaks](../../../docs/reference/frame-rate.md#time-constraint-computation-declared-below-the-threads-peaks). ## Open - Climbing ladders still misplays at 60 fps (with the community's cheat too). - Once, loading a save on the iPhone stayed on the loading screen (no save write, no red line); the next launch loaded it, and it has not come back. - SDK 4.5.0 is called "the oldest so far" in `game.tsv`, the bring-up notes and `AGENTS.md`, while Mario Kart 8 Deluxe ([0100152000022000](../0100152000022000/README.md)) is SDK 4.4.0, also called the oldest. -- [04](../../../docs/agents/04-graphics.md#seen-in-dark-souls-remastered) speaks of `main`'s 72 GLSLC outputs (44 of which failed the old check), `game.tsv` of 59 shaders in `main`; the sources do not say how the two counts relate. +- [shaders.md](../../../docs/reference/shaders.md#glslc-outputs-read-through-their-section-headers) speaks of `main`'s 72 GLSLC outputs (44 of which failed the old check), `game.tsv` of 59 shaders in `main`; the sources do not say how the two counts relate. diff --git a/dm/games/010051701FB46000/README.md b/dm/games/010051701FB46000/README.md index cb97f6e..ca44d23 100644 --- a/dm/games/010051701FB46000/README.md +++ b/dm/games/010051701FB46000/README.md @@ -64,11 +64,11 @@ Update 1.1.1 (v131072) over the base game, SDK 18.3.5. Unity 2022.3.54f1 IL2CPP General host changes first needed here; each applies to every game: -- **`svcGetInfo` type 24, FreeThreadCount** (`thr_limit_used`): [03, Threads](../../../docs/agents/03-bringup.md#threads). -- **Movie decoder output slots per decoder kind** (`MD_NOUT`, `MD_NOUT_AAC`, `MD_NOUT_VIDEO`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **`svcGetInfo` type 24, FreeThreadCount** (`thr_limit_used`): [Free thread count (`svcGetInfo` type 24)](../../../docs/reference/threads.md#free-thread-count-svcgetinfo-type-24). +- **Movie decoder output slots per decoder kind** (`MD_NOUT`, `MD_NOUT_AAC`, `MD_NOUT_VIDEO`): [Output slot counts per decoder kind](../../../docs/reference/movies.md#output-slot-counts-per-decoder-kind), [Audio output slots for players that hold every output](../../../docs/reference/movies.md#audio-output-slots-for-players-that-hold-every-output). - **`nvnSyncSetDebugLabel`** as a no-op (no other source describes it). -- **`ReadFile` re-prepared on EFAULT** against the dirty-page tracker (`h_fs_ReadFile`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Unity `ComputeShader` objects** (`unity_compute`, `COMPUTE_2022_3`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **`ReadFile` re-prepared on EFAULT** against the dirty-page tracker (`h_fs_ReadFile`): [ReadFile into write-protected texture pool pages](../../../docs/reference/services.md#readfile-into-write-protected-texture-pool-pages). +- **Unity `ComputeShader` objects** (`unity_compute`, `COMPUTE_2022_3`): [Unity ComputeShader objects](../../../docs/reference/shaders.md#unity-computeshader-objects). - **`present_interval display`** for a game that steps by frame time: [dm/games/README.md, Keys](../README.md#keys). - The spin-wait search takes the movie library's thread loop wider in this game's `subsdk0`: [development.md, Power](../../../docs/development.md#power). diff --git a/dm/games/01005270232F2000/README.md b/dm/games/01005270232F2000/README.md index 6b263a1..7941888 100644 --- a/dm/games/01005270232F2000/README.md +++ b/dm/games/01005270232F2000/README.md @@ -78,13 +78,13 @@ No `pad_layout` key: the iPhone gamepad starts in the 2D layout ([README](../../ General host changes first needed here; each applies to every game (or every Unreal game): -- **Unreal's allocator replaced by the host's** (`hle_uemalloc.c`, a rewriter cut of the constructor), explained in [Memory](#memory) above; the later Unreal builds and their vtable layouts: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). +- **Unreal's allocator replaced by the host's** (`hle_uemalloc.c`, a rewriter cut of the constructor), explained in [Memory](#memory) above; the later Unreal builds and their vtable layouts: [Unreal's FMallocBinned3 replaced by the host allocator](../../../docs/reference/memory.md#unreals-fmallocbinned3-replaced-by-the-host-allocator). - **Unreal's frame limits written in game memory** (`hle_fpslock.c`, `fps_target`), explained in [Frame rate](#frame-rate) above. -- **Game arguments set before `nninitStartup`** (`main.c`, profile key `game_args`): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **Unreal pak reading and Oodle-compressed shaders decoded by the game's own Oodle** (`uepak.py`, `DM_OODLE_JOB`, `decode_ue_shaders`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content), [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **Texture slots picked at run time** (`tex_dyn_slots`, `NVN_RB_TEXD` / `NVN_RB_SAMPD`), **image atomics bound to the image's storage** (`NVN_RB_IMGA`, `atom_scratch`), **buffer textures below 16 bytes**: [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). -- **Rewriter:** byte tables far from their `br` (`jump_bytes_far`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). Not first found here but met here: long functions analysed in pieces (`pieces`, same section) and spin waits that store (`patch_nso.py` "Spin waits", [development.md, Power](../../../docs/development.md#power)). -- **Undefined weak imports bound to 0** (`loader.c`), **`vi::GetLatestFrameNumber`** (`hle_vi.c`), **`fs::GetSaveDataSizeMax`** (`hle_fs.c`), **NVN pool getters and `nvnCommandBufferFenceSync`** (`OP_FENCE_SYNC`), **the guest clock left running behind the software keyboard** (`hle_swkbd.c`, `front_ios.m`, `guest_clock_freeze_for`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Game arguments set before `nninitStartup`** (`main.c`, profile key `game_args`): [Unreal's engine address region and its options](../../../docs/reference/memory.md#unreals-engine-address-region-and-its-options). +- **Unreal pak reading and Oodle-compressed shaders decoded by the game's own Oodle** (`uepak.py`, `DM_OODLE_JOB`, `decode_ue_shaders`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content), [Unreal Engine 5 shader code archives (Oodle)](../../../docs/reference/shaders.md#unreal-engine-5-shader-code-archives-oodle). +- **Texture slots picked at run time** (`tex_dyn_slots`, `NVN_RB_TEXD` / `NVN_RB_SAMPD`), **image atomics bound to the image's storage** (`NVN_RB_IMGA`, `atom_scratch`), **buffer textures below 16 bytes**: [Texture slots picked at run time](../../../docs/reference/shaders.md#texture-slots-picked-at-run-time), [Image atomics](../../../docs/reference/shaders.md#image-atomics), [Buffer textures at offsets below 16 bytes](../../../docs/reference/rendering.md#buffer-textures-at-offsets-below-16-bytes). +- **Rewriter:** byte tables far from their `br` (`jump_bytes_far`): [Byte tables whose add is far from the br](../../../docs/reference/rewriter.md#byte-tables-whose-add-is-far-from-the-br). Not first found here but met here: long functions analysed in pieces (`pieces`, [Long functions analysed in pieces](../../../docs/reference/rewriter.md#long-functions-analysed-in-pieces)) and spin waits that store (`patch_nso.py` "Spin waits", [development.md, Power](../../../docs/development.md#power)). +- **Undefined weak imports bound to 0** (`loader.c`), **`vi::GetLatestFrameNumber`** (`hle_vi.c`), **`fs::GetSaveDataSizeMax`** (`hle_fs.c`), **NVN pool getters and `nvnCommandBufferFenceSync`** (`OP_FENCE_SYNC`), **the guest clock left running behind the software keyboard** (`hle_swkbd.c`, `front_ios.m`, `guest_clock_freeze_for`): [Undefined weak imports bind to 0](../../../docs/reference/services.md#undefined-weak-imports-bind-to-0), [Layer frame numbers](../../../docs/reference/services.md#layer-frame-numbers), [Save data size queries](../../../docs/reference/services.md#save-data-size-queries), [Pool size getters, texture address and command memory size](../../../docs/reference/nvn-objects.md#pool-size-getters-texture-address-and-command-memory-size), [Fence syncs recorded in command buffers](../../../docs/reference/nvn-objects.md#fence-syncs-recorded-in-command-buffers), [The software keyboard blocks only its caller](../../../docs/reference/services.md#the-software-keyboard-blocks-only-its-caller). - **`DM_ALIAS_WINDOWS`**, the alias-region window census: [05, Memory growth](../../../docs/agents/05-debugging.md#memory-growth). ## Open diff --git a/dm/games/0100535012974000/README.md b/dm/games/0100535012974000/README.md index 90fac01..0c6bab3 100644 --- a/dm/games/0100535012974000/README.md +++ b/dm/games/0100535012974000/README.md @@ -49,4 +49,4 @@ General host changes first needed here; each applies to every game: ## Open - The memory partition is Link's Awakening's, not measured for this game (`game.tsv`). -- Online SDKs in NROs: always load them, refuse them (as now) or decide per profile is an open question for the maintainer ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk), the Absolum entry). +- Online SDKs in NROs: always load them, refuse them (as now) or decide per profile is an open question for the maintainer ([Online SDKs in NROs are never loaded](../../../docs/reference/services.md#online-sdks-in-nros-are-never-loaded)). diff --git a/dm/games/010055D009F78000/README.md b/dm/games/010055D009F78000/README.md index 7af4613..879b35c 100644 --- a/dm/games/010055D009F78000/README.md +++ b/dm/games/010055D009F78000/README.md @@ -56,7 +56,7 @@ Game-specific findings. The general host changes they led to are listed under [F - RomFS: the LINKDATA archive (`DATA0.bin` is an index of 32-byte entries `{u64 offset, u64 length, u64 stored length, u64 compressed}` into `DATA1.bin`, 6.9 GB), loose overrides in `patch1`–`patch4` (listed by `patchN/INFO0.bin`, 0x120-byte entries ending in a `rom:/patchN/…` path), WebM movies in `nx/movie` and KTSL sound banks (`*.ktsl2stbin`). Compressed entries and `*.gz` files are Koei Tecmo's chunked zlib (`kt_gz`; [01](../../../docs/agents/01-extract.md#compressed-romfs-content)). - Shaders: 1,364. `shader_split.py` (`kt_shaders`) reads the SPKG packages (G2S containers, as in Age of Calamity) in `DATA1.bin` (132 entries) and `patch*/nx/shader`, a `_SPK4100` post-effect package without G2S (`DATA1.bin` entry 549), map and model archives carrying packages of their own, and the BNSH and GLSLC outputs in `main`. `DATA1.bin` is split into 64 scan jobs by entry range (`linkdata_jobs`). -- Its GLSLC (output version 0x1d.2) writes 0x848 / 0x850-byte control sections and compute code with no header, which the reader used to drop: 24 shaders before, 1,355 after. A few outputs of version 0x1d.0 keep something else at control +0x20 (odd values such as 0x136b), so a control length must be a multiple of 4; `glslc_section` later took 8 more of them ([04, Seen in Dark Souls Remastered](../../../docs/agents/04-graphics.md#seen-in-dark-souls-remastered)). +- Its GLSLC (output version 0x1d.2) writes 0x848 / 0x850-byte control sections and compute code with no header, which the reader used to drop: 24 shaders before, 1,355 after. A few outputs of version 0x1d.0 keep something else at control +0x20 (odd values such as 0x136b), so a control length must be a multiple of 4; `glslc_section` later took 8 more of them ([GLSLC outputs read through their section headers](../../../docs/reference/shaders.md#glslc-outputs-read-through-their-section-headers)). - **Shaders compiled at run time:** KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (`glslcCompile` from `main+0x9c89f8`; the sources are strings in `main`): 2D primitives, text, the loading quill. No game file holds their output, so they are captured on the Mac: the game runs with `DM_GLSLC_CAPTURE=/glslc_capture` (`hle_glslc.c`), and `shader_split.py` reads that folder. The outputs are identical on every run. The NSP script does this by itself in its `GLSLC capture` stage, so the NSPs and prod.keys are enough: it builds the Mac host into `_build/tools/` and boots the game until the 9 files are in `_build/extract/v327680/glslc_capture`. Without the folder the split tables would differ from the profile, and the script would stop at its table check. By hand, run the game for a few seconds with that variable (step 5 of the build by hand). ## Mac routes @@ -69,11 +69,11 @@ Game-specific findings. The general host changes they led to are listed under [F General host changes first needed here; each applies to every game: -- **Submissions serialized across queues** (`g_sub_mu`) and **`GetCommandMemoryUsed` counted from the last `AddCommandMemory`**: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **SDK 6:** `account::OpenPreselectedUser` returning a Result, SDK 6's DSP command header (`capture` in `hle_audren.c`) and **run-time GLSLC capture** (`DM_GLSLC_CAPTURE`, `hle_glslc.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk); the NSP script's capture stage: [development.md](../../../docs/development.md#the-packing-script-in-detail). -- **Overlapping texture / sampler pool IDs** kept apart (`bind_handles`, pool base in `cmd_t`), **integer vertex inputs fed a float-type format** (`vconv_bind`) and **other-signedness integer formats** (`vconv_int`, since replaced by shadow vertex inputs), **depth clamp** (`OP_DEPTH_CLAMP`), **`nvnQueueFinish` waits only for the GPU** (`mtl_finish`): [04, Seen in Fire Emblem: Three Houses](../../../docs/agents/04-graphics.md#seen-in-fire-emblem-three-houses). -- **Row wrap** and **shadow vertex inputs**: [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). -- **GLSLC 0x848 / 0x850 controls and headerless compute code** (`glslc_shaders`) and the **SPKG / LINKDATA reader** (`kt_shaders`): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **Submissions serialized across queues** (`g_sub_mu`) and **`GetCommandMemoryUsed` counted from the last `AddCommandMemory`**: [Submissions from two queues on two threads](../../../docs/reference/nvn-objects.md#submissions-from-two-queues-on-two-threads), [What `GetCommandMemoryUsed` counts](../../../docs/reference/nvn-objects.md#what-getcommandmemoryused-counts). +- **SDK 6:** `account::OpenPreselectedUser` returning a Result, SDK 6's DSP command header (`capture` in `hle_audren.c`) and **run-time GLSLC capture** (`DM_GLSLC_CAPTURE`, `hle_glslc.c`): [SDK 6's `account::OpenPreselectedUser` returns a Result](../../../docs/reference/services.md#sdk-6s-accountopenpreselecteduser-returns-a-result), [SDK 6: the DSP command header layout](../../../docs/reference/audio.md#sdk-6-the-dsp-command-header-layout), [Built-in shaders compiled with the game's own GLSLC](../../../docs/reference/shaders.md#built-in-shaders-compiled-with-the-games-own-glslc); the NSP script's capture stage: [development.md](../../../docs/development.md#the-packing-script-in-detail). +- **Overlapping texture / sampler pool IDs** kept apart (`bind_handles`, pool base in `cmd_t`), **integer vertex inputs fed a float-type format** (`vconv_bind`) and **other-signedness integer formats** (`vconv_int`, since replaced by shadow vertex inputs), **depth clamp** (`OP_DEPTH_CLAMP`), **`nvnQueueFinish` waits only for the GPU** (`mtl_finish`): [Texture and sampler pools with overlapping IDs](../../../docs/reference/shaders.md#texture-and-sampler-pools-with-overlapping-ids), [CPU conversion of vertex attributes the input type cannot take](../../../docs/reference/rendering.md#cpu-conversion-of-vertex-attributes-the-input-type-cannot-take), [Depth clamp](../../../docs/reference/rendering.md#depth-clamp), [`nvnQueueFinish` waits for the GPU, not the display](../../../docs/reference/nvn-objects.md#nvnqueuefinish-waits-for-the-gpu-not-the-display). +- **Row wrap** and **shadow vertex inputs**: [Buffer textures read by 1D integer fetches (row wrap)](../../../docs/reference/rendering.md#buffer-textures-read-by-1d-integer-fetches-row-wrap), [Shadow vertex inputs](../../../docs/reference/rendering.md#shadow-vertex-inputs). +- **GLSLC 0x848 / 0x850 controls and headerless compute code** (`glslc_shaders`) and the **SPKG / LINKDATA reader** (`kt_shaders`): [SPKG packages in KTGL LINKDATA](../../../docs/reference/shaders.md#spkg-packages-in-ktgl-linkdata). - **Yield back-off** and the **yield-progress rule** (`thr_yield`; this game's limiter is a poll that keeps its registers): [development.md, Power](../../../docs/development.md#power). ## Open diff --git a/dm/games/0100606026F12000/README.md b/dm/games/0100606026F12000/README.md index 910ce2e..391b45f 100644 --- a/dm/games/0100606026F12000/README.md +++ b/dm/games/0100606026F12000/README.md @@ -61,9 +61,9 @@ Status: brought up on the Mac: the first-boot brightness screen, the title, a ne General host changes first needed here; each applies to every game: -- **UE 5.6's 32-slot FMalloc** (`hle_uemalloc.c` layout `UE5_6`): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **IoStore shader libraries read through the game's own Oodle with no boot** (`uepak.py` `Toc`, `io_archive`, `OodleHost`, `DM_OODLE_SERVE`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content), [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **Rewriter:** text before the first FDE as one function (`func_of`) and long functions analysed in pieces (`pieces`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). +- **UE 5.6's 32-slot FMalloc** (`hle_uemalloc.c` layout `UE5_6`): [Unreal's FMallocBinned3 replaced by the host allocator](../../../docs/reference/memory.md#unreals-fmallocbinned3-replaced-by-the-host-allocator). +- **IoStore shader libraries read through the game's own Oodle with no boot** (`uepak.py` `Toc`, `io_archive`, `OodleHost`, `DM_OODLE_SERVE`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content), [Unreal Engine 5.6 IoStore shader libraries](../../../docs/reference/shaders.md#unreal-engine-56-iostore-shader-libraries). +- **Rewriter:** text before the first FDE as one function (`func_of`) and long functions analysed in pieces (`pieces`): [Code before the first unwind entry is one function](../../../docs/reference/rewriter.md#code-before-the-first-unwind-entry-is-one-function), [Long functions analysed in pieces](../../../docs/reference/rewriter.md#long-functions-analysed-in-pieces). ## Open diff --git a/dm/games/0100620012D6E000/README.md b/dm/games/0100620012D6E000/README.md index 29604cf..71cb9e5 100644 --- a/dm/games/0100620012D6E000/README.md +++ b/dm/games/0100620012D6E000/README.md @@ -30,8 +30,8 @@ No `patches.py`, and `dm/tools/sdk_patches.py` has no cut for its `sdk` build (` ## Audio -- Its `AudioRenderSystem::Update` lock is `SdkMutexType::Lock` at +0x1d8 (SDK 10.4–11.4); `update_lock` (`hle_audren.c`) was checked with sound on it ([03, Audio](../../../docs/agents/03-bringup.md#audio)). -- **Resolved:** when queued catch-up frames still signalled the renderer event, every frame of this game became a catch-up frame (about 38,000 catch-up renders instead of 37). A queued frame no longer signals the event ([03, Audio](../../../docs/agents/03-bringup.md#audio)). +- Its `AudioRenderSystem::Update` lock is `SdkMutexType::Lock` at +0x1d8 (SDK 10.4–11.4); `update_lock` (`hle_audren.c`) was checked with sound on it ([The renderer's update lock](../../../docs/reference/audio.md#the-renderers-update-lock-update_lock)). +- **Resolved:** when queued catch-up frames still signalled the renderer event, every frame of this game became a catch-up frame (about 38,000 catch-up renders instead of 37). A queued frame no longer signals the event ([A sound driver that never waits on the renderer event](../../../docs/reference/audio.md#a-sound-driver-that-never-waits-on-the-renderer-event)). ## Movies @@ -41,7 +41,7 @@ No `patches.py`, and `dm/tools/sdk_patches.py` has no cut for its `sdk` build (` ## Services -SDK 11.4.3 calls it needed ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)): +SDK 11.4.3 calls it needed ([Hardware Opus decoders](../../../docs/reference/audio.md#hardware-opus-decoders-built-on-the-sdks-software-decoders), [Directory, file data cache and save-thumbnail calls](../../../docs/reference/services.md#directory-file-data-cache-and-save-thumbnail-calls), the rest in [services.md](../../../docs/reference/services.md)): - **Abort in `sf::hipc::InitializeHipcServiceResolution` under `codec::GetHardwareOpusDecoderWorkBufferSize(int, int)`** (`main+0x31f8e8`, early in `nnMain`): SDK 11's form without the option argument; the host had only the three-argument one. - From the import list before the next boot (`main`'s undefined `nn::` symbols no hook names): `fs::EnableIndividualFileDataCache` / `DisableIndividualFileDataCache`; `account::LoadCachedNintendoAccountInfo` (fails: no Nintendo Account; the game then skips the birthday and country getters); the IR camera (`irsensor`: every camera unconnected, processors cannot run; Bezel's input layer makes one per controller and asks its status, `main+0x497f84`); `ldn::GetNetworkConfig`; and SDK 11's socket overloads with plain `sockaddr` / `int` arguments (`Connect`, `Accept`, `GetSockName`, `Get` / `SetSockOpt`, `Fcntl`, `Recv`, `Send`). @@ -65,10 +65,10 @@ SDK 11.4.3 calls it needed ([03, Services and SDK](../../../docs/agents/03-bring General host changes first needed here; each applies to every game: -- **`sys_resource`**: the host reports the NPDM's system resource instead of 32 MB for every game: [dm/games/README.md, Keys](../README.md#keys), [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Movie decoder Flush from several threads** (a send claims an index a flush returned, `pend[]` cannot overflow, an index held across a flush is withheld for 250 ms; a flush keeps each slot's last picture, and in native-texture mode an output with no picture leaves the texture as it was; `hle_movie.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **`sys_resource`**: the host reports the NPDM's system resource instead of 32 MB for every game: [dm/games/README.md, Keys](../README.md#keys), [Virtual address memory follows the NPDM's system resource](../../../docs/reference/memory.md#virtual-address-memory-follows-the-npdms-system-resource). +- **Movie decoder Flush from several threads** (a send claims an index a flush returned, `pend[]` cannot overflow, an index held across a flush is withheld for 250 ms; a flush keeps each slot's last picture, and in native-texture mode an output with no picture leaves the texture as it was; `hle_movie.c`): [Input indices returned by a flush while a client holds them](../../../docs/reference/movies.md#input-indices-returned-by-a-flush-while-a-client-holds-them). - **SIGABRT caught by the crash handler**: before it was, this game died once with exit 134, nothing in the log and no report: [05, Reading a crash report](../../../docs/agents/05-debugging.md#reading-a-crash-report). -- **BNSH in Bezel Engine archives** (`bea_members`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **BNSH in Bezel Engine archives** (`bea_members`): [BNSH in Bezel Engine archives](../../../docs/reference/shaders.md#bnsh-in-bezel-engine-archives). - The spin-wait search's store rule takes one more loop in its `main`, a real flag wait (`main+0x1734e0`): [development.md, Power](../../../docs/development.md#power). ## Open diff --git a/dm/games/010069401ADB8000/README.md b/dm/games/010069401ADB8000/README.md index 722b50c..76990d5 100644 --- a/dm/games/010069401ADB8000/README.md +++ b/dm/games/010069401ADB8000/README.md @@ -25,7 +25,7 @@ No `patches.py`, no `pad_layout` (the iPhone pad starts in the 2D layout). ## Audio and services -Each found by one boot ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)): +Each found by one boot ([Hardware Opus decoders built on the SDK's software decoders](../../../docs/reference/audio.md#hardware-opus-decoders-built-on-the-sdks-software-decoders), [Cryptographically random bytes](../../../docs/reference/services.md#cryptographically-random-bytes)): - **Resolved: abort in `sm::Initialize` under `nn::codec::GetHardwareOpusMultiStreamDecoderWorkBufferSize(int, int, int, int, int)`** (at `nnMain`, CRI's Opus streams). The host now has the C interface of the hardware multi-stream decoder, including the SDK 12+ overload with an option argument (Age of Calamity only needed the class and the four-argument size). Each call goes to the `sdk`'s software `OpusMultiStreamDecoder` C interface, with a host-held state per game object as for the single-stream decoder (`hle_codec.c`). `GetHardwareOpusMultiStreamDecoderProcessingTime` reports 0. - **Resolved: abort in `sm::Initialize` under `nn::crypto::GenerateCryptographicallyRandomBytes`** (`std::random_device` → the `sdk`'s own `getentropy` → `InitializeCsrng`). The `sdk` defines `getentropy`, so the host's `getentropy` hook never sees the call, but the call into `nn::crypto` goes through the `sdk`'s PLT. An `HLE_FORCE` on it fills from the host's entropy stream (`hle_os.c`). @@ -49,7 +49,7 @@ Each found by one boot ([03, Services and SDK](../../../docs/agents/03-bringup.m General host changes first needed here; each applies to every game: - `alloc_big_align` was first needed on 13 Sentinels ([its note](../01003FC01670C000/README.md#memory), [the profile keys](../README.md#keys)). -- **The hardware multi-stream Opus decoder's C interface** forwarded to the SDK's software decoder (`hle_codec.c`) and **`GenerateCryptographicallyRandomBytes` under the `sdk`'s own `getentropy`** (`HLE_FORCE`, `hle_os.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **The hardware multi-stream Opus decoder's C interface** forwarded to the SDK's software decoder (`hle_codec.c`) and **`GenerateCryptographicallyRandomBytes` under the `sdk`'s own `getentropy`** (`HLE_FORCE`, `hle_os.c`): [Hardware Opus decoders built on the SDK's software decoders](../../../docs/reference/audio.md#hardware-opus-decoders-built-on-the-sdks-software-decoders), [Cryptographically random bytes](../../../docs/reference/services.md#cryptographically-random-bytes). ## Open diff --git a/dm/games/01006A800016E000/README.md b/dm/games/01006A800016E000/README.md index 4f60a15..1e0c41c 100644 --- a/dm/games/01006A800016E000/README.md +++ b/dm/games/01006A800016E000/README.md @@ -53,10 +53,10 @@ No `pad_layout` key (the iPhone pad starts in the 3D layout). General host changes first needed here; each applies to every game: -- **NRO modules loaded at run time:** collected by the NRRs' SHA-256s (`nro_collect.py`, `nro.tsv`), rewritten (`patch_nso.py` reads NROs), mapped at start and activated by `nn::ro::LoadModule` / `UnloadModule` (`loader.c` "NRO modules", `hle_ro.c`; `DM_NRO=0` turns it off), embedded on iOS (`gen_embed.py`, `nro_pieces`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk), [development.md, The packing script in detail](../../../docs/development.md#the-packing-script-in-detail). -- **Register copies followed back to a jump table's base** (`copied_base`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **`nvnSetExternalDebuggerEnabledNVX`** as a no-op (`names.txt`, `NVN_NOPROTO`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **The residency set committed at present** (`present_now` → `res_commit`): [04, Seen in Super Smash Bros. Ultimate](../../../docs/agents/04-graphics.md#seen-in-super-smash-bros-ultimate). +- **NRO modules loaded at run time:** collected by the NRRs' SHA-256s (`nro_collect.py`, `nro.tsv`), rewritten (`patch_nso.py` reads NROs), mapped at start and activated by `nn::ro::LoadModule` / `UnloadModule` (`loader.c` "NRO modules", `hle_ro.c`; `DM_NRO=0` turns it off), embedded on iOS (`gen_embed.py`, `nro_pieces`): [Loading NROs with `ro::LoadModule`](../../../docs/reference/services.md#loading-nros-with-roloadmodule), [development.md, The packing script in detail](../../../docs/development.md#the-packing-script-in-detail). +- **Register copies followed back to a jump table's base** (`copied_base`): [Far table base passed through other registers](../../../docs/reference/rewriter.md#far-table-base-passed-through-other-registers). +- **`nvnSetExternalDebuggerEnabledNVX`** as a no-op (`names.txt`, `NVN_NOPROTO`): [Extensions fetched by name and called unchecked](../../../docs/reference/nvn-objects.md#extensions-fetched-by-name-and-called-unchecked). +- **The residency set committed at present** (`present_now` → `res_commit`): [The residency set committed before a present](../../../docs/reference/rendering.md#the-residency-set-committed-before-a-present). - **The indexed archive reader** (`smash_arc.py`, `smash_arc_jobs`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). ## Open diff --git a/dm/games/01006BB00C6F0000/README.md b/dm/games/01006BB00C6F0000/README.md index 55bb61e..81de260 100644 --- a/dm/games/01006BB00C6F0000/README.md +++ b/dm/games/01006BB00C6F0000/README.md @@ -61,12 +61,12 @@ Game-specific findings; the general host changes they led to are listed under [F General host changes first needed here; each applies to every game: -- **Lower-left window origin** (`g_origin_ll`), **copies drawn as passes** (`copy_t2t_draw`), **compute `c0` with the dispatch size** (`encode_args_`, `nvn_refl.m`), **stencil in depth-stencil copies**: [04, Seen in Link's Awakening](../../../docs/agents/04-graphics.md#seen-in-the-legend-of-zelda-links-awakening). +- **Lower-left window origin** (`g_origin_ll`), **copies drawn as passes** (`copy_t2t_draw`), **compute `c0` with the dispatch size** (`encode_args_`, `nvn_refl.m`), **stencil in depth-stencil copies**: [Lower-left window origin](../../../docs/reference/rendering.md#lower-left-window-origin), [Copies a blit cannot do, drawn](../../../docs/reference/rendering.md#copies-a-blit-cannot-do-drawn), [Compute `c0`: dispatch size and uniform buffer table](../../../docs/reference/shaders.md#compute-c0-dispatch-size-and-uniform-buffer-table), [Depth-stencil copies carry the stencil](../../../docs/reference/rendering.md#depth-stencil-copies-carry-the-stencil). - **Per-depth-mode vertex entry points** (`vs_depth`, `main0_z`, `mtl_set_depth_zero`): [04, section 5](../../../docs/agents/04-graphics.md#5-depth-convention). -- **Vertex outputs start at (0, 0, 0, 1)** (`vs_init`): [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). +- **Vertex outputs start at (0, 0, 0, 1)** (`vs_init`): [Unwritten components of a written output start as (0, 0, 0, 1)](../../../docs/reference/shaders.md#unwritten-components-of-a-written-output-start-as-0-0-0-1). - **`shader_facts_manual.tsv`** for facts the machine code cannot tell: [dm/games/README.md](../README.md). -- **The DeviceSink found by its size** (`capture` in `hle_audren.c`), the SDK 8 parameter size, `oe` notifications, the npad style event: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Jump tables checked on another register** (`jump_offsets`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). +- **The DeviceSink found by its size** (`capture` in `hle_audren.c`), the SDK 8 parameter size, `oe` notifications, the npad style event: [SDK 8: command IDs numbered differently](../../../docs/reference/audio.md#sdk-8-command-ids-numbered-differently), [SDK 8's longer `AudioRendererParameter`](../../../docs/reference/audio.md#sdk-8s-longer-audiorendererparameter), [The oe notification event and its queue](../../../docs/reference/threads.md#the-oe-notification-event-and-its-queue), [Npad style set update event](../../../docs/reference/services.md#npad-style-set-update-event). +- **Jump tables checked on another register** (`jump_offsets`): [Offset jump tables with the bounds check on another register](../../../docs/reference/rewriter.md#offset-jump-tables-with-the-bounds-check-on-another-register). - **Racy crashes under CPU load and observation gates**: [05, Racy crashes](../../../docs/agents/05-debugging.md#racy-crashes-load-trace-gates). ## Open diff --git a/dm/games/01006C300E9F0000/README.md b/dm/games/01006C300E9F0000/README.md index 7c039aa..3bd95f9 100644 --- a/dm/games/01006C300E9F0000/README.md +++ b/dm/games/01006C300E9F0000/README.md @@ -70,14 +70,14 @@ Status: brought up on the Mac through the opening movie, the title (name entry, General host changes first needed here; each applies to every game: -- **Unreal Engine 4.18 shaders inline in the cooked packages** (`ue_legacy_jobs`, `ue_inline_shaders`) and **the game's own LZ5 served in the Mac host's tool mode** (`hle_oodle.c`, `DM_OODLE_SERVE`, `uepak.py` `Pak.read`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content), [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **Slice-writing geometry shaders folded into the vertex shader** (`gs_slice`, `vs_layer_entry`, `prog_fold_gs`, `draw_gs_check`) and **cube map arrays counted in layer-faces** (`nvn_tex.c` `shape`, `tex_fit`): [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). -- **Catch-up frames that wait for queue space** (`hle_audren.c`): [03, Audio](../../../docs/agents/03-bringup.md#audio). -- **Movie decoder events read at signal time** (`md_event`), **no input after end-of-stream** (`in_eos`), **32 audio decoder outputs** (`MD_NOUT`, now 64): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Shop service accessors** (`hle_misc.c`), **`nvnTextureGetTextureAddress` / `nvnCommandBufferGetCommandMemorySize`**: same section. +- **Unreal Engine 4.18 shaders inline in the cooked packages** (`ue_legacy_jobs`, `ue_inline_shaders`) and **the game's own LZ5 served in the Mac host's tool mode** (`hle_oodle.c`, `DM_OODLE_SERVE`, `uepak.py` `Pak.read`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content), [Unreal Engine 4.18 shaders inline in cooked packages](../../../docs/reference/shaders.md#unreal-engine-418-shaders-inline-in-cooked-packages). +- **Slice-writing geometry shaders folded into the vertex shader** (`gs_slice`, `vs_layer_entry`, `prog_fold_gs`, `draw_gs_check`) and **cube map arrays counted in layer-faces** (`nvn_tex.c` `shape`, `tex_fit`): [Geometry shaders that only pick the layer](../../../docs/reference/shaders.md#geometry-shaders-that-only-pick-the-layer), [Cube map arrays count layer-faces](../../../docs/reference/rendering.md#cube-map-arrays-count-layer-faces). +- **Catch-up frames that wait for queue space** (`hle_audren.c`): [A sound driver that never waits on the renderer event](../../../docs/reference/audio.md#a-sound-driver-that-never-waits-on-the-renderer-event). +- **Movie decoder events read at signal time** (`md_event`), **no input after end-of-stream** (`in_eos`), **32 audio decoder outputs** (`MD_NOUT`, now 64): [Client events read through the registered pointer](../../../docs/reference/movies.md#client-events-read-through-the-registered-pointer), [End of stream is the last input until `Flush` or `Stop`](../../../docs/reference/movies.md#end-of-stream-is-the-last-input-until-flush-or-stop), [Audio decoders keep more outputs than a pre-decoding player fills](../../../docs/reference/movies.md#audio-decoders-keep-more-outputs-than-a-pre-decoding-player-fills). +- **Shop service accessors** (`hle_misc.c`), **`nvnTextureGetTextureAddress` / `nvnCommandBufferGetCommandMemorySize`**: [Shop service accessors](../../../docs/reference/services.md#shop-service-accessors), [Pool size getters, texture address and command memory size](../../../docs/reference/nvn-objects.md#pool-size-getters-texture-address-and-command-memory-size). - **Vorbis type-2 residues with more than two channels**: [stb_vorbis README](../../third_party/stb_vorbis/README.md). -- **Rewriter:** an index copied before its bounds check (`check_before_copy`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **The node-area and 32 MB alignment cuts on SDK 7 / 8 builds** (the `orr w9, wzr, #0x3fe00000` form), later taken by Super Smash Bros. Ultimate's and Advance Wars 1+2's `sdk` builds: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). +- **Rewriter:** an index copied before its bounds check (`check_before_copy`): [Index copied before the bounds check](../../../docs/reference/rewriter.md#index-copied-before-the-bounds-check). +- **The node-area and 32 MB alignment cuts on SDK 7 / 8 builds** (the `orr w9, wzr, #0x3fe00000` form), later taken by Super Smash Bros. Ultimate's and Advance Wars 1+2's `sdk` builds: [The SDK's address-region node area](../../../docs/reference/memory.md#the-sdks-address-region-node-area), [1 GB alignment of regions of 1 GB and more](../../../docs/reference/memory.md#1-gb-alignment-of-regions-of-1-gb-and-more). ## Open diff --git a/dm/games/01006F8002326000/README.md b/dm/games/01006F8002326000/README.md index 5b307eb..ed8fa55 100644 --- a/dm/games/01006F8002326000/README.md +++ b/dm/games/01006F8002326000/README.md @@ -55,10 +55,10 @@ On the Mac: a new game through Tom Nook's counter (name entry on the game's own General host changes first needed here; each applies to every game: -- **Halfword jump tables padded to their largest entry** (`half_count` in `patch_nso.py`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **Yields that make progress return at once** (`thr_yield` in `hle_thread.c`): [03, Threads](../../../docs/agents/03-bringup.md#threads) and [development.md, Power](../../../docs/development.md#power) ("Yield loops"). -- **The game's own HIPC clients answered by host sessions** (`hle_sf.c`, `DM_SF_TRACE`), **device save data and shared transactions** (`mount_ex`), **`fs::Commit` with a mount list or a `CommitOption`**, and the adjustable user clock, region code and SDK 20 album and steady-clock calls: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **16 output slots for video movie decoders** (`MD_NOUT_VIDEO` in `hle_movie.c`): same section. +- **Halfword jump tables padded to their largest entry** (`half_count` in `patch_nso.py`): [Halfword jump table landing pads sized to the largest entry](../../../docs/reference/rewriter.md#halfword-jump-table-landing-pads-sized-to-the-largest-entry). +- **Yields that make progress return at once** (`thr_yield` in `hle_thread.c`): [Yields that make progress return at once (`thr_yield`)](../../../docs/reference/threads.md#yields-that-make-progress-return-at-once-thr_yield) and [development.md, Power](../../../docs/development.md#power) ("Yield loops"). +- **The game's own HIPC clients answered by host sessions** (`hle_sf.c`, `DM_SF_TRACE`), **device save data and shared transactions** (`mount_ex`), **`fs::Commit` with a mount list or a `CommitOption`**, and the adjustable user clock, region code and SDK 20 album and steady-clock calls: [Game-linked HIPC clients of system services](../../../docs/reference/services.md#game-linked-hipc-clients-of-system-services), [Device save data on the account save's directory](../../../docs/reference/services.md#device-save-data-on-the-account-saves-directory), [`fs::Commit` with a list of mounts](../../../docs/reference/services.md#fscommit-with-a-list-of-mounts), [Steady clock, clock snapshots and user clocks](../../../docs/reference/services.md#steady-clock-clock-snapshots-and-user-clocks), [Region code from the language](../../../docs/reference/services.md#region-code-from-the-language), [Album calls fail with no album](../../../docs/reference/services.md#album-calls-fail-with-no-album). +- **16 output slots for video movie decoders** (`MD_NOUT_VIDEO` in `hle_movie.c`): [Video output slots: how far the extractor gets ahead](../../../docs/reference/movies.md#video-output-slots-how-far-the-extractor-gets-ahead). ## Open diff --git a/dm/games/01007EF00011E000/README.md b/dm/games/01007EF00011E000/README.md index f7dc542..48bb690 100644 --- a/dm/games/01007EF00011E000/README.md +++ b/dm/games/01007EF00011E000/README.md @@ -109,11 +109,11 @@ Used at `DM_FPS_TARGET=0` and for the interval the framework keeps while the VFR General host changes first needed here; each applies to every game: - **The host frame step in whole vsyncs** (`nvn_fstep.h`, `fstep_tick`, `FRAME_STEP` stubs, `fps_target`), explained in full under [Frame rate](#the-frame-step-current); its probes (`nvn_fstep_gpu_slowed`, `nvn_fstep_same`): [development.md, Power](../../../docs/development.md#power). -- **Real GPU timestamps per segment** (`gpu_timestamps`, `rep_ticks`), **submissions in segments** (`sub_cut`), **deferred texture binds** (`bh_fresh`), **duplicate colour slots** (`rt_dup_slot`), **cube textures as 2D arrays** (`tex_fit`), **compressed render targets through shadows** (`bcrt_attach` / `bcrt_flush`), **pool lookups** (`pool_find`), **memory fixes** (`defer_drain`'s pool, `g_ghost`, growing pool table), **`nvnTextureCompare` with the channel mapping**, **first-use fill of flushed storage** (`nvn_flushed`, `tex_init_fill`), **partial-cover uniform shadows** (`ubp_get`, `cb_extent.tsv`): [04, Seen in Breath of the Wild](../../../docs/agents/04-graphics.md#seen-in-the-legend-of-zelda-breath-of-the-wild). -- **Display-link waits per present interval** (`mtl_link_wait_tick`): [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). +- **Real GPU timestamps per segment** (`gpu_timestamps`, `rep_ticks`), **submissions in segments** (`sub_cut`), **deferred texture binds** (`bh_fresh`), **duplicate colour slots** (`rt_dup_slot`), **cube textures as 2D arrays** (`tex_fit`), **compressed render targets through shadows** (`bcrt_attach` / `bcrt_flush`), **pool lookups** (`pool_find`), **memory fixes** (`defer_drain`'s pool, `g_ghost`, growing pool table), **`nvnTextureCompare` with the channel mapping**, **first-use fill of flushed storage** (`nvn_flushed`, `tex_init_fill`), **partial-cover uniform shadows** (`ubp_get`, `cb_extent.tsv`): [GPU timestamps per submission segment](../../../docs/reference/frame-rate.md#gpu-timestamps-per-submission-segment), [Submissions handed to the GPU in segments](../../../docs/reference/frame-rate.md#submissions-handed-to-the-gpu-in-segments), [Deferred texture binds read at execution](../../../docs/reference/shaders.md#deferred-texture-binds-read-at-execution), [The same texture attached at two colour slots](../../../docs/reference/rendering.md#the-same-texture-attached-at-two-colour-slots), [Cube textures held as 6-layer 2D arrays](../../../docs/reference/rendering.md#cube-textures-held-as-6-layer-2d-arrays), [Compressed textures as render targets through an uncompressed shadow](../../../docs/reference/rendering.md#compressed-textures-as-render-targets-through-an-uncompressed-shadow), [Address to memory pool lookups by binary search](../../../docs/reference/frame-rate.md#address-to-memory-pool-lookups-by-binary-search), [Deferred-release drain without an autorelease pool](../../../docs/reference/memory.md#deferred-release-drain-without-an-autorelease-pool), [Dead texture serials kept in a ghost table](../../../docs/reference/nvn-objects.md#dead-texture-serials-kept-in-a-ghost-table), [Memory-pool tracking table grows in blocks](../../../docs/reference/nvn-objects.md#memory-pool-tracking-table-grows-in-blocks), [`nvnTextureCompare` compares the channel mapping](../../../docs/reference/rendering.md#nvntexturecompare-compares-the-channel-mapping), [First-use fill of storage the game flushed](../../../docs/reference/nvn-objects.md#first-use-fill-of-storage-the-game-flushed), [Partial-cover uniform shadows keep passes open](../../../docs/reference/rendering.md#partial-cover-uniform-shadows-keep-passes-open). +- **Display-link waits per present interval** (`mtl_link_wait_tick`): [Frames of a dynamic present interval held on screen and at the acquire](../../../docs/reference/frame-rate.md#frames-of-a-dynamic-present-interval-held-on-screen-and-at-the-acquire). - **Spin waits that take a spin lock** (`patch_nso.py` "Spin waits", `dm_spin_wait`) and **yields that make progress** (`thr_yield`): [development.md, Power](../../../docs/development.md#power). -- **`DT_RELR`** (`relr_apply`) and **halfword jump tables** (`half_count`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). **NVN calls on the big host stack** (`nvn_big_entry`): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **SDK 20 service calls** and **record sizes against `GetCommandMemoryUsed`** (`draw_rec`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). **SDK 20's audio renderer layout** (`render_once`): [03, Audio](../../../docs/agents/03-bringup.md#audio). **Running a 30 fps game at 60**: [03, Frame rate](../../../docs/agents/03-bringup.md#frame-rate). +- **`DT_RELR`** (`relr_apply`) and **halfword jump tables** (`half_count`): [Packed relative relocations (DT_RELR)](../../../docs/reference/rewriter.md#packed-relative-relocations-dt_relr), [Halfword jump table landing pads sized to the largest entry](../../../docs/reference/rewriter.md#halfword-jump-table-landing-pads-sized-to-the-largest-entry). **NVN calls on the big host stack** (`nvn_big_entry`): [NVN entry points on the big host stack](../../../docs/reference/nvn-objects.md#nvn-entry-points-on-the-big-host-stack). +- **SDK 20 service calls** and **record sizes against `GetCommandMemoryUsed`** (`draw_rec`): [SDK 20 six-axis sensor and amiibo (`nfp`) calls](../../../docs/reference/services.md#sdk-20-six-axis-sensor-and-amiibo-nfp-calls), [Applet resource user ID, launch parameters and VR mode](../../../docs/reference/services.md#applet-resource-user-id-launch-parameters-and-vr-mode), [Shared font load request](../../../docs/reference/services.md#shared-font-load-request), [Model heaps sized by the host's command record sizes](../../../docs/reference/memory.md#model-heaps-sized-by-the-hosts-command-record-sizes). **SDK 20's audio renderer layout** (`render_once`): [SDK 20's `AudioRenderSystem` layout](../../../docs/reference/audio.md#sdk-20s-audiorendersystem-layout). **Running a 30 fps game at 60**: [sead's dynamic present interval, `ksys::VFR` and the host's frame step](../../../docs/reference/frame-rate.md#seads-dynamic-present-interval-ksysvfr-and-the-hosts-frame-step). - **Hitch dumps and guest thread names** (`DM_HITCH_DUMP`), **heap probes** (`DM_PROBE_HEAP*`), **autorelease leaks**: [05, Scripted input and thread scheduling](../../../docs/agents/05-debugging.md#scripted-input-and-thread-scheduling), [05, Memory growth](../../../docs/agents/05-debugging.md#memory-growth). ## Open diff --git a/dm/games/010086A022444000/README.md b/dm/games/010086A022444000/README.md index 4e849f1..5a179eb 100644 --- a/dm/games/010086A022444000/README.md +++ b/dm/games/010086A022444000/README.md @@ -17,7 +17,7 @@ On the Mac: the boot, the title, a new game, the intro and the descent into the - As Hollow Knight: Silksong: 16 MB of system resource, no heap (`main` never sets one), Unity 6 reserving and committing its memory itself through `nn::os::AllocateAddressRegion` and `AllocateMemoryPages` in the alias region, Engage's 5 GB block, and the node-area cut on this `sdk` build. - **Node area cut** on `sdk` build `8bfebccd6f5c1392597b364b9ab22a8d`: its AddressRegionManager constructor is the SDK 20.5.22 build's (Tears of the Kingdom's, [0100F2C0115B6000](../0100F2C0115B6000/README.md)) at the same offsets: the constructor at `sdk+0x145cfc`, the limit at `sdk+0x145d0c`, the occupied length at `sdk+0x145d78` (`mov w8 / w9,#0x3fe00000` → `#0x3e00000`). -- The provisional profile was right as generated; the `sdk` build only needed the cut ([03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space)). +- The provisional profile was right as generated; the `sdk` build only needed the cut ([The SDK's address-region node area](../../../docs/reference/memory.md#the-sdks-address-region-node-area)). - Measured on the Mac over 10 minutes of the first dive and the base: the alias region's high-water mark +2.00 GB of 4.375 GB, 1.26 GB mapped, 1.6 GB resident, 0.7 GB of GPU memory, flat. ## Rendering @@ -28,7 +28,7 @@ On the Mac: the boot, the title, a new game, the intro and the descent into the - 38,099 shaders (18,913 vertex, 18,917 pixel, 116 compute, 76 hull and domain, one geometry), 11 of them the player's own GLSLC outputs in `main`; everything is under `Data/` (`shader_groups /Data/=data`). - Addressables: 11 bundles in `StreamingAssets/aa/Switch`; FMOD banks in `StreamingAssets` (one stereo AudioOut). -- The loose serialized files (`resources.assets`, `sharedassets1.assets`, the built-in resources) have no type trees. They borrow the `Shader` tree that 10 of the bundles carry (the same type hash; `shader_split.py` defers a file that raises `NoTree` to a second pass), so `unityfs.py` needed no layout, as for Silksong ([04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container)). +- The loose serialized files (`resources.assets`, `sharedassets1.assets`, the built-in resources) have no type trees. They borrow the `Shader` tree that 10 of the bundles carry (the same type hash; `shader_split.py` defers a file that raises `NoTree` to a second pass), so `unityfs.py` needed no layout, as for Silksong ([Unity 6 Shader objects](../../../docs/reference/shaders.md#unity-6-shader-objects)). - `romfs_marker` is `Data/sharedassets1.assets`. ## Input diff --git a/dm/games/010089B00D09C000/README.md b/dm/games/010089B00D09C000/README.md index 20d64ff..1caca4f 100644 --- a/dm/games/010089B00D09C000/README.md +++ b/dm/games/010089B00D09C000/README.md @@ -63,8 +63,8 @@ The three geometry shaders were the test case of the general geometry-shader com General host changes first needed here; each applies to every game: -- **H.264 drain on a released output slot** (`ReleaseOutputBufferIndex` goes on with the drain; the end-of-stream output takes the first queued picture, `jflush`, `hle_movie.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Per-stage composite taken again after a replay state restore** (`prog_resolve` in `nvn_mtl.m`, after `pso_predict_rest`, the fusion probe and the deferred-draw replay): [04, the entry under Seen in Metroid Dread](../../../docs/agents/04-graphics.md#seen-in-metroid-dread). +- **H.264 drain on a released output slot** (`ReleaseOutputBufferIndex` goes on with the drain; the end-of-stream output takes the first queued picture, `jflush`, `hle_movie.c`): [H.264 drain at the end of a stream goes on with released slots](../../../docs/reference/movies.md#h264-drain-at-the-end-of-a-stream-goes-on-with-released-slots). +- **Per-stage composite taken again after a replay state restore** (`prog_resolve` in `nvn_mtl.m`, after `pso_predict_rest`, the fusion probe and the deferred-draw replay): [Composite program taken again after a replay state restore](../../../docs/reference/rendering.md#composite-program-taken-again-after-a-replay-state-restore). - **Geometry shaders as compute** (native VS capture, GS dispatch, output-reader vertex function; `t_geometry`): [geometry support and limits](../../../docs/geometry-metal-research.md). - The spin-wait search's store rule takes one wider loop in its `main`: [development.md, Power](../../../docs/development.md#power). diff --git a/dm/games/01008BC024FDE000/README.md b/dm/games/01008BC024FDE000/README.md index 0167cc0..0df5817 100644 --- a/dm/games/01008BC024FDE000/README.md +++ b/dm/games/01008BC024FDE000/README.md @@ -29,7 +29,7 @@ The pad layout is the default 2D one (no `pad_layout` row) and the operation mod ## Movies and audio - 87 H.264 / AAC MP4 movies in `movie/`, demuxed by the SDK's `movie::Extractor` in `subsdk0` and decoded by the host (`hle_movie.c`): decoders opened with output 4 (NV12 into the client's buffer, 1280 × 720 × 1.5 bytes) in mode 1. The title menu's background is one of those movies. -- **Resolved: a null read at `sdk+0x1f2314` as the first movie starts** (the title menu's background, right after the first press), in `nn::audio::SetAudioOutVolume`, called from `main+0x1c4fbc` on the AudioOut the game opens for the movie's AAC sound (`OpenDefaultAudioOut` without an event, `main+0x1c48c0`). The SDK's version calls the audout session's virtual method, and the host never made the session. The game sets the volume every frame while a movie plays. The host keeps it per output and scales the PCM16 it plays, saturating; `GetAudioOutVolume` reads it back (`hle_audio.c`). It was among `main`'s imports that no host hook names; the others there fill the game's own structures or run in `sdk` (the CPU audio renderer, the swkbd and hid argument builders). It was the one call the game needed ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)). +- **Resolved: a null read at `sdk+0x1f2314` as the first movie starts** (the title menu's background, right after the first press), in `nn::audio::SetAudioOutVolume`, called from `main+0x1c4fbc` on the AudioOut the game opens for the movie's AAC sound (`OpenDefaultAudioOut` without an event, `main+0x1c48c0`). The SDK's version calls the audout session's virtual method, and the host never made the session. The game sets the volume every frame while a movie plays. The host keeps it per output and scales the PCM16 it plays, saturating; `GetAudioOutVolume` reads it back (`hle_audio.c`). It was among `main`'s imports that no host hook names; the others there fill the game's own structures or run in `sdk` (the CPU audio renderer, the swkbd and hid argument builders). It was the one call the game needed ([AudioOut volume kept per output](../../../docs/reference/audio.md#audioout-volume-kept-per-output)). - Its sound goes through the hardware Opus decoder calls (`hle_codec.c`). ## Files and shaders @@ -50,9 +50,9 @@ The pad layout is the default 2D one (no `pad_layout` row) and the operation mod General host changes first needed here; each applies to every game: -- **`nn::audio::SetAudioOutVolume` / `GetAudioOutVolume`** per output (`hle_audio.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Storage takeover in the storage-sharing texture table** (`alias_take_over` in `nvn_mtl.m`, with its guards for live texture objects and lookups in the last 16 submissions, and the `shared textures N (…)` memory line): [04, Seen in STEINS;GATE RE:BOOT](../../../docs/agents/04-graphics.md#seen-in-steinsgate-reboot). -- The halfword jump-table padding (`half_count`) changed 1 to 6 tables in some `main`s, this one among them: [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). +- **`nn::audio::SetAudioOutVolume` / `GetAudioOutVolume`** per output (`hle_audio.c`): [AudioOut volume kept per output](../../../docs/reference/audio.md#audioout-volume-kept-per-output). +- **Storage takeover in the storage-sharing texture table** (`alias_take_over` in `nvn_mtl.m`, with its guards for live texture objects and lookups in the last 16 submissions, and the `shared textures N (…)` memory line): [Shared-storage entries taken over by a new texture](../../../docs/reference/rendering.md#shared-storage-entries-taken-over-by-a-new-texture). +- The halfword jump-table padding (`half_count`) changed 1 to 6 tables in some `main`s, this one among them: [Halfword jump table landing pads sized to the largest entry](../../../docs/reference/rewriter.md#halfword-jump-table-landing-pads-sized-to-the-largest-entry). ## Open diff --git a/dm/games/010093801237C000/README.md b/dm/games/010093801237C000/README.md index 757a32f..7c4e26d 100644 --- a/dm/games/010093801237C000/README.md +++ b/dm/games/010093801237C000/README.md @@ -74,17 +74,17 @@ The game plays VP9 WebM movies through the SDK's own demuxer and a decoder in na General host changes first needed here; each applies to every game: -- **Patch areas before the module base** for modules with a giant bss (negative `PATCH_OFF`, `FILE_IMAGE_LEN`, `IMAGE_SIZE`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **Used memory counts the bss of front-patch modules** (`big_bss_bytes`): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **`sleep_precise`** against macOS timer slack, **same-core preemption** (`core_preempt`, `svc_thread` / `dm_yield_to_core_peers`, `DM_PREEMPT`) and **observation gates right after a `bl`**: [03, Threads](../../../docs/agents/03-bringup.md#threads). -- **Native-texture movies** (`nvn_host_write_texture`, `nvn_host_signal_sync`), the `D1` destructor, `md_event`, the SDK 11 names, `FinishStartupLogo`, `SendVibrationValues` and lexical `..` (`norm_rest`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Per-stage program binding** (`bind_stages`, composite programs, `Z` lines in pass dumps), **in-place `WriteTexels`** (`write_region_bl`) and the **MSHD reader**: [04, Seen in Metroid Dread](../../../docs/agents/04-graphics.md#seen-in-metroid-dread) and [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **Patch areas before the module base** for modules with a giant bss (negative `PATCH_OFF`, `FILE_IMAGE_LEN`, `IMAGE_SIZE`): [Patch area before the module base for a large bss](../../../docs/reference/rewriter.md#patch-area-before-the-module-base-for-a-large-bss). +- **Used memory counts the bss of front-patch modules** (`big_bss_bytes`): [Heap sized from the application pool less used memory](../../../docs/reference/memory.md#heap-sized-from-the-application-pool-less-used-memory). +- **`sleep_precise`** against macOS timer slack, **same-core preemption** (`core_preempt`, `svc_thread` / `dm_yield_to_core_peers`, `DM_PREEMPT`) and **observation gates right after a `bl`**: [Timer slack in guest sleeps (`sleep_precise`)](../../../docs/reference/threads.md#timer-slack-in-guest-sleeps-sleep_precise), [Approximating same-core preemption (`core_preempt`)](../../../docs/reference/threads.md#approximating-same-core-preemption-core_preempt). +- **Native-texture movies** (`nvn_host_write_texture`, `nvn_host_signal_sync`), the `D1` destructor, `md_event`, the SDK 11 names, `FinishStartupLogo`, `SendVibrationValues` and lexical `..` (`norm_rest`): [Decoding into the client's NVN texture](../../../docs/reference/movies.md#decoding-into-the-clients-nvn-texture), [The complete-object destructor](../../../docs/reference/movies.md#the-complete-object-destructor), [Client events read through the registered pointer](../../../docs/reference/movies.md#client-events-read-through-the-registered-pointer), [Older SDK names for the same calls](../../../docs/reference/services.md#older-sdk-names-for-the-same-calls), [The startup logo](../../../docs/reference/services.md#the-startup-logo), [Vibration values flushed by the SDK's vibration graph](../../../docs/reference/services.md#vibration-values-flushed-by-the-sdks-vibration-graph), [Paths with `.` and `..` components](../../../docs/reference/services.md#paths-with--and--components). +- **Per-stage program binding** (`bind_stages`, composite programs, `Z` lines in pass dumps), **in-place `WriteTexels`** (`write_region_bl`) and the **MSHD reader**: [Programs bound per stage](../../../docs/reference/rendering.md#programs-bound-per-stage), [`WriteTexels` regions written in place by block-linear address](../../../docs/reference/nvn-objects.md#writetexels-regions-written-in-place-by-block-linear-address) and [MSHD in MercurySteam's bshdat files](../../../docs/reference/shaders.md#mshd-in-mercurysteams-bshdat-files). - **The submit-thread autorelease leak** (`g_sub_pool`): [05, Memory growth](../../../docs/agents/05-debugging.md#memory-growth). - **Inline uniform updates inside a pass** (`DM_NVN_UB_DEFER`): [development.md, Power](../../../docs/development.md#power). ## Open -- [04, Seen in Metroid Dread](../../../docs/agents/04-graphics.md#seen-in-metroid-dread) still says every copy or update ends the host's render pass (about 180 one-draw passes in cutscenes); [development.md, Power](../../../docs/development.md#power) says an inline uniform update no longer does (about 246 passes a frame before). The cutscene pass count with the deferred update is not given. +- 04's Metroid Dread section, now [Inline uniform updates inside a render pass](../../../docs/reference/rendering.md#inline-uniform-updates-inside-a-render-pass), said every copy or update ends the host's render pass (about 180 one-draw passes in cutscenes); [development.md, Power](../../../docs/development.md#power) says an inline uniform update no longer does (about 246 passes a frame before). The cutscene pass count with the deferred update is not given. - `game.tsv` opens with "Bring-up has just started: the memory values are Mario's until measured", while its memory section gives measured values. - [dm/games/README.md, Adding a game](../README.md#adding-a-game) gives `0x90000000` as this game's heap region in the guesser's 4 GB layout; in `game.tsv` `0x90000000` is the whole block (`mem_reserve`) and the heap region is 1.5 GB. - Only the merged 2.1.0 copy is extracted; the base game's own build IDs are not in the profile. diff --git a/dm/games/010099C022B96000/README.md b/dm/games/010099C022B96000/README.md index fcf84c6..d54e267 100644 --- a/dm/games/010099C022B96000/README.md +++ b/dm/games/010099C022B96000/README.md @@ -56,7 +56,7 @@ Update 1.3.1 (v327680) over the base game, SDK 18.3.5. The Wii game (codename Co ## Services - **Resolved: `sm::Initialize` aborted right after `ConnectToNamedPort` (SVC 0x1F) and `SendSyncRequest` (0x21).** An SDK call the host did not hook opened its service: `nn::hid::InitializeTouchScreen(int, int)`, the SDK 18 overload of the hooked `InitializeTouchScreen()`. -- Focus handling mode 3 (no focus notifications; the system suspends the game out of focus). The host queues message 15 only under modes 1 and 2 ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk), the Galaxy 2 entry). +- Focus handling mode 3 (no focus notifications; the system suspends the game out of focus). The host queues message 15 only under modes 1 and 2 ([Focus handling mode decides whether focus changes are notified](../../../docs/reference/services.md#focus-handling-mode-decides-whether-focus-changes-are-notified)). ## Damaged NSP @@ -71,10 +71,10 @@ Update 1.3.1 (v327680) over the base game, SDK 18.3.5. The Wii game (codename Co General host changes first needed here; each applies to every game: -- **Vulkan layer additions** (push constants, descriptor arrays, dual-source blending `vk__d`, GX vertex formats, viewport and scissor across passes, copies inside a pass, INHERIT transform and the vi layer crop, linear images, queries, `fix_strides`): [04, Seen in Super Mario Galaxy (Vulkan)](../../../docs/agents/04-graphics.md#seen-in-super-mario-galaxy-vulkan). +- **Vulkan layer additions** (push constants, descriptor arrays, dual-source blending `vk__d`, GX vertex formats, viewport and scissor across passes, copies inside a pass, INHERIT transform and the vi layer crop, linear images, queries, `fix_strides`): [Push constants in the next free buffer slot](../../../docs/reference/rendering.md#push-constants-in-the-next-free-buffer-slot), [Descriptor arrays bound to consecutive slots](../../../docs/reference/rendering.md#descriptor-arrays-bound-to-consecutive-slots), [Dual-source blending from output location 1](../../../docs/reference/rendering.md#dual-source-blending-from-output-location-1), [Scaled and three-component vertex formats](../../../docs/reference/rendering.md#scaled-and-three-component-vertex-formats), [Viewport and scissor persist across render passes](../../../docs/reference/rendering.md#viewport-and-scissor-persist-across-render-passes), [Copies inside a render pass](../../../docs/reference/rendering.md#copies-inside-a-render-pass), [Swapchain pre-transform and layer crop](../../../docs/reference/rendering.md#swapchain-pre-transform-and-layer-crop), [Linear images alias their memory](../../../docs/reference/rendering.md#linear-images-alias-their-memory), [Timestamp and occlusion queries](../../../docs/reference/rendering.md#timestamp-and-occlusion-queries), [Vulkan shaders generated at run time](../../../docs/reference/rendering.md#vulkan-shaders-generated-at-run-time). - **SPIR-V captured at run time** (`DM_VK_CAPTURE`, `vk_capture.tsv`, the NSP script's capture stage): [development.md, The packing script in detail](../../../docs/development.md#the-packing-script-in-detail) and [dm/games/README.md](../README.md#what-the-game-files-decide-at-every-build). -- **Sleeps under 20 µs take the yield back-off** (`sleep_precise`, `hle_thread.c`): [03, Threads](../../../docs/agents/03-bringup.md#threads). -- **SDK 18's `InitializeTouchScreen(int, int)`** hooked: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Sleeps under 20 µs take the yield back-off** (`sleep_precise`, `hle_thread.c`): [Sleeps under 20 µs take the yield path](../../../docs/reference/threads.md#sleeps-under-20-µs-take-the-yield-path). +- **SDK 18's `InitializeTouchScreen(int, int)`** hooked: [SDK calls that open a system service through sm](../../../docs/reference/services.md#sdk-calls-that-open-a-system-service-through-sm). - **`op_mode`** (the operation mode at start, `DM_OPMODE`): [dm/games/README.md, Keys](../README.md#keys). ## Open diff --git a/dm/games/01009BF0072D4000/README.md b/dm/games/01009BF0072D4000/README.md index 65b3da8..7a428aa 100644 --- a/dm/games/01009BF0072D4000/README.md +++ b/dm/games/01009BF0072D4000/README.md @@ -41,7 +41,7 @@ No `patches.py`, no `dm/tools/sdk_patches.py` cut for its `sdk` build (`3ae34e68 General host changes first needed here; each applies to every game: -- **GLSLC captures only when a module defines `glslcCompile`** (`shader_split.py`, as `profile_guess.py`): [04, Seen in Captain Toad](../../../docs/agents/04-graphics.md#seen-in-captain-toad-treasure-tracker). +- **GLSLC captures only when a module defines `glslcCompile`** (`shader_split.py`, as `profile_guess.py`): [Built-in shaders compiled with the game's own GLSLC](../../../docs/reference/shaders.md#built-in-shaders-compiled-with-the-games-own-glslc). - **Routes planned from a position probe and the stage map**: [05, Scripted input and thread scheduling](../../../docs/agents/05-debugging.md#scripted-input-and-thread-scheduling). ## Open diff --git a/dm/games/0100A00019DE0000/README.md b/dm/games/0100A00019DE0000/README.md index 4731252..82de779 100644 --- a/dm/games/0100A00019DE0000/README.md +++ b/dm/games/0100A00019DE0000/README.md @@ -41,7 +41,7 @@ Base game v0 (the NACP's display version is 0.0.126424), SDK 19.3.9. Supergiant General host changes first needed here; each applies to every game: - **Combined texture slots past 31** (the combined table up to 0x168, 41 slots, sharing words with the image table; a bound texture written there when the image slot is unbound): [04, Find the binding layout by census](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census). -- **The SDK 19 `nn::ro::LoadModule` form** and `ro::Finalize` hooked (`hle_ro.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **The SDK 19 `nn::ro::LoadModule` form** and `ro::Finalize` hooked (`hle_ro.c`): [`nn::ro` and its port](../../../docs/reference/services.md#nnro-and-its-port). ## Open diff --git a/dm/games/0100A5C00D162000/README.md b/dm/games/0100A5C00D162000/README.md index 794d49b..544a813 100644 --- a/dm/games/0100A5C00D162000/README.md +++ b/dm/games/0100A5C00D162000/README.md @@ -40,13 +40,13 @@ Update 1.3.7 (v655360) over the base game; its DLC (The Delicious Last Course) i General host changes first needed here; each applies to every game: -- **A size the hardware never reports as 0 is never 0 on the host** (`zcull_size`): [03, NVN object sizes](../../../docs/agents/03-bringup.md#nvn-object-sizes-memory-corruption). +- **A size the hardware never reports as 0 is never 0 on the host** (`zcull_size`): [Sizes the hardware never reports as 0 (ZCull save area)](../../../docs/reference/nvn-objects.md#sizes-the-hardware-never-reports-as-0-zcull-save-area). - **Overlapping copies in the pass dump** as the sign of a broken game allocator: [05, Intermittent rendering glitches](../../../docs/agents/05-debugging.md#intermittent-rendering-glitches). - **`mem_used`** for rewritten modules over 64 MB, which the NSP script now sets after the rewrite: [dm/games/README.md, Keys](../README.md#keys) and [Adding a game](../README.md#adding-a-game). -- **Unity 2017.4 serialized and `Shader` formats** (`unityfs.py`, `shader_split.py`) and the rewriter's jump-table form (`patch_nso.py`), by armyja: [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container), [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). +- **Unity 2017.4 serialized and `Shader` formats** (`unityfs.py`, `shader_split.py`) and the rewriter's jump-table form (`patch_nso.py`), by armyja: [shaders.md, Containers by engine](../../../docs/reference/shaders.md#containers-by-engine), [Offset jump tables with the bounds check on another register](../../../docs/reference/rewriter.md#offset-jump-tables-with-the-bounds-check-on-another-register). ## Open - "SDK 5.5.0 (the oldest so far)" in `game.tsv` and docs/development.md is stale: DARK SOULS: REMASTERED (4.5.0) and Mario Kart 8 Deluxe (4.4.0) are older. -- The tracing text in [03](../../../docs/agents/03-bringup.md#nvn-object-sizes-memory-corruption) names observation gates "in its `patches.py`"; the profile has no `patches.py` now. +- The tracing text in [nvn-objects.md](../../../docs/reference/nvn-objects.md#sizes-the-hardware-never-reports-as-0-zcull-save-area) names observation gates "in its `patches.py`"; the profile has no `patches.py` now. - `game.tsv` has no `sys_resource` row, so the host reports its default, 32 MB (`game.py`, `hle_svc.c`), although the NPDM declares none and the notes describe the game as having no virtual address memory. Not checked against the game's behaviour. diff --git a/dm/games/0100A6301214E000/README.md b/dm/games/0100A6301214E000/README.md index e9ad822..3861bf7 100644 --- a/dm/games/0100A6301214E000/README.md +++ b/dm/games/0100A6301214E000/README.md @@ -26,7 +26,7 @@ No `pad_layout`: the virtual gamepad starts in the 2D layout. `present_interval` - The former 6 GB block left no margin on a phone: without the extended virtual addressing entitlement iOS allows one reservation of at most 6 GB, and Metal needs address space too (Engage allocates about 1 GB of GPU memory on the Mac). - **Resolved: a crash a few minutes into the prologue battle (scripted run, about 357 s).** A write to `NULL + 0x1fffe0` at `main+0x9a4664`: Unity's TLSF heap allocator (`main+0x9a45d0`) added a pool and got no address region for it. The alias region was out of address space, not memory: at the crash Engage held 4.17 of the 4.375 GB left after the SDK's ~1 GB node area (StandardAllocator 3.14 GB, Unity about 1.05 GB, the rest fragments). On hardware the alias region is 64 GB. The node area cut (the two `mov w8/w9, #0x3fe00000` in the AddressRegionManager constructor, `sdk+0x346420`, → 62 MB, about 1.35 million 0x30-byte nodes; a few hundred regions are live in measurements) fixed it; a 12-minute scripted run now plays through the battle. - **Resolved: the idle attract demo crashed after about 220 s.** Unity's particle geometry jobs wrote through a NULL index pointer on several job threads at once (`main+0xdb0ad4`, `0xdaea58`, `0xdadb5c`). The scheduler (`main+0xd91e0c`) takes its vertex and index chunks from a fallback that creates and maps a fresh buffer each time (`main+0x7b9c64`; Unity's Switch renderer always takes it, as on hardware), and from about 180 s 32-bit mesh indices also turned up inside Unity's texture objects (the replay's red lines on rejected texture objects showed their first bytes as 32-bit triangle indices; they stopped once the alias region had address space left). Same cause: past 2 MB the index buffers got no mapping. With the cut an 850 s idle run is clean; with the old SDK the same run dies at 159 s in Unity's heap allocator. To reproduce the old state: `boot.sh 245` with no input on an SDK rewritten without the cut. -- How it was measured: probe `VammManager::AllocateAddressRegion` at its success path (`ReleaseWriteLock` return: x21 = size, x22 = address) and `FreeAddressRegion` after its prologue (x1 = address) and sum live regions by caller; probe `VammManager::AllocateMemoryPages` / `FreeMemoryPages` (x1 address, x2 size) for commits. The `[mem]` log lines report the alias high-water mark and the mapped total ([03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space)). +- How it was measured: probe `VammManager::AllocateAddressRegion` at its success path (`ReleaseWriteLock` return: x21 = size, x22 = address) and `FreeAddressRegion` after its prologue (x1 = address) and sum live regions by caller; probe `VammManager::AllocateMemoryPages` / `FreeMemoryPages` (x1 address, x2 size) for commits. The `[mem]` log lines report the alias high-water mark and the mapped total ([Measuring address regions and commits](../../../docs/reference/memory.md#measuring-address-regions-and-commits)). ## Rendering @@ -70,13 +70,13 @@ No `pad_layout`: the virtual gamepad starts in the 2D layout. `present_interval` General host changes first needed here; each applies to every game: -- **Unity's x18 shapes** (`site_br_tab`, `site_blr`, `site_sp`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **Node area cut by `sdk` build** (`dm/tools/sdk_patches.py`) and **sizing a virtual-address-memory game's reservation**: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **C++ exception unwinding through `svcQueryMemory`** (IL2CPP throws for every managed exception), **`movie::Decoder` on MediaCodec's protocol**, **applet `Show…` hooks**, **RomFS directory index** (`dir_index_get`), **add-on content from `DM_AOC_DIR`**, **program keys in `SetShaders`** (`prog_key`), **CPU-written block-linear textures** (`mtl_tex_cpu_written`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Texture objects copied into records** (`REC_SNAP` / `tex_snap`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Several audio outputs logged apart** (`audio #N`): [03, Audio](../../../docs/agents/03-bringup.md#audio). -- **Pool finalize evicting shared textures** (`mtl_alias_drop_pool`), deferred release of pool buffers, the memory line's `serial objects` / `freed but alive` / `shared textures`: [04, Seen in Engage](../../../docs/agents/04-graphics.md#seen-in-engage-unity). -- **Unity `Shader` objects** (`unityfs.py`): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container); **separate textures** (`NVN_SEPTEX_SLOT0`): [04, section 3](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census). +- **Unity's x18 shapes** (`site_br_tab`, `site_blr`, `site_sp`): [Offset jump tables through x18 read at rewrite time](../../../docs/reference/rewriter.md#offset-jump-tables-through-x18-read-at-rewrite-time), [Indirect calls through x18](../../../docs/reference/rewriter.md#indirect-calls-through-x18), [Stack pointer writes through x18](../../../docs/reference/rewriter.md#stack-pointer-writes-through-x18). +- **Node area cut by `sdk` build** (`dm/tools/sdk_patches.py`) and **sizing a virtual-address-memory game's reservation**: [The SDK's address-region node area](../../../docs/reference/memory.md#the-sdks-address-region-node-area), [Sizing the pool and alias region for virtual address memory](../../../docs/reference/memory.md#sizing-the-pool-and-alias-region-for-virtual-address-memory). +- **C++ exception unwinding through `svcQueryMemory`** (IL2CPP throws for every managed exception), **`movie::Decoder` on MediaCodec's protocol**, **applet `Show…` hooks**, **RomFS directory index** (`dir_index_get`), **add-on content from `DM_AOC_DIR`**, **program keys in `SetShaders`** (`prog_key`), **CPU-written block-linear textures** (`mtl_tex_cpu_written`): [`svcQueryMemory` must report each module's real layout](../../../docs/reference/memory.md#svcquerymemory-must-report-each-modules-real-layout), [`movie::Decoder` taken over as a MediaCodec-style protocol](../../../docs/reference/movies.md#moviedecoder-taken-over-as-a-mediacodec-style-protocol), [System applets opened by `Show…` calls](../../../docs/reference/services.md#system-applets-opened-by-show-calls), [RomFS lookups on large directories](../../../docs/reference/services.md#romfs-lookups-on-large-directories), [Add-on content from `DM_AOC_DIR`](../../../docs/reference/services.md#add-on-content-from-dm_aoc_dir), [Program keys computed inside `nvnProgramSetShaders`](../../../docs/reference/shaders.md#program-keys-computed-inside-nvnprogramsetshaders), [Textures written with `nvnTextureWriteTexels`](../../../docs/reference/nvn-objects.md#textures-written-with-nvntexturewritetexels). +- **Texture objects copied into records** (`REC_SNAP` / `tex_snap`): [Texture objects copied into command records](../../../docs/reference/rendering.md#texture-objects-copied-into-command-records). +- **Several audio outputs logged apart** (`audio #N`): [Several outputs at once (`audio #N`)](../../../docs/reference/audio.md#several-outputs-at-once-audio-n). +- **Pool finalize evicting shared textures** (`mtl_alias_drop_pool`), deferred release of pool buffers, the memory line's `serial objects` / `freed but alive` / `shared textures`: [Finalized pools kept alive by residency and the sharing table](../../../docs/reference/nvn-objects.md#finalized-pools-kept-alive-by-residency-and-the-sharing-table). +- **Unity `Shader` objects** (`unityfs.py`): [Unity 2020.3 Shader objects in UnityFS bundles](../../../docs/reference/shaders.md#unity-20203-shader-objects-in-unityfs-bundles); **separate textures** (`NVN_SEPTEX_SLOT0`): [04, section 3](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census). - **Prewarm lists**: [development.md](../../../docs/development.md#prewarm-lists). ## Open diff --git a/dm/games/0100A83022A22000/README.md b/dm/games/0100A83022A22000/README.md index 4091014..11192f9 100644 --- a/dm/games/0100A83022A22000/README.md +++ b/dm/games/0100A83022A22000/README.md @@ -17,7 +17,7 @@ No `pad_layout` key: the iPhone gamepad starts in the 2D layout ([README](../../ ## Memory -- The NPDM declares 16 MB of system resource (virtual address memory), and `main` never calls `SetMemoryHeapSize`: everything lives in address regions of the alias region. `profile_guess.py` gives every game with an NPDM system resource the 5 GB block, and this game came out right with it ([03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space)). +- The NPDM declares 16 MB of system resource (virtual address memory), and `main` never calls `SetMemoryHeapSize`: everything lives in address regions of the alias region. `profile_guess.py` gives every game with an NPDM system resource the 5 GB block, and this game came out right with it ([Unity 6 takes its memory with no StandardAllocator and no heap](../../../docs/reference/memory.md#unity-6-takes-its-memory-with-no-standardallocator-and-no-heap)). - **Resolved: `nn::diag` abort at `sdk+0x12dad0` in `nninitStartup`, called from `main+0x756310`.** That is malloc's `StandardAllocator::Initialize(nullptr, 0xa00000000)`: 40 GB of address space (`main+0x28bff4`; `nninitStartup` is at `main+0x1b54d0`, the call goes through `main+0x7562d0`). - How the needs were measured: one boot on the Mac with a 64 GB alias region (`mem_reserve 0x1030000000`, `mem_alias 0x1008000000`), probing `VammManager::AllocateAddressRegion`'s success path (`sdk+0x13bf8c`: x21 size, x22 address) with `DM_SVC=1`'s map log. The regions, from boot into the first level: - malloc's: 2.19 GB used, 1.75 GB of it FNA3D's GPU heap, allocated whole at `FNA3D_CreateDevice` (`main+0xc03660` → `main+0x563370`); its blocks reach 332 MB from the region's start and 1.82 GB from its end, and no more after the title. The cut gives it 2.5 GB, 300 MB to spare. @@ -41,7 +41,7 @@ The effects (`.xnb`) are XNA's D3D9 effects (MOJOSHADER is linked in and parses ## Code - **Resolved: `RuntimeError: 788 ldr x18, #0xd1eb0` in `patch_nso.py`.** The text segment starts with 1.7 MB of data (`main+0x0`–`+0x1b5000`, high-entropy, no FDE covers it, and no `adrp` or relocation points into it): 62% of its words do not decode, and 6,858 of the rest read as instructions that use `x18`, plus three "`svc`" words with numbers such as `#0x91e7`. The linear sweep rewrote them all. `dm/tools/nso_code.py` finds such data blocks (a stretch of at least 4 KB between the `.eh_frame` FDE ranges in which more than a quarter of the words do not decode); the sweep skips them (`DATA_BLOCKS`, logged as `data blocks left alone`), and so does `insn_census.py`, whose counts then match the rewriter's again (11,744 `x18` sites in this `main`). Code without unwind information is still swept: it decodes (zlib at `main+0x75261c`, libunwind's register restore at `main+0x28dbd4`). -- Padding halfword jump tables only to their largest entry (`half_count`) changed one to six tables in this `main`, as in a few other games' ([03, Rewriter](../../../docs/agents/03-bringup.md#rewriter)). +- Padding halfword jump tables only to their largest entry (`half_count`) changed one to six tables in this `main`, as in a few other games' ([Halfword jump table landing pads sized to the largest entry](../../../docs/reference/rewriter.md#halfword-jump-table-landing-pads-sized-to-the-largest-entry)). ## Services @@ -58,14 +58,14 @@ The effects (`.xnb`) are XNA's D3D9 effects (MOJOSHADER is linked in and parses General host changes first needed here; each applies to every game: -- **Data blocks at the start of the text left alone** (`nso_code.py`, `DATA_BLOCKS`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **`getenv` answered from the profile** (`game_env`, `hle_os.c`) and **the 32 MB alignment cut for large regions on SDK 18**, later taken by Paper Mario: The Thousand-Year Door's and Celeste's SDK 15.3.2 build: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **`vi::CreateLayer` with a size**, **AudioIn absent**, **online SDK NROs refused** (`ONLINE_SDKS`), **`nso_syms.py host_hooks` reading `dm/gfx`**, **NVIDIA's GPU drivers never loaded** (`GPU_DRIVERS`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **H.264 decoders made at `Configure` switched to RGBA** (`md_avc_open`): same section. -- **Viewport swizzles** (`OP_VP_SWIZZLE`): [04, Seen in Absolum](../../../docs/agents/04-graphics.md#seen-in-absolum). -- **GLSLC outputs in loose RomFS files**: [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **Data blocks at the start of the text left alone** (`nso_code.py`, `DATA_BLOCKS`): [Data blocks at the start of the text](../../../docs/reference/rewriter.md#data-blocks-at-the-start-of-the-text). +- **`getenv` answered from the profile** (`game_env`, `hle_os.c`) and **the 32 MB alignment cut for large regions on SDK 18**, later taken by Paper Mario: The Thousand-Year Door's and Celeste's SDK 15.3.2 build: [.NET NativeAOT malloc and the GC region range](../../../docs/reference/memory.md#net-nativeaot-malloc-and-the-gc-region-range), [1 GB alignment of regions of 1 GB and more](../../../docs/reference/memory.md#1-gb-alignment-of-regions-of-1-gb-and-more). +- **`vi::CreateLayer` with a size**, **AudioIn absent**, **online SDK NROs refused** (`ONLINE_SDKS`), **`nso_syms.py host_hooks` reading `dm/gfx`**, **NVIDIA's GPU drivers never loaded** (`GPU_DRIVERS`): [`vi::CreateLayer` with the layer's size](../../../docs/reference/services.md#vicreatelayer-with-the-layers-size), [No audio input devices](../../../docs/reference/audio.md#no-audio-input-devices), [Online SDKs in NROs are never loaded](../../../docs/reference/services.md#online-sdks-in-nros-are-never-loaded), [Listing a game's unhooked imports before they are met](../../../docs/reference/services.md#listing-a-games-unhooked-imports-before-they-are-met), [GPU driver modules that are never loaded](../../../docs/reference/services.md#gpu-driver-modules-that-are-never-loaded). +- **H.264 decoders made at `Configure` switched to RGBA** (`md_avc_open`): [Parameter sets in `Configure` made the H.264 decoder without RGBA](../../../docs/reference/movies.md#parameter-sets-in-configure-made-the-h264-decoder-without-rgba). +- **Viewport swizzles** (`OP_VP_SWIZZLE`): [Viewport swizzles applied as a flipped viewport](../../../docs/reference/rendering.md#viewport-swizzles-applied-as-a-flipped-viewport). +- **GLSLC outputs in loose RomFS files**: [GLSLC outputs in main and in loose files](../../../docs/reference/shaders.md#glslc-outputs-in-main-and-in-loose-files). ## Open - Shader counts differ: development.md says 162 GLSLC outputs in `shaders.nx.bin`; 04 and `dm/games/README.md` say 163 outputs (162 unique). -- Loading online SDKs: always load, refuse them (as now), or decide per profile is an open question for the maintainer ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)). +- Loading online SDKs: always load, refuse them (as now), or decide per profile is an open question for the maintainer ([Online SDKs in NROs are never loaded](../../../docs/reference/services.md#online-sdks-in-nros-are-never-loaded)). diff --git a/dm/games/0100A9D01C446000/README.md b/dm/games/0100A9D01C446000/README.md index cb9375c..aac33a0 100644 --- a/dm/games/0100A9D01C446000/README.md +++ b/dm/games/0100A9D01C446000/README.md @@ -73,12 +73,12 @@ The desktop renderer with forward shading: one RG11B10 colour pass draws every o General host changes first needed here; each applies to every game: -- **Unreal Engine 4.27's FMalloc layout and the allocator made on the first allocation** (`hle_uemalloc.c` layout `UE4_27`, cut + `OBS_UE_MALLOC` gate): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **LZ4 Unreal shaders decoded by `uepak.py` itself** (`lz4_shaders`): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container), [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). -- **Stencil-mode texture builders** (`tex_t.stencil`), **texture slots 32–39 reading the bound image** (`tex_slot_h`) and **image size queries typed from the image** (`dmsh_xlate` `ImageTypesOf`): [04, Seen in DRAGON QUEST VII Reimagined](../../../docs/agents/04-graphics.md#seen-in-dragon-quest-vii-reimagined). -- **A 32 kHz audio renderer resampled 3:2 on the CPU** (`up_3_2`): [03, Audio](../../../docs/agents/03-bringup.md#audio). -- **`nv12-uv-stride`, `nv12-width`, `nv12-height` in the movie output format** (`md_format`), **`GetAudioDeviceOutputVolume`**, **`kpr::GetKeyCodeMap`** and the shop / Nintendo Account calls: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Rewriter:** gates before code that writes the scratch registers first (`scratch_dead`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). +- **Unreal Engine 4.27's FMalloc layout and the allocator made on the first allocation** (`hle_uemalloc.c` layout `UE4_27`, cut + `OBS_UE_MALLOC` gate): [Unreal's FMallocBinned3 replaced by the host allocator](../../../docs/reference/memory.md#unreals-fmallocbinned3-replaced-by-the-host-allocator). +- **LZ4 Unreal shaders decoded by `uepak.py` itself** (`lz4_shaders`): [Unreal Engine 4.27 shader code archives (LZ4)](../../../docs/reference/shaders.md#unreal-engine-427-shader-code-archives-lz4), [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). +- **Stencil-mode texture builders** (`tex_t.stencil`), **texture slots 32–39 reading the bound image** (`tex_slot_h`) and **image size queries typed from the image** (`dmsh_xlate` `ImageTypesOf`): [Stencil-mode textures from the texture builder](../../../docs/reference/rendering.md#stencil-mode-textures-from-the-texture-builder), [A texture slot that shares its c2 word with an image slot](../../../docs/reference/shaders.md#a-texture-slot-that-shares-its-c2-word-with-an-image-slot). +- **A 32 kHz audio renderer resampled 3:2 on the CPU** (`up_3_2`): [A renderer below 48 kHz with no upsampler on the CPU](../../../docs/reference/audio.md#a-renderer-below-48-khz-with-no-upsampler-on-the-cpu). +- **`nv12-uv-stride`, `nv12-width`, `nv12-height` in the movie output format** (`md_format`), **`GetAudioDeviceOutputVolume`**, **`kpr::GetKeyCodeMap`** and the shop / Nintendo Account calls: [Output format keys (`md_format`)](../../../docs/reference/movies.md#output-format-keys-md_format), [Audio device output volume and names](../../../docs/reference/audio.md#audio-device-output-volume-and-names), [Mouse, keyboard and key code maps](../../../docs/reference/services.md#mouse-keyboard-and-key-code-maps), [Shop service accessors](../../../docs/reference/services.md#shop-service-accessors), [Account, friends, play data and Nintendo Account calls](../../../docs/reference/services.md#account-friends-play-data-and-nintendo-account-calls). +- **Rewriter:** gates before code that writes the scratch registers first (`scratch_dead`): [Observation gates where x9–x17 are dead](../../../docs/reference/rewriter.md#observation-gates-where-x9x17-are-dead). - **`DM_CTL` route scripts** (`run/ctl.sh`): [05, Scripted input and thread scheduling](../../../docs/agents/05-debugging.md#scripted-input-and-thread-scheduling). ## Open diff --git a/dm/games/0100ABF008968000/README.md b/dm/games/0100ABF008968000/README.md index 766c594..3a2483b 100644 --- a/dm/games/0100ABF008968000/README.md +++ b/dm/games/0100ABF008968000/README.md @@ -19,7 +19,7 @@ No `patches.py`. ## Services -SDK 7 calls, one boot each; the import list against the hooks and every other game's imports named all of them beforehand (33 `nn::` imports no host hook names and no other game imports) ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)): +SDK 7 calls, one boot each; the import list against the hooks and every other game's imports named all of them beforehand (33 `nn::` imports no host hook names and no other game imports) ([Listing a game's unhooked imports before they are met](../../../docs/reference/services.md#listing-a-games-unhooked-imports-before-they-are-met)): - **Resolved: the Poké Ball Plus** (`hid::*Palma*`, 26 functions; the first boot crashed in `hid::detail::SetPalmaBoostMode`, a null read at `sdk+0x238eb0`, the hid service never opened). The host answers that none is connected: no npad has the Palma style, so the game never gets a connection handle, and every call on a handle fails (`hle_hid.c`). - **Resolved: `pdm::QueryApplicationPlayStatistics`** (the second boot: a null read at `sdk+0x336ccc`). The game asks how often this user launched two other games (the Let's Go play statistics) and reads only the launch count, so each is never played (`hle_offline.c`). @@ -53,6 +53,6 @@ SDK 7 calls, one boot each; the import list against the hooks and every other ga General host changes first needed here; each applies to every game: -- **The Poké Ball Plus answered as absent** (`hle_hid.c`), **play statistics as never played** (`hle_offline.c`), `err::ShowNetworkServiceError`, `nvnDeviceGetWindowOriginMode` / `nvnDeviceGetDepthMode`: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **The GFPAK reader** (`gfpak_members`) and **`.tfxo` streams in `main`** (`tfxo_streams`, `tfxo_shaders`): [04, Seen in Pokémon Sword](../../../docs/agents/04-graphics.md#seen-in-pokémon-sword), [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). +- **The Poké Ball Plus answered as absent** (`hle_hid.c`), **play statistics as never played** (`hle_offline.c`), `err::ShowNetworkServiceError`, `nvnDeviceGetWindowOriginMode` / `nvnDeviceGetDepthMode`: [Poké Ball Plus (Palma)](../../../docs/reference/services.md#poké-ball-plus-palma), [Account, friends, play data and Nintendo Account calls](../../../docs/reference/services.md#account-friends-play-data-and-nintendo-account-calls), [System applets opened by `Show…` calls](../../../docs/reference/services.md#system-applets-opened-by-show-calls), [Device getters that were red stubs](../../../docs/reference/nvn-objects.md#device-getters-that-were-red-stubs). +- **The GFPAK reader** (`gfpak_members`) and **`.tfxo` streams in `main`** (`tfxo_streams`, `tfxo_shaders`): [BNSH entries of GFPAK archives](../../../docs/reference/shaders.md#bnsh-entries-of-gfpak-archives), [Shaders in a zlib stream in main (YEBIS .tfxo)](../../../docs/reference/shaders.md#shaders-in-a-zlib-stream-in-main-yebis-tfxo), [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). - **A mixed float / integer pixel shader entry point** (`fs_uint`, `main0_m`, `uint_mask_m`, `pso_mixed_uint`), **cube views counted in cubes** (`tex_by_id`) and **packaged textures whose shared edge pages are written but whose own bytes are not, protected again instead of re-uploaded** (`pkg_edges_take`, `pkg_edges_clean`; `t_nvn` 6c1): same section. diff --git a/dm/games/0100B04011742000/README.md b/dm/games/0100B04011742000/README.md index f63f344..b9d9e03 100644 --- a/dm/games/0100B04011742000/README.md +++ b/dm/games/0100B04011742000/README.md @@ -39,7 +39,7 @@ Game-specific findings. The general host changes they led to are listed under [F - A whole-level clear of a texture made without render-target use (a 32x80 packaged one) upgrades it first (`tex_upgrade`). - About 30 compute shaders read storage buffer addresses from the game's own uniform buffer (c3 + 0x38), marked `sbuf_ubo`. - 218 pixel shaders keep a bindless handle in a loop variable (`c2[0x1A0] | 0x10500000`: separate texture slot 7, constant sampler 0x105); Ryujinx's bindless elimination gave up on the loop phi and the shader did not load (`no resolved slot number`) until the vendored translator followed the phis ([third_party/ryujinx/README.md](../../third_party/ryujinx/README.md)). -- Four size queries on image handles (`cp_t_tcb_48` / `_4A`) are of the kind `tex_slot_h` / `ImageTypesOf` handle ([04, Seen in DRAGON QUEST VII Reimagined](../../../docs/agents/04-graphics.md#seen-in-dragon-quest-vii-reimagined)). +- Four size queries on image handles (`cp_t_tcb_48` / `_4A`) are of the kind `tex_slot_h` / `ImageTypesOf` handle ([A texture slot that shares its c2 word with an image slot](../../../docs/reference/shaders.md#a-texture-slot-that-shares-its-c2-word-with-an-image-slot)). - Shadow vertex inputs: 16,781 side libraries, 344 MB on iOS (21% of its shader libraries, 1.5% of its IPA); its Mac and iPhone runs so far load none. Its 8- and 16-bit integer formats feeding float inputs (red line 99, about 60,000 vertices per submission on its Mac route into the opening) still go through the CPU, since shadows cover only integer inputs. - **Geometry shaders** (53) do not translate: SPIRV-Cross writes `unknown main0_out main0()` for the geometry stage, and Metal has none. @@ -64,7 +64,7 @@ Game-specific findings. The general host changes they led to are listed under [F - The base game is a cartridge dump: the card header's magic `HEAD` sits at 0x100, and its root HFS0 holds the `update`, `logo`, `normal` and `secure` partitions; the game's NCAs are in `secure` (the 7 GB program NCA, control, legal information, HTML manual and the meta NCA), with their key in the key area, so there is no ticket ([01](../../../docs/agents/01-extract.md#nsp--nca--exefs--romfs)). - Everything sits in KPKA PAK archives: `re_chunk_000.pak` plus `re_chunk_000.pak.patch_001.pak` … `patch_016` (19.6 GB in all; KPKA v4, zstd / deflate entries, file names only as murmur hashes), later patch PAKs overriding earlier ones by path hash ([01](../../../docs/agents/01-extract.md#compressed-romfs-content)). `romfs_marker` is `re_chunk_000.pak.patch_001.pak`. -- Shaders: 63,111 in 2,685 SDF files (30,233 vertex, 21,475 pixel, 11,271 compute, 53 geometry, 33 hull, 46 domain) and 102,345 vertex / pixel pairs, in 5 s with 12 jobs (`shader_split.py` `kpka_jobs` / `sdf_shaders`); an SDF entry's stages are one program; control sections 0x880 bytes, compute code without a header ([04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container)). +- Shaders: 63,111 in 2,685 SDF files (30,233 vertex, 21,475 pixel, 11,271 compute, 53 geometry, 33 hull, 46 domain) and 102,345 vertex / pixel pairs, in 5 s with 12 jobs (`shader_split.py` `kpka_jobs` / `sdf_shaders`); an SDF entry's stages are one program; control sections 0x880 bytes, compute code without a header ([SDF definitions in RE Engine PAKs](../../../docs/reference/shaders.md#sdf-definitions-in-re-engine-paks)). ## Rewriter @@ -84,16 +84,16 @@ Run with `DM_FPS_TARGET=30`: the frame numbers count at 30. General host changes first needed here; each applies to every game: -- **Rewriter:** the patch area before the base for big text (`BIG_TEXT`, `DM_PATCH_FRONT`) and disassembly past undecodable words (`disasm_fn`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **`mem_used`** covering the modules: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). -- **Sparse virtual pools** (`vpool_*`, `vm_remap`) and their **4 KB page table** (`vpt_*`, `nvn_sparse_copy`, `nvn_sparse_since`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **SDK 11 calls**, the **inline keyboard** (window size, `ikb_str_arg`, `Calc` results), **`ProfanityFilter` vtable** (`ngc_vtable`), **native-texture H.264 / AAC movies** (`mavc_set_rgba`), the **notification event following the queue** (`mq_ev_sync`, `DM_OE_SCRIPT`) and focus handling modes: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Rewriter:** the patch area before the base for big text (`BIG_TEXT`, `DM_PATCH_FRONT`) and disassembly past undecodable words (`disasm_fn`): [Patch area right before the base for a long text](../../../docs/reference/rewriter.md#patch-area-right-before-the-base-for-a-long-text), [Disassembly goes on after undecodable words](../../../docs/reference/rewriter.md#disassembly-goes-on-after-undecodable-words). +- **`mem_used`** covering the modules: [Heap sized from the application pool less used memory](../../../docs/reference/memory.md#heap-sized-from-the-application-pool-less-used-memory). +- **Sparse virtual pools** (`vpool_*`, `vm_remap`) and their **4 KB page table** (`vpt_*`, `nvn_sparse_copy`, `nvn_sparse_since`): [Sparse virtual pools built from separate physical pools](../../../docs/reference/nvn-objects.md#sparse-virtual-pools-built-from-separate-physical-pools), [Textures read through a 4 KB page table of a sparse pool](../../../docs/reference/nvn-objects.md#textures-read-through-a-4-kb-page-table-of-a-sparse-pool). +- **SDK 11 calls**, the **inline keyboard** (window size, `ikb_str_arg`, `Calc` results), **`ProfanityFilter` vtable** (`ngc_vtable`), **native-texture H.264 / AAC movies** (`mavc_set_rgba`), the **notification event following the queue** (`mq_ev_sync`, `DM_OE_SCRIPT`) and focus handling modes: [Listing a game's unhooked imports before they are met](../../../docs/reference/services.md#listing-a-games-unhooked-imports-before-they-are-met), [Inline keyboard window size, setters and callbacks](../../../docs/reference/services.md#inline-keyboard-window-size-setters-and-callbacks), [Inline keyboard text callbacks and `Calc` states](../../../docs/reference/services.md#inline-keyboard-text-callbacks-and-calc-states), [`nn::ngc::ProfanityFilter` vtable and by-name overloads](../../../docs/reference/services.md#nnngcprofanityfilter-vtable-and-by-name-overloads), [H.264 into an RGBA texture and AAC through AudioToolbox](../../../docs/reference/movies.md#h264-into-an-rgba-texture-and-aac-through-audiotoolbox), [The oe notification event and its queue](../../../docs/reference/threads.md#the-oe-notification-event-and-its-queue), [Focus handling mode decides whether focus changes are notified](../../../docs/reference/services.md#focus-handling-mode-decides-whether-focus-changes-are-notified). - **Both texture tables at once** (separate binds at slots 128 + n, `samp_id`): [04, section 3](../../../docs/agents/04-graphics.md#3-find-the-binding-layout-by-census). -- **One-lane subgroups** (`spv_single_lane`), **indirect-count multi-draws** (`mdx_run`, `mdx_next`), **sampled textures in sparse pools uploaded through the page table**, **the sparse texture layout** (`level_walk`), **`sparse_view`**, **`pend_read`**, **`tex_upgrade`**, **`sbuf_ubo`**, **bindless handles through loop phis**: [04, Seen in Monster Hunter Rise](../../../docs/agents/04-graphics.md#seen-in-monster-hunter-rise). +- **One-lane subgroups** (`spv_single_lane`), **indirect-count multi-draws** (`mdx_run`, `mdx_next`), **sampled textures in sparse pools uploaded through the page table**, **the sparse texture layout** (`level_walk`), **`sparse_view`**, **`pend_read`**, **`tex_upgrade`**, **`sbuf_ubo`**, **bindless handles through loop phis**: [Subgroup operations rewritten for a one-lane subgroup](../../../docs/reference/shaders.md#subgroup-operations-rewritten-for-a-one-lane-subgroup), [Indirect-count multi-draws](../../../docs/reference/rendering.md#indirect-count-multi-draws), [Sampled textures in a sparse pool uploaded through the page table](../../../docs/reference/nvn-objects.md#sampled-textures-in-a-sparse-pool-uploaded-through-the-page-table), [Sparse textures laid out in 64 KB tiles](../../../docs/reference/rendering.md#sparse-textures-laid-out-in-64-kb-tiles), [Sparse textures bound from their first mapped level](../../../docs/reference/rendering.md#sparse-textures-bound-from-their-first-mapped-level), [Handles in uniform buffers written earlier in the same submission](../../../docs/reference/shaders.md#handles-in-uniform-buffers-written-earlier-in-the-same-submission), [Render-target use added on demand](../../../docs/reference/rendering.md#render-target-use-added-on-demand), [Storage buffer addresses in the game's own uniform buffer](../../../docs/reference/shaders.md#storage-buffer-addresses-in-the-games-own-uniform-buffer), [Bindless handles through a loop phi](../../../docs/reference/shaders.md#bindless-handles-through-a-loop-phi). - **Spin waits through stores** (`patch_nso.py` "Spin waits"; this game's `main` has the most new loops): [development.md, Power](../../../docs/development.md#power). ## Open -- Texture layout: NVN's automatic block height now truncates for every game (`pick_tile(ah * 3 / 16)`), which matters for this game's sparse textures; not re-checked here ([04, Seen in Tears of the Kingdom](../../../docs/agents/04-graphics.md#seen-in-the-legend-of-zelda-tears-of-the-kingdom)). +- Texture layout: NVN's automatic block height now truncates for every game (`pick_tile(ah * 3 / 16)`), which matters for this game's sparse textures; not re-checked here ([Automatic block height truncates](../../../docs/reference/rendering.md#automatic-block-height-truncates)). - The 53 geometry shaders and the 121 fully dynamic handles (type 4) are not supported. - 8- and 16-bit integer formats feeding float inputs still go through the CPU conversion (red line 99). diff --git a/dm/games/0100B7C00933A000/README.md b/dm/games/0100B7C00933A000/README.md index 700b20d..51521c5 100644 --- a/dm/games/0100B7C00933A000/README.md +++ b/dm/games/0100B7C00933A000/README.md @@ -58,9 +58,9 @@ Status: on the Mac, the boot, the title (a VP9 movie behind the logo), the main General host changes first needed here; each applies to every game: -- **A network request kept on hold for 50 ms** (`hle_net.c`, `nn::nifm`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Rewriter:** an `adrp` re-encoded inside a copied exclusive window (`window_adrp`) and gates before a `ret` (`scratch_dead`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). Spin waits that store (`patch_nso.py` "Spin waits"): [development.md, Power](../../../docs/development.md#power). -- **An Unreal engine region whose default is cut instead of capped by `game_args`**: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). +- **A network request kept on hold for 50 ms** (`hle_net.c`, `nn::nifm`): [Network requests stay on hold before they settle](../../../docs/reference/services.md#network-requests-stay-on-hold-before-they-settle). +- **Rewriter:** an `adrp` re-encoded inside a copied exclusive window (`window_adrp`) and gates before a `ret` (`scratch_dead`): [`adrp` inside an exclusive window](../../../docs/reference/rewriter.md#adrp-inside-an-exclusive-window), [Observation gates where x9–x17 are dead](../../../docs/reference/rewriter.md#observation-gates-where-x9x17-are-dead). Spin waits that store (`patch_nso.py` "Spin waits"): [development.md, Power](../../../docs/development.md#power). +- **An Unreal engine region whose default is cut instead of capped by `game_args`**: [Unreal's engine address region and its options](../../../docs/reference/memory.md#unreals-engine-address-region-and-its-options). ## Open diff --git a/dm/games/0100B99019412000/README.md b/dm/games/0100B99019412000/README.md index af30903..187cb8d 100644 --- a/dm/games/0100B99019412000/README.md +++ b/dm/games/0100B99019412000/README.md @@ -50,8 +50,8 @@ The cutscenes are MP4 files (`Movie/*.mp4`, H.264 1080p and AAC stereo) played t General host changes first needed here; each applies to every game: -- **32 outputs for AAC movie decoders and one AAC frame from `GetOutputBufferSize`** (`MD_NOUT_AAC` in `hle_movie.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Advanced blend modes** (`blend_st`, `adv_blend`): [04, Seen in Mario vs. Donkey Kong](../../../docs/agents/04-graphics.md#seen-in-mario-vs-donkey-kong). +- **32 outputs for AAC movie decoders and one AAC frame from `GetOutputBufferSize`** (`MD_NOUT_AAC` in `hle_movie.c`): [AAC decoders offer at most 32 outputs](../../../docs/reference/movies.md#aac-decoders-offer-at-most-32-outputs), [AAC output buffer size is one frame](../../../docs/reference/movies.md#aac-output-buffer-size-is-one-frame). +- **Advanced blend modes** (`blend_st`, `adv_blend`): [Advanced blend modes as Porter-Duff factors](../../../docs/reference/rendering.md#advanced-blend-modes-as-porter-duff-factors). - **A value filter for `DM_PROBE_POLL`** (`=`, for a word that changes often before the bad store): [05, Probes without a debugger](../../../docs/agents/05-debugging.md#probes-without-a-debugger-dm_probe). ## Open diff --git a/dm/games/0100BF00112C0000/README.md b/dm/games/0100BF00112C0000/README.md index 9856aa9..ce0a1df 100644 --- a/dm/games/0100BF00112C0000/README.md +++ b/dm/games/0100BF00112C0000/README.md @@ -61,10 +61,10 @@ Brought up on the Mac: the ATLUS logo, the title, the puzzle type choice, the Go General host changes first needed here; each applies to every game: -- **GLSLC GPU code version range follows the game's own GLSLC** (`hle_glslc.c` `dm_glslc_gpu_code`, `hle_nvn.c`; on Bayonetta 3 the reported newest version came to follow the SDK, 1.15 from SDK 8 after this game's GLSLC 8.2.1): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **VertexA / VertexB merge** (`shader_split.py` `sph_stage`, `dmsh_build.py` `vertex_ab`), explained in full under [Shaders](#shaders): [04, Seen in Catherine: Full Body](../../../docs/agents/04-graphics.md#seen-in-catherine-full-body). +- **GLSLC GPU code version range follows the game's own GLSLC** (`hle_glslc.c` `dm_glslc_gpu_code`, `hle_nvn.c`; on Bayonetta 3 the reported newest version came to follow the SDK, 1.15 from SDK 8 after this game's GLSLC 8.2.1): [GLSLC GPU code version against the driver's range](../../../docs/reference/shaders.md#glslc-gpu-code-version-against-the-drivers-range), [Newest GPU code version reported by the driver](../../../docs/reference/nvn-objects.md#newest-gpu-code-version-reported-by-the-driver). +- **VertexA / VertexB merge** (`shader_split.py` `sph_stage`, `dmsh_build.py` `vertex_ab`), explained in full under [Shaders](#shaders): [VertexA and VertexB parts merged into one vertex shader](../../../docs/reference/shaders.md#vertexa-and-vertexb-parts-merged-into-one-vertex-shader). - **Commit at the first pass boundary after a buffer → texture copy** (`g_cut_copy`, `sub_cut`; always on, even with `DM_NVN_CUT=0`): same section. -- **`nv12-uv-stride` in the movie decoder's output format** (`md_format` in `hle_movie.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **`nv12-uv-stride` in the movie decoder's output format** (`md_format` in `hle_movie.c`): [Output format keys (`md_format`)](../../../docs/reference/movies.md#output-format-keys-md_format). - **`nn::settings::GetAvailableLanguageCodeCount`** (`hle_sdk16.c`): same section. ## Open diff --git a/dm/games/0100C2A01A730000/README.md b/dm/games/0100C2A01A730000/README.md index a686058..339caaf 100644 --- a/dm/games/0100C2A01A730000/README.md +++ b/dm/games/0100C2A01A730000/README.md @@ -44,8 +44,8 @@ Update 1.1.3 (v851968) over the base game, with its four DLC. SDK 17.5.3 (Ace At General host changes first needed here; each applies to every game: - **Multi-program updates:** `nx_extract` takes program index 0's NCAs: [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). -- **The newer RDB** (`.fdata`, `idrk_data`): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **`OpenAudioRenderer` calling its no-event form through the PLT** (`t_in_open` in `hle_audren.c`): [03, Audio](../../../docs/agents/03-bringup.md#audio). +- **The newer RDB** (`.fdata`, `idrk_data`): [G2S containers in KTGL RDB archives](../../../docs/reference/shaders.md#g2s-containers-in-ktgl-rdb-archives). +- **`OpenAudioRenderer` calling its no-event form through the PLT** (`t_in_open` in `hle_audren.c`): [The SDK's open with an event calls the open without one (`t_in_open`)](../../../docs/reference/audio.md#the-sdks-open-with-an-event-calls-the-open-without-one-t_in_open). ## Open diff --git a/dm/games/0100CD801CE5E000/README.md b/dm/games/0100CD801CE5E000/README.md index 6f46294..e8ad848 100644 --- a/dm/games/0100CD801CE5E000/README.md +++ b/dm/games/0100CD801CE5E000/README.md @@ -56,10 +56,10 @@ LÖVE's graphics module is its own OpenGL backend, so its choices follow what th General host changes first needed here; each applies to every game: -- **The GL layer's core-profile paths** (`GL_CONTEXT_PROFILE_MASK`, persistently mapped buffers and fences, attribute bindings and current values, uniform queries, texture storage, 3D and array textures, swizzles, mipmaps, base-vertex and instanced draws, blits) in `hle_gl.m`, and **`DM_GL_STUBS=1`**: [04, Seen in Balatro](../../../docs/agents/04-graphics.md#seen-in-balatro-löve-114-on-opengl). +- **The GL layer's core-profile paths** (`GL_CONTEXT_PROFILE_MASK`, persistently mapped buffers and fences, attribute bindings and current values, uniform queries, texture storage, 3D and array textures, swizzles, mipmaps, base-vertex and instanced draws, blits) in `hle_gl.m`, and **`DM_GL_STUBS=1`**: [The GL core profile and persistently mapped buffers](../../../docs/reference/rendering.md#the-gl-core-profile-and-persistently-mapped-buffers), [GL uniforms, pipelines and attributes](../../../docs/reference/rendering.md#gl-uniforms-pipelines-and-attributes), [GL textures, swizzles and uploads](../../../docs/reference/rendering.md#gl-textures-swizzles-and-uploads), [GL draws and framebuffers](../../../docs/reference/rendering.md#gl-draws-and-framebuffers), [Which GL functions a game calls](../../../docs/reference/rendering.md#which-gl-functions-a-game-calls). - **`loader_census.py` following registers** (LÖVE's glad), and `vkGetInstanceProcAddr` left out of `profile_guess.py`'s Vulkan count: same section. - **GL shaders captured at run time** (`DM_GL_CAPTURE`, `gl_capture.tsv`): [development.md, The packing script in detail](../../../docs/development.md#the-packing-script-in-detail). -- **A renderer opened without an event** starts the render thread (`ar_open`): [03, Audio](../../../docs/agents/03-bringup.md#audio). +- **A renderer opened without an event** starts the render thread (`ar_open`): [A renderer opened without an event](../../../docs/reference/audio.md#a-renderer-opened-without-an-event). - **The Tap pad layout** (`pad_layout touch`, `DmGamepad.m`): [dm/games/README.md](../README.md). ## Open diff --git a/dm/games/0100D9F01D474000/README.md b/dm/games/0100D9F01D474000/README.md index 7eb4530..407fd1e 100644 --- a/dm/games/0100D9F01D474000/README.md +++ b/dm/games/0100D9F01D474000/README.md @@ -41,9 +41,9 @@ One BNSH, `shader/shader_NX.bnsh` (2 vertex and 33 pixel shaders, all 2D), with General host changes first needed here; each applies to every game: -- **Initial content of `CPU_NO_ACCESS` pools** (`tex_init_fill`) and **linear textures in any pool** (`tex_linear`): [04, Seen in Rhythm Heaven Groove](../../../docs/agents/04-graphics.md#seen-in-rhythm-heaven-groove). -- **Truncated automatic block height** (`pick_tile(ah * 3 / 16)` in `nvn_tex.c`): same section, and [04, Seen in Tears of the Kingdom](../../../docs/agents/04-graphics.md#seen-in-the-legend-of-zelda-tears-of-the-kingdom). -- **SDK 21's DSP command list without magic** (`capture` in `hle_audren.c`), **`pl::GetSharedFont`** for every font at once (`hle_sdk16.c`) and **`vi::GetDefaultDisplayPanelIdentifier`** (`hle_vi.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Initial content of `CPU_NO_ACCESS` pools** (`tex_init_fill`) and **linear textures in any pool** (`tex_linear`): [A pool's storage at creation is its textures' content](../../../docs/reference/nvn-objects.md#a-pools-storage-at-creation-is-its-textures-content), [Linear textures in CPU-visible pools](../../../docs/reference/nvn-objects.md#linear-textures-in-cpu-visible-pools). +- **Truncated automatic block height** (`pick_tile(ah * 3 / 16)` in `nvn_tex.c`): [Automatic block height truncates](../../../docs/reference/rendering.md#automatic-block-height-truncates). +- **SDK 21's DSP command list without magic** (`capture` in `hle_audren.c`), **`pl::GetSharedFont`** for every font at once (`hle_sdk16.c`) and **`vi::GetDefaultDisplayPanelIdentifier`** (`hle_vi.c`): [SDK 21 DSP command header without a magic](../../../docs/reference/audio.md#sdk-21-dsp-command-header-without-a-magic), [Shared fonts requested all at once](../../../docs/reference/services.md#shared-fonts-requested-all-at-once), [Default display panel identifier](../../../docs/reference/services.md#default-display-panel-identifier). - **`perf_mode`** (Boost whatever the operation mode, `hle_oe.c`): [dm/games/README.md, Keys](../README.md#keys). ## Open diff --git a/dm/games/0100DCA0064A6000/README.md b/dm/games/0100DCA0064A6000/README.md index d53c66d..8c34c78 100644 --- a/dm/games/0100DCA0064A6000/README.md +++ b/dm/games/0100DCA0064A6000/README.md @@ -58,9 +58,9 @@ A deferred renderer: the scene draws into a G-buffer of two integer targets (R32 General host changes first needed here; each applies to every game: -- **Single texture binds recorded in the hardware's layout** (`OP_BIND_TEX1`, `p_bind_tex1`, `bind_rec`): [03, NVN object sizes](../../../docs/agents/03-bringup.md#nvn-object-sizes-memory-corruption). -- **An integer-output entry point** (`fs_uint`, `main0_u`, `pso_all_uint`), **driver-table handle arrays** (`c2_array`, `tex_c2a` / `samp_c2a`, `NVN_RB_TEXA` / `NVN_RB_SAMPA`, `c2_handle`), **depth textures drawn through a colour view** (`shrt_block`) and the **`draw with geometry shader … which has no library`** line: [04, Seen in Luigi's Mansion 3](../../../docs/agents/04-graphics.md#seen-in-luigis-mansion-3). -- **Pixel-format-view usage on every depth-stencil texture** (`tex_desc`, PR #106): [04, Seen in Pokémon Legends: Z-A](../../../docs/agents/04-graphics.md#seen-in-pokémon-legends-z-a). +- **Single texture binds recorded in the hardware's layout** (`OP_BIND_TEX1`, `p_bind_tex1`, `bind_rec`): [Recorded commands patched in place by the game (`OP_BIND_TEX1`)](../../../docs/reference/nvn-objects.md#recorded-commands-patched-in-place-by-the-game-op_bind_tex1). +- **An integer-output entry point** (`fs_uint`, `main0_u`, `pso_all_uint`), **driver-table handle arrays** (`c2_array`, `tex_c2a` / `samp_c2a`, `NVN_RB_TEXA` / `NVN_RB_SAMPA`, `c2_handle`), **depth textures drawn through a colour view** (`shrt_block`) and the **`draw with geometry shader … which has no library`** line: [Integer entry points for integer colour outputs](../../../docs/reference/shaders.md#integer-entry-points-for-integer-colour-outputs), [An array of handles from the driver table c2](../../../docs/reference/shaders.md#an-array-of-handles-from-the-driver-table-c2), [A depth texture drawn through a colour view](../../../docs/reference/rendering.md#a-depth-texture-drawn-through-a-colour-view), [Geometry shaders with no Metal stage](../../../docs/reference/shaders.md#geometry-shaders-with-no-metal-stage). +- **Pixel-format-view usage on every depth-stencil texture** (`tex_desc`, PR #106): [Stencil-mode texture views read through `X32_Stencil8`](../../../docs/reference/rendering.md#stencil-mode-texture-views-read-through-x32_stencil8). - **The engine's `.dict` / `.data` / `.patch` archives** (`nlg_shaders`, `nlg_patch_section`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). - **The frame limiter's milliseconds written in game memory** (`hle_fpslock.c`, `fps_target` with `present_interval display`): [the profile keys](../README.md#keys). diff --git a/dm/games/0100E44016C22000/README.md b/dm/games/0100E44016C22000/README.md index a6a51f7..7a23ee7 100644 --- a/dm/games/0100E44016C22000/README.md +++ b/dm/games/0100E44016C22000/README.md @@ -41,9 +41,9 @@ Update 1.0.1 (v65536) over the base game, SDK 13.3.7. Unity 2020.3.30f1 IL2CPP w General host changes first needed here; each applies to every game: -- **The shared-font address is the TTF** unless the game is on SDK 11 or later and imports `ScalableFontEngineHelper::Decode` (`font_skip`, `dm_main_imports`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk), [dm/games/README.md, Keys](../README.md#keys) (`shared_font`). -- **Unity 2020.3's `Shader` layout written out** (`SHADER_2020_3`, `_SHADER_COMMON`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **Finding the AddressRegionManager constructor on a new `sdk` build**: [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). +- **The shared-font address is the TTF** unless the game is on SDK 11 or later and imports `ScalableFontEngineHelper::Decode` (`font_skip`, `dm_main_imports`): [Shared font address: BFTTF header or bare TTF](../../../docs/reference/services.md#shared-font-address-bfttf-header-or-bare-ttf), [dm/games/README.md, Keys](../README.md#keys) (`shared_font`). +- **Unity 2020.3's `Shader` layout written out** (`SHADER_2020_3`, `_SHADER_COMMON`): [Unity Shader layouts written out for players without bundles](../../../docs/reference/shaders.md#unity-shader-layouts-written-out-for-players-without-bundles). +- **Finding the AddressRegionManager constructor on a new `sdk` build**: [The SDK's address-region node area](../../../docs/reference/memory.md#the-sdks-address-region-node-area). ## Open diff --git a/dm/games/0100E65002BB8000/README.md b/dm/games/0100E65002BB8000/README.md index a5678b2..11a04f1 100644 --- a/dm/games/0100E65002BB8000/README.md +++ b/dm/games/0100E65002BB8000/README.md @@ -45,7 +45,7 @@ The provisional profile's guesses held. General host changes first needed here (PR #120, before this profile's change); each applies to every game: -- **Compressed XNB** decompressed before the MGFX reader (`xnb_decompress` in `shader_split.py`): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). +- **Compressed XNB** decompressed before the MGFX reader (`xnb_decompress` in `shader_split.py`): [MGFX effects in MonoGame XNB files](../../../docs/reference/shaders.md#mgfx-effects-in-monogame-xnb-files). - **The Mac software keyboard's text box** releases the buttons that opened it and stays up below the minimum length (`front_mac.m`): described in [Input](#input) above. - **Spin waits through stores** (`patch_nso.py` "Spin waits"): one more loop in this game's `main`: [development.md, Power](../../../docs/development.md#power). diff --git a/dm/games/0100ECD018EBE000/README.md b/dm/games/0100ECD018EBE000/README.md index 5459d07..3de2aea 100644 --- a/dm/games/0100ECD018EBE000/README.md +++ b/dm/games/0100ECD018EBE000/README.md @@ -40,11 +40,11 @@ Update 1.0.1 (v65536), SDK 15.3.2. Intelligent Systems' engine on NintendoWare ( ## Services -- **Resolved: abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodes`** (early in `nnMain`, `main+0x327888`). The game asks for up to 32 codes and takes the index of `oe::GetDesiredLanguage`'s code in the list as its language. The host lists the system's codes in `nn::settings::Language` order, the table `LanguageCode::Make` uses (`hle_sdk16.c`). It was the one import of `main` that no host hook names and that reaches a service; `fs::MountRamDisk` and the swkbd and hid helpers it also imports run in `sdk` or were never called ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)). +- **Resolved: abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodes`** (early in `nnMain`, `main+0x327888`). The game asks for up to 32 codes and takes the index of `oe::GetDesiredLanguage`'s code in the list as its language. The host lists the system's codes in `nn::settings::Language` order, the table `LanguageCode::Make` uses (`hle_sdk16.c`). It was the one import of `main` that no host hook names and that reaches a service; `fs::MountRamDisk` and the swkbd and hid helpers it also imports run in `sdk` or were never called ([Listing a game's unhooked imports before they are met](../../../docs/reference/services.md#listing-a-games-unhooked-imports-before-they-are-met)). ## Rendering -11,858 shaders (3,154 vertex, 7,826 pixel, 874 compute, 4 domain), all BNSH; every one builds with Mario's binding layout ([04, Seen in Paper Mario](../../../docs/agents/04-graphics.md#seen-in-paper-mario-the-thousand-year-door)). +11,858 shaders (3,154 vertex, 7,826 pixel, 874 compute, 4 domain), all BNSH; every one builds with Mario's binding layout ([Zstd files recognised by their magic](../../../docs/reference/shaders.md#zstd-files-recognised-by-their-magic)). - **Current: a texture picked at run time between bound slots** (2,386 pixel shaders: 5,476 cube samples, 487 2D). The handle is read from the driver table of combined texture handles (c2 `0x20 + 8 × slot`, slots 0, 7, 8, 13 and 28, constant offsets only) and chosen by branches, so the translator falls back to `array, 32> fp_tcube_cb0_40000000`. `tex_dyn_slots` puts `r | r << 20` in the combined table's slot r, so the handle names element r, which holds the texture bound in slot r. The array's other elements hold whatever the other slots do, mostly 2D textures where the array is of cubes; such an element gets the zero cube without a type mismatch report (`tex_fit_elem`). - **Resolved: an array of handles in a uniform buffer** (the reflection shaders, `reflection.bfsha`: `array, 32> fp_t2d_cb14_0` and its samplers). A loop reads one handle per light from the game's uniform buffer c14 (uniform slot 11) at a computed offset; Ryujinx's texture array from a buffer indexes it by the handle's byte offset ÷ 8 (an NVN handle is 64-bit, the shader computes `(word >> 2) << 1 | (word & 3) >> 1`). The build read the name as one bindless texture (`tex_ubo`, offset 0), so the argument buffer was 32 entries longer than the reflection said and the host stopped at the shader's first load (`unexpected argument buffer layout: encodedLength=640 maxAid=48`). The first run with the fix showed elements 0–3, 6 and 8–12 holding textures, among them the shader's fixed handles at +8, +16 and +24 and the lights' from +64. No chain change: the build already wrote the array type. @@ -76,9 +76,9 @@ Update 1.0.1 (v65536), SDK 15.3.2. Intelligent Systems' engine on NintendoWare ( General host changes first needed here; each applies to every game: - **Spin waits:** the rewriter copies pure poll loops and sends their back-edges through a gate that polls 2 ms, then sleeps (`patch_nso.py` "Spin waits", `dm_spin_wait`, `DM_SPIN`, `DM_SPIN_POLL`, `DM_SPIN_LIST`): [development.md, Power](../../../docs/development.md#power), [05, Scripted input and thread scheduling](../../../docs/agents/05-debugging.md#scripted-input-and-thread-scheduling). -- **`sub sp, sp, xN` rounded up to 16** (`site_sp_align`; `insn_census.py` counts the sites): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **Zstd by magic** in `shader_split.py`, **`tex_dyn_slots` for combined handles** (`tex_fit_elem`), **handle arrays in a uniform buffer** (`NVN_RB_TEXU` / `SAMPU` with `n`, array length from the MSL type), **integer clears** converted at replay, **render-target use for a copy destination** (`tex_upgrade`) and **an unbound uniform slot reading the all-zero bank**: [04, Seen in Paper Mario](../../../docs/agents/04-graphics.md#seen-in-paper-mario-the-thousand-year-door). -- **`nn::settings::GetAvailableLanguageCodes`** (`hle_sdk16.c`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **`sub sp, sp, xN` rounded up to 16** (`site_sp_align`; `insn_census.py` counts the sites): [Stack arguments reserved with sub sp, sp, xN](../../../docs/reference/rewriter.md#stack-arguments-reserved-with-sub-sp-sp-xn). +- **Zstd by magic** in `shader_split.py`, **`tex_dyn_slots` for combined handles** (`tex_fit_elem`), **handle arrays in a uniform buffer** (`NVN_RB_TEXU` / `SAMPU` with `n`, array length from the MSL type), **integer clears** converted at replay, **render-target use for a copy destination** (`tex_upgrade`) and **an unbound uniform slot reading the all-zero bank**: [Zstd files recognised by their magic](../../../docs/reference/shaders.md#zstd-files-recognised-by-their-magic), [Texture slots picked at run time](../../../docs/reference/shaders.md#texture-slots-picked-at-run-time), [An array of handles in a uniform buffer](../../../docs/reference/shaders.md#an-array-of-handles-in-a-uniform-buffer), [Integer colour clears](../../../docs/reference/rendering.md#integer-colour-clears), [Render-target use added on demand](../../../docs/reference/rendering.md#render-target-use-added-on-demand), [An unbound uniform slot reads the zero bank](../../../docs/reference/rendering.md#an-unbound-uniform-slot-reads-the-zero-bank). +- **`nn::settings::GetAvailableLanguageCodes`** (`hle_sdk16.c`): [Available language codes and their count](../../../docs/reference/services.md#available-language-codes-and-their-count). ## Open diff --git a/dm/games/0100F2C0115B6000/README.md b/dm/games/0100F2C0115B6000/README.md index 079f030..b84565f 100644 --- a/dm/games/0100F2C0115B6000/README.md +++ b/dm/games/0100F2C0115B6000/README.md @@ -46,11 +46,11 @@ Game-specific findings. The general host changes they led to are listed under [F ## Command memory -- **Resolved (issue #61): `CRASH sig=10` in `mtl_exec` after minutes of play**, the fault address equal to `x19` and shaped like `0x1000000003`. The game gives each model's prerecorded lists fixed slots of the size a recording of that list measured (`GetCommandMemoryUsed`), slot k at base + size × k, and passes `AddCommandMemory` max(size, 512) (`main+0xbb120`); its memory callback is empty. A JUMP the host wrote after a 496-byte list reached 8 bytes into the next slot. The host now never writes past what `GetCommandMemoryUsed` reported (`OP_RESERVE`, [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)). +- **Resolved (issue #61): `CRASH sig=10` in `mtl_exec` after minutes of play**, the fault address equal to `x19` and shaped like `0x1000000003`. The game gives each model's prerecorded lists fixed slots of the size a recording of that list measured (`GetCommandMemoryUsed`), slot k at base + size × k, and passes `AddCommandMemory` max(size, 512) (`main+0xbb120`); its memory callback is empty. A JUMP the host wrote after a 496-byte list reached 8 bytes into the next slot. The host now never writes past what `GetCommandMemoryUsed` reported (`OP_RESERVE`, [Prerecorded lists in slots of their measured size](../../../docs/reference/nvn-objects.md#prerecorded-lists-in-slots-of-their-measured-size)). ## Services -- SDK 20 calls it needed (each aborted in `sm::Initialize` or read a null applet proxy, one per boot): `oe::GetPreviousProgramIndex`, the hardware Opus decoder's C interface, cache storage, `fs::IsSaveDataExisting` for another title (it asks for Breath of the Wild's save: none), `MountSaveDataReadOnly`, `time::ClockSnapshot::CreateWithStandardSystemClock` ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)). +- SDK 20 calls it needed (each aborted in `sm::Initialize` or read a null applet proxy, one per boot): `oe::GetPreviousProgramIndex`, the hardware Opus decoder's C interface, cache storage, `fs::IsSaveDataExisting` for another title (it asks for Breath of the Wild's save: none), `MountSaveDataReadOnly`, `time::ClockSnapshot::CreateWithStandardSystemClock` ([Applet resource user ID, launch parameters and VR mode](../../../docs/reference/services.md#applet-resource-user-id-launch-parameters-and-vr-mode), [Hardware Opus decoders](../../../docs/reference/audio.md#hardware-opus-decoders-built-on-the-sdks-software-decoders), [Cache storage and other titles' save data](../../../docs/reference/services.md#cache-storage-and-other-titles-save-data), [Steady clock, clock snapshots and user clocks](../../../docs/reference/services.md#steady-clock-clock-snapshots-and-user-clocks)). - `main` imports `nn::ro` (it loads NRO modules at run time). - Focus handling mode 2 (focus notifications are sent). @@ -68,10 +68,10 @@ Game-specific findings. The general host changes they led to are listed under [F General host changes first needed here; each applies to every game: -- **Conditional rendering** evaluated at replay (`OP_RENDER_COND`, profile key `render_cond`) and **CPU-written textures with no flush** (`tex_dirty_upload`, `ft_add`): [04, Seen in Tears of the Kingdom](../../../docs/agents/04-graphics.md#seen-in-the-legend-of-zelda-tears-of-the-kingdom). +- **Conditional rendering** evaluated at replay (`OP_RENDER_COND`, profile key `render_cond`) and **CPU-written textures with no flush** (`tex_dirty_upload`, `ft_add`): [Conditional rendering evaluated at replay](../../../docs/reference/rendering.md#conditional-rendering-evaluated-at-replay), [CPU writes with no flush, caught by page protection](../../../docs/reference/nvn-objects.md#cpu-writes-with-no-flush-caught-by-page-protection-tex_dirty_upload). - **Mid-pass, partial-mask and partial-scissor clears** drawn as a triangle (`clear_color_tri` / `clear_ds_tri`), **8-bit indices widened on the GPU** when written in the same submission (`wide_run`), **`ReadTexels` from guest memory on any thread** with `nvn_ft_poll` marking textures written after first use, **attachment-less passes** (`bare_dims`): same section. -- **Rewriter:** out-of-line atomic retry blocks (`retry_block` / `site_retry_window`, 8 sites) and jump tables relative to an `adr` (`jump_adr_offsets`, `main+0x393bb0`): [03, Rewriter](../../../docs/agents/03-bringup.md#rewriter). -- **Slotted command lists** (`OP_RESERVE`, `t_nvn` "slotted lists"): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). +- **Rewriter:** out-of-line atomic retry blocks (`retry_block` / `site_retry_window`, 8 sites) and jump tables relative to an `adr` (`jump_adr_offsets`, `main+0x393bb0`): [Out-of-line retry blocks of atomic updates](../../../docs/reference/rewriter.md#out-of-line-retry-blocks-of-atomic-updates), [Offset tables relative to an adr](../../../docs/reference/rewriter.md#offset-tables-relative-to-an-adr). +- **Slotted command lists** (`OP_RESERVE`, `t_nvn` "slotted lists"): [Prerecorded lists in slots of their measured size](../../../docs/reference/nvn-objects.md#prerecorded-lists-in-slots-of-their-measured-size). ## Open diff --git a/dm/games/0100F43008C44000/README.md b/dm/games/0100F43008C44000/README.md index 5fa91d2..403d6c7 100644 --- a/dm/games/0100F43008C44000/README.md +++ b/dm/games/0100F43008C44000/README.md @@ -24,7 +24,7 @@ Update 2.0.2 (v393216) over the base game, SDK 19.3.11 (NVN 55). Game Freak's Tr ## Services -The import list against the hooks (the survey's `nso_syms.py` table, then `main`'s undefined `nn::` symbols that no `HLE_FORCE` names) left 86 candidates; most only fill the game's own structures (error and web argument builders, `prepo` report buffers, `swkbd` configuration, the audio resampler, `nn::async`). Those that reach a service were hooked before they were met where the answer is clear: `time::ToPosixTime` with an explicit rule (the host's zone, as for `ToCalendarTime`), `album::SaveCurrentScreenshot` (no album: fails), `err::ShowApplicationError` (logged) and the `ShowShopAddOnContentList(UserHandle)` overload. Then one boot each ([03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk)): +The import list against the hooks (the survey's `nso_syms.py` table, then `main`'s undefined `nn::` symbols that no `HLE_FORCE` names) left 86 candidates; most only fill the game's own structures (error and web argument builders, `prepo` report buffers, `swkbd` configuration, the audio resampler, `nn::async`). Those that reach a service were hooked before they were met where the answer is clear: `time::ToPosixTime` with an explicit rule (the host's zone, as for `ToCalendarTime`), `album::SaveCurrentScreenshot` (no album: fails), `err::ShowApplicationError` (logged) and the `ShowShopAddOnContentList(UserHandle)` overload. Then one boot each ([Listing a game's unhooked imports before they are met](../../../docs/reference/services.md#listing-a-games-unhooked-imports-before-they-are-met)): - **Resolved: abort in `sm::Initialize` under `crypto::GenerateCryptographicallyRandomBytes`** (the first call in `nnMain`, `main+0x1734b4`): the bytes come from the csrng service; the host fills them from its entropy source, the same stream as `getentropy` (`hle_os.c`). - **Resolved: a null read in `audio::GetAudioOutBufferCount` on Wwise's sink wakeup thread** (`sdk+0x1e7ed0`, from `main+0x30faae8`): after every release event Wwise asks how many buffers are still queued and marks the sink starved at 0; the SDK's version read the audout session the host never made. It answers from the host queue: appended and not yet played to the end (`hle_audio.c`). @@ -64,11 +64,11 @@ Each item was found from a red line or a pass dump on the way through the langua General host changes first needed here; each applies to every game: -- **The pool enforced as on Horizon** (`mem_map_limit`, `hle_svc.c`): [the profile keys](../README.md#keys), [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Trinity packs decoded with the game's own Oodle** (`trpfs_jobs` / `trinity_shaders`, `hle_oodle.c`, `DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script for any game with `ONEPACK` files): [04, section 1](../../../docs/agents/04-graphics.md#1-find-every-shader-container), [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). -- **SDK 19 calls:** csrng bytes, `GetAudioOutBufferCount` from the host queue, NVN state getters, `WaitForAddress` type 3, `OP_FENCE_SYNC`: [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **Transform feedback** (`tf_layout`, `vs_tf` / `main0_tf`, `TF_SLOT`, `OP_TF_BIND` / `OP_TF_STATE` / `OP_RAST_DISCARD`, `tf_draw`, `pend_add` kind 4), the **feedback encoder ending `g_blit`**, **layered pipelines** (`vs_layer`, `layer_out`), **function constants specialized** (`shader_load`), the **zero cube for a 2D texture** (`tex_fit`), **layered passes** (`g_rp_layered`, `g_pass.layered`), **stencil-mode views** (`tex_by_id`, `X32_Stencil8`; every depth-stencil texture with pixel-format-view usage, `tex_desc`, PR #106, `t_stencil_view`), **the base instance in the driver segment** (`encode_args`, `c0[0x644]`; `nvn_refl.m` treats any declared driver-segment binding as read), **reduction samplers** (`samp_by_id`) and **pool creation retries** (`mtl_pool_add`): [04, Seen in Pokémon Legends: Z-A](../../../docs/agents/04-graphics.md#seen-in-pokémon-legends-z-a). -- **Image atomics into a scratch buffer as large as the image** (`atom_scratch`): [04, section 6](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). +- **The pool enforced as on Horizon** (`mem_map_limit`, `hle_svc.c`): [the profile keys](../README.md#keys), [Maps past the application pool fail as on Horizon](../../../docs/reference/memory.md#maps-past-the-application-pool-fail-as-on-horizon). +- **Trinity packs decoded with the game's own Oodle** (`trpfs_jobs` / `trinity_shaders`, `hle_oodle.c`, `DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script for any game with `ONEPACK` files): [BNSH in Trinity packs](../../../docs/reference/shaders.md#bnsh-in-trinity-packs), [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). +- **SDK 19 calls:** csrng bytes, `GetAudioOutBufferCount` from the host queue, NVN state getters, `WaitForAddress` type 3, `OP_FENCE_SYNC`: [Cryptographically random bytes](../../../docs/reference/services.md#cryptographically-random-bytes), [AudioOut buffer count from the host queue](../../../docs/reference/audio.md#audioout-buffer-count-from-the-host-queue), [State read back through NVN getters](../../../docs/reference/nvn-objects.md#state-read-back-through-nvn-getters), [WaitForAddress type 3](../../../docs/reference/threads.md#waitforaddress-type-3-64-bit-wait-if-equal), [Fence syncs recorded in command buffers](../../../docs/reference/nvn-objects.md#fence-syncs-recorded-in-command-buffers). +- **Transform feedback** (`tf_layout`, `vs_tf` / `main0_tf`, `TF_SLOT`, `OP_TF_BIND` / `OP_TF_STATE` / `OP_RAST_DISCARD`, `tf_draw`, `pend_add` kind 4), the **feedback encoder ending `g_blit`**, **layered pipelines** (`vs_layer`, `layer_out`), **function constants specialized** (`shader_load`), the **zero cube for a 2D texture** (`tex_fit`), **layered passes** (`g_rp_layered`, `g_pass.layered`), **stencil-mode views** (`tex_by_id`, `X32_Stencil8`; every depth-stencil texture with pixel-format-view usage, `tex_desc`, PR #106, `t_stencil_view`), **the base instance in the driver segment** (`encode_args`, `c0[0x644]`; `nvn_refl.m` treats any declared driver-segment binding as read), **reduction samplers** (`samp_by_id`) and **pool creation retries** (`mtl_pool_add`): [Transform feedback captured by a vertex entry point](../../../docs/reference/rendering.md#transform-feedback-captured-by-a-vertex-entry-point) and the other entries of [rendering.md](../../../docs/reference/rendering.md) that name this game. +- **Image atomics into a scratch buffer as large as the image** (`atom_scratch`): [Image atomics](../../../docs/reference/shaders.md#image-atomics). ## Open diff --git a/dm/games/0100F5D01DA3A000/README.md b/dm/games/0100F5D01DA3A000/README.md index 62e4649..db9ecea 100644 --- a/dm/games/0100F5D01DA3A000/README.md +++ b/dm/games/0100F5D01DA3A000/README.md @@ -36,8 +36,8 @@ Base game 1.0.0 (v0), SDK 16.2.4. Unity 2022.3.5f1 IL2CPP with FMOD on NVN. Modu General changes first needed here; each applies to every game: -- **Written-out `Shader` layouts for Unity players with no bundle to borrow a tree from** (`KNOWN_TREES`, `tree_from_layout`, given in the split's second pass): [04, Find every shader container](../../../docs/agents/04-graphics.md#1-find-every-shader-container). -- **The node area cut on an `sdk` build whose first game never needed it** (Unicorn Overlord's): [03, Memory and address space](../../../docs/agents/03-bringup.md#memory-and-address-space). +- **Written-out `Shader` layouts for Unity players with no bundle to borrow a tree from** (`KNOWN_TREES`, `tree_from_layout`, given in the split's second pass): [Unity Shader layouts written out for players without bundles](../../../docs/reference/shaders.md#unity-shader-layouts-written-out-for-players-without-bundles). +- **The node area cut on an `sdk` build whose first game never needed it** (Unicorn Overlord's): [The SDK's address-region node area](../../../docs/reference/memory.md#the-sdks-address-region-node-area). ## Open diff --git a/dm/games/0100FD8022DAA000/README.md b/dm/games/0100FD8022DAA000/README.md index 71a09f6..8d69371 100644 --- a/dm/games/0100FD8022DAA000/README.md +++ b/dm/games/0100FD8022DAA000/README.md @@ -58,11 +58,11 @@ Mac repro of both: `DM_OE_SCRIPT=bg=1@20,bg=0@25`. The game froze after switchin General host changes first needed here; each applies to every game: -- **`precise::` min / max / clamp** in every translator (`dmsh_build.py` `precise`, `vk_shaders.py`, `gl_shaders.py`; issue 72): [04, Other per-title surprises seen in Mario](../../../docs/agents/04-graphics.md#6-other-per-title-surprises-seen-in-mario). -- **Vulkan descriptors hold the Metal objects** (issue 68): [04, Seen in Super Mario Galaxy (Vulkan)](../../../docs/agents/04-graphics.md#seen-in-super-mario-galaxy-vulkan). -- **Vulkan acquire and the vsync event thread follow the display link** (`mtl_link_wait_tick`, `mtl_link_wait_vsync`): [03, Frame rate](../../../docs/agents/03-bringup.md#frame-rate). -- **Focus message 15 only under focus handling modes 1 and 2** (`hle_oe.c`) and **the guest clock waits in the background** (`guest_clock_read_wait`): [03, Services and SDK](../../../docs/agents/03-bringup.md#services-and-sdk). -- **A yield whose callee-saved registers moved counts as progress** (`thr_yield`; this game's poll keeps them): [03, Threads](../../../docs/agents/03-bringup.md#threads). +- **`precise::` min / max / clamp** in every translator (`dmsh_build.py` `precise`, `vk_shaders.py`, `gl_shaders.py`; issue 72): [min, max and clamp with a NaN operand](../../../docs/reference/shaders.md#min-max-and-clamp-with-a-nan-operand). +- **Vulkan descriptors hold the Metal objects** (issue 68): [Descriptors hold the Metal objects they were written with](../../../docs/reference/rendering.md#descriptors-hold-the-metal-objects-they-were-written-with). +- **Vulkan acquire and the vsync event thread follow the display link** (`mtl_link_wait_tick`, `mtl_link_wait_vsync`): [Vulkan acquire and vsync event follow the display link](../../../docs/reference/frame-rate.md#vulkan-acquire-and-vsync-event-follow-the-display-link). +- **Focus message 15 only under focus handling modes 1 and 2** (`hle_oe.c`) and **the guest clock waits in the background** (`guest_clock_read_wait`): [Focus handling mode decides whether focus changes are notified](../../../docs/reference/services.md#focus-handling-mode-decides-whether-focus-changes-are-notified), [Guest clock reads wait while the app is in the background](../../../docs/reference/services.md#guest-clock-reads-wait-while-the-app-is-in-the-background). +- **A yield whose callee-saved registers moved counts as progress** (`thr_yield`; this game's poll keeps them): [Yields that make progress return at once](../../../docs/reference/threads.md#yields-that-make-progress-return-at-once-thr_yield). ## Open diff --git a/dm/games/profile_guess.py b/dm/games/profile_guess.py index 3ba2829..e09b35a 100644 --- a/dm/games/profile_guess.py +++ b/dm/games/profile_guess.py @@ -159,7 +159,7 @@ def guess(F, hooks=None): if F.get('main_markers', set()) & {'Unreal', 'Unreal 4'} and R['exts'].get('.pak') and 'nvnBootstrapLoader' in imp: G['shader_groups'] = ('ShaderArchive-Global-=global ShaderArchive-=game', 'the decoded shader code archives: the engine\'s global shaders and the game\'s material shaders') G['mem_reserve'] = ('0x1028000000', 'Unreal: nninitStartup reserves a 62 GB address region and its FMallocBinned3 takes 37 GB of it, so a 64 GB alias region for the bring-up on the Mac. ' - 'An iPhone needs the allocator cut for the build and a capped region (game_args), as Octopath Traveler 0\'s profile (docs/agents/03)') + 'An iPhone needs the allocator cut for the build and a capped region (game_args), as Octopath Traveler 0\'s profile (docs/reference/memory.md)') G['mem_alias'] = ('0x1008000000', '') G.update(preset) return G diff --git a/dm/gfx/nvn/nvn_mtl.m b/dm/gfx/nvn/nvn_mtl.m index b2777c7..8ec2fbe 100644 --- a/dm/gfx/nvn/nvn_mtl.m +++ b/dm/gfx/nvn/nvn_mtl.m @@ -3575,7 +3575,7 @@ static void tex_mis_note(NvShader *sh, int stage, int slot, int tid, id replaces the list. Caller holds g_rmu static char g_afr_keys[48][20]; static int g_afr_nk = -1; static int afr_want(NvShader *sh) { diff --git a/dm/host/hle_audio.c b/dm/host/hle_audio.c index e77ddc8..f6a13f3 100644 --- a/dm/host/hle_audio.c +++ b/dm/host/hle_audio.c @@ -148,7 +148,7 @@ static OSStatus au_render(void *ctx, AudioUnitRenderActionFlags *f, const AudioT aout *a = ctx; atomic_fetch_add_explicit(&g_render_calls, 1, memory_order_relaxed); atomic_fetch_add_explicit(&a->cb_n, 1, memory_order_relaxed); { - // Every 2000 callbacks of each output, log one line: frames requested per call, pending blocks at callback start, underruns (check this line on a new device; docs/agents/03-bringup.md "Crackling on iOS only"). + // Every 2000 callbacks of each output, log one line: frames requested per call, pending blocks at callback start, underruns (check this line on a new device; docs/reference/audio.md "RemoteIO's default I/O buffer duration"). // Kept per output: with two outputs open, combined numbers mislead (one fully fed and one fed by nobody look like "every other callback underruns") uint32_t qd = atomic_load(&a->q_tail) - atomic_load(&a->q_head); if (!a->st_calls || frames < a->st_fmin) a->st_fmin = frames; if (frames > a->st_fmax) a->st_fmax = frames; a->st_qsum += qd; diff --git a/docs/agents/02-survey.md b/docs/agents/02-survey.md index 647e2ce..ff837ff 100644 --- a/docs/agents/02-survey.md +++ b/docs/agents/02-survey.md @@ -57,7 +57,7 @@ The tables below are what to check by hand afterwards. | What | Where to look | Mario | |---|---|---| | Shaders | Every container that embeds NVN shader binaries, not just the obvious archive. Scan all of RomFS and `main` (see [04](04-graphics.md)) | `Shader/*.bfsha` (BNSH), `Lib/agl/*.sarc` + `Shader/Secred.sarc` (agl BAHS `sharcb`), `Effect/*.esetb` (BNSH), `Layout/*.blarc` (BNSH), and two BNSH inside `main` | -| Heap budget | A heap-definition file, if the engine has one | `System/HeapSize/*.HeapDef.byml`: per-subsystem heaps in KB, and a 1.75 GB `VirtualAddressMemory` mapped on demand into the alias region (drives the address-space partition, [03](03-bringup.md)) | +| Heap budget | A heap-definition file, if the engine has one | `System/HeapSize/*.HeapDef.byml`: per-subsystem heaps in KB, and a 1.75 GB `VirtualAddressMemory` mapped on demand into the alias region (drives the address-space partition, [memory.md](../reference/memory.md#partitioning-the-guest-block-between-alias-region-stacks-and-heap)) | | Resource sizes | Resource size table | `System/Resource/ResourceSizeTable.*.rsizetable` (RESTBL, CRC32 of the path → decompressed size) | | Fonts | Font archives and use of the system shared font | `Font/*.bfarc`; the shared font goes through `nn::fontll::ScalableFontEngineHelper::Decode` (BFTTF) | | Languages | Per-language archives | `Mals/.sarc`, `Voice/Resource/` | diff --git a/docs/agents/03-bringup.md b/docs/agents/03-bringup.md index cca41b6..1988b29 100644 --- a/docs/agents/03-bringup.md +++ b/docs/agents/03-bringup.md @@ -27,320 +27,6 @@ Useful switches: `DM_FS=1` (file opens, and not-found probes), `DM_SVC=1` (memor ## Failure catalogue -Each entry: symptom → root cause → fix (where the fix lives). Check these first; most new-title failures are one of them in a new form. +The failures met so far are filed by subsystem, one entry per mechanism (symptom, cause, fix, the games it was found on): [rewriter](../reference/rewriter.md), [memory and address space](../reference/memory.md), [threads](../reference/threads.md), [services and SDK calls](../reference/services.md), [movies](../reference/movies.md), [audio](../reference/audio.md), [frame rate](../reference/frame-rate.md), [NVN objects](../reference/nvn-objects.md) and [game state](../reference/game-state.md); graphics in [rendering](../reference/rendering.md) and [shaders](../reference/shaders.md). Most new-title failures are one of them in a new form, so search the [symptom index](symptoms.md) for the assertion, `[RED]` text, abort site or visible result first, then read the entry and the game notes it names. -### Rewriter - -- **`jump table ` assertion in `patch_nso.py`** (Unity / IL2CPP `main`) → the compiler used `x18` for other jump-table shapes: an offset table (`ldrsw x18, [xB, x18, lsl #2]; add x18, x18, xB; br x18`), an indirect call (`blr x18`) and an `alloca` (`mov sp, x18`). The rewriter reads the offset table's targets at rewrite time (`site_br_tab`), calls through `x16` with the original return address so C++ unwinding still finds its frames (`site_blr`), and restores its scratch registers relative to the old `sp` (`site_sp`). Engage `main`: 7 jump tables, 1 call, 1 `sp` write; Mario's output stays byte-identical. -- **`jump table ` assertion in a `subsdk`** (Link's Awakening's GLSLC and movie modules) → the offset-table form `ldrsw x18, [xB, xI, lsl #2]; add x18, x18, xB; br x18` with the bounds check on another register: the index in `w0` (`sub w0, w18, #0xd1; cmp w0, #0x31`), a copy (`cmp w21, #7; …; mov w18, w21`) or a masked copy (`and w16, w18, #0x3f; cmp w16, #0xe; …; and x18, x18, #0x3f`). `jump_offsets` follows `mov` copies back to the `cmp` and accepts a check on the same `and` of the same, unchanged source. Engage's output stayed byte-identical. The entry may also be loaded into another register, with only the `add` writing `x18` (Cuphead, Unity 2017.4: `cmp w17, #7; ldrsw x17, [x15, x17, lsl #2]; add x18, x17, x15`). -- **`jump table ` assertion on a `br x18` with no table in front of it** (Ace Attorney Investigations Collection, Unity 2022.3 IL2CPP: `main+0x1d271bc` and `+0x204ebec`) → not a jump table but an indirect tail call: an IL2CPP thunk loads the method pointer with `ldr x18, [x0, #0x18]`, moves its stack arguments with every other scratch register (`ldr x16, [sp]`, `ldp x17, x15, [sp, #8]`, …) and ends with `br x18`. When neither table form is found, the last write of `x18` is a plain load and the `br` ends the function, it becomes `x16 = guest x18; br x16` (`tail_call` / `site_br_call`; at a call `x16` / `x17` are free, as for `blr x18`). It is tried only where the rewriter used to stop, so every module it accepted before comes out byte-identical. -- **Job system hangs; a queue claims thousands of tasks but every slot is empty** → in an `ldxr … cmp … b.ne out … stxr … cbnz` window, the rewritten stub emitted its exits in the wrong order, so successful stores took the failure path → emit the fall-through exit first (`patch_nso.py`). D2R had no such windows; Mario had 82. -- **`branch into window ` assertion** (The Legend of Zelda: Tears of the Kingdom 1.4.3 `main`, 8 sites) → the compiler moved an atomic update's retry path out of line: the failed `stxr` branches to a block `ldxr; cmp; b.eq ` that re-enters the window at its store. That block cannot jump into the window's patch copy (the `x18` stand-in would be loaded between its exclusive load and the store), so it gets a copy of its own: the retry block and the window's tail from the store on, under one prologue (`retry_block` / `site_retry_window`). A window also never spans a second exclusive load: before this rule, a retry block's `ldxr` opened a window that ran on into the next, unrelated `ldxr … stxr` loop (`main+0x5a4674`), and branches to that loop's head landed mid-window. -- **`jump table ` assertion, entries relative to an `adr`** (Tears of the Kingdom `main+0x393bb0` and others) → `cmp wI, #N; b.hi; …; adr x18, #L; ldrsw xE, [xT, xI, lsl #2]; add x18, x18, xE; br x18`: target = L + entry, not table + entry (`jump_adr_offsets`, tried only where the table-relative form is not found). The bounds check can be on a register copied into the index (`mov w16, w15`), and the table base can be built far from the dispatch; then every `adrp` + `add` pair in the function that builds that register is tried, and a table is accepted only if exactly one makes every target an instruction of the function. -- **`jump table ` assertion on a byte table indexed by x30** (Hyrule Warriors: Age of Calamity `main+0x138216c`: `adr x18, #L; ldrb w30, [x14, x12]; add x18, x18, x30, lsl #2; br x18`) → capstone names x30 `lr` (and x29 `fp`), so the index register looked like no register and the `ldrb` never matched. The byte / halfword table check maps those names back to 30 / 29. -- **`jump table ` assertion on a byte table whose `add` is far from the `br`** (Octopath Traveler 0 `main+0x37f9bc8`: `adr x18,#L; ldrb w0,[x17,x15]; add x18,x18,x0,lsl #2`, then 17 register moves before `br x18`) → the byte / halfword search looked 16 instructions back. `jump_bytes_far` looks 32 back, requires that nothing writes x18 between the `add` and the `br`, and runs only when every other form has failed, so sites that rewrote before come out the same. One of the 12 `br x18` sites in this `main`; a census of all of them first (each shape within 16 / 32 / 48 instructions) saves a 12-minute rewrite per site. -- **`jump table ` assertion where the index is a copy made before the check** (DRAGON QUEST XI S `main+0x2dd8cd0`: `mov w0,w18`, then 41 instructions holding two other jump tables on `x0`, then `cmp w18,#4; b.hi; ldrsw x18,[x11,x0,lsl #2]; add x18,x18,x11; br x18`) → every other form follows copies made after the bounds check. `check_before_copy` accepts the `cmp` and `b.hi` right before the load when a `mov wI, wS` up to 64 instructions back is the last writer of both registers; capstone lists a `cmp`'s first operand as written, which the search skips. It runs only when every other form has failed, so modules that rewrote before come out the same. Before fixing one site, a census of all of them saves a run per site: a copy of `patch_nso.py` whose assert prints and continues found this one alone among the 25,508 `x18` sites. That `main` has no unwind entries for its first 70 MB, so the rewriter disassembles them as one function and takes 6 minutes. -- **`jump table ` assertion on an offset table whose base was set far back** (ASTRAL CHAIN 1.0.1 `main+0x6db4a0`, SDK 6: `cmp w18,#5; b.hi; ldrsw x18,[x24,x18,lsl #2]; add x18,x18,x24; br x18`) → the function's first block loads six table bases with `adrp` + `add` into registers that its loop keeps (`x24` 441 instructions before this load), and a later block one more (`x3`, 549 before the load at `+0x6db70c`); `jump_offsets` follows the base's writer chain 256 instructions back. Where nothing writes the base in those 256, its last attempt (`far_base`) follows the chain to the function's start (`table_base` `far`), through no call when the base is caller-saved, and accepts the table only when every target is an instruction of the function. The function-wide search that `jump_adr_offsets` uses for a hoisted base (the one `adrp` + `add` pair that fits) found two for `x3`, the other a local table of a `br x17` further on. Three of that `main`'s five `br x18` sites need it; it runs only where every other form has failed, so modules that rewrote before come out the same (Tears of the Kingdom, Age of Calamity, Cuphead, DARK SOULS: REMASTERED, Luigi's Mansion 3 and Metroid Dread `main`s: byte-identical). - - **The far base passed through other registers** (Super Smash Bros. Ultimate 13.0.5 `main+0x2b74458`, SDK 8.2.1: `ldrb w18,[x14]; sub w18,w18,#1; cmp w18,#6; b.hi; ldrsw x18,[x7,x18,lsl #2]; add x18,x18,x7; br x18`) → `x7` is built by `adrp` + `add` 688 instructions before, then parked in `x30` around a loop (`mov x30,x7` … `mov x7,x30`), so the far chain meets a `mov` and stops; the function-wide search finds a second `adrp x7` + `add` pair (another table further on) whose seven targets also lie in the function. The last attempt now follows register copies back to the `adrp` + `add` (`copied_base`, through no call while the register followed is caller-saved; the same target check). It was that `main`'s only unhandled site; it runs only where the rewriter asserted before, and Mario Kart 8 Deluxe's three modules and Smash's `sdk` and movie module come out byte-identical. -- **`B()` assertion at a spin wait: a branch into the patch area beyond the ±128 MB a `b` reaches** (Animal Crossing: New Horizons 3.0.3 `main`, 63 MB of text) → each halfword jump table (`ldrh wE, [xT, xI, lsl #1]; …; adr x18, #L; add x18, x18, xE, lsl #2; br x18`) got a landing pad for every one of the 65,536 values an entry can hold (768 KB a table), whatever the switch's size: that `main` has 51 of them, 40 MB, and the spin-wait stubs placed after them were out of reach. `half_count` reads the table as `jump_adr_offsets` does (the index's bounds check, following register copies, then the base from `adrp` + `add`, or the one candidate in the function whose every target is an instruction of it) and pads only up to its largest entry; a table it cannot read keeps all 65,536 (7 of the 51). The patch area went from 42 MB to 7.9 MB. In the other titles' modules only such tables changed: one in the `sdk` of several SDK 11–20 builds (936 to 1,320 pads), 1 to 6 in some `main`s (Tears of the Kingdom, Breath of the Wild, Silksong, Monster Hunter Rise, Absolum, STEINS;GATE RE:BOOT) and one in some movie and middleware `subsdk`s. -- **`observation gate neither at a function entry nor after a bl`** (DRAGON QUEST VII Reimagined: the gate on `mov x0, x19` at `main+0x32758e4`, after a cut `bl` and an `adrp`) → the host may change x9–x17 in a gate, so a gate sat only where they carry nothing. A site is also accepted when the straight-line code from it to the next `bl` reads none of x9–x17 before writing it (`scratch_dead`); it is tried only after the other two rules fail. The gate table takes the original instruction as a word value (`aa1303e0`), the cut table as bytes (`e00313aa`). -- **`jump table ` assertion on an ordinary byte table near the start of the text** (FINAL FANTASY RESONANCE 1.0.1 `main`, Unreal Engine 5.6.1: `main+0x219234`, `adr x18; ldrb w0,[x17,x16]; add x18,x18,x0,lsl #2; br x18`) → the module has no unwind entry before `main+0x6a75d4`, and `func_of` made every site before the first FDE a function of one instruction, so no form saw the `adr` in front of the `br`. The text before the first FDE is now one function, from the end of the data block before it (or the text's start), as a gap between two FDEs already was. Of the 120 modules of 33 games on hand, only DRAGON QUEST VII Reimagined's `main` has other `x18` sites there (110 before `+0xb310`, none in a window or a `br x18`), and its rewrite is byte-identical. -- **The rewrite never finishes, the Mac swaps, the rewriter's footprint reaches 100 GB** (the same `main`: 82 MB at `main+0x6c4f88`–`+0x59220c0` and the 6.65 MB before the first FDE have no unwind entries) → a stretch without FDEs is one function to `func_of`, and its detailed disassembly costs about 880 bytes an instruction once built (2.6 KB while capstone builds it): 21.6 million instructions. Octopath Traveler 0 (71 MB), DRAGON QUEST XI S (68 MB) and DRAGON QUEST VII Reimagined (two of 23 MB) have such stretches too; they got through on a quieter Mac with the compressor and swap. A function longer than 256 KB is now analysed in pieces (`pieces` in `patch_nso.py`), cut only right after a `ret` / `br` / `b` where no conditional branch and no `b` within 64 KB crosses, so a window, a jump table's bounds check and table base, and a retry block stay in one piece. After all pieces, every farther `b` / `bl` and every jump table target is checked against the windows and retry blocks of all pieces. Counting `b` up to 1 MB left only 21,385 cut points in the 82 MB (tail calls between neighbours) and pieces of up to 6.5 MB; with 64 KB there are 185,228, and no piece is over 256 KB (the two stretches: 28 and 344 pieces). The rewrite then takes 13 minutes at about 2.7 GB resident on a loaded Mac. DRAGON QUEST VII Reimagined's `main` came out byte-identical to its rewrite by the previous `patch_nso.py`. FINAL FANTASY RESONANCE's came out the same as with `b` counted up to 1 MB (3 and 136 pieces), and DRAGON QUEST XI S's (292 pieces) and Octopath Traveler 0's the same as with 1 MB pieces and `b` within 256 KB (75 pieces for DQ XI S), so the analysis does not depend on where the cuts fall. -- **`AssertionError: 0x…` in `site_window` on an `adrp` inside an exclusive window** (Pikmin 4 1.1.1 `main+0x43da14c`, Unreal Engine 4.26.2: `ldaxr x9,[x9]; cmp x9,x25; b.ne out; adrp x10 + add x10` (the same address again, for the store); `mov x18,x26; stlxr w9,x8,[x10]`, then loads and `cbnz w9` back) → a window that uses x18 is copied into the patch area, and an `adrp` there would build another page. It is re-encoded for the copy's address with the same target page (x18 as its stand-in), in both window copiers (`window_adrp`). Only windows that stopped the rewrite change, so every other module rewrites byte for byte as before. -- **`observation gate neither at a function entry, after a bl nor before a call` on an epilogue** (Pikmin 4: the gate on `ldp x29, x30, [sp, #0x10]` right after the cut allocator constructor call, `main+0x353f78c`; the rule checks the image after the cuts, so the `bl` before it is a `nop` by then) → `ret` ends the code ahead as a `bl` does: under AAPCS64 none of x9–x17 carries a result, so the straight-line scan (`scratch_dead`) accepts a `ret` too. -- **Checking a rewriter change**: rewrite every extracted module of every title with the old and the new `patch_nso.py` and compare all outputs; only `build_src.sha256` (the rewriter's own hash) may differ for titles the change is not meant for. -- **SIGILL / trap on a system register read** → newer SDK reads a register macOS traps (`CTR_EL0` in SDK 16's `FlushDataCache`) → rewrite to a constant. -- **`B()` range assertion in `patch_nso.py` (`site_plain`), only for one module** (Metroid Dread `main`) → its bss is 1.8 GB, the engine's static heap, so a patch area appended after the image starts 1.9 GB from the code and no `B` (±128 MB) reaches it. When an appended patch area would start beyond 112 MB, the rewriter puts it 32 MB *before* the module base (negative `PATCH_OFF`). `patched.bin` then holds only text + ro + data (`FILE_IMAGE_LEN`) followed by the patch bytes, and the loader maps the bss and the key page as zero pages (`IMAGE_SIZE`). The Mac loader, the iOS embedding (`gen_embed.py`: no bss in the executable) and `QueryMemory` handle both layouts; Mario's and Engage's output is byte-identical to before. -- **`B()` range assertion, then `patch area of … bytes does not fit in the … before the base`** (Monster Hunter Rise `main`: 125.3 MB of text) → with the patch area 32 MB before the base, sites near the end of the text are more than 128 MB away. When the text ends within 32 MB of the reach (`BIG_TEXT`), the patch area goes right before the base: the stubs follow the sites in address order, so each site's stub is about as far from it as the text is long. The first pass tries the gap that keeps every site in reach (2.6 MB here); when the patch area is larger (3.6 MB, 34,322 sites), the rewriter runs itself again with exactly that size (`DM_PATCH_FRONT`, set only by the rerun) and `B()` checks every branch. -- **`KeyError` at `site_br` for a `br x18` that is an ordinary byte jump table** (Monster Hunter Rise `main+0x6b346d0`) → capstone stops at the first word it cannot decode, and the function held a trap word (`0xe7ffdefe`, `main+0x6b34540`) before the jump table, so the table was never analysed. Function disassembly now goes on after undecodable words (`disasm_fn`). -- **`RuntimeError: 788 ldr x18, #0xd1eb0` in `patch_nso.py`, or any rewrite site in data** (Absolum `main`, .NET NativeAOT) → the text segment starts with 1.7 MB of data (`main+0x0`–`+0x1b5000`, high-entropy, no FDE covers it, and no `adrp` or relocation points into it): 62% of its words do not decode, and 6,858 of the rest read as instructions that use `x18`, plus three "`svc`" words with numbers such as `#0x91e7`. The linear sweep rewrote them all. `dm/tools/nso_code.py` finds such data blocks: a stretch of at least 4 KB between the `.eh_frame` FDE ranges in which more than a quarter of the words do not decode. The sweep skips them (`DATA_BLOCKS`, logged as `data blocks left alone`), and so does `insn_census.py`, whose counts then match the rewriter's again (11,744 `x18` sites in Absolum's `main`). Code without unwind information is still swept: it decodes (Absolum's zlib at `main+0x75261c`, libunwind's register restore at `main+0x28dbd4`). No module of the 19 other games on hand has a data block, and their rewrites are byte-identical. - -- **`CRASH sig=10 addr=0` on an `stp` relative to `sp`, with `sp` not a multiple of 16** (Paper Mario: The Thousand-Year Door: `main+0x2bbba4`, `sub sp, sp, #0xc0` then `stp d15, d14, [sp, #0x60]` with sp ending in `…9dc`) → Horizon leaves stack alignment checking off for user code; Apple's kernel faults on an sp-based access while sp is misaligned. The game's script engine calls native functions through six hand-written bridges (`main+0x17c140` and five more beside it): each reserves the callee's stack arguments with `sub sp, sp, x19`, x19 being their sizes added up (4 for one int, `[x0 + 0x238]`), and gives them back with `add sp, sp, x19` after the call, so the callee ran with sp 4 bytes off. The rewriter now sends every `sub sp, sp, xN` (extended-register form, no shift) through a stub that rounds xN up to 16 first (`site_sp_align`); the `add` then gives back the same amount. Compiled code keeps such sizes 16-aligned, so there it changes nothing; of the 22 games' modules on hand only this `main` has the form (6 sites), so every other rewrite is byte-identical. `insn_census.py` counts the sites as `sub sp, sp, xN`. -- **Every pointer in `.data` / the GOT is a module offset, crashes everywhere** (SDK 20: The Legend of Zelda: Breath of the Wild 1.9.0) → the modules pack their relative relocations as `DT_RELR` (tag 36, size tag 35): `main` has 223,994 of them and its `DT_RELA` only symbol relocations. Each 8-byte entry is either an even offset (relocate it, move to the next slot) or an odd bitmap whose bits 1–63 stand for the following 63 slots; the addend is the slot's own value (`relr_apply` in `loader.c`; `DM_LOADHASH` and `nvn_slots_got.py` read it too). The loader's log line counts them as `relr=`. - -### Memory and address space - -- **`StandardAllocator::Initialize` aborts in `nninitStartup`** → the NPDM declares a system resource (offset 0x14 of `META`), so `IsVirtualAddressMemoryEnabled` is true and the allocator reserves all free memory (about 3.2 GB for Engage) as one region in the alias region, after the ~1 GB node area → the profile's `mem_alias` / `mem_reserve` must hold it (Engage: 5.5 / 8 GB on the Mac). -- **C++ exception aborts in `nn::ro::detail::GetExceptionInfo`** → the SDK's unwinder finds a module from `svcQueryMemory`: it walks back over Code+RX regions to the module start, forward over Code then CodeData, and reads `MOD0` for `.eh_frame_hdr`. The host must report each loaded module's real layout and report other host memory as free (`hle_svc.c`). IL2CPP throws a C++ exception for every managed exception, so Unity games need this; Mario never threw. -- **First `AllocateAddressRegion` fails with 0x1803** → SDK 16's AddressRegionManager takes 0x3fe00000 at the start of the alias region as a node area → the alias region must be larger than 1 GB. -- **The game calls `exit(1)` in `nninitStartup`** (Octopath Traveler 0, Unreal Engine: `main+0x12e74`) → Unreal's `nninitStartup` needs virtual address memory and reserves one address region for the engine with `AllocateAddressRegion`, 62 GB by default (`-SwitchVirtualAddressSpaceSize=` in MB, or in bytes above 4095; the other options it reads are `-SwitchMaxMemorySize=`, `-NVNCommandBufferGranularity=`, `-NVNControlBufferSize=`, `-SwitchMainThreadStackSize=`, `-NVNTempMemoryPageSize=`, `-ResX=` / `-ResY=` and `-SwitchDisableVirtualAllocator`, which also exits). It reads them from the game arguments, which the host used to set only after `nninitStartup`; they are set before it now, as on hardware, and the profile key `game_args` supplies them on iOS. Its allocator, an FMallocBinned3, then reserves 37 GB of that region (74 pools of 512 MB, pool index = offset >> 29; the null read in the out-of-memory report at `main+0x1005824` is the symptom when the region is smaller). The pools are uneven (at character creation the largest holds 68 MB, the 74 together 732 MB; measured with `DM_ALIAS_WINDOWS=512`), so no smaller window fits a phone's ~5 GB. The constructor call is cut (its `patches.py`) and the host puts its own allocator in the object (`hle_uemalloc.c`: the FMalloc vtable on a host malloc zone, the trivial slots kept). With it the region holds only what the engine maps itself (709 MB at character creation) and is capped at 2 GB; SDK 19's AddressRegionManager node area is cut to 62 MB as Engage's (`sdk+0x1423fc` / `+0x142468`), so the guest block is 3.75 GB. - - **Unreal Engine 4.27** (DRAGON QUEST VII Reimagined, SDK 19.3.1): the same 62 GB region from `nninitStartup` (`main+0xe8a2c0`), and an FMallocBinned3 whose 37 GB of pools are cut out of that region (the mapped high-water mark reached +38 GB of a 64 GB alias region at boot), so a capped region would starve it. Here the allocator is not in the bss: `GCreateMalloc` (`main+0x32758b0`) mallocs 0x1c38 bytes on the first allocation, during the static constructors, and constructs it there. Found from the commit calls (probe `nn::os::AllocateMemoryPages` past its `adrp`, `sdk+0x140f84`: every backtrace runs through the allocator's methods around `main+0x3270000`), the relative relocations that point at them (the vtable `main+0x5e4c0e8`; this build has no RTTI) and the one store of that vtable outside the destructors. The constructor call is cut and an observation gate right after it (`OBS_UE_MALLOC`, x19 = the new object) puts the host allocator's vtable in (`hle_uemalloc.c`); no code outside the allocator reads its pool base or calls its methods directly. UE 4.27's FMalloc has one FExec slot, not four, so the slot numbers differ from UE5's (Malloc 3 … GetAllocationSize 9; `Free` is `main+0x10`, right after the 16-byte module header). With `-SwitchVirtualAddressSpaceSize=2048` the region is mapped up to +0.75 GB through the opening and the new-game menus. - - **Unreal Engine 5.6.1** (FINAL FANTASY RESONANCE, SDK 20.5.17): `nninitStartup` (`main+0x13370`) reserves the same 62 GB region and constructs its FMallocBinned3 in place in the bss, as Octopath Traveler 0's (`main+0x77c6210`; the call at `main+0x137f0` is cut, and its constructor would reserve 38 GB). UE 5.5 and later add `MallocZeroed` / `TryMallocZeroed` after `Free`, two TLS cache calls (`MarkTLSCachesAsUsed` / `Unused`) and two cached-memory queries, so the FMalloc vtable has 32 slots (layout `UE5_6` in `hle_uemalloc.c`: Malloc 6 … Free 10, QuantizeSize 13, GetAllocationSize 14). The base vtable shows it: its three pure slots are 0, and its slots 11 and 12 are one function that calls slot 6, then `memset`. The `sdk` is STEINS;GATE RE:BOOT's build; its node area takes the same cut (`sdk+0x145cbc` / `+0x145d28`). DRAGON QUEST VII Reimagined's 2 GB region aborted in the first battle (`nn::diag` abort from `main+0x11bc`, no message): the engine hands its region out with a page allocator of its own (`main+0x13ef0`), a bump pointer plus free lists per size class, so the address space it has used only grows, and past the region's end it calls `OnOutOfMemory` (`main+0x13c74`), with about 1 GB mapped. Probed where each new block comes back (`DM_PROBE=main:13fcc`: x0 the block, x21 its size, x8 the region's end): a 1 GB block at boot from a static constructor, +1.5 GB 15 s in, +2.0 GB in the battle, level at +2.03 GB over seven minutes. This SDK build aligns regions of 2 MB and more to 2 MB only (`VammManager::AllocateAddressRegion`, `sdk+0x145e00`), so it needs no alignment cut, and the profile gives the region 4064 MB in Engage's 5 GB block, as DRAGON QUEST XI S's. - - **Unreal Engine 4.26.2** (Pikmin 4 1.1.1, SDK 10.4.8): the allocator is made as in 4.27, with the same 21-slot FMalloc vtable (`GCreateMalloc` `main+0x353f770` mallocs 0x1c38 bytes and calls the constructor `main+0x35ba610`, the one store of the vtable `main+0x8da4758` outside its two destructors; found from the FMallocBinned3 error strings through the relocated slots). The cut and the gate are as DRAGON QUEST VII's, the gate on the `ldp x29, x30` after the cut call (Rewriter above). The region option is the trap: this `nninitStartup` reserves the region only when `-SwitchVirtualAddressSpaceSize=` is absent (`ldr x8,[x22,#0x130]; cbnz x8` past the default 62 GB and its `AllocateAddressRegion`, `main+0x353f564`), so with the argument the base stayed 0 and the first commit, `AllocateMemoryPages(0, 2 MB)` while the RHI starts (`main+0x353ff8c`, result 0xe03), went into Unreal's fatal error (`nn::diag::Abort` from `main+0x352fbec`). The default is cut to 4064 MB instead (two `orr #0xf80000000` → `mov #0xfe000000`, its `patches.py`) and the profile has no `game_args`. The SDK 10.4.8 build's AddressRegionManager is the 10.4.1 build's at the same offsets and takes the same three cuts. With Engage's 5 GB block, from the boot into the Olimar prologue: up to +1.5 GB of the alias region with 1.0 GB mapped, an Unreal heap of 1 GB. - - **Unreal Engine 4.18** (DRAGON QUEST XI S, SDK 7.3.2) reads the same option and the others from `argv[1]` on (`main+0x177b5a4`; `main+0x177c440` parses each one: `atoi`, MB up to 4095, bytes above). Its region (16 GB by default) needs no allocator cut: the SDK's `NumberLineAllocator` hands it out in 64 KB units (`main+0x177c4d0`), and the engine maps only what it uses. A value of 4224 was taken as bytes: a 4 KB region, and the first 2 MB request crashed in `NumberLineAllocator::Compaction` at boot. With the node area cut and 32 MB alignment for large regions on that SDK build (`sdk_patches.py`), a 4064 MB region starts 64 MB into Engage's 4.375 GB alias region. On the Mac the mapped total stays at 1.5 GB from the title to the first field (Cobblestone Tor), used up to +1.75 GB of the alias region; the process holds about 2.1 GB, 1.1 GB of it Metal allocations. -- **A null-based write minutes into play, at the pool size minus 0x20 (Engage: `addr=0x1fffe0` at `main+0x9a4664`)** → Unity's TLSF heap added a pool, and its low-level virtual allocator (vtable at `main+0x61ffeb0`; reserve `main+0x9aa0f4` → `nn::os::AllocateAddressRegion`) got no address region and returned 0. The alias region ran out of *address space*, not memory: the AddressRegionManager aligns each region by its size (64 KB under 2 MB, 2 MB under 4 MB, 4 MB under 32 MB, 32 MB under 1 GB, else 1 GB) and Unity asks for its block plus alignment slack, so a 2.25 MB block takes a 4 MB slot and a 16.25 MB block a 20 MB one. At the crash Engage held 4.17 of the 4.375 GB left after the node area (StandardAllocator 3.14 GB, Unity about 1.05 GB, the rest fragments). On hardware the alias region is 64 GB. → shrink the node area with a rewriter cut for that SDK build (`dm/tools/sdk_patches.py`, Engage: the two `mov w8/w9, #0x3fe00000` in the constructor → 62 MB). To measure, probe the SDK's `VammManager::AllocateAddressRegion` at its success path (`ReleaseWriteLock` return: x21 = size, x22 = address) and `FreeAddressRegion` after its prologue (x1 = address), and sum the live regions by caller. -- **Sizing the reservation for a title that uses virtual address memory (Engage)** → such a title may never call `SetHeapSize`: everything comes from `MapPhysicalMemory` in the alias region. The `[mem]` log lines report the alias high-water mark (one line per 256 MB), the total actually mapped (page bitmap) and every heap growth. What fills the alias region first is address space, not memory: StandardAllocator reserves the whole free pool (pool − 64 MB, 1 GB aligned) but Engage commits only 0.58 GB of it, and Unity's own regions commit 0.97 GB. So Engage's profile reports a 2.5 GB pool (`app_pool`) and uses a 4.375 GB alias region in a 5 GB block (reserved address space peaks at 3.47 GB of 4.31 GB). iOS without the extended-virtual-addressing entitlement allows a single 6 GB reservation, but Metal needs address space too; 5 GB is the total proven on a phone. To measure commits, probe `VammManager::AllocateMemoryPages` / `FreeMemoryPages` (x1 address, x2 size) and split them by region. -- **`StandardAllocator::Initialize` aborts in `nninitStartup` on SDK 17, then the alias region fills in minutes** (Ace Attorney Investigations Collection, Unity 2022.3, SDK 17.5.3) → the same node area as Engage's SDK: `CentralHeap::Init(nullptr, pool − used)` got no 1 GB-aligned region for 3.14 GB after the 1 GB node area. The constructor is at `sdk+0x139458` on that build and takes the same cut. With Engage's 2.5 GB pool the game then ran, but probes on `VammManager::AllocateAddressRegion`'s success path (`sdk+0x1395a8`: x21 size, x22 address) and `FreeAddressRegion` (`sdk+0x139df0`) showed StandardAllocator's 2.44 GB reservation beside Unity's pools (1.06 GB live, from `main+0x40f1d4`) ending 4.37 GB into the 4.375 GB alias region after 140 s, while the `MapPhysicalMemory` log showed StandardAllocator committing 0.19 GB. The profile reports a 1 GB pool, a little less than the 1.02 GB the game maps at its peak (the host does not enforce the pool, and only `nninitStartup` reads it); ten minutes then peak at +2.85 GB of address space, against +3.91 GB with a 1.5 GB pool. -- **The same abort on SDK 16.2.4** (Brain Memory Training Academy, Unity 2022.3.5f1: `nn::diag` abort at `sdk+0x126490`, `StandardAllocator::Initialize`, called from `nninitStartup` at `main+0x73adcc`; the NSP script's heap check reports "the host stopped before nnMain") → `nninitStartup` is Ace Attorney's, `Initialize(nullptr, pool − used)` with virtual address memory and no `SetMemoryHeapSize`. The `sdk` build is Unicorn Overlord's, whose NPDM declares no system resource, so it never needed the cut; the constructor (`AddressRegionManagerC2`, `sdk+0x134768`) takes it at `+0x134778` / `+0x13480c`. With Ace Attorney's 1 GB pool and 5 GB block the game boots to its title in 5 s with no red line; StandardAllocator ends at +1.0 GB, and five minutes of level loads take the alias high-water mark to +2.25 GB with at most 0.5 GB mapped. -- **The same abort on SDK 13.3.7** (Patrick's Parabox, Unity 2020.3.30f1: `StandardAllocator::Initialize` at `sdk+0x324dec`, called from `nninitStartup` at `main+0x1cd080`) → the same `nninitStartup` and the same cut, on this build's constructor (`sdk+0x37c3a0`: `+0x37c3b0` / `+0x37c44c`). With Ace Attorney's 1 GB pool and 5 GB block (in place of the provisional profile's hardware pool) it boots to its title with no red line, and the alias region is mapped up to +1.5 GB with 0.27 GB in use. To find the constructor on a new `sdk` build, search its text for the two `mov w8 / w9, #0x3fe00000` words (`08fca752`, `09fca752`); `sdis.py --fn` names the function `AddressRegionManagerC2Emm` from its dynamic symbols. -- **The same abort on SDK 10.4.1** (Advance Wars 1+2: Re-Boot Camp, Unity 2020.2.1f3: `StandardAllocator::Initialize` at `sdk+0x2edf90`, called from `nninitStartup` at `main+0x365310`; the heap check reports "the host stopped before nnMain") → the same `nninitStartup`. The `sdk` build is The Great Ace Attorney Chronicles', which has no virtual address memory; its constructor (`sdk+0x340e90`) loads the node area with `orr w8 / w9, wzr, #0x3fe00000` (`e8230b32` at `+0x340ea0`, `e9230b32` at `+0x340f3c`), the SDK 7 / 8 form, so the `mov` search above finds nothing there. `AllocateAddressRegion` (`sdk+0x341030`) aligns a region of 1 GB or more to 1 GB (`+0x34109c`), as DRAGON QUEST XI S's SDK 7.3.2 build, and takes the same 32 MB cut. - - **Then the frames stop at the first loading screen, a graphics worker at 100% CPU** (17 s in, `UnityGfxDeviceWorker` running, the main thread waiting on a condition variable, no red line) → the heap check's 1 GB pool (the hardware pool had used the alias region up to +4.25 GB) left the game's malloc too little: Unity's GPU memory pool block allocation (`main+0x96f430`) calls `aligned_alloc` through `main+0x866810`, halves the request after each failure, and once the request rounds to 0 it calls on for ever with a size of 0. `sample ` on the Mac host shows such a spin (the guest frames are `???` in the host's image list; subtract the module base from the log's `base` lines); it worked here where lldb does not. With a 2 GB pool the game maps up to 2.26 GB of the alias region through its menus and first battle. -- **Unity 6 takes its memory with no StandardAllocator and no heap** (Hollow Knight: Silksong, Unity 6000.0.50f1 on SDK 20.5.8) → `nninitStartup` (`main+0x118cb70`) only calls `QueryMemoryInfo` and `SetMemoryAllocatorForThreadLocal`; Unity's own allocator reserves address regions with `nn::os::AllocateAddressRegion` and commits pages with `AllocateMemoryPages`, so nothing aborts at startup as with 2022.3. `main` imports no `SetMemoryHeapSize`, and the provisional profile took that for a game living in its bss (a 1 GB block, 256 MB alias region) while Unity maps 1.5 GB in the first room; `profile_guess.py` now gives every game with an NPDM system resource the 5 GB block (Absolum and Pokémon Legends: Z-A, heapless too, came out right with it). The node area still sits at the start of the alias region: without the cut the high-water mark was +1.00 GB at `nninitStartup` and +3.00 GB in the first room, with it (`sdk+0x145bfc` / `+0x145c68`) +2.26 GB. BALL x PIT (Unity 6000.0.62f1, SDK 20.5.19) is the same: the provisional profile was right as generated, and its `sdk` build only needed the same cut (`sdk+0x145d0c` / `+0x145d78`, the SDK 20.5.22 build's offsets): +2.00 GB in the base. -- **`nn::diag` abort from `StandardAllocator::Initialize` long after `nninitStartup`, right after the first Vulkan query pool** (Super Mario Galaxy 2, SDK 18.3.5: `sdk+0x12dad0` ← `main+0x62c52c`) → the movie player initializes its own allocator with virtual address memory, `Initialize(nullptr, 384 MB)` (`main+0x62c280`); Galaxy, on the same SDK, allocates that block from its main allocator first. The guessed 256 MB alias region holds no region after the 1 GB node area. The same node area cut for that `sdk` build (`sdk+0x13be4c` / `+0x13bee0`) and a 512 MB alias region: the region is 32 MB aligned, so it starts 64 MB in. -- **`nn::diag` abort at `sdk+0x12dad0` in `nninitStartup`, called from `main+0x756310`** (Absolum, .NET NativeAOT on SDK 18.3.2) → malloc's `StandardAllocator::Initialize(nullptr, 0xa00000000)`: 40 GB of address space (`main+0x28bff4`). To see what such a game really needs, boot it once on the Mac with a 64 GB alias region (`mem_reserve 0x1030000000`, `mem_alias 0x1008000000`) and probe `VammManager::AllocateAddressRegion`'s success path (`sdk+0x13bf8c`: x21 size, x22 address) with `DM_SVC=1`'s map log. Absolum's regions: malloc's (2.19 GB used into the first level, 1.75 GB of it FNA3D's GPU heap, allocated whole at `FNA3D_CreateDevice`), the .NET GC's region range (3.75 GB) and a few small ones. The GC sizes its range as 5 × its hard limit (768 MB = 75% of the 1 GB its `GetPhysicalMemoryLimit`, `main+0x28bffc`, reports), and it reads `DOTNET_GCRegionRange` through `getenv` first. The host now answers `getenv` from the profile key `game_env` (`hle_os.c`; every other name goes to the SDK's own `getenv`, an empty environment), which sets 1.5 GB. The GC then uses its range from both ends, 756 MB from the start and 748 MB from the end, where it used only the first 1.14 GB of 3.75. A cut on `main` gives malloc 2.5 GB. One more thing kept it from fitting: SDK 18's `AllocateAddressRegion` aligns a region of 1 GB or more to 1 GB in absolute address (`sdk+0x13c03c`; 32 MB from 32 MB up). The two large regions, with a 4 KB one between them, lost up to 1 GB each to alignment, depending on where `mmap` put the guest block. A cut on that build aligns them to 32 MB as well, beside the node-area cut. The regions then end 4.13 GB into the 4.375 GB alias region of Engage's 5 GB block. -- **The game aborts in its static initialisers, before `nnMain`, when its own 62 GB address region fails** (Super Smash Bros. Ultimate 13.0.5, SDK 8.2.1: `main+0x37c9bd0`, `AllocateAddressRegion(&p, 0xf80000000)` at `main+0x37c9a5c`, the allocator behind `nu::VirtualAllocHook`) → a 64 KB page allocator whose own tables fill the region's last 14 MB, so the size, the header offset, the page count and three region-end bounds all follow from 62 GB: the profile's `patches.py` cuts all 11 instructions to a 4 GB region. Booted once with a 64 GB alias region (`mem_reserve 0x1030000000`, `mem_alias 0x1008000000`) and `DM_SVC=1`'s map log, the game commits from the region's start upward (1.73 GB contiguous at the title) and its page tables from the top; a battle commits up to 2.5 GB (`DM_ALIAS_WINDOWS`). The cut region still did not fit: SDK 8 has the same 1 GB node area and 1 GB alignment of regions of 1 GB and more as the newer SDKs (constructor `sdk+0x336b40`, `orr w9, wzr, #0x3fe00000` at `+0x336b50` / `+0x336bf0`; alignment at `+0x336d5c`), cut in `sdk_patches.py` as DRAGON QUEST XI S's SDK 7.3.2 build; the region then starts 64 MB into the 4.375 GB alias region of the 5 GB block. -- **Large resources silently skipped, later a null deref where the resource is used** → the game reserves "virtual address memory" (Mario: 1.75 GB in `HeapDef.byml`) and maps it into the alias region with `MapPhysicalMemory` on demand; the alias region left after the node area was too small. Also visible as `SetHeapSize` asking for much less than the pool. → repartition the guest block (`hle_svc.c` `mem_init`): Mario uses alias 2.875 GB, stacks 256 MB, heap 1.625 GB (it asks for 1.47 GB), 5 GB in total. Measure before sizing (`DM_SVC=1`: the `SetHeapSize` request and the highest `MapPhysicalMemory` address). On iOS without the extended-virtual-addressing entitlement a single 6 GB reservation succeeds and 7 GB fails, but Metal and the rest of the process need address space too; 5 GB is the total proven on a phone (D2R). -- **`nn::diag` abort at `sdk+0x11ff34` (`StandardAllocator::Initialize`) in `nninitStartup`, SDK 15.3.2** (Paper Mario: The Thousand-Year Door, called from `main+0x31370c`) → the game's malloc is a StandardAllocator over 62 GB of address space (`Initialize(nullptr, 0xf80000000, false)`, `main+0x3136e8`), out of which it cuts 256 MB, 16 MB and 1 MB allocators. Booted once with a 64 GB alias region, the region came 0.69 GB past the 1 GB node area, at the next 1 GB boundary: this `sdk` build (Celeste's, which has no virtual address memory) aligns regions of 1 GB and more to 1 GB, as Absolum's did. Absolum's three cuts on it (the AddressRegionManager constructor `sdk+0x12e0e0`: `+0x12e0f0` / `+0x12e184`; `AllocateAddressRegion`'s alignment at `+0x12e2e0`) and a cut on `main` to 4 GB of address space, as Pokémon Legends: Z-A's, put the region at alias +62 MB to +4.06 GB in Engage's 5 GB block. The allocator maps 1.5 GB at boot and 2.0 GB by the arrival in Rogueport. -- **`MapPhysicalMemory` faults on part of a page** → round to 16 KB host pages. -- **`nnMain` returns right after `nv::InitializeGraphics`, with no error** (Hyrule Warriors: Age of Calamity) → the game's `nninitStartup` sizes its heap as total − used memory (`QueryMemoryInfo`), calls `SetMemoryHeapSize` and hands the block to its own malloc. The host reports 64 MB used beside the heap, but `SetHeapSize` checks a request against the real size of the loaded modules (66.6 MB here), so `SetHeapSize(pool − 64 MB)` failed with ResultLimitReached, `AllocateMemoryBlock` returned 0x1003, malloc was never set up, the 866 MB graphics heap allocation came back null and `nnMain` returned. Probe `DM_SVC=1`'s `svc 0x1` request and the `AllocateMemoryBlock` result. The profile key `mem_used` sets the used-memory figure (72 MB for this game: the image plus stacks). Monster Hunter Rise's modules take 320 MB (`main` alone 314 MB), so its `SetHeapSize(pool − 64 MB − bss)` failed the same way (`[mem] SetHeapSize … + modules … > app pool`); 328 MB gives a 2.88 GB heap. -- **`SetHeapSize` asks for almost the whole application pool and fails at boot** (Metroid Dread: 0xc9400000, larger than the heap region) → the SDK sizes the heap as total − used memory, and the host reported "used" as heap + 64 MB. That 64 MB covers ordinary modules (Mario's two are about 74 MB, and its heap request matches hardware), but Dread's `main` carries a 1.8 GB bss that the Switch counts as used. Used memory now adds the bss of modules in the front-patch layout (`big_bss_bytes` in `hle_svc.c`); Dread then asks for 1.38 GB. -- **Some textures are re-uploaded every frame (several MB per frame)** → the dirty-page tracker registers each memory pool rounded to 16 KB host pages, but NVN pools are only 4 KB aligned, so neighbouring pools share their first / last host page and one address can lie in several registered ranges. Taking the first range that contains an address judged textures in the second pool untracked (always dirty), released only one range on a write fault, and could unregister the neighbour when a pool was destroyed. `nvn_dirty.c` now uses the range that holds the whole span, releases every range covering a faulting page, and removes a range by the pool's exact start (`t_dirty_slots` covers it). -- **A null write as soon as the main thread starts recording a command buffer** (13 Sentinels: a constructor at `main+0x2fa9a0` runs on `this = 0` when the prologue enters its first battle) → the object came from one of the engine's ring heaps (`main+0x2d83ec`), and that heap had base 0. At startup the engine takes one GPU memory pool, sized as exactly the sum of three ring heaps, and cuts the heaps out at 64 KB alignment *relative to the pool's GPU address* (`main+0x30c118`). On the Switch a pool's GPU address is big-page aligned; the host's GPU address is the CPU address, and the storage came from the game's `malloc` rounded only to 4 KB. Up to 60 KB was lost at the front, the third heap (16 MB) did not fit, and the sub-allocator returned 0. The game's `malloc` is `nn::mem::StandardAllocator::Allocate(size)`; on this build the host routes allocations of 64 KB or more to `Allocate(size, 0x10000)` (`hle_os.c`). Probing the heap object at the allocation (`DM_PROBE=main:2bec60@x0+8`) showed the zero base directly. -- **SIGBUS inside a host hook (`snprintf` under `mount_ex` / `h_fs_MountSave`), then the process hangs** (Super Mario 3D World + Bowser's Fury) → the game's save thread has an 8 KB stack (CreateThread stack top `0x…9fe0`, fault `0x21a0` below it). Host hooks run on the calling guest thread's stack, and the file system hooks size their path buffers by PATH_MAX: one file open takes 6–7 KB (`-fstack-usage`: `OpenFile` 1.3 KB → `host_path_ex` 1.9 KB → `path_component` 3.2 KB); on hardware the same call is one IPC. Every hook in `hle_fs.c` now registers through `HLE_FORCE_BIG` (`hle.h`): a compile-time stub loads the implementation into x16 and jumps to `dm_bigstack_enter` (`hle_bigstack.c`), which moves the call to a per-thread 512 KB host stack (with a guard page) when sp lies in the guest address block, passing x0–x8 and d0–d7 through. Only for functions with register arguments only (no variadic or stack arguments). iOS forbids generating code at run time, so the stubs are static. -- **SIGBUS inside Metal (`AGCDeserializedReply`, an AGX compiler block) at an address in the guest stack region** (Breath of the Wild) → a host NVN call did Metal work on a game thread with a small stack: `nvnQueueSubmitCommands` replays the frame, and ending a render pass made the driver compile its fast-clear blit program on the calling thread. NVN functions are not registered through `HLE_FORCE`, so they got their own big-stack entry: every implemented NVN function except the `nvnCommandBuffer*` recording calls goes through `nvn_big_entry` (`nvn_entry.S`), which finds the implementation by slot and hands it to `dm_bigstack_enter`. -- **Write through a null pointer at +0x1c0 during startup** → `GetInfo(UserExceptionContextAddress)` must return a real buffer. -- **Heap allocations fail late in a busy scene** → the reported application pool (3285 MB, `APP_POOL`) and used memory decide the game's heap size; report Switch-accurate numbers. - -### Threads - -- **Per-core job queues never drained** → a `SetThreadCoreMask` that pins a thread to other cores (ideal core −3 = keep) must move the ideal core into the new mask, as Horizon does, so `GetCurrentProcessorNumber` reports a core the thread can run on. -- **The main thread waits forever on an event right after `nnMain` starts, with only three or four threads created** (Monster Train 2, Unity 2022.3.54f1 on SDK 18.3.5: `main+0x3e7288` starts a thread and waits for it to signal `main+0x539cf80`) → Unity's thread start (`main+0x3e1a50`) creates a thread only when `nn::os::GetFreeThreadCount()` is at least 1, and that is `svcGetInfo` type 24 (FreeThreadCount, Horizon 11 and later), which the host answered 0 like every type it did not know: the thread was never asked for (`DM_THR=1` showed no `CreateThread` for it). The kernel answers the thread resource limit less the threads alive (Atmosphère `kern_svc_info.cpp`); `pm` gives an application 96 threads (`pm_spec.cpp`, 256 with a boost). The host now answers 96 less the game's threads that have not exited (`thr_limit_used`, guest threads the host spawns for itself left out). Of the 20 games extracted on this machine, only this `main` imports the function. With it the game creates 40 threads and reaches its title. -- **Garbage-collected titles (Unity / IL2CPP): `SetThreadActivity` (0x32) and `GetThreadContext3` (0x33) must really pause and report registers** → the Boehm collector pauses each thread, reads its registers for roots and scans its stack. As placeholders that only reported success, threads kept running and the context was empty: roots held only in registers were missed and live objects were freed, with their finalizers releasing native resources. In `hle_thread.c` a thread inside an SVC counts as paused at once, with the gate frame as its context (`thr_svc_enter` / `thr_svc_leave` around every gate SVC; it waits at the SVC exit while paused). A thread in guest code gets SIGUSR2, and the handler saves the ucontext and waits there. A thread in host code is retried every 100 µs. The context is Horizon's 0x320-byte `ThreadContext`. -- **`SignalToAddress` type 2 (`SignalAndModifyByWaitingCountIfEqual`) with count ≤ 0 and waiters present** → the new value is `value − 2`, not `value − 1` (mesosphere `KAddressArbiter`; Ryujinx and yuzu agree). The host had −1. Fixing it did not change Engage's demo crash (that was the address-space shortage above), but it is the specified behaviour. -- **Several threads crash at once and the dumps interleave, then the process hangs instead of exiting** → Unity's job threads tend to crash together. `crash.c` lets only the first crashing thread print (`crash_gate`); the others stop in `pause()`, and a 15 s alarm ends the process if the dump itself stalls. -- **Crash indexing a per-core table with a huge offset (index −2)** → threads created with ideal core −2 (*use process default*) were recorded as core −2 and `GetCurrentProcessorNumber` returned −2 → resolve −2 to the process default core (0) and never return a negative core (`hle_thread.c`). Seen in Mario's font engine (`main+0x107dbc0`). -- **A 30 Hz game thread runs at 27 Hz, and whatever it paces comes out slow** (Metroid Dread: cutscenes at 36 ms per frame instead of 33.3; gameplay was fine at 60) → macOS timer slack. `nanosleep` and `mach_wait_until` overshoot by about 10% at any QoS (measured: 33 ms → 36.4 ms, 5 ms → 6.1 ms, 1 ms → 1.26 ms). Dread's sound thread (`main+0xf95b0`) reads a millisecond clock, updates, then sleeps "33 ms minus this round's work", so on the host its period was 36.9 ms. The cutscene loop renders a frame whenever the audio play position, which that thread refreshes, moves (`main+0x119a3d0`; the position is played samples ÷ sample rate of the cutscene's stream), so cutscenes ran at about 27 fps. `SleepThread` now sleeps three quarters of the remaining time per step until within 20 µs (`sleep_precise` in `hle_thread.c`): cutscenes at 33.3 ms, the sound thread at 33.2 ms. Mario's frame-time p95 also went from 19.7 to 18.4 ms. Timed waits (`WaitSynchronization`, condition variables) still use the host's own timeouts. -- **Crashes around movie changes in the game's own player threads, about one run in two** (Metroid Dread: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`) → the game relies on Horizon's per-core preemption. Its main thread (priority 44, core 0) posts "unlink these holders" to a dispatcher thread and leaves messages for a worker thread (both priority 30, only core 0), then frees the object and builds the next one in the same memory without waiting. On hardware the two threads preempt the main thread as soon as it signals them, so their work is done first; on the host all threads run in parallel. Two fixes, both needed (with only the second, 3 of 6 runs still crashed; with both, 12 of 12 were clean): - - an observation gate after each request the game posts (OBS 19 / 20 in its `patches.py`, this build only) waits until the dispatcher's request ring is empty (`movie_ring_wait` in `hle_movie.c`). Gates may now sit right after a `bl` as well as at function entries; - - an approximation of same-core preemption (`svc_thread` / `dm_yield_to_core_peers` in `hle_thread.c`): after a thread SVC, the caller waits (at most 5 ms) while a higher-priority thread pinned to its core is running guest code. "Running" means the Mach thread state is running, the thread is not in a guest wait or timed sleep, and it is not doing host NVN submit / present work (`dm_host_work`). It is a profile key, `core_preempt`, on only for Dread: Mario's loader threads would wait for busy decompression threads, and its first frame moved from 0.6 s to 2.9 s. -- **PhysX worker crashes in a task's `removeReference` with a null task manager, right as the game switches to 30 fps** (Link's Awakening: `pc=main+0x10ccf28`, `lr=main+0x13c236c`; on the iPhone and in 2 of 5 Mac runs, in every Mac run with 12 busy loops loading the CPU) → a destructor that runs a moment too late. The game rebuilds each physics world's completion task in place before every `simulate` (`main+0x15730`, main thread). A worker runs the task (`fetchResults`, sets the world's done flag, notifies, unlocks) and then calls its `release`, which calls the task's own destructor (vtable back to the `PxLightCpuTask` base, `mTm = 0`). At 30 fps the game steps twice per frame, so the unlock wakes a main thread that steps again at once. On the Switch the `release` follows the unlock within nanoseconds; on Apple devices the worker can be descheduled between them, the main thread rebuilds the task, and the late destructor wipes the new one. The next simulation's `NpSceneCompletion::release` then dereferences `mTm`. A per-build cut makes that one `release` a `ret` (its `patches.py`); the destructor frees nothing and the step rewrites every field. How it was found is in [05](05-debugging.md#racy-crashes-load-trace-gates). - -- **The main thread waits forever on a job's done event at boot while every worker sleeps, two of them on one event** (FINAL FANTASY TACTICS - The Ivalice Chronicles: 4 of 6 Mac runs, `main+0x1426c` waiting on `Job::m_doneEvent`) → the engine's job pools wake one worker through the pool's single auto-clear event. A job goes into worker k's own queue (`main+0x154c0`), and if k's state reads asleep the caller signals the event (`main+0x12554`); the kernel wakes the waiter that slept first, which is k only while the workers sleep in index order. A worker woken for another's job runs only its own queue and sleeps again, so with two workers in one pool the signal is lost: at boot when worker 1 went to sleep before worker 0, and later as a job left in worker 0's queue for 12 s. The thread dump's `wait #` (the order condition waiters arrived in, which `SignalProcessWideKey` wakes them in at equal priority) showed the order in every hung run. The game's own wait wrapper (`main+0x12500`, `{event, pending}`) signals the event once more when it takes a nonzero pending word after waking, and nothing set it: a build cut makes the signal set it first (its `patches.py`), so a worker woken in another's place passes the wake on; 6 of 6 runs boot. A host change that made `StartThread` return only once the new thread runs (as a free Switch core takes it within microseconds) brought the hangs to 1 in 6 and was dropped. -- **A guest thread keeps a core at 100% with billions of `SleepThread` calls** (Super Mario Galaxy: Hagi polls with `SleepThread(100 ns)`, `main+0x517c`; 1.4 billion calls in 200 s, and the render thread nearly as busy) → `sleep_precise` returns at once for anything under 20 µs (its nanosleep would overshoot), so such a loop never sleeps. Sleeps under 20 µs now take the yield path and its back-off (plain yields for the first 2 ms of a run, then short sleeps): the two threads went from 199 s and 171 s of CPU per 200 s to 5 s and 19 s per 120 s, at the same frame rate. -- **A loading screen sits for a minute with the CPU nearly idle; one thread yields thousands of times a second, every yield a back-off sleep** (Animal Crossing: New Horizons: a black screen with the plane icon for 77 s after the opening movie, `SaveProc` at about 2,600 yields a second, the power line's top yielder with as many back-off sleeps; also about 40 s of the same at boot) → the thread was working, not waiting: its grid walks over the new island (`main+0x157d020`) call `nn::os::YieldThread` once per cell, and the back-off made each yield a 250 µs sleep. A yield whose caller's x19–x28 changed since the last one is now progress (`thr_yield` in `hle_thread.c`; the SDK reaches the SVC through tail branches, so they are the loop's own): it returns at once with no host call and the run starts again. Polls keep those registers (Three Houses' limiter, Galaxy 2's Hagi sleep poll at `main+0x11ec`); counters move them. The island now shows about 10 s after the movie, as with `DM_YIELD_BACKOFF=0`. Check such a change on Breath of the Wild outdoors: its `PlacementMgr` and `RadarMgr` turned out to yield per item too, and a `sched_yield` on each of their 200,000 yields a second cost a third of a core more than the back-off had; with no host call, CPU energy fell instead (numbers in [development.md](../development.md) "Yield loops"). A temporary log of which of x19–x28 move at each yield, per thread, is what told the two kinds apart. - -### Services and SDK -- **`sm::Initialize` aborts right after `ConnectToNamedPort` (SVC 0x1F) and `SendSyncRequest` (0x21)** (Super Mario Galaxy, SDK 18) → an SDK call the host does not hook opened its service: here `nn::hid::InitializeTouchScreen(int, int)`, the SDK 18 overload of the hooked `InitializeTouchScreen()`. Symbolize the abort backtrace against the SDK's exports to find the caller, and hook that name. -- **Absolum (SDK 18.3.2, FNA on .NET NativeAOT with SDL3), one boot each**: - - **SIGSEGV in `vi::CreateLayer(Layer**, Display*, int, int)`** (`sdk+0x3f9bc4`, from SDL3's Switch video driver at `main+0x239304`) → the overload with the layer's size; the host had only the two-argument one. It returns the static layer (`hle_vi.c`). The survey's import table listed every vi call as unhooked: `nso_syms.py host_hooks` read only `dm/host`, and the vi hooks are in `dm/gfx` (it reads `dm/gfx` and `dm/gfx/nvn` now). - - **AudioIn** (SDL3's audio driver lists the input devices when its audio starts) → the host has no microphone. `ListAudioIns` reports none, both opens fail, and the calls that need an open AudioIn answer without touching the game's object (`hle_audio.c`). - - **Epic Online Services in an NRO** (`rom:/nro/EOSSDK-Switch-Shipping.nro`, loaded with `ro::LoadModule`; `main` imports the `EOS_*` functions) → the host loaded no NRO, so the imports stayed `UNIMPL` stubs and `EOS_Platform_Create` handed back a garbage handle. The game checks it (`[EOS] Failed to create platform`) and plays offline. The host now loads NROs (below), but never an online SDK, whatever the game: an NRO that exports `EOS_Platform_Create` is left out of the pack (`dm/tools/nro_collect.py` `ONLINE_SDKS`, a `refused` row in `nro.tsv`) and `ro::LoadModule` refuses it with one log line (`refused …: an online SDK (Epic Online Services), not loaded`), also by the export when an older `nro.tsv` has it. An NRR that registers only such modules fails `RegisterModuleInfo`, as every registration did before the host loaded NROs, so the game takes the path it took then: Hades never asks for the module (checked on 1.0.38233: `refused an NRR that registers only online SDKs (1 hashes)`, no `LoadModule`, its frames as before). Hades' `main` imports only EOS Auth, Connect and PlayerDataStorage, Epic-account cross-save the host cannot serve; the real SDK would at best fail its login and at worst wait on time-outs or contact Epic's servers, with an unknown risk to the player's account. So both games behave as before: offline, local saves. Open question for the maintainer: always load, refuse online SDKs (as now), or decide per profile. - - **The modules**: `main` imports EGL (SDL3's GL support) and `vkGetInstanceProcAddr` (its Vulkan support), but FNA3D renders with NVN (`FNA3D_Driver_NVN.cpp`). NVIDIA's OpenGL and Vulkan drivers (`subsdk1`, `subsdk0`) program the GPU through NvRm, which the host does not have, so they are never loaded (`profile_guess.py` `GPU_DRIVERS`); the EGL imports stay unbound stubs that nothing calls. - -- **A library's `free` rejects a pointer (`CentralHeap::Free` → `NotNMallocPtrError`)** → the game's `main` defines its own `operator new` / `delete` (Unity's memory manager) and `sdk` has a copy forwarding to `malloc` / `free`. Symbols are resolved globally in load order (the first module's definition wins, also for a module's references to its own exports); only local / non-default-visibility symbols bind to the module itself (`loader.c`). The loader logs how many references bind to an earlier module. -- **`[RED] fs ReadFile into … failed at +0 (EFAULT): VM region … protection 1`, now and then** (Monster Train 2 at 60 fps: two runs in three as the route map loads, never at 30 fps) → the destination was texture pool memory, which the dirty-page tracker write-protects after each upload (`nvn_dirty.c`). `ReadFile` lifts the protection first (`dm_guest_write_prepare`), but the graphics layer can protect the pages again before `pread` writes them, and a system call gets EFAULT where a plain write would fault into the tracker's handler; at 60 fps uploads come twice as often. On EFAULT the host now prepares the rest of the buffer again (unprotect, mark dirty, so the texture is uploaded again) and reads on, up to eight times (`h_fs_ReadFile`; the first case is logged, `[fs] ReadFile … prepared again`). Four runs at 60 fps: one retry, no failed read. -- **IL2CPP "Could not open …/global-metadata.dat" (then a crash in its logger)** → IL2CPP checks `nn::fs::IsMounted("rom")` before opening; the SDK's own `IsMounted` reads a mount table the host never fills → answer from the host's mount table (`hle_fs.c`). -- **Crash in `nn::ro::InitializeWithPortName`** → `nn::ro` connects to `ldr:ro`; take over `Initialize` and answer `LookupSymbol` from the loaded modules (`hle_ro.c`). Check the mangled names against the new title's imports: SDK 19's `LoadModule` takes one more bool (`_ZN2nn2ro10LoadModuleEPNS0_6ModuleEPKvPvmib`, Hades II), so the hook for the older form did not catch it and the SDK's own code would have run. -- **A jump to pc=0 right after the NVN bootstrap loader** (Super Smash Bros. Ultimate 13.0.5: `lr=main+0x37fe930`, `[RED] NVN address lookup of an unknown name: nvnSetExternalDebuggerEnabledNVX`) → the game fetches an NVX extension of the SDK 8 driver by name and calls it without checking the pointer. Unknown names return null, as the real driver does for a name it lacks; this one the driver has, so it joins `dm/gfx/nvn/names.txt` (thunks regenerated with `gen_protos.py`; no prototype in `nvn.h`, so `NVN_NOPROTO`) as a no-op. -- **The frame loop runs, but the game waits for code it loads itself** (Super Smash Bros. Ultimate 13.0.5: `ro::RegisterModuleInfo` / `ro::GetBufferSize` red lines; `main` imports 1,045 `lua2cpp::create_agent_*` functions, global, that no ExeFS module defines, bound to stubs at start) → the fighters' code is 98 NROs inside `data.arc`, registered by `.nrr/cross2_*.nrr`. The SDK's own `LoadModule` (SDK 8.2.1 `LoadModuleInternal`, `sdk+0x347460`) uses the service only to map the image, then zeroes bss, relocates through rtld's module lists, calls the module's init and binds the variables of the modules loaded before; the host replaces rtld, so it takes over the whole call. The NROs are collected at pack time by the SHA-256s the NRRs list (`dm/tools/nro_collect.py`: loose files, the current files of an indexed archive, or a scan for stored and zstd-framed NROs; Smash's NRRs of older releases also register 666 superseded copies, which `data.arc` still holds and which are left out), rewritten like any module (`patch_nso.py` reads NROs: the flat image, segments at +0x20, build ID at +0x40) and listed in `/nro.tsv`. The host maps them all at start and keeps them out of symbol lookups; `LoadModule` finds one by the image's SHA-256, restores its data and bss (after the first load), relocates it against the modules loaded then in load order, binds the earlier modules' symbol relocations it defines (main's `lua2cpp` stubs), fills `nn::ro::Module` as the SDK does (+0 module, +8 state 1, +0x10 image, +0x18 buffer) and calls its `DT_INIT`; `UnloadModule` calls `DT_FINI` and binds every relocation that pointed into it again (`loader.c` "NRO modules", `hle_ro.c`). `GetBufferSize` returns the bss size from header +0x38 as the SDK does. The startup init pass must skip NROs (their init ran unrelocated and crashed). Smash loads six NROs in its first 30 s and calls none of the stubs; only data is written at a load, so it works with code embedded in the iOS executable: `gen_embed.py` packs the NROs into three sections (`__dm_nro_x0`, `__DM_NRO_W`, `__dm_nro_x1`) and a table of their pieces (`__dm_nro_tab`), since a Mach-O file has at most 255 sections and three per NRO would pass it, and `map_image` maps an NRO from there (`loader.c` `nro_pieces`). Played on an iPhone. -- **SDK 7 calls of Pokémon Sword** (one boot each; the import list against the hooks and every other game's imports named all of them beforehand: 33 `nn::` imports no host hook names and no other game imports): the Poké Ball Plus (`hid::*Palma*`, 26 functions; the first boot crashed in `hid::detail::SetPalmaBoostMode`, a null read at `sdk+0x238eb0`, the hid service never opened) answers that none is connected: no npad has the Palma style, so the game never gets a connection handle, and every call on a handle fails (`hle_hid.c`); `pdm::QueryApplicationPlayStatistics` (the second boot: a null read at `sdk+0x336ccc`) asks how often this user launched two other games and reads only the launch count, so each is never played (`hle_offline.c`); `err::ShowNetworkServiceError` is logged. `nvnDeviceGetWindowOriginMode` and `nvnDeviceGetDepthMode` were red stubs: the game sets neither mode and reads both. -- **SDK 12 names** → older SDKs export other names for the same calls (`ListAudioDeviceName` / `GetActiveAudioDeviceName`, singular `hid::GetNpadState`, `OpenDefaultAudioOut`); hook both generations. -- **SDK 11 names** (Metroid Dread) → `fs::CommitSaveData(mount)` is the older name of `fs::Commit`; `fs::DeleteDirectory` (an empty directory) was missing too. -- **`fs::Commit(const char* const*, int)`, the mount-list overload** (Animal Crossing: New Horizons, PR #25) → the SDK's own code opened fsp-srv and aborted, leaving the save writes uncommitted. The host commits the list as one transaction when every name is the same host save; cache and temporary storage in the list are skipped (their writes are already on disk, as for the single-name `Commit`), and a list of two separate saves is refused without committing either. -- **An `nn::diag` abort with Result 0x2aca in the first frames of `nnMain`** (Hollow Knight: Silksong, Unity 6000.0.50f1: `bt: sdk+0x514580 main+0x1296c64 …`) → `main+0x1296c60` calls `hid::GetDebugPadStates(states, 16)` every frame, and the SDK's own reads a debug pad resource nothing initialized (module 202, hid). The host had only the single-state `GetDebugPadState`; the history form now answers every requested entry as not connected, newest first with consecutive sampling numbers (`hle_hid.c`). -- **More SDK 11 calls that reach a service** (13 Sentinels) → `hid::GetMouseState` (the first boot aborted in it: the SDK's mouse resource was never initialized), `fs::CleanDirectoryRecursively`, `fs::EnableGlobalFileDataCache`, `oe::SetUserInactivityDetectionTimeExtendedUnsafe`, `time::ToPosixTime` (host `mktime`) and `audio::GetAudioOutName`. Finding them before the first boot: list `main`'s `nn::` imports that no host hook names (the survey's import table, filtered against `HLE_FORCE` names); most `nn::os` / `nn::mem` imports run natively in `sdk` and need nothing. -- **SDK 11 calls of Monster Hunter Rise** (one boot each; the import list against the hooks and the other SDK 11 games found most of them before the first boot): `oe::IsCpuOverclockEnabled` (a null read in `sdk+0x35258`, apm's session, the first call in `nnMain`), `album::Initialize` (abort in `sm::Initialize` under `capsrv::InitializeForApplication`; saving to the album now fails, `hle_offline.c`), `oe::GetDefaultDisplayResolution` and its change event, `oe::SendServerMaintenanceOverlayNotification`, `time::LoadTimeZoneRule` and the `ToCalendarTime` overload with a rule (the host's zone is used), `time::StandardNetworkSystemClock::IsAccuracySufficient` (offline: no), `hid::GetKeyboardStates`, `fs::GetSaveDataSize` / `ExtendSaveData` and the one-argument `fs::MountCacheStorage`. `nvnDeviceGetTexelFetchHandle` returns the texture ID with sampler 0. -- **Texture creation returns null right after the inline keyboard's `Initialize`, then a null read at `main+0x7713044`** (Monster Hunter Rise) → the game makes three textures of `swkbd::InlineKeyboard::GetWindowSize`, and the host answered 0 × 0 (its creation function returns null for a size of 0, `main+0x77112b8`). It is 1280 × 720 as on hardware now. SDK 11 adds keyboard setters (window mode, key top alpha / scale / position, touch, hardware keyboard, user words, dictionaries; ignored) and two callbacks the applet makes from `Calc`: finished-initialize after `Initialize` and released-user-words after `UnsetUserWordInfo`. The host makes them on the next `Calc`. -- **A null-pointer crash early in `nnMain`, right after NVN setup should have happened** (Super Mario 3D World + Bowser's Fury, `main+0x759e08`: the device singleton `main+0x1a28ea0` is still null) → the game checks `nvnDeviceGetInteger(API_MAJOR / API_MINOR)` for major == 53 and minor ≥ 313 (`main+0x711c88`) and quietly skips creating its device otherwise. The reported version follows the SDK (`hle_nvn.c` `nvn_api`: 53.313 for SDK 10 and older, 55.15 for later ones, Mario's requirement); the profile key `device_api` overrides it. NVN 53 also brings `nvnQueueAcquireTexture` (the older window acquire: the queue waits instead of a sync object), `nvnSamplerCompare` and the debug group calls (no-ops). -- **Crash after the shared fonts load, looking up the "?" glyph (`main+0x6c73c0`, a font cache lookup returning null)** (3D World) → SDK 10's `ScalableFontEngine::LoadFont` failed with error 403 and `SetFont` with 301 on both stand-in fonts. This game records shared fonts as type 0 (`main+0x975658`), calls no `ScalableFontEngineHelper::Decode` and hands the `GetSharedFontAddress` pointer straight to `LoadFont`, so the pointer must be the plain TTF; Mario decodes the same pointer and needs the 8-byte BFTTF header in front. SDK 10 and older games all take the plain TTF; the host decides by the SDK version (`hle_sdk16.c` `font_skip`, the profile key `shared_font` overrides). Found by probing the engine's error code after each call (`DM_PROBE` on the instruction after `GetError`). A game on a newer SDK can import `Decode` (and so get the header) yet hand the address straight to `LoadFont` the same way (The Legend of Zelda: Echoes of Wisdom 1.1.0, SDK 20.5.1: `main+0x8b37c` crashed at address 0x38); the host's `LoadFont` takeover passes the TTF past the header whenever it is given the host's own BFTTF base, so the address form `font_skip` picks only matters to `Decode` and other readers (`hle_sdk16.c` `h_font_LoadFont`). -- **More SDK 10 calls that reach a service** (3D World): `settings::LanguageCode::Make` and the reversed `operator==(LanguageCode, Language)` (the first comparison in `nnMain` aborted in `sm`), `account::ListAllUsers`, `oe::GetPerformanceConfiguration`, `oe::SetAlbumImageOrientation`, the `nifm` IP queries with SDK 10's `socket::InAddr` / `in_addr` mangling, `prepo::AddSessionId` / `AddInternetConnectionStatus` (entering the save select), and `vi::GetIndirectImageMemoryRequirement`. -- **Null read in `applet::CreateLibraryApplet` under `swkbd::InlineKeyboard::Initialize`** (3D World, entering the save select after the character select) → the inline keyboard is a resident library applet whose results arrive through `Calc`. `hle_swkbd.c` takes over every entry that touches the applet: `Appear` arms a request, and the next `Calc` asks for the text the way `ShowKeyboard` does (`DM_KBD`, else the front end's input box) and hands it to the game's changed-string and enter callbacks, or calls its cancel callback; `GetImage` reports no picture. `GetImageMemoryRequirement` is not a pure computation: it asks vi, hence the vi hook above. -- **The sound stutters and the input box lags while the player types a name** (Octopath Traveler 0 on an iPhone: one 4.7 s frame, 5,035 wakeups a second instead of about 1,800, and no audio underruns) → the host froze the guest clock while `swkbd::ShowKeyboard` was up (d2r-mobile's choice, so that monsters did not attack while the player typed). Unreal calls it off its game thread, so the game kept running, and every thread that waits on the clock spun while its audio mixing starved. On the Switch the applet blocks only the caller while the clock runs; the host now does the same (`hle_swkbd.c`). The iOS input box froze the clock a second time around its own wait (`front_ios.m`), so the first fix changed nothing on the phone (4,977 ms and 2,577 ms frames, 5,181 wakeups a second): look for every caller of `guest_clock_freeze_for`, the front ends included. On the Mac `DM_KBD= DM_TEST_SWKBD_HOLD_MS=8000` holds the keyboard up: frozen, 8.8 fps and 7,936 wakeups a second in that window; running, 30 fps and 1,788. The inline keyboard keeps the freeze: its game (Monster Hunter Rise) does not block on hardware at all, and the host's input box blocks it. -- **The hunter's name box stays empty, then stays open with the name in it** (Monster Hunter Rise, character creation) → the inline keyboard used to count every `Appear` as a cancel. Now the text arrives through the enter callback, whose argument is the text inline (`char16_t[502]`, the length at +0x3EC; the changed-string argument adds three cursor positions at +0x3F0..+0x3F8: `ikb_str_arg`, read off the game's callbacks at `main+0x6fb3910` / `main+0x6fb3afc`). The game then finishes the entry only when `Calc` returns 1 (ready / hidden) after a non-zero value (`main+0x6fb3e6c`); the hook returned nothing. `Calc` now returns 0 until initialization finishes, 3 while shown and 1 otherwise. `SetFinishedKeyboardCallback` is called on the `Calc` after `Disappear`. AppearArg's `textMaxLength` (+0x20) limits the text. -- **`CRASH sig=10` at a heap address, called from `main+0x6bca160` right after the name was entered** (Monster Hunter Rise) → `nn::ngc::ProfanityFilter` is polymorphic and the game masks the name through its vtable (slot 8, `MaskProfanityWordsInText(int *, char16_t *, uint32_t)`). The constructor hook did nothing, so the object kept what the heap held. It now gets a host vtable (`ngc_vtable` in `hle_misc.c`): each slot of the SDK's `_ZTVN2nn3ngc15ProfanityFilterE` is matched by address with the class's methods, and slots the loader already bound to forced take-overs keep them. Nothing is masked or reported. -- **Abort in `nn::sm::Initialize` under `nn::oe::FinishStartupLogo`** (Metroid Dread, right after the first frames) → it reaches the applet proxy through `am::GetApplicationFunctions`. The host has no startup logo: a no-op hook (`hle_oe.c`). -- **Abort in `nn::sm::Initialize` under `nn::hid::VibrationNode::Update`** (Metroid Dread) → the SDK's vibration graph flushes queued values with `hid::detail::SendVibrationValues`, which opens the hid service. The game sets handles on its `VibrationTarget`s even when the host reports no vibration device. The flush calls it through the PLT, so an `HLE_FORCE` on that symbol catches it and returns success (`hle_hid.c`). -- **An Atk task aborts in a `switch` default (`UnexpectedDefaultImpl`, then `AbortImpl`) while a stream sound loads** (Metroid Dread, `main+0x1225a8c`) → the stream path was `rom:/system/snd/../../sounds/streams/…`, and the host answered "not found" for any `..` component. Horizon's path normalizer resolves `.` and `..` lexically and fails only above the mount root; `host_path_ex` now does the same (`norm_rest` in `hle_fs.c`). -- **Abort in NVIDIA's video path (`NvRmChannelOpen` → `NvOsDrvOpen`)** → the SDK movie library's hardware decoder opens nvdrv directly. The host takes over `movie::Decoder` (`hle_movie.c`, next entry); nvdrv opens fail (`hle_nvdrv.c`). -- **The game waits forever at a movie (Engage's 94 s opening `S01F.webm`; frames keep presenting, nothing happens)** → Unity demuxes the WebM itself and feeds VP9 packets to `movie::Decoder`; with no working decoder the movie never ends. `movie::Decoder` is a thin wrapper over Android's MediaCodec, and its vtable entries are symbol relocations, so `HLE_FORCE` on each method also catches Unity's virtual calls. The host implements the protocol: four `nn::os` events (`DecoderEvents` = input available, output ready, format changed, error), input indices (`AcquireInputBufferIndex`, none = 0x1011; `GetInputBuffer` fills the client's `movie::Buffer`; `SendInputBufferForDecode(index, offset, size, µs, flags)`), output indices (`AcquireOutputBufferIndex`, none = 0x100e; one NV12 frame per packet copied into the buffer Unity allocated, w × h × 1.5; `ReleaseOutputBufferIndex`). While every output slot is taken, packets wait and their input indices are not returned, as with MediaCodec; dropping them instead lost frames. `Buffer`, `MediaData` and events are handled through the SDK's own exported functions (`SetDataAndCapacity`, `SetRange`, `SetInt32`, `SetRect`, `SignalEvent`). Unity reads `width`, `height`, `crop`, `nv12-y-stride`, `nv12-y-offset`, `nv12-uv-offset` and `slice-height`. VP9 packets are decoded with libvpx (`dm/third_party/libvpx`) on one host thread per decoder: guest threads run on the stacks the SDK gives them, too small for libvpx. The guest thread that sends a packet or returns an output index waits for the decode and signals the events itself, because `SignalEvent` is guest code and cannot run on a host thread. Packets are decoded in arrival order from one queue. When the movie ends the game continues to its title screen. -- **Abort with no message at start-up in a service call the host has no hook for** (Super Mario Odyssey, SDK 7: `account::OpenPreselectedUser`, then `bcat::Initialize`, then `audio::AcquireAudioDeviceSwitchNotification`; a null read in `account::GetLastOpenedUser`) → the SDK's IPC to the service fails and it calls `AbortImpl` (`Break` from `sdk+…` with an `nn::diag` backtrace). Resolve the frames with `sdis.py --sym` on the decompressed module, list the game's imports of that service the host does not hook (the survey's import table against the `HLE_FORCE` names), and add the ones that talk to a service; library helpers that only fill the game's own buffers need nothing. Look at how the game uses the result before choosing it (`bcat` mount failure is the game's own "nothing delivered" path). -- **The same abort from `ec::InitializeForShopServiceAccessors`, called from Unreal's `FOnlineSubsystemSwitch::Init` at the start of `nnMain`** (DRAGON QUEST XI S, SDK 7.3.2: `SVC 0x15` / `0x1f` / `0x21` placeholders, then `Break` with `sm::Finalize` under `ShopServiceAccessServer::Proxy::Initialize`) → the shop service accessor library. The library init succeeds with nothing to set up, an accessor never initializes, and its async response is complete at once with no data (`hle_misc.c`); the accessor and response are stubbed whole because the SDK's destructor aborts unless `Finalize` succeeds. Listing the rest of `main`'s imports with no hook (hooks are `HLE_FORCE`, `SSF`, `SKF`, `IKB_IGNORE` across `dm/host` and `dm/gfx`; `strings` misses names the symbol table stores as the tail of a longer one, so list them with `nso_syms.dynsyms`) gave `oe::Begin` / `EndAutoSleepDisabledSection` and `IsAutoSleepDisabled` (counted, `hle_oe.c`), `pctl::CheckFreeCommunicationPermission` (never), SDK 7's `album::SaveScreenshot` without an orientation (fails), `friends::GetBlockedUserList`, `DeclareOpen` / `DeclareCloseOnlinePlaySession` (unavailable) and `fs::DisableGlobalFileDataCache`. The game then runs its opening movie at 30 fps. -- **Buttons and menus work, the sticks do nothing in play** (Super Mario Odyssey: its first hint, "Look Around", waited forever) → the game reads all 16 entries of `GetNpadStates` and ignores the returned count; the host filled only the newest. It polls every controller slot each frame, which moves the shared sampling number on by about 16, so the stick samples it combined were mostly empty, while buttons and the stick's direction bits (which menus use) still came through. The host now fills every requested entry with the current state under consecutive sampling numbers. `DM_KEYS` takes `RLeft` / `RUp` / `RRight` / `RDown` for the right stick. -- **Motion moves never happen; shaking does nothing** (Super Mario Odyssey: cap throws by shaking) → the host handed out six-axis handles but reported no sensor. It now reports one per connected controller: two for handheld or dual Joy-Cons, one otherwise, with the style, npad id and device in the handle. At rest the sensor lies flat: acceleration (0, 0, −1) G, no rotation, identity direction. Samples are 5 ms apart on the guest clock and numbered by time, so a history read returns the samples at those times. `DM_BTN_SHAKE` (`dm_input.h`: the iPhone shaken, Space on the Mac, `DM_KEYS` `Shake`) plays a side-to-side shake in whole cycles. Odyssey reads all 16 history entries per sensor and ignores the count, like its npad history; find a game's swing detector by probing the accelerometer value it reads (`DM_PROBE=main:626248@x8+10` in Odyssey 1.4.1). -- **Movies through the SDK's own demuxer and a decoder in native-texture mode** (Metroid Dread): - - `movie::Extractor` (libwebm in `subsdk0`) runs natively. The host used to force `SetDataSource` to fail because Engage's Unity demuxes by itself, so the game crashed on a null right after the failed open. - - The decoder is created with output format 3 and mode 2 (`DecoderMode_NativeTexture`). The client's `movie::Buffer` then carries the `NVNtexture` object itself as its data, 8 bytes of capacity (an `RGBX8_SRGB` texture embedded in the game's texture pool entry, `main+0x226c98`), plus `SetNvnDev` / `SetNvnSync`. The host converts the VP9 picture to RGBA (BT.709 / BT.601 and range from the stream) and writes it into the texture's storage like `WriteTexels`, then marks the sync as signalled (`nvn_host_write_texture` / `nvn_host_signal_sync` in `hle_nvn.c`). - - The game calls the complete-object destructor `movie::Decoder::~Decoder` (`D1`), which the host now takes over too; the SDK's own one destroyed the host object as a `DecoderImpl`. - - `RegisterClientEvents` copied the four event pointers, because Dread's struct had been overwritten by its second movie. DRAGON QUEST XI S (Unreal's Switch movie player, VP9 with 6-channel Vorbis) does the opposite: it registers its struct right after constructing the decoder (`main+0x39edab0`, through the decoder's vtable) and fills it later, so the copy held leftovers (`0xe00000010`) and the first deferred signal crashed in `InternalCriticalSectionImplByHorizon::Enter`. The SDK keeps only the pointer (`DecoderImpl::RegisterClientEvents` stores it at +0xd0) and reads the entry each time it signals, skipping a null one (`SendOutputBufferEventAvailableToClient` and the others, `subsdk0+0x1feb0`). The host now does the same (`md_event` in `hle_movie.c`). When the client's entry is not a readable, 8-byte aligned pointer, it falls back to the entry's value at registration if that was one (Dread's case), and otherwise logs a red line. - - As with MediaCodec, `Flush` announces nothing (the game ends a movie with `Flush`, `Flush`, `Stop`, destroy, and calls `Start` after a `Flush` when it loops), and no input or output index is handed out while the decoder is stopped. - - Also as with MediaCodec, an input marked end-of-stream is the last until `Flush` / `Stop`: no input index is handed out or announced after it (`in_eos` in `hle_movie.c`). DRAGON QUEST XI S's movie player sends an end-of-stream packet (length 0, time 0) for every input it gets once its extractor is done, until the game stops the movie. When the prologue movie was skipped, each one became an end-of-stream output that the player turned into an audio sample: 2.25 GB were mapped within seconds, past the application pool, and the game crashed in its allocator. - - The "input available" after `Start` is signalled about 2 ms later from a guest thread, as the SDK's codec thread would. Pending signals are dropped at `Stop`, `Flush` and destruction. -- **Null read in `sdk` under `nn::applet::CreateLibraryApplet` (Engage: title menu → DLC → `nn::ec::ShowShopAddOnContentList`)** → every `Show…` call that opens a system applet (eShop, friend lists, web pages, error screens) goes through the applet proxy that `oe::Initialize` would have set up, and the host has none. Hook each `Show…` overload the game imports: list `main`'s undefined symbols containing `Show` / `Applet` and grep the host sources for each mangled name; one overload missing is enough to crash. The shop calls return `void` and act as if the player closed the shop (`hle_misc.c`); the friends and web ones return failure (`hle_friends.c`, `hle_offline.c`). -- **The opening movie never starts, in about half the boots (DRAGON QUEST XI S: a black screen and no sound from boot on)** → Square Enix's `UMediaPlayer` has two natives of its own, `StartPreDecoding` and `IsPreDecodingCompleted`. The opening's Blueprint polls the second once a frame and only then plays its Matinee, whose event key calls `Play`. Its Switch player (vtable `main+0x77921b8`, `IsPreDecodingCompleted` at `main+0x42d4d20`) is complete when every selected track has at least one decoded output queued. The player starts its clock and its decoders on the first `TickInput` after `StartPreDecoding`. The render thread's first fetch (`main+0x39f07d0`, the `+0x5c7` flag) takes every decoded picture and shows the newest. The host decoded all eight video outputs (to 236 ms) within 36 ms. Each Vorbis decoder's 8 outputs were full by then, because nothing consumes audio before `Play`, so the extractor waited for an audio input and no picture followed the ones the render thread took. The movie started only when the game thread's poll came between the first decoded picture and that fetch. Audio decoders now keep 32 outputs (`MD_NOUT`, 64 since Monster Train 2 below; video keeps 8), so video keeps flowing after the first fetch: 8 boots out of 8 start. To find such a gate: probe the player's `SetRate` (`DM_PROBE=main:`) to see whether the game ever asks to play, then follow the Blueprint native that calls it (the `FNameNativePtrPair` tables give the names). -- **A long black screen at boot with nothing in the log (Engage: about 17 s on the Mac, 32 s on an iPhone)** → sample the process (`sample 3`). If the top of stack is `getxattr` under `path_component`, the game is walking RomFS, and every path component scanned its whole parent directory. That scan maps the game's case-sensitive names onto APFS, so it calls `getxattr` on each entry, and Engage has directories of 38,000 files. Read-only mounts now use a per-directory name index built at the first lookup (`dir_index_get` in `hle_fs.c`), which also speeds up every file Wwise opens during play. The first frame then came at 5.7 s on the Mac. The rest was the listing itself: `ReadDirectory` called `stat`, `getxattr` and a failing `fopen(".holes")` for every entry. Now only `.dm-case-*` names (the only ones that carry the xattr) read it, directories take their type from `d_type`, and files use `fstatat` / `openat` relative to the directory. The first frame comes at 1.8 s on the Mac (9.9 s → expected about 3 s on an iPhone), and the opening movie starts at 7 s instead of 49 s. -- **DLC the player owns is not unlocked** → the host reports add-on content from `DM_AOC_DIR`, one numbered folder per DLC index (title ID − base − 0x1000) holding that DLC's RomFS. `build_ipa_from_nsp.py` extracts the DLC NSPs in the folder with `nx_extract --aoc`, `build_ipa.sh` copies them to `/aoc/`, and the app sets `DM_AOC_DIR`. `aoc::CountAddOnContent` / `ListAddOnContent` report those indices (`hle_misc.c`), and `fs::MountAddOnContent` mounts the folder read-only (`hle_fs.c`). Engage's two DLC packages each hold one empty marker file (`e0`, `h0`), and the content is already in update 2.0.0. `DLCManager.RefreshHasList` marks a content owned when its mount succeeded (probe `main+0x29f3c88`). D2R keeps its fixed `{1}` without `DM_AOC_DIR`; `DM_AOC_OWNED=0` reports none. -- **Abort in `nn::sm::Initialize` under `nn::pctl::IsFreeCommunicationAvailable` (Engage, entering its first battle map)** → the SDK connects to the parental-control service through `sm`. The host answers it like the other free-communication calls: not available (`hle_misc.c`). -- **Abort inside `nn::sm::Initialize`** → an SDK path reached a system service the host doesn't replace → hook the public SDK entry point named in the backtrace (`HLE_FORCE` in the relevant `hle_*.c`). The log shows it as `SVC 0x1f` (ConnectToNamedPort, port name `sm:`) and `SVC 0x21` answered by placeholders, then `nn::diag` abort. -- **A weakly imported function the game calls although nothing defines it** (Octopath Traveler 0: the `thread_local` init function `_ZTHN13LowLevelTasks5FTask10ActiveTaskE`, an `UNIMPL` line, then an abort) → the loader bound every undefined import to a named stub, so the address test the compiler emits for a weak symbol saw a function. An undefined weak symbol that no module defines and the host does not implement is now 0, as Horizon's rtld binds it (`loader.c`, logged as `undefined weak … bound to 0`). -- **SIGSEGV in `android::Surface::getQueuedFrameNumber`** under `nn::vi::GetLatestFrameNumber` (Octopath Traveler 0: Unreal paces frames by the layer's frame number, `main+0x10140e8`) → the SDK reads the vi service's surface, which the host never made. `hle_vi.c` answers with the frames presented so far; `ListFrameHistory` returns no entries. -- **SIGSEGV at address 0 in `nn::fs::GetSaveDataSizeMax`, about 13 minutes into a new game** (Octopath Traveler 0: Unreal's save system asks before its first save) → the SDK's version reads the fsp-srv object the host never made. It reports 256 MB of data and journal, as `GetSaveDataSize` does (`hle_fs.c`). Finding the rest before they crash: list main's undefined imports that no host hook names, by namespace, and read each one's SDK code; for this game that gave `fs::GetCacheStorageSize` and `fs::GetFreeSpaceSize` (the same answers), `account::PushOpenUsers` (nothing to hand over), `web::ShowOfflineHtmlPage` (cut, as `ShowWebPage`) and `oe::ExecuteProgram` (logged red: the host runs one program). The rest were pure computations (the swkbd presets, the web argument constructors, `time::GetDayOfWeek`, the audio resampler) or socket calls behind a `Socket()` that already fails offline. -- **DRAGON QUEST VII Reimagined (SDK 19.3.1, Unreal Engine 4.27)**: an abort in `sm::Initialize` under `audio::GetAudioDeviceOutputVolume` (the game thread, `main+0xee8edc`); the volume `SetAudioDeviceOutputVolume` sets is now kept and read back (`hle_audren.c`). The import list against the hooks then gave the rest before they were met: `ec::FinalizeForShopServiceAccessors`, the shop response's `Cancel` and `GetErrorCode`, `ec::ShowShopSubscriptionDetails`, the Nintendo Account request's `GetAuthorizationCode` and `GetState` (both fail, as `GetIdToken`), and `kpr::GetKeyCodeMap`, which asks the settings service (`settings::detail::GetKeyCodeMap`) for the keyboard layout's map; the game zeroes a 0x1000-byte map and fetches it only once a keyboard reports keys (`main+0x1f0eb4`), and the host leaves it zero. The rest fill the game's own structures or sit behind a socket that fails offline. -- **The boot sits at a black loading screen for good, CPU nearly idle, every thread waiting, about 1 boot in 6 (3 in 4 with four Mac hosts at once)** (Pikmin 4, Unreal Engine 4.26.2: the title movie is never opened, and the save files are never probed) → a race the host's instant offline answers lost. The game's boot state machine (`main+0x1f2f0e0`, state at `this+0xd4`, found by probing the save probe's call stack and dumping the state each tick) stayed in state 7, which polls the status word (+0x14) of the game's BCAT news runnable ("OlimarMemo") for 0, 4, 5, 6 or 7; it read 1. The runnable's constructor (`main+0x203f3b0`) clears the status, starts `FBCATThread` and only then stores 1; the thread's `Run` (`main+0x203f510`) stores 2, calls `nn::nifm::SubmitNetworkRequest`, sleeps while `IsNetworkRequestOnHold`, and stores 6 when `IsNetworkAvailable` is false. On a Switch the request stays on hold while the nifm service evaluates it, so the constructor's 1 always comes first; the host answered "not on hold" at once, and a thread that ran before the constructor's store finished at 6, which the 1 then overwrote. The host now keeps a submitted request on hold for 50 ms, then settles it as unavailable, as Horizon's asynchronous request does (`hle_net.c`): 12 of 12 boots with four hosts at once reached the title. How it was found: the hung boots' logs diverged from good ones at the save probe (`DM_FS=1`), the thread dumps showed nothing but idle waits, and `DM_PROBE` on the state machine, then `peek` lines in a hung run's `DM_CTL` file, followed the pointers to the status word (a probe on the hot path slowed the boot enough to hide the race: 8 of 8 good). -- **A movie in grey with vertical bands of colour, its luma sharp** (DRAGON QUEST VII Reimagined: CRIWARE's Sofdec hands its VP9 video to `movie::Decoder` in mode 1, NV12 into its own buffers, and draws the planes with its shaders) → the bands were the top row's chroma: every chroma row was read from the first one. Sofdec reads `nv12-uv-stride` from the output format, which the host never set (Unity reads `nv12-uv-offset` only). The host now sets everything the SDK's `DecoderImpl::GetOutputBufferFormatFromDecoder` (SDK 19 `subsdk0+0x121f50`) does: also `nv12-uv-stride` (the Y stride), `nv12-width` and `nv12-height` (`md_format` in `hle_movie.c`). The same missing key crashed Catherine: Full Body's Sofdec2 player (below). -- **NVN calls new with Unreal** (Octopath Traveler 0): `nvnTexturePoolGetSize` / `nvnSamplerPoolGetSize` and their memory pool / offset getters (an abort in device setup when they returned 0), and `nvnCommandBufferFenceSync`, which Unreal's RHI records in its command buffers (`OP_FENCE_SYNC`: the replay fences the sync with the submission that reaches it). Unreal Engine 4.18 (DRAGON QUEST XI S) also asks for `nvnTextureGetTextureAddress` (pool address + offset, as `nvnMemoryPoolGetBufferAddress`) and `nvnCommandBufferGetCommandMemorySize` (used + free, as in NVN; `GetCommandMemoryFree` is what is left in the current segment). -- **A game update aborts at boot where the base game ran** → the update imports SDK functions the base game didn't. Diff the undefined dynamic symbols of the two `main` modules before the first run. Mario 1.2.1 added 28, among them `nn::nfp` (amiibo, initialized at boot), `pl::IsRunningOnOunce` (is this a Switch 2), `ec::ShowShopApplicationInformationForUpgrade`, `account::PopOpenUsers` and `aoc::CountAddOnContent`. Read the call sites before choosing answers: Mario only reads the count from `PopOpenUsers` and falls back to `TryOpenPreselectedUser` on 0, and only counts add-on content when running on a Switch 2. The host answers as a Switch 1 with no amiibo and no add-on content (`hle_sdk16.c`, `hle_account.c`, `hle_misc.c`). -- **SDK 20 calls that reach a service** (Breath of the Wild 1.9.0, SDK 20.5.22): `applet::GetAppletResourceUserId` (SDK 20's `OpenAudioRenderer` aborts on an ARUID of 0, which the SDK's own getter returns without am; the host reports 0x80), `pl::RequestSharedFontLoad`, the VR-mode calls (`pl::IsVrMode` / `BeginVrMode` / `EndVrMode`), `pctl::CheckStereoVisionPermission` (bool, permitted), `oe::TryPopLaunchParameter` (none), `time::StandardSteadyClock::GetCurrentTimePoint` (host monotonic seconds, fixed source ID; `GetSpanBetween` runs in `sdk`), the console six-axis sensor (`hid::*SevenSixAxisSensor*`: zero states) and more `nfp` calls (`AttachActivateEvent` / `AttachDeactivateEvent` crashed in the SDK at `sdk+0x3f2a78`; also two-argument `Mount`, `Unmount`, `GetCommonInfo`, `GetRegisterInfo`, `GetState`, `Finalize`). Two new imports need nothing: `nvnPrivate0` is a flag in `sdk`'s bss that enables an inlined command-buffer fast path when the NVN driver sets it (it stays 0), and `nn::ro::ProtectRelro` (called from `main`'s DSO init) runs natively on the host's no-op `SetMemoryPermission` / `SetMemoryAttribute`. -- **More SDK 20 calls that reach a service** (Tears of the Kingdom 1.4.3, the same `sdk` build as Breath of the Wild 1.9.0; each aborted in `sm::Initialize` or read a null applet proxy, one per boot): `oe::GetPreviousProgramIndex` (−1: launched from the home menu); the hardware Opus decoder's C interface (`GetHardwareOpusDecoderWorkBufferSize`, `InitializeHardwareOpusDecoder`, `DecodeOpusInterleavedWithHardware`, …), forwarded to the `sdk`'s software `OpusDecoder` C interface with a host-held decoder state per game object; cache storage (`OpenCacheStorageList` / `ReadCacheStorageList` report none, `CreateCacheStorage` / `MountCacheStorage` use a host folder beside the temporary storage); `fs::IsSaveDataExisting(ApplicationId, Uid)` for another title (it asks for Breath of the Wild's save: none) and `MountSaveDataReadOnly`; `time::ClockSnapshot::CreateWithStandardSystemClock` (0xD0 bytes, filled on the host from the steady clock, the user clock and the calendar; the network fields stay 0). The import list itself says where to look: diff the undefined symbols against the closest title the host already runs, drop what the host hooks, and the rest are the candidates. To find which `sdk` function aborted, map the backtrace offsets to the nearest exported symbol. -- **STEINS;GATE RE:BOOT (SDK 20.5.17, M2's ADV engine), the one call it needed**: a null read at `sdk+0x1f2314` as the first movie starts (the title menu's background, right after the first press), in `nn::audio::SetAudioOutVolume`, called from `main+0x1c4fbc` on the AudioOut the game opens for the movie's AAC sound (`OpenDefaultAudioOut` without an event, `main+0x1c48c0`). The SDK's version calls the audout session's virtual method, and the host never made the session. The game sets the volume every frame while a movie plays. The host keeps it per output and scales the PCM16 it plays, saturating; `GetAudioOutVolume` reads it back (`hle_audio.c`). It was among `main`'s imports that no host hook names; the others there fill the game's own structures or run in `sdk` (the CPU audio renderer, the swkbd and hid argument builders). -- **Animal Crossing: New Horizons (SDK 20.5.4)**: - - **Abort in `sm::Initialize` under `sf::hipc::InitializeHipcServiceResolution`, right after the first frames** (from `main+0x5b8198`) → `main` links its own clients of `olsc:u` (cloud save data backup settings) and `npns` (push notifications), which talk HIPC through `sm` and `svcSendSyncRequest`. For each user the game asks whether backup is on and switches it off where it is (`main+0x2809c8c`); a failed connect aborts as well. `hle_sf.c` takes over the SDK's HIPC client entry points (`InitializeHipcServiceResolution`, `ConnectToHipcService`, `SendSyncRequest`, `CloseClientSessionHandle` and the control requests `QueryPointerBufferSize`, `ConvertSessionToDomain`, `DuplicateSession(Ex)`): a service it serves (`olsc:u`) gets a host session, and every request on it succeeds with zeroed output, which reads as backup off for every user. Any other name is refused with a red line, as `sm` refuses a service nobody registered (`npns` was never connected). `DM_SF_TRACE=1` dumps each request. - - **Device save data**: 3.0.3's `main` mounts no account save data; the island is in device save data (`MountDeviceSaveData`, `GetDeviceSaveDataSize`, `ExtendDeviceSaveData`), which the host gives the account save's host directory and transaction (one player). A second mount name over a save already mounted now shares that mount's transaction (`mount_ex`); it used to clone the committed directory again, over the first name's pending writes. Also `fs::Commit` with a `CommitOption` (its one flag clears the restore flag) and `GetSaveDataRestoreFlag` (never restored from a backup). - - Found in `main`'s imports before the first boot: the adjustable user clock (`time::AdjustableUserSystemClock::GetCurrentTime`, which the game's days follow) reads the time service's shared memory in the SDK (`MonotonicSystemClock::GetCurrentTime`, `sdk+0x4421cc`), which the host never maps, so it is the host's clock, as the user clock; `time::GetStandardSteadyClockCurrentTimePoint`, the free-function form of the steady clock; `oe::GetRegionCode` (from the language: Japan, the Americas, Europe or Hong Kong / Taiwan / Korea); the SDK 20 album calls `SaveAndShareScreenshot` with a share ID and `GetSharePostedLastUrl` (no album: they fail). -- **Abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodes`** (Paper Mario: The Thousand-Year Door, SDK 15.3.2, early in `nnMain`, `main+0x327888`) → the game asks for up to 32 codes and takes the index of `oe::GetDesiredLanguage`'s code in the list as its language. The host lists the system's codes in `nn::settings::Language` order, the table `LanguageCode::Make` uses (`hle_sdk16.c`). It was the one import of `main` that no host hook names and that reaches a service; `fs::MountRamDisk` and the swkbd and hid helpers it also imports run in `sdk` or were never called. -- **Unicorn Overlord (SDK 16.2.4, Vanillaware's engine as in 13 Sentinels), one boot each**: - - **Abort in `sm::Initialize` under `nn::codec::GetHardwareOpusMultiStreamDecoderWorkBufferSize(int, int, int, int, int)`** (at `nnMain`, CRI's Opus streams) → the C interface of the hardware multi-stream decoder, including the SDK 12+ overload with an option argument. Age of Calamity only needed the class and the four-argument size. Each call goes to the `sdk`'s software `OpusMultiStreamDecoder` C interface, with a host-held state per game object as for the single-stream decoder (`hle_codec.c`). `GetHardwareOpusMultiStreamDecoderProcessingTime` reports 0. - - **Abort in `sm::Initialize` under `nn::crypto::GenerateCryptographicallyRandomBytes`** (`std::random_device` → the `sdk`'s own `getentropy` → `InitializeCsrng`) → the `sdk` defines `getentropy`, so the host's `getentropy` hook never sees the call, but the call into `nn::crypto` goes through the `sdk`'s PLT. An `HLE_FORCE` on it fills from the host's entropy stream (`hle_os.c`). - - **Write to `0x268` right after the title screen** (`main+0x5d6708`, a constructor whose `this` is `0x200`) → `operator new` (`main+0x5a55d4`) takes the thread's ring heap, a bump allocator whose cursor starts at its base. One ring heap kept base 0, as in 13 Sentinels: the engine cuts ring heaps out of a memory pool at 64 KB GPU alignment, and the host's pool storage from malloc is only 4 KB aligned. The first allocation returned 0, which `operator new`'s null check skipped, and the next returned `0x200`. The profile key `alloc_big_align` aligns allocations of 64 KB or more to 64 KB ([dm/games/README.md](../../dm/games/README.md)); the game then plays from the title screen into its opening story scene. -- **SDK 17 C library (Ace Attorney Investigations Collection, 17.5.3, Unity 2022.3 IL2CPP)**: - - **Addressables never start (no `StreamingAssets/aa/settings.json` in the `DM_FS=1` output at all), then a null asset crash (`main+0x31100ec`)** → Unity 2022's C# file API reaches the SDK's C library, not `nn::fs`: `File.Exists` → `SystemNative_Stat` → `stat` = `open` + `fstat` + `close` (`sdk+0x948b0`). nnmusl's path check (`sdk+0x904e0`, then the converter `sdk+0x8ef50`) asks `nn::fs::IsPathMounted`, `GetPathLengthMax`, `GetUtf16PathLengthMax`, `GetEntryNameLengthMax` and `GetUtf16EntryNameLengthMax` before it calls `nn::fs::GetEntryType` / `OpenFile`, and the SDK's own versions read the empty mount table (`IsPathMounted` false; the next one aborted with Result 0x35f202). The host answers them from its mount table, with Horizon's 0x300-byte limits and no UTF-16 limit (`hle_fs.c`). The path .NET builds is `/rom:/…`; nnmusl's converter turns it back into `rom:/…`. armyja's fork (GitHub issue #18) patched the managed stat wrappers by address instead; these hooks need no per-build table. - - **`std::terminate` (`Break` from `sdk+0x5bfa48`) at the first real-time read** (right after the first-boot dialog, `main+0x3f3f68` → `system_clock::now()`) → `clock_gettime(CLOCK_REALTIME)` calls through a source pointer (`sdk+0xb05010`) that `nntimeStartup` sets once `time::Initialize` succeeds: 1 = `nn::time::GetStandardUserSystemClockCurrentTimespec`, 2 = the adjustable clock's (`sdk+0x8e414`). Both asked the time service; they are host take-overs now (`hle_time.c`), and `system_clock::now()` works on any build. - - Not needed in the end: the process heap of pull request #16 (`nninitStartup` calls `SetMemoryHeapSize` / `AllocateMemoryBlock` only without virtual address memory, and nothing else in `main` calls them) and the tolerant common string table of pull request #15 (1.0.0.2's `main` has the table intact at `main+0x46169ac`, 109 entries ending at `Hash128`). -- **SDK 19 (Pokémon Legends: Z-A, 19.3.11, Game Freak's Trinity engine)**. The import list against the hooks (the survey's `nso_syms.py` table, then `main`'s undefined `nn::` symbols that no `HLE_FORCE` names) left 86 candidates; most only fill the game's own structures (error and web argument builders, `prepo` report buffers, `swkbd` configuration, the audio resampler, `nn::async`). The ones that reach a service were hooked before they were met where the answer is clear: `time::ToPosixTime` with an explicit rule (the host's zone, as for `ToCalendarTime`), `album::SaveCurrentScreenshot` (no album: fails), `err::ShowApplicationError` (logged), and the `ShowShopAddOnContentList(UserHandle)` overload. One boot each: - - **`StandardAllocator::Initialize` aborts in `nninitStartup`** (`main+0x1b2e690`) → with virtual address memory (a 16 MB system resource) the game initializes its allocator over 12 GB of address space, `StandardAllocator::Initialize(nullptr, 0x300000000)` (`main+0x1b3e3ec`), as the Switch's 64 GB alias region allows. The alias region must hold the SDK's 1 GB node area and that region, 13 GB, more than a phone gives a process. Two cuts for 2.0.2 (its `patches.py` and `sdk_patches.py`) make it fit the 5 GB block Tears of the Kingdom runs in: the allocator's address space 12 GB → 4 GB (`main+0x1b3e3e4`) and the node area 1 GB → 62 MB (the AddressRegionManager constructor, `sdk+0x1423fc` and `+0x142468`, Engage's cut). SDK 19's AddressRegionManager puts the region right after the node area (+0x3fe2c000 before the cut), not 1 GB aligned as Engage's SDK did, so a 4.25 GB alias region holds both (`mem_alias 0x110000000`). `QueryMemoryInfo` reporting more than 4 GB would take the game's big-memory path (heaps of allocatable − 4.27 GB instead of − 2.76 GB). - - **The alias region's mapped total runs past the application pool in the first minute** → `nninitStartup` allocates and frees everything allocatable (pool − used, 3.15 GB, mapped at the top of the region) and the SDK keeps the freed pages mapped; later allocations map new pages at the bottom until a map fails, and only then does it unmap the cached block and retry. The host never refused a map, so the block stayed mapped beside the 0.4 GB the game used. Profile key `mem_map_limit` (`hle_svc.c`): a map past the pool fails with ResultLimitReached, and the used memory `GetInfo` reports counts the mapped pages, as on Horizon (without that, the startup probe itself, sized as total − used, was refused). Measured: one refused 4 MB map, the 3.15 GB unmap, the same map again. - - **Abort in `sm::Initialize` under `crypto::GenerateCryptographicallyRandomBytes`** (the first call in `nnMain`, `main+0x1734b4`) → the bytes come from the csrng service. The host fills them from its entropy source, the same stream as `getentropy` (`hle_os.c`). - - **A null read in `audio::GetAudioOutBufferCount` on Wwise's sink wakeup thread** (`sdk+0x1e7ed0`, from `main+0x30faae8`) → after every release event Wwise asks how many buffers are still queued and marks the sink starved at 0; the SDK's version read the audout session the host never made. It answers from the host queue: appended and not yet played to the end (`hle_audio.c`). - - **NVN state read back through getters** (the red `NVN unimplemented function called` lines for `nvnColorStateGetBlendEnable`, `nvnBlendStateGetBlendFunc`, `nvnPolygonStateGetCullFace`, `nvnDepthStencilStateGetDepthTestEnable`, `nvnBufferBuilderGetSize`, `nvnTextureBuilderGetWidth`, …) → Trinity builds its pipeline states from state objects it reads back; the red-reporting stub returned whatever was in x0, the object's address, so a blend enable or depth test read as on and a texture builder's width as an address. The getters read the packed state objects (`hle_nvn.c`); the multisample setters the host lacked (alpha-to-coverage dither, raster samples, coverage modulation and coverage-to-colour) are recorded in fields `ms_st` already had. - - **`WaitForAddress` type 3** → Horizon 18 added a 64-bit "wait if equal" arbitration type, which SDK 19 exports as `nn::os::WaitForAddress(const long*, long)`; the host used to fall into its 32-bit "wait if less than" test, so such a wait could return at once or block when it should not. Z-A does not call it, but the SDK has it (`hle_thread.c`). - - **`nvnCommandBufferFenceSync` unimplemented, every `SyncWait` timed out at once** → the game fences once per frame from a command buffer and waits on it. A record (`OP_FENCE_SYNC`) fences the sync at replay with the fence value of the submission segment being encoded, so it signals once everything before it has completed. -- **SDK 4 (Dark Souls Remastered, 4.5.0, the oldest so far), one boot each**: - - **A call through a null pointer in the host's `movie::Decoder::Configure` right after `nnMain` starts the logos** → SDK 4's `Decoder` constructor has no priority argument (`_ZN5movie7DecoderC1ENS_11DecoderTypeENS_19DecoderOutputFormatENS_11DecoderModeE`). The host took over only the four-argument one, so the SDK built its own decoder and the host's `Configure` read that object as its own. Both forms are taken over now (`hle_movie.c`). - - **`FATAL movie: missing SDK symbol _ZNK5movie6Buffer10GetNvnSyncEv`** at the first frame of the native-texture movie (H.264 into the game's RGBA texture, as in Monster Hunter Rise) → SDK 4's `movie::Buffer` carries the texture object but no sync object. The lookup is optional now; the game reads the texture once `GetOutputBuffer` returns, which the CPU write satisfies. - - **Abort in `sm::Initialize` under `nn::friends::GetBlockedUserList`** after the opening movie → SDK 4's overload without the `Uid` (`EPiPNS_7account23NetworkServiceAccountIdEii`). It reports an empty list with success: the game turns a failure into an error for the error applet (`main+0x8ff9f0`). The overload with a `Uid` keeps reporting a failure, which DRAGON QUEST XI S takes as it is. - - **Crash in `lmem::AllocateFromExpHeap` under `ngc::ProfanityFilter::MaskProfanityWordsInText` right after the character name is entered** → the game calls the UTF-16 overload (`EPiPtj`) by name; the host had taken it over only in the filter's vtable, so the SDK's version ran on a filter whose word list the host never loaded. The `unsigned short` and `char16_t` overloads are taken over by name too (`hle_misc.c`). - - **One thread at 100% CPU from boot, with one SVC in 99 s** (`NexusRevolution Socket`, `main+0x92c7c4`, the game's network thread) → it calls `nn::socket::Poll(fds, 0, 33)`: no descriptors and a 33 ms timeout, which on hardware is a 33 ms sleep. The host failed every socket call at once, so the thread spun 11,000 times a second (88,000 hits of a probe in 8 s). `Poll` and `Select` with nothing to wait on (no descriptors, or only negative ones, which POSIX ignores) now wait out their timeout and return 0, in every mangling the host takes over, including SDK 4's `Select(int, fd_set*, …, timeval*)`; an endless timeout returns 0 after 100 ms, since a thread inside a host hook pauses for IL2CPP's collector only once the hook returns (`hle_net.c`, `dm_guest_sleep_ns` in `hle_thread.c`). Guest CPU fell from 1,030 to 60 ms per second. - - **Menus ignore A** → not a host fault: the game maps its buttons by position, B confirms and A goes back (its prompts say so). The first boot shows two notices before the title menu ("Unable to connect to the network", then "Game will start in offline mode"), and a new game first walks through Brightness, Options and Button settings. -- **SDK 4.4.0 (Mario Kart 8 Deluxe 4.0.0, sead / agl / gsys), one boot each**: - - **`[RED] sf: the game connects to service mii:e … mii:u itself`, then an abort** → `nn::mii::Database` keeps an `IDatabaseService` of the mii service at +0 (`Database::Initialize`, `sdk+0x1c5664`), and `AppendMii` / `ShowMiiEdit` start the MiiEdit applet. `hle_mii.c` takes over the public methods: a console with no Mii of its own (an empty database: no index, nothing to update) that still has the six default Miis every console has. The game asks for those with `Get(…, 6, SourceFlag 2)` and never reads the count (`main+0x81e954`): its boot-time list of Mii icons (`main+0x419dc8`) builds a head model for each of the six, and zeroed entries failed the model's `CharInfoAccessor::IsValid` (`CharModel::Initialize` returned 0xd67e, `main+0x8f15b0`) and crashed at `main+0x8f152c`. A CharInfo the host builds is the default Mii's parameters with Switch common colours, a create ID with version 4 and variant bits (fixed per default Mii, so a saved choice finds the same Mii; fresh for `BuildRandom`, which the title's demo race calls for 12 racers) and the nickname "no name"; every field is within `CharInfoRaw::Verify`'s ranges (`sdk+0x1b68c8`). The Mii editor is cancelled (0xE7E, which `ShowMiiEdit` itself treats as success). - - **Abort in `fs::detail::GetFileSystemProxyServiceObject` under `fs::QueryMountSystemDataCacheSize`** (`main+0x8f23bc`) → the game mounts system data 0100000000000802, MiiModel, as `MiiResource:` and reads `NXTexture{Low,Mid}{Linear,SRGB}.dat` and `Shape{Mid,High}.dat` whole; the SDK's Mii rendering is linked into `main` (`main+0x8f1400`–`+0x8fc000`). Firmware content is not the host's to ship, so the host writes a stand-in of the same layout with every part empty (`dm_mii_standin` in `hle_mii.c`, mounted by `hle_fs.c` `MountSystemData`; any other system data answers "target not found"). The layout as the code reads it: texture file +0xc the largest decompressed size of each of 11 part kinds, +0x38 their alignments, from +0x64 one 0x1c-byte entry per part (offset, size, stored size, 4 zlib parameters, then width, height, format, mips); shape file +0xc / +0x3c the same for 12 kinds, from +0x6c one 0x58-byte entry per part (offset, size, stored size, parameters, five vertex attribute offsets and sizes, index offset and byte size, bounding box), then hair (132 × 0x48, +0x12704) and faceline (12 × 0x24, +0x14c24) transforms. A texture of size 0 is skipped (`main+0x8faaf4`); shapes cannot all be absent, because the faceline is drawn without a check (a null buffer read at `main+0xa690d0` under `main+0x8286c4`), so every shape entry points at one shared mesh of one vertex and the triangle (0, 0, 0), 32 bytes of zeros in a stored zlib block. Heads build and draw nothing; bodies and favourite colours show. The largest sizes must not be 0: they size the model's memory pool (`main+0x8f2e6c`), whose alignment is the first one. - - **SIGSEGV in `crypto::detail::CbcModeAes128Helper::DecryptBlocks` with a null destination** (`main+0x8737e8`, right after `aoc:` is mounted) → the add-on content (0100152000023001: `contents.dat` and `version.txt`, the courses are in the update) is decrypted in `AocHeap`, a 0x3130-byte heap that also holds the mount cache. The host answered `QueryMountAddOnContentCacheSize` with 4 KB, and the 0x1400-byte decryption buffer then did not fit. On Horizon the cache is the RomFS's file table as `RomFsFileSystem::GetRequiredWorkingMemorySize` sizes it: both hash tables (4 bytes a bucket, the builder's bucket count) and all directory and file entries (0x18 / 0x20 bytes plus the name, padded to 4), 0x88 bytes for these two files. The host computes that from the folder it mounts (`romfs_work_size` in `hle_fs.c`); `rom` keeps its minimum answer, which is too small rather than too large. - - **Every boot asks to pick a Mii again** (`DM_FS=1`: `UserSave:/userdata.dat` not found although the host save has it) → the game mounts `CommonSave` with `MountDeviceSaveData` and `UserSave` with `MountSaveData(uid)` (`main+0x8b0c84`), two saves on Horizon; the host made device save data the account save (for Animal Crossing: New Horizons, which mounts only device save data), so both names shared one directory. At boot the save manager deletes `CommonSave:/userdata.dat` (`main+0x8b1910`, through its file device's `DeleteFile`, `main+0x633afc`; a probe there with `@*x1+8` shows the name) and commits, and in the shared directory that was the account save's own file. A game whose `main` imports `MountSaveData` too gets its device save as `_device`, a transaction of its own (`hle_fs.c`); of the games on hand only this one imports both, so Animal Crossing's islands stay where they are. - - **No sound: `[RED] audren: no DeviceSink (ID 0x10) in the command list, 0 commands of 0x148 bytes`** → SDK 4's command list has SDK 6's header, but its DeviceSink (`CommandBuffer::GenerateDeviceSinkCommand`, `sdk+0x19e690`) is ID 0xf and 0x130 bytes: device name +0x14, upsample buffer +0x118, input count +0x120, input indices as bytes from +0x124. `capture` takes the one 0x130-byte command with 1 to 6 inputs when there is no 0x148-byte one (`g_sink4` in `hle_audren.c`). Its `AudioRenderSystem::Update` (`sdk+0x19b77c`) locks an `nn::os::MutexType` at +0x1d0, not SDK 16's `SdkMutexType` at +0x1f0; see Audio below. -- **SDK 6 (Fire Emblem: Three Houses, 6.5.0)**: - - **The first loading screen never ends; `DM_FS=1` shows `save:/system` "not found" and no `fs mount save`** → in SDK 6 `account::OpenPreselectedUser` returns a Result, and the game skips its save mount when it is not 0 (`main+0x53b7e0`). The host's hook returned void and left the handle pointer in w0. It returns 0 now; SDK 7 and later callers ignore it. - - **No sound; red `audren: command 1 in the command list is malformed (magic 0 size 0)`** → SDK 6's DSP command header is +0x04 size (s16), +0x06 command ID (s16), +0x10 enabled, where SDK 8–20 have +0x04 enabled, +0x05 ID, +0x06 size (`CommandListProcessor::Process`, `sdk+0xaaa00`); its DeviceSink is ID 0x10. `capture` in `hle_audren.c` takes the layout whose walk finds the 0xcafebabe magic on every command of the first full list. - - **Shaders missing from the map that no game file holds** → KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (`glslcCompile` from `main+0x9c89f8`; the sources are strings in `main`). `DM_GLSLC_CAPTURE=` saves every compile's output (`hle_glslc.c`; the compile object's first word points at {status, output}), and `shader_split.py` reads `/glslc_capture` (the NSP script makes it for any game whose `main` holds GLSL sources and imports `glslcCompile`). The outputs are identical on every run. -- **SDK 6.4.0 (Digimon Story Cyber Sleuth: Complete Edition, Media.Vision's MVGL engine on OpenGL), one boot each**: - - **`UNIMPL eglGetDisplay`, then `Break` right after `nv::InitializeGraphics` and `vi::OpenDefaultDisplay`** (`main+0x5ed9d0`) → the game renders with NVIDIA's desktop OpenGL through EGL; its NVN device (made just before) serves only a helper beside the movie player. The host's GL layer (`dm/gfx/gl/hle_gl.m`) answers the EGL imports and `eglGetProcAddress` ([04](04-graphics.md)). - - **SIGBUS in `__vfprintf` under `aoc_load`, from `h_aoc_Count`** (a store to `0x…67ff0`, right below a page of the guest thread's stack) → the first add-on content query lists `DM_AOC_DIR` and logs, a few KB of stack, and the game asks from a thread with little left. `aoc::CountAddOnContent` and `ListAddOnContent` run on the big host stack (`HLE_FORCE_BIG`, `hle_misc.c`). -- **SDK 18.3.2 (Balatro, LÖVE 11.4 on OpenGL with SDL and LuaJIT), one boot each**: - - **The NSP script's provisional profile said Vulkan, and the capture boot crashed in the GL layer** → `main` imports `vkGetInstanceProcAddr` for SDL's Vulkan loader, which nothing uses; the game draws with EGL and GL. `profile_guess.py` now leaves that import out of its Vulkan count ([04](04-graphics.md)). - - **A jump to address 0 right after `glGetFloatv(0xb12)`** (`lr=main+0x2d58c4`) → a GL function the layer did not have (here `glVertexAttrib4fv`, LÖVE's default colours). `DM_GL_STUBS=1` hands out a stub per missing name that reports its first call; `loader_census.py` lists what the code calls before the first boot. - - **A crash in `main+0x339c00` a second or two into the title, in love-nx's video module teardown** (`love::video::theora::Video` → `~Worker`; its thread released the worker that the closing module deletes too) → the Lua state was closing: on the Switch, Balatro's `love.errhand` returns at once, so any Lua error quits. A `print` at the top of `love.errhand` in a copy of the RomFS (`DM_ROMFS=`) gave the error: `Shader uniform 'distortion_fac' does not exist` (the capture run had no shader library, so every program linked with no uniforms), then `bad argument #2 to 'send' (number expected, got boolean)` (a `bool` uniform reported as `GL_UNSIGNED_INT`). Probing the destructors (`DM_PROBE`) and logging every object the Lua GC released (`DM_PROBE=main:0x18b920@x0+0`: the module proxies all at once) showed it was a shutdown, not a premature collection. - - **Silent** (the log has `audren: OpenAudioRenderer (no event)` and no `render thread started`) → see [Audio](#audio). -- **SDK 8 (Link's Awakening, 8.2.0)**: - - **Abort in `os::WaitSystemEvent` under `oe::PopNotificationMessage`** → the blocking pop waits on the system event `oe::Initialize` would have created, and the host's `Initialize` is a no-op. The host now implements it on its notification queue (a condition variable, then the `TryPop` path). - - **SIGSEGV in `vi::SetLayerCrop`** → the SDK's version dereferences vi service objects the host never made; `SetLayerCrop`, `DestroyLayer`, `CloseDisplay` and `vi::Finalize` are host no-ops. - - **Abort with Result 0x499 in `audio::GetAudioRendererWorkBufferSize`** → SDK 8's `AudioRendererParameter` is one word longer than SDK 16's: its revision sits at +0x38. The host copies the game's parameter before switching it to CPU rendering with manual execution, and copied only 0x38 bytes, so `CheckValidRevision` read garbage. It copies 0x40 now. - - **No music and no sound effects, only the movies' sound** (the log repeats `audren: first-frame commands …` every frame and never prints `DeviceSink inputs`) → SDK 8 numbers the DSP commands differently: its command list goes 4×6, 7×24, 11×2, 16×1, 21×1 where SDK 20's goes 7×6, 11×24, 15×2, 20×1, 25×1, and the output command (DeviceSink, the same 0x148-byte layout with device name `MainAudioOut`, six inputs) is 0x10, not SDK 16's 0x13. The host never found it and played silence. When the expected ID is missing from the first full command list, `capture` in `hle_audren.c` now takes the one 0x148-byte command with 1 to 6 inputs as the sink and logs the ID it adopted. - - **The game never reads the controller** (the title's "Press L+R" never sees a press; no `GetNpadState` call at all) → it waits for the event bound by `hid::BindNpadStyleSetUpdateEvent` before reading. The host now signals it once when it is bound for a connected controller, and again when a player's controller connects or goes away. - - **`nvnWindowGetPresentInterval`, `nvnCommandBufferSetCopyImageStride`** → implemented (the latter is the layer stride of texture ↔ buffer copies). - - **Movie sound** → the decoder is asked for type 5 (Vorbis) with output format 1 (PCM16); see the movie entries above and `hle_movie.c`. -- **SDK 8.2.1 (Catherine: Full Body, Atlus's GFD engine, CRI middleware), one boot each**: - - **A null read at `main+0x3316dc` in `nnMain`, right after the window setup** (the program object's interface pointers at +0xd0 / +0xd8 both null) → the game compiles two built-in programs at boot with its own GLSLC (`subsdk0`, the same build as Link's Awakening's), and first checks that GLSLC's GPU code version against `nvnDeviceGetInteger` items 0x2E–0x31 (the driver's supported range, `main+0x330dcc`). The host reported Super Mario Odyssey's SDK 7 driver range, 1.5–1.14; this GLSLC writes 1.15, so nothing was compiled and the first uniform lookup read the empty program. Every SDK's driver runs the code of the GLSLC in the same SDK, so the host now reports at least the version the game's own `glslcGetVersion` returns (`hle_glslc.c` `dm_glslc_gpu_code`, `hle_nvn.c`). The two outputs are captured on the Mac as Three Houses' are (SDK 6, above). - - **Abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodeCount()`** (`main+0x117590`: it compares `LanguageCode::Make(0 … count − 1)` with `GetDesiredLanguage`'s code) → the count of the host's language table, beside Paper Mario's `GetAvailableLanguageCodes` (`hle_sdk16.c`). The other imports no hook names run in `sdk` (the `LanguageCode` comparison, `ClockSnapshot`'s constructor and getters, fibers, `NumberLineAllocator`). - - **The memory pool for the game's 96 MB GPU heap has no storage (`MemoryPoolInitialize` red line), then a null read at `main+0x2eb954`** → `[mem] SetHeapSize … + modules 0x5c00000 > app pool`: the rewritten modules take 92 MB, so `SetHeapSize(pool − 64 MB)` failed and the game ran on a 32 MB heap, as Age of Calamity did (Memory, above). The profile's `mem_used` is 96 MB, the NSP script's guess after the rewrite. - - **A read far outside guest memory at `main+0x344f74` on the first movie picture** (CRI Sofdec2's NV12 → RGB conversion; its H.264 goes through `movie::Decoder`, mode 1, output format 4) → the player reads `nv12-y-stride`, `nv12-y-offset`, `nv12-uv-stride` and `nv12-uv-offset` from the output format and checks none of the results (`main+0x34e694`); the host set no `nv12-uv-stride`, so the chroma stride was stack garbage. It is the luma stride now, as for any NV12 picture (`hle_movie.c` `md_format`). -- **Monster Hunter Rise's movies** → H.264 (type 1) into the client's NVN texture (output 3 = RGBA, mode 2), with AAC 5.1 sound (type 4, PCM16). VideoToolbox decodes to BGRA for that mode (`mavc_set_rgba`), and AAC goes through AudioToolbox's converter with an MPEG-4 ES descriptor built from the AudioSpecificConfig in `csd-0` (as FFmpeg's AudioToolbox decoder does). The game ends a stream with time −1 and the whole 4 MB input buffer as its length: that packet carries no data (decoding it gave VideoToolbox −12909 and AudioToolbox −50). It also deletes its decoders (`D0`), so the host frees the object with the game's `operator delete` after its own state. -- **A native-texture movie shows as a strip of small grey copies at the top of the screen** (Absolum's H.264 logos, FNA3D's `nx_video_player.cpp`: type 1, output 3, mode 2) → the texture held NV12 bytes. In RGBA8, 405 rows of 1920 pixels are exactly a 1920×1080 NV12 frame, rows 0–270 the luma at video black (16) and rows 270–405 the chroma (128). This game's MP4 demuxer hands the parameter sets to `Configure` (`csd-0` / `csd-1`), which made the H.264 decoder there without switching it to RGBA, and the first packet then found it made. Both paths now go through `md_avc_open` (`hle_movie.c`). Monster Hunter Rise's movies never brought parameter sets to `Configure`. -- **A movie stays black after its first picture, the sound never starts, and the game keeps presenting frames** (Monster Train 2's 90 s intro, `StreamingAssets/intro_fmv.mp4`: H.264 High with 2 reorder frames and AAC interleaved packet by packet, played by Unity 2022.3's movie player: `movie::Extractor` in `subsdk0` feeds both decoders from one thread in file order) → the player takes every output a decoder offers and starts playing only once it holds about half a second of audio. The host then gave each decoder 8 output slots (before DRAGON QUEST XI S's 32 for audio, above) and takes input only while one is free (as MediaCodec does). `DM_MOVIE_TRACE=1` (each line now names the decoder): Unity held all 8 audio outputs, the audio decoder kept the next 4 packets and handed out no input index, the video packets behind them in the file never came, and the video decoder had given out one picture (the host releases H.264 pictures once more than 4 are queued). Slot counts per decoder kind, measured: 8 audio / 8 video and 8 audio / 32 video hang; 64 audio / 8 video plays, starting with 27 audio and 8 video outputs held (peak 30 audio). 32 audio slots hold that peak with two to spare, so audio decoders now have 64 (`MD_NOUT`; a PCM slot is 128 KB, allocated when first used), and video keeps 8 (`MD_NOUT_VIDEO`: a 1080p picture is 3 MB, 8 MB as RGBA). The game's A, B and + do not skip this intro. -- **A movie shows as a black screen with the loading icon, one picture every few seconds, while its sound decoder keeps up** (Animal Crossing: New Horizons' in-flight movie `Movie/OpeningMovie.webm`, VP9 1080p and Vorbis through `movie::Extractor`) → this player also holds every output it is offered (16 of 16 video, 59 of 64 audio), so how far its extractor can get ahead of the picture on screen is the video slots plus the 4 queued packets. With 8 video slots that was 0.4 s, and the audio behind the next video packet, whose clock the player follows, ran dry: 86 pictures in 25 s. With 16 or 32 slots all 1,960 play at 30 fps. A video decoder now has 16 (`MD_NOUT_VIDEO`); a slot is allocated when first used, so the 8 more cost 24 MB of NV12 at 1080p only in a player that holds them all. -- **The game never leaves a movie that has played to its end; the last picture stays on screen** (AI: THE SOMNIUM FILES, Unity 2017.4: the 6 s Spike Chunsoft logo `StreamingAssets/Movie/al_titel_cl_sc_1k_30.mp4`, H.264 High, before the title) → `DM_MOVIE_TRACE=1`: the end-of-stream packet came with H.264 pictures still in the reorder queue, its output went out as a black picture without the end-of-stream flag, and the queued pictures were to follow one per free output slot (the drain). Unity held all 16 video slots by then. `ReleaseOutputBufferIndex` decoded again only when packets were queued, and none were, so the drain never went on: no flag, and the game waited for the movie's end for good. A released slot now goes on with a drain too, and the end-of-stream packet's own output takes the first queued picture (`jflush`), so no black picture shows before a movie's last frames (`hle_movie.c`). The audio decoder was never affected: AAC has no reorder queue. -- **A crash in the game's heap free (`sead::ExpHeap`) on a movie thread, a second into the first movie, at the same address every boot** (Mario vs. Donkey Kong 1.0.1, SDK 15.3.0: its opening movie `Movie/Cutscene01.mp4`, H.264 and AAC through `movie::Extractor`; `pc=main+0x42f980`, a free-list unlink through a link of 0x3f, `addr=0x47`) → `DM_PROBE_WATCH` on the link broke the movie's file reads (the page also held a 64 KB read buffer, and `read()` into a protected page fails with EFAULT), so the movie stopped and nothing crashed; `DM_PROBE_POLL=
+8=3f` (a snapshot only at the change to that value) showed it being written right after the block was relinked. The values around it were pairs {u32 output index, u32 flags, u64 time}: the game's movie player keeps the outputs it holds in a ring of 32 such entries per track (`main+0xe93d4c`: 0x200 bytes) and pushes with no bounds check (`main+0xe94140`), and the host's audio decoder offered 64 outputs, so the 33rd outstanding one, index 63, went past the ring. The console's AAC decoder cannot offer more than 32, or the game would overflow there too: AAC decoders now have 32 outputs (`MD_NOUT_AAC`; Monster Train 2 peaked at 30 held), Vorbis keeps 64. -- **The movie then froze on its first second, its sound decoder full and the game taking no more audio** (the same movie) → the player opens an AudioOut of its own (`main+0xe92c2c`), sizes its buffers by the audio decoder's `GetOutputBufferSize`, decodes straight into them (`SetDataAndCapacity` with that size) and appends each whole, advancing its audio clock by size / 4 samples (`main+0xe92a74`). The host answered 128 KB, a bound for any Vorbis packet, so every 4 KB AAC frame played as 0.68 s and the clock crawled (a temporary log of `AppendAudioOutBuffer` showed 131,072-byte buffers). An AAC decoder now reports one frame: 1024 samples per channel, 2048 with SBR (object type 5 or 29 in the AudioSpecificConfig), which is what each output holds; Vorbis keeps the bound, its packets vary in length. -- **One guest thread at about 1000 ms/s on the iPhone only, the phone hot, CPU energy 5–7 W** (Monster Hunter Rise's `MessageSuperviseThread`, priority 32, in the `[nvn] threads` lines) → the thread waits on `oe::GetNotificationMessageEvent()` with `os::WaitAny`, which does not clear an event, then pops notifications until the queue is empty. The host only ever signalled its notification event, so after the first focus change (iOS changes focus at launch; the Mac does not) it stayed set and the loop never blocked again. The event now follows the queue as the am service's does: set while a notification is queued, cleared when the last one is popped (`hle_oe.c` `mq_ev_sync`, `hle_clear_system_event`). Mac repro: `DM_OE_SCRIPT=focus=2@5,focus=1@6` (the script also starts at the game's first `GetNotificationMessageEvent`, since such a game pops only once the event fires): before 995 ms/s, CPU 109%, 6.97 W; after, the thread is idle and CPU energy is 0.05 W. -- **The game freezes after switching apps and back, picture still, sound on** (Super Mario Galaxy 2, two causes; Mac repro `DM_OE_SCRIPT=bg=1@20,bg=0@25`): - - **The game's own pause deadlocks** (the main loop waits in its pause handshake, `main+0x5d10d4` / `main+0x5cea68`) → the game set focus handling mode 3. The SDK (`oe::SetFocusHandlingMode`, SDK 18 `sdk+0x3c5770`) turns modes 0 and 3 into "no focus notifications, the system suspends the game out of focus"; only 1 and 2 notify. The host sent message 15 anyway, and Galaxy 2's Wii layer ran a pause the console never triggers. `hle_oe.c` now queues 15 only under modes 1 and 2 (or when no mode was set). Modes seen: 1 Monster Hunter Rise; 2 Wonder, Breath of the Wild, Tears of the Kingdom, Link's Awakening, Cuphead; 3 Galaxy, Galaxy 2, 3D World, Odyssey. - - **One guest thread at 100% after the return, in a tight loop** (`main+0x9abd0`, `x10 = 0`) → the frame pacing measured a frame period of 0 (`main+0x5926a8`: the clock difference between two frames). The host freezes the guest clock in the background but its threads went on, so two frames read the same time. Guest reads of the system counter (the `cntpct` gate, `svc GetSystemTick`) now wait while the app is in the background (`hle_os.c` `guest_clock_read_wait`); host work and the keyboard freeze keep the frozen clock. -- **SDK 9.6 (Hyrule Warriors: Age of Calamity)**: - - **Null read in `oe::InitializeCopyrightFrameBuffer`** (right after the window setup) → the copyright frame buffer is an overlay the system draws only into screenshots and recordings; the SDK hands it to am as transfer memory. `InitializeCopyrightFrameBuffer`, `SetCopyrightImage` and `SetCopyrightVisibility` are host no-ops (`hle_oe.c`). - - **`HardwareOpusMultiStreamDecoder`** → the game builds it with the imported constructor and then calls it through its vtable. Its 12 slots match the SDK's software `OpusMultiStreamDecoder` one by one, so the constructor hook builds the software decoder in place (0x38 of the 0x48 bytes), as for the single-stream `HardwareOpusDecoder`; `GetHardwareOpusMultiStreamDecoderWorkBufferSize` returns the software size (`hle_codec.c`). Both destructors are a bare `ret`. -- **SDK 21 (Rhythm Heaven Groove, 21.4.0), one boot each**: - - **Abort in `nn::sm::Initialize` under `nn::pl::sharedresource::GetPlatformSharedResourceManager`** (first thing in `nnMain`) → the game asks for every shared font at once with `pl::GetSharedFont(int *, SharedFontData *, int, LanguageCode)` (`sdk+0x40f270`), not by type. `SharedFontData` is 0x18 bytes {u32 type, address, size}; the SDK adds the pl service's offsets, which point past each font's 8-byte BFTTF header, to the shared memory base, and the game hands each address straight to `ScalableFontEngine::LoadFont` (`main+0x501ff0`). The hook returns the six types in order with the stand-in TTFs past the header, whatever the profile's `shared_font` (`hle_sdk16.c`). - - **SIGSEGV in `vi::GetDefaultDisplayPanelIdentifier`** (`main+0xa9a4`, before the window) → the SDK's version calls through the vi service object the host never made. The game reads a u32 kind and, from kind 2, 0x14 more bytes; anything else makes it use zeros. The host answers an all-zero identifier (`hle_vi.c`). - - **No sound; red `audren: command 0 in the command list is malformed (first word 0x181901 size 0)`** → SDK 21's DSP command header has no magic: +0x00 enabled (bit 0), +0x01 command ID, +0x02 size (s16), +0x04 estimated time, +0x08 node ID (`CommandListProcessor::Process`, `sdk+0x24fd50`), and every command body moved 8 bytes forward. DeviceSink is still ID 0x14 but 0x140 bytes: input count +0x128, input indices +0x12c, upsample buffer +0x120 (written at `sdk+0x23a7b0`). `AudioRenderSystem` keeps SDK 20's offsets (update lock +0x2d0). `capture` tries this layout when neither magic walk works, bounded by the generated list's length; its first lists hold 4 commands or fewer, so the layout is now chosen on the first non-empty list. - - **Not a host bug: the TV latency calibration ("press A when you hear the fourth beat") ignores A.** Docked, the first boot runs it (scene script `main+0x4c0820`). It starts listening only `0x3c0` frames (16 s) after the prompt, through the active controller (`main+0x22b0` id `0x3ed`, virtual button 4 = A), and passes when about ten presses keep a steady offset within −5 to +13 frames of the cue (evaluation `main+0x2948c0`; offset = cue ticks at `+0x34` / about 17 ticks per frame, about 480 per beat). Scripted presses at fixed seconds drift off the beat; use frame-timed `DM_KEYS` at the bar period (117.12 frames at its 123 BPM) and read each press's offset with `DM_PROBE=main:2951f0@34`. A passed calibration is saved per TV panel identity (the host's all-zero one included) and not asked again. The profile plays it handheld with Boost (`op_mode handheld`, `perf_mode boost`), which skips the TV setup and keeps the 1080p draw size. - - **Black screen with every geometry right (`DM_NVN_HILITE` shows each sprite and glyph in place)** → its sprite shader discards a fragment whose alpha is 0 and every texture read 0: the textures sit in `CPU_NO_ACCESS` pools made on memory the files were read into (the initial content of such pools, [04](04-graphics.md)). -- **SDK 10.4.1 (The Great Ace Attorney Chronicles, Capcom's MT Framework)**: the first boot crashed in `oe::DisableAlbumImageTakenNotification` (a null read at `sdk+0x338070`, the first call in `nnMain`, `main+0x6b188c`: the SDK asks am through the applet proxy the host never made). The import list against the hooks (`nso_syms.py`, then `main`'s undefined `nn::` symbols no `HLE_FORCE` names) then gave the rest before they were met: `oe::IsInStoreDemoModeEnabled` (no), `vi::ListDisplays` (one display, "Default"; the game lists 10 into a stack buffer and does not read them, `main+0x808f08`), `fs::RenameDirectory` (as `RenameFile`), `account::StoreSaveDataThumbnailImage` / `DeleteSaveDataThumbnailImage` (nothing kept), the `ec::ShowShop…` overloads without an `ApplicationId` (the calling application's own pages), `friends::StartFacedFriendRequest` / `StartSendingFriendRequest` (fail), and `ngc::ProfanityFilter::MaskProfanityWordsInText(int *, char *, uint32_t)`, `SetMaskMode` and `Finalize`, which this game calls by name rather than through the vtable. The touch-screen states (`GetTouchScreenState<16>`) were already hooked by a macro the import table does not see. Two NVN calls were red stubs: `nvnDeviceSetIntermediateShaderCache` and `nvnCommandBufferDiscardColor` (a discarded target is undefined, so keeping its content is correct). After that it runs its frame loop at 30 fps; the black screen it showed was its textures, never uploaded ([04](04-graphics.md), Seen in The Great Ace Attorney Chronicles). -- **SDK 11.4.3 (Big Brain Academy: Brain vs. Brain, NintendoWare Bezel Engine)**: - - **`nn::diag` abort in `StandardAllocator::Initialize(void *, size_t, bool)` from `nninitStartup`** (`sdk+0x2f98e4` ← `main+0x5b69e0`) → the game's `nninitStartup` (`main+0x5b6984`) asks `os::IsVirtualAddressMemoryEnabled()`: when true it gives StandardAllocator a 60 GB address region (`0xf00000000`), when false it sets a heap of pool − used (2 MB aligned) and hands that over. The SDK's answer is `GetInfo(TotalSystemResourceSize) != 0` (`VammManager::IsVirtualAddressMemoryEnabled`, `sdk+0x34e074`), and the host reported a 32 MB system resource to every game, while this game's NPDM declares none. The profile key `sys_resource` reports the NPDM's size (0 here, so the heap path, as on hardware); the survey writes it for every new game, and profiles without it keep the 32 MB answer. - - **Abort in `sf::hipc::InitializeHipcServiceResolution` under `codec::GetHardwareOpusDecoderWorkBufferSize(int, int)`** (`main+0x31f8e8`, early in `nnMain`) → SDK 11's form without the option argument; the host had only the three-argument one. - - **From the import list before the next boot** (`main`'s undefined `nn::` symbols no hook names; [02](02-survey.md)): `fs::EnableIndividualFileDataCache` / `DisableIndividualFileDataCache`, `account::LoadCachedNintendoAccountInfo` (fails: no Nintendo Account; the game then skips the birthday and country getters), the IR camera (`irsensor`: every camera unconnected, processors cannot run; Bezel's input layer makes one per controller and asks its status, `main+0x497f84`), `ldn::GetNetworkConfig`, and SDK 11's socket overloads with plain `sockaddr` / `int` arguments (`Connect`, `Accept`, `GetSockName`, `Get` / `SetSockOpt`, `Fcntl`, `Recv`, `Send`). The survey's table counts only names given to `HLE` / `HLE_FORCE` directly: hooks made through macros (`SKF` and `SSF` in `hle_net.c`, `TOUCH(N)` in `hle_hid.c`) show as missing. - - **The process dies when an activity starts, 2 runs in 5, with no `CRASH` line** (the run ends with exit 137, SIGKILL; `~/Library/Logs/DiagnosticReports/dmhost-*.ips` has `EXC_BREAKPOINT` in `_os_unfair_lock_corruption_abort` ← `pthread_mutex_destroy` ← `md_free`, the movie decoder's destructor) → each activity's preview is an H.264 movie (`movie/Movie/*.mp4`) the game loops with Flush and Start, from four threads at once (`DM_MOVIE_TRACE=1` logs every decoder call with its thread). Its event thread acquired an input index while the control thread's Flush waited for the packet being decoded; the flush then returned that index to the free list, it was handed out a second time, two packets shared one buffer (VideoToolbox -12909 on the overwritten one, in exactly the runs that died) and a fifth pending packet wrote past `pend[]` into the decoder's `npend` and `dmu`. A send now claims an index a flush returned, and `pend[]` cannot overflow. Handing such an index out last was not enough (with three hosts running at once one run still sent an index twice): an index the client held across a flush is now withheld for 250 ms unless the client sends it first, and an acquire refused only for that announces input when the first one comes back, so a client that abandons its indices, as MediaCodec's rules expect, loses none (`hle_movie.c`). The previews also flashed black at the end of every loop: the game reads the buffers of the end-of-stream output (no picture) and of an output it acquired before the flush, and the host painted its texture black for both. A flush now keeps each slot's last picture, and in native-texture mode an output with no picture leaves the texture as it was, as the SDK's GPU consumer has nothing to draw. -- **SDK 12.3.3 (Taiko Risshiden V DX, Koei Tecmo's KTGL)**: - - **Null read at `sdk+0x362b8` (`apm::SetCpuOverclockEnabled`) in the first calls of `nnMain`** (`main+0x2e1e4c`) → `oe::SetCpuOverclockEnabled(bool)` goes through the apm session the host never opened, as `oe::IsCpuOverclockEnabled` did for Monster Hunter Rise. The host keeps the flag and the query reports it (`hle_oe.c`); it does not scale clocks. - - **Taps in handheld mode do nothing, though the game reads the touch screen** (the title's PRESS ANY BUTTON, the menus' cards) → its touch handler (`main+0x42f1ac`) uses `GetTouchScreenState` only to track the finger and takes its taps, pans and swipes from `hid::GetGestureStates` (it acts on types 5–9), which the host answered with no entries. The host now recognizes gestures from the same touch points, by yuzu's rules plus an 8 px slop before a touch counts as moving (`hle_touch.c` `gesture_sample`, unit test `t_gesture`; only in handheld mode, as touches): Touch, Press after 0.5 s, Pan, Pinch / Rotate with two fingers, Cancel on a change in the finger count, then Tap (IsDoubleTap within 0.35 s), Swipe above 400 px/s or Complete, and Idle. `DM_TOUCH_LOG=1` logs each gesture entry, and a `DM_CTL` line `drag [seconds]` moves a finger for testing pans and swipes on the Mac. Every menu, the cards, the map's drag and flick worked by touch with it. -- **SDK 13.3.2 (Bayonetta 3, PlatinumGames' engine)**: - - **Null read at `main+0xb8462c` right after the first two textures, called from `main+0x670eb0`** → not the fault itself: `nnMain`'s start-up (`main+0x6705c0`) returns false when any of its 30 steps fails, the game then skips its main loop and goes straight to shutdown, and the shutdown walks a pool (`main+0x4efce70`) whose initialization (`main+0xb840d0`) came after the failed step. `DM_PROBE` on each step's entry (lldb was unavailable: developer mode off) showed the last one entered, `main+0x69c698`, the built-in shaders of `shader2.dat` (`hwfillcolor`, `hwshaderpf`, …), and probes inside it the step that failed: creating a shader from its object (`main+0xd0d998`). The engine checks each shader object's GPU code version (byte 6 major, byte 7 minor of the object) against the driver's range (`nvnDeviceGetInteger` 0x2E–0x31: newest major, oldest major, newest minor, oldest minor) and refuses newer code; the host reported 1.14, the table measured on Super Mario Odyssey's SDK 7 driver, and `hwfillcolor.vso` is 1.15 (the game's objects run from 1.9 to 1.16). A driver runs the code of its own SDK's GLSLC, so the newest version the host reports now follows the SDK: 1.15 from SDK 8 (Catherine: Full Body's GLSLC 8.2.1), 1.16 from SDK 13 (Bayonetta 3's newest objects), and at least the version of a GLSLC the game carries, as before (`hle_nvn.c`). A wider range only lets more shaders pass such checks. - - Two NVN getters were red stubs: `nvnWindowGetCrop` (read back every few frames; before any `SetCrop` it is the whole first window texture) and `nvnSamplerBuilderGetLodClamp`. - - The first boot sat on the loading clock for about a minute, the CPU idle, with every file read done after 12 s: that run compiled its Metal libraries and pipelines on first use (cold cache). With a warm cache the title is up after about 15 s; what looked like a 40-second wait in the read log (`DM_FSLOG`) was the title screen itself, which reads nothing. Screenshots every few seconds (`ctl.sh snap` from a script) told the two apart. -- **SDK 3.6.0 (Bayonetta 2 and Bayonetta, PlatinumGames' engine in its Wii U form)**: the oldest SDK so far booted to its frame loop with the provisional profile as generated and no host change: the services, the NVN API version (53, as for every SDK up to 10), the audio output and the heap all answered as for SDK 4. What Bayonetta 2 needed was graphics (a depth clear folded into a smaller pass, single-sample `Downsample`; [04](04-graphics.md)) and a reader for its CRI CPK archives ([01](01-extract.md)). Bayonetta, on the same `sdk` and `subsdk0` builds, then needed no change at all: its profile is Bayonetta 2's with its own build row and tables, and the route from the boot to the first fight logged no red line. -- **Code inherited from another title calls fixed addresses** → harmless on the build it was written for, silent memory corruption on any other. d2r-mobile's 3D render scale read a renderer pointer at `main+0x31e0848` and called `main+0x8582f0`; in Mario v0 that is the middle of a static initializer and only did nothing because the pointer happened to read 0. Such code checks `module_t.build_id` now (written into `fixups.tsv` by the rewriter) and stays off on other builds. -- **Device / subsystem setup returns quietly and the next stage never starts** → a version or capability check failed (NVN API version, a feature query) → compare what the game queries with what the host answers. -- **Font engine runs on garbage** → the system shared font must look like BFTTF: the SDK calls `ScalableFontEngineHelper::Decode(ptr, size)`, which XOR-decodes in place unless the first word is the "already decoded" marker (LE `0x18029a7f`) → serve `marker + bswap(size ^ 0x49621806) + TTF` (`hle_sdk16.c`). That is the address only for a game on SDK 11 or later whose `main` imports `Decode` (Wonder, Breath of the Wild, Tears of the Kingdom, Pokémon Legends: Z-A); every other game gets the TTF itself (next entry). -- **Japanese, Korean or Chinese text draws nothing in a Unity game, Latin text is fine** (Patrick's Parabox, SDK 13: its language menu showed three blank lines) → the game's one font (Jost, a Unity dynamic font) falls back to the OS fonts for those glyphs, and Unity's Switch player loads them from the shared fonts: `main+0x2390b0` maps the names `Switch_Font_STD`, `Switch_Font_CN_Simplified`, `Switch_Font_CN_Extended`, `Switch_Font_CN_Traditional`, `Switch_Font_KR` and `Switch_Font_NIN_Extended` to types 0–5 and hands `GetSharedFontAddress` / `GetSharedFontSize` straight to FreeType's `FT_New_Memory_Face` (`main+0x23b2e0`), as libnx homebrew does with the same pl offsets. The host gave every SDK 11+ game the 8-byte BFTTF header in front, so every face failed to open. The SDK's `GetSharedFontAddress` is the shared memory base plus the pl service's offset, with no adjustment, and that offset points past the header (yuzu's pl service: "font offset and size do not account for the header"); Captain Toad (SDK 8) records its shared fonts as plain TTFs and calls `Decode` only for its BFTTF formats. So the address is the TTF unless the game is on SDK 11 or later and its `main` imports `nn::fontll::ScalableFontEngineHelper::Decode` (`font_skip` in `hle_sdk16.c`, `dm_main_imports` in `loader.c`). Of the games on hand, the ones that call `GetSharedFontAddress` without importing `Decode` are all Unity players (Engage, Ace Attorney Investigations Collection, Brain Memory Training Academy, Patrick's Parabox). To see which fonts a Unity game asks for, probe its name mapper's entry (`DM_PROBE=main:2390b0` on Patrick's Parabox 1.0.1; the function is the one `main` caller of `GetSharedFontLoadState`). -- **File-not-found noise** → games probe many optional paths (debug parameters, other-language voice files); with `DM_FS=1` these show as `fs … not found`. Only a missing file followed by a failure matters. - -- **A buffer is created with a null pool right after a scene change** → NVN *virtual* memory pools (flag 0x200) have no storage until `nvnMemoryPoolMapVirtual` maps physical pool pages in; if the host returns failure, the game treats the pool as unusable. Mario maps its single 128 MB virtual pool onto one physical pool in one request, so the host aliases it (`hle_nvn.c`). True sparse mappings are still unimplemented. -- **`[RED] MapVirtual: non-contiguous mapping (true sparse) not implemented`** (Monster Hunter Rise) → RE Engine's GPU heap is one 64 GB virtual pool built from separately allocated 4 MB physical pools, each mapped at a virtual offset of its own (45 at boot, over the first 180 MB). A virtual pool larger than the application pool cannot be one alias of a physical pool, so it gets an address range of its own at `MemoryPoolInitialize` (reserved, inaccessible: 64 GB on the Mac, at most 1 GB on iOS), and each mapping request `vm_remap`s the physical pages over its part of the range, so virtual addresses read and write the same memory, as the GPU's page table does. Each run of mapped pages is registered with the replay like a pool of its own and rebuilt when a request joins or splits runs (`vpool_*` in `hle_nvn.c`). Mario's 128 MB pool, mapped whole onto one physical pool, keeps the alias. The log reports the mapped high-water mark per 128 MB. -- **`CRASH sig=10` in `memmove` from `pkg_upload` → `nvn_deswizzle`, at an address 32 GB into a sparse virtual pool** (Monster Hunter Rise) → the game's streamed textures (flag 0x20, packaged layout) live 32 GB into its virtual pool; it writes their tiles on the CPU into 4 MB physical pools and then maps them, with no GPU copy. NVN maps at 4 KB and the host aliases at its 16 KB page, so a request whose offsets differ modulo 16 KB cannot be aliased: the ASTC 6x6 textures' mip tail starts 0x81000 into the texture. Every request also goes into a 4 KB page table (`vpt_*` in `hle_nvn.c`: the physical page behind each virtual page and the generation of the request that set it). The replay reads a sparse pool's textures through it (`nvn_sparse_copy`, unmapped pages as 0) and uploads one again only when a mapping in its range changed or the CPU wrote a physical page behind it (`nvn_sparse_since`; the physical pools are dirty-tracked). Before that the untracked alias counted as written on every look: 32 textures, 18 MB, were uploaded again every frame, and those with unaliased tails came out magenta. Requests beyond the reserved range (iOS reserves 1 GB: the buffers stay under 0x20100000 through the opening) are kept in the table only, like unaligned ones: textures can read them, the GPU cannot address them. -- **A null-pool buffer even with virtual pools working** → the game's own heap ran out. Trace it with probes ([05](05-debugging.md)): the gsys GPU allocator at `main+0x4290c0` and the heap's `tryAlloc` (vtable slot +0x30) show which heap, its size and every allocation into it; the `!xN` heap walk shows the free and used blocks at the moment of failure. - - Mario: each resource gets its own `sead::ExpHeap` sized *resource-size-table value + 0x1008* (`main+0x1e603c` → `ExpHeap::create` `main+0x4aac38`). The table values are Nintendo's measured peaks with almost no margin (Yoshi's animation file uses 99 % of its budget). `Model/NpcCommonerA.bfres` in the opening course came up about 0x70 bytes short: its last GPU chunk (0x9000 bytes, 4 KB aligned) fit, but the chunk's 0x598-byte descriptor did not. Breath of the Wild's identical failure came from sizes the game takes from the host (below); whether Mario's chunk is sized the same way is not checked yet. - - Fix: a registered code cut gives every resource heap +16 KB (`CUTS` in the profile's `patches.py`, keyed by the module build ID, original bytes verified). Per-title code patches go there, with the reason; those of Nintendo's `sdk` modules in `dm/tools/sdk_patches.py`. - - Breath of the Wild (`main+0x155a7a4` writes through a null `operator new` at the world load, about 1 run in 3): each model's heap is its table entry + 0x1008, the file goes in 4 KB aligned, and the setup (`main+0x155a2ac`) takes the rest as a frame heap (vtable `main+0x1d4f950`, head `+0xe0`, tail `+0xe8`). Its use is the same in every run; only the file's alignment pad varies (0 to 0xff8), and the entries leave room for any pad, about 0xd00 to spare on most models. Three models were a 4 KB page short, so the loads with a large pad failed. The cause: the game sizes the model's prerecorded commands by recording each mesh's draw and each shape's per-pass state into a scratch command buffer and summing `GetCommandMemoryUsed` (`main+0xf9aeb4`, `main+0xf9bb54`), and our records were larger than the driver's (draws carried the indirect and multi-draw fields, every recording began with a BEGIN record). Plain draws now leave those fields out and recordings start with their first command. To check a whole scene, probe where the setup heap is cut (`main+0x155a520` with `!x21`, the resource heap) and the end of the setup (`main+0x155a5f4` with `!x2`, the frame heap), with `DM_PROBE_HEAP=28,38,40,e0,f8 DM_PROBE_HEAPFRAME=1d4f950 DM_PROBE_HEAPWORDS=1` ([05](05-debugging.md)). A model fits any placement when entry + 0x1008 covers the resource heap's header and object (0x310), the file, the frame heap's block header and its peak use, plus 0xff8 of pad. - - Read the probe log carefully: `tryAlloc` probes show *requests*, not successes, so a running total that already exceeds the heap means an earlier request failed. - -- **`FATAL NVN command memory: the game's pool is empty…`** → our command records are larger than real NVN commands, so command buffers the game sizes for hardware overflow; when the game's out-of-memory callback declines to add memory, the host lends a segment. Many small command buffers each overflowing once per frame exhausted a fixed pool of 64 × 1 MB. The pool now grows on demand (256 KB segments, abort only beyond 1 GB; `hostseg_take` in `hle_nvn.c`). A segment is released only when the game re-grants memory overlapping the guest bytes of the recording that jumped into it (`hostseg_release` in `AddCommandMemory`): the game records setup blocks once and keeps `CallCommands`-ing them for many frames, so recycling segments after a fixed number of frames would overwrite live recordings. An empty-pool grant of at most 4 KB outside a recording gets a 4 KB segment (Super Mario Odyssey hands out many 512-byte ones before recording, and with no recording to reclaim them by, 256 KB each reached the cap entering the Wooded Kingdom), a record larger than 256 KB gets a segment that holds it, and the cap counts every segment's bytes (`hostseg_size`). -- **Frames stop after about 15 s; a game thread spins in its command-memory callback** (Fire Emblem: Three Houses: a ring allocator at `main+0x9cc120` waiting for fences that have all signalled) → KTGL gives its main command buffer one 16 MB chunk of a ring and, in its callback (`main+0x9ca960`), takes chunk start + `GetCommandMemoryUsed` as the position the GPU has reached. NVN counts used bytes from the last `AddCommandMemory`; the host counted from `BeginRecording`, so the position stayed at one frame's size and the ring never freed. The count now resets at `AddCommandMemory` (Super Mario Bros. Wonder and Super Mario Odyssey checked unchanged). -- **`FATAL submit re-entered` in the first frames** (Fire Emblem: Three Houses) → the game initializes two NVN queues and submits on both from two threads, while the Metal replay keeps one command buffer for the submission in progress. `nvnQueueSubmitCommands` now holds a lock across the replay (`g_sub_mu` in `hle_nvn.c`). -- **`GetCommandMemoryUsed` must count everything the recording used**, including the parts that went into host segments (plus 16 bytes for the jump). Mario marks a recorded block as empty when the count is 0 and then skips calling it (see the next entry for how that happened). -- **`CRASH sig=10` in `mtl_exec` after minutes of play, the fault address equal to `x19` and shaped like `0x1000000003`** (Tears of the Kingdom 1.4.3, GitHub issue #61) → the fault is the record-header load at the top of `mtl_exec`'s loop, `x19` is the replay pointer, and the address is itself a record header (`{op 3 = OP_CALL, size 16}`): a JUMP's target had been overwritten by the next recording. Tears of the Kingdom gives each model's prerecorded lists fixed slots of the size a recording of that list measured (`GetCommandMemoryUsed`), slot k at base + size × k, and passes `AddCommandMemory` max(size, 512) (`main+0xbb120`); its memory callback is empty. The host kept 24 bytes free after every record for "a JUMP or an END", so a 496-byte list's END went to a host segment and the 16-byte JUMP at 488 reached 8 bytes into the next slot. The host now keeps 16 bytes (one JUMP), lets the END take a segment's last bytes, and links only recordings that have records, so a JUMP only stands where a record of 16 bytes or more was to go and never writes past what `GetCommandMemoryUsed` reported (`OP_RESERVE` in `hle_nvn.c`; `t_nvn` "slotted lists"). To read such a crash: `x8` holds the jump-table target of the previous record's case, and the registers a callee saved below `sp` give the previous replay pointer (`x19`) and payload (`x26`). -- **Frames stop, no crash, the main thread sits in `nvnSyncWait(sync, -1)`, and the watchdog's wait-state line shows the main fence one short (`main fence submitted N / completed N-1`)** → a GPU command buffer never completes. Check GPU load from outside: `ioreg -r -d 1 -c IOAccelerator | grep 'Device Utilization'` at 100 % means a shader is looping forever. Find the shaders first used in the last frames (`DM_NVN_PASSDUMP`, compare the `K` keys) and look at their `.metal` for an empty `do {} while (true);` with no `return` (see [04](04-graphics.md) section 1: the compute program start). - -- **Metal aborts on a texture descriptor with an absurd size (`height (1869367126) greater than … 16384`)** → a command record kept a pointer to the game's `NVNtexture` and read it at replay. NVN commands do not depend on the API object once recorded, so the host now copies the texture object into the record (`REC_SNAP` / `tex_snap` in `hle_nvn.c`: render targets, texture copies and clears). Replay rejects objects without the texture magic and sizes beyond Metal's limits, with a red line instead of an abort. The first red lines name the call site and the object's first bytes; in Engage those bytes were 32-bit triangle indices, i.e. another write had already overrun into the texture object; that stopped once the alias region had address space left (see the Engage entry in [04](04-graphics.md)). -- **Shaders reported missing at random, and their dumped control sections are garbage (Unity)** → NVN lets the caller free a program's control section once `nvnProgramSetShaders` returns; Unity does, and the host used to hash it at the first draw. The keys are now computed inside `SetShaders` (`hle_nvn.c` `prog_key`). Mario calls `SetShaders` for some programs with a code address that is not mapped yet, so a stage whose code is not in a registered memory pool at that moment is hashed at its first draw, as before. -- **Video frames, or a game's built-in default textures, are all zero** → the texture is block-linear, has no packaged layout and is written from the CPU with `nvnTextureWriteTexels` (Engage's video frames: Y plane `R8` + UV plane `RG8`, every frame; Mario's and Unity's 4×4 fallback textures, which hold black, white, transparent and red texels). The host uploaded only packaged textures from guest memory and made everything else a GPU-only texture. `WriteTexels` now records the texture, and it resolves like a packaged one: uploaded from guest memory, re-uploaded when its pages are dirtied (`mtl_tex_cpu_written`). Mario's frames got slightly brighter where materials sample the default white texture. -- **Menus and dialogue boxes draw but contain no text; images and pictogram digits are fine** → the game rasterizes glyphs on the CPU into a *linear* texture in a CPU-visible pool (Mario: 2048×8192 R8, flags IMAGE|LINEAR). A private Metal texture never sees those writes. Linear textures in CPU-visible pools are now created on the pool buffer (`tex_linear` in `nvn_mtl.m`). `DM_NVN_TEXCPU=1` lists non-packaged textures in CPU-visible pools. -- **Characters (skinned models) invisible or stretched into streaks while the level renders** → integer vertex inputs (bone indices) translated as float; see [04](04-graphics.md) section 6. To find the draws: log model file ranges with `DM_PROBE=main:480f0` (x1 = file data, x2 = size), match the vertex/index buffer addresses in `DM_NVN_PASSDUMP_ALL` against them, then `DM_NVN_HILITE` their fragment shader. -- **Sky black** → CPU-written linear colour ramps (see the linear-texture entry above). -- **Controller-button icons in text show as `?`** → those glyphs come from the system shared font's Nintendo-extension set (private-use code points; Mario's `GameMsg/Key.msbt` uses U+E0E0–E0F0 for A/B/X/Y/L/R/ZL/ZR/SL/SR/D-pad/+/−, U+E0AB–E0AE for D-pad directions, U+E090–E097 for stick arrows, U+E100/E101 for sticks). `dm/tools/make_ext_font.py` generates a stand-in `nintendo_ext.ttf` with those icons; the host serves it for shared-font type 5 (`DM_FONT_DIR` or `DM_SHARED_FONT_EXT`) and a desktop font for the other types. To find which code points a title uses, scan the `TXT2` sections of its MSBT files for U+E000–F8FF. -- **Shared fonts on iOS**: the macOS stand-in path (`/System/Library/Fonts/Supplemental/Arial Unicode.ttf`) does not exist on a phone, and `GetSharedFontLoadState` would report "not loaded". The iOS app ships `dm/font/standard.ttf` and `dm/font/nintendo_ext.ttf` (copied / generated by `build_ipa.sh`) and sets `DM_FONT_DIR`. - -### Frame rate - -- **Running a 30 fps game at 60** (Breath of the Wild): find where the present interval comes from and how the logic time step follows it before patching anything. BotW's `sead::GameFrameworkNx` main loop (`main+0x112660c`) runs a dynamic interval: from the last frame's time it computes how many vertical syncs were needed, clamps that to [2, 3] (the maximum from the interval controller built at `main+0x1126c04`), steps up after three overrunning frames and down one level every six frames, and sets the result through `setWaitVsyncInterval` (vtable +0x130, `main+0x11269b8` → `nvnWindowSetPresentInterval`). The game logic follows on its own: `ksys::VFR` is initialized with a constant base interval of 2 (`main+0x127e740`) and every frame `VFR::updateInterval` (`main+0x151d3d0`) takes the framework's current interval, giving a frame ratio of interval / 2. Five rewriter cuts lower the floor from 2 to 1 (its `patches.py`, build `cd57b23f…`): the candidate floor (two places) and the hysteresis targets. Probes confirmed the result: the framework presents at interval 1, VFR reports interval 1 with ratio 0.5, and when a frame runs long the game steps back to 2 with ratio 1, so the game speed does not change. Then check what the controller measures: BotW's measured only the main thread's CPU time (about 3 ms a frame), so a GPU-bound device stayed at interval 1 and played in slow motion (GitHub issue #9). Two more cuts make it always take its GPU-time path, and the host's timestamp reports had to carry real GPU times for that path to see anything (profile key `gpu_timestamps`, [04](04-graphics.md)). The zeldaret/botw decompilation (`VFR.cpp`) gave the semantics; 1.9.0's addresses were found by the constants (`60 / interval`, the `fmov s11, #2.0` floor). -- **A CPU frame limiter with a constant** (Dark Souls Remastered: 30 on the Switch, 60 now). The game presents at interval 1, yet every frame took exactly 33.3 ms of "frame work": the cap is the render thread's own limiter, `main+0x707090(this, fps)`, which sleeps out 1000 / fps ms since its previous call, called once with the constant 30 (`main+0x706fa4`). Found from the render thread's backtrace in the thread dump (`HGCommandDispatcher`), not from the NVN calls. Before changing the constant, check how the game steps time, and not by eye: a 0.8 s walk that looked the same at 30 and 60 fps on the Mac was taken for a time-based game, and on the iPhone everything ran twice as fast. The frame update takes a fixed step, the float 1/30 at `main+0x167a6ec`, loaded at `main+0x667904` and passed to `main+0x6a337c`; a second cut loads the 1/60 at `main+0x167a834` (the community's 60 fps cheat for this build changes the same two words). Some 20 other loads of that 1/30 turn parameters counted in 30 fps frames into seconds or step single systems; they stay. Probing every site that loads such a constant (`DM_PROBE`, 16 per run) tells per-frame steps from one-off conversions, but which of them sets the game's speed only shows by timing the game. -- **55 fps with every frame 18.2 ms, the main thread asleep most of the time** (Taiko Risshiden V DX in its menus and town; its movies, paced by their own player, ran at 60) → KTGL's frame end (`main+0x443e8c`) waits until 1/60 s has passed since the frame's start tick, polling `GetSystemTick` and sleeping 2 ms a round (`main+0x4c9f24`), so with about 2 ms of work the rounds end at 18.2 ms, never at 16.7, and every 11th frame stayed on screen two vsyncs. Found by sampling the host (`sample `: 90% of the main thread in `dm_guest_sleep_ns` under a guest return address of `main+0x443d74`) and a probe on the target's multiply (x20 = 1 frame, x0 = 19.2 MHz). The game presents at interval 1, so a build cut makes the check jump to the end and the host's present pacing (the acquire waits) holds 60: frame time p50 16.66 / p95 16.85 ms, wakeups 2,050 → 320 a second (its `patches.py`). -- **Stalls of 30–60 ms about once a second, at 30 fps and at 60** (Dark Souls Remastered in the Asylum cell: 99th percentile 63–127 ms) → the thread that acquires the window (here the render thread) runs under a time-constraint policy whose computation the host re-declares every 120 frames; it was 1.25 × the 90th percentile of that thread's CPU time per frame. This render thread needs 1.9–2.0 ms a frame with one or two frames in 120 near 4.4 ms, against 2.5 ms declared, and each frame past the declaration stalled the thread. Probes on both threads' waits showed the main thread waiting 20–34 ms for the render thread once a second; adding probes slowed the render thread and the stalls vanished, and `DM_RT=0` or a fixed `DM_RT_COMP=5` / `10` removed them too (99th percentile 18–21 ms) at the same CPU energy. The declaration now also covers 1.1 × the window's peak (`rt_comp_tick` in `hle_nvn.c`): 5.6 ms here, 60.0 fps, 99th percentile 21 ms. -- **A steady game whose frame-rate counter wanders between 50 and 60 on the iPhone only** (Super Mario Galaxy 2, Vulkan) → the `display link up to tick` lines show about as many empty ticks as frames dropped by merging: frames arrive two in one tick and none in the next. The Vulkan layer's acquire and the `vi::GetDisplayVsyncEvent` thread each kept a 60 Hz clock of their own, and Galaxy runs the Wii's VI retrace on that event, so its frames drifted against the screen. While the display link presents, both follow its ticks now (`hle_vk.m` acquire → `mtl_link_wait_tick`, as NVN's; `hle_sdk16.c` vsync thread → `mtl_link_wait_vsync`), with the old clocks as the fallback. A Mac window measures nothing here unless it is on screen: macOS fires an occluded window's display link about once a second (`display link recovered: stalled 1017 ms`). - -### Audio - -- **Check whether the SDK carries the renderer.** List the `sdk` module's exports for `nn::audio::server::` and `nn::audio::dsp::CommandListProcessor`. If they are there (Mario, SDK 16), do not write a mixer: switch `AudioRendererParameter` to rendering device = CPU (+0x2c = 1) and execution mode = manual (+0x30 = 1) in `GetAudioRendererWorkBufferSize` and both `OpenAudioRenderer` overloads, call the SDK originals (`guest_sym` returns the module's own code even for a forced symbol), and drive `ExecuteAudioRendererRendering` from a guest thread (`dm_guest_thread_spawn`). The open then goes through `CreateAudioRendererManagerByDfc`; no IPC. -- **SDK 20's `AudioRenderSystem` has another layout** (Breath of the Wild): the getters the host used (`GetExecutionMode`, `GetRenderingDevice`, `GetCommandBufferWorkBufferSize`) are inlined away, `GenerateCommand` takes and returns a `Span` (the generated list comes back in x0 / x1), the device and mode are words at +0x48 / +0x4c, the command buffer is at +0xf8 / +0x100, and `UpdateImpl` locks the mutex at +0x2d0 instead of +0x1f0. The DeviceSink command became ID 0x14 (SDK 20 inserted a command before it) with the same layout. `render_once` picks the layout by whether the Span version of `GenerateCommand` exists (`hle_audren.c`). The render thread's call also takes another route: the public `ExecuteAudioRendererRendering` (`sdk+0x221e90`) reaches `AudioRendererImpl::ExecuteAudioRendererRendering` (`sdk+0x22ca88`, `x0 += 0x60` and a jump through the PLT) through the vtable, so the host's takeover of the `Impl` function was never called. The SDK's own `AudioRenderSystem::ExecuteAudioRendererRendering` ran instead (no update lock), every `RequestUpdate` rendered a catch-up frame because the host's render counter never moved, and the two threads wrote the same command buffer: `CommandListProcessor::Process` crashed now and then (SIGSEGV under `render_main`, once at 60 s and once at 134 s in the shrine). The host now also takes over `AudioRenderSystem::ExecuteAudioRendererRendering`, the PLT target; catch-ups fell from one per update to 3 per run. -- **The CPU processor drops the final mix.** `dsp::SendCommandBuffer`, `dsp::Wait` and `dsp::Signal` are stubs in the application build, and DeviceSink (command 0x13) is a no-op in `CommandListProcessor::Process`. Take the output after `Process` from the mix buffers the DeviceSink command names (layout in `hle_audren.c`). -- **Generation needs the renderer's lock.** `AudioRenderSystem::Update` holds a mutex in `AudioRenderSystem`; `GenerateCommand` does not take it, so the host does (the auto-mode system thread does the same in `SendCommandToDsp`). Which mutex depends on the SDK: `nn::os::LockMutex(MutexType *)` at +0x1d0 in SDK 4.4 and at +0x1d8 in 6.4–8.2, `SdkMutexType::Lock` at +0x1d8 in 10.4–11.4, +0x1f0 in 13.3–16.2 and +0x1f8 in 17.5 (the `sdk` builds of 24 games on hand), inlined in 19 and 20 (+0x2d0, the Span layout above). The host took SDK 16's for every SDK before 20, which wrote a lock word into another field and excluded nothing; Mario Kart 8 Deluxe (4.4) was the first game it was plainly wrong for. `update_lock` in `hle_audren.c` reads it off the SDK's own `Update`: the register the prologue adds the offset to and moves into x0 before its first `bl`, whose PLT slot must hold `LockMutex` or `SdkMutexType::Lock`; anything else keeps SDK 16's. Checked with sound on Big Brain Academy (11.4, +0x1d8), DRAGON QUEST XI S (7.3, `MutexType` +0x1d8) and Super Mario Odyssey (7.3, which is silent for its first minute with either lock). -- **At least one rendered frame between two updates.** An effect's "needs init" parameter state is set only in the update where the game flags it new; the next update copies the game's value over it. Mario updates twice in a row at start-up; with a real audio device the first frame came after both, and the light limiter later crashed on a null state pointer (`ApplyLightLimiterEffect`, `sdk+0x228494`). The `AudioRenderSystem::Update` hook renders a catch-up frame first when nothing was rendered since the last update. `DM_AUDREN_TRACE=1` logs every update with the number of frames rendered since the previous one. -- **A renderer opened without an event** (Balatro: LÖVE's OpenAL Soft on `OpenAudioRenderer(handle, parameter, work, size)`, the form with no system event). The game paces its own updates, and the renderer must still run every 5 ms; the host used to start its render thread only for the form with an event, so only the game's updates rendered (catch-up frames that nothing played) and the game was silent. Both forms now go through one open that starts the thread (`hle_audren.c` `ar_open`). -- **`[RED] audren: a second renderer` right after the first open, with the same work buffer and "original device 1 mode 1"** (ROMANCE OF THE THREE KINGDOMS 8 REMAKE, SDK 17.5.3) → not a second renderer: SDK 17.5.3's `OpenAudioRenderer` with an event opens it by calling the form without one through the PLT (`sdk+0x1e489c`), which the loader binds to the host's hook since that form has one (the entry above), so `ar_open` ran twice for one renderer. The inner call started the render thread before the outer one had created the event, and the outer call took the renderer for a second one. A thread-local flag (`t_in_open`) now hands the inner call straight to the SDK with the host's CPU / manual copy of the parameter, and only the outer call starts the thread. -- **A sound driver that never waits on the renderer event** (DRAGON QUEST XI S, Square Enix's `SQEX::Sd::Driver::Core::CoreAudioOut`, its loop at `main+0x442c5c0`). It appends a wave buffer, mixes the next block on the same thread, calls `RequestUpdate` and takes the next released buffer. It waits on the renderer event only when no buffer is released. Every update there finds no frame rendered since the last one and renders a catch-up frame, which releases the buffer just appended, so the loop never waits. When a full catch-up queue discarded the frame, the loop ran 11,000 updates a second on one core, the sound ran about 50 times faster than the output. An update with a full queue now waits (at most 50 ms) for the render thread to play one queued frame, so catch-up frames keep real time. The render thread then only plays queued frames, and the driver's updates render every frame. A queued frame does not signal the renderer event (the SDK signals it in each render): signalled, an ordinary game woke, updated with nothing rendered since and rendered the next catch-up frame, so every frame became one (Big Brain Academy: Brain vs. Brain, about 38,000 catch-up renders instead of 37). `audren: rendered 12000 frames, N of them not silent` shows whether a game's sound reaches the output. -- **Crackling on iOS only** → RemoteIO asks for as many frames per callback as the audio session's I/O buffer duration, about 21 ms by default (1024 frames at 48 kHz), while the renderer keeps 4 blocks × 5 ms = 20 ms in flight. Every callback came up 1.3 ms short and was padded with zeros, about 47 gaps per second. Reproduced on the Mac by forcing 1024-frame callbacks (`DM_DBG_AU_FRAMES=1024` with `DM_AUDIO=1`: 1999 underruns in 2000 callbacks). Fix: `AVAudioSession` `setPreferredIOBufferDuration:0.005` (and 48 kHz) before the game opens audio and again on return to the foreground (`front_ios_audio_session` in `front_ios.m`); the device then calls back with 240 frames and nothing underruns. When checking a new device, read the host's `audio #N device callbacks` line, logged every 2000 callbacks: frames per callback, queued blocks, underruns. -- **Several outputs at once.** Engage opens one AudioOut from Wwise (`main+0x177d924`, no release event, the game polls). On the phone it opened a second one later; the Mac runs never did. Unity's FMOD also has an output (`main+0x4aec0c` / `0x4aec4c`). The host logs each output as `audio #N`: the open with its call site, Start / Stop / Close (the first 32 per output), and the callback line every 2000 of that output's callbacks. Combined numbers mislead: one fully fed output plus one fed by nobody look like "every other callback underruns, 2.0 blocks queued". -- **The background round trip on the Mac**: `DM_AUDIO=1 DM_AUDIO_MUTE=1` takes the real Core Audio path with the output silenced, and `DM_OE_SCRIPT="bg=1@60,bg=0@66"` runs the same sequence as the iOS lifecycle callbacks (focus, audio take-over, guest clock freeze, graphics suspend). Send `dmhost` `SIGSTOP` / `SIGCONT` between the two to imitate iOS suspending the process. The silent sink that stands in while in the background does not make up the time the process was suspended (it used to pull seconds' worth at once and log hundreds of underruns). -- **Sound stays silent after Siri, an alarm, a call or another app's audio** → iOS stops calling the RemoteIO unit back without stopping it, so the host still thinks the output plays; a game whose audio thread waits for its buffers back (Wwise's poll-only output, FMOD) then waits forever. Three layers handle it: `front_ios.m` observes `AVAudioSessionInterruptionNotification` (began → the silent sink stands in, ended → session up again and resume; becoming active also ends an interruption whose "ended" never came) and `AVAudioSessionMediaServicesWereResetNotification` (`dm_audio_reset` recreates every AudioUnit); and a watchdog in `hle_audio.c` switches any started output whose device has not called back for 200 ms in focus to the silent sink, retrying the AudioUnit after 1, 2, 4, 8 s. A start that fails (session still held by a call) falls back the same way instead of aborting. The log shows `audio session: interrupted`, `the device has not called back for N ms` and `AudioUnit restarted`. `tests/t_audio.c` scenario 3 stops a playing AudioUnit behind the host's back. -- **The main thread spins at start-up in a resource wait (a `YieldThread` loop, no crash), right after `OpenAudioRenderer`** (Hyrule Warriors: Age of Calamity) → Koei Tecmo's sound engine (KTSL) failed its setup quietly. The init hands it three heaps cut from one block (15 MB engine memory, 2 MB, 142 MB sound data; the sizes are constants at `main+0x8d9610`). In the 15 MB heap the setup allocates the renderer config and work buffer (0x760000: `AudioRenderSystem::GetWorkBufferSize`, 0x40a9c8 of it command buffer) and one block per player (up to 0x1b5860; three of the four mix 5.1). The fourth block came back null, the setup released its memory pools (`RequestAttachMemoryPool` then `Detach` with no update between) and returned −51, the first sound archive (ASRS) never loaded, and the resource the main thread waited for depended on it. How it was found: probe the pump the wait calls (`main+0x8e1c0`: first missing stage bit → the type's stage handler), the handler's dependency list (an incomplete entry of type `0xbbd39f2d`, ASRS), then the memory pool states (`MemoryPoolInfo` +0x10: 1 new, 3 detached, 4 request attach, 5 attached, 6 unused) and the engine allocator's arguments and results. A per-build cut moves 4 MB from the sound data heap to the engine heap (its `patches.py`); why hardware fits in 15 MB is not known. -- **A renderer at 32 kHz: SIGSEGV at address 0x18 in `UpsamplerManager::Allocate` on the render thread** (DRAGON QUEST VII Reimagined: rate 32000, 160 samples per frame, `GenerateSinkCommands` `sdk+0x225a80`) → a DeviceSink below 48 kHz is upsampled by the SDK's upsampler, and `AudioRenderSystem::SetUpRenderResourceSet` allocates the `UpsamplerManager` only for the DSP (`sdk+0x21eb6c`: rate 0x7d00 and rendering device +0x48 == 0); with the host's CPU rendering the manager pointer (`AudioRenderSystem+0x150`, the render context's +0x20) is null. The host answers `Allocate` on a null manager with no upsampler, so the generator skips the upsample command (`sdk+0x225b40`), and resamples the DeviceSink's stereo downmix 3:2 itself (a 16-tap windowed sinc in three phases, `up_3_2` in `hle_audren.c`), 240 samples per 5 ms frame for the 48 kHz output. Until then the output code assumed an upsampled buffer at 32 kHz that no game had produced. -- **Checking audio without listening**: `DM_AUDIO_DUMP=file` writes the PCM the host plays (16-bit, 48 kHz, stereo). RMS per second, clipping share, spectral flatness (music ≈ 0.1–0.5, white noise ≈ 0.85) and the strongest FFT peaks (they should land on musical notes) tell real music from noise or silence. - -### NVN object sizes (memory corruption) - -The game allocates NVN objects with the sizes of *its* SDK's headers, which shrank in newer SDKs. A host record larger than the game's allocation overwrites neighbouring memory: - -- `NVNtexture` is 0x80 bytes in SDK 16 (heap corruption: the old 0x88 record overwrote the next heap block's links). -- `NVNtextureView` is 16 bytes in SDK 16 (stack smash: a 16-byte local next to the saved frame pointer; the function returned to `0x100000000`). -- `NVNprogram` is smaller in Mario than the 0xC0 bytes in `nvn.h`: growing the host record from 112 to 168 bytes corrupted the next object (a crash in SHA-256 on the code address 0x16300). Keep `prog_t` at 112 bytes; per-program host data goes in a side table (`prog_key`). - -Check every host object against the new SDK: `_Static_assert(sizeof(host) <= N)` next to the struct, and when a crash returns to a nonsense address, look at the 0x100 bytes below `sp` in the crash report for a zeroed frame. - -Host answers that size the game's own allocations count too. A size the hardware never reports as 0 must not be 0 on the host, even where the host needs no memory: Cuphead (Unity 2017.4) allocates each depth target's ZCull save area from its NVN pool allocator, which gives a zero-size request the offset of a live block. Its map of allocations then lost that block, and freeing it later released a garbage-sized range: buffers overlapped, and sprites broke apart in some runs only, depending on the allocation order. It was found by tracking the allocator from observation gates (`OBS` in its `patches.py`) on its free-list push and after its map insert, checking at every event that each block allocated so far was still in the game's map. - -The game may also write into an NVN object directly, without an NVN call, through code inlined from the SDK headers. Host records stored in place must leave those bytes alone: - -- **Recorded commands patched in place** (Luigi's Mansion 3: the title showed one stretched 64×64 icon over the 3D scene, a new game a white screen). The engine records a command buffer, then rewrites texture handles in it before submitting, as the hardware lays the command out: before `nvnCommandBufferBindTexture` it takes the command memory base + `GetCommandMemoryUsed` (`main+0x7c8964`, the only reader of that NVN entry point), and every frame it stores a 32-bit handle at that address + 0x10 (`main+0xb7d0d4`, on a job worker). In a host bind record that word was the count, and the replay read 535 million handles from whatever followed, so the post-process passes kept stale texture handles in their slots. Found with a check that reports a bind record holding more than 32 handles (`[RED] bind record … overwritten after recording`), then `DM_PROBE_WATCH` on that word. A single texture bind is now recorded as `OP_BIND_TEX1`, its handle 16 bytes in (`nvn_obj.h` `p_bind_tex1`, read through `bind_rec`). Any other command a game patches this way needs the same: find the readers of the `GetCommandMemoryUsed` slot first. -- **`NVNcommandBuffer` +0x10**: Mario writes the byte 1 there (`main+0x4da5c`, `main+0xb8dec`). In the host record it was the low byte of `end`. Each write shrank the game's 512-byte material setup blocks, so their recordings overflowed into host segments; `GetCommandMemoryUsed` (then `cur − begin`) returned 0, the game marked the setups empty and skipped calling them, and the draws ran with the previous draw's vertex buffers: out-of-range vertices, NaN, a black band and flicker. It was seen on the iPhone; Mac runs of the same scene looked clean. `cmd_t` now reserves the byte (`nvn_flag10` in `nvn_obj.h`). To find such writes: put the field you suspect behind a guard value and log when it changes outside the host's own code, or force overflows on the Mac by shrinking the command memory the game gets. - -### Game state - -- **In a scripted run, a screen goes back by itself right after a confirm (Engage's appearance choice)** → the confirm opened the system keyboard (`swkbd`, name entry). Without a window the host answers "cancel" by design; pass `DM_KBD=` to answer it. The log shows `swkbd: … → no window, treated as cancel`. -- **A player join screen that ignores A** (3D World: a character select with four "L + R to join" slots after the title) → the game calls `hid::StartLrAssignmentMode` and waits for L and R pressed together on the join screen itself, then a character choice with A. In `DM_KEYS` that is `L+R~0.5` while the screen is up (presses on the title before it are lost). On the game select, holding the stick right for a second (`7:LRight~1`) switches to Bowser's Fury, which also uses `nvnTextureGetSwizzle` and `nvnTextureGetStorageClass`. Map and level movement read the left stick only: `DM_KEYS` takes `LLeft` / `LUp` / `LRight` / `LDown`. A working route into World 1-1 on a fresh save: `DM_KEYS=6:A,10:A,14:L+R~0.5,17:A*3/2,24:Plus*6/3,45:A*6/3,66:LUp~1,70:A,74:LRight~1,78:A,82:LLeft~1,86:A` (World 1-1 by frame ~6000). -- **"Stuck" at a screen with one draw per frame** → usually not stuck: most draws are in recorded command buffers replayed later; dump the passes ([04](04-graphics.md)) before assuming the game logic is blocked. +A new finding goes to one place: a general mechanism as an entry in its reference (with a line in the symptom index), a fact about one game in that game's note (`dm/games//README.md`). diff --git a/docs/agents/04-graphics.md b/docs/agents/04-graphics.md index 37cf04f..fe326e4 100644 --- a/docs/agents/04-graphics.md +++ b/docs/agents/04-graphics.md @@ -6,34 +6,7 @@ The graphics layer records NVN commands into its own command format (`dm/gfx/nvn Do not stop at the obvious archive. In Mario the main `bfsha` held 4,888 of the 12,811 shaders the game actually uses. -`dm/gfx/tools/shader_split.py []` scans **every** RomFS file (zstd-decompressed, SARC expanded recursively) and `main`, and recognizes the containers below. The pairs table lists the (vertex key, pixel key) of every BNSH variation or Unity program (section 6). Unity shaders need a second pass: files whose type trees are missing are scanned again once the bundles have supplied the `Shader` tree (or with the one `unityfs.py` has written out for that hash, when no bundle has it). For a base game plus an update, give RomFS and `main` as comma-separated lists, base first: keys are content hashes, so the output is the union of both versions and one shader build serves both (Mario 1.2.1 recompiled 3,451 of its pixel shaders; v0 rows keep their package indices). - -- **BNSH** (`BNSH\0\0\0\0` + `grsc` block at +0x60): inside `bfsha`, particle effects (`esetb`, VFXB), UI layouts (`blarc`), and `main` itself. Variation table at +0x80 (count at +0x7c); binary program at variation +0x10; six stage pointers from program +0x08 (vs, hs, ds, gs, ps, cs); stage code descriptor: control ptr +0x08, code ptr +0x10, code size +0x18, control size +0x1c. All pointers relative to the BNSH start. -- **Unity Shader objects** (Engage): Unity 2020.3 keeps each shader's compiled programs in its `Shader` object (class 48), in UnityFS bundles (`StreamingAssets/aa`) and in the player's own serialized files (`data.unity3d`, `Resources/unity default resources`). `dm/gfx/tools/unityfs.py` reads them: UnityFS blocks (LZ4 / LZMA), the serialized file, the object through its type tree, then `compressedBlob` per platform (LZ4 segments; each segment starts with a u32 count and 12-byte `{offset, length, segment}` entries). The player's files have no type trees; they borrow the bundles' `Shader` tree with the same type hash (same Unity build, same layout). Type-tree strings with the high bit set index Unity's common string table, which `unityfs.common_strings` reads from the game's own `main` (`AABB\0AnimationClip\0…`). On Switch every subprogram is one whole NVN program: its code is a 0x60-byte header of four `i32 ×6` arrays indexed by NVN stage (0 vertex, 1 fragment, 2 geometry, 3 tess control, 4 tess evaluation, 5 compute): an unused / scratch size, the block offset from +0x60 (−1 = no stage), the control size and the code size. Each block is the control section followed by the code area. The vertex and fragment stages of one program form a pair. Engage: 209 `Shader` objects in 188 bundles plus the engine files, 9,269 stage shaders (4,634 vertex / pixel pairs and one geometry shader), no compute shaders. -- **Unity 2022 Shader objects** (Ace Attorney Investigations Collection, Unity 2022.3.17f1): the program code keeps the 0x60-byte stage header, but two things changed. From Unity 2022 the blob also holds each program's parameter tables as entries of their own (the parsed form's `m_ParameterBlobIndices`; their bytes start `PGlobals`), and the player's programs are listed in `m_PlayerSubPrograms[tier][].m_BlobIndex` (`m_SubPrograms` is empty); `unity_code_blobs` keeps only those. UnityPy's type trees for 2020.3 (Engage) and 2021.3 have neither field, so those read every entry as before. Subprogram version 202012090 (from Unity 2021.2) also has no local keyword list, so `unityfs.subprogram` reads no second keyword array (with one it read the code length as a keyword count). The player's serialized files have no type trees and the Addressables bundles hold no `Shader`, except the `unitybuiltinshaders` bundle, whose tree has the same hash and is borrowed in the second pass. 4,951 stage shaders: 2,475 vertex / pixel pairs (4,294 in the player's files, 517 built-ins, 140 in bundles) and one geometry shader, which Metal cannot express (its build fails). armyja's issue #17 proposed UnityPy and a reader of its own; neither was needed. -- **A Unity player with no bundle to borrow a tree from** (Brain Memory Training Academy, Unity 2022.3.5f1: only `Data/*.assets`, `level*` and the two engine files in `Data/Resources`, none with type trees) → the split stopped with "no bundle has a Shader type tree of the same hash to borrow". The `Shader` type hash there (`ea424a5f…`) is the one in Ace Attorney Investigations Collection's bundles (2022.3.17f1): the same layout. `unityfs.py` now carries that layout written out (`KNOWN_TREES`: about 30 structs, `tree_from_layout` builds the tree a file with type trees would have) and the split's second pass gives it to the stripped files when no bundle lent a tree of that hash (only there: taken in the first pass, Ace Attorney's player files came before its `unitybuiltinshaders` bundle and the built-ins' source column and package indices moved, while with the second pass its tables come out byte-identical). Checked against Ace Attorney's own tree: the same 2,324 nodes with the same alignment flags, and its 40 bundle `Shader` objects decode identically with either. 496 stage shaders: 439 engine built-ins, 46 of the game's, 11 of the player's in `main`, and the built-in VR blit's geometry shader, which does not build. A Unity version whose `Shader` hash is not in the table still stops with the same message; read its tree from any bundle of that Unity version and add it. Patrick's Parabox (Unity 2020.3.30f1, built-in shaders only, no bundles) stopped there with hash `a30e3dcf…`, which one 18 KB shader bundle of Fire Emblem Engage (2020.3.18f1) carries; its tree is now in the table too (`SHADER_2020_3`: per subprogram global and local keyword indices, every subprogram in `m_SubPrograms`, no parameter blobs, keyword names or stage counts), equal to Engage's node for node (2,179 nodes, the alignment flags included), and each of the game's 48 `Shader` objects reads to its exact end. The structs both layouts share are written once (`_SHADER_COMMON`), and the 2022.3 layout comes out byte-identical. Raging Loop (Unity 2018.3.10f1, no bundles: 48 `Shader` objects in `Data/*.assets` and `Data/Resources`) stopped there with hash `8db76007…`, and no game of that Unity version with a bundle was at hand, so `SHADER_2018_3` is written from AssetRipper's TypeTreeDumps (`StructsDump/release/2018.3.10f1.dump`), checked node for node against the dump (1,325 nodes, the alignment flags included). It is a layout of its own, older than the common structs: no conservative raster state, `BufferBinding` without `m_ArraySize`, `ConstantBuffer` without `m_IsPartialCB`, the parameters in each subprogram and no `m_CommonParameters`, one segment per platform (the pre-2019.3 form Cuphead's trees gave `shader_programs`). Each of the 48 objects reads to its exact end: 450 subprograms of version 201802150 (older than the local keyword list, which starts at 201806140). To take one file out of an NSP without extracting the RomFS, filter `nx_extract`'s `Copy` by path. -- **Unity ComputeShader objects** (Monster Train 2, Unity 2022.3.54f1) → the 3D battle scene was black behind a correct UI, with `[RED] shader 854a8afc9d07f88d is not in the shader map` and `the dispatched program has no compute shader`. `DM_NVN_SHDUMP` saved the program; its code bytes were in `Data/sharedassets2.assets`, uncompressed, inside a `ComputeShader` (class 72): the post-processing stack's colour grading LUT baker (`Lut3DBaker#KGenLut3D_AcesTonemap`), so the LUT the scene is graded through stayed black. The split read only `Shader` objects. A `ComputeShader` holds per platform variant a list of kernels, each with its unique variants; a variant's `code` is the program code of a `Shader` subprogram (the 0x60-byte stage header, only the compute block present) and its `threadGroupSize` matches control +0x748. `unity_compute` in `shader_split.py` reads them beside the `Shader` objects of every serialized file; the game has no bundle with a class 72 tree, so `unityfs.py` carries Unity 2022.3.54f1's layout (`COMPUTE_2022_3`, written from AssetRipper's TypeTreeDumps for that version; all 12 objects decode, `resourcesResolved` true at each end). 85 compute shaders came in (86 with the player's own). A file whose `ComputeShader` has no tree still goes to the second pass; if it has none there either, the split prints a `[RED]` line and leaves its kernels out instead of stopping, so profiles whose compute shaders were never read keep their tables: Cuphead (2) and Fire Emblem Engage (4) have `ComputeShader` objects only in files without type trees, of hashes no layout covers. So does Raging Loop (Unity 2018.3: the two built-ins in `unity default resources`, whose 2018 layout lists kernels with no unique variants), which dispatches no compute. -- **MSHD** (Metroid Dread, MercurySteam's engine: `system/shd/*.bshdat`, the only place its shaders live; nothing in `main` or the `.pkg` packs): each stage is a control section, whose length is at control +0x20, sometimes followed by a small compiler-option table (`fastmath true`), then the code area at the next code header. Every file holds vertex / pixel stages in alternating order (1, 2 or 4 programs) and is paired two by two. The control section carries NV assembly text after its first 0x880 bytes, and the SASS disassembly follows each code area, so the binding layout can be read straight from the files (`TEX` / `TEXS` handle immediates). 478 files, 1,342 stages, 1,010 unique keys (359 vertex, 651 pixel), 659 pairs. -- **Unity 6 Shader objects** (Hollow Knight: Silksong, Unity 6000.0.50f1) → read with no change: its 2,068 Addressables bundles carry their type trees (they report the engine version `0.0.0`, Unity's stripped version string), so no layout had to be written out. 2,450 stage shaders: 1,189 vertex, 1,193 pixel, 67 compute and one geometry shader, which does not build; 11 of them are the player's own in `main`. BALL x PIT (6000.0.62f1) keeps most of its 38,099 in the player's loose serialized files (`resources.assets`, `sharedassets1.assets`, the built-in resources), which have no type trees; they borrow the `Shader` tree its 10 shader bundles carry (`shader_split.py` defers a file that raises `NoTree` to a second pass), so it needed no layout either. -- **GLSLC outputs in `main`** (13 Sentinels, Vanillaware's engine: no shader file in RomFS, whose six CRI CPK archives hold textures, scripts, CRI audio and USM movies): 838 GLSLC compiler outputs sit in `main`'s read-only segment, each referenced from a `{pointer, size, 0}` table in `.data`. An output starts with the magic `0x19866891`, its total size is at +0x44, and it holds exactly one control section with the code area right after it (control +0x20 = control length); a debug section with the NV assembly and the GLSL source follows. Engage's `main` also has 11 of them (Unity's own), and `shader_split.py` reads them in every game's `main`, accepting only outputs that pass the control / code / length check. The stage comes from SPH word 0, as for BAHS. Loose RomFS files no other reader takes are searched the same way: FINAL FANTASY TACTICS - The Ivalice Chronicles keeps 257 whole outputs in its technique files (`*.tec`, below), Absolum 163 in `shaders.nx.bin`, Catherine: Full Body one per file in `data/shaders_binary` (3,050, its vertex shaders in two parts, below); in the other profiles' files only Metroid Dread's MSHD files hold any, which `mshd_shaders` reads. An output may also hold several stages compiled together as one program (Baba Is You: the 120 outputs in `Assets.dat`, each a pixel and a vertex shader, then a reflection section): such an output is read by its section headers (+0x4c section count, section *i*'s header at +0x90 + 0x90 *i*, type 0 = GPU code; `glslc_program`), and its stages are paired with each other, not as single-stage programs. Before that reader the split found no shader in the game, and the NSP script now stops at the split when it finds none. -- **G2S in RDB archives** (Hyrule Warriors: Age of Calamity, Koei Tecmo's KTGL): each material's shaders are a G2S container (`_S2G3000`, type KTID `0x7bcd279f`; 23,108 entries in `asset/system.rdb.bin*`, 2.3 GB inflated) of GLSLC outputs, as in 13 Sentinels' `main`, inside zlib-chunked IDRK entries ([01](01-extract.md)); one `_DOK0000` container (type `0x20a6a0bb`, the engine's system resources) holds 151 more bare GLSLC outputs, 21 of them drawn in the first battle. `shader_split.py` maps the RDB files and inflates only entries that start with those two magics (`idrk_shaders`), and the 117 outputs in `main` as in every game. Many outputs carry 0x850-byte control sections and the compute ones have no code header, so `glslc_shaders` accepts controls from 0x760 bytes and takes the stage of headerless code from the thread group size at control +0x748. 40,726 shaders (37,043 pixel, 3,440 vertex, 241 compute, 2 geometry), Mario's texture handle layout (`0x20 + 8 × slot`); pairs come from the single-stage cover (`separable_pairs`). The older KTGL of Three Houses packs the same G2S containers differently (next item). ROMANCE OF THE THREE KINGDOMS 8 REMAKE's newer KTGL keeps them (`_S2G5000` / `6000` / `7000`, and `_DOK0000`) in `.fdata` files whose IDRK entries are LZ4 or zlib in chunks of another form ([01](01-extract.md)): 59,214 outputs there and 149 in `main`, all of them built. -- **SPKG in LINKDATA** (Fire Emblem: Three Houses, the older KTGL): shader packages (`SPKG`: `patch*/nx/shader/*.spkg` and 132 entries of `DATA1.bin`) of G2S containers as above, one `_SPK4100` package without G2S (the post effects, `DATA1.bin` entry 549), and map and model archives carrying packages of their own. `kt_shaders` inflates every LINKDATA entry and loose `*.gz` / `*.spkg` file ([01](01-extract.md)), takes every GLSLC output that passes the triple check, and scans its SARCs for BNSH; `DATA1.bin` is split into 64 scan jobs by entry range (`linkdata_jobs`). Its GLSLC (output version 0x1d.2) writes the 0x848 / 0x850-byte controls and headerless compute code described above, which `glslc_shaders` used to drop: 24 shaders before, 1,355 after. A few outputs of version 0x1d.0 keep something else at control +0x20 (odd values such as 0x136b), so a control length must be a multiple of 4. Nine more shaders exist only at run time (Seen in Fire Emblem: Three Houses, below). -- **SDF in RE Engine PAKs** (Monster Hunter Rise): shader definitions (`SDF\0`, 2,685 files in the PAK view, [01](01-extract.md)): u16 version 1, u16 entry count, u64, then 0x108-byte entries from +0x10. Entry +0x10 holds six u64 offsets (vertex, hull, domain, geometry, pixel, compute; 0 = none), each to `{u32 control length, u32 code length}` followed by the control section (0x880 bytes) and the code area; compute code has no header. An entry's stages are one program, so its vertex / pixel stages are a pair. Empty definitions are 16 bytes of version 0. `kpka_jobs` maps the PAKs, takes the newest copy of each path, peeks at 4 decompressed bytes and inflates only SDF entries (`sdf_shaders`): 63,111 shaders (30,233 vertex, 21,475 pixel, 11,271 compute, 53 geometry, 33 hull, 46 domain) and 102,345 pairs, in 5 s with 12 jobs. -- **Unreal Engine shader code archives** (Octopath Traveler 0: `ShaderArchive-Global-SF_GLSL_SWITCH_MOBILE-SP_GLSL_SWITCH_MOBILE.ushaderbytecode`, 1,219 shaders, and `ShaderArchive-Octopath_Traveler0_Chunk0-…`, 9,377, both stored in the pak, [01](01-extract.md)): version 2 archives of shader map hashes, shader hashes, shader map entries, shader entries `{u64 offset, u32 size, u32 raw size, u8 frequency}` (17 bytes packed; frequency 0 vertex, 3 pixel, 5 compute), preload entries and shader indices, then the code. Every shader is Oodle-compressed (Mermaid, block header `8C 0A`); there is no Oodle decoder in this repository, so the game's own `OodleLZ_Decompress` (statically linked into `main`; the host finds it as the one function that takes the address of the library's own name string "OodleLZ_Decompress", its profiler scope: Octopath Traveler 0 `main+0x4f77e10`, Pokemon Legends: Z-A `main+0x1a9c9e0`) decodes them in the Mac host: `DM_OODLE_JOB` (`dm/host/hle_oodle.c`), the NSP script's `decode_ue_shaders` for any game with Unreal paks. A decoded shader is an `LSLG` record: `LSLG`, a stage tag (`SV`, `SP`, `SC`), a header with the source name (`/Engine/Private/…usf:MainPS`), then `u32` code length, the code area, `u32` control length (0x880), the control section and a 24- or 29-byte tail, all at unaligned offsets (`uepak.lslg`). The host keys came out exactly: 37 of 37 shaders a first run reported missing were in the two archives. 10,324 unique shaders (1,457 vertex, 6,346 pixel, 2,521 compute); vertex and pixel shaders combine per draw, so every vertex shader is paired with the single-stage cover. Bindings: separate textures and samplers, as Engage's (`0x168` / `0x568`); 61 vertex shaders pick a texture slot at run time (profile key `tex_dyn_slots`, below), 2,084 shaders do atomics on buffer images (below). 16 compute shaders do atomics on a 3D `r32ui` image, which SPIRV-Cross's MSL 2.3 image-atomic emulation indexes with an `int3` (`array subscript is not an integer`): all are Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run), so they stay unbuilt. - DRAGON QUEST VII Reimagined (Unreal Engine 4.27, the desktop renderer: `ShaderArchive-Global-SF_GLSL_SWITCH.ushaderbytecode`, 1,764 shaders, and `ShaderArchive-DOLL_Chunk0-SF_GLSL_SWITCH.ushaderbytecode`, 63,185, stored in the pak) has the same version 2 archives and the same `LSLG` records, but every shader is an LZ4 block (the first bytes are LZ4 tokens followed by `LSLG`; LZ4 is Unreal's default shader compression since 4.26), which `uepak.py` decodes itself (`lz4_shaders`: every compressed shader must decode to its raw size as an `LSLG` record, else the archive goes to the Oodle job as before). 62,901 unique shaders (10,114 vertex, 50,756 pixel, 1,968 compute, 63 geometry). Its fragment shaders carry no output types either: one global shader writes integers bit-cast into a float output to an `R8Uint` target, listed in the profile's `shader_facts_manual.tsv`. - FINAL FANTASY RESONANCE (Unreal Engine 5.6.1, the desktop renderer: `SF_GLSL_SWITCH`) has the same `LSLG` records, but no archive in its pak: its two libraries are IoStore shader libraries in `FFRS-Switch.ucas`, whose shader groups are Oodle-compressed as a whole ([01](01-extract.md)). Each shader entry packs its frequency (bits 0–3), group (4–33) and offset in the group's decoded bytes (from bit 34), and a shader runs to the next one's offset; `uepak.py` writes them out as version 2 archives, so the split reads them as any other. 73,496 unique shaders (9,253 vertex, 61,305 pixel, 2,934 compute, 4 geometry). -- **Unreal Engine 4.18 shaders inline in the cooked packages** (DRAGON QUEST XI S: legacy version 4 paks, LZ5-compressed blocks, [01](01-extract.md)): there is no shader code archive. Each material's shader map is serialized in its `.uexp` (38 to 87 shaders in the ones looked at) and the engine's global shaders in `Engine/GlobalShaderCache-GLSL_SWITCH.bin` (945: 597 pixel, 104 vertex, 241 compute, 3 geometry). Each shader's code is a byte array (`i32` length, then the bytes) holding one zlib stream that inflates to the same `LSLG` record as above. `shader_split.py` (`ue_legacy_jobs`, `ue_inline_shaders`) decodes every file of the newest copy of each path with the game's own LZ5 and takes each zlib stream that inflates whole from exactly the length in front of it and starts with `LSLG` (every such stream in the files looked at). Records stored uncompressed are taken too, but none were seen. 38,892 unique shaders (4,654 vertex, 33,981 pixel, 234 compute, 23 geometry) from 1,740 files; vertex and pixel shaders are paired as single-stage programs, as for Octopath Traveler 0. -- **BNSH in Trinity packs** (Pokémon Legends: Z-A, Game Freak's engine): every shader is a BNSH inside the Oodle-compressed files of `arc/data.trpfs` ([01](01-extract.md)). For any game with Trinity packs (`ONEPACK` files), `trpfs_jobs` / `trinity_shaders` in `shader_split.py` name the packs from `data.trpfd`, decode every file with the game's own Oodle in the Mac host (`DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script) and scan it like a loose file. 28,603 shaders (11,633 vertex, 16,845 pixel, 125 compute) and 26,051 variation pairs: the material shaders in `system_resource/gfx2/shader/NX64/env_ikkaku` (`material/standard.bnsh`, `material/transparent.bnsh`, `system/light.bnsh`, and the `special/imgui.trsha` pack with the engine's system shaders), particle effects (`.ptcl` / `.trtml` in `ik_effectdata`, 19,842 of the shaders), UI layouts (`__ArchiveShader.bnsh` in the `.truiv` packs of `uidata`) and the 40 BNSH in `main`. The layout archives' control sections are 104 KB long, and their stage descriptors run 456 bytes past the control's own length (control +0x20), with bytes that differ between copies of one shader; `same_shader` compares copies only up to the control's own length. -- **BNSH in Bezel Engine archives** (Big Brain Academy: Brain vs. Brain, NintendoWare Bezel Engine 1.56): the shaders are BNSH in the `.bfsha` entries of the `.bea` archives (zstd entries, [01](01-extract.md)), most of them the engine's own in `System.nx.bea` (`_BezelSystemResources//shader/*.bfsha`: PBR shadow maps 846, post effects 770, …), the rest in the layouts' `__ArchiveShader.bnsh` and the per-archive shader packs (`.bnbshpk`), plus one BNSH in `main`. `shader_split.py` scans each archive entry as a file of its own (`bea_members`): 2,125 shaders in about 90 s. -- **SPK in an MT Framework ARC** (The Great Ace Attorney Chronicles, Capcom's MT Framework): the engine's one shader package, `sc\NX_64\root` (`SPK\0`, type hash `0x02358e1a`) in `nativeNXx64/sa/NX_64/root.arc` ([01](01-extract.md)). After the package's tables every shader is a code area followed directly by its control section, back to back to the end of the file: 230 vertex shaders, then 230 pixel shaders. Each control section names its code length (+0x704), so the code is the bytes right before it, and every block must start where the previous one ended (`mtspk_shaders`). The tables that say which vertex and pixel shaders the engine combines are not read: the vertex shaders are paired with the single-stage cover (one pixel shader reads every input any of them reads). `CustomShaderPackage.mfx` (`MFX\0`, the effect descriptions) holds no shader code, and neither `main` nor any other ARC entry does. All 460 build, with Mario's binding layout; every binding decodes. -- **Control sections loose in `main`** (Advance Wars 1+2: Re-Boot Camp, Unity 2020.2 with Bink 2's Unity plugin): the movie player builds its programs from 32 control sections and code areas in `main`'s read-only segment that are in no GLSLC output and no BNSH file. 26 are 0x20-byte records in two tables in `.data` (`{code area pointer, control section pointer, u32 code length, u32 control length}`, both pointers relative relocations); the code takes the other 6 by address (`adrp` + `add` to the control section and, a few instructions away, to the code area it copies with `memcpy`), the code area right after the control section's own length (+0x20, 16-byte aligned) or right before it, and two control sections share one code area. Both neighbours can pass the header and length checks, so the reader (`raw_shaders`) takes only a code area the code builds the address of, the nearest one to the control section's. The 8 compute shaders' code areas have no header (SDK 10). Without them the shaders were not in the map (two `[RED] shader … is not in the shader map` lines and `program has no vertex shader` as the opening movie starts) and every movie drew white. Of 25 `main`s on hand no other has such control sections. -- **MonoGame effects** (Celeste: MonoGame for the Switch, its C# compiled to C++ by Sickhead's BRUTE): the 13 XNB files in `Content/Effects` (XNB platform `S`, uncompressed, read by `EffectReader`; a compressed XNB, MonoGame's LZX flag 0x80 or LZ4 flag 0x40 as in Stardew Valley, is decompressed first, `xnb_decompress`) each hold one MGFX effect, and `main` holds SpriteBatch's built-in one; no other file and no GLSLC output. MGFX version 8 is MonoGame's own layout: `MGFX`, u8 version, u8 profile, u32 effect key, the constant buffers, then each shader as u8 vertex (1) / pixel (0), u32 length, the bytecode (u32 control length, the control section, u32 code length, the code area: GLSLC's output without its header), the samplers, the constant buffer indices and an attribute list (as MonoGame's OpenGL backend writes it), then the parameters and the techniques with their passes (`mgfx_effect`). The reader parses every effect to its end, which checks the layout; in `main`, four bytes `MGFX` count only when version 8 and a control section follow. A pass names its vertex and pixel shader by index, 255 for none, and 10 of the 13 effects have no vertex shader: SpriteBatch binds its own and the effect's pass only replaces the pixel shader. Vertex and pixel shaders therefore combine across effects, and they are paired as single-stage programs. 26 shaders (20 pixel, 6 vertex), all with Mario's binding layout; every binding decodes. -- **GLSLC outputs in FromSoftware's dvdbnd archives** (Dark Souls Remastered): RomFS is two archives, `dvdbnd0.bdt` and `dvdbnd1.bdt` (3.6 GB), each indexed by the `.bhd5` beside it: `BHD5`, 0xff (little-endian), u32 1, u32 file size, u32 bucket count, u32 0, u64 bucket table offset; each 16-byte bucket `{u32 count, u32, u64 offset}` points at 16-byte entries `{u32 path hash, u32 size, u64 offset}` (paths only hashed). 5,184 entries are DCX containers (`DCX\0`, big-endian header, `DFLT` = a zlib stream at +0x4c), most of them BND3 archives (28-byte entries `{u8 flags, i32 size, u64 offset, i32 ID, u32 name offset, i32 unpacked size}` in formats 0x5c / 0x7c). Six BND3 archives in `dvdbnd0.bdt` hold the shaders, one GLSLC output per entry (`.vpo` vertex, `.fpo` pixel, named `Shader/FRPG_Sfx/NVNShaderBinary/ps/…` or by base name in the two FLVER material archives, 2,602 + 486 entries): `bhd5_entries`, `undcx`, `bnd3_members`, `bdt_shaders` in `shader_split.py`, in 64 jobs by entry range, about 10 s. Only archives that contain a GLSLC magic are parsed (the others use format 0x78). SDK 4's GLSLC needs its section headers read (Dark Souls Remastered in section 6). -- **BAHS** (agl `sharcb`, engine utility shaders): code region at header +0x1c (size +0x18). Binary entries in the table area are 16 bytes `{code offset, 0, control size (0x880/0x88c; 0x848 in BAHS version 9, Super Mario 3D World), signed offset from this entry to the control section}`. Control sections are shared between programs, so pair by entry, never by searching for code near a control. -- **Shorter control sections (SDK 10, Super Mario 3D World + Bowser's Fury)**: the control section stores its own length at +0x20. Mario's, Engage's, Dread's and 13 Sentinels' are all at least 0x880 bytes; 3D World has 2,888 of 0x850 bytes (its ShaderData bfsha and six layouts) and 0x848 in BAHS version 9, each followed directly by other data. The key therefore hashes the first `min(0x880, control +0x20)` bytes (`ctl_len` in `shader_split.py`, `keys_hash` in `hle_nvn.c`); the other games' tables came out byte-identical. -- **Compute code without a header (SDK 10)**: 3D World's particle compute shaders start their instructions at code +0 (like D2R), not after a `0x12345679` header. `shader_split.py` accepts both; `dmsh_build.py` strips only a header that is there. -- **Code area**: graphics = 0x30-byte header (magic `0x12345678`) + 0x50 SPH + instructions (at +0x80); compute = 0x30-byte header (magic `0x12345679`) + **zero padding, instructions at +0x100** (all 71 of Mario's). **Control section**: magic `0x98761234`; +0x700 = c1 offset in the code area, +0x704 = code size, +0x748 = compute thread-group size (u32 ×3), +0x754 = compute shared memory bytes. -- **The program start must be exact.** Ryujinx's decoder treats every 32-byte boundary *relative to the start* as a scheduling word. Mario's compute code was first fed from +0x30: the groups were misaligned by 0x10, instructions were dropped or misread, and 12 compute shaders (particles, post effects) translated into an endless loop that hung the GPU at the first gameplay scene. Check a new title: the first non-zero word after the header must be a control word, and the trailing `EXIT` (`e30000000007000f`) / `BRA`-to-self (`e2400fffff87000f`) must sit in instruction slots of 32-byte groups counted from your chosen start. `dmsh_build.py` now refuses compute code whose padding is not zero. +`dm/gfx/tools/shader_split.py []` scans **every** RomFS file (zstd-decompressed, SARC expanded recursively) and `main`, and recognizes the containers listed by engine in [reference/shaders.md](../reference/shaders.md#containers-by-engine). The pairs table lists the (vertex key, pixel key) of every BNSH variation or Unity program. Unity shaders need a second pass: files whose type trees are missing are scanned again once the bundles have supplied the `Shader` tree (or with the one `unityfs.py` has written out for that hash, when no bundle has it). For a base game plus an update, give RomFS and `main` as comma-separated lists, base first: keys are content hashes, so the output is the union of both versions and one shader build serves both (Mario 1.2.1 recompiled 3,451 of its pixel shaders; v0 rows keep their package indices). Validate a new title's coverage at run time: `DM_NVN_SHDUMP=` makes the host dump the control section and code of every program it can't find in the map; search those bytes in the game files to find the container that was missed. @@ -81,156 +54,9 @@ So the translator leaves z alone (`QueryGraphicsState` depth mode false in `Prog - Not rewriting a GL game: the full-screen copy into the window is clipped away, so the window stays black even with a solid-colour fragment shader. - Metal function constants could pick the variant too, but any function with a function constant has to be specialized before a pipeline can use it (measured: about 2.4 ms per vertex shader on an M4 Pro); the second entry point only costs build time and about 30% more library size. -## 6. Other per-title surprises seen in Mario +## 6. Where the per-title findings are -- **8-bit index buffers**: Metal has only 16/32-bit indices. Widened to 16-bit at replay into the frame arena. Mario's full-screen passes and its final copy use them. -- **Texture swizzle**: `TextureBuilderSetSwizzle` / `TextureViewSetSwizzle` are applied with Metal swizzled texture views (NVN and Metal channel enums have the same values). The font atlas is `(1, 1, 1, R)`; without it glyphs render as white boxes. The swizzle names components of the NVN format, which the format's own mapping finds in Metal's (`nvn_fmt_to_mtl`: alpha 1 for RGBX-type formats; red and blue swapped for the RGB565 family, but only in textures uploaded from guest memory, since what Metal renders into one is in Metal's order; GitHub PR #66, The Legend of Zelda: Echoes of Wisdom's hearts). Metal makes a swizzled view read-only (usage 0x1), so an image slot gets the same view without the mapping (`img_fit`), and the on-chip read merge leaves a swizzled sample alone. -- **Render-target views that change the format**: Mario's last pass writes the `RGBA8` swapchain texture through an `RGBA8_SRGB` view (`TextureViewSetFormat` 0x38 on a 0x25 texture), so the GPU encodes the linear shader output to sRGB on write. The host used to attach the base texture, which stored the linear values unencoded. The whole picture came out darker and more saturated on the Mac and the iPhone: comparing with a Switch screenshot, every channel at every level equalled the Switch value decoded from sRGB once more (0.55 → 0.26, 0.75 → 0.52). `attach` now attaches a Metal view of the view's format, so the pipelines get that format too; the view is kept until the submission completes. A format change that is not an sRGB / linear pair needs `MTLTextureUsagePixelFormatView` on the base texture; without it the host draws in the base format and reports `render target view format … cannot be attached`. `DM_NVN_PASSDUMP` prints these views as `V c view format …`; Mario uses only this one (1 of 80 passes in a course frame). The present step has the matching rule: the Switch scans out the stored bytes, so an sRGB-typed swapchain texture (`RGBA8_SRGB` / `BGRA8_SRGB`) is sampled through a non-sRGB view (`present_raw_src`); sampling it directly would decode it once more on screen only, which a `DM_NVN_SHOT` capture (raw bytes) does not show. Mario's swapchain is `RGBA8`, so this path is for other games (`t_present_crop` covers it). -- **Transient render targets**: the game builds new texture objects every frame over the same storage and never finalizes them. Host textures are shared by `(pool, offset, description)`; otherwise each new object allocates a new private Metal texture (≈13 MB/frame leak). -- **Inline uniform updates** (`CommandBufferUpdateUniformBuffer`) are replayed as in-order GPU copies; reading the uniform buffer from the CPU at encode time gives the frame's *last* value, not the draw's. -- **Compute code areas have a header** in Mario (`0x12345679`), not in D2R; `dmsh_build.py` strips it. -- **Virtual memory pools** (flag 0x200): aliased onto the physical pool named by `MapVirtual` when all requests share one base (Mario's case); otherwise unimplemented. -- **SPIRV-Cross failures**: 2 agl post-effect compute shaders still produce SPIR-V that SPIRV-Cross can't parse (`nullptr` exception). Before the compute start-offset fix it was 7: misdecoded code produced the bad SPIR-V. -- **CPU-written linear textures**: a texture with `NVN_TEXTURE_FLAGS_LINEAR` or `LINEAR_RENDER_TARGET` in a CPU-visible pool (Mario's glyph cache; its four 64×1 RGBA16F sky colour ramps, without which the sky is black) is created on the pool's `MTLBuffer` (`tex_linear`), so CPU writes (Mario's glyph cache) are visible to the GPU. Block-linear CPU-written textures still need the packaged-texture re-upload path. -- **Integer vertex inputs**: without PC reflection the translator declares every vertex input `float4`. Skinned models read their bone index (`R8UI` / `RG8UI` / `RGBA8UI`) as raw integer bits (`as_type(in.m_N.x) * 48`); Metal converted the integers to floats, every bone index was garbage, and all characters were stretched into streaks. `dmsh_build.py` (`int_inputs`) now re-translates a vertex shader with `uint` inputs when every use of an input is an integer bit cast (529 of 4,854 in Mario), and the host makes the Metal vertex format agree with the declared type (`vfmt_for_ct`; signedness swaps of 32-bit components and 32-bit float→int reinterpretation keep the bits, while float-type formats and 8- and 16-bit integers of the other sign go through shadow inputs, below). A remaining mismatch is reported as `vertex attribute format … does not match shader input type …`. -- **Shadow vertex inputs** (`_bv`, `dm/gfx/tools/dmsh_vin.py`): an input that `int_inputs` declares an integer may still be fed a format of another class. A float-type format (half, unorm / snorm, 10:10:10:2, 11:11:10) leaves the converted float32 bits in the register; an 8- or 16-bit integer of the other signedness is widened by the format's own sign (Three Houses below). Metal reads a format only into an input of its own class, so the replay used to convert those attributes on the CPU every draw (`vconv_bind`, red lines 89 and 96): about a quarter of Age of Calamity's replay thread in battle. The build now also writes a side library for each such vertex shader. In it, every integer input is declared again as a `float4` and as the integer of the other sign, at Metal attribute indices 16–30 (NVN uses 0–15), and the body starts with `in.m_N = dm_in.m_N | as_type(dm_in.m_N_vf) | as_type(dm_in.m_N_vo)`. The host sends each attribute to the index of its format's class and the zero buffer to the others (`vin_ct`, `vin_on` in `nvn_mtl.m`; pipeline keys unchanged), so a pipeline with any mix of formats reads Maxwell's register bits. The float shadows are assigned first: with more than 7 integer inputs the last ones get only the float shadow (169 of 8,201 inputs in Age of Calamity). A format with no shadow keeps the CPU conversion, as do transform feedback draws (`main0_tf` has no variant) and `DM_NVN_VCONV_CPU=1`. Age of Calamity: 2,259 libraries, 77 MB each for Mac and iOS. The host loads a side library only for a pipeline that routes an attribute to a shadow, but the build cannot tell which shaders will need one (an input used only as an integer may hold bone indices or float bits), so it writes them all. Other games have 0–2,000 (at most 31 MB on iOS), except Monster Hunter Rise: 16,781 libraries, 344 MB on iOS (21% of its shader libraries, 1.5% of its IPA), and its Mac and iPhone runs so far load none. Its 8- and 16-bit integer formats feeding float inputs (red line 99, about 60,000 vertices per submission on its Mac route into the opening) still go through the CPU, since shadows cover only integer inputs. Age of Calamity's Mac battle at 60 fps against the CPU conversion: 0 vertices converted instead of 545,000 per submission, submit + present 10.0 → 5.7 ms a frame, CPU 3.97 → 2.44 W, GPU unchanged, the same picture. -- **Vertex formats**: Mario uses `RGB10A2SN` (0x69) → `MTLVertexFormatInt1010102Normalized`. Unmapped formats fall back to a 4-component read and print `vertex attribute format … not mapped` once. The integer-read-as-float formats (`*_UI2F` / `*_I2F`, 0x51–0x68), which Metal has no vertex format for, are converted on the CPU at replay like the other conversions (`vconv_i2f`); Breath of the Wild has an `RG8_UI2F` attribute. -- **Instanced array draws** (`DrawArraysInstanced`) carry instance count and base instance in the `OP_DRAW` record. -- **Metal 4 does not order operations inside one encoder.** The host batches consecutive copies, fills and inline uniform updates (`UpdateUniformBuffer`) into one compute encoder. Mario writes the same uniform slot twice in a row each frame (4 bytes of −1, then 16 bytes of wave coefficients); without an intra-encoder barrier the writes occasionally landed in reverse order, the fountain's GPU wave simulation (a 512×1 RGBA16F texture updated by render → copy back → splash pass every frame) ran with −1 as its self coefficient, diverged to ±65504, and its vertex shader pushed the water surface to the top of the screen (tall blue / textured panels that flickered frame to frame). `blit_buf` now inserts `barrierAfterEncoderStages` when an operation reads or writes a guest range an earlier operation in the same encoder wrote; texture copies always get one. `DM_NVN_BLIT_ORDER=0` turns the barriers off to reproduce. Render passes have their own encoder with full barriers, so they were not affected; compute dispatches share the copy encoder and always get a barrier, as texture copies do (Age of Calamity below). -- **A texture slot picked at run time** (Octopath Traveler 0: 61 Unreal vertex shaders, particles reading their data from buffer textures) → the shader reads one of several c2 texture-table words, chosen by branches, and ORs it with a sampler-table word. The translator cannot trace that to one slot and falls back to Ryujinx's bindless arrays, `array, 32> vp_tb_cb0_40000000` and `array vp_s_cb0_40000000`, indexed by the handle's texture ID (low 20 bits) and sampler ID (high 12), capped at 31; the build leaves them without a slot and they used not to load. With the profile key `tex_dyn_slots` the reflection reads them as `NVN_RB_TEXD` / `NVN_RB_SAMPD` arrays (`nvn_refl.m`), the replay writes slot indices into that shader's c2 tables (texture slot r → r, sampler slot r → r << 20) and binds slot k at element k, so the shader's own arithmetic lands on the binding. Check first that the handles come only from c2's tables: in Octopath Traveler 0 every integer read of c2 in those shaders is a table slot (texture slots 0-10, samplers 0-4). The host logs each such shader's first encode with the slots that held textures. Host only: the chain version does not change. -- **Image atomics** → SPIRV-Cross's MSL 2.3 emulation declares the image twice: the texture (`texture_buffer cp_i_cb2_4C_r32ui`) and a `device atomic_uint* cp_i_cb2_4C_r32ui_atomic` pointer to the same memory, which the build read as a second image of the same slot, so the replay put a texture resource ID where the shader wanted an address. Octopath Traveler 0 has 2,084 such shaders (Unreal's GPU particles); three ran in the first 100 seconds. The pointer is now its own kind (`NVN_RB_IMGA`), bound to the storage of the image in that slot: a buffer texture is a view of its memory pool, so the atomics and the texture reads see the same words. A block-linear image is a deswizzled Metal copy whose pool memory has another layout, so its atomics go to a scratch buffer (logged red) until a game needs them followed; a 3D image does not compile at all (above). The scratch buffer is at least as large as the image (`atom_scratch`): SPIRV-Cross indexes the atomics by the image bound in the same slot (`spvImage2DAtomicCoord`: x + y × its width), and in the 64 KB dummy buffer they used before, Pokémon Legends: Z-A's atomic max into a 480x270 r32ui 2D array (fragment shader `2bd5037003980716`, the scene after the story opening's first cutscene) ran about 1.4 MB past its end. They wrote into zero-copy pools over the game's heap (pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced, and the game or the replay crashed) and into unmapped addresses (`kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers; GitHub issue #101). Metal shader validation (`MTL_SHADER_VALIDATION=1`) stops out-of-bounds accesses, so under it the scene played. -- **Buffer textures at offsets below 16 bytes** → Metal reports a 16-byte alignment for texture buffer views, and the host used to leave a view at a smaller offset as a dummy (Octopath Traveler 0: one at pool offset `0x168`, read by the particle shaders above). On an Apple GPU a view at any multiple of the element size reads the right elements (measured on an M6 with R32Float views at offsets 4, 8 and 12), so the host now creates it there; below the element size it stays a dummy. -- **Buffer textures read by 1D integer fetches (row wrap)** → Maxwell fetches a texture buffer and a 1D texture with the same instructions (TLD with integer coordinates only, TLDS `Texture1DLodZero`), and the bound descriptor decides which it is. Without PC reflection the translator has no type answer and reads such a slot as a 2D texture at `(x, 0)`, and the host could only bind a dummy when the game put a buffer texture there: the slot read 0 (`texture type mismatch: shader declares 2, texture is 9`). Age of Calamity has 16,438 such shaders (98,970 bindings; slots 16–21 hold KTGL's light and decal lists). The vendored translator (`InstEmitTexture.cs`, marked `nxnative:`) now addresses every such fetch as `(x & 0x3FFF, x >> 14)` and reports the slot (`fetch1d` in the reflection). The host binds a buffer texture in a flagged slot as a linear 2D texture of 16,384-texel rows over the same pool memory (`tbuf_as_2d` in `nvn_mtl.m`, `nvn_tbuf_rows` in `nvn_tex.c`), cached by texture serial beside the views. A real 1D texture, which the host creates as a single row at most 16,384 wide, reads as before. Limits: a size query on a buffer of more than 16,384 elements returns 16,384 (no shader both size-queries and fetches such a slot in Age of Calamity; the build summary counts them), and the rows stop at the end of the Metal buffer. A mismatch no view can bridge is logged once per shader and slot, with its texture ID, NVN target and formats (`tex_mis_note`). Age of Calamity's 15-minute Mac battle bound 15.7 million of them through 320 row textures, none failed; the scene is darker and more saturated than with the zeros (mean luma 163 against 178 at the start of the battle), GPU time unchanged. -- **New textures start as zero.** Metal leaves a new texture's content undefined; a freshly mapped Switch memory pool reads as 0, and feedback effects (such as the wave simulation above) never clear their state texture. Non-packaged colour textures are zero-filled from a shared zero buffer when created (`tex_zero_fill`, batched with the uploads). -- **Scissored colour clears**: NVN `ClearColor` honours the scissor. Clears are normally folded into the next pass's load action, which only works for a full-target clear; Mario clears sub-rectangles of small targets to zero (pyramid levels packed into 480×272, 256×256 and 64×36 targets). A zero clear outside a pass with a partial scissor is now a copy from the shared zero buffer into that rectangle of the attachment's level / slice (`clear_rect_zero`). `DM_NVN_CLEARLOG=1` logs clears that are still skipped (mid-pass, partial channel mask or non-zero colour with a partial scissor). -- **Inline uniform updates are snapshotted at encode time** (`mtl_sub_update`): the data is copied into a host-owned ring instead of being read from command memory when the GPU runs. -- **Every pixel shader input must be a vertex shader output.** Maxwell attributes are untyped slots: a pixel shader that reads a slot the vertex shader never wrote gets a default. Metal refuses to build the pipeline (`Fragment input(s) user(locnN) mismatching vertex shader output type(s) or not written by vertex shader`), and the draw is skipped. Translating one shader at a time cannot know what the partner reads, so `shader_split.py` writes the pairs of each BNSH variation (`shader_pairs.tsv`) and `dmsh_build.py` gives each vertex shader the outputs its partners read but it doesn't write, as `float4` (0, 0, 0, 1) (`vs_pad`; vertex shaders with partners are finished after the pixel shaders, and the partner list is part of the shader's cache rule). In Mario 331 of 9,043 pairs failed; the opening course used none of them, later levels did (about 11,500 failed builds in one 11-minute session). Check all pairs offline after a build (recipe in [05](05-debugging.md)). The host caches failed pipelines and reports each once with its vertex / fragment keys (`[RED] failed to create render pipeline (vertex … fragment …)`); before that, every draw retried the failing compile. -- **Unwritten components of a written output are (0, 0, 0, 1) too.** A vertex shader may write only part of an output (Link's Awakening writes a world position's x, y and z) while the pixel shader reads the rest (it divides by w). On the hardware the unwritten components read as (0, 0, 0, 1); Ryujinx gets the same by initializing the components the next stage reads when it links stages. The translator's MSL starts every vertex shader with `main0_out out = {};`, all zeros, so `dmsh_build.py` (`vs_init`) sets every `float4` output to (0, 0, 0, 1) right after it and the translated body overwrites what it writes. In Link's Awakening 12,281 of 18,410 vertex outputs never write w. -- **Positions must be invariant across shaders.** A game may draw the same geometry with two programs and rely on identical depth, for example a depth pre-pass and then a less-or-equal G-buffer pass (Super Mario Odyssey). Maxwell evaluates identical position math identically; Metal only does for outputs marked `[[invariant]]` (the translation marks `gl_Position`) when compiled with `-fpreserve-invariance`, which `dmsh_build.py` (Mac) and `ios_metal.py` (iOS) pass to every compile. Without it Odyssey's ground kept stale 32×32 blocks where the G-buffer draw failed the depth test. -- **A geometry shader that only picks the layer** (DRAGON QUEST XI S: Unreal Engine 4.18 builds its 32×32×32 colour-grading LUT with `WriteToSliceMainVS` + `FWriteToSliceGS`, one slice per instance; without it the tone mapper read an empty LUT and the screen came out a flat blue). Metal has no geometry stage. `dmsh_build.py` recognises such a shader in its SPIR-V (`gs_slice`: triangles in, at most 3 vertices out, positions and other inputs copied, `gl_Layer` taking the bits of one input component). It builds no library for it and records the component in its reflection. Each vertex shader that writes integer bits from the instance index into an output component gets two more entry points (`vs_layer_entry`: `main0_l` / `main0_lz` write that component as `[[render_target_array_index]]`). At bind time `prog_fold_gs` (`nvn_mtl.m`) swaps the vertex shader for its layer variant and drops the geometry stage when the components match. Unreal binds each stage from a program of its own, so a vertex shader is often paired for a moment with the previous draw's geometry shader. A stub that does not fit stays in the program, and only a draw that uses it prints a red line (`draw_gs_check`). The other 22 geometry shaders in DQ11 (one-pass cube shadows that emit transformed triangles) still do not build. -- **Cube map arrays count layer-faces** (DRAGON QUEST XI S, when its prologue movie ends: an abort in `MTLTextureDescriptor` validation, `arrayLength (510) … greater than the max allowed size (341)`) → as in OpenGL, an NVN cube map array's depth is its number of layer-faces (a multiple of 6: 510 for 85 cubes), while a Metal cube array's `arrayLength` counts cubes. The host took the depth as a cube count in three places: the Metal texture, the upload's layer count and the guest layout (`nvn_tex.c` `shape`), so `GetStorageSize` reported six times the storage. A cube array is now created like a cube, as a 2D array of all its faces, and `tex_fit` makes the cube-array view a shader asks for (`t_nvn` checks the storage size against a 2D array's). -- **iOS math mode**: the iOS libraries are compiled with `-fmetal-math-mode=relaxed` (`dm/ios/tools/ios_metal.py`; the Vulkan and GL libraries in `vk_shaders.py` and `gl_shaders.py` too). The translated shaders reproduce Maxwell's NaN behaviour with `isnan` selects (8,337 of 12,809 use them), and the default fast mode allows the GPU compiler to assume there is no NaN. Relaxed keeps fast math's reassociation but not that assumption. -- **min / max / clamp with a NaN operand** (every game; GitHub issue 72, Super Mario Galaxy 2 on an iPhone: the half of Mario facing the camera's left was white). spirv-cross writes `FMin` / `FMax` / `FClamp` as `fast::min` / `max` / `clamp`, which leave a NaN operand's result to the GPU; Maxwell's `FMNMX` returns the other operand and its saturation gives 0, as Metal's `precise::` functions (IEEE `fmin` / `fmax`) do, so `dmsh_build.py` (`precise`), `vk_shaders.py` and `gl_shaders.py` write those instead. Plain `max` / `min` / `clamp` compile to the `fast_` intrinsics in every Metal math mode, relaxed included. The NVN chain had about 124,000 `fast::max`, 98,000 `fast::clamp` and 41,000 `fast::min` in Paper Mario: The Thousand-Year Door, none guarded by its `isnan` selects. Galaxy's GX lighting normalizes light directions the game leaves at (0, 0, 0) (the key and fill lights of its characters) and takes `max(0, dot(ldir, NaN))` as the spot term: 0 on the M4 Pro with either function, so the attenuation is 1. The fill light sits at the camera's left in view space, so its terminator runs down the middle of a character facing the camera. - -- **Dynamic present interval (Mario 1.2.1 on the iPhone)**: Mario times every frame (`main+0xcfd24`); after a frame that ran past one vertical sync it sets present interval 2 for the next frame *and* sets its own time step to the interval (a float at `main+0x36ceff0`), so that frame catches up 2/60 s. A user's 2-minute iPhone session switched 123 times, mostly for one frame. In `display` mode those frames were shown for 16.7 ms and the game visibly jumped ahead ("sometimes the speed increases when jumping"). In `game` mode (the default since 2026-10-15) the display-link callback keeps a frame whose period is over 25 ms on screen for its whole period before taking the next one (`g_link_hold_until` in `nvn_mtl.m`; the `display link up to tick` log line counts these ticks as `held for the present interval`), and the 30 fps target that drops the display link to 30 Hz (`nvn_game_fps` in `hle_nvn.c`) waits until interval 2 has held for a full second, so one-frame switches no longer flip the display-link mode. **Fixed: the game side of that hold (BotW on the iPhone, "when pressing B, Link's move pace is speeding up").** The acquire waited for one display-link tick per frame whatever the interval, so the game kept producing frames every 16.7 ms while the screen held each interval-2 frame for 33 ms and dropped the one in between (`old frames dropped by merging`). BotW sprinting flaps between interval 1 and 2 for runs of 6 or more frames, and its VFR time step at interval 2 is a whole 30 fps tick, so those runs went at double speed. `mtl_link_wait_tick` now waits for as many ticks as the game's present interval (times 2 on the 120 Hz link, half rounded up on the 30 Hz link), as the Switch blocks the next acquire for N vertical syncs. The windowless Mac host was never affected: there the acquire sleeps until the interval's slot. `DM_NVN_TSV` writes each frame's interval as `@itv`. `DM_TEST_PRESENT_LAG=90,30` (sleep 30 ms before every 90th present) makes Mario do this on the Mac; run it with `DM_WINDOW=1` to get the display link. -- **Screens whose rate is not a multiple of 60 (144 or 165 Hz Mac displays; from PR #27)**: asked for 60 Hz, the system fires the display link on every third refresh of a 165 Hz screen, 55 Hz, so games paced by it ran at 55 fps (27.5 at 30). There `link_range` (`nvn_mtl.m`) runs the link at the screen's rate and counts as ticks only the refreshes nearest a steady 1/60 s clock: 60 a second on average, each frame shown 12.1 or 18.2 ms at 165 Hz. A tick lies up to half a refresh off its slot, so two ticks can be one refresh nearer each other than their slots: the hold of frames over 25 ms tolerates one refresh more (in a simulation of the callback without it, 30 fps frames held a third tick: 22.5 fps at 165 Hz, 24 at 144), and the frame step lets a frame on time show one refresh longer than its hold (`shown_slack`, `nvn_fstep.h`; a cheap 50 ms frame shows 54.5 ms at 165 Hz, and counted as late it kept Breath of the Wild at 20 fps for good). `DM_LINK_VTICK=0` keeps the system's choice. iOS screens run at multiples of 60 and are not affected. Not yet run on such a screen here; the PR's author measured at 165 Hz. -- **In fullscreen, the screen switched to a rate the frame rate divides (from PR #79)** (`front_mac.m` `front_screen_match`, called from `link_sync` with the link's rate: 60, 30, or 120 with `DM_PRESENT_LINK=2`): virtual ticks keep the average, but a frame still stays up two or three refreshes in turn (13.9 / 20.8 ms at 144 Hz), and Apple's HUD showed the frame interval jumping between them. When the window is fullscreen on a screen whose mode is not a multiple of the link's rate, the host switches that screen to the mode of the same size with the highest rate that is (an XB273U at 2560x1440: 144 → 120 Hz for 60 fps), with `kCGConfigureForAppOnly`, so macOS switches it back when the host quits, crashes included (the PR's author checked: 144 Hz again after quitting); leaving fullscreen, or the screen, switches it back at once. A window keeps the screen's rate and the virtual ticks: the switch changes the whole screen for every app, and most external screens go black for a second or more. The link sets its range again whenever the screen's rate changes (`link_sync`; before, a move to another screen kept the first screen's range), with no virtual ticks after a switch. A screen with no such mode keeps the virtual ticks; `DM_SCREEN_MATCH=0` keeps the screen's rate. The PR's author ran Hades at 144 Hz (switched in a window then): the HUD's frame interval line steady at 16.7 ms, where it jumped before. Here (a built-in 120 Hz screen, a multiple of every link rate) a harness drove `front_mac.m` asked for 50 fps: no switch in the window, 120 → 50 Hz in fullscreen, 120 Hz again on leaving it, and 120 Hz after quitting in fullscreen; at 60 fps and with `DM_SCREEN_MATCH=0` nothing switched. The link's new range after a switch has not run here (Hades windowed and fullscreen paced at 60 as before). - -### Seen in Engage (Unity) - -- **Program control sections are freed after `nvnProgramSetShaders`**: shader keys are computed inside that call ([03](03-bringup.md), graphics entries). -- **CPU-written block-linear textures** (`nvnTextureWriteTexels`: video frames, engine default textures) are uploaded and re-uploaded from guest memory like packaged textures ([03](03-bringup.md)). -- **Video**: Unity's `Hidden/VideoDecode` samples luma as `.w` of an `R8` texture with the swizzle `(0, 0, 0, R)` (Unity's Alpha8) and chroma from an `RG8` texture at half size. To check the movie path apart from the decoder, fill the frames with a gradient in `hle_movie.c`; it should appear on screen. -- **One geometry shader** (in `unity_builtin_extra`): Metal has no geometry stage, so it fails to compile (`unknown main0_out`); nothing in the opening scenes uses it. -- **Fixed: a crash a few minutes into the prologue battle (scripted run, about 357 s).** It wrote to `NULL + 0x1fffe0` at `main+0x9a4664`: Unity's TLSF heap allocator (`main+0x9a45d0`) got no address region for a new 2 MB pool. The alias region was out of address space, not memory; see [03](03-bringup.md), "A null-based write minutes into play". A 12-minute scripted run now plays through the battle. -- **Fixed: GPU memory grew with every scene change** (the idle title demo reached 5.4 GB of GPU memory and 3.6 GB resident in 850 s; it now stays at 1.2 to 1.4 GB). Unity frees memory pools on scene changes and builds new ones, which Mario never did in its first course. Two host structures kept freed pools alive: the residency set still held each pool's no-copy `MTLBuffer` after `nvnMemoryPoolFinalize` (now retired through deferred release), and the storage-sharing texture table (`g_alias`, keyed by pool object, offset and description) never evicted, so the old pools' render targets stayed, and linear textures built on a pool buffer kept the whole buffer alive. Finalizing a pool now evicts its entries (`mtl_alias_drop_pool`). The `memory up to frame` line now also reports `serial objects` (host objects by serial), `freed but alive` (finalized pool buffers still alive) and `shared textures` (sharing table size and evictions); growth in GPU memory with those counts flat points to an allocation outside them. -- **Probably fixed: about one run in four stalled early** (around the opening movie or the title; no log output, not even the time-limit report). The stalls happened before the real thread pause for IL2CPP's GC ([03](03-bringup.md)); since then 12 runs (eight of them 200 s runs through the movie and title, four of 330 to 850 s) have not stalled once, where one in four used to. Without the pause the collector ran while other threads kept changing the heap. A second candidate, fixed later: a race in the dirty-page tracker (`nvn_dirty.c`, ported from d2r-mobile) could leave a page read-only while marked writable, so a game thread writing it spun forever in the fault handler; Engage's movie decoder writes frames into texture storage from a host thread while the replay thread protects pages (`tests/t_dirty_race.c` reproduces it). If a stall comes back, sample the process when its log stops growing (`sample 3`) to see where every thread waits; the crash handler now writes one line with `write()` and exits if the stderr lock stays taken for 1 s, so a stall that holds that lock shows up as such. -- **Fixed: the idle attract demo crashed after about 220 s.** Unity's particle geometry jobs wrote through a NULL index pointer on several job threads at once (`main+0xdb0ad4`, `0xdaea58`, `0xdadb5c`). The scheduler (`main+0xd91e0c`) takes its vertex and index chunks from a fallback that creates and maps a fresh buffer each time (`main+0x7b9c64`; Unity's Switch renderer always takes it, as on hardware), and from about 180 s 32-bit mesh indices also turned up inside Unity's texture objects. The cause was the alias region running out of address space, as in the battle crash ([03](03-bringup.md), "A null-based write minutes into play"): past 2 MB the index buffers got no mapping. With the node-area cut an 850 s idle run is clean, with no corrupted texture objects; with the old SDK the same run dies at 159 s in Unity's heap allocator. Reproduce the old state: `boot.sh 245` with no input on an SDK rewritten without the cut. - -### Seen in Metroid Dread - -- **Movies in native-texture mode**: the decoder writes each frame into an `RGBX8_SRGB` `NVNtexture` that the game hands over in a `movie::Buffer` ([03](03-bringup.md), movie entry). The host writes the texture's block-linear storage on the CPU and the replay layer re-uploads the dirtied pages, as for `WriteTexels`. -- **`WriteTexels` one texel at a time**: the game's set-pixel helper (`main+0x160e80`) writes a 1×1 region per call (probably the map / minimap mask). The host used to decode the whole level, patch it and encode it again for every call, and after a few minutes of play the main thread spent whole frames there (frames stopped at about 175 s). Non-3D regions are now written in place by block-linear address (`write_region_bl` in `hle_nvn.c`). `t_nvn` checks 15 regions in RGBA8, BC1 and a two-layer RGBA16F array, from 1×1 to the whole level, byte for byte against the old whole-level path. -- **Scenes and characters black, only foliage, light shafts and the HUD drawn** → NVN's `BindProgram(program, stageBits)` binds per stage, and a null program unbinds just those stages. Dread's depth prepass calls `BindProgram(program, vertex | fragment)`, then `BindProgram(null, fragment)`, and draws depth with the vertex shader alone. The host replaced its single bound program on every bind, so a null bind meant "no program bound", and all of the prepass's ~140 draws per frame were dropped. The depth buffer stayed empty, and the G-buffer passes, which test depth `EQUAL` without writing, passed no pixel. The replay now tracks the program per stage (`bind_stages` in `nvn_mtl.m`). When the stages come from different programs, or the fragment stage is unbound, it builds a composite program, which gives a vertex-only pipeline for the prepass. Mario's and Engage's binds always bind every stage from one program, so they are unchanged. The combinations Dread uses: (program, 0x3), (null, 0x2), (null, 0x7). Pass dumps now print a `Z` line wherever a draw is dropped (no program, no pipeline, empty scissor, both faces culled, no render pass). That is how this one was found: 144 `Z … draw skipped: no program bound` lines in a frame whose G-buffer pass had draws but stayed black. -- **A draw with the vertex shader of another Shader object** (AI: THE SOMNIUM FILES, Unity 2017.4: `[RED] failed to create render pipeline (vertex 790573a52b9c5053_b fragment f39fb42850f50ecd_b): Fragment input(s) user(locn0),user(locn1) … not written by vertex shader`, once on entering the first investigation) → the vertex shader was `Legacy Shaders/VertexLit`'s shadow caster, the pixel shader `PSYNC/BG_AlphaTest`'s, whose own pass pairs it with `00b462fe0dec6e06`. Unity binds the vertex and fragment programs apart, so every draw's pair is a composite of `bind_stages`, one of a ring of 64. The precompile prediction (`pso_predict_rest`, at a submission's first pipeline miss) saves the replay state, replays the rest of the submission's binds and restores it; in a 280-draw submission it made more than 64 composites, reused the slot the saved `S.prog` named, and the draws after it up to the next bind ran with that slot's new pair. The fusion probe and redraw and the deferred-draw replay restore `S` from copies the same way. Each restore now takes the composite again from the per-stage bindings it restored (`prog_resolve`, logged as `combined program taken again after a state restore`); a draw already under way keeps its `NvProg`, whose release is deferred. Two runs each from one save with one key script, prewarm off: the old host logged `texture type mismatch: shader declares a cube, texture is 2D` both times (a pixel shader run with another pair's bindings), the fixed host no red line, with the slot taken again twice. -- **Cutscenes at 30 fps, gameplay at 60, as on the Switch**: the game keeps present interval 1 throughout. In a cutscene its main loop spins in 1 ms steps until the cutscene's audio clock advances, then renders a frame; that clock moves with a 30 Hz sound thread ([03](03-bringup.md), timer slack entry). -- **One draw per render pass**: the cutscene frames have about 180 passes with one draw each. The game updates uniforms inline (`CommandBufferUpdateUniformBuffer`) between draws, and every copy or update ends the host's render pass. This is correct but costs submit time (about 5 ms per frame on the Mac). - -### Seen in 13 Sentinels - -- **Single-stage programs**: the game builds one NVN program per GLSLC output (838 `ProgramSetShaders` at startup) and combines vertex and pixel programs per draw with `BindProgram` stage bits, so the composite-program path from Metroid Dread does all the pairing. The real pairs exist only at draw time. Every pixel shader input is a plain `float4` at locations 1–9 (the SPH input map, words 6–13, matches the translated MSL inputs for all 375 pixel shaders), so `shader_split.py` pairs every vertex shader with a *cover*: the fewest pixel shaders whose inputs together are every input any pixel shader reads (here one shader). `vs_pad` then gives each vertex shader all of them, and any vertex / pixel combination links. -- **`dmsh_build.py`'s shader cache was keyed without the pairs**: a vertex shader whose pairs changed was restored unpadded from `cas/`. The per-shader key now covers the pairs, and a paired vertex shader's also its pixel shaders' keys (`shader_key`, `pair_keyed`), so a change to a profile's pairs rebuilds what it affects. -- **Triangle fans and indexed quads**: Metal has neither. Non-indexed fans draw through a shared `(0, i, i+1)` index buffer with `first` as the base vertex; indexed fans and quads are expanded on the CPU into a 32-bit triangle list in the frame arena at replay (`nvn_mtl.m`, as for 8-bit indices). -- **Vertex streams 14 and 15**: the host bound NVN stream *n* to Metal vertex buffer 16 + *n*, but Metal has slots 0–30 only (30 is the host's constant fallback), and stream 15 hit the assertion `buffer index (31) must be < 31` a minute and a half into the prologue. Translated shaders use only `buffer(0)`, so streams 14 and 15 now go to slots 14 and 15 (`VB_SLOT`). -- **Packaged texture data from CPU memory**: every texture is created with `TextureBuilderSetPackagedTextureData`; NVN copies that data into the texture storage at `TextureInitialize` (a compressible pool cannot be written by the CPU). The host copies it and marks the texture CPU-written, so it is de-laid-out and uploaded like a packaged texture. Linear textures (the save thumbnails) are excluded: in a CPU-visible pool they are read straight from the pool buffer. - -### Seen in Super Mario 3D World + Bowser's Fury - -- **Shader containers**: bfsha (BNSH) and agl sharcb (BAHS version 9) in `ShaderData/*.szs`, particle effects (BNSH, including 25 compute shaders) in `EffectData`, six layouts and `SystemData/AglResource.szs` (BAHS), and two BNSH in `main`: 11,080 shaders. All bind with Mario's texture handle layout (`0x20 + 8 × slot`); the one failure is a geometry shader (fur fins, `RenderFinFur`), which Metal cannot express. -- **1D textures**: the material shaders sample 64×1 R32F lookup tables created as NVN 1D textures (and filled with `WriteTexels`). Translated shaders never declare `texture1d` (none of the five games' libraries does), so a Metal 1D texture bound to their `texture2d` argument is a type mismatch. NVN 1D and 1D-array textures are now created as one-row 2D / 2D-array Metal textures. -- **Fixed: daylight scenes washed out to white.** The game creates its deferred pre-lighting buffer (the G-buffer's fourth target, R11G11B10F) as a *one-layer 2D array* texture, and the lighting shader samples it as 2D. On Maxwell the sample instruction carries its own dimension and a non-array sample of an array texture reads layer 0; on Metal a 2D array bound to a `texture2d` argument is a type mismatch and every read returned one fixed value, so the whole picture got one pale ambient colour (bloom then blew it out further). The host now compares each texture binding's declared type (the reflection's `dtt`) with the texture it resolves to and binds a compatible view: 2D array ↔ 2D, 2D array → cube / cube array (`tex_fit` in `nvn_mtl.m`, cached per texture-pool entry); any other mismatch is reported once. How it was found: a blit-based pass snapshot of that target showed a correct, colourful image while a lighting shader edited to output only that input showed a uniform colour, and both resolved to the same Metal texture. Earlier suspects that turned out correct: fog, bindings, the 1D lookup tables, clear colours, the depth convention, the final sRGB view, blend states, GPU luminance readback. There is no tone-mapping pass in this game: the final composite writes `scene + bloom` into an RGB10A2 target. -- **Targets the fragment shader does not write**: volume draws after the G-buffer (`50d813761ca3f1f9` and others) write only outputs 0 and 2, with blending and all channels enabled on targets 1 and 3. Maxwell leaves targets outside the shader's output map untouched; on Metal such an attachment gets undefined values blended in. Pipelines now get a zero write mask and no blending for colour outputs the shader does not write (reflection `colors`, `color_known` in `nvn_refl.m`). All five games' regressions were unchanged by it. -- **Open: one scaled texture copy** (1920×1080 → 1280×720, RGB10A2, flags `ENGINE_2D`) when leaving the title: Metal blits cannot scale, so it is skipped with a red line; it is probably the captured frame for the transition. -- **iPhone only: solid magenta blocks in World 1-1 (fix in place; waiting for a device run).** The blocks are exactly (255, 0, 255), hard-edged and aligned to the 16-pixel grid (16×16, sometimes two stacked). `DM_NVN_MAGSCAN` on the A20 Pro found them first in the 1920×1080 RGB10A2 composite (`ser=30`), in the pass whose shader `66a71a3081b91b9a` renders into that texture while sampling it. They kept appearing with the sparkle particle `5e4de7561c670fee` dropped, so that particle, the earlier suspect (picked by position), is not the cause. Writing and sampling one attachment within a render pass is undefined in Metal: the M4 Pro happens to return the old content, while the A20 Pro now and then returns whole 16×16 blocks as (1, 0, 1), probably blocks of the compressed render target caught mid-write. On the Switch the read returns the content from before the draw. The host now ends the pass before such a draw, copies the attachment's level and layer into a snapshot texture in the replay stream, and binds the snapshot for that draw (`fb_prepare` in `nvn_mtl.m`). It does so only for the fragment shaders the profile's `feedback_snap` key lists (`66a71a3081b91b9a`, plus `30e3ccb0bd304ead`, which samples a G-buffer target of its own pass on the title screen). In 1-1 that is one 8 MB copy per frame. Doing it for every shader is too expensive: Breath of the Wild has over a thousand draws a frame that sample the G-buffer of their own pass (Mac frame time 16.6 → 36.6 ms), and Mario Wonder's submit time went from 1.4 to 4.0 ms. Those games' feedback draws (Mario Wonder `a7e5c6fd501cf21f` and `22a61cb1ec732aef` on its RG11B10F scene; BotW `4d3d5a4408c2e8d9`, `753d0169ae3ce4f0` and others on its 1600×900 G-buffer) have shown no blocks on the iPhone so far; if they do, `DM_NVN_MAGSCAN` names the pass and the shader goes into that profile's `feedback_snap`. A sampled view that spans more levels than the attachment's one is left as before with a red line: `3e8567da37eb28a1` renders level 1 of a texture while its view covers levels 0–4 (mip-chain passes). - -### Seen in The Legend of Zelda: Breath of the Wild - -- **Shader containers**: 19,592 shaders: `Shader/*.sbfsha` (BNSH) and the gsys / ksys / terrain resource packs inside `Pack/Bootup_Graphics.pack` and `Pack/TitleBG.pack`, the particle effects in `Effect/*.sesetlist` (11,373 shaders, 101 compute), layouts (`.sblarc`) and 37 in `main`. All bind with Mario's texture handle layout; the full build had no failure and no unresolved binding. -- **Fixed: the 3D scene black, only particles and the HUD drawn.** The engine binds almost every texture with `nvnCommandBufferBindTexturesDeferred` (`main+0xec2e64`, one handle, the pointer being the handle field inside the texture-sampler object). "Deferred" means the GPU reads the handle from that address *when the command executes*; the game records its command blocks once, submits them every frame and writes the handles afterwards. The host copied the handles at recording, mostly zeros, so the deferred-lighting composite (one full-screen triangle per material class, blending into the HDR target) read the dummy texture 0 for all of its inputs. The bind record now keeps the source address (`p_bind_handles.defer / src`) and every replay-time reader refreshes the handles first (`bh_fresh` in `nvn_mtl.m`). How it was found: a frame data-flow table built from `DM_NVN_PASSDUMP_ALL` (per pass: attachments, shader keys, sampled serials) showed the half-resolution lighting outputs never being read, and the composite pass's `B` lines were all `tid=0 dummy`. Without `_ALL` the dump lists only the first draw of each pass, which hid the composite. -- **Fixed: the same texture attached at two colour slots.** The G-buffer passes attach the albedo and another buffer twice each (c1 = c2, c3 = c4); the duplicate slots have write mask 0. On Maxwell the ROP writes only masked channels and the slots do not interfere. On Apple's TBDR each attachment has its own tile memory and the whole tile is stored at the end of the pass, so the copy stored later (loaded, never written) overwrote the other one's writes, in a different order for each 32×32 tile: the albedo held only scattered squares. A slot whose texture, level and layer repeat an earlier slot is now left unattached (`rt_dup_slot`), also in pipeline prediction; a draw that really writes such a slot is reported once (`color slot … holds the same texture as an earlier slot`). Records hold copies of texture objects (`tex_snap`), so slots are compared by serial, not by pointer. -- **Cube textures sampled as cube arrays.** 1,512 shaders sample the environment probes as `texturecube_array` while a single cube texture is bound. Metal can make a cube-array view only of a 2D-array base texture, so NVN cube textures are now created as 6-layer 2D arrays and bound through cube / cube-array views by `tex_fit` (which also maps a cube array read as a cube to its first cube). -- **Fixed: a compressed texture as a render target.** agl's BC4 block encoder (`8e2cc2671cae6020`) writes a 32×32 RGTC1 (BC4) texture through an `RGBA16UI` view (one texel per 8-byte block: BC4 encoded on the GPU). Metal can neither attach a compressed format nor make an uncompressed view of one, and the pipeline-descriptor validation aborts the process, so the attachment used to be left out and its draws did nothing (`render target is a compressed-format texture …`). Such a pass now draws into an uncompressed shadow of blocks × blocks in the view's format: the level's (or layer's, or depth plane's) blocks are copied into it through a buffer before the pass and back after it (`bcrt_attach` / `bcrt_flush` in `nvn_mtl.m`; the method comes from PR #11). The shadows stay resident and are reused, and they are kept out of the load-skip and store-elision bookkeeping. The encoder's output must be declared `uint`: the game has no PC version, so the profile's `shader_facts_manual.tsv` lists it (without it the pipeline fails and the red line prints the row to add), and so does its twin for a 2D source (`93aa5e22a8202409`), whose failure showed up in an iPhone run. -- **Big-stack NVN entry points**: the render thread's stack is small, and Metal's driver compiles internal programs (a fast-clear blit) on the calling thread while the host ends a render pass inside `nvnQueueSubmitCommands`. That overflowed the guest stack (SIGBUS in `AGCDeserializedReply`, address in the guest stack region). All implemented NVN functions except the `nvnCommandBuffer*` recording calls now run on the per-thread 512 KB host stack ([03](03-bringup.md), big-stack entry; `nvn_big_entry` in `nvn_entry.S`). -- **60 fps** is now the host's frame step in whole vsyncs ([Breath of the Wild's notes](../../dm/games/01007EF00011E000/README.md)): a stub at the framework's `setWaitVsyncInterval` hands the game the host's step, and frames stay on screen exactly that long. The notes below are about the method before it, which `DM_FRAME_STEP=legacy` still runs: a rewriter cut on the game's frame-interval logic ([03](03-bringup.md)); the engine's variable frame rate (`ksys::VFR`) scales its time step by the present interval, so the speed is right as long as the interval matches the frames. **Fixed: slow motion when the GPU is the bottleneck (GitHub issue #9).** The interval controller (`main+0x112660c`) measured a frame as framework +0x250, the main thread's CPU span from the start of `procFrame` to before its draw step (`main+0x1126cec` → `main+0x1126d20`): about 2.7 ms in the shrine (probe at `main+0x11266a4`, `@x19+0x250`). Only while the object at `[main+0x1e3a638] + 0x1d2b8` exists did it take max(CPU span, GPU time), in 64 of 6,046 shrine frames. So the interval stepped up only on CPU spikes: an iPhone 16 Pro Max at a 25 ms frame p50 never left interval 1 in 4,800 frames, and the game ran at about two thirds of its speed. The GPU time (framework +0x220) is the difference between the frame's two timestamp reports: `TIMESTAMP_TOP` at the start of the frame's command buffer (`main+0xe94584`) and `TIMESTAMP` at its end (`main+0xe94664`), read after the draw step has waited for the frame's fence (`main+0xe949bc`). It read 0–6 ticks, because the host wrote all 5 reports of the frame's one submission with one time in its completion handler. Two more cuts (`main+0x112691c`, `main+0x1126974`: cbz → b) make the controller always take the GPU path. The profile key `gpu_timestamps segment` makes each report take the GPU start (`TIMESTAMP_TOP`) or end (other types) of the `sub_cut` segment it was encoded in (`rep_ticks` in `nvn_mtl.m`: MTL4CommitFeedback times converted to the guest clock, written in the last segment's handler as before). The GPU time now reads the frame's GPU span, gaps where the GPU waits for the host's translation included. Measured on the Mac with two BotW runs sharing the GPU under a heavy CPU load (load average about 75, a shader rebuild running): GPU time p50 16–23 ms; the fixed run stayed at interval 2 (3 while the GPU time passed 1.9 vsyncs, 1 for under 2% of the frames) at 0.93–0.98 of real time (Σ VFR ratio / 30 per wall second, probe `main+0x151d3e0` `@x0+0x30`: +0x30 ratio, +0x34 interval). At the same time a run with the old timestamps (`DM_NVN_GPU_TS=submission`) spent 10–70% of its frames at interval 1 and ran at 0.78–0.95. The same GPU time drives the game's dynamic resolution (`main+0x1267070`): a running average as a percentage of a 60 Hz frame scales the resolution down above 175% (29 ms) and back up below 150% (25 ms). With 0 it always sat at its maximum, and it still does whenever the game holds 30. -- **Render scale (`DM_RENDER_SCALE`)**: the dynamic-resolution method (`main+0x12666e4`) keeps its scale in s8 (0.8–1.0 in 0.05 steps, or the forced 0.75 / 0.5 of `+0x284 & 0x1800`) and applies it from `main+0x1266880`: the views' scale (`+0x28a0` / `+0x28a4`), from which the size function (`main+0x106340c`) sizes the scene buffers each frame, the shader ratio `uking_dynamic_resolution_ratio` and the other readers. That block runs only when the dynamic path has just recomputed the scale (`+0x280` bit 4, set at `main+0x1266830`) or in a forced mode, else `main+0x126686c` skips it. The stub sets s8 at the block and, for a factor other than 1.0, takes the block every frame, so the factor holds wherever the dynamic path does not run. A multiply in the size function (Tears of the Kingdom's way) would leave the ratio and those readers at the old scale. -- **Fixed: the iPhone app was killed for memory in the first shrine (GitHub issue #9, iPhone 16 Pro Max, handheld).** An 8 GB iPhone gives this app about 3.4 GB (resident plus `headroom to the system memory limit` stayed at 3,375 MB). Resident memory grew about 0.2 MB a frame from 2.6 GB at the shrine load until the kill, while the residency set and the texture counts stayed flat. The cause was the deferred-release drain running on the guest submit thread with no autorelease pool, the Metroid Dread leak (`defer_drain` now has its own `@autoreleasepool`): on the Mac a host without that pool grew 1 MB a frame in the shrine (GPU 3.0 → 4.4 GB in 2,400 frames); with it the shrine stays at 2.7–2.9 GB resident and 1.5–1.7 GB GPU, idle or walking (vmmap at 2.8 GB: about 1.6 GB guest memory, 0.75 GB private textures, 0.17 GB IOAccelerator). Found on the way: (1) about 10 serial-table entries a frame were never deleted: the game records its transient render targets (the HUD captures, the 640×360 buffers) and finalizes them before it submits, and replay put the dead serials back into `g_by_serial` (276,000 after 25,000 frames). Dead serials now go to a 512-entry table (`tex_put`, `g_ghost`; the memory line's `finalized textures still referenced`). (2) The memory-pool tracking table had a fixed 4,096 slots and the phone reached 4,006 pools; a pool past the end would have its textures re-uploaded at every use. It now grows in blocks of 1,024 (`nvn_dirty.c`, up to 262,144). -- **Address → memory pool lookups**: BotW keeps about 1,760 memory pools alive, and `mtl_addr` / `mtl_gpu_addr` scanned them all for every buffer binding of every draw: a CPU sample of the submit thread had about 1,000 of its samples under those scans (vertex buffers in `draw`, uniform buffers in `encode_args`). They now try the last hit, then a binary search over pools sorted by start address (`pool_find`, rebuilt when pools are added or removed). Host submit time in the shrine went from 21 ms to 7 ms per frame (p50), the median frame from 38 ms to 21 ms. -- **Submissions handed to the GPU in segments.** The game does not submit frame N+1 until frame N's GPU work is done: a stack sample in the shrine had the main thread in `nvnSyncWait` (the fence of the frame just submitted) for 59% of the time and the render thread in `nvnQueueSubmitCommands` for 40%. On the Switch that submit costs nothing; here it is the host's translation (about 3,600 draws and 347 passes, 7 ms), so per frame the GPU ran 13–14 ms and then sat idle 7.7 ms (`DM_NVN_TSV` rows `@gpu_beg_ms` / `@gpu_end_ms` against `@call_ms`). The replay now commits the command buffer encoded so far every 64 draws, between two passes (`sub_cut` in `nvn_mtl.m`, called after `rp_flush` in `OP_SET_RT`; `DM_NVN_CUT=`, 0 = off). It cuts only with no open encoder and nothing deferred, merged or hoisted, commits the pending upload batch first, and continues on the same command allocator. The fence signal, the completion work (`post` ops, `g_done`) and the allocator's fence stay on the last segment: the queue runs command buffers in commit order, and every encoder starts with a queue barrier. GPU idle per frame went from 7.7 to 3.8 ms; most of the rest is the game's own work between its fence wait and the next submit (about 2.5 ms). The other five games' frame times were unchanged (Dread's p95 dropped from 19.6 to 16.9 ms). -- **Outdoor frames: passes, not shading (2026-10-21, the user's Great Plateau save, Mac M4 Pro).** A frame had about 386 render passes and loaded and stored about 900 MB of attachments (on an iPhone at 60 fps about 55 GB/s, near the memory's bandwidth); the GPU waits for each pass to finish before the next. 220 of the passes ended only because the game updates the first 64 bytes of a 64 KB vertex uniform block (`0x70c38c0000`) before each draw group while its shaders read up to 544 bytes of it (the terrain pixel shader `eb0afe555b86da79`, which indexes the block with an integer made from an interpolated value, the whole block). The shadows read from memory (`ubp_get`, `DM_NVN_UB_PARTIAL`) and the per-shader read extents of `cb_extent.tsv` keep those passes open: 386 → about 165 passes, GPU p50 about 9.3 → 6.5 ms, submit 5.7 → 4.9 ms; `DM_NVN_UBP_CHECK=1` found every shadow equal to the GPU's memory (968,000 checked). Lazy clears (`DM_NVN_LAZY_CLEAR`) fold the four clear-only passes before the G-buffer into its first pass (about 30 MB less loaded and stored a frame at 1600x900). What still splits the big passes (a `DM_NVN_PASSDUMP` frame dump): the NintendoWare VFX GPU particles run one compute dispatch per emitter between draws (ten 800x450 half-resolution passes and two G-buffer passes, about 190 MB a frame; the G-buffer ones read the depth being drawn, so they cannot move ahead of the pass), and decals that sample the G-buffer right after the game sets it again. The decals sample targets of their own pass (Metal leaves that undefined; the Mac reads the stored content); `DM_NVN_FB_SNAP=all` would snapshot them at a large cost. Tried and dropped: `DM_NVN_TBDR=1` (10–15% fewer stores, more CPU, red ledger lines, and it turns the deferred uniform updates off), `DM_NVN_BARRIER=game` (no gain), same-target continuation outside TBDR mode (the decals end the pass anyway) -- **Frame rate in the Shrine of Resurrection on the Mac (M4 Pro)**, after the above: docked (1600×900) GPU about 13 ms, frame p50 16.7 ms (60 fps; p90 17.6 ms), was 20.8 ms before segmenting; handheld (`DM_OPMODE=0`, 1280×720) GPU p50 9.3 ms and a steady 60. Host translation is 8 ms per frame (22,000 instructions per draw), spread over Metal's encoder creation and draw calls, argument encoding, Objective-C retain / release and pool lookups. The per-pass GPU timer (`DM_NVN_PASSGPU`) is switched off under Metal 4 (`pg_render` returns at once), so GPU time cannot yet be attributed to passes. The frame loads 376 MB and stores 432 MB of render-target data (`@rt_load_mb` / `@rt_store_mb`; 325 of 347 passes load and store everything), which is the next thing to look at. -- **Fixed: dark boxes and cream cloud rectangles behind HUD prompts, a black stamina wheel and a black System menu background (layout captures; GitHub PR #6).** Panes tagged `CaptureOn` (about 140 layouts, e.g. `PaGuideKeySmoke_00`; user data `A8`, `CaptureScale` 0.5, `MultiFilter`) are drawn into a target, blurred at half size and copied into a capture texture that the layout's materials sample in place of the placeholder `FBLayout_00`. The game creates that texture with its per-format default mapping (one call site, `main+0xecb0bc`: R8 → (R, 0, 0, 1), RG8 → (R, G, 0, 1), and so on), then rebuilds it on the same storage with the capture's own mapping: `main+0xf597dc` picks (1, 1, 1, R) for an `A8` capture (the material colour kept, the mask as alpha) and alpha 1 when the pane says `CaptureOutputAlpha` 255 (read at `main+0xf58e98`; an RGB10A2 capture then reads (R, G, B, 1)); `main+0xecacf0` stores the mapping and rebuilds the texture (`main+0xecabe8`), and `main+0xec33f0` replaces the registered descriptor only if `nvnTextureCompare` reports the two textures different. The host compared the description and the storage but not the mapping, so every rebuild was dropped and the captures kept (R, 0, 0, 1), whose alpha is always 1: hard-edged dark boxes behind every prompt (the shadow pane `e6a88e12dbae9743`: black, alpha = capture alpha × vertex alpha 180/255) and rectangles filled with cream cloud (the glow `aa209f57b133e862`: alpha = min(cloud alpha, capture alpha)). Before the cause was known, the profile key `alpha_from_red` listed all 26 pixel-shader variants of the layout library (`main`'s BNSH at 0x1d83000) and bound such render targets through a view mapped (R, 0, 0, R) (`tex_alpha_red` in `nvn_mtl.m`). That fixed the alpha but zeroed green and blue, so tinted panes such as the stamina wheel came out black, and the System menu background (a capture with alpha forced to 1) stayed black. `nvnTextureCompare` now compares the mapping too, an unset mapping counting as (R, G, B, A), and BotW's profile no longer lists `alpha_from_red`. Checked on the Mac by the PR's author (M4 Pro, unchanged shader libraries): green stamina, the purple noise meter and the game's own blurred System menu background. `t_nvn` covers the comparison. -- **BC textures are never attached or viewed raw (GitHub issue #4, reported from an iPhone 14 Pro / A16).** The game encodes one 32×32 BC4 texture on the GPU, drawing its blocks through an RGBA16UI view (one texel = one 8-byte block). Metal can neither attach a compressed texture nor make an uncompressed view of one, so a pass that draws into it uses an uncompressed shadow (above), and `attach`, the pipeline prediction and the sampling views go by the NVN format. The A16 has no BC formats; the host used to store BC textures decompressed on such GPUs (about 3.9 GB instead of 2.8 GB for this game, over iOS's limit on a 6 GB iPhone), and that path has since been removed: only GPUs with BC are supported. -- **Fixed: the sky was brownish grey, the distant haze brown (GitHub issue #52, Mac and iPhone).** The sky is precomputed atmospheric scattering: each frame a pass (`08fe25061d583637`, `uking_pass_shader.sharcb`) turns a 256×32×16 RGBA16F 3D table of in-scattered light into a 256×256 sky map, which the sky draw (`8110c74078548172`) and the materials' aerial perspective sample. The table and two 2D ones (transmittance 256×64, irradiance 64×64) come from `System/KSys/sky.skybin` in `Pack/Bootup_Graphics.pack` (1,212,416 bytes, the three textures' storage back to back). The game makes a `CPU_CACHED | GPU_CACHED` pool for each texture, creates the textures, then copies the file's data into the pools with `memcpy` and flushes them (`main+0x1257d50`, once, at the sky's setup; the flush goes through `main+0xebf4c4`, which flushes only CPU-cached pools). The host created the 3D table as a GPU texture of zeros at its first use, so the sky had no scattered light at all: only the clouds and the sun's colour were left. `DM_NVN_TEXRAW=212,214` (all zero) and `DM_NVN_TEXINFO=256x32` (`storage 1048576 bytes non-zero 781567` at first use, `swizzled=0`) showed it. Every game now fills a new GPU texture from its storage once when the game flushed that storage before the texture's first use, as it already did for a pool the CPU cannot access (`nvn_flushed` in `hle_nvn.c`, `tex_init_fill` in `nvn_mtl.m`): the GPU has not written such a texture yet, so its content is what the CPU put there. Unlike the profile's `tex_flush_upload` it never re-uploads, so a texture the game renders into afterwards stays a GPU texture. In the Great Plateau save it fills that one texture; the flushes (about 15 a frame) merge into about 530 ranges, and frame times did not change. DRAGON QUEST XI S (title, free play, menus; 300 s) filled none. `DM_NVN_FLUSH_FILL=0` turns it off. - -### Seen in The Legend of Zelda: Link's Awakening - -- **Lower-left window origin.** It never calls `nvnDeviceSetWindowOriginMode`, so it keeps NVN's default, lower-left (OpenGL), where NDC y = -1 lands on a render target's first memory row. The six games before it all switch to upper-left. Read as upper-left, every texture the game renders and samples came out flipped and cancelled out, so the scene looked right; but text it renders into textures for the title and the menus came out upside down, and it copies the frame with `MIRROR_Y`. Viewport and scissor rectangles, fragment coordinates, copies and texture coordinates address the same memory rows in both conventions, so the host only flips the direction inside the viewport (negative viewport height), reverses the front-face winding, and flips the window image at present and in raw screenshots (`g_origin_ll` in `nvn_mtl.m`). -- **Copies a blit cannot do** (`copy_t2t_draw`): RG11B10F → RGBA16F every frame, half-size depth-stencil copies (depth and stencil are written from a fragment shader, stencil through `[[stencil]]` export from an `X32_Stencil8` view of the source), and a scaled `MIRROR_Y` copy of the frame to 1280×720 RGBA8. Each is one render pass per layer; this also covers Super Mario 3D World's scaled copy when leaving the title. -- **Fixed: missing palm tree and flowers** (the pipeline `147c829e79fb322f` + `0ebba5ff6a44aa52`, `shadowmap_pbr.bfsha` variant 641, failed to build). The translation declared vertex inputs 1–5 as `uint4` because the shader only moves their bits (x bit-cast to an integer, y copied as a float), and the game feeds them 16-bit normalized formats, which Metal will not pair with an integer input. These are the foliage's shadow-map and depth pre-pass draws; the main pass shades with depth test EQUAL, so without the pre-pass the plants never appeared. The profile's `shader_facts_manual.tsv` declares the five inputs float (the bits Maxwell would see), which changes only this profile's chain version. Four outdoor shaders with the same loop (`03c43da3e3f46da8`, `079700f3577e311f`, `15c02683a10f8e9d`, `2344874045b1e758`) failed the same way on the iPhone in Mabe Village (Half2, RG8 / RG16 normalized) and are listed too. The guess is not wrong in general: 1,437 of the game's vertex shaders have integer-typed inputs, many of them bone indices the shader multiplies as integers, so overrides are added only for shaders seen failing with float formats (`failed to create render pipeline … Cannot convert attribute …`). -- **Fixed: no ambient occlusion (stencil lost in the depth downsample).** The occlusion pass (`aa05724ffa62cb82`, a 480×270 R8 target cleared to 1.0) only writes where stencil bit 1 of the quarter-size depth-stencil copy is clear. The copy left the stencil as the new texture's undefined content (new depth-stencil textures are not zero-filled), so on the Mac the mask kept 1.0 everywhere and the scene had no occlusion; on a device the result depends on whatever that memory holds. The copy now carries the stencil. -- **Fixed: flickering hard-edged white tiles on the step of Marin and Tarin's doorway (Mac and iPhone).** The room shaders use tiled forward lighting: the fragment position picks a 64×64-pixel tile (`trunc(y / 64) × ceil(width / 64) + trunc(x / 64)`), whose light count and light indices come from two storage buffers that a compute shader (`light_culling_compute.bfsha`, one thread per workgroup, one workgroup per tile) fills every frame. The culling shader computes its tile index as workgroup y × `c0[0x200]` + workgroup x, reading the dispatch's workgroup count from NVN's driver segment `c0`. The host passed that shader an all-zero `c0`: `dmsh_build.py`'s `drv_reads` scan matches only fragment-shader names (`fp_c0` / `fp_c2`), so vertex and compute shaders that read `c0` were never marked. Every row's workgroups wrote row 0's entries, the last writer won (a different one each frame), and the bottom tile row, where the step is, got the lights of other rows. A dump of the tile lists showed only the first of 17 rows filled. The fix is in the host: a declared driver-segment binding now counts as read (`nvn_refl.m`), and for compute `c0` carries the dispatch size at 0x200 (x, y, z) and the uniform buffer table at 0x210 + 16 r (address, size) in front of the storage buffer table at 0x310 (`encode_args_`). After it every tile row has its lights and the doorway shows the game's soft light from outside. Breath of the Wild (101 compute and 4 vertex shaders read `c0`) and Super Mario 3D World (11 and 1) render the same as before on the Mac. -- **Fixed: outdoors the whole screen was blurred (Mac and iPhone).** The depth-of-field composite (`52fcbb4ca2a78907`, `postprocess.bfsha`) blends the sharp frame with two blurred levels by a blur amount computed from the half-size depth copy: `pow(clamp((c5[2].y / (depth + c5[2].x) − c5[3].x) · c5[3].y), c5[3].z)` above a screen line, by screen position below it. The game sets depth mode near = 0, but every vertex shader was translated for GL, so all depths sat in 0.5..1 and the computed distance put everything out of focus. Fixed by the per-mode entry points in §5. Found with a save file from the iPhone (Mabe Village outside the house), `DM_NVN_PASSDUMP` / `DM_NVN_TEXDUMP` of that frame, and LASRecomp, which rewrites z only for the GL mode. -- **Fixed: outdoors was dim and nothing cast a sun shadow (Mac and iPhone).** The sun's shadow map (2048×2048, depth 32-bit float) was correct, but every lookup into it failed, so the whole scene was lit as if in shadow. The surface shaders compute the lookup from a world position the vertex shader passes in `locn24`, divided by its w; the vertex shader writes only x, y and z of it. The translation started that output at 0 instead of the hardware's (0, 0, 0, 1), so w was 0 and every compare value was infinite (found with `DM_NVN_TEXRAW` on the shadow map, the light matrices from `DM_NVN_CBDUMP`, and a hand-edited copy of one surface shader that wrote the compare value and the stored depth as its colour). The fix is general: every vertex shader output now starts at (0, 0, 0, 1) (see "Unwritten components" above). Compared with forcing the compare to pass (`compareFunction` always): the same brightness, plus shadows from the fences, the house and the trees. -- **Open: `SR_Y_DIRECTION`.** 2,487 of the game's pixel shaders read the window's Y-direction system register, which is −1 under a lower-left origin. The Ryujinx recompiler answers a constant +1 (a TODO in `InstEmitMove.cs`; LASRecomp, a static recompilation of this game, answers −1). In the room shader the value only scales a depth slope under `abs()`; other shaders may use its sign (`dFdy`-based effects). Answering it per profile means a translator change, i.e. every game's chain. -- **Open: two pixel shaders do not translate** (`9d8f6b8543355374`, `9338e3f1b597fe7c`: variants 0 and 1 of one program in `postprocess.bfsha` @0x252000; SPIRV-Cross throws on a null pointer). No run so far has loaded either. -- **GLSLC.** `subsdk0` is NVN's runtime GLSL compiler and `main` imports `glslcCompile*`, but up to the house, the beach and the menus every shader the game loaded was in the offline tables. - -### Seen in Fire Emblem: Three Houses - -- **Two texture pools and two sampler pools with overlapping IDs**: the scene binds a 12,544-entry texture pool and a 3,856-entry sampler pool, the 2D UI's command buffer binds 4,096-entry ones, and both fill IDs from 256. The replay had one ID table, so the scene's default 8×8 white texture overwrote the UI's textures: the title menu's labels drew as plain bars, other menus were empty except for the cursor. Each pool now has a base in the table, in creation order; `RegisterTexture` / `RegisterSampler` add it, and binds recorded on a command buffer add the base of the pool that command buffer last set (`bind_handles` in `hle_nvn.c`, the base kept in `cmd_t`; log line `pool #2 (…): numbered from N`), and handles read from uniform buffers take the base of the pool bound at replay (see Dark Souls Remastered below). Deferred binds are not remapped (red line). The table's capacity is the sum of the pools: the scene registers each draw's textures into fresh IDs of a ring (8,729 and up a few seconds in), which used to read as empty slots and left the screen black. -- **An integer vertex input fed a float-type format**: `int_inputs` declares an input `uint` when its first translation only bit-casts it, and KTGL's character and soldier shaders then take it back with `as_type`; the game binds a Half2 there (UVs). Maxwell's fetch converts by the format and the register holds the float32 bits. Metal refuses such a pipeline (`Vertex attribute m_36(5) of type uint4 cannot be read using MTLAttributeFormatHalf2`: the characters' shading draws were dropped, black silhouettes), and reading the raw bytes as integers gives denormals (UVs collapse onto one texel: flat tan characters, also Age of Calamity's soldiers). The replay now converts such attributes (half, unorm / snorm 8 and 16, 10:10:10:2, 11:11:10, BGRA8 into an int / uint input) on the CPU into float32 bits, four components with the fetch's (0, 0, 0, 1.0) defaults, in the frame arena, and binds them as UInt4 / Int4 on buffer slots 1–12 (`vconv_bind`, once per range and layout per submission; red line 89). Shadow vertex inputs (above) have since replaced that conversion; it remains for the formats they cannot take. -- **Depth clamp**: the game turns `SetDepthClamp` on; it is now recorded (`OP_DEPTH_CLAMP`) and replayed as Metal's depth clip mode (it used to be ignored). -- **Fixed: the ground under close-up battle cameras was flat steel blue.** The terrain's vertex shader (`56778feb1cfdee7e`) reads its grid position from an `RGBA16I` attribute: x and z the grid cell, y the height, which spans the whole int16 range. So does the side pass that draws the river surface into targets of its own (`db725c85ca060403`). `int_inputs` declared that input `uint4`, and Metal refuses a signed format for an unsigned input, so `vfmt_for_ct` swapped `Short4` for `UShort4`. That keeps the 16 bits but widens them with zeros: every negative height read as 32,768–65,535, and those vertices went far up out of view. Maxwell widens by the format's signedness, whatever type the shader gives the register. Where the ground was missing nothing wrote the depth (reversed-Z, cleared to 0), and the lighting takes depth 0 for sky and paints the fog colour. The G-buffer colour targets are never cleared, so they kept an old image there: cobblestones of a mesh under the terrain, which from the map camera showed through the gaps like a road. 8- and 16-bit integer formats whose signedness differs from the input's now go through the CPU conversion like the float-type formats (`vconv_int` in `nvn_mtl.m`, red line 96 once; since replaced by the other-sign shadow inputs, above): 32-bit values extended by the format's own sign, missing components (0, 0, 0, 1) as integers. How it was traced, since two steps misled: a `HILITE` colour over the gap was stale too (the never-cleared albedo), and the end-of-frame depth showed 0 there. Recomputing every draw's clip positions from `CBDUMP` data showed that no draw of the shader reached the gap. Then the side pass's layer, which showed only scattered tiles of the river, led to the vertex format. Ruled out on the way: depth clamp (Metal's clip and clamp modes gave the same depth), the pass without a depth attachment (142595 is a screen-space decal pass: it samples the depth and writes albedo and normals), polygon offset (none), dropped draws. **To confirm:** 64-pixel rectangles in the albedo where decals land. The decal pass (`417a523053457352`, slots 16 and 18) and the lighting (`094a2d11253f126c`, `f54830a75e07e14a`, slots 18–20) read their decal and light lists from texture buffers that the translation declared `texture2d`, because those handles have no PC type (`texture type mismatch: shader declares 2, texture is 9`); skipping the decal pass removed the rectangles. Since row wrap (above, general) they read the real lists: slot 18 holds 1,052,640 elements, 65 rows. On the Mac prologue route the torches and burning houses now light the battle map and the close-up ground (they lit nothing before), and the blood decals take the scene's light instead of showing as dark blue patches. The rectangles did not appear on that route in either build (no 64-pixel structure in the close-up frames), so the scene where they were seen still has to be checked. -- **Fixed: the first loading screen took 78 s on the iPhone (5 s on the Mac).** KTGL's loader uploads each resource with a copy on the queue followed by `nvnQueueFinish`, hundreds of times per loading screen (effect banks, face models). `mtl_finish` used to call `mtl_present_flush` first, and with a window that waits until the display link has taken every queued present: one to two display frames per call. The windowless Mac runs present at once, so they never showed it; `DM_WINDOW=1` does (one step took 20 s instead of 1.7 s). `nvnQueueFinish` now waits only for the GPU work submitted so far, as on the Switch, where it does not wait for the display. A swapchain texture is still not handed back before its previous present is processed (`AcquireTexture` waits for that ticket), and `mtl_drain` keeps its flush. Windowed Mac run after the fix: first load 6.1 s, frame pacing unchanged. How it was found on the device: `Documents/env.txt` with `DM_WATCH=50` (thread table at 50 s: only the yield-polling main thread used CPU) and `DM_FSLOG=` (the reads came in bursts, with stalls of 4 / 24 / 13 s right after the effect banks). -- **Run-time GLSLC**: the 2D primitive, text and loading-feather shaders come from the game's own compiler at boot and are captured once on the Mac ([03](03-bringup.md), SDK 6). - -### Seen in Super Mario Odyssey - -- **Layout captures as in BotW.** Odyssey's layouts rebuild their captures through the same agl code (`main+0x982320` replaces the registered descriptor only if `nvnTextureCompare` reports a difference), so the comparison fix above covers them too. The profile's `alpha_from_red` (all 26 layout pixel shaders, `main`'s BNSH at 0x132c000) had turned the coin counters' solid white cards into the right shapes. With the fix it no longer fires: on the Mac smoke route to frame 20,100 it converted no texture (main's host converted its first, a 640×80 R8 target, at frame 4129), and the HUD at frames 17,300 and 17,600 is pixel-identical to main's. The list stays until the change has been seen on an iPhone. -- **Fixed: the Cap Kingdom fog never parted around Mario (reported on an iPhone; fix checked on the Mac).** On the Switch the ground fog opens around Mario as he walks through it. The fog's shading reads velocity fields that a small fluid simulation updates every frame: two 256×256 and two 128×128 RG16F targets, with a two-pass blur and a blended composite. Its only input is a point-sprite splat per mover, drawn with `alPointSpriteShader`: one vertex per mover, its size in the vertex data (Mario's main splat asks for about 18 px, others 3–5 px) and a soft disc in the fragment shader (`clamp(1 - 4r², 0, 1)` × velocity). The vertex shader writes `gl_PointSize = min(size, c0[0x610])`, and the host had nothing at that offset of the driver segment, so every splat had size 0. The simulation fields stayed exactly zero (raw dumps with `DM_NVN_TEXDUMP` + `DM_NVN_SNAPRAW`; the 8-bit dump shows velocities as black either way) and the fog stayed whole. `c0 + 0x610` holds the largest point size, and the host now writes 2047 there for the graphics stages. Writing the `SetPointSize` value instead (always 1 in this game) left one-pixel splats whose disc weight came out 0 (the fields held `-0.0`). After the fix the fields hold velocities up to ±44, and the fog clears around Mario. The build's `drv_reads` scan in `dmsh_build.py` only matches fragment-shader names (`fp_c0`), so vertex and compute reads of the driver segment are not listed in the JSON; the host treats a declared driver-segment binding as read (`nvn_refl.m`). -- **Fixed: the Cap Kingdom ground showed 32×32 blocks of a stale image on every fourth frame (Mac).** Presents 4k were brighter and blocky, 4k+1 and 4k+2 clean, 4k+3 partly blocked, repeating to the decimal across runs. The blocks were already in the HDR target when the G-buffer pass ended, and only with the ground's material (`b6384a85afd3ef94`) drawn. The cause: in some frames the game first draws a depth pre-pass (`alRenderDepthOnly`) for objects its occlusion queries report visible (the queries rotate through three slots, hence the cycle), then draws the G-buffer with other vertex shaders and depth test less-or-equal, without clearing the colour targets. Metal compiled the two vertex shaders' identical position math to slightly different depths, the G-buffer fragments failed the test, and those pixels kept the previous content. Every Metal compile now uses `-fpreserve-invariance` (see "Positions must be invariant" above). Ruled out on the way: the TBDR optimizations, same-pass feedback, the present path; evaluating the game's conditional rendering on the CPU from earlier frames' results made it worse (the condition belongs to the same frame on the GPU), so the host still ignores the condition and draws. - -### Seen in The Legend of Zelda: Tears of the Kingdom - -- **Fixed: the 3D scene was black (conditional rendering).** TOTK wraps nearly every `ClearColor` / `ClearDepthStencil` in `SetRenderEnableConditional(RENDER_IF_NOT_EQUAL, rec)` … `SetRenderEnable(true)`. Each clear is followed by `ReportCounter(TIMESTAMP_TOP, rec)`, the CPU sets `rec` to 1 when a target needs a clear, and `rec + 16` is never written. The condition compares the 64-bit counters at `rec` and `rec + 16`. A timestamp report puts 0 in the counter field (the time goes in the second field), so after one clear the two are equal and later conditional clears of that target are skipped. The host ignored the condition and wrote the time into both fields, so every clear ran. The first tile-lighting round, the half-resolution scene depth the light volumes test against (back faces `GREATER` against a buffer just cleared to 1.0 can never pass) and the light accumulation were all cleared before the composite read them. Profile key `render_cond`: `OP_RENDER_COND` is evaluated at replay from the records in memory. A timestamp report replayed earlier in the same submission counts as 0, and a condition on a samples-passed result (an occlusion query, known only when the GPU completes) keeps drawing. A pass dump shows the switches as `E` lines (`conditional render … → not drawn`) and a skipped clear as `c … skipped by conditional rendering`. -- **Fixed: the cave was magenta (textures filled with no NVN flush).** The terrain draws sample two 121-layer ASTC 8×8 arrays (42 MB each) that TOTK creates on zeroed memory and then fills from the CPU, with no `nvnMemoryPoolFlushMappedRange` or `FlushTexels` over them (its cache maintenance runs natively). `tex_flush_upload` never marked them, so they stayed GPU-only with content 0, and Metal decodes an all-zero ASTC block as the error colour (1, 0, 1). A zero BC block decodes black, which looked like missing light. Profile key `tex_dirty_upload`: `ft_add` write-protects a registered texture's pages, and on first use (submit thread) a write since then makes it CPU-written. `DM_NVN_TEXINFO=WxH` tells the cases apart: it prints how many bytes of the texture's guest memory are non-zero at first use. -- **How the tile lighting was traced**: the G-buffer class byte selects per-class half-resolution passes through a depth buffer of class values (`gl_FragDepth = mix(0.5, 1, nibble-swapped class / 255)`, then full-screen or per-tile quads with depth `EQUAL`), so `DM_NVN_DEPTHOVR` (depth test forced to `ALWAYS`) with `HILITE` tells "rejected by depth" from "never rasterized". `TEXSNAP` copies used to run before the pass they were meant to show (classic-queue command buffer committed ahead of the Metal 4 one); they now go through the replay's copy encoder. -- **Clears inside a pass, with a partial channel mask or with a partial scissor** were dropped (red lines for unimplemented mid-pass, partial-mask and partial-scissor clears). NVN clears obey the scissor and the clear's own channel or stencil write mask, and can come between draws, while a Metal load action clears a whole attachment at the start of a pass. Such clears are now drawn as one triangle over the pass in the open render encoder (`clear_color_tri` / `clear_ds_tri` in `nvn_mtl.m`, adapted from MacNativeGames, MIT). The pipeline has the pass's formats, the clear's mask as the write mask of its slot and no writes to the other slots. Depth is `ALWAYS` (and written for a depth clear), stencil `REPLACE` through the clear's write mask, the scissor is clamped as for draws, and visibility counting is off for the triangle. The value comes from the frame arena through the triangle's own argument tables, and the draws' tables are put back after it. A clear outside a pass opens the pass as a draw would. Clears a load action can express keep it, and two cases are still cleared whole, logged once as `depth-stencil clear with a scissor or partial stencil mask, cleared whole`: a depth / stencil clear with a scissor outside a pass, and a partial stencil write mask. TOTK issues the first (a depth clear scissored to 8×8 of a 64×64 target), and so does Wonder. With `render_cond` TOTK's scissored colour clears are all skipped; without it they are 50×50 tiles of a 500×400 target cleared to opaque black, about five a frame. Metroid Dread logged the red line once while loading on its smoke route. -- **8-bit indices written earlier in the same submission** were read stale (a red line: the 8-bit index buffer had GPU-side writes in the submission). Metal has no 8-bit index type, so replay widens 8-bit indices to 16-bit on the CPU. When a copy, update or fill encoded earlier in the submission covers them (`pend_hits`), the CPU reads the bytes from before that write. Those draws are now widened on the GPU (`wide_run`): a compute pass reads the indices from a 16-byte-aligned address, with the byte shift as a constant, and writes 16-bit ones into the frame arena. As in `ixc_run`, the first such draw widens every such draw after it up to the next buffer write outside a render pass, so only a draw met inside an open render pass with no widening yet would split it. In TOTK about one draw per widening, up to a dozen widenings a frame on the title and load screens, no pass split (log line `8-bit index GPU expansion`). Draws whose indices no write covers keep the CPU widening. -- **Fixed: the stone blocks and steps of the opening cave got no torch light (Mac and iPhone).** On the Switch they are grey stone lit warm on the side facing the torch; here they kept their texture and the ambient light but stayed black. Their fragment shader `54e8c08998050e79` takes its roughness from a 5120×5120 BC4 atlas (roughness = the square root of its red, packed into the upper six bits of blue in colour target 3, which the tile-light shaders read); the atlas was all zero, roughness 0 gives no point light. TOTK fills the atlas on the CPU: it fills 1024×1024 BC4 material textures from the CPU with no NVN flush, reads them back with `nvnTextureReadTexels` on its own threads before any draw uses them, and copies the blocks into the atlas. `tex_dirty_upload` only marked such a texture CPU-written when the submit thread first resolved it, so `ReadTexels` returned the never-uploaded host texture: zeros, and the atlas got zeros. Now `mtl_read_texels` reads any texture whose content is in guest memory there (a packaged layout, a CPU-written texture, or a `tex_dirty_upload` texture written since registration), on any thread, without an upload; and `nvn_ft_poll` (`hle_nvn.c`, at each submission after guest memory changed, at most 128 MB of textures per look, going on where it stopped) marks registered textures written after their first use, because the atlas itself is filled after it was first bound. Textures the GPU renders into or writes (`mtl_tex_gpu_owned`) leave that list without an upload. Found by running MacNativeGames' build of the same game on the same save: their atlas held data and their blocks were lit (their `ReadTexels` reads guest memory). How it was traced: `DM_NVN_TEXDUMP` (the blocks' roughness bits in colour target 3), the shader's MSL, `DM_NVN_TEXRAW` on the atlas and `DM_NVN_TEXINFO` (guest bytes), then lldb reading the atlas memory inside their process. Fast loop: the in-cave autosave (`run/save`, keys `f2500:A,f2800:A,f3100:A,f3600:A`, shot at frame 4500); `DM_DETCLOCK` cannot be used (TOTK stops at frame 1 waiting for guest time). -- **Fixed: draws into a 32×32×32 BC4 3D texture were lost** (`render target is a compressed-format texture …`). agl's BC4 block encoder (`0544282c162305db`, the same shader Mario Wonder ships) encodes the texture one depth plane per pass through an `RGBA16UI` view. It now draws through the shadow described under Breath of the Wild, with its output declared `uint` in `shader_facts_manual.tsv`. A raw dump at frame 1500 of the new-game route: the texture was all zeros before; now all 32 planes hold encoded blocks. -- **Fixed: a draw with no render target.** An agl fragment shader that only stores to a storage buffer draws one triangle into a 1×1 viewport with no colour or depth target set (it samples a 64×64 cube map, and the game reads the result back from the buffer). The host found nothing to attach and skipped the pass (`no render target could get an MTLTexture`); it now opens an attachment-less pass at the viewport's size (`bare_dims` in `nvn_mtl.m`, from PR #11). Compared at the same frames of the new-game route, the cave's rock and stone blocks now get ambient light they lacked, and Zelda's bloom halo in the descent cutscene is gone. -- **Integer fragment outputs.** With no PC version the translator has no output types and declares every output float, and Metal refuses a pipeline whose float output meets an integer attachment. The profile's `shader_facts_manual.tsv` declares them: the BC4 encoder above and the Ultrahand mask (`e3fb0bb58a691e10`, R8Uint, found by PR #11; the Mac route does not reach it). When another one fails, the pipeline red line prints the row to add. `nvnDeviceRegisterFastClearDepth`, a hardware hint, now returns success instead of reaching the red stub. -- **Texture layout (open):** MacNativeGames found that NVN chooses the automatic block height by truncating `height_in_blocks * 1.5 / 8`, while `nvn_tex.c` rounds the division up, so some sizes (a 160×160 BC3 texture: H3 instead of H2) are read with the wrong layout. Rhythm Heaven Groove confirmed it on its two 40-block-row textures (below), and Touchable (a d2r-mobile port of the same game) on 16 of its texture files; `nvn_tex.c` now truncates for every game (`pick_tile(ah * 3 / 16)`). It changes layouts at 6–10, 17–21, 38–42 and 81–85 block rows, which matters only for data the game lays out itself (no packaged layout: Tears of the Kingdom's flushed textures, Monster Hunter Rise's sparse ones); those were not re-checked. - -### Seen in Pokémon Legends: Z-A - -Game Freak's Trinity engine on SDK 19 (NVN 55); every shader comes from the Oodle-compressed packs (section 1). Each item was found from a red line or a pass dump on the way through the language screen, character creation and the first minutes in Lumiose City: - -- **Characters collapsed into a spike (transform feedback skinning).** The character preview drew one thin spike. Trinity skins on the GPU with transform feedback: a vertex-only program (28 such shaders, e.g. `681cc42e48c0bbc0` in the `special/imgui.trsha` pack; skin indices in attribute 12, weights in 13) runs under `SetRasterizerDiscard` between `BeginTransformFeedback` / `EndTransformFeedback` and writes positions (location 0 xyz, 12 bytes a vertex) and packed normal and tangent bits (locations 1 and 2, x only, 4 bytes) into three feedback buffers; the character draws then read those buffers as vertex streams 0–2 (`RGB32F`, `RGB10A2SN`, `RGB10A2SN`). The host ignored all of it, so the streams kept whatever was there. The pass dump gave it away: a pass of 19 draws with a vertex shader and no fragment shader (`fs=-`) before the character pass. Metal has no feedback stage, so: - - the layout comes from the program's control section, where the NVN driver would read it (`tf_layout` in `dmsh_build.py`): +0x6dc u16 strides, +0x6e8 per-buffer enables, +0x6ec buffer count, and per buffer b the captured output (0x20 + n) at +0xdc + 0x80 b and its component count at +0x4dc + 0x80 b. Every buffer in this game holds one output from x on (strides 12/4/4, 12/12/12, 12/4, 12). Any other form gets a red line and no capture entry point, so one shader cannot stop a game's build (Tears of the Kingdom's 14,766 and Ace Attorney's 2,475 vertex shaders have no feedback at all); - - such a vertex shader gets a third entry point, `main0_tf` (`vs_tf`): the same body, returning nothing, storing the captured components as their bits into `device uint *` buffers at Metal vertex buffer slots 8–11 (`TF_SLOT`), indexed by vertex ID; the reflection lists `entry_tf` and `tf_strides`; - - the binds, begin / end / pause / resume and rasterizer discard are records now (`OP_TF_BIND`, `OP_TF_STATE`, `OP_RAST_DISCARD`). At replay a draw under rasterizer discard runs `tf_draw` in `nvn_mtl.m`: with feedback active and a capture entry point, an encoder of its own with no attachments (the game draws these while its frame target is bound), a pipeline with rasterization off, the buffers bound at their current offset less first × stride, the offsets advanced and the written ranges queued as GPU writes (`pend_add` kind 4); without feedback the draw rasterizes nothing, as on the GPU (it used to draw, depth writes included). Only plain array draws are captured; indexed, indirect or instanced ones give a red line. `DM_NVN_PASSDUMP` prints the records as `Y tf …` and each capture as a `T … tf capture` line. -- **A crash in the driver's compute encoder end, two minutes into a new game (a feedback encoder beside the copy encoder).** `SIGSEGV` in `-[AGXG18GFamilyComputeContext_mtlnext performEndEncoding]` (from `rp_begin_ex_`, or in its `dealloc` on the encoding thread) about frame 3390 on the Mac, with the encoding thread on and off. Metal validation (`MTL_DEBUG_LAYER=1 MTL_DEBUG_LAYER_ERROR_MODE=nslog`) named it: "Command buffer already has an open command encoder". `tf_draw` opened its render encoder while the copy encoder (`g_blit`) was open, which the inline updates `ubd_land` copies there leave behind, as does a copy or a dispatch just before it; every other helper pass ends `g_blit` before it opens its own encoder, and now `tf_draw` does too (GitHub issue #101). -- **A layered pass whose pipeline could not be built.** Two full-screen vertex shaders (`57246f9a04af8529`, `b29f08619f379a83`: one triangle per instance, layer = instance + offset, 20 pixel shaders read the layer) write `render_target_array_index`. Metal builds such a pipeline only with an input primitive class, and the translator writes a point size in every vertex shader, which needs the point class: no class fits both. `vs_layer` drops the point size from vertex shaders that write the layer, and the host gives their pipelines the triangle class once Metal names the reason (`layer_out` in `pso_get_` and in the background prewarm). -- **`fragmentFunction main0 cannot be used to build a pipeline state` (a Metal validation assertion).** SPIRV-Cross declares an optional function constant, `spvLinearTextureAlignmentOverride`, in shaders that read a texel buffer (one imgui pixel shader here), and Metal refuses any function with function constants until it is specialized. `shader_load` specializes such functions with an empty set of values. -- **Integer fragment outputs.** Two system pixel shaders write an ID into an `R8_UI` target on the character creation screen; their rows are in the profile's `shader_facts_manual.tsv`. A scan for outputs built from integer operations was not trusted: the translated code keeps float bits in integer variables (register reuse), so such a scan cannot tell an integer output from raw float bits stored into a float target. -- **A 2D texture in a cube slot** (a placeholder for an environment map) has no cube view: the zero cube is bound instead of a type mismatch with undefined reads (`tex_fit`). The same for a 2D array of fewer than six layers, which is how the host holds some NVN 2D textures: Mario Kart 8 Deluxe binds one (format 0x44) at slot 10 of two pixel shaders that declare a cube there as a race loads, and it was bound as is (`texture type mismatch … declares slot 10 as Metal type 5, texture … is type 3`). -- **Black world (layered rendering without a layered pass).** The 3D scene came out black once the opening reached Lumiose. The game builds its 32×32×32 colour grading LUT with `b29f08619f379a83`, one instance per depth plane, each writing its plane through `render_target_array_index`, and tone-maps the frame through that LUT. A Metal vertex shader selects the attachment layer per primitive only when the render pass has `renderTargetArrayLength`, so every plane went to plane 0 and the rest stayed black. A draw whose vertex shader writes the layer now opens a layered pass (`g_rp_layered` in `draw`, `g_pass.layered` in `rp_begin_ex`): the pass gets every layer the attachments have from their attached one on (3D textures: depth planes), and it is split when the next draw's shader differs. `DM_NVN_PASSDUMP` prints `L … layered pass: n layers`. -- **Striped lighting (a stencil-mode texture view).** Every lit surface had black and white bands that followed depth, as if every pixel were in or out of shadow at random. The deferred lighting pixel shader (`ead6e2a32540b19a`, 31 bindings) first reads the depth-stencil buffer through a view whose `nvnTextureViewSetDepthStencilMode` is `STENCIL`, tests stencil bit 6 and discards. The host ignored the mode and bound depth, so the test read bit 6 of the depth value's float bits. `tex_by_id` binds an `X32_Stencil8` view for such a view. The shader declares a float texture (the translator has no integer texture type) and takes the raw bits with `as_type`, and a read of the integer view gives the stencil value in those bits. Metal allows such a view only of a texture with the pixel-format-view usage: Apple GPUs read it correctly without, but the API validation layer aborts at the first one (PR #106, Luigi's Mansion 3). Upgrading the texture when the view appears recreated the depth buffer while it was attached, and the next occlusion-counted draw crashed inside `setVisibilityResultMode`, so every depth-stencil texture has that usage from the start (`tex_desc`; Vulkan depth-stencil images too, `vkCreateImage`), which costs nothing measured (on an M4 Pro the same allocated size and GPU time); `t_stencil_view` checks it under validation. Inputs ruled out on the way: the G-buffer albedo, the normals (12-bit pairs packed into 3 bytes, noisy by design), the screen-space shadow mask, the scaled depth copies and conditional rendering (`render_cond` changes nothing in this game and stays off). -- **Characters' legs folded back from the hips once they walked (the base instance in the driver segment).** From the first step on, the player's legs stuck straight out behind her at hip height, and the rival's forward in a later cutscene; standing still beforehand looked right. The bone palette the game computes on the CPU was already bent, so the GPU side only drew it. The cause was six occlusion queries a frame, one 36-vertex box each (`a793f436dcabd350`, base instance 61, 62, …), drawn against mip level 2 of the depth pyramid: every query came back with 0 samples, and the game took its characters for hidden and posed their legs that way. Forcing every result to visible fixed the legs, and depth testing with ALWAYS did not, so the boxes never rasterized where they should. The box shader picks its transform with (instance index − base instance) + `c0[0x644]`: Maxwell's instance ID leaves the base instance out, and the NVN driver keeps it in its constant segment, where GLSLC reads it for `gl_InstanceID + gl_BaseInstance`. The host left `0x644` at 0, so every box was box 0. `encode_args` now writes the draw's base instance there for graphics stages and does not reuse a stage's argument block across draws with different base instances; 793 of the game's vertex shaders read it. The word before it, `0x640`, is added to the vertex ID by 58 shaders and stays 0 until what the driver puts there is known. `drv_reads` in `dmsh_build.py` only scans fragment-shader names (`fp_c0`), so these reads never showed up in the reflection; `nvn_refl.m` treats any declared driver-segment binding as read. How it was found: the raycasts were logged with scratch observation gates (all correct), then `DM_HLECOUNT` showed nothing walking-specific, then the counter reports. -- **Sampler reduction (min / max).** The game builds that depth pyramid with a min-reduction sampler (`SamplerBuilderSetReductionFilter 1`; `151607c7a92386cc` samples the level above at an explicit LOD), which the host used to average. Metal 26 has the same thing (`MTLSamplerDescriptor.reductionMode`), but ignores it with a nearest or no mip filter, so reduction samplers filter mips linearly (`samp_by_id`): at the integer LODs reductions are used with, that is the same level. -- **`newBufferWithBytesNoCopy` refused under memory pressure.** A memory pool created while the game maps and unmaps its 4 GB region was sometimes refused once, then accepted. `mtl_pool_add` retries up to 12 times with a short back-off and prints `[RED] memory pool … created at retry N`; on a final failure it logs the VM regions around the pool. +What earlier titles needed beyond sections 1–5 is filed by mechanism: textures, passes, clears, vertex input, uniforms, conditional rendering and the Vulkan and OpenGL layers in [reference/rendering.md](../reference/rendering.md); containers by engine, control sections, translation, bindings and run-time shaders in [reference/shaders.md](../reference/shaders.md); memory pools, command memory, queues, syncs and CPU-written textures in [reference/nvn-objects.md](../reference/nvn-objects.md); present intervals and display rates in [reference/frame-rate.md](../reference/frame-rate.md). Each entry names the games it was found on, and each game's own findings (its passes, shader keys, measurements, profile keys) are in its note, `dm/games//README.md` ([games table](../../dm/games/README.md#games)). Search [symptoms.md](symptoms.md) for what you see first. ## 7. Diagnostic switches (host) @@ -274,197 +100,3 @@ Game Freak's Trinity engine on SDK 19 (NVN 55); every shader comes from the Oodl | `MTL_DEBUG_LAYER=1 MTL_DEBUG_LAYER_ERROR_MODE=nslog` | Metal validation without aborting | A first-frame bring-up that worked for Mario: screenshot is black → `TEXDUMP` shows which targets have content → `PASSDUMP` the pass that writes the window → `HILITE` its fragment shader → `CBDUMP` its uniforms → read the translated `.metal` of its vertex shader. - -### Seen in Hyrule Warriors: Age of Calamity - -- **Compute has a binding of its own.** `BindProgram(program, stageBits)` with the compute bit used to replace the one bound program, so a draw after a dispatch found the compute program (or none: `no program bound before the draw`, `program has no vertex shader`) and was dropped, and a dispatch after a graphics bind found no compute shader (`the dispatched program has no compute shader`). The replay now keeps the compute program apart (`S.cprog` in `nvn_mtl.m`, and `g_pp_cprog` in the pass analysis). The game interleaves dispatches and draws; its intro movie's NV12 → RGB draw was dropped and the movie played black under its "+ Skip" prompt. -- **Images registered on their own**: `nvnTexturePoolRegisterImage` writes the same table as `RegisterTexture` (image handles are pool indices). -- **Compute dispatches share the copy encoder** (`dispatch_enc`). After its first cutscene the first battle runs about 900–1,500 small dispatches a frame on the Mac (an iPhone log's call counts suggest more), each after a game barrier, at the same draws and passes. Each dispatch had a compute encoder of its own with full queue barriers on both sides, a GPU drain per dispatch: about 10 µs of GPU each on the Mac (GPU 9 → 18 ms a frame); the iPhone's went from 15 to 51 ms (17 fps). They now go into the open copy encoder, each after a barrier inside it (as texture copies are, `blit`), with the compute argument table set once per encoder: about 1.3 µs each, GPU 7.3 → 8.1 ms across the cutscene. `DM_NVN_DISPATCH_OWN=1` (and `DM_NVN_PASSGPU`, which times each dispatch) gives each its own encoder again. -- **Texture views cached by texture** (`vc_get`). KTGL registers a texture into the next ID of a ring before every bind (`RegisterTexture`, `GetTextureHandle` and `BindTexture` counts are equal: 8,000 a frame in battle, up to 28,000 after the cutscene). Only 1 in 6 registrations repeated what the ID held, so the per-ID view (`treg_t.view`) was rebuilt for half of all binds: `newTextureViewWithPixelFormat` took 45% of the replay thread. Views are now also kept by texture serial and view parameters (4 per texture), checked against the current base texture on each hit, and dropped when the texture or its pool is finalized; a registration identical to what the ID holds keeps its view and the registration generation. 2,036 views were made in a 15-minute Mac battle run, 211 million reused; the replay before the cutscene went from 14.3 to 10.8 ms a frame. -- **Indirect draws and dispatches**: `nvnCommandBufferDrawArraysIndirect`'s record `{count, instances, first, base instance}` has the layout of `MTLDrawPrimitivesIndirectArguments` (and `DrawElementsIndirect`'s `{count, instances, first index, base vertex, base instance}` that of `MTLDrawIndexedPrimitivesIndirectArguments`), so the replay issues Metal indirect draws from the pool buffer: the GPU writes the record earlier in the frame, so the CPU cannot read it at replay. The game's GPU-driven effects are indirect quads, which Metal has no primitive for. `ixc_run` in `nvn_mtl.m` converts such records (quads, fans, and indexed ones with 8-bit indices) with a compute pass into indexed indirect arguments in the frame arena, and the draw indexes the shared quad / fan index buffers (indexed ones get a 32-bit list expanded on the GPU). A compute pass cannot run inside a render pass. So when the first such draw is met, it converts every such draw up to the next buffer write outside a render pass (copy, update, fill, texture → buffer, dispatch; they bump `g_wr_epoch`). Each record is then read after its last writer, and only a draw met inside an already open render pass splits it. `DispatchComputeIndirect` → `dispatchThreadgroupsWithIndirectBuffer`; when the shader reads the driver segment, the GPU copies the counts into c0 + 0x200 first. `DM_NVN_IXCLOG=` logs the first n conversions (record → arguments) once the GPU has run them. -- **Movie frames through flushed CPU writes**: the decoder's NV12 output is copied by the game into a 1280×720 R8 and a 640×360 RG8 texture in a CPU-visible pool and published with `nvnMemoryPoolFlushMappedRange`; the profile turns on `tex_flush_upload`. Buffer textures (a 1,048,576-texel R32F one) stay out of that path: they are views onto the pool buffer, and an upload would have sent them past Metal's 2D size limit. - -### Seen in Monster Hunter Rise - -- **Subgroup operations in vertex shaders** (about 4 % of them): `SPIRV-Cross threw an exception: Subgroup builtins are not available in this type of function`. Metal has no SIMD-group functions in vertex functions, and SPIRV-Cross's own subgroup emulation covers only `Elect`. RE Engine scalarizes uniform loads: ballot the active lanes, shuffle the first lane's value to all and take a fast path when all agree. With a one-lane subgroup that is exact, so `dmsh_build.py` rewrites the vertex shaders' SPIR-V that way before SPIRV-Cross (`spv_single_lane`: the ballot is the lane's own bit, shuffles and broadcasts return its own value, all / any are the predicate, all-equal and elect are true, lane ID 0, size 1). -- **Geometry shaders** (53) do not translate: SPIRV-Cross writes `unknown main0_out main0()` for the geometry stage, and Metal has none. -- **Indirect-count multi-draws** (`nvnCommandBufferMultiDrawElementsIndirectCount` / `MultiDrawArraysIndirectCount`): the scene is GPU-driven. Compute culling writes the draw records (stride 20) and a u32 count, then up to `max` draws (1–49 per call) are made from them. Both were empty, and the G-buffer pass had no draws: the title screen showed only the UI. Metal has no draw count read by the GPU, so each call is recorded as `max` ordinary indirect draws that carry the count's address and their index (capped at 1,024). A compute pass copies each such draw's record into the frame arena, or zeros past the count, and the draw reads the copy, so a draw past the count draws nothing (`mdx_run` in `nvn_mtl.m`). Like `ixc_run`, it runs at the first such draw with no copy and takes every later one up to the next write that could change a record or count; copies are keyed by record and count address, since the game draws the same records with two counts. Draws of one multi-draw replayed back to back are encoded without the rest of `draw` (`mdx_next`). The replay used to commit its work and wait for the GPU before the first such draw after each GPU write, then read the count on the CPU (`mdi_wait`, now only for quads, fans and 8-bit indices). In Kamura Village that wait held the submitting thread 14 of its 20 ms a frame on an iPhone (36–54 fps). On the Mac the same scene's submit took 20 ms of wall time for 6.1 ms of work, and now 5.1 ms, all of it work. -- **Textures in the sparse virtual pool hold what the CPU wrote into physical pages before mapping them**: the streamed textures (flag 0x20, packaged layout) and plain sampled ones with no packaged layout. The replay took the plain ones for GPU textures, so their Metal copies stayed 0: the hunter's innerwear (ASTC 6x6) came out magenta, the ASTC error colour, and the face black. A sampled texture in a sparse pool (not a render target, image, linear or buffer texture) is now uploaded like a packaged one, through the pool's page table (03, sparse pools). -- **Sparse textures have a layout of their own** (`nvn_tex.c` `level_walk`): they are mapped in 64 KB tiles, one row of blocks 128 GOBs / (block height × block depth) wide. A level at least one tile wide, high and deep is laid out with its width padded to whole tiles, the levels from the first smaller one on form the mip tail (packed as usual), and the storage is whole tiles, 64 KB aligned. The game's files hold its streamed textures pre-swizzled that way, and it places each level at the offset the host reports. Laid out unpadded, the 1024×1024 ASTC 6×6 clothes of Hinoa and Minoto (43 GOBs wide instead of 48, level 1 at 0x81000 instead of 0x90000) read 2,183 all-zero blocks in level 0 and 3,043 in level 1, which ASTC decodes as its error colour: their trousers, sleeves and gloves were magenta, worst in Kamura Village. `DM_NVN_ZBLOCK` finds such textures. -- **Sparse textures are sampled only where the game has mapped them**: the translator drops TEX's LOD clamp and its residency predicate, so streamed textures were sampled at levels the game had not mapped yet, which read 0 (magenta on the title screen's clothes). A sparse texture is bound as a view starting at its first level whose pages, and those of every smaller level, are all mapped (`sparse_view` in `nvn_mtl.m`). That samples exactly as a minimum LOD would; the view follows each upload. -- **Bindless handles in uniform buffers written earlier in the same submission**: the replay reads handles on the CPU, and a write queued earlier in the submission has not landed yet. Inline uniform updates keep where their data sits, so the handle is read from the newest one covering it (`pend_read`). The game queues more than the old limit of 256 writes per submission, after which every address counted as written; pending writes are now indexed by 4 KB page (8,192 per submission). -- **Clear of a texture made without render-target use** (a 32x80 packaged one): a whole-level clear upgrades the texture to render-target use first, as format views are upgraded (`tex_upgrade`). -- **Storage buffer addresses in the game's own uniform buffer** (about 30 compute shaders: c3 + 0x38): Ryujinx reports such a buffer with `cb_slot` 3, which had no NVN slot. The build marks it `sbuf_ubo` (uniform slot `cb_slot - 3`, byte offset), and the replay reads the 64-bit address there at bind time, as it reads bindless handles. -- **Bindless handles through a loop phi** (218 pixel shaders): the handle is `c2[0x1A0] | 0x10500000` (separate texture slot 7, constant sampler 0x105), kept in a loop variable. Ryujinx's bindless elimination compared only the phi's direct inputs, gave up, and turned the access into an index into a 32-entry texture-pool array (`fp_t2d_cb0_40000000`, handle type Direct), which the replay cannot bind (`no resolved slot number`, the shader does not load). The vendored translator now follows the phis that feed a phi, and treats two `constant buffer | constant` handles as the same value ([third_party/ryujinx/README.md](../../dm/third_party/ryujinx/README.md)). Those accesses become ordinary type-3 handles. - -### Seen in Super Mario Galaxy (Vulkan) - -The game renders through Vulkan (`dm/gfx/vk/hle_vk.m`, written for Hades) and needed these additions; each was found from a red line or a screenshot on the route to the Star Festival: - -- **Shaders generated at run time**: Hagi emits SPIR-V for each GX configuration and creates every pipeline of its pipeline database at boot (3,089 modules, 4,535 pipelines). `DM_VK_CAPTURE=` writes the modules; `vk_shaders.py` reads capture folders like a RomFS. Its SPIR-V puts `ArrayStride` on the struct member (`OpMemberDecorate`) instead of the array type, which NVIDIA's driver accepts and SPIRV-Cross refuses ("Struct member does not have ArrayStride set"): `fix_strides` copies it onto the type before translating. -- **Push constants**: SPIRV-Cross binds the block at `buffer(0)`, where the first renumbered uniform buffer goes; the translator moves it to the next free buffer slot (`p:0.0=` in the table) and the replay sets the recorded bytes there (`setVertexBytes` / `setFragmentBytes`, the pipeline layout's range). -- **Descriptor arrays**: the vertex shaders read the Wii's transform memory as arrays of uniform buffers (4 position-matrix blocks, 2 normal, 2 dual-texture) and the fragment shaders 8 combined image samplers; SPIRV-Cross gives an array n consecutive slots, so the table says `slot*n` and the replay binds element k at slot + k. -- **Descriptors hold the Metal objects** (Super Mario Galaxy 2, GitHub issue 68): the replay binds every element of an array the shader takes, and a set may still name an image view the game has destroyed (the hardware reads a descriptor only when a draw samples it). The layer's descriptors were pointers to its own view, sampler and buffer structs: Galaxy 2 crashed in `bind_res` on an iPhone, reading a freed view's texture from element 1 of the 8-texture array when the ship took off after the first galaxy. A descriptor now holds the texture, sampler state and buffer it was written with (retained), as a hardware descriptor holds a copy of the handle. Through the opening and the first storybook stage the game allocates 22,785 descriptor sets and frees none, and destroys only 5 image views (and 1,244 images), so a stale element is rare: a Mac run that kept destroyed objects to catch their binds found none there. -- **Dual-source blending from location 1**: the GX shaders write the second blend source as an ordinary output at location 1 (no `Index` decoration), and pipelines blend with `SRC1` factors; NVIDIA's hardware feeds MRT 1 to source 1. Metal refuses the pipeline ("Fragment shader does not write to render target color(0), index(1)"). `vk_shaders.py` builds a variant `vk__d` with that output at `[[color(0), index(1)]]`, which the host takes for pipelines whose blend reads source 1. -- **GX vertex formats**: positions and colours arrive as integers read as unnormalized floats (`R16G16B16_SSCALED`, `R8G8B8A8_USCALED`, …) and in three-component formats; Metal's integer vertex formats convert to the shader's float type. A buffer layout no mapped attribute uses is left out: Metal's validation aborts on it. -- **Viewport and scissor persist across render passes** within a command buffer, as Vulkan says; the layer used to reset them to the render target at every pass. In handheld mode the game sets a 1280x720 viewport once and draws through several passes into its 1920x1080 swapchain, so its menus came out 1.5x too big (the file-creation dialog cut off at the right edge). -- **Copies inside a render pass**: the game copies from its EFB with `vkCmdCopyImage` between `vkCmdBeginRenderPass` and `vkCmdEndRenderPass`, which NVIDIA's driver runs. Render passes now start with deferred store actions; a copy stores every attachment and ends the encoder, the next draw resumes the pass with load actions, and the pass's own store actions are set when it really ends. Before, the rest of the pass was dropped (`draw outside a render pass`). -- **Swapchain**: the game asks for `preTransform` INHERIT and asserts (a stripped trap, `main+0x4d26e0`) when the surface does not list it. In handheld mode it keeps a 1920x1080 swapchain, draws into the top-left 1280x720 and calls `vi::SetLayerCrop` every frame; the host keeps that crop (`hle_vi.c`) and the Vulkan present shows only that rectangle. -- **Linear images**: staging images (`VK_IMAGE_TILING_LINEAR`, filled by the CPU and copied with `vkCmdCopyImage`) and one sampled 4x4 image alias their memory as on the Switch: the texture is made over the bound `MTLBuffer` with Metal's linear row alignment, which `vkGetImageSubresourceLayout` reports. -- **Queries**: two timestamp pools and a 3,072-entry occlusion pool. Timestamps are their submit's completion time; occlusion queries count into a per-submit visibility buffer, copied to the pool when the submit completes. -- **Not translated**: two geometry shaders (GX points and lines expanded to quads); those draws come out as one-pixel points and lines. GX's sampler LOD bias has no Metal sampler equivalent (red line). - -### Seen in Rhythm Heaven Groove - -Nintendo's own 2D engine, SDK 21.4.0: one BNSH (`shader/shader_NX.bnsh`, 2 vertex and 33 pixel shaders), Mario's binding layout (every binding decodes), no packaged textures. - -- **Fixed: black screen, every sprite and glyph in place under `HILITE`.** The sprite shader (`40adb3021c79b155`) computes texture × colour multiply + colour add and discards a fragment whose alpha is 0. The uniform buffer was right (a `CBDUMP`: multiply 1, add 0), so every texture read 0. The game reads each texture file into memory and makes a `CPU_NO_ACCESS | GPU_CACHED` pool (flags 0x21) on it: 245 pools by the first screens, each holding one texture at offset 0, 17 block-linear BC7 (flag `IMAGE`, no packaged layout) and 228 linear R8 (`IMAGE | LINEAR`). NVN keeps what a pool's storage held at `nvnMemoryPoolInitialize` as its content, but the host uploaded from guest memory only packaged, `WriteTexels` or flushed textures, and made a linear texture on its pool's buffer only in a pool the CPU can see. Now a new texture in a `CPU_NO_ACCESS` pool whose storage holds data starts with that content, uploaded once (`tex_init_fill` in `nvn_mtl.m`); it stays a GPU texture that can be rendered into, with no re-upload on page writes. A linear one is made on the pool's buffer in any pool (`tex_linear`). Left out: storage of zeros, `COMPRESSIBLE` textures (render targets here, in their own 0xa1 pools; their memory means nothing without the compression tags), multisample, depth / stencil and sparse ones. Found by a census of every texture's pool flags, flags, format and whether its storage held data at creation (a temporary line under `DM_NVN_LOG`). -- **Fixed: the latency calibration's beat numbers torn and its background in blue steps.** Two BC7 textures without a packaged layout were read with the wrong block height. A dump of the guest bytes deswizzled in Python with 1 to 16 GOBs and decoded with `texture2ddecoder` shows the right one by eye; a smoothness score does not, because a block height too tall reads padding and looks smooth. Truncating height in blocks × 1.5 / 8, as MacNativeGames found (above), matches every size checked by eye: 672×160 and 1232×160 (40 block rows: 4 GOBs, where rounding up gives 8), and 112×112 (4), 288×208 (8) and 880×432 (16), where both roundings agree and `floor(rows / 8)`, the other rule that fits 40 rows, does not. `nvn_tex.c` now uses it for every game (see the Tears of the Kingdom entry). - - -### Seen in FINAL FANTASY TACTICS - The Ivalice Chronicles - -Square Enix's engine on SDK 19.3.9, NVN (`main` imports only `nvnBootstrapLoader`; the bundled OpenGL driver `subsdk0` is not loaded). 306 shaders: 257 GLSLC outputs in the engine's technique files (`resources_nx/release/master/shader/*.tec`, magic `TEC\0`, each output whole after a table of parameter names) and 49 in `main`. `shader_split.py` reads GLSLC outputs in any file no other reader takes. Mario's binding layout (combined textures at `0x20 + 8 × slot`, images at `0x120`); every binding decodes. One geometry shader (`bahamuteffect.tec`) does not build, as Metal has no geometry stage. - -- **Fixed: black screen; the UI drew in the right places under `HILITE`.** The UI reads its textures from `.tex` files the engine loads straight into a `CPU_UNCACHED | GPU_CACHED | COMPRESSIBLE` pool (flags 0xa2): 231 textures by the title screen, no packaged layout, no `WriteTexels`, no copies. Writes to an uncached pool need no flush, so neither the flush path nor `tex_init_fill` (which only takes `CPU_NO_ACCESS` pools) saw them. The profile sets `tex_flush_upload` and `tex_dirty_upload`: the registered textures' pages are write-protected, and a write after `nvnTextureInitialize` uploads them. -- **Fixed: translucent UI (the frosted text bands, button guides, dialog headers) yellow and white.** The last passes have no render target: fragment shaders composite each UI layer into the window texture with image stores, blending in software. The composite (`uirender.tec`, `4cf1975ed1e36590`) loads the window texture's texel as `r32ui`, unpacks the four bytes and stores the blend through a float view of the same image. The host bound the RGBA8Unorm texture to the `texture2d` argument as it was, whose reads are undefined (Metal's validation layer does not check textures in argument tables). An image slot the shader declares as uint / int now gets a view of the same texel size in that class (`img_fit` in `nvn_mtl.m`). The format-view usage costs lossless compression, so a texture gets it only when a shader needs such a view: it is upgraded (copied into a texture with the usage) before the next draw pass opens, as the copy cannot run inside the draw's own pass; until then that one draw reads it as before. -- **Fixed: dialog boxes with holes the size of their text, and white specks where text faded.** Each UI element is drawn into its own layers and then composited into the window image by one draw per layer, all in one attachment-less pass, each a read-modify-write of the image, with `nvnCommandBufferBarrier(ORDER_FRAGMENTS | INVALIDATE_TEXTURE)` (bits 0x12) between them. The host took a barrier only as an order for the next pass, and Metal orders nothing between draws of one render pass for shader writes: the text element's composite read the window before the parchment had landed and wrote the old pixels back. A barrier with `ORDER_FRAGMENTS` or `ORDER_FRAGMENTS_TILED` now ends the open pass when a draw in it wrote an image or a storage buffer (`swrite_note` in `nvn_mtl.m`), so the next draw opens a pass behind the full consumer barrier; reflection marks an image binding written when its type has write access (`nvn_refl.m`, which left `rw` unset for images). -- **Fixed: `texture clear case not implemented … mask 0x1`.** The game clears an R8 render target with channel mask 0x1. A mask that covers every channel the format has is a whole clear (`mtl_pf_chmask`). -- **New NVN entry points:** `nvnCommandBufferBindTextures` (the plural of `BindTexture`), `nvnSamplerPoolRegisterSamplerBuilder` (a sampler registered from its builder), `nvnTextureWriteTexelsStrided` (`WriteTexels` with row and image strides) and device item 0x67, which the game reads before every program it loads and uses as the alignment of its copies of the shader (answered 256, not measured). - -### Seen in Absolum - -FNA (XNA 4 in C#) compiled with .NET NativeAOT, SDL3, and FNA3D's NVN driver (`FNA3D_Driver_NVN.cpp`) on SDK 18.3.2. The effects (`.xnb`) are XNA's D3D9 effects (MOJOSHADER is linked in); the shaders the GPU runs are 163 GLSLC outputs in `rom:/shaders.nx.bin` (a u32 count, then per entry a u32 hash, a u32 size and one whole output), which `shader_split.py` reads as a loose file no other reader takes: 162 unique shaders, 83 pixel and 79 vertex, every vertex shader paired with the single-stage cover. Every binding decodes with the usual layout. - -- **Fixed: the whole picture upside down** (the title logo and "Press Any Button" mirrored top to bottom). The game keeps NVN's lower-left window origin (handled as for Link's Awakening) and, twice a frame, sets viewport 0's swizzle with `nvnCommandBufferSetViewportSwizzles` to (x, −y, z, w), which negates clip-space y: how a renderer with OpenGL heritage flips its picture. The host had it as a red-reporting stub. It is recorded now (`OP_VP_SWIZZLE`) and applied at the draw as a viewport flipped about its centre, the same mapping. Facing is decided after the swizzle on the GPU as in Metal, so the winding stays. Other swizzles (axis exchanges, negated x / z / w) give a red line and are drawn unswizzled. -- **Fixed: the logo movie as a strip of small grey copies at the top**: NV12 bytes in an RGBA texture ([03](03-bringup.md), movie entries). - -### Seen in Captain Toad: Treasure Tracker - -Nintendo EPD Tokyo's engine (the `al` library over sead / agl, as Super Mario 3D World), SDK 8.2.0. 7,258 shaders: bfsha (BNSH) in `ShaderData`, agl sharcb (BAHS) and layout BNSH in `SystemData`, particle effects in `EffectData` and 40 BNSH in `main`, all with Mario's binding layout (every binding decodes). The one that does not build is the fin fur geometry shader, as in 3D World. `main` imports `glslcCompile` and holds agl's GLSL sources, but no module defines it, so nothing is compiled at run time; `shader_split.py` now asks for captured GLSLC outputs only when a module of the profile's defines `glslcCompile`, as `profile_guess.py` does. - -- **Same-pass feedback on the Mac too.** agl's post-effect program 987 (`54f58a7cab608766`, the same program as 3D World's `66a71a3081b91b9a` in another build) writes the 1920x1080 RGB10A2 image as an attachment while sampling it. In 3D World the 16x16 magenta blocks showed only on the iPhone; here a Mac (Apple M6) showed one in about half the frames of the opening course, moving with the falling leaves. `DM_NVN_FB_SNAP=all` cleared every frame and named this one shader, now in the profile's `feedback_snap`. - -### Seen in The Great Ace Attorney Chronicles - -Capcom's MT Framework, SDK 10.4.1: 460 shaders in one shader package (section 1), Mario's binding layout (combined textures at `0x20 + 8 × slot`, slots 0–2), every binding decodes and every shader builds. Upper-left window origin, depth mode near = 0, a 1920x1080 RGBA8 swapchain; the game asks for present interval 1 and paces itself at 30 fps. - -- **Fixed: black screen at boot, with the draws of a dialog box in the pass dump** (sampling 256x256 and 1024x256 textures of format 0x4d). No texture had been uploaded from guest memory (`packaged textures: first uploads 0`), and `DM_NVN_TEXINFO=1024x256` showed storage that already held data at first use (226,555 of 262,144 bytes non-zero): no packaged layout, block-linear, in a 1000 MB `CPU_CACHED | GPU_CACHED | COMPRESSIBLE` pool (flags 0xa4). A log of the pool flushes, texture creations and file reads (temporary lines under `DM_NVN_LOG`) showed how they are filled: every `.tex` is read into staging buffers (512 KB `ReadFile`s, none into the pool), and the CPU copies the texels into the texture's storage after `nvnTextureInitialize`, with no `nvnMemoryPoolFlushMappedRange` over them; only the four linear 1024x64 text textures are flushed, every frame. `main` has no cache maintenance instruction at all, so there is no flush for the host to take as the signal. The profile sets `tex_flush_upload` and `tex_dirty_upload` (as FINAL FANTASY TACTICS - The Ivalice Chronicles): a registered texture whose pages the CPU writes after creation is uploaded at its first use. With `tex_flush_upload` alone the screen stayed black. - -### Seen in STEINS;GATE RE:BOOT - -M2's ADV engine on SDK 20.5.17: 487 shaders (483 pixel, 4 vertex) in five BNSH in `main`; the build decodes every binding. Its frames are light: a background, the E-mote characters, the text window. - -- **Fixed: GPU memory grew with every image shown.** The game decodes each picture it shows (ASTC 4x4 images up to 4096 wide, RGBA8 render targets, R8 masks) into one scratch region of a pool that lives for good, finalizes that texture when it is done with it and builds the next one in the same region with another size or format; during some transitions it builds a new render target every frame, one pixel wider each time (614x720, 615x720, …). The storage-sharing table (`g_alias`, keyed by pool, offset and description; section 6, transient render targets) kept every one of them until the pool was finalized: from a quick save, 297 entries and 1.29 GB of GPU memory at frame 6,600; 468 entries and 1.58 GB 24,000 frames into a new game. A new shared texture now takes over the entries of its pool whose storage it overlaps under another description (`alias_take_over` in `nvn_mtl.m`), unless a live texture object still resolves to one (Wonder's transient render targets, rebuilt on new objects every frame and never finalized) or a lookup found it in the last 16 submissions (Breath of the Wild finalizes its transient render targets before every submit and builds them again the next frame; targets sharing storage would otherwise evict each other every frame). An entry that only a finalized serial in `g_ghost` still holds loses the table's reference and is retired with that serial. On the same route it held 97 entries and 0.93 GB at frame 6,600; 15 minutes of story from the quick save (53,700 frames) stayed at 1.08–1.15 GB of GPU memory and 1.02–1.13 GB resident, and a new game from boot stays at 0.97–1.03 GB from its first story scene to frame 27,300 (with the 16-submission guard). Captain Toad: Treasure Tracker's opening course, rendered with agl's transient targets, keeps its 26 entries and takes none over, with the same memory and frame times as before. The memory line reports `shared textures N (M of them unreferenced; evicted with pools …, on storage takeover …)`: the unreferenced ones wait for their storage to be reused or their pool to go. -- **1280x720 in both modes, by the game's own choice.** The window is 1920x1080 and the game crops it (`nvnWindowSetCrop`) to the 1280x720 it draws; the present scales the crop to the screen. Its render size setting (engine object + 0x6000) holds 1920x1080 docked from its constructor (`main+0x383eb0`) until its start-up code sets 1280x720 (`main+0x17c50` → setter `main+0x389e90`), whatever the host reports. Raw screenshots (`DM_NVN_SHOT`) show the whole swap texture, the picture in its lower-left corner (lower-left window origin); `DM_NVN_SHOT_OUT=960x540` gives the presented picture. -- **A film look**: the backgrounds carry a green scratch line, coloured dust specks, chromatic fringes and displaced rows; by every sign the game's own effect (not yet compared with a Switch capture). -- **Movies in buffer mode**: H.264 decoders opened with output 4 (NV12 into the client's buffer, 1280 × 720 × 1.5 bytes) in mode 1; the AAC sound plays through an AudioOut whose volume the game sets every frame ([03](03-bringup.md)). - -### Seen in Big Brain Academy: Brain vs. Brain - -NintendoWare Bezel Engine 1.56, SDK 11.4.3: 2,125 shaders (955 vertex, 1,153 pixel, 12 geometry, 5 compute) from the Bezel Engine archives and `main` (section 1). The window origin is lower left, depth mode near = 0, the swapchain 1920x1080 RGBA8 sRGB (format 0x38; cropped to 1280x720 in handheld Normal), present interval 1 at 60 fps. Everything drew right at the first frame with shaders; no texture or pool key was needed. - -- **Not done: Bezel's line renderer.** Its 12 geometry shaders (`_BezelSystemResources/line/shader/line.bfsha`) take a line strip with adjacency (primitive type 11) and emit up to 12 vertices per segment as triangle strips: thick lines with joins and caps (28 emit sites). Metal has no geometry stage, so they do not build (`unknown main0_out`) and their draws are skipped (`[RED] primitive type 11 not implemented`). From the main menu on, every frame draws one such line (width 10 in its uniform block) into the top-left 512x512 of the frame and copies that region into a texture, so the menu's chart loses its outlines. The vendored Ryujinx translator has its geometry-to-compute transforms (`VertexToCompute`, `GeometryToCompute`), which the chain does not use yet. -- **Not done: ten pixel shaders pick their texture at run time** (Ryujinx's fallback arrays, `fp_t2d_cb0_40000000`): six in `03GamePerception.bnbshpk`, two PBR, two in `main`. In `e10f24835bbaa5f6` the handle is c3 +0x130 or c12 +0x1d0 of the game's own uniform buffers, chosen by a flag in c12, packed as texture ID | sampler ID << 20; `tex_dyn_slots` covers only handles from the driver's c2 tables, so these do not load and their draws are skipped. Resolving them needs the translator to report such a choice of two uniform-buffer handles, a chain change for every profile. None of the activities checked uses them; a long run of activities met one once. -- **Conditional rendering**: the engine calls `SetRenderEnableConditional`; drawn unconditionally (the profile has no `render_cond`), no screen played looked wrong. - -### Seen in Paper Mario: The Thousand-Year Door - -Intelligent Systems' engine on NintendoWare (G3d, Vfx), SDK 15.3.2: 11,858 shaders (3,154 vertex, 7,826 pixel, 874 compute, 4 domain), all BNSH in zstd files (section 1). Every one builds with Mario's binding layout; 2,386 pick a texture at run time. - -- **Zstd files named `.zst`** (8,726 of the RomFS's 9,959 files, the shader archives among them) → `shader_split.py` decompressed only `.zs` files and found 0 shaders. It now goes by the zstd magic (`28 b5 2f fd`) whatever the name; of the other 21 games' RomFS on hand only Super Mario Galaxy's `GDDR3.zst` (a Vulkan game, no NVN split) has such a file. -- **A texture picked at run time between bound slots** (2,386 pixel shaders: 5,476 cube samples, 487 2D): the handle is read from the combined texture table (c2 `0x20 + 8 × slot`, slots 0, 7, 8, 13 and 28, constant offsets only) and chosen by branches, so the translator falls back to `array, 32> fp_tcube_cb0_40000000`. The same case as Octopath Traveler 0's separate textures (section 6): the profile key `tex_dyn_slots` puts `r | r << 20` in the combined table's slot r, so the handle names element r, which holds the texture bound in slot r. The array's other elements hold whatever the other slots do, mostly 2D textures where the array is of cubes; an array element bound to a 2D texture gets the zero cube without a type mismatch report (`tex_fit_elem`), since the shader does not pick it there. -- **An array of handles in a uniform buffer** (the reflection shaders, `reflection.bfsha`: `array, 32> fp_t2d_cb14_0` and its samplers) → a loop reads one handle per light from the game's uniform buffer c14 (uniform slot 11) at a computed offset; Ryujinx's texture array from a buffer indexes it by the handle's byte offset ÷ 8 (an NVN handle is 64-bit, the shader computes `(word >> 2) << 1 | (word & 3) >> 1`). The build reads the name as one bindless texture (`tex_ubo`, offset 0), so the argument buffer was 32 entries longer than the reflection said and the host stopped at the shader's first load (`unexpected argument buffer layout: encodedLength=640 maxAid=48`). The host's reflection reader now takes the array length from the MSL type, and the replay fills element k from the handle at byte 8k of that uniform slot (`NVN_RB_TEXU` / `SAMPU` with `n`); elements whose bytes are not a registered texture or sampler ID read the dummy, silently. The first run showed elements 0–3, 6 and 8–12 holding textures, among them the shader's fixed handles at +8, +16 and +24 and the lights' from +64. No chain change: the build already wrote the array type. -- **`nvnCommandBufferClearColori` / `ClearColorui`** (an integer target cleared once Mario stands on Rogueport's docks, about frame 12,300 of a new file) → recorded as a float clear; both replay paths convert the value to the attachment's integer type. -- **A format-converting copy into a packaged texture** (a 128x128 RGBA8 render target into an RG8 texture, mirrored in Y, as the boat arrives in Rogueport, about frame 10,950) → `copy_t2t_draw` renders into the destination and declined one made without render-target use. The destination now gets that use first (`tex_upgrade`, as a whole-level clear of such a texture does). -- **An unbound uniform slot** (`stage 0 cb5 has no uniform buffer bound`, at the title) → the argument buffer kept a null pointer there; the slot now reads the all-zero bank, as an unbound texture slot reads the zero texture. - -### Seen in Dark Souls Remastered - -FromSoftware's engine on SDK 4.5.0: 3,099 shaders (391 vertex, 2,708 pixel), GLSLC outputs in the dvdbnd archives and `main` (section 1), single-stage programs as in 13 Sentinels. Every binding decodes. Forward rendering into a 1920x1080 RGBA16F target, depth of field, light shafts and bloom at quarter size; 60 fps with rewriter cuts on its frame limiter and its fixed frame step (30 on the Switch, [03](03-bringup.md)). - -- **SDK 4's GLSLC outputs**: version 0x20.4. Pixel shaders' control sections store at +0x20 a value 12 bytes short of their length, so the code area is not right after it, and 44 of `main`'s 72 outputs and most of the archives' pixel shaders failed the reader's check. An output's own section headers say where everything is: the output's +0x4c is the section count, section i's header is at +0x90 + 0x90 i (`{size, data offset from the output, type}`, type 0 = GPU code), and a GPU code header adds +0x30 control offset, +0x34 code offset (both from the section's data), +0x38 code length, +0x3c control length. `glslc_section` reads them for outputs the old rule rejects. On every output the old rule keeps (1,073 in 13 Sentinels, Three Houses, FINAL FANTASY TACTICS, Engage and Age of Calamity) both give the same key; the new reading also takes 8 outputs of Three Houses and 3 of Age of Calamity (GLSLC version 0x1d.0, whose control +0x20 holds odd values) that used to be dropped, so those two profiles' tables gained them. -- **Fixed: distant walls and floors smeared sideways.** The map materials (`FRPG_Phn_*Lit*`) read their combined texture / sampler handles from a uniform buffer (`fp_t_cb13_*`, resolved at replay by `ubo_handle`), and the game keeps 12 sampler pools of 2,304 entries, each with its own samplers under the same IDs (ID 261 is clamp-to-edge with LOD 0..0 in one pool, repeat with mips in others), one per command buffer that records the scene. The replay resolved uniform-buffer handles in the first pool, so the walls' tiling base textures (u ≈ −3.45) took a clamp-to-edge sampler and repeated their edge column across the wall. The replay now keeps the texture and sampler pools bound (`OP_SET_TEX_POOL` / `OP_SET_SAMP_POOL`, which it used to skip; a command buffer that binds none keeps the queue's, as on the GPU) and adds their bases to handles read from uniform buffers (`ubo_tex_id` / `ubo_samp_id` in `nvn_mtl.m`). Found by highlighting the six most common fragment shaders of the scene pass (`DM_NVN_HILITE`), then the next six, and reading the wall draw's binding lines (`DM_NVN_PASSDUMP_ALL`). -- **The bases of bound handles live in the command buffer** (`cmd_t` `tex_base` / `samp_base`): the list of command buffers that bound a non-first pool had 64 entries and was never cleared, and this game binds a non-first pool in more than 64 (6,734 red lines in one run). -- **Conditional rendering**: `SetRenderEnableConditional` is drawn unconditionally (no `render_cond`), as in Big Brain Academy; nothing seen looked wrong. - -### Seen in Catherine: Full Body - -Atlus's GFD engine on SDK 8.2.1: 3,128 shaders (1,244 vertex, 1,872 pixel, 12 compute), all GLSLC outputs of one stage each: 3,050 loose files in `data/shaders_binary` (`*.vs` / `*.ps`, read like FINAL FANTASY TACTICS' technique files, section 1), 76 in `main` and 2 the game compiles at boot ([03](03-bringup.md), SDK 8.2.1). Single-stage programs paired with the cover, as in 13 Sentinels. Every binding decodes with Mario's layout. - -- **Shader type 0 in every vertex shader's SPH** (word 0 = `0x0061`; every other game's vertex shaders have `0x0461`) → `shader_split.py` found no stage for them and stopped at the first captured output (`no GLSLC output`). Ryujinx's `ShaderHeader` reads a graphics header's type 0 as a vertex shader; `sph_stage` does too. A census of the other games' packages and of every `main` on hand found no other type-0 vertex shader. -- **Fixed: every post pass and the window drew one texel's colour (a black screen with sound).** That shader type is Maxwell's first vertex slot (VERTEX_CULL_BEFORE_FETCH), and this GLSLC (GPU code version 1.15) writes every vertex shader as two programs in one code area: part A at code +0 (the SPH above) loads the position attribute and writes only the position; part B, at control +0x700 with a 0x30-byte header of zeros and an SPH of type 1, reads the other attributes and writes the other outputs. Control +0x748 lists both as {SPH offset, length from the SPH, ?}. The hardware runs A, then B. Translated alone, A left every varying at (0, 0, 0, 1), so the full-screen passes sampled their sources at (0, 0) and the 2D draws their atlases' first texel. `dmsh_build.py` (`vertex_ab`) merges the two as Ryujinx's `TranslatorContext.Translate(VertexA)` does: A's instructions with each unconditional `EXIT` turned into a `BRA` to B's first instruction, then B's, under B's SPH with A's flags and input / output maps ORed in and the larger local memory sizes. All 1,244: A writes no generic output, A ends in an `EXIT` predicated on CC FCSM_TR (which Ryujinx never takes; left as it is), an unconditional `EXIT` and a `BRA` to itself, and neither part has an absolute or table jump, so moving B keeps every branch. c1 is what follows part B (zero padding in all of them). A type-0 shader with no part B listed is translated as it is. -- **Fixed: the cutscene subtitle band was a hard-edged dark box with a magenta stroke through it.** The band is a 32x512 BC3 gradient the game copies from a staging buffer (`nvnCommandBufferCopyBufferToTexture`, recorded at the start of a frame, `main+0x305e64`), and the GPU copy read the next texture's blocks: the source was right when recorded and when encoded, but the game's resource thread wrote the next texture into the same staging block 64 ms later, before that first submission had even been committed, because the host spent 300 ms building pipelines inside it. On the Switch the GPU runs a submission's copies as soon as the submit returns. A segment is now committed at the first pass boundary after any buffer → texture copy (`g_cut_copy`, `sub_cut` in `nvn_mtl.m`), so the copies run while the host encodes the rest; the `residency set` line counts these cuts (`N after buffer → texture copies`, a few per 300 frames in this game, only while it loads). Found with `DM_NVN_HILITE` on the band's pixel shader, `DM_NVN_TEXRAW` of the texture (decoded: opaque black and magenta blocks, not a fade) and `DM_PROBE_WATCH` on the staging address beside a log of the copy's submission completing. - -### Seen in Digimon Story Cyber Sleuth: Complete Edition (OpenGL) - -Media.Vision's MVGL engine on SDK 6.4.0 renders with **NVIDIA's desktop OpenGL 4.5 core profile through EGL**, not NVN. The host implements GL itself on Metal (`dm/gfx/gl/hle_gl.m`), as it does Vulkan for Hades; NVIDIA's GL driver (`subsdk0`) is not loaded. - -- **Telling the API apart**: `main` imports `nvnBootstrapLoader` and 13 EGL functions. Its graphics setup (`main+0x5ed8f0`) makes an NVN device first, which only a helper beside the movie player uses (`main+0x528000`–`+0x52c400`: a memory pool and four textures next to the movie's GL textures). Then `eglGetDisplay`, `eglBindAPI(EGL_OPENGL_API)`, a 4.5 core context, a window surface on `vi::CreateLayer(w, h)` and a glad loader (`main+0x52c460`). Absolum imports the same EGL functions (SDL3) but renders with NVN and names no GL function. `profile_guess.py` decides `graphics gl` when `main` imports `eglCreateWindowSurface` and names `glShaderSource`; of the 22 games extracted here only Digimon does. -- **Which GL functions**: glad asks `eglGetProcAddress` for 2,956 names and stores each in a global its code reaches through the GOT. A census of the GOT slots the code loads outside the loader gives the 79 functions the game calls: buffers, vertex arrays, textures (2D and cube, S3TC), framebuffers and renderbuffers, programs with plain `glUniform*`, `glDrawElements` / `glDrawArrays`, and three compatibility calls (`glTexEnvi`, `glColorTable`, `glPixelTransferi`). No uniform buffers, instancing or buffer mapping. Every other name gets a null pointer. -- **Shaders are GLSL ES 3.20 source**, compiled at run time with `glShaderSource` / `glCompileShader`: 1,682 `.shad` files in the MVGL archives (615 distinct) and 37 strings in `main` (post effects, FXAA, shadows, the movie's YUV). `gl_shaders.py` translates them offline: glslang (`-V -R`: relaxed Vulkan rules gather the loose uniforms into one `gl_DefaultUniformBlock`, whose member offsets the table lists) → SPIR-V → SPIRV-Cross. The host finds each shader by the hash of its text and its stage. Varyings are matched by name, as GL links them: `[[user(locnN)]]` becomes `[[user()]]`, so a vertex and a fragment shader translated apart still meet. `--fixup-clipspace` maps GL's clip z to Metal's. -- **What NVIDIA's compiler accepts and glslang does not** (fix-ups on the preprocessed text): the GLSL ES 1.0 functions `textureCube` / `texture2D` in ES 3.20 code (89 fragment shaders; with `-V` glslang reads `textureCube` as Vulkan's texture type, so a macro cannot rename it), `out vec4 gl_Position` redeclared through a macro (the built-in sprite shader), and `textureOffset` with an offset that is not a constant (FXAA), which becomes a plain sample at P + offset ÷ level 0 size. A string in `main` says nothing of its stage, so it is translated as each stage that compiles (one fragment shader defines `vPosition` as `gl_Position`). -- **Orientation**: the layer renders as GL does, bottom row first, through the NVN layer's lower-left origin (`mtl_set_origin_lower_left`): flipped viewports, reversed winding, a flipped present. Scissor rectangles and `gl_FragCoord` need no change, and render targets read back with GL's texture coordinates. -- **State**: uniforms are CPU copies per stage, set as bytes at a draw after they change; pipelines are cached per program by vertex layout, attachment formats, blend and colour mask; disabled vertex arrays read the attribute's current value (GL's default (0, 0, 0, 1) until `glVertexAttrib4f`) through a constant-step buffer. Without `glBindAttribLocation` the game's attribute index is the Metal one: `glGetAttribLocation` returns the shader's own attribute numbers (Balatro below binds its own). -- **Textures** are made on first use from the levels given so far (the loader defines them level by level). An upload into a texture a pending command buffer may still read is a blit in command order; any other is a CPU write. Metal reads a channel a format lacks as 0 (alpha 1) as GL does, so only luminance / alpha formats get a swizzle: Metal refuses a swizzle on a texture that can be a render target (`Texture swizzling is not compatable with MTLTextureUsageRenderTarget`, the first story scene). -- **Buffers**: `glBufferData` on a buffer the GPU may still read gets a new `MTLBuffer`, as GL drivers orphan it. **Clears**: a whole-framebuffer clear becomes the next pass's load action; a scissored or masked one is a quad. -- **Present**: three window textures through the NVN layer's present path (`mtl_ext_tex`, as the Vulkan swapchain), paced by the swap interval in display-link ticks. The game sets the interval to 60 ÷ its frame rate before each swap: 1 at the title and in the DigiLab, 2 from the story's opening on. -- **Movies**: the USM movies' H.264 is decoded through the SDK's movie library (the host's decoder) into NV12 buffers (3,110,400 bytes a 1920x1080 frame); the game uploads the planes into a `GL_RED` and a `GL_RG` texture (two `glTexSubImage2D` a frame) and converts YUV in its own shader (in `main`, `SamplerY` / `SamplerUV`). - -### Seen in Balatro (LÖVE 11.4 on OpenGL) - -LÖVE ("love-nx", with SDL and LuaJIT) renders with the same NVIDIA desktop GL 4.5 through EGL; its graphics module is LÖVE's own OpenGL backend, so its choices follow what the context reports. - -- **Telling the API apart**: `main` imports the EGL functions and `vkGetInstanceProcAddr`, SDL's Vulkan loader, which nothing calls (`subsdk1` is NVIDIA's Vulkan driver). `profile_guess.py` leaves `vkGetInstanceProcAddr` out of its Vulkan count: Hades and both Galaxies import 99 or 100 Vulkan functions, `vkCreateInstance` among them. -- **Which GL functions**: LÖVE's glad stores each pointer straight into a global of its own (no GOT slot per name), and the code reaches those globals through GOT entries that hold their addresses, which it often keeps in callee-saved registers across a loop, or picks between with `csel` (`glEnableVertexAttribArray` or `glDisableVertexAttribArray`). `loader_census.py` follows registers in a straight line for that (adrp, add, loads through relocated GOT entries, `mov`, `csel`): 2,133 names asked, about 150 called. `DM_GL_STUBS=1` gives every name the layer lacks a stub of its own that reports its first call instead of a null pointer (a jump to address 0 names nothing). -- **Core profile**: LÖVE reads `GL_CONTEXT_PROFILE_MASK`; on a core profile it streams vertices through persistently mapped buffers (`glBufferStorage` + `glMapBufferRange` of a 3-frame ring, a `glFenceSync` per frame and `glClientWaitSync` three frames later), on a compatibility one from client memory, which Metal cannot draw from. The layer answers core: the mapping is the shared `MTLBuffer`'s memory, a fence is the serial of the command buffer being recorded, and a wait on a fence of that command buffer commits it first. The shaders then come as `#version 330 core`. -- **Attributes**: LÖVE binds VertexPosition, VertexTexCoord, VertexColor and ConstantColor to 0–3 with `glBindAttribLocation` before each link, keeps VertexColor and ConstantColor as disabled arrays and sets their values with `glVertexAttrib4f` (the draw colour); the layer maps each Metal input to the game's index at the link. -- **Uniforms**: LÖVE lists them with `glGetActiveUniform` (types as GL enums, arrays as `name[0]`) and reads their initial values back with `glGetUniformfv` / `iv`. A GLSL `bool` is a `uint` in the SPIR-V block; `gl_shaders.py` keeps the declared type, as LÖVE's `Shader:send` takes only a boolean for a bool. -- **Textures**: made with `glTexStorage2D` (every level up front: the game's images are mipmapped, filled by `glGenerateMipmap` as a blit) or `glTexStorage3D`: LÖVE makes a 1x1 default texture of each type, 3D and 2D array among them, and the layer's units hold all four targets (96 of them, as `GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS` says). Font glyphs are RG8 with a `GL_TEXTURE_SWIZZLE_*` of R, R, R, G, applied when the texture is made (a swizzled texture cannot be a render target). -- **Draws and framebuffers**: batched quads share one 16-bit index buffer through `glDrawElementsBaseVertex`; canvases bind GL_FRAMEBUFFER (read and draw), check completeness, discard depth and stencil after use (`glInvalidateFramebuffer`) and clear with `glClearBufferfv`. `glBlitFramebuffer` (colour, same size) and `glReadPixels` (RGBA8) are there for multisample resolves and `Canvas:newImageData`, which the game does not use. -- **Shaders are put together at run time**: LÖVE builds each source in Lua from its header, its built-in code and the game's `.fs` file; the 35 are all made before the first frame. The NSP script captures them on the Mac (`gl_capture.tsv`). A capture run has no library: every program then links with no uniforms, the game's first `Shader:send` raises a Lua error and, on the Switch, its error handler quits ([03](03-bringup.md)). - -### Seen in DRAGON QUEST VII Reimagined - -Unreal Engine 4.27's desktop renderer with forward shading (one RG11B10 colour pass draws every object, lit per pixel from the light grid, the volumetric lightmap and the screen-space shadow mask), HEXA DRIVE's toon lighting on top. 62,901 shaders, LZ4 in the shader code archives (section 1); the game has no PC version, so the facts table holds only compute group sizes. - -- **The hero drawn as a dithered silhouette inside a dithered square** → Unreal's stencil shader resources are second textures on the depth buffer's storage built with `nvnTextureBuilderSetDepthStencilMode(STENCIL)`, which the host only logged, so the character occlusion mask (`edbfc7a088be545e`) read depth where it tests stencil bits. The builder's mode now goes to the texture (`tex_t.stencil`), whose sampling view is the `X32_Stencil8` one a stencil-mode texture view already got. -- **Characters and props glowing white in the opening cutscene only, about 10⁴ too bright, while walls and gameplay look right** → the per-pixel volumetric lightmap read brick (0, 0, 0) of the brick atlas for almost every cell. Unreal merges each streamed level's lightmap into one atlas on the GPU: brick copies (`0a7ff4171bd1a82a`), then a patch of the shared 16³ `RGBA8_UINT` indirection texture (`7b958a7b21e72801`) whose bounds come from `GetDimensions` on the very `RWTexture3D` it writes. That size query is a TXQ on the image's handle, c2 `0x120` (image slot 0): the build numbers it combined texture slot 32, the slot that shares that word, and with no type in the facts table the translator read it as 2D. The host bound the empty texture slot 32, the 1×1 dummy's size passed only thread (0, 0, 0), so only that cell was written, every other cell kept size 0, and the shader fell back to the atlas origin: the first daytime level's bricks, baked about 2¹⁵ brighter (the same bricks in gameplay hold 0.5–1.4). Two general fixes: a texture binding of slot 32–39 reads the image bound in slot 0–7 when there is one, as c2 holds it (`nvn_mtl.m` `tex_slot_h`; Hades II's texture slot 34 without an image still reads the texture), and a query on an image handle the facts do not type is translated again with the type of that image from the first translation (`dmsh_xlate` `ImageTypesOf`). Monster Hunter Rise has four such size queries (`cp_t_tcb_48` / `_4A`). How it was found: the raw scene colour (`texraw`) put the characters at 1,000–2,000 and the walls at 0.05; a debug build of one character shader that wrote its terms in alternating 8-pixel columns isolated the volumetric lightmap ambient term; dumps of the atlas (`texlist` / `texraw`), of the uploaded level bricks (the 3D upload dump below) and of the indirection texture, with the cell computed from the primitive's origin and View `VolumetricLightmapWorldToUVScale / Add` (View 220 / 221), gave the rest. -- **Integer textures and images read and written through float declarations** (the indirection `RGBA8_UINT` texture read by `texture3d` and bit-cast, `R16_UINT` / `R32_UINT` light grid buffers through `texture_buffer`, integer stores into a float-declared `texture3d` image) → checked on an Apple GPU: all keep the integer bits, so they need no facts. - -### Seen in Luigi's Mansion 3 - -Next Level Games' deferred renderer on SDK 7.3.2 (NVN 53): the scene draws into a G-buffer of two integer targets (R32Uint, RGBA8Uint) and a D32S8 depth, light volumes and a full-screen resolve light it into RG11B10F targets, and a chain of 960×540 to 60×33 bloom passes ends in the window. All 5,317 shaders come from `init` (one program per stage), and most textures are ASTC. - -- **Every scene draw skipped** (`failed to create render pipeline … output of type float4 is not compatible with a MTLPixelFormatR32Uint color attachment`, about 1,400 pixel shaders): with no PC version the translator types every fragment output float, and the G-buffer shaders write bit-packed integers through `as_type`. A hand-written row per shader (`shader_facts_manual.tsv`, Breath of the Wild's way) does not scale to that. `dmsh_build.py` `fs_uint` gives a pixel shader a second entry point, `main0_u`, with every colour output the unsigned integer vector of the same bits, when it writes an output from integer bits or nothing but constants (a clear drawn into the R32Uint target); the hardware writes the output's bits to an integer target whatever the shader computed, so the copy is exact. The host takes it for a pipeline whose written colour attachments are all unsigned integer formats (`pso_all_uint`); one with a float target as well keeps `main0` and the red line. -- **Shadow cascades from an array of driver-table handles** (586 pixel shaders: `array, 32> fp_t2das_cb2_0`, `shaders with a binding whose slot was not decoded 588`): Ryujinx's texture array from a buffer, as Paper Mario's uniform-buffer arrays, but over the driver table c2, indexed by (0x80 + a cascade offset) ÷ 8. `dmsh_build.py` `c2_array` names it `tex_c2a` / `samp_c2a` with the c2 byte offset of element 0; the replay fills element k from the slot whose c2 word is at that offset + 8k (combined, else separate), and the dummy elsewhere (`NVN_RB_TEXA` / `NVN_RB_SAMPA`, `c2_handle`). -- **A depth texture drawn through a colour view** (the boot aborted in Metal's pipeline validation: `MTLPixelFormatDepth16Unorm is not color renderable`): a 1024×1024 D16 texture drawn through an R16 view. Metal has no colour view of a depth texture, so such a pass draws into a colour shadow of the same pixel size, copied in before and back after the pass through a buffer, as a compressed render target is (`shrt_block`; D16 and D32F, whose Metal pixel is the NVN pixel; not D24S8). -- **Post-process passes with stale texture handles**: the game patches handles into recorded commands ([03](03-bringup.md)). -- **Not done:** the tessellation stages (15 hull, 18 domain shaders; `primitive type 14 not implemented`, their patch draws are skipped) and 10 geometry shaders (one expands a box from two corners). Neither is drawn much. Two tessellated programs (vertex `28a39709d0adfcfa` / hull `51d2d156bb12c260` / domain `0b95742e863bec50`, and `d4d89931b5a9a514` / `8aa6feeb7d33e9f6` / `f254b4dce2d6c012`) draw twice a frame for about 30 frames, in the title scene behind the file select as it slides out before a new game. No draw used a geometry shader from the title through the opening cutscene to the lobby. A program whose geometry shader has no library used to draw silently without that stage, with the vertex shader's positions taken as clip coordinates; `draw with geometry shader … which has no library` now names it. SPIRV-Cross already writes the hull shaders as kernels and the domain shaders as post-tessellation vertex functions, but Metal 4's render encoder and pipelines have no tessellation (no `drawPatches`, no factor buffer), so the draws would need a tessellator in a compute kernel and domain shaders rewritten as plain vertex functions. The hull kernels also write quad factors (`MTLQuadTessellationFactorsHalf`) for triangle patches. The vertex shaders that feed a hull or geometry shader can be found from the shader headers (SPH): the generic attribute map at +0x18 of the consumer's header (inputs) equals the one at +0x36 of the vertex shader's (outputs). In LM3, 40 vertex shaders match a hull shader and 49 a geometry shader. A few pixel shaders declare a 3D texture in a slot the game fills with a 32×32 2D texture (`texture type mismatch`), on paths not seen taken. - -### Seen in Super Smash Bros. Ultimate -- **A pink window, every command buffer ignored, while headless runs looked right** (13.0.5, `DM_WINDOW=1`): the first presents failed on the display command buffer with `kIOGPUCommandBufferCallbackErrorPageFault`, and after a few of those macOS ignored every submission of the process ("Ignored (for causing prior/excessive GPU errors)"). The game presents its window texture in its first frames before drawing into it; the texture was made on that first resolution in the present path and added to the residency set (`res_add`), but only a submission commits the set, so the display command buffer sampled a texture outside the committed set. `present_now` now commits the residency set after resolving the presented texture (`res_commit`, free when nothing changed), for every game. Headless runs never present, which is why they looked right; Breath of the Wild windowed with the change: its title, 60 fps, no GPU error. - -### Seen in Bayonetta 3 -- **Shader containers**: PlatinumGames' shader objects, 8,024 of them in `root_000.pkz` (`shader.dat`, `shader2.dat`, `shadereff*.dat`, `autoshadereff.dat`): `VSO\0` / `PSO\0` / `CSO\0` / `HSO\0` / `DSO\0` (the stage), u32 compiler version, u32 control length, u32 control offset (0x18), u32 code length, u32 code offset, and nothing else; 6,424 distinct keys (4,288 pixel, 2,047 vertex, 78 compute, 6 domain, 5 hull). The engine makes a program of each and binds them per stage at draw time, so they are single-stage programs, paired with a cover pixel shader (`shader_split.py` `pg_shader`, [01](01-extract.md) for the archives). It finds its built-in ones by name (`hwfillcolor.vso`, …) and checks each object's GPU code version against the driver's ([03](03-bringup.md), SDK 13.3.2). -- **Two domain shaders do not build** (`DS_Vanish05_*.dso`: `SPIRV-Cross threw an exception: Subgroup builtins are not available in this type of function`); the one-lane rewrite (`spv_single_lane`) covers vertex shaders only, and tessellated draws are skipped anyway. -- From the title to the prologue's first play the picture is right as far as compared (title scene, menus, the VP9 opening movie, the first real-time scene, play with the HUD). Two red lines, once each: a pixel shader declares a cube texture where the game binds a 2D one (the zero cube is bound), and a draw whose stage 0 reads cb1 with none bound. - -### Seen in Bayonetta 2 -- **Shader containers**: BNSH files of one stage each, the `.vso` / `.pso` members of PlatinumGames DAT archives (`shader.dat`, `shader2.dat`, `shadereff.dat`) inside a CRI CPK archive, all CRILAYLA-compressed ([01](01-extract.md); `shader_split.py` `cpk_jobs`): 1,856 distinct keys (1,547 pixel, 309 vertex). Each file is one variation with one stage, so the split pairs nothing; as in Hades II, the vertex shaders write every output their pixel shaders read and no pipeline failed to link. -- **Fixed: a clear folded into a pass that covers less than the cleared attachment.** The story's opening narration showed its scene in a 32x32 or 256x256 square in the top-left corner of a black screen, the earlier opening scenes had speckled and blocky patches, and a full-screen grain overlay showed one tile. Every frame starts with a pass whose colour target is 32x32 (then 256x256 ones) beside the 1280x720 depth buffer, and the game clears the depth there. A Metal render pass covers only the smallest of its attachments, and so does a clear in its load action, so only that corner of the depth buffer was cleared; the scene behind failed the depth test against the previous frames' depth everywhere else (the texture dump right after the scene pass showed the picture only in the corner, and `DM_NVN_HILITE` on the scene's shaders lit only the corner). NVN clears the whole target. `rp_begin_ex_` now clears an attachment larger than the pass in a pass of its own first and loads it in the real one (`C … cleared in a pass of its own` in a pass dump). -- **Fixed: `Downsample` of a single-sample texture** (`[RED] Downsample case not implemented: 320x176 1 samples format 70 → 80x48`). The game builds its bloom with `nvnCommandBufferDownsample` from a 320x176 target into 80x48 and 48x32 textures, and from its 1280x720 BGRA8 frame into an RGBA8 texture of the same size; the host took Downsample only as a multisample resolve and skipped the rest. A single-sample source is now drawn into the destination as a whole-level scaling copy with bilinear filtering, converting the format (`copy_t2t`, through `copy_t2t_draw`). - -### Seen in Bayonetta -- **Shader containers**: as in Bayonetta 2, BNSH files of one stage each, the `.vso` / `.pso` members of `shader.dat` in `data00.cpk` (CRILAYLA): 458 distinct keys (236 pixel, 222 vertex). All build and the picture was right on the route to the first fight (the clock tower, the graveyard, Witch Time's tint, the practice screens, the HUD) with no change: the two fixes made for Bayonetta 2 cover this engine. - -### Seen in Pokémon Sword -- **Shader containers**: 41,516 shaders. Nearly all are BNSH entries of the 7,699 GFPAK archives under `bin/archive` (the models' and effects' shader archives, LZ4 entries, [01](01-extract.md); `shader_split.py` `gfpak_members`), beside loose BNSH files (`bin/battle/waza/model/**/*.bnsh_vsh` / `_fsh`, `bin/graphics`, `system_resource/shader`) and two in `main`. The provisional split, with no GFPAK reader, found 3,126. -- **Shaders in no file: YEBIS** (Silicon Studio's post-effect library: bloom, glare, depth of field, vignette; `C:/workspace/YEBIS_draft/src/GPU/…` in `main`). After the GFPAK reader, 38 shaders the opening drew were still missing from the map (`DM_NVN_SHDUMP`): no RomFS file and no module held their code, stored or LZ4, and none was a patched copy of a split shader (the nearest ones of the same length differ in hundreds of bytes). The game does not compile them at run time either (`main` imports `glslcCompile` from `subsdk1` but never calls it; it holds no GLSL source). A temporary trace in `nvnProgramSetShaders` (the key's code and control addresses, a backtrace and the heap around them) showed control sections packed back to back with technique names such as `ps_BrightPassT1_THRESHOLD_LUMINANCE_S1` beside them. The names are in `main`, and so is the data: the only zlib stream in `main` that inflates (`main+0x1c98ed9`, 39.5 MB) is YEBIS's `.tfxo file`, a text header (the shader count, `hlsl`, `TexUtil_NVN_NX`, the technique list) and 3,155 shaders, each a control section (its own length 0x850 on SDK 7) directly followed by its code area (`tfxo_streams`, `tfxo_shaders`; 3,015 distinct, 3,112 pixel and 43 vertex, single-stage programs). They covered all 38. -- **A G-buffer of float and integer targets** (`failed to create render pipeline … output of type float4 is not compatible with a MTLPixelFormatRGBA16Uint color attachment`, 10 pipelines as the opening's stadium loads): the scene shaders write colour 0 and 2 as floats and colour 1 from integer bits (`as_type`) into an RGBA16Uint target. Luigi's Mansion 3's integer entry point (`main0_u`, every output an integer) does not fit a pipeline with a float target as well. `fs_uint` now also gives a shader whose outputs are of both kinds a third entry point, `main0_m`, that returns only the outputs written from integer bits as unsigned integers, and records their colour indices (`uint_mask_m` in its reflection, colour 1 in all 336 of Sword's); the host takes it for a pipeline whose integer attachments are exactly those (`pso_mixed_uint`). -- **Cube views of one layer** (an abort in Metal's validation as the player first walks out of the house: `newSliceRange.length(1) must be a multiple of (6)`) → the environment maps are 6-layer textures (target 2D array, depth 6) sampled through views with target CUBEMAP and layers 0 + 1. As in the hardware's texture header, a cube view counts its layers in cubes; Metal counts faces. A cube view whose layer count is not a multiple of 6 now takes its layers as cubes when the texture has that many faces (`tex_by_id`; else it is read as a 2D array with a red line). -- **Writes beside a texture** (73,232 re-uploads in 70 s from a save, 12 GB of upload staging in 8 minutes, frame times rising to 75 ms): the game keeps each texture of its models in a memory pool of its own, the pool's storage 4 KB after its own object, in the same 16 KB host page, and writes there every frame. The dirty-page tracker marks whole host pages, so every such texture counted as changed at each use; hashing showed every re-uploaded content unchanged. A packaged texture now keeps the bytes of its storage that lie in its first and last host pages when they are shared (`pkg_edges_take`); when only those pages were written and the bytes are as uploaded, it is protected again instead of re-uploaded (`pkg_edges_clean`, counted in the `packaged textures` line as `writes beside a texture only`). The same 70 s: 0 re-uploads and 72,584 such checks; `t_nvn` 6c1 covers it. - -### Seen in Mario vs. Donkey Kong -- **Level scenes black behind the HUD** (1.0.1; `[RED] SetRenderEnableConditional: the host draws anyway`): the game skips clears under render conditions, as Tears of the Kingdom and Animal Crossing: New Horizons do; the profile key `render_cond` makes the replay evaluate them. -- **Advanced blend modes** (`nvnBlendStateSetAdvancedMode 8`, SRC_OVER, on one blend state in the levels): the host used to log it and blend with the state's own functions, which the hardware ignores while an advanced mode is set. The blend state now keeps the mode, the premultiplied-source flag and the overlap (`blend_st` in `nvn_obj.h`), and the pipeline (`adv_blend` in `nvn_mtl.m`) blends the Porter-Duff modes (ZERO, SRC, DST, SRC/DST_OVER, _IN, _OUT, _ATOP, XOR, the PLUS modes) with the fixed-function factors they amount to under uncorrelated overlap, with premultiplied colours as NV_blend_equation_advanced defines them; a source that is not premultiplied is multiplied by As where the source factor is ONE. Multiply, screen, overlay, the HSL modes and the other overlaps need the destination colour in the shader and keep the state's functions, with a red line naming the mode. diff --git a/docs/agents/05-debugging.md b/docs/agents/05-debugging.md index ab1f5f8..dc76bf9 100644 --- a/docs/agents/05-debugging.md +++ b/docs/agents/05-debugging.md @@ -13,7 +13,7 @@ x0=… x28=… - `pc` and `lr` are module-relative (`main+…`, `sdk+…`) or host symbols. Module offsets are the same as in the decompressed NSO and the ELF (Ghidra adds 0x100000). - Look at the registers before the code: a pointer that is a valid pointer plus garbage high bits, an index of `0xfffffffe`, or `pc == lr == 0x100000000` each tell a different story. a fault at `0x74a740e114 = 0x70a740c570 + 0xfffffffe·4 + 0x1bac` was an index of −2 widened as unsigned; `pc == lr` on a nonsense value is a `ret` through an overwritten saved LR (look at the zeroed frame below `sp`). -- The handler takes SIGSEGV, SIGBUS, SIGILL, SIGTRAP, SIGALRM (the run's time limit) and SIGABRT (`abort()`: an assertion or an uncaught exception in the host or a framework; before it was handled, Big Brain Academy: Brain vs. Brain died once with exit 134, nothing in the log and no report). A run that ends with exit 137 and no `CRASH` line was killed with SIGKILL: macOS does that after a trap in a system library (`_os_unfair_lock_corruption_abort` destroying a corrupted mutex, the movie decoder entry in [03](03-bringup.md)), and its report is in `~/Library/Logs/DiagnosticReports/dmhost-*.ips`, a JSON document whose `faultingThread` names the thread whose `frames` to read. +- The handler takes SIGSEGV, SIGBUS, SIGILL, SIGTRAP, SIGALRM (the run's time limit) and SIGABRT (`abort()`: an assertion or an uncaught exception in the host or a framework; before it was handled, Big Brain Academy: Brain vs. Brain died once with exit 134, nothing in the log and no report). A run that ends with exit 137 and no `CRASH` line was killed with SIGKILL: macOS does that after a trap in a system library (`_os_unfair_lock_corruption_abort` destroying a corrupted mutex, the movie decoder entry [Input indices returned by a flush while a client holds them](../reference/movies.md#input-indices-returned-by-a-flush-while-a-client-holds-them)), and its report is in `~/Library/Logs/DiagnosticReports/dmhost-*.ips`, a JSON document whose `faultingThread` names the thread whose `frames` to read. - When the frame-pointer backtrace is empty, the stack scan's negative offsets (below `sp`) show the return addresses of the calls the crashing function made last. - `[dm] *xN:` lines show the first six words each register points at (read safely, so a bad pointer is skipped). A word inside a module prints as `main+…`: the first word of an object is its vtable, which names its class, and comparing it with the class's own vtable shows whether the object was destroyed (a base-class vtable) or rebuilt. @@ -57,7 +57,7 @@ The `!xN` heap walk takes the layout of the game's sead version from `DM_PROBE_H An exclusive or atomic store (`STXR`, `STLR`, `CAS`, `SWP`, `LD`) that reaches the watched bytes prints `WATCH … exclusive store` (or `atomic store`) with its pc and backtrace, every time, though it is not emulated. A page that takes many atomics stays unprotected most of the time, so even that can miss: Tears of the Kingdom's memory-shortage flag (a word its frame check compares with 1 and clears) shares its page with atomic counters, and the watch saw neither the write nor the check's own clear. -**When the watch misses: `DM_PROBE_POLL=
[+]`** (hex guest address, 4 bytes by default, up to 8). A host thread spins on the bytes; on each change it stops every thread at once, reads each one's pc, lr and 8 frame-pointer frames, lets them run again, then prints `POLL thread …` for the threads with guest code in those frames, the change (`POLL … old → new`) and, for the first two changes, the full guest thread dump. The snapshot comes a few microseconds after the store; a writer that went on into a wait shows only its wait (for that flag the one thread in guest code was `PlacementMgr` in `main+0x5aaac0`, both times). It costs one core while it runs. A word that changes thousands of times before the bad store takes a value filter, `DM_PROBE_POLL=
+=`: every change is still logged, but the threads are snapshotted and dumped only at changes to that value (Mario vs. Donkey Kong: a heap block's list link, rewritten by every allocation beside it, until an output index landed in it, [03](03-bringup.md)). +**When the watch misses: `DM_PROBE_POLL=
[+]`** (hex guest address, 4 bytes by default, up to 8). A host thread spins on the bytes; on each change it stops every thread at once, reads each one's pc, lr and 8 frame-pointer frames, lets them run again, then prints `POLL thread …` for the threads with guest code in those frames, the change (`POLL … old → new`) and, for the first two changes, the full guest thread dump. The snapshot comes a few microseconds after the store; a writer that went on into a wait shows only its wait (for that flag the one thread in guest code was `PlacementMgr` in `main+0x5aaac0`, both times). It costs one core while it runs. A word that changes thousands of times before the bad store takes a value filter, `DM_PROBE_POLL=
+=`: every change is still logged, but the threads are snapshotted and dumped only at changes to that value (Mario vs. Donkey Kong: a heap block's list link, rewritten by every allocation beside it, until an output index landed in it, [AAC decoders offer at most 32 outputs](../reference/movies.md#aac-decoders-offer-at-most-32-outputs)). A probe's backtrace reads each frame with `vm_read_overwrite`: code built without frame pointers keeps data in x29, and the walk used to fault in the host on such a value (x29 = 0x430 at `main+0x38921c` in the same game). @@ -118,7 +118,7 @@ To see the order of events, put temporary observation gates (`OBS` in the profil 4. GPU allocated climbing while the residency set, `shared textures` and the upload counts stay flat: retired objects kept alive by an autorelease pool that never drains. Any objc call on the guest submit thread outside `g_sub_pool` leaks what it autoreleases (Metroid Dread; Breath of the Wild, GitHub issue #9). Build the host with the suspect wrapped in `@autoreleasepool` and compare. 5. `serial objects` climbing by the same amount every frame with GPU memory flat: entries the serial table never deletes. A census of `g_by_serial` by class and size in `mtl_mem_stat` shows which (BotW: finalized textures replay resolved again; they now go to the small table counted as `finalized textures still referenced`). -6. Address space rather than memory (an iPhone has about 5 GB of it without the extended virtual addressing entitlement): `DM_ALIAS_WINDOWS=` logs every 30 s how much of each window of that size in the alias region is mapped (windows aligned to their size in absolute addresses). For an allocator that reserves fixed windows and commits each from its start, the windows are its pools: Unreal's FMallocBinned3 in Octopath Traveler 0 showed 74 pools of 512 MB holding 732 MB together, the largest 68 MB, so no smaller uniform window fitted ([03](03-bringup.md)). Unreal's own heap on the host malloc zone (`hle_uemalloc.c`) logs its peak as `[mem] Unreal heap … in use`. +6. Address space rather than memory (an iPhone has about 5 GB of it without the extended virtual addressing entitlement): `DM_ALIAS_WINDOWS=` logs every 30 s how much of each window of that size in the alias region is mapped (windows aligned to their size in absolute addresses). For an allocator that reserves fixed windows and commits each from its start, the windows are its pools: Unreal's FMallocBinned3 in Octopath Traveler 0 showed 74 pools of 512 MB holding 732 MB together, the largest 68 MB, so no smaller uniform window fitted ([memory.md](../reference/memory.md#unreals-fmallocbinned3-replaced-by-the-host-allocator)). Unreal's own heap on the host malloc zone (`hle_uemalloc.c`) logs its peak as `[mem] Unreal heap … in use`. Instruments' Metal Resource Events didn't record this process from the command line (`xctrace record --launch`, with `--env` for the variables); the in-host probes were faster. @@ -153,4 +153,4 @@ A glitch that comes and goes between frames or runs needs a detector before bise 3. Bisect with `DM_NVN_SKIP_FS` (up to 16 fragment shader keys per run) until one draw remains, then `DM_NVN_HILITE` it to see its geometry. 4. Dump that draw's inputs across frames (`DM_NVN_CBDUMP` + `DM_NVN_CBDUMP_KEY`). If every input is byte-identical between good and bad frames, the difference is on the GPU side: follow the textures it samples (`DM_NVN_TEXINFO`, `DM_NVN_TEXRAW`) to whatever writes them. 5. For a GPU-side simulation, trace its state on the GPU timeline (`DM_NVN_T2T_TRACE`), re-implement the shader step in numpy with the logged uniform updates (the `Y update_ub` lines print the first four floats), and compare frame by frame. Frames that disagree show which input the GPU really used; for the fountain, a coefficient of −1 instead of 1.81 explained every bad texel and led to the missing intra-encoder barrier. -6. A dump read at encode, or after the submission completed, may not be what the draw read: later work of the submission can rewrite the buffer. When a draw's indices or vertices look wrong, list in command order every copy and inline update of the frame from the pass dump (`DM_NVN_PASSDUMP_ALL=1`, the `Y copy_b2b` / `Y update_ub` lines): a copy whose source an earlier GPU write of the same frame covers reads that write, not the bytes the CPU staged. Cuphead's broken runs had dozens of those a frame, and the clean runs none. The game's allocations overlapped, so the cause was in the game's allocator, which a host answer had broken ([03](03-bringup.md), NVN object sizes). +6. A dump read at encode, or after the submission completed, may not be what the draw read: later work of the submission can rewrite the buffer. When a draw's indices or vertices look wrong, list in command order every copy and inline update of the frame from the pass dump (`DM_NVN_PASSDUMP_ALL=1`, the `Y copy_b2b` / `Y update_ub` lines): a copy whose source an earlier GPU write of the same frame covers reads that write, not the bytes the CPU staged. Cuphead's broken runs had dozens of those a frame, and the clean runs none. The game's allocations overlapped, so the cause was in the game's allocator, which a host answer had broken ([Sizes the hardware never reports as 0](../reference/nvn-objects.md#sizes-the-hardware-never-reports-as-0-zcull-save-area)). diff --git a/docs/agents/README.md b/docs/agents/README.md index ef161d2..717cb74 100644 --- a/docs/agents/README.md +++ b/docs/agents/README.md @@ -8,9 +8,10 @@ Everything here was learned on Diablo II: Resurrected (SDK ~12, NVN 53, the orig |---|---| | [01-extract.md](01-extract.md) | Keys, NSP → NCA → ExeFS / RomFS, update NSPs, the workspace layout | | [02-survey.md](02-survey.md) | Static survey of a new title before touching the host: SDK version, imports vs. host hooks, SVCs, instructions to rewrite, NVN surface, data containers, Ghidra / REA setup | -| [03-bringup.md](03-bringup.md) | Getting the host to boot the game: the run loop, how to read the log, and a catalogue of every failure pattern met so far with its root cause and fix | -| [04-graphics.md](04-graphics.md) | NVN on Metal for a new title: object sizes, shader containers and keys, binding layouts, bindless textures, depth mode, index sizes, swizzle, transient render targets, the diagnostic switches | +| [03-bringup.md](03-bringup.md) | Getting the host to boot the game: the run loop and how to read the log; the failures met so far are in the [subsystem references](../reference/README.md), found through the [symptom index](symptoms.md) | +| [04-graphics.md](04-graphics.md) | NVN on Metal for a new title: finding every shader container, shader keys, binding layouts, bindless textures, depth mode, the diagnostic switches; what other titles needed is in the [rendering](../reference/rendering.md), [shaders](../reference/shaders.md) and [NVN objects](../reference/nvn-objects.md) references | | [05-debugging.md](05-debugging.md) | Recipes: crash reports, lldb scripts, Ghidra headless decompilation, objdump, memory growth, render-pass and texture dumps, on-device diagnostics, offline pipeline link checks | +| [symptoms.md](symptoms.md) | One line per symptom met so far (an assertion, a `[RED]` line, an abort or crash site, a visible result) → the reference entry that explains it | ## Ground rules @@ -31,9 +32,9 @@ Everything here was learned on Diablo II: Resurrected (SDK ~12, NVN 53, the orig 2. Extract base (and update, if in scope) — [01](01-extract.md). Finish the survey — [02](02-survey.md). Write down: SDK version, NVN API version, module list, imports the host lacks, SVCs, instruction census, NVN loader pattern, shader containers. 3. Add a game profile `dm/games//game.tsv`, named after the base game's title ID (start from `010015100B514000/game.tsv`: identity, `build` rows, `modules`; keep the memory / NVN values until they are measured) and build the host for it with `make -C dm/host GAME=<id>`. 4. Run the rewriter and check its counts against `insn_census.py`. -5. Boot on the Mac and iterate on the failure catalogue in [03](03-bringup.md) until the game reaches its frame loop. +5. Boot on the Mac and iterate ([03](03-bringup.md)) until the game reaches its frame loop: look up each failure in the [symptom index](symptoms.md), which leads to its entry in the [subsystem references](../reference/README.md). 6. Extract and translate the shaders, find the binding layout by census, and get the first frame — [04](04-graphics.md). 7. Check memory over a minute of running (`footprint`, the host's memory report line); fix growth before moving on. 8. Input, audio, then the iOS bundle (d2r-mobile's `dm/ios`, RomFS inside the app). -9. Test on a phone: an unsigned IPA the user sideloads, switches through `env.txt`, results through `Documents/log.txt` ([05](05-debugging.md)). Several bugs only showed there (the command-buffer byte and the audio buffer size in [03](03-bringup.md), pipeline link failures in later levels in [04](04-graphics.md)). +9. Test on a phone: an unsigned IPA the user sideloads, switches through `env.txt`, results through `Documents/log.txt` ([05](05-debugging.md)). Several bugs only showed there (the [command-buffer byte](../reference/nvn-objects.md#inlined-sdk-code-writes-into-nvncommandbuffer-0x10) and the [audio buffer size](../reference/audio.md#remoteios-default-io-buffer-duration), pipeline link failures in later levels in [04](04-graphics.md)). 10. For a game update: extract it over the base game, diff the imports, carry the per-build addresses over, and rebuild the shader tables for both versions ([01](01-extract.md)). diff --git a/docs/agents/symptoms.md b/docs/agents/symptoms.md new file mode 100644 index 0000000..1614b1e --- /dev/null +++ b/docs/agents/symptoms.md @@ -0,0 +1,386 @@ +# Symptom index + +One line per symptom in the [subsystem references](../reference/): what you see (an assertion, a `[RED]` line, an abort or crash site, a visible result) and the entry that explains it. Search this page for the text in your log first (`grep -i` the function name, the `[RED]` text or the address form), then read the entry and the game notes it names. + +## Rewriter + +- `jump table <addr>` assertion in `patch_nso.py`, in a Unity / IL2CPP `main`. → [Offset jump tables through x18 read at rewrite time](../reference/rewriter.md#offset-jump-tables-through-x18-read-at-rewrite-time) +- `jump table <addr>` assertion in `patch_nso.py`, in a `subsdk`. → [Offset jump tables with the bounds check on another register](../reference/rewriter.md#offset-jump-tables-with-the-bounds-check-on-another-register) +- `jump table <addr>` assertion where the index is a copy made before the check (`mov w0,w18`, then 41 instructions holding two other jump tables on `x0`, then `cmp w18,#4; b.hi; ldrsw x18,[x11,x0,lsl #2]; add x18,x18,x11; br x18`). → [Index copied before the bounds check](../reference/rewriter.md#index-copied-before-the-bounds-check) +- `jump table <addr>` assertion on an offset table whose base was set far back (`cmp w18,#5; b.hi; ldrsw x18,[x24,x18,lsl #2]; add x18,x18,x24; br x18`). → [Offset table base set far back in the function](../reference/rewriter.md#offset-table-base-set-far-back-in-the-function) +- `jump table <addr>` assertion on an offset table whose base was set far back and moved through other registers (`ldrb w18,[x14]; sub w18,w18,#1; cmp w18,#6; b.hi; ldrsw x18,[x7,x18,lsl #2]; add x18,x18,x7; br x18`). → [Far table base passed through other registers](../reference/rewriter.md#far-table-base-passed-through-other-registers) +- `jump table <addr>` assertion, entries relative to an `adr`. → [Offset tables relative to an adr](../reference/rewriter.md#offset-tables-relative-to-an-adr) +- `jump table <addr>` assertion on a byte table indexed by x30 (`adr x18, #L; ldrb w30, [x14, x12]; add x18, x18, x30, lsl #2; br x18`). → [Byte tables indexed by x30 (capstone register names)](../reference/rewriter.md#byte-tables-indexed-by-x30-capstone-register-names) +- `jump table <addr>` assertion on a byte table whose `add` is far from the `br` (`adr x18,#L; ldrb w0,[x17,x15]; add x18,x18,x0,lsl #2`, then 17 register moves before `br x18`). → [Byte tables whose add is far from the br](../reference/rewriter.md#byte-tables-whose-add-is-far-from-the-br) +- `jump table <addr>` assertion in `patch_nso.py`, in a Unity / IL2CPP `main`, at a `blr x18`. → [Indirect calls through x18](../reference/rewriter.md#indirect-calls-through-x18) +- `jump table <addr>` assertion on a `br x18` with no table in front of it. → [Indirect tail calls through x18 in IL2CPP thunks](../reference/rewriter.md#indirect-tail-calls-through-x18-in-il2cpp-thunks) +- job system hangs; a queue claims thousands of tasks but every slot is empty. → [Exit stub order in exclusive windows](../reference/rewriter.md#exit-stub-order-in-exclusive-windows) +- `branch into window <addr>` assertion. → [Out-of-line retry blocks of atomic updates](../reference/rewriter.md#out-of-line-retry-blocks-of-atomic-updates) +- branches to an `ldxr … stxr` loop's head landed mid-window. → [A window never spans a second exclusive load](../reference/rewriter.md#a-window-never-spans-a-second-exclusive-load) +- `AssertionError: 0x…` in `site_window` on an `adrp` inside an exclusive window (`ldaxr x9,[x9]; cmp x9,x25; b.ne out; adrp x10 + add x10` (the same address again, for the store); `mov x18,x26; stlxr w9,x8,[x10]`, then loads and `cbnz w9` back). → [adrp inside an exclusive window](../reference/rewriter.md#adrp-inside-an-exclusive-window) +- `B()` assertion at a spin wait: a branch into the patch area beyond the ±128 MB a `b` reaches. → [Halfword jump table landing pads sized to the largest entry](../reference/rewriter.md#halfword-jump-table-landing-pads-sized-to-the-largest-entry) +- `B()` range assertion in `patch_nso.py` (`site_plain`), only for one module. → [Patch area before the module base for a large bss](../reference/rewriter.md#patch-area-before-the-module-base-for-a-large-bss) +- `B()` range assertion, then `patch area of … bytes does not fit in the … before the base`. → [Patch area right before the base for a long text](../reference/rewriter.md#patch-area-right-before-the-base-for-a-long-text) +- `observation gate neither at a function entry nor after a bl`; `observation gate neither at a function entry, after a bl nor before a call` on an epilogue. → [Observation gates where x9–x17 are dead](../reference/rewriter.md#observation-gates-where-x9x17-are-dead) +- `jump table <addr>` assertion on an ordinary byte table near the start of the text (`adr x18; ldrb w0,[x17,x16]; add x18,x18,x0,lsl #2; br x18`). → [Code before the first unwind entry is one function](../reference/rewriter.md#code-before-the-first-unwind-entry-is-one-function) +- the rewrite never finishes, the Mac swaps, the rewriter's footprint reaches 100 GB. → [Long functions analysed in pieces](../reference/rewriter.md#long-functions-analysed-in-pieces) +- `KeyError` at `site_br` for a `br x18` that is an ordinary byte jump table. → [Disassembly goes on after undecodable words](../reference/rewriter.md#disassembly-goes-on-after-undecodable-words) +- `RuntimeError: 788 ldr x18, #0xd1eb0` in `patch_nso.py`, or any rewrite site in data. → [Data blocks at the start of the text](../reference/rewriter.md#data-blocks-at-the-start-of-the-text) +- `jump table <addr>` assertion in `patch_nso.py`, in a Unity / IL2CPP `main`, at an `alloca` (`mov sp, x18`). → [Stack pointer writes through x18](../reference/rewriter.md#stack-pointer-writes-through-x18) +- `CRASH sig=10 addr=0` on an `stp` relative to `sp`, with `sp` not a multiple of 16 (`main+0x2bbba4`, `sub sp, sp, #0xc0` then `stp d15, d14, [sp, #0x60]` with sp ending in `…9dc`). → [Stack arguments reserved with sub sp, sp, xN](../reference/rewriter.md#stack-arguments-reserved-with-sub-sp-sp-xn) +- SIGILL / trap on a system register read. → [System register reads that macOS traps](../reference/rewriter.md#system-register-reads-that-macos-traps) +- every pointer in `.data` / the GOT is a module offset, crashes everywhere. → [Packed relative relocations (DT_RELR)](../reference/rewriter.md#packed-relative-relocations-dt_relr) + +## Memory and address space + +- large resources silently skipped, later a null deref where the resource is used; also visible as `SetHeapSize` asking for much less than the pool. → [Partitioning the guest block between alias region, stacks and heap](../reference/memory.md#partitioning-the-guest-block-between-alias-region-stacks-and-heap) +- `nnMain` returns right after `nv::InitializeGraphics`, with no error (Hyrule Warriors: Age of Calamity); `[mem] SetHeapSize … + modules … > app pool` (Monster Hunter Rise), `[mem] SetHeapSize … + modules 0x5c00000 > app pool` (Catherine: Full Body); `SetHeapSize` asks for almost the whole application pool and fails at boot (Metroid Dread: 0xc9400000, larger than the heap region); the memory pool for the game's 96 MB GPU heap has no storage (`MemoryPoolInitialize` red line), then a null read at `main+0x2eb954` (Catherine: Full Body); heap allocations fail late in a busy scene. → [Heap sized from the application pool less used memory](../reference/memory.md#heap-sized-from-the-application-pool-less-used-memory) +- `StandardAllocator::Initialize` aborts in `nninitStartup` (Fire Emblem Engage); `nn::diag` abort in `StandardAllocator::Initialize(void *, size_t, bool)` from `nninitStartup` (`sdk+0x2f98e4` ← `main+0x5b69e0`, Big Brain Academy: Brain vs. Brain). → [Virtual address memory follows the NPDM's system resource](../reference/memory.md#virtual-address-memory-follows-the-npdms-system-resource) +- the first `AllocateAddressRegion` fails with 0x1803; `StandardAllocator::Initialize` aborts in `nninitStartup` (an `nn::diag` abort; the NSP script's heap check reports "the host stopped before nnMain"); the alias region fills in minutes; regions a game reserves later fail (Size-aligned regions exhaust the alias address space, The movie player's own StandardAllocator). → [The SDK's address-region node area](../reference/memory.md#the-sdks-address-region-node-area) +- a null-based write minutes into play, at the pool size minus 0x20 (Engage: a write to `NULL + 0x1fffe0`, `addr=0x1fffe0` at `main+0x9a4664`); NULL index pointers written on several job threads at once (`main+0xdb0ad4`, `0xdaea58`, `0xdadb5c`), with 32-bit mesh indices turning up inside texture objects. → [Size-aligned regions exhaust the alias address space](../reference/memory.md#size-aligned-regions-exhaust-the-alias-address-space) +- a cut engine region still does not fit; two large regions lose up to 1 GB each to alignment, depending on where `mmap` put the guest block (Absolum); the region comes 0.69 GB past the 1 GB node area, at the next 1 GB boundary (Paper Mario: The Thousand-Year Door). → [1 GB alignment of regions of 1 GB and more](../reference/memory.md#1-gb-alignment-of-regions-of-1-gb-and-more) +- nothing aborts at startup as with Unity 2022.3, but the provisional profile gave a 1 GB block and a 256 MB alias region while Unity maps 1.5 GB in the first room. → [Unity 6 takes its memory with no StandardAllocator and no heap](../reference/memory.md#unity-6-takes-its-memory-with-no-standardallocator-and-no-heap) +- `nn::diag` abort from `StandardAllocator::Initialize` long after `nninitStartup`, right after the first Vulkan query pool (`sdk+0x12dad0` ← `main+0x62c52c`). → [The movie player's own StandardAllocator](../reference/memory.md#the-movie-players-own-standardallocator) +- `StandardAllocator::Initialize` aborts in `nninitStartup` (Pokémon Legends: Z-A: `main+0x1b2e690`); `nn::diag` abort at `sdk+0x11ff34` (`StandardAllocator::Initialize`) in `nninitStartup`, SDK 15.3.2, called from `main+0x31370c` (Paper Mario: The Thousand-Year Door). → [StandardAllocators over 12 to 62 GB of address space](../reference/memory.md#standardallocators-over-12-to-62-gb-of-address-space) +- `nn::diag` abort at `sdk+0x12dad0` in `nninitStartup`, called from `main+0x756310`. → [.NET NativeAOT malloc and the GC region range](../reference/memory.md#net-nativeaot-malloc-and-the-gc-region-range) +- the game aborts in its static initialisers, before `nnMain`, when its own 62 GB address region fails (`main+0x37c9bd0`, `AllocateAddressRegion(&p, 0xf80000000)` at `main+0x37c9a5c`). → [A 62 GB page allocator behind `nu::VirtualAllocHook`](../reference/memory.md#a-62-gb-page-allocator-behind-nuvirtualallochook) +- the game calls `exit(1)` in `nninitStartup` (Octopath Traveler 0: `main+0x12e74`); `nn::diag::Abort` from `main+0x352fbec` (Unreal's fatal error) after the first commit, `AllocateMemoryPages(0, 2 MB)` while the RHI starts (`main+0x353ff8c`, result 0xe03) (Pikmin 4); a crash in `NumberLineAllocator::Compaction` at boot (DRAGON QUEST XI S, region option 4224). → [Unreal's engine address region and its options](../reference/memory.md#unreals-engine-address-region-and-its-options) +- a null read in the out-of-memory report at `main+0x1005824` when the engine region is smaller than the allocator's pools (Octopath Traveler 0). → [Unreal's FMallocBinned3 replaced by the host allocator](../reference/memory.md#unreals-fmallocbinned3-replaced-by-the-host-allocator) +- `nn::diag` abort from `main+0x11bc`, no message, in the first battle, with about 1 GB mapped (FINAL FANTASY RESONANCE with DRAGON QUEST VII Reimagined's 2 GB region). → [Unreal 5.6's page allocator never gives address space back](../reference/memory.md#unreal-56s-page-allocator-never-gives-address-space-back) +- write to `0x268` right after the title screen (`main+0x5d6708`, a constructor whose `this` is `0x200`). → [Ring heaps cut from a pool at 64 KB alignment](../reference/memory.md#ring-heaps-cut-from-a-pool-at-64-kb-alignment) +- C++ exception aborts in `nn::ro::detail::GetExceptionInfo`. → [`svcQueryMemory` must report each module's real layout](../reference/memory.md#svcquerymemory-must-report-each-modules-real-layout) +- `MapPhysicalMemory` faults on part of a page. → [`MapPhysicalMemory` in 16 KB host pages](../reference/memory.md#mapphysicalmemory-in-16-kb-host-pages) +- the alias region's mapped total runs past the application pool in the first minute. → [Maps past the application pool fail as on Horizon](../reference/memory.md#maps-past-the-application-pool-fail-as-on-horizon) +- a write through a null pointer at +0x1c0 during startup. → [`GetInfo(UserExceptionContextAddress)` returns a real buffer](../reference/memory.md#getinfouserexceptioncontextaddress-returns-a-real-buffer) +- a null-pool buffer even with virtual pools working (Virtual memory pools mapped onto one physical pool). → [A null-pool buffer from a game heap that ran out](../reference/memory.md#a-null-pool-buffer-from-a-game-heap-that-ran-out) +- a null-pool buffer for `Model/NpcCommonerA.bfres` in the opening course. → [Per-resource heaps sized from Nintendo's measured peaks](../reference/memory.md#per-resource-heaps-sized-from-nintendos-measured-peaks) +- `main+0x155a7a4` writes through a null `operator new` at the world load, about 1 run in 3. → [Model heaps sized by the host's command record sizes](../reference/memory.md#model-heaps-sized-by-the-hosts-command-record-sizes) +- the frames stop at the first loading screen, a graphics worker at 100% CPU (17 s in, `UnityGfxDeviceWorker` running, the main thread waiting on a condition variable, no red line). → [Unity's GPU pool block allocation spins at size 0](../reference/memory.md#unitys-gpu-pool-block-allocation-spins-at-size-0) +- SIGBUS inside a host hook (`snprintf` under `mount_ex` / `h_fs_MountSave`), then the process hangs (Super Mario 3D World + Bowser's Fury); SIGBUS inside Metal (`AGCDeserializedReply`, an AGX compiler block) at an address in the guest stack region (Breath of the Wild). → [Host calls on small guest stacks](../reference/memory.md#host-calls-on-small-guest-stacks) +- the iPhone app is killed for memory; resident memory grows steadily (about 0.2 MB a frame) while the residency set and the texture counts stay flat. → [Deferred-release drain without an autorelease pool](../reference/memory.md#deferred-release-drain-without-an-autorelease-pool) + +## Threads + +- per-core job queues never drained. → [Core mask changes move the ideal core into the new mask](../reference/threads.md#core-mask-changes-move-the-ideal-core-into-the-new-mask) +- crash indexing a per-core table with a huge offset (index −2). → [Ideal core −2 resolves to the process default core](../reference/threads.md#ideal-core-2-resolves-to-the-process-default-core) +- the main thread waits forever on an event right after `nnMain` starts, with only three or four threads created; `DM_THR=1` shows no `CreateThread` for the thread it waits on. → [Free thread count (`svcGetInfo` type 24)](../reference/threads.md#free-thread-count-svcgetinfo-type-24) +- in garbage-collected titles (Unity / IL2CPP), live objects freed, with their finalizers releasing native resources. → [Pausing threads and reading their context for garbage collectors](../reference/threads.md#pausing-threads-and-reading-their-context-for-garbage-collectors) +- crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`. → [Approximating same-core preemption (`core_preempt`)](../reference/threads.md#approximating-same-core-preemption-core_preempt) +- several threads crash at once and the dumps interleave, then the process hangs instead of exiting. → [Simultaneous crashes of several threads](../reference/threads.md#simultaneous-crashes-of-several-threads) +- none seen; found while investigating Fire Emblem Engage's demo crash, which it did not change (that was the address-space shortage, Size-aligned regions exhaust the alias address space). → [`SignalToAddress` type 2 with count ≤ 0](../reference/threads.md#signaltoaddress-type-2-with-count--0) +- a 64-bit wait could return at once or block when it should not (not met in a game). → [`WaitForAddress` type 3: 64-bit wait if equal](../reference/threads.md#waitforaddress-type-3-64-bit-wait-if-equal) +- a PhysX worker crashes in a task's `removeReference` with a null task manager, right as the game switches to 30 fps (`pc=main+0x10ccf28`, `lr=main+0x13c236c`); on the iPhone and in 2 of 5 Mac runs, in every Mac run with 12 busy loops loading the CPU. → [Late PhysX task destructor after the unlock](../reference/threads.md#late-physx-task-destructor-after-the-unlock) +- a 30 Hz game thread runs at 27 Hz, and whatever it paces comes out slow (cutscenes at 36 ms per frame instead of 33.3; gameplay was fine at 60). → [Timer slack in guest sleeps (`sleep_precise`)](../reference/threads.md#timer-slack-in-guest-sleeps-sleep_precise) +- a guest thread keeps a core at 100% with billions of `SleepThread` calls. → [Sleeps under 20 µs take the yield path](../reference/threads.md#sleeps-under-20-µs-take-the-yield-path) +- a loading screen sits for a minute with the CPU nearly idle; one thread yields thousands of times a second, every yield a back-off sleep (the power line's top yielder, with as many back-off sleeps). → [Yields that make progress return at once (`thr_yield`)](../reference/threads.md#yields-that-make-progress-return-at-once-thr_yield) +- the main thread waits forever on a job's done event at boot while every worker sleeps, two of them on one event (`main+0x1426c` waiting on `Job::m_doneEvent`); 4 of 6 Mac runs. → [One auto-clear event shared by a pool's workers](../reference/threads.md#one-auto-clear-event-shared-by-a-pools-workers) +- abort in `os::WaitSystemEvent` under `oe::PopNotificationMessage` (The Legend of Zelda: Link's Awakening); one guest thread at about 1000 ms/s on the iPhone only, the phone hot, CPU energy 5–7 W (Monster Hunter Rise's `MessageSuperviseThread`, priority 32, in the `[nvn] threads` lines). → [The oe notification event and its queue](../reference/threads.md#the-oe-notification-event-and-its-queue) + +## Services and SDK calls + +- `sm::Initialize` aborts right after `ConnectToNamedPort` (SVC 0x1F) and `SendSyncRequest` (0x21); abort inside `nn::sm::Initialize`, the log showing `SVC 0x1f` (ConnectToNamedPort, port name `sm:`) and `SVC 0x21` answered by placeholders, then an `nn::diag` abort; abort with no message at start-up in a service call the host has no hook for (`Break` from `sdk+…` with an `nn::diag` backtrace). → [SDK calls that open a system service through sm](../reference/services.md#sdk-calls-that-open-a-system-service-through-sm) +- a game update aborts at boot where the base game ran. → [A game update imports SDK calls the base game did not](../reference/services.md#a-game-update-imports-sdk-calls-the-base-game-did-not) +- abort in `sm::Initialize` under `sf::hipc::InitializeHipcServiceResolution`, right after the first frames (from `main+0x5b8198`). → [Game-linked HIPC clients of system services](../reference/services.md#game-linked-hipc-clients-of-system-services) +- a call the host hooks under its newer name reaches the SDK's own code on an older game. → [Older SDK names for the same calls](../reference/services.md#older-sdk-names-for-the-same-calls) +- abort in `sm::Initialize` under `nn::friends::GetBlockedUserList` after the opening movie. → [SDK 4's `friends::GetBlockedUserList` without a `Uid`](../reference/services.md#sdk-4s-friendsgetblockeduserlist-without-a-uid) +- the first loading screen never ends; `DM_FS=1` shows `save:/system` "not found" and no `fs mount save`. → [SDK 6's `account::OpenPreselectedUser` returns a Result](../reference/services.md#sdk-6s-accountopenpreselecteduser-returns-a-result) +- IL2CPP "Could not open …/global-metadata.dat" (then a crash in its logger). → [The SDK's own `fs::IsMounted` reads an empty mount table](../reference/services.md#the-sdks-own-fsismounted-reads-an-empty-mount-table) +- Addressables never start (no `StreamingAssets/aa/settings.json` in the `DM_FS=1` output at all), then a null asset crash (`main+0x31100ec`). → [C library path checks ask the file system's limits](../reference/services.md#c-library-path-checks-ask-the-file-systems-limits) +- an Atk task aborts in a `switch` default (`UnexpectedDefaultImpl`, then `AbortImpl`) while a stream sound loads. → [Paths with `.` and `..` components](../reference/services.md#paths-with--and--components) +- a long black screen at boot with nothing in the log (Engage: about 17 s on the Mac, 32 s on an iPhone); sampling the process (`sample <pid> 3`) shows `getxattr` under `path_component` at the top of stack. → [RomFS lookups on large directories](../reference/services.md#romfs-lookups-on-large-directories) +- `[RED] fs ReadFile into … failed at +0 (EFAULT): VM region … protection 1`, now and then. → [ReadFile into write-protected texture pool pages](../reference/services.md#readfile-into-write-protected-texture-pool-pages) +- `fs … not found` lines with `DM_FS=1`. → [File-not-found noise from optional paths](../reference/services.md#file-not-found-noise-from-optional-paths) +- a missing `fs::DeleteDirectory`; undefined `nn::fs::` / `nn::account::` imports with no hook, found from the import list (Listing a game's unhooked imports before they are met) before they were met. → [Directory, file data cache and save-thumbnail calls](../reference/services.md#directory-file-data-cache-and-save-thumbnail-calls) +- the SDK's own code opened fsp-srv and aborted, leaving the save writes uncommitted. → [`fs::Commit` with a list of mounts](../reference/services.md#fscommit-with-a-list-of-mounts) +- SIGSEGV at address 0 in `nn::fs::GetSaveDataSizeMax`, about 13 minutes into a new game. → [Save data size queries](../reference/services.md#save-data-size-queries) +- each aborted in `sm::Initialize` or read a null applet proxy, one per boot. → [Cache storage and other titles' save data](../reference/services.md#cache-storage-and-other-titles-save-data) +- the island lives in device save data, which the host did not mount. → [Device save data on the account save's directory](../reference/services.md#device-save-data-on-the-account-saves-directory) +- every boot asks to pick a Mii again (`DM_FS=1`: `UserSave:/userdata.dat` not found although the host save has it). → [Device and account save data as two saves](../reference/services.md#device-and-account-save-data-as-two-saves) +- abort in `fs::detail::GetFileSystemProxyServiceObject` under `fs::QueryMountSystemDataCacheSize` (`main+0x8f23bc`). → [System data stand-in for the MiiModel archive](../reference/services.md#system-data-stand-in-for-the-miimodel-archive) +- DLC the player owns is not unlocked. → [Add-on content from `DM_AOC_DIR`](../reference/services.md#add-on-content-from-dm_aoc_dir) +- SIGSEGV in `crypto::detail::CbcModeAes128Helper::DecryptBlocks` with a null destination (`main+0x8737e8`, right after `aoc:` is mounted). → [Add-on content mount cache sized as RomFS working memory](../reference/services.md#add-on-content-mount-cache-sized-as-romfs-working-memory) +- SIGBUS in `__vfprintf` under `aoc_load`, from `h_aoc_Count` (a store to `0x…67ff0`, right below a page of the guest thread's stack). → [Add-on content queries on a small guest stack](../reference/services.md#add-on-content-queries-on-a-small-guest-stack) +- buttons and menus work, the sticks do nothing in play (Super Mario Odyssey: its first hint, "Look Around", waited forever); an `nn::diag` abort with Result 0x2aca in the first frames of `nnMain` (Hollow Knight: Silksong, Unity 6000.0.50f1: `bt: sdk+0x514580 main+0x1296c64 …`). → [History reads fill every requested entry](../reference/services.md#history-reads-fill-every-requested-entry) +- the game never reads the controller (the title's "Press L+R" never sees a press; no `GetNpadState` call at all). → [Npad style set update event](../reference/services.md#npad-style-set-update-event) +- motion moves never happen; shaking does nothing (Super Mario Odyssey: cap throws by shaking). → [Six-axis sensors](../reference/services.md#six-axis-sensors) +- SDK 20 calls that reach a service; `nfp`'s `AttachActivateEvent` / `AttachDeactivateEvent` crashed in the SDK at `sdk+0x3f2a78`. → [SDK 20 six-axis sensor and amiibo (`nfp`) calls](../reference/services.md#sdk-20-six-axis-sensor-and-amiibo-nfp-calls) +- abort in `nn::sm::Initialize` under `nn::hid::VibrationNode::Update`. → [Vibration values flushed by the SDK's vibration graph](../reference/services.md#vibration-values-flushed-by-the-sdks-vibration-graph) +- taps in handheld mode do nothing, though the game reads the touch screen (the title's PRESS ANY BUTTON, the menus' cards). → [Touch gestures recognized from the touch points](../reference/services.md#touch-gestures-recognized-from-the-touch-points) +- the first boot aborted in `hid::GetMouseState` (the SDK's mouse resource was never initialized). → [Mouse, keyboard and key code maps](../reference/services.md#mouse-keyboard-and-key-code-maps) +- the first boot crashed in `hid::detail::SetPalmaBoostMode`, a null read at `sdk+0x238eb0` (the hid service never opened). → [Poké Ball Plus (Palma)](../reference/services.md#poké-ball-plus-palma) +- undefined `irsensor` imports found from the import list (Listing a game's unhooked imports before they are met). → [IR camera reported unconnected](../reference/services.md#ir-camera-reported-unconnected) +- null read in `sdk` under `nn::applet::CreateLibraryApplet` (Engage: title menu → DLC → `nn::ec::ShowShopAddOnContentList`); `Show…` overloads found in the imports before they were met. → [System applets opened by `Show…` calls](../reference/services.md#system-applets-opened-by-show-calls) +- the sound stutters and the input box lags while the player types a name (Octopath Traveler 0 on an iPhone: one 4.7 s frame, 5,035 wakeups a second instead of about 1,800, and no audio underruns). → [The software keyboard blocks only its caller](../reference/services.md#the-software-keyboard-blocks-only-its-caller) +- null read in `applet::CreateLibraryApplet` under `swkbd::InlineKeyboard::Initialize` (3D World, entering the save select after the character select). → [The inline keyboard as a resident library applet](../reference/services.md#the-inline-keyboard-as-a-resident-library-applet) +- texture creation returns null right after the inline keyboard's `Initialize`, then a null read at `main+0x7713044`. → [Inline keyboard window size, setters and callbacks](../reference/services.md#inline-keyboard-window-size-setters-and-callbacks) +- the hunter's name box stays empty, then stays open with the name in it (Monster Hunter Rise, character creation). → [Inline keyboard text callbacks and `Calc` states](../reference/services.md#inline-keyboard-text-callbacks-and-calc-states) +- `CRASH sig=10` at a heap address, called from `main+0x6bca160` right after the name was entered (Monster Hunter Rise); crash in `lmem::AllocateFromExpHeap` under `ngc::ProfanityFilter::MaskProfanityWordsInText` right after the character name is entered (DARK SOULS: REMASTERED); filter methods among the undefined imports with no hook, found from the import list before they were met (The Great Ace Attorney Chronicles). → [`nn::ngc::ProfanityFilter` vtable and by-name overloads](../reference/services.md#nnngcprofanityfilter-vtable-and-by-name-overloads) +- SIGSEGV in `vi::CreateLayer(Layer**, Display*, int, int)` (`sdk+0x3f9bc4`, from SDL3's Switch video driver at `main+0x239304`). → [`vi::CreateLayer` with the layer's size](../reference/services.md#vicreatelayer-with-the-layers-size) +- SIGSEGV in `vi::SetLayerCrop`. → [vi layer and display calls without vi service objects](../reference/services.md#vi-layer-and-display-calls-without-vi-service-objects) +- SIGSEGV in `android::Surface::getQueuedFrameNumber` under `nn::vi::GetLatestFrameNumber`. → [Layer frame numbers](../reference/services.md#layer-frame-numbers) +- SIGSEGV in `vi::GetDefaultDisplayPanelIdentifier` (`main+0xa9a4`, before the window). → [Default display panel identifier](../reference/services.md#default-display-panel-identifier) +- the first boot crashed in `oe::DisableAlbumImageTakenNotification`: a null read at `sdk+0x338070`, the first call in `nnMain` (`main+0x6b188c`); other `oe` calls that reach am, found in the imports before they were met. → [oe calls answered without an applet proxy](../reference/services.md#oe-calls-answered-without-an-applet-proxy) +- abort in `nn::sm::Initialize` under `nn::oe::FinishStartupLogo`, right after the first frames. → [The startup logo](../reference/services.md#the-startup-logo) +- null read in `oe::InitializeCopyrightFrameBuffer`, right after the window setup. → [Copyright frame buffer overlay](../reference/services.md#copyright-frame-buffer-overlay) +- SDK 20 calls that reach a service, each one boot; SDK 20's `OpenAudioRenderer` aborts on an ARUID of 0. → [Applet resource user ID, launch parameters and VR mode](../reference/services.md#applet-resource-user-id-launch-parameters-and-vr-mode) +- a null read in `sdk+0x35258` (apm's session) in `oe::IsCpuOverclockEnabled`, the first call in `nnMain`; null read at `sdk+0x362b8` (`apm::SetCpuOverclockEnabled`) in the first calls of `nnMain` (`main+0x2e1e4c`). → [CPU overclock calls without an apm session](../reference/services.md#cpu-overclock-calls-without-an-apm-session) +- the game freezes after switching apps and back, picture still, sound on; the main loop waits in its pause handshake (`main+0x5d10d4` / `main+0x5cea68`). Mac repro `DM_OE_SCRIPT=bg=1@20,bg=0@25`. → [Focus handling mode decides whether focus changes are notified](../reference/services.md#focus-handling-mode-decides-whether-focus-changes-are-notified) +- abort in `sm::Initialize` under `capsrv::InitializeForApplication` (`album::Initialize`); calls that reach the album service, found in the imports before they were met. → [Album calls fail with no album](../reference/services.md#album-calls-fail-with-no-album) +- a null read at `sdk+0x336ccc` in `pdm::QueryApplicationPlayStatistics` (Pokémon Sword, the second boot); undefined `nn::account::` / `nn::friends::` imports with no hook, found from the import list (Listing a game's unhooked imports before they are met) before they were met. → [Account, friends, play data and Nintendo Account calls](../reference/services.md#account-friends-play-data-and-nintendo-account-calls) +- `[RED] sf: the game connects to service mii:e … mii:u itself`, then an abort. → [Mii database answered by the host](../reference/services.md#mii-database-answered-by-the-host) +- the abort with no message (`SVC 0x15` / `0x1f` / `0x21` placeholders, then `Break` with `sm::Finalize` under `ShopServiceAccessServer::Proxy::Initialize`) from `ec::InitializeForShopServiceAccessors`, called from Unreal's `FOnlineSubsystemSwitch::Init` at the start of `nnMain`. → [Shop service accessors](../reference/services.md#shop-service-accessors) +- abort in `nn::sm::Initialize` under `nn::pctl::IsFreeCommunicationAvailable` (Engage, entering its first battle map). → [Parental controls and free communication](../reference/services.md#parental-controls-and-free-communication) +- play report calls met entering the save select (3D World); undefined `nn::ldn::` / `nn::socket::` imports with no hook, found from the import list before they were met (Big Brain Academy). → [Network interface, LDN, socket and play report calls](../reference/services.md#network-interface-ldn-socket-and-play-report-calls) +- the boot sits at a black loading screen for good, CPU nearly idle, every thread waiting, about 1 boot in 6 (3 in 4 with four Mac hosts at once); the title movie is never opened, and the save files are never probed. → [Network requests stay on hold before they settle](../reference/services.md#network-requests-stay-on-hold-before-they-settle) +- one thread at 100% CPU from boot, with one SVC in 99 s (`NexusRevolution Socket`, `main+0x92c7c4`, the game's network thread). → [Socket `Poll` and `Select` with nothing to wait on](../reference/services.md#socket-poll-and-select-with-nothing-to-wait-on) +- the first comparison in `nnMain` aborted in `sm`. → [Language codes](../reference/services.md#language-codes) +- abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodes` (early in `nnMain`, `main+0x327888`); abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodeCount()` (`main+0x117590`). → [Available language codes and their count](../reference/services.md#available-language-codes-and-their-count) +- the font engine runs on garbage; crash after the shared fonts load, looking up the "?" glyph (`main+0x6c73c0`, a font cache lookup returning null), with SDK 10's `ScalableFontEngine::LoadFont` failing with error 403 and `SetFont` with 301 on both stand-in fonts; or Japanese, Korean or Chinese text draws nothing in a Unity game while Latin text is fine (Patrick's Parabox's language menu showed three blank lines). → [Shared font address: BFTTF header or bare TTF](../reference/services.md#shared-font-address-bfttf-header-or-bare-ttf) +- abort in `nn::sm::Initialize` under `nn::pl::sharedresource::GetPlatformSharedResourceManager` (first thing in `nnMain`). → [Shared fonts requested all at once](../reference/services.md#shared-fonts-requested-all-at-once) +- controller-button icons in text show as `?`. → [Nintendo-extension glyphs for button icons](../reference/services.md#nintendo-extension-glyphs-for-button-icons) +- on a phone, `GetSharedFontLoadState` would report "not loaded". → [Shared fonts on iOS](../reference/services.md#shared-fonts-on-ios) +- SDK calls that reach the time service, each aborting in `sm::Initialize` (or found in the imports before they were met). → [Steady clock, clock snapshots and user clocks](../reference/services.md#steady-clock-clock-snapshots-and-user-clocks) +- `std::terminate` (`Break` from `sdk+0x5bfa48`) at the first real-time read (right after the first-boot dialog, `main+0x3f3f68` → `system_clock::now()`). → [`clock_gettime(CLOCK_REALTIME)` through the time source pointer](../reference/services.md#clock_gettimeclock_realtime-through-the-time-source-pointer) +- after switching apps and back, one guest thread at 100% in a tight loop (`main+0x9abd0`, `x10 = 0`); the game freezes, picture still, sound on. Mac repro `DM_OE_SCRIPT=bg=1@20,bg=0@25`. → [Guest clock reads wait while the app is in the background](../reference/services.md#guest-clock-reads-wait-while-the-app-is-in-the-background) +- abort in `sm::Initialize` under `nn::crypto::GenerateCryptographicallyRandomBytes` (Unicorn Overlord: `std::random_device` → the `sdk`'s own `getentropy` → `InitializeCsrng`; Z-A: the first call in `nnMain`, `main+0x1734b4`). → [Cryptographically random bytes](../reference/services.md#cryptographically-random-bytes) +- a library's `free` rejects a pointer (`CentralHeap::Free` → `NotNMallocPtrError`). → [Global symbol resolution in load order](../reference/services.md#global-symbol-resolution-in-load-order) +- a weakly imported function the game calls although nothing defines it: an `UNIMPL` line, then an abort. → [Undefined weak imports bind to 0](../reference/services.md#undefined-weak-imports-bind-to-0) +- crash in `nn::ro::InitializeWithPortName`. → [`nn::ro` and its port](../reference/services.md#nnro-and-its-port) +- the frame loop runs, but the game waits for code it loads itself (`ro::RegisterModuleInfo` / `ro::GetBufferSize` red lines; `main` imports 1,045 `lua2cpp::create_agent_*` functions, global, that no ExeFS module defines, bound to stubs at start). → [Loading NROs with `ro::LoadModule`](../reference/services.md#loading-nros-with-roloadmodule) +- `[EOS] Failed to create platform` (the game plays offline); `refused …: an online SDK (Epic Online Services), not loaded`; `refused an NRR that registers only online SDKs (1 hashes)`. → [Online SDKs in NROs are never loaded](../reference/services.md#online-sdks-in-nros-are-never-loaded) +- `main` imports EGL (SDL3's GL support) and `vkGetInstanceProcAddr` (its Vulkan support), but the game renders with NVN (`FNA3D_Driver_NVN.cpp`). → [GPU driver modules that are never loaded](../reference/services.md#gpu-driver-modules-that-are-never-loaded) +- harmless on the build it was written for, silent memory corruption on any other. → [Code inherited from another title at fixed addresses](../reference/services.md#code-inherited-from-another-title-at-fixed-addresses) + +## Movies + +- abort in NVIDIA's video path (`NvRmChannelOpen` → `NvOsDrvOpen`). → [The SDK's hardware decoder opens nvdrv](../reference/movies.md#the-sdks-hardware-decoder-opens-nvdrv) +- the game waits forever at a movie (Engage's 94 s opening `S01F.webm`; frames keep presenting, nothing happens). → [`movie::Decoder` taken over as a MediaCodec-style protocol](../reference/movies.md#moviedecoder-taken-over-as-a-mediacodec-style-protocol) +- a movie in grey with vertical bands of colour, its luma sharp; or a read far outside guest memory at `main+0x344f74` on the first movie picture (CRI Sofdec2's NV12 → RGB conversion). → [Output format keys (`md_format`)](../reference/movies.md#output-format-keys-md_format) +- a call through a null pointer in the host's `movie::Decoder::Configure` right after `nnMain` starts the logos. → [SDK 4's `movie::Decoder` constructor without a priority](../reference/movies.md#sdk-4s-moviedecoder-constructor-without-a-priority) +- the first deferred signal crashed in `InternalCriticalSectionImplByHorizon::Enter` (DRAGON QUEST XI S). → [Client events read through the registered pointer](../reference/movies.md#client-events-read-through-the-registered-pointer) +- the process dies when an activity starts, 2 runs in 5, with no `CRASH` line: the run ends with exit 137 (SIGKILL), and `~/Library/Logs/DiagnosticReports/dmhost-*.ips` has `EXC_BREAKPOINT` in `_os_unfair_lock_corruption_abort` ← `pthread_mutex_destroy` ← `md_free` (the movie decoder's destructor). → [Input indices returned by a flush while a client holds them](../reference/movies.md#input-indices-returned-by-a-flush-while-a-client-holds-them) +- when the prologue movie was skipped, 2.25 GB were mapped within seconds, past the application pool, and the game crashed in its allocator (DRAGON QUEST XI S). → [End of stream is the last input until `Flush` or `Stop`](../reference/movies.md#end-of-stream-is-the-last-input-until-flush-or-stop) +- decoding the end-of-stream packet gave VideoToolbox −12909 and AudioToolbox −50. → [End-of-stream packets with no data, and deleted decoders](../reference/movies.md#end-of-stream-packets-with-no-data-and-deleted-decoders) +- the game never leaves a movie that has played to its end; the last picture stays on screen. → [H.264 drain at the end of a stream goes on with released slots](../reference/movies.md#h264-drain-at-the-end-of-a-stream-goes-on-with-released-slots) +- the opening movie never starts, in about half the boots (DRAGON QUEST XI S: a black screen and no sound from boot on). → [Audio decoders keep more outputs than a pre-decoding player fills](../reference/movies.md#audio-decoders-keep-more-outputs-than-a-pre-decoding-player-fills) +- a movie stays black after its first picture, the sound never starts, and the game keeps presenting frames. → [Audio output slots for players that hold every output](../reference/movies.md#audio-output-slots-for-players-that-hold-every-output) +- a movie shows as a black screen with the loading icon, one picture every few seconds, while its sound decoder keeps up. → [Video output slots: how far the extractor gets ahead](../reference/movies.md#video-output-slots-how-far-the-extractor-gets-ahead) +- a crash in the game's heap free (`sead::ExpHeap`) on a movie thread, a second into the first movie, at the same address every boot (`pc=main+0x42f980`, a free-list unlink through a link of 0x3f, `addr=0x47`). → [AAC decoders offer at most 32 outputs](../reference/movies.md#aac-decoders-offer-at-most-32-outputs) +- the movie then froze on its first second, its sound decoder full and the game taking no more audio. → [AAC output buffer size is one frame](../reference/movies.md#aac-output-buffer-size-is-one-frame) +- `FATAL movie: missing SDK symbol _ZNK5movie6Buffer10GetNvnSyncEv` at the first frame of the native-texture movie (H.264 into the game's RGBA texture, as in Monster Hunter Rise). → [SDK 4's `movie::Buffer` has no sync object](../reference/movies.md#sdk-4s-moviebuffer-has-no-sync-object) +- a native-texture movie shows as a strip of small grey copies at the top of the screen. → [Parameter sets in `Configure` made the H.264 decoder without RGBA](../reference/movies.md#parameter-sets-in-configure-made-the-h264-decoder-without-rgba) +- looping movie previews flash black at the end of every loop. → [Last picture kept across a flush and at end of stream](../reference/movies.md#last-picture-kept-across-a-flush-and-at-end-of-stream) +- the game crashed on a null right after the failed open. → [`movie::Extractor` runs natively](../reference/movies.md#movieextractor-runs-natively) + +## Audio + +- abort with Result 0x499 in `audio::GetAudioRendererWorkBufferSize`. → [SDK 8's longer `AudioRendererParameter`](../reference/audio.md#sdk-8s-longer-audiorendererparameter) +- the game is silent; only the game's updates render (catch-up frames that nothing plays). The log has `audren: OpenAudioRenderer (no event)` and no `render thread started`. → [A renderer opened without an event](../reference/audio.md#a-renderer-opened-without-an-event) +- `[RED] audren: a second renderer` right after the first open, with the same work buffer and "original device 1 mode 1". → [The SDK's open with an event calls the open without one (`t_in_open`)](../reference/audio.md#the-sdks-open-with-an-event-calls-the-open-without-one-t_in_open) +- none of its own; command generation races the game's `Update` on SDKs whose lock is not at SDK 16's offset. → [The renderer's update lock (`update_lock`)](../reference/audio.md#the-renderers-update-lock-update_lock) +- `CommandListProcessor::Process` crashes now and then (SIGSEGV under `render_main`, once at 60 s and once at 134 s in the shrine); every `RequestUpdate` renders a catch-up frame. → [The render thread's route through the public `ExecuteAudioRendererRendering` (SDK 20)](../reference/audio.md#the-render-threads-route-through-the-public-executeaudiorendererrendering-sdk-20) +- SIGSEGV at address 0x18 in `UpsamplerManager::Allocate` on the render thread. → [A renderer below 48 kHz with no upsampler on the CPU](../reference/audio.md#a-renderer-below-48-khz-with-no-upsampler-on-the-cpu) +- the main thread spins at start-up in a resource wait (a `YieldThread` loop, no crash), right after `OpenAudioRenderer`. → [A sound engine's heap too small for its renderer and players](../reference/audio.md#a-sound-engines-heap-too-small-for-its-renderer-and-players) +- the CPU processor drops the final mix. → [The final mix is taken from the DeviceSink's mix buffers](../reference/audio.md#the-final-mix-is-taken-from-the-devicesinks-mix-buffers) +- no sound: `[RED] audren: no DeviceSink (ID 0x10) in the command list, 0 commands of 0x148 bytes`. → [SDK 4: DeviceSink is command 0xf of 0x130 bytes](../reference/audio.md#sdk-4-devicesink-is-command-0xf-of-0x130-bytes) +- no sound; red `audren: command 1 in the command list is malformed (magic 0 size 0)`. → [SDK 6: the DSP command header layout](../reference/audio.md#sdk-6-the-dsp-command-header-layout) +- no music and no sound effects, only the movies' sound (the log repeats `audren: first-frame commands …` every frame and never prints `DeviceSink inputs`). → [SDK 8: command IDs numbered differently](../reference/audio.md#sdk-8-command-ids-numbered-differently) +- no sound; red `audren: command 0 in the command list is malformed (first word 0x181901 size 0)`. → [SDK 21 DSP command header without a magic](../reference/audio.md#sdk-21-dsp-command-header-without-a-magic) +- crackling on iOS only. → [RemoteIO's default I/O buffer duration](../reference/audio.md#remoteios-default-io-buffer-duration) +- sound stays silent after Siri, an alarm, a call or another app's audio; a game whose audio thread waits for its buffers back (Wwise's poll-only output, FMOD) then waits forever. → [Interruptions that stop RemoteIO callbacks](../reference/audio.md#interruptions-that-stop-remoteio-callbacks) +- combined numbers mislead: one fully fed output plus one fed by nobody look like "every other callback underruns, 2.0 blocks queued". → [Several outputs at once (`audio #N`)](../reference/audio.md#several-outputs-at-once-audio-n) +- a null read at `sdk+0x1f2314` as the first movie starts (the title menu's background, right after the first press), in `nn::audio::SetAudioOutVolume`, called from `main+0x1c4fbc`. → [AudioOut volume kept per output](../reference/audio.md#audioout-volume-kept-per-output) +- a null read in `audio::GetAudioOutBufferCount` on Wwise's sink wakeup thread (`sdk+0x1e7ed0`, from `main+0x30faae8`). → [AudioOut buffer count from the host queue](../reference/audio.md#audioout-buffer-count-from-the-host-queue) +- an abort in `sm::Initialize` under `audio::GetAudioDeviceOutputVolume` (the game thread, `main+0xee8edc`). → [Audio device output volume and names](../reference/audio.md#audio-device-output-volume-and-names) +- SDL3's audio driver lists the input devices (AudioIn) when its audio starts. → [No audio input devices](../reference/audio.md#no-audio-input-devices) +- an abort in `sm::Initialize` under the hardware Opus decoder's C interface; for the multi-stream one, under `nn::codec::GetHardwareOpusMultiStreamDecoderWorkBufferSize(int, int, int, int, int)` (at `nnMain`, CRI's Opus streams). An abort in `sf::hipc::InitializeHipcServiceResolution` under `codec::GetHardwareOpusDecoderWorkBufferSize(int, int)` (`main+0x31f8e8`, early in `nnMain`). A game that builds `HardwareOpusMultiStreamDecoder` with the imported constructor and then calls it through its vtable. → [Hardware Opus decoders built on the SDK's software decoders](../reference/audio.md#hardware-opus-decoders-built-on-the-sdks-software-decoders) +- the light limiter crashed on a null state pointer (`ApplyLightLimiterEffect`, `sdk+0x228494`). → [At least one rendered frame between two updates](../reference/audio.md#at-least-one-rendered-frame-between-two-updates) +- the sound runs about 50 times faster than the output; the loop runs 11,000 updates a second on one core. → [A sound driver that never waits on the renderer event](../reference/audio.md#a-sound-driver-that-never-waits-on-the-renderer-event) + +## Frame rate + +- a 30 fps game to run at 60; after the floor was lowered, slow motion on a GPU-bound device (GitHub issue #9). → [sead's dynamic present interval, `ksys::VFR` and the host's frame step](../reference/frame-rate.md#seads-dynamic-present-interval-ksysvfr-and-the-hosts-frame-step) +- slow motion when the GPU is the bottleneck: a game's interval controller that times the GPU never steps up (GitHub issue #9). → [GPU timestamps per submission segment](../reference/frame-rate.md#gpu-timestamps-per-submission-segment) +- the game presents at interval 1, yet every frame takes exactly 33.3 ms of "frame work"; after only the limiter was raised, everything ran twice as fast on the iPhone. → [A render-thread limiter with a constant and a fixed time step](../reference/frame-rate.md#a-render-thread-limiter-with-a-constant-and-a-fixed-time-step) +- 55 fps with every frame 18.2 ms, the main thread asleep most of the time; every 11th frame stays on screen two vsyncs. → [A frame-end wait polled in 2 ms sleeps](../reference/frame-rate.md#a-frame-end-wait-polled-in-2-ms-sleeps) +- "sometimes the speed increases when jumping" (Mario 1.2.1 on the iPhone): in `display` mode interval-2 frames were shown for 16.7 ms and the game visibly jumped ahead. "When pressing B, Link's move pace is speeding up" (Breath of the Wild on the iPhone); `old frames dropped by merging`. → [Frames of a dynamic present interval held on screen and at the acquire](../reference/frame-rate.md#frames-of-a-dynamic-present-interval-held-on-screen-and-at-the-acquire) +- a steady game whose frame-rate counter wanders between 50 and 60 on the iPhone only; the `display link up to tick` lines show about as many empty ticks as frames dropped by merging: frames arrive two in one tick and none in the next. → [Vulkan acquire and vsync event follow the display link](../reference/frame-rate.md#vulkan-acquire-and-vsync-event-follow-the-display-link) +- asked for 60 Hz, the system fires the display link on every third refresh of a 165 Hz screen, 55 Hz, so games paced by it ran at 55 fps (27.5 at 30). → [Virtual 60 Hz ticks on screens that are not a multiple of 60](../reference/frame-rate.md#virtual-60-hz-ticks-on-screens-that-are-not-a-multiple-of-60) +- virtual ticks keep the average, but a frame still stays up two or three refreshes in turn (13.9 / 20.8 ms at 144 Hz), and Apple's HUD showed the frame interval jumping between them. → [Fullscreen switches the screen to a rate the frame rate divides](../reference/frame-rate.md#fullscreen-switches-the-screen-to-a-rate-the-frame-rate-divides) +- the GPU sits idle each frame while the host translates the submission (a game that does not submit frame N+1 until frame N's GPU work is done). → [Submissions handed to the GPU in segments](../reference/frame-rate.md#submissions-handed-to-the-gpu-in-segments) +- the submit thread's CPU samples dominated by `mtl_addr` / `mtl_gpu_addr` (vertex buffers in `draw`, uniform buffers in `encode_args`). → [Address to memory pool lookups by binary search](../reference/frame-rate.md#address-to-memory-pool-lookups-by-binary-search) +- stalls of 30–60 ms about once a second, at 30 fps and at 60 (99th percentile 63–127 ms); the main thread waits 20–34 ms for the render thread once a second. → [Time-constraint computation declared below the thread's peaks](../reference/frame-rate.md#time-constraint-computation-declared-below-the-threads-peaks) + +## NVN objects + +- heap corruption; a stack smash where a function returned to `0x100000000`; a crash in SHA-256 on the code address 0x16300. → [Host records must fit the game's SDK object sizes](../reference/nvn-objects.md#host-records-must-fit-the-games-sdk-object-sizes) +- out-of-range vertices, NaN, a black band and flicker; seen on the iPhone, while Mac runs of the same scene looked clean. → [Inlined SDK code writes into `NVNcommandBuffer` +0x10](../reference/nvn-objects.md#inlined-sdk-code-writes-into-nvncommandbuffer-0x10) +- sprites broke apart in some runs only, depending on the allocation order; buffers overlapped. → [Sizes the hardware never reports as 0 (ZCull save area)](../reference/nvn-objects.md#sizes-the-hardware-never-reports-as-0-zcull-save-area) +- a null write as soon as the main thread starts recording a command buffer (13 Sentinels: a constructor at `main+0x2fa9a0` runs on `this = 0` when the prologue enters its first battle). → [Heaps cut from a pool at alignment relative to its GPU address](../reference/nvn-objects.md#heaps-cut-from-a-pool-at-alignment-relative-to-its-gpu-address) +- `[RED] memory pool … created at retry N`. → [`newBufferWithBytesNoCopy` retried under memory pressure](../reference/nvn-objects.md#newbufferwithbytesnocopy-retried-under-memory-pressure) +- a buffer is created with a null pool right after a scene change. → [Virtual memory pools mapped onto one physical pool](../reference/nvn-objects.md#virtual-memory-pools-mapped-onto-one-physical-pool) +- `[RED] MapVirtual: non-contiguous mapping (true sparse) not implemented`. → [Sparse virtual pools built from separate physical pools](../reference/nvn-objects.md#sparse-virtual-pools-built-from-separate-physical-pools) +- `CRASH sig=10` in `memmove` from `pkg_upload` → `nvn_deswizzle`, at an address 32 GB into a sparse virtual pool; textures with unaliased tails came out magenta. → [Textures read through a 4 KB page table of a sparse pool](../reference/nvn-objects.md#textures-read-through-a-4-kb-page-table-of-a-sparse-pool) +- GPU memory grows with every scene change (Engage's idle title demo reached 5.4 GB of GPU memory and 3.6 GB resident in 850 s). → [Finalized pools kept alive by residency and the sharing table](../reference/nvn-objects.md#finalized-pools-kept-alive-by-residency-and-the-sharing-table) +- the serial table grows by about 10 entries a frame (276,000 after 25,000 frames). → [Dead texture serials kept in a ghost table](../reference/nvn-objects.md#dead-texture-serials-kept-in-a-ghost-table) +- `FATAL NVN command memory: the game's pool is empty…`. → [Host-lent command memory segments](../reference/nvn-objects.md#host-lent-command-memory-segments) +- frames stop after about 15 s; a game thread spins in its command-memory callback (a ring allocator at `main+0x9cc120` waiting for fences that have all signalled). → [What `GetCommandMemoryUsed` counts](../reference/nvn-objects.md#what-getcommandmemoryused-counts) +- `CRASH sig=10` in `mtl_exec` after minutes of play, the fault address equal to `x19` and shaped like `0x1000000003`. → [Prerecorded lists in slots of their measured size](../reference/nvn-objects.md#prerecorded-lists-in-slots-of-their-measured-size) +- the title showed one stretched 64×64 icon over the 3D scene, a new game a white screen; post-process passes with stale texture handles; `[RED] bind record … overwritten after recording`. → [Recorded commands patched in place by the game (`OP_BIND_TEX1`)](../reference/nvn-objects.md#recorded-commands-patched-in-place-by-the-game-op_bind_tex1) +- `nvnCommandBufferFenceSync` unimplemented, every `SyncWait` timed out at once. → [Fence syncs recorded in command buffers](../reference/nvn-objects.md#fence-syncs-recorded-in-command-buffers) +- `FATAL submit re-entered` in the first frames. → [Submissions from two queues on two threads](../reference/nvn-objects.md#submissions-from-two-queues-on-two-threads) +- frames stop, no crash, the main thread sits in `nvnSyncWait(sync, -1)`, and the watchdog's wait-state line shows the main fence one short (`main fence submitted N / completed N-1`). → [A GPU command buffer that never completes](../reference/nvn-objects.md#a-gpu-command-buffer-that-never-completes) +- loading screens many times slower on the iPhone than on the Mac (Three Houses' first loading screen 78 s on the iPhone, 5 s on the Mac). → [`nvnQueueFinish` waits for the GPU, not the display](../reference/nvn-objects.md#nvnqueuefinish-waits-for-the-gpu-not-the-display) +- menus and dialogue boxes draw but contain no text, while images and pictogram digits are fine; the sky is black (CPU-written linear colour ramps). → [Linear textures in CPU-visible pools](../reference/nvn-objects.md#linear-textures-in-cpu-visible-pools) +- video frames, or a game's built-in default textures, are all zero. → [Textures written with `nvnTextureWriteTexels`](../reference/nvn-objects.md#textures-written-with-nvntexturewritetexels) +- after a few minutes of play the main thread spends whole frames in `WriteTexels` (Metroid Dread's frames stopped at about 175 s). → [`WriteTexels` regions written in place by block-linear address](../reference/nvn-objects.md#writetexels-regions-written-in-place-by-block-linear-address) +- black screen with every geometry right (`DM_NVN_HILITE` shows each sprite and glyph in place); the uniform buffer right (a `CBDUMP`), so every texture read 0. → [A pool's storage at creation is its textures' content](../reference/nvn-objects.md#a-pools-storage-at-creation-is-its-textures-content) +- a texture the game filled with `memcpy` and flushed reads as zeros (BotW's sky brownish grey, the distant haze brown). → [First-use fill of storage the game flushed](../reference/nvn-objects.md#first-use-fill-of-storage-the-game-flushed) +- a movie plays black (Age of Calamity: its intro movie under the "+ Skip" prompt). → [Mapped-range flushes taken as uploads (`tex_flush_upload`)](../reference/nvn-objects.md#mapped-range-flushes-taken-as-uploads-tex_flush_upload) +- magenta surfaces (Metal decodes an all-zero ASTC block as the error colour (1, 0, 1)); a zero BC block decodes black, which looks like missing light; black screen while the UI (or a dialog box) draws in the right places under `HILITE` or in the pass dump; `packaged textures: first uploads 0`. → [CPU writes with no flush, caught by page protection (`tex_dirty_upload`)](../reference/nvn-objects.md#cpu-writes-with-no-flush-caught-by-page-protection-tex_dirty_upload) +- a texture the game builds from CPU-filled textures read back with `nvnTextureReadTexels` comes out zero. → [`ReadTexels` from guest memory and textures written after first use](../reference/nvn-objects.md#readtexels-from-guest-memory-and-textures-written-after-first-use) +- textures that should hold CPU-written data come out magenta (ASTC's error colour) or black. → [Sampled textures in a sparse pool uploaded through the page table](../reference/nvn-objects.md#sampled-textures-in-a-sparse-pool-uploaded-through-the-page-table) +- some textures are re-uploaded every frame (several MB per frame). → [Dirty-page ranges of pools that share a host page](../reference/nvn-objects.md#dirty-page-ranges-of-pools-that-share-a-host-page) +- packaged textures re-uploaded at every use with unchanged content; upload staging and frame times rising. → [Writes beside a packaged texture in a shared host page](../reference/nvn-objects.md#writes-beside-a-packaged-texture-in-a-shared-host-page) +- about one run in four stalled early (around the opening movie or the title), with no log output, not even the time-limit report. → [Dirty-page tracker race and early stalls](../reference/nvn-objects.md#dirty-page-tracker-race-and-early-stalls) +- device / subsystem setup returns quietly and the next stage never starts. → [Version and capability checks that fail quietly](../reference/nvn-objects.md#version-and-capability-checks-that-fail-quietly) +- a null-pointer crash early in `nnMain`, right after NVN setup should have happened (Super Mario 3D World + Bowser's Fury, `main+0x759e08`: the device singleton `main+0x1a28ea0` is still null). → [The device's API version](../reference/nvn-objects.md#the-devices-api-version) +- null read at `main+0xb8462c` right after the first two textures, called from `main+0x670eb0`. → [Newest GPU code version reported by the driver](../reference/nvn-objects.md#newest-gpu-code-version-reported-by-the-driver) +- a jump to pc=0 right after the NVN bootstrap loader (`lr=main+0x37fe930`, `[RED] NVN address lookup of an unknown name: nvnSetExternalDebuggerEnabledNVX`). → [Extensions fetched by name and called unchecked](../reference/nvn-objects.md#extensions-fetched-by-name-and-called-unchecked) +- SIGBUS in `AGCDeserializedReply`, address in the guest stack region. → [NVN entry points on the big host stack](../reference/nvn-objects.md#nvn-entry-points-on-the-big-host-stack) +- red `NVN unimplemented function called` lines for `nvnColorStateGetBlendEnable`, `nvnBlendStateGetBlendFunc`, `nvnPolygonStateGetCullFace`, `nvnDepthStencilStateGetDepthTestEnable`, `nvnBufferBuilderGetSize`, `nvnTextureBuilderGetWidth`, …. → [State read back through NVN getters](../reference/nvn-objects.md#state-read-back-through-nvn-getters) +- an abort in device setup when `nvnTexturePoolGetSize` / `nvnSamplerPoolGetSize` returned 0. → [Pool size getters, texture address and command memory size](../reference/nvn-objects.md#pool-size-getters-texture-address-and-command-memory-size) +- red stub lines for NVN calls. → [NVN calls that were red stubs](../reference/nvn-objects.md#nvn-calls-that-were-red-stubs) +- `nvnWindowGetPresentInterval`, `nvnCommandBufferSetCopyImageStride` missing. → [SDK 8 window and copy-stride entry points](../reference/nvn-objects.md#sdk-8-window-and-copy-stride-entry-points) + +## Rendering + +- Metal aborts on a texture descriptor with an absurd size (`height (1869367126) greater than … 16384`). → [Texture objects copied into command records](../reference/rendering.md#texture-objects-copied-into-command-records) +- glyphs render as white boxes (the font atlas is `(1, 1, 1, R)`). → [Texture swizzle through swizzled Metal views](../reference/rendering.md#texture-swizzle-through-swizzled-metal-views) +- dark boxes and cream cloud rectangles behind HUD prompts, a black stamina wheel and a black System menu background (layout captures). → [`nvnTextureCompare` compares the channel mapping](../reference/rendering.md#nvntexturecompare-compares-the-channel-mapping) +- `newTextureViewWithPixelFormat` takes a large share of the replay thread (45% in Age of Calamity) when a game registers textures into a ring of IDs before every bind. → [Texture views cached by texture and view parameters](../reference/rendering.md#texture-views-cached-by-texture-and-view-parameters) +- daylight scenes washed out to white (every read of a texture returned one fixed value); `texture type mismatch` red lines. → [Compatible views for a mismatched declared texture type](../reference/rendering.md#compatible-views-for-a-mismatched-declared-texture-type) +- a type mismatch when a Metal 1D texture is bound to a shader's `texture2d` argument. → [1D textures as one-row 2D textures](../reference/rendering.md#1d-textures-as-one-row-2d-textures) +- shaders that sample a cube texture as `texturecube_array` get a type mismatch. → [Cube textures held as 6-layer 2D arrays](../reference/rendering.md#cube-textures-held-as-6-layer-2d-arrays) +- an abort in `MTLTextureDescriptor` validation, `arrayLength (510) … greater than the max allowed size (341)`, when the prologue movie ends. → [Cube map arrays count layer-faces](../reference/rendering.md#cube-map-arrays-count-layer-faces) +- an abort in Metal's validation: `newSliceRange.length(1) must be a multiple of (6)`. → [Cube views count their layers in cubes](../reference/rendering.md#cube-views-count-their-layers-in-cubes) +- `texture type mismatch … declares slot 10 as Metal type 5, texture … is type 3` (undefined reads). → [A 2D texture in a cube slot binds the zero cube](../reference/rendering.md#a-2d-texture-in-a-cube-slot-binds-the-zero-cube) +- translucent UI (frosted text bands, button guides, dialog headers) yellow and white. → [Integer views for images a shader declares uint or int](../reference/rendering.md#integer-views-for-images-a-shader-declares-uint-or-int) +- `texture type mismatch: shader declares 2, texture is 9`; the slot read 0. → [Buffer textures read by 1D integer fetches (row wrap)](../reference/rendering.md#buffer-textures-read-by-1d-integer-fetches-row-wrap) +- torn digits and a background in blue steps (Rhythm Heaven Groove's latency calibration). → [Automatic block height truncates](../reference/rendering.md#automatic-block-height-truncates) +- magenta (the ASTC error colour) on streamed textures in a sparse pool; `DM_NVN_ZBLOCK` finds such textures (all-zero blocks). → [Sparse textures laid out in 64 KB tiles](../reference/rendering.md#sparse-textures-laid-out-in-64-kb-tiles) +- streamed textures read 0 (magenta on Monster Hunter Rise's title screen's clothes). → [Sparse textures bound from their first mapped level](../reference/rendering.md#sparse-textures-bound-from-their-first-mapped-level) +- ≈13 MB/frame leak. → [Transient render targets shared by storage](../reference/rendering.md#transient-render-targets-shared-by-storage) +- GPU memory grows with every image shown; the memory line's `shared textures` count keeps rising. → [Shared-storage entries taken over by a new texture](../reference/rendering.md#shared-storage-entries-taken-over-by-a-new-texture) +- the whole picture came out darker and more saturated on the Mac and the iPhone: comparing with a Switch screenshot, every channel at every level equalled the Switch value decoded from sRGB once more (0.55 → 0.26, 0.75 → 0.52). → [Render-target views that change the format](../reference/rendering.md#render-target-views-that-change-the-format) +- a clear of, or a format-converting copy into, a texture made without render-target use is declined. → [Render-target use added on demand](../reference/rendering.md#render-target-use-added-on-demand) +- `render target is a compressed-format texture …`; the pass's draws did nothing (the attachment was left out), or the pipeline-descriptor validation aborts the process. → [Compressed textures as render targets through an uncompressed shadow](../reference/rendering.md#compressed-textures-as-render-targets-through-an-uncompressed-shadow) +- the boot aborts in Metal's pipeline validation: `MTLPixelFormatDepth16Unorm is not color renderable`. → [A depth texture drawn through a colour view](../reference/rendering.md#a-depth-texture-drawn-through-a-colour-view) +- a G-buffer target holds only scattered squares (a different order for each 32×32 tile). → [The same texture attached at two colour slots](../reference/rendering.md#the-same-texture-attached-at-two-colour-slots) +- `no render target could get an MTLTexture` (the pass skipped). → [Passes with no render target](../reference/rendering.md#passes-with-no-render-target) +- a black world: everything rendered through a 3D LUT built by layered draws comes out black (every plane went to plane 0); `DM_NVN_PASSDUMP` shows no `L` line. → [Layered passes for vertex shaders that write the layer](../reference/rendering.md#layered-passes-for-vertex-shaders-that-write-the-layer) +- `texture clear case not implemented … mask 0x1`. → [A channel mask that covers every channel is a whole clear](../reference/rendering.md#a-channel-mask-that-covers-every-channel-is-a-whole-clear) +- red lines for unimplemented mid-pass, partial-mask and partial-scissor clears (the clears were dropped); `depth-stencil clear with a scissor or partial stencil mask, cleared whole` (logged once). → [Clears inside a pass, with a partial mask or a partial scissor](../reference/rendering.md#clears-inside-a-pass-with-a-partial-mask-or-a-partial-scissor) +- solid magenta blocks, exactly (255, 0, 255), hard-edged and aligned to the 16-pixel grid (16×16, sometimes two stacked): Super Mario 3D World showed them only on the iPhone; Captain Toad on a Mac, Apple M6, in about half the frames of the opening course, moving with the falling leaves. → [Same-pass feedback read through a snapshot](../reference/rendering.md#same-pass-feedback-read-through-a-snapshot) +- text a game renders into textures for the title and the menus comes out upside down, while the scene looks right. → [Lower-left window origin](../reference/rendering.md#lower-left-window-origin) +- a pink window with every command buffer ignored while headless runs look right: the first presents fail on the display command buffer with `kIOGPUCommandBufferCallbackErrorPageFault`, and after a few of those macOS ignores every submission of the process ("Ignored (for causing prior/excessive GPU errors)"). → [The residency set committed before a present](../reference/rendering.md#the-residency-set-committed-before-a-present) +- undefined values blended into attachments the fragment shader leaves alone. → [No writes to colour outputs the shader does not write](../reference/rendering.md#no-writes-to-colour-outputs-the-shader-does-not-write) +- `nvnBlendStateSetAdvancedMode 8` (SRC_OVER) logged and blended with the state's own functions. → [Advanced blend modes as Porter-Duff factors](../reference/rendering.md#advanced-blend-modes-as-porter-duff-factors) +- striped lighting: black and white bands that follow depth on every lit surface, as if every pixel were in or out of shadow at random. → [Stencil-mode texture views read through `X32_Stencil8`](../reference/rendering.md#stencil-mode-texture-views-read-through-x32_stencil8) +- a character drawn as a dithered silhouette inside a dithered square (DRAGON QUEST VII Reimagined's hero). → [Stencil-mode textures from the texture builder](../reference/rendering.md#stencil-mode-textures-from-the-texture-builder) +- characters (skinned models) invisible or stretched into streaks while the level renders; `vertex attribute format … does not match shader input type …` for a remaining mismatch. → [Integer vertex inputs](../reference/rendering.md#integer-vertex-inputs) +- red lines 89 and 96 (`vconv_bind`, the CPU conversion every draw); about a quarter of Age of Calamity's replay thread in battle. → [Shadow vertex inputs](../reference/rendering.md#shadow-vertex-inputs) +- `Vertex attribute m_36(5) of type uint4 cannot be read using MTLAttributeFormatHalf2` (draws dropped: black silhouettes); UVs collapse onto one texel (flat tan characters); vertices far out of view (negative 16-bit values widened with zeros). → [CPU conversion of vertex attributes the input type cannot take](../reference/rendering.md#cpu-conversion-of-vertex-attributes-the-input-type-cannot-take) +- `vertex attribute format … not mapped` (printed once). → [Vertex formats Metal lacks](../reference/rendering.md#vertex-formats-metal-lacks) +- the assertion `buffer index (31) must be < 31`. → [Vertex streams 14 and 15 at Metal slots 14 and 15](../reference/rendering.md#vertex-streams-14-and-15-at-metal-slots-14-and-15) +- scenes and characters black, only foliage, light shafts and the HUD drawn; pass dumps full of `Z … draw skipped: no program bound`. → [Programs bound per stage](../reference/rendering.md#programs-bound-per-stage) +- `[RED] failed to create render pipeline (vertex 790573a52b9c5053_b fragment f39fb42850f50ecd_b): Fragment input(s) user(locn0),user(locn1) … not written by vertex shader`, once on entering the first investigation; `texture type mismatch: shader declares a cube, texture is 2D` (a pixel shader run with another pair's bindings). → [Composite program taken again after a replay state restore](../reference/rendering.md#composite-program-taken-again-after-a-replay-state-restore) +- `no program bound before the draw`, `program has no vertex shader` (a draw after a dispatch is dropped); `the dispatched program has no compute shader` (a dispatch after a graphics bind). → [Compute programs bound apart from graphics programs](../reference/rendering.md#compute-programs-bound-apart-from-graphics-programs) +- a red line: the 8-bit index buffer had GPU-side writes in the submission (indices read stale). → [8-bit indices widened on the GPU after a write in the same submission](../reference/rendering.md#8-bit-indices-widened-on-the-gpu-after-a-write-in-the-same-submission) +- indirect draws whose record the GPU writes earlier in the frame cannot be read on the CPU at replay; GPU-driven effects drawn as indirect quads, which Metal has no primitive for. → [Indirect draws and dispatches read by the GPU](../reference/rendering.md#indirect-draws-and-dispatches-read-by-the-gpu) +- an empty G-buffer pass (the title screen showed only the UI); later, the submitting thread waiting for the GPU before each such draw (36–54 fps in Kamura Village on an iPhone). → [Indirect-count multi-draws](../reference/rendering.md#indirect-count-multi-draws) +- characters collapsed into one thin spike (vertex streams read from feedback buffers that were never written); a pass dump shows a pass of draws with a vertex shader and no fragment shader (`fs=-`) before the pass that reads them. → [Transform feedback captured by a vertex entry point](../reference/rendering.md#transform-feedback-captured-by-a-vertex-entry-point) +- `SIGSEGV` in `-[AGXG18GFamilyComputeContext_mtlnext performEndEncoding]` (from `rp_begin_ex_`, or in its `dealloc` on the encoding thread); Metal validation (`MTL_DEBUG_LAYER=1 MTL_DEBUG_LAYER_ERROR_MODE=nslog`) names it: "Command buffer already has an open command encoder". → [Helper encoders end the open copy encoder first](../reference/rendering.md#helper-encoders-end-the-open-copy-encoder-first) +- the whole picture upside down (Absolum's title logo and "Press Any Button" mirrored top to bottom); a red-reporting stub for `nvnCommandBufferSetViewportSwizzles`. → [Viewport swizzles applied as a flipped viewport](../reference/rendering.md#viewport-swizzles-applied-as-a-flipped-viewport) +- tall blue / textured panels that flickered frame to frame: the fountain's vertex shader pushed the water surface to the top of the screen. → [Metal 4 does not order operations inside one encoder](../reference/rendering.md#metal-4-does-not-order-operations-inside-one-encoder) +- many render passes with one draw each and high submit time: Metroid Dread's cutscene frames had about 180 passes with one draw each (about 5 ms of submit time per frame on the Mac). → [Inline uniform updates inside a render pass](../reference/rendering.md#inline-uniform-updates-inside-a-render-pass) +- hundreds of render passes a frame and heavy attachment traffic when a game updates a small part of a large uniform block before each draw group. → [Partial-cover uniform shadows keep passes open](../reference/rendering.md#partial-cover-uniform-shadows-keep-passes-open) +- `stage 0 cb5 has no uniform buffer bound`. → [An unbound uniform slot reads the zero bank](../reference/rendering.md#an-unbound-uniform-slot-reads-the-zero-bank) +- GPU time per frame doubles with many small dispatches (Age of Calamity: GPU 9 → 18 ms a frame on the Mac; the iPhone's 15 → 51 ms, 17 fps). → [Compute dispatches share the copy encoder](../reference/rendering.md#compute-dispatches-share-the-copy-encoder) +- dialog boxes with holes the size of their text, and white specks where text faded. → [Fragment-ordering barriers between draws that write images](../reference/rendering.md#fragment-ordering-barriers-between-draws-that-write-images) +- a texture copied from a staging buffer shows another texture's blocks (Catherine: Full Body's cutscene subtitle band a hard-edged dark box with a magenta stroke through it). → [A segment committed after a buffer → texture copy](../reference/rendering.md#a-segment-committed-after-a-buffer--texture-copy) +- a scaled texture copy skipped with a red line (Metal blits cannot scale); format-changing, depth-stencil and mirrored copies. → [Copies a blit cannot do, drawn](../reference/rendering.md#copies-a-blit-cannot-do-drawn) +- no ambient occlusion (Mac); on a device the result depends on whatever that memory holds. → [Depth-stencil copies carry the stencil](../reference/rendering.md#depth-stencil-copies-carry-the-stencil) +- the 3D scene black (targets cleared again before they are read); `[RED] SetRenderEnableConditional: the host draws anyway`; with a game that skips clears under render conditions, scenes black behind the HUD (Mario vs. Donkey Kong's levels, 1.0.1). → [Conditional rendering evaluated at replay](../reference/rendering.md#conditional-rendering-evaluated-at-replay) +- no SPIR-V file in the RomFS; SPIRV-Cross refuses a module with "Struct member does not have ArrayStride set". → [Vulkan shaders generated at run time](../reference/rendering.md#vulkan-shaders-generated-at-run-time) +- a crash in `bind_res` reading a freed view's texture (Super Mario Galaxy 2 on an iPhone, element 1 of the 8-texture array, when the ship took off after the first galaxy; GitHub issue 68). → [Descriptors hold the Metal objects they were written with](../reference/rendering.md#descriptors-hold-the-metal-objects-they-were-written-with) +- Metal refuses the pipeline: "Fragment shader does not write to render target color(0), index(1)". → [Dual-source blending from output location 1](../reference/rendering.md#dual-source-blending-from-output-location-1) +- Metal's validation aborts on a buffer layout no mapped attribute uses. → [Scaled and three-component vertex formats](../reference/rendering.md#scaled-and-three-component-vertex-formats) +- menus 1.5x too big in handheld mode (the file-creation dialog cut off at the right edge). → [Viewport and scissor persist across render passes](../reference/rendering.md#viewport-and-scissor-persist-across-render-passes) +- `draw outside a render pass`; the rest of the pass was dropped. → [Copies inside a render pass](../reference/rendering.md#copies-inside-a-render-pass) +- an assert (a stripped trap, `main+0x4d26e0`) when the surface does not list `preTransform` INHERIT. → [Swapchain pre-transform and layer crop](../reference/rendering.md#swapchain-pre-transform-and-layer-crop) +- GX points and lines drawn as one-pixel points and lines; a red line for the sampler LOD bias. → [Not translated in the Vulkan layer: geometry shaders and LOD bias](../reference/rendering.md#not-translated-in-the-vulkan-layer-geometry-shaders-and-lod-bias) +- `UNIMPL eglGetDisplay`, then `Break` right after `nv::InitializeGraphics` and `vi::OpenDefaultDisplay` (`main+0x5ed9d0`). → [Games that render with desktop OpenGL through EGL](../reference/rendering.md#games-that-render-with-desktop-opengl-through-egl) +- the NSP script's provisional profile said Vulkan, and the capture boot crashed in the GL layer. → [Telling a GL game from an NVN or Vulkan one](../reference/rendering.md#telling-a-gl-game-from-an-nvn-or-vulkan-one) +- a jump to address 0 right after `glGetFloatv(0xb12)` (`lr=main+0x2d58c4`). → [Which GL functions a game calls](../reference/rendering.md#which-gl-functions-a-game-calls) +- in a capture run every program links with no uniforms; the game's first `Shader:send` raises a Lua error and, on the Switch, its error handler quits (Any Lua error quits a LÖVE game). → [GL shaders put together at run time and captured](../reference/rendering.md#gl-shaders-put-together-at-run-time-and-captured) + +## Shaders + +- `shader_split.py` finds 0 shaders in a RomFS of zstd files named `.zst`. → [Zstd files recognised by their magic](../reference/shaders.md#zstd-files-recognised-by-their-magic) +- a provisional split finds only a fraction of the shaders (3,126 of Pokémon Sword's 41,516). → [BNSH entries of GFPAK archives](../reference/shaders.md#bnsh-entries-of-gfpak-archives) +- the split found no shader in the game (0 shaders). → [GLSLC outputs holding several stages](../reference/shaders.md#glslc-outputs-holding-several-stages) +- 24 shaders before the control-length fix, 1,355 after. → [SPKG packages in KTGL LINKDATA](../reference/shaders.md#spkg-packages-in-ktgl-linkdata) +- shaders the game draws are missing from the map (`DM_NVN_SHDUMP`), in no RomFS file and no module, stored or LZ4, and not compiled at run time. → [Shaders in a zlib stream in main (YEBIS .tfxo)](../reference/shaders.md#shaders-in-a-zlib-stream-in-main-yebis-tfxo) +- two `[RED] shader … is not in the shader map` lines and `program has no vertex shader` as the opening movie starts; every movie drew white. → [Loose control sections and code areas in main (Bink 2)](../reference/shaders.md#loose-control-sections-and-code-areas-in-main-bink-2) +- the split stopped with `no bundle has a Shader type tree of the same hash to borrow`. → [Unity Shader layouts written out for players without bundles](../reference/shaders.md#unity-shader-layouts-written-out-for-players-without-bundles) +- the 3D battle scene was black behind a correct UI, with `[RED] shader 854a8afc9d07f88d is not in the shader map` and `the dispatched program has no compute shader`. → [Unity ComputeShader objects](../reference/shaders.md#unity-computeshader-objects) +- shaders reported missing at random, and their dumped control sections are garbage (Unity). → [Program keys computed inside `nvnProgramSetShaders`](../reference/shaders.md#program-keys-computed-inside-nvnprogramsetshaders) +- 12 compute shaders (particles, post effects) translated into an endless loop that hung the GPU at the first gameplay scene; SPIRV-Cross failed on 7 shaders instead of 2 (SPIRV-Cross parse failures). → [Exact program start for the decoder](../reference/shaders.md#exact-program-start-for-the-decoder) +- GLSLC outputs fail the reader's check and are dropped (44 of DARK SOULS: REMASTERED's `main`'s 72 outputs and most of its archives' pixel shaders). → [GLSLC outputs read through their section headers](../reference/shaders.md#glslc-outputs-read-through-their-section-headers) +- `shader_split.py` finds no stage for vertex shaders and stops at the first captured output (`no GLSLC output`). Translated, every post pass and the window draw one texel's colour (a black screen with sound); every varying at (0, 0, 0, 1). → [VertexA and VertexB parts merged into one vertex shader](../reference/shaders.md#vertexa-and-vertexb-parts-merged-into-one-vertex-shader) +- SPIRV-Cross can't parse the SPIR-V (`nullptr` exception). → [SPIRV-Cross parse failures](../reference/shaders.md#spirv-cross-parse-failures) +- `Fragment input(s) user(locnN) mismatching vertex shader output type(s) or not written by vertex shader` and the draw is skipped; the host reports `[RED] failed to create render pipeline (vertex … fragment …)`. → [Pixel shader inputs the vertex shader does not write](../reference/shaders.md#pixel-shader-inputs-the-vertex-shader-does-not-write) +- outdoors dim and nothing casts a sun shadow (every shadow-map lookup fails). → [Unwritten components of a written output start as (0, 0, 0, 1)](../reference/shaders.md#unwritten-components-of-a-written-output-start-as-0-0-0-1) +- the ground kept stale 32×32 blocks where the G-buffer draw failed the depth test: blocks of a stale image where a depth pre-pass and a later pass with other vertex shaders meet (Odyssey: 32×32 blocks on every fourth frame). → [Position invariance across shaders](../reference/shaders.md#position-invariance-across-shaders) +- the half of Mario facing the camera's left was white (GitHub issue 72, on an iPhone). → [min, max and clamp with a NaN operand](../reference/shaders.md#min-max-and-clamp-with-a-nan-operand) +- a pipeline red line for a float output meeting an integer attachment (the red line prints the row to add); a BC encoder's pipeline fails. → [Integer fragment outputs declared by hand](../reference/shaders.md#integer-fragment-outputs-declared-by-hand) +- `failed to create render pipeline … output of type float4 is not compatible with a MTLPixelFormatR32Uint color attachment` (or `MTLPixelFormatRGBA16Uint`); every draw into the target skipped. → [Integer entry points for integer colour outputs](../reference/shaders.md#integer-entry-points-for-integer-colour-outputs) +- `failed to create render pipeline … Cannot convert attribute …`; draws missing (a pipeline that fails to build). → [Integer-typed vertex inputs fed normalized formats](../reference/shaders.md#integer-typed-vertex-inputs-fed-normalized-formats) +- outdoors the whole screen blurred (depth of field out of focus everywhere). → [Depth mode near = 0 needs the per-mode entry points](../reference/shaders.md#depth-mode-near--0-needs-the-per-mode-entry-points) +- `SPIRV-Cross threw an exception: Subgroup builtins are not available in this type of function`. → [Subgroup operations rewritten for a one-lane subgroup](../reference/shaders.md#subgroup-operations-rewritten-for-a-one-lane-subgroup) +- the tone mapper read an empty LUT and the screen came out a flat blue. → [Geometry shaders that only pick the layer](../reference/shaders.md#geometry-shaders-that-only-pick-the-layer) +- a geometry shader fails to compile with `unknown main0_out`: SPIRV-Cross writes `unknown main0_out main0()` for the geometry stage, and the shader does not build. → [Geometry shaders with no Metal stage](../reference/shaders.md#geometry-shaders-with-no-metal-stage) +- `primitive type 14 not implemented`; patch draws skipped. → [Tessellation stages are not drawn](../reference/shaders.md#tessellation-stages-are-not-drawn) +- a vertex or compute shader reads zeros from NVN's driver segment `c0`. → [Driver-segment reads outside fragment shaders](../reference/shaders.md#driver-segment-reads-outside-fragment-shaders) +- flickering hard-edged white tiles (tiled lighting with every tile row getting row 0's or other rows' lights). → [Compute `c0`: dispatch size and uniform buffer table](../reference/shaders.md#compute-c0-dispatch-size-and-uniform-buffer-table) +- point sprites of size 0 (Odyssey's Cap Kingdom fog never parted around Mario). → [Graphics `c0 + 0x610`: the largest point size](../reference/shaders.md#graphics-c0--0x610-the-largest-point-size) +- occlusion queries that always return 0 samples (Z-A: characters' legs folded back from the hips once they walked); every instanced draw's transform taken for instance 0. → [Graphics `c0 + 0x644`: the draw's base instance](../reference/shaders.md#graphics-c0--0x644-the-draws-base-instance) +- a menu's labels drawn as plain bars, other menus empty except for the cursor (one pool's textures overwritten by another's); a black screen when IDs past a fixed table read as empty slots. → [Texture and sampler pools with overlapping IDs](../reference/shaders.md#texture-and-sampler-pools-with-overlapping-ids) +- red lines from the list of command buffers that bound a non-first pool (6,734 in one DARK SOULS: REMASTERED run). → [Bases of bound handles kept in the command buffer](../reference/shaders.md#bases-of-bound-handles-kept-in-the-command-buffer) +- distant walls and floors smeared sideways (DARK SOULS: REMASTERED): tiling textures took a clamp-to-edge sampler and repeated their edge column. → [Handles from uniform buffers take the bound pools' bases](../reference/shaders.md#handles-from-uniform-buffers-take-the-bound-pools-bases) +- the 3D scene black, only particles and the HUD drawn; pass dump `B` lines all `tid=0 dummy`. → [Deferred texture binds read at execution](../reference/shaders.md#deferred-texture-binds-read-at-execution) +- bindless handles read from a uniform buffer name the wrong texture when the game writes them earlier in the same submission. → [Handles in uniform buffers written earlier in the same submission](../reference/shaders.md#handles-in-uniform-buffers-written-earlier-in-the-same-submission) +- a storage buffer reported with `cb_slot` 3, which had no NVN slot. → [Storage buffer addresses in the game's own uniform buffer](../reference/shaders.md#storage-buffer-addresses-in-the-games-own-uniform-buffer) +- `no resolved slot number`, the shader does not load; the translation has a 32-entry texture-pool array (`fp_t2d_cb0_40000000`, handle type Direct). → [Bindless handles through a loop phi](../reference/shaders.md#bindless-handles-through-a-loop-phi) +- the shaders' bindless arrays have no slot and used not to load; the translator falls back to a 32-entry array such as `array<texturecube<float>, 32> fp_tcube_cb0_40000000`. → [Texture slots picked at run time](../reference/shaders.md#texture-slots-picked-at-run-time) +- Ryujinx's fallback arrays (`fp_t2d_cb0_40000000`) in pixel shaders whose handle comes from the game's own uniform buffers; the shaders do not load and their draws are skipped. → [A texture chosen at run time between two uniform-buffer handles](../reference/shaders.md#a-texture-chosen-at-run-time-between-two-uniform-buffer-handles) +- the host stops at the shader's first load: `unexpected argument buffer layout: encodedLength=640 maxAid=48`. → [An array of handles in a uniform buffer](../reference/shaders.md#an-array-of-handles-in-a-uniform-buffer) +- `shaders with a binding whose slot was not decoded 588`; shadow cascades read through `array<texture2d_array<float>, 32> fp_t2das_cb2_0`. → [An array of handles from the driver table c2](../reference/shaders.md#an-array-of-handles-from-the-driver-table-c2) +- characters and props glowing white, about 10⁴ too bright, in DRAGON QUEST VII Reimagined's opening cutscene only, while walls and gameplay look right. → [A texture slot that shares its c2 word with an image slot](../reference/shaders.md#a-texture-slot-that-shares-its-c2-word-with-an-image-slot) +- the replay put a texture resource ID where the shader wanted an address; with the old 64 KB dummy buffer, pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced and the game or the replay crashed, or `kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers (GitHub issue #101). → [Image atomics](../reference/shaders.md#image-atomics) +- shaders missing from the map that no game file holds. → [Built-in shaders compiled with the game's own GLSLC](../reference/shaders.md#built-in-shaders-compiled-with-the-games-own-glslc) +- a null read at `main+0x3316dc` in `nnMain`, right after the window setup (the program object's interface pointers at +0xd0 / +0xd8 both null). → [GLSLC GPU code version against the driver's range](../reference/shaders.md#glslc-gpu-code-version-against-the-drivers-range) +- a vertex shader whose pairs changed was restored unpadded from `cas/`. → [Shader cache key covering the pairs](../reference/shaders.md#shader-cache-key-covering-the-pairs) +- `fragmentFunction main0 cannot be used to build a pipeline state` (a Metal validation assertion). → [Function constants specialized at load](../reference/shaders.md#function-constants-specialized-at-load) +- a pipeline for a vertex shader that writes `render_target_array_index` cannot be built. → [Layer-writing vertex shaders get the triangle class](../reference/shaders.md#layer-writing-vertex-shaders-get-the-triangle-class) +- the first boot sat on the loading clock for about a minute, the CPU idle, with every file read done after 12 s. → [A cold Metal cache looks like a long load](../reference/shaders.md#a-cold-metal-cache-looks-like-a-long-load) + +## Game state + +- "stuck" at a screen with one draw per frame. → [Most draws are in command buffers replayed later](../reference/game-state.md#most-draws-are-in-command-buffers-replayed-later) +- menus ignore A. → [Buttons mapped by position: B confirms](../reference/game-state.md#buttons-mapped-by-position-b-confirms) +- a crash in `main+0x339c00` a second or two into the title, in love-nx's video module teardown (`love::video::theora::Video` → `~Worker`; its thread released the worker that the closing module deletes too). → [Any Lua error quits a LÖVE game](../reference/game-state.md#any-lua-error-quits-a-löve-game) +- the TV latency calibration ("press A when you hear the fourth beat") ignores A. → [TV latency calibration's late, beat-timed listening window](../reference/game-state.md#tv-latency-calibrations-late-beat-timed-listening-window) +- in a scripted run, a screen goes back by itself right after a confirm; the log shows `swkbd: … → no window, treated as cancel`. → [The software keyboard in scripted runs (`DM_KBD`)](../reference/game-state.md#the-software-keyboard-in-scripted-runs-dm_kbd) +- a player join screen that ignores A (a character select with four "L + R to join" slots after the title). → [Player join screens (`hid::StartLrAssignmentMode`)](../reference/game-state.md#player-join-screens-hidstartlrassignmentmode) diff --git a/docs/reference/README.md b/docs/reference/README.md new file mode 100644 index 0000000..c1065f8 --- /dev/null +++ b/docs/reference/README.md @@ -0,0 +1,19 @@ +# Subsystem references + +How the host meets each part of a Switch game, and every failure met so far, one entry per mechanism: symptom, cause, fix (file and function), the games it was found on (linked to their notes in `dm/games/<id>/README.md`). Start from the [symptom index](../agents/symptoms.md) when you have a log line or a visible result; the workflows that lead here are in [docs/agents](../agents/README.md). + +| File | Covers | +|---|---| +| [rewriter.md](rewriter.md) | The NSO rewriter `patch_nso.py`: jump tables, exclusive windows, patch area reach, cuts and observation gates, functions without unwind entries, stack and system registers, relocations | +| [memory.md](memory.md) | Guest layout, heap and application pool, virtual address memory and the alias region (the SDK's node area), engine allocators, mapping SVCs, running out of memory | +| [threads.md](threads.md) | Thread creation and cores, address arbitration, yields and spin waits, events | +| [services.md](services.md) | SDK calls and system services: sm connections, names that changed between SDKs, file system and saves, add-on content, input, keyboard and applets, display and operation mode, accounts and network, settings and fonts, C library and time, modules | +| [movies.md](movies.md) | The SDK's movie library: decoder lifecycle, output slots, native-texture output, demuxers and codecs, game movie players | +| [audio.md](audio.md) | The audio renderer and its command lists by SDK, AudioOut and devices, codecs, timing | +| [frame-rate.md](frame-rate.md) | The host's frame step, engine frame limiters, present intervals and display rates, CPU cost | +| [nvn-objects.md](nvn-objects.md) | NVN object sizes, memory pools, command memory, queues and syncs, CPU-written textures, NVN entry points | +| [rendering.md](rendering.md) | The replay on Metal: textures, passes and clears, blend / depth / stencil, vertex input, uniforms and ordering, conditional rendering; the Vulkan and OpenGL layers | +| [shaders.md](shaders.md) | Shader containers by engine, control sections and code, translation, bindings and handles, shaders compiled at run time, pipelines and caches | +| [game-state.md](game-state.md) | Saves, scripted state, and game behaviour that looks like a host bug but is not | + +A new general finding goes into the file and section it belongs to, in the same form, with a line in the symptom index; a fact about one game goes into that game's note. diff --git a/docs/reference/audio.md b/docs/reference/audio.md new file mode 100644 index 0000000..3234f7a --- /dev/null +++ b/docs/reference/audio.md @@ -0,0 +1,205 @@ +# Audio + +The audio renderer (`dm/host/hle_audren.c`), AudioOut, audio devices and codecs. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Audio renderer + +### Running the SDK's own renderer on the CPU + +Check whether the SDK carries the renderer: list the `sdk` module's exports for `nn::audio::server::` and `nn::audio::dsp::CommandListProcessor`. If they are there ([Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md), SDK 16), do not write a mixer: + +- switch `AudioRendererParameter` to rendering device = CPU (+0x2c = 1) and execution mode = manual (+0x30 = 1) in `GetAudioRendererWorkBufferSize` and both `OpenAudioRenderer` overloads; +- call the SDK originals (`guest_sym` returns the module's own code even for a forced symbol); +- drive `ExecuteAudioRendererRendering` from a guest thread (`dm_guest_thread_spawn`). + +The open then goes through `CreateAudioRendererManagerByDfc`; no IPC. + +### SDK 8's longer `AudioRendererParameter` + +- **Symptom:** abort with Result 0x499 in `audio::GetAudioRendererWorkBufferSize`. +- **Cause:** SDK 8's `AudioRendererParameter` is one word longer than SDK 16's: its revision sits at +0x38. The host copies the game's parameter before switching it to CPU rendering with manual execution, and copied only 0x38 bytes, so `CheckValidRevision` read garbage. +- **Fix:** it copies 0x40 now. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0). + +### A renderer opened without an event + +- **Symptom:** the game is silent; only the game's updates render (catch-up frames that nothing plays). The log has `audren: OpenAudioRenderer (no event)` and no `render thread started`. +- **Cause:** the game opens the renderer with `OpenAudioRenderer(handle, parameter, work, size)`, the form with no system event, and paces its own updates, but the renderer must still run every 5 ms. The host used to start its render thread only for the form with an event. +- **Fix:** both forms go through one open that starts the thread (`hle_audren.c` `ar_open`). +- **Found on:** [Balatro](../../dm/games/0100CD801CE5E000/README.md) (SDK 18.3.2, LÖVE's OpenAL Soft; one boot). + +### The SDK's open with an event calls the open without one (`t_in_open`) + +- **Symptom:** `[RED] audren: a second renderer` right after the first open, with the same work buffer and "original device 1 mode 1". +- **Cause:** not a second renderer: SDK 17.5.3's `OpenAudioRenderer` with an event opens it by calling the form without one through the PLT (`sdk+0x1e489c`), which the loader binds to the host's hook since that form has one ([A renderer opened without an event](#a-renderer-opened-without-an-event)), so `ar_open` ran twice for one renderer. The inner call started the render thread before the outer one had created the event, and the outer call took the renderer for a second one. +- **Fix:** a thread-local flag (`t_in_open`) hands the inner call straight to the SDK with the host's CPU / manual copy of the parameter, and only the outer call starts the thread (`hle_audren.c`). +- **Found on:** [ROMANCE OF THE THREE KINGDOMS 8 REMAKE](../../dm/games/0100C2A01A730000/README.md) (SDK 17.5.3). + +### The renderer's update lock (`update_lock`) + +- **Symptom:** none of its own; command generation races the game's `Update` on SDKs whose lock is not at SDK 16's offset. +- **Cause:** `AudioRenderSystem::Update` holds a mutex in `AudioRenderSystem`; `GenerateCommand` does not take it, so the host does (the auto-mode system thread does the same in `SendCommandToDsp`). Which mutex depends on the SDK: `nn::os::LockMutex(MutexType *)` at +0x1d0 in SDK 4.4 and at +0x1d8 in 6.4–8.2, `SdkMutexType::Lock` at +0x1d8 in 10.4–11.4, +0x1f0 in 13.3–16.2 and +0x1f8 in 17.5 (the `sdk` builds of 24 games on hand), inlined in 19 and 20 (+0x2d0, the Span layout: [SDK 20's `AudioRenderSystem` layout](#sdk-20s-audiorendersystem-layout)). The host took SDK 16's for every SDK before 20, which wrote a lock word into another field and excluded nothing. +- **Fix:** `update_lock` in `hle_audren.c` reads it off the SDK's own `Update`: the register the prologue adds the offset to and moves into x0 before its first `bl`, whose PLT slot must hold `LockMutex` or `SdkMutexType::Lock`; anything else keeps SDK 16's. +- **Found on:** [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md) (4.4), the first game it was plainly wrong for. +- **Check:** checked with sound on [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (11.4, +0x1d8), [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (7.3, `MutexType` +0x1d8) and [Super Mario Odyssey](../../dm/games/0100000000010000/README.md) (7.3, which is silent for its first minute with either lock). + +### The render thread's route through the public `ExecuteAudioRendererRendering` (SDK 20) + +- **Symptom:** `CommandListProcessor::Process` crashes now and then (SIGSEGV under `render_main`, once at 60 s and once at 134 s in the shrine); every `RequestUpdate` renders a catch-up frame. +- **Cause:** the public `ExecuteAudioRendererRendering` (`sdk+0x221e90`) reaches `AudioRendererImpl::ExecuteAudioRendererRendering` (`sdk+0x22ca88`, `x0 += 0x60` and a jump through the PLT) through the vtable, so the host's takeover of the `Impl` function was never called. The SDK's own `AudioRenderSystem::ExecuteAudioRendererRendering` ran instead (no update lock), every `RequestUpdate` rendered a catch-up frame because the host's render counter never moved, and the two threads wrote the same command buffer. +- **Fix:** the host also takes over `AudioRenderSystem::ExecuteAudioRendererRendering`, the PLT target (`hle_audren.c`); catch-ups fell from one per update to 3 per run. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (SDK 20). +- **Note:** the same SDK's other `AudioRenderSystem` layout is in [SDK 20's `AudioRenderSystem` layout](#sdk-20s-audiorendersystem-layout). + +### A renderer below 48 kHz with no upsampler on the CPU + +- **Symptom:** SIGSEGV at address 0x18 in `UpsamplerManager::Allocate` on the render thread. +- **Cause:** a DeviceSink below 48 kHz is upsampled by the SDK's upsampler, and `AudioRenderSystem::SetUpRenderResourceSet` allocates the `UpsamplerManager` only for the DSP (`sdk+0x21eb6c`: rate 0x7d00 and rendering device +0x48 == 0); with the host's CPU rendering the manager pointer (`AudioRenderSystem+0x150`, the render context's +0x20) is null. +- **Fix:** the host answers `Allocate` on a null manager with no upsampler, so the generator skips the upsample command (`sdk+0x225b40`), and resamples the DeviceSink's stereo downmix 3:2 itself (a 16-tap windowed sinc in three phases, `up_3_2` in `hle_audren.c`), 240 samples per 5 ms frame for the 48 kHz output. +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md): rate 32000, 160 samples per frame, `GenerateSinkCommands` `sdk+0x225a80`. +- **Note:** until then the output code assumed an upsampled buffer at 32 kHz that no game had produced. + +### A sound engine's heap too small for its renderer and players + +- **Symptom:** the main thread spins at start-up in a resource wait (a `YieldThread` loop, no crash), right after `OpenAudioRenderer`. +- **Cause:** Koei Tecmo's sound engine (KTSL) failed its setup quietly. The init hands it three heaps cut from one block (15 MB engine memory, 2 MB, 142 MB sound data; the sizes are constants at `main+0x8d9610`). In the 15 MB heap the setup allocates the renderer config and work buffer (0x760000: `AudioRenderSystem::GetWorkBufferSize`, 0x40a9c8 of it command buffer) and one block per player (up to 0x1b5860; three of the four mix 5.1). The fourth block came back null, the setup released its memory pools (`RequestAttachMemoryPool` then `Detach` with no update between) and returned −51, the first sound archive (ASRS) never loaded, and the resource the main thread waited for depended on it. +- **Fix:** a per-build cut moves 4 MB from the sound data heap to the engine heap (its `patches.py`). +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md). +- **Note:** how it was found: probe the pump the wait calls (`main+0x8e1c0`: first missing stage bit → the type's stage handler), the handler's dependency list (an incomplete entry of type `0xbbd39f2d`, ASRS), then the memory pool states (`MemoryPoolInfo` +0x10: 1 new, 3 detached, 4 request attach, 5 attached, 6 unused) and the engine allocator's arguments and results. Why hardware fits in 15 MB is not known. + +## Command lists by SDK + +### The final mix is taken from the DeviceSink's mix buffers + +- **Symptom:** the CPU processor drops the final mix. +- **Cause:** `dsp::SendCommandBuffer`, `dsp::Wait` and `dsp::Signal` are stubs in the application build, and DeviceSink (command 0x13) is a no-op in `CommandListProcessor::Process`. +- **Fix:** take the output after `Process` from the mix buffers the DeviceSink command names (layout in `hle_audren.c`). + +### SDK 4: DeviceSink is command 0xf of 0x130 bytes + +- **Symptom:** no sound: `[RED] audren: no DeviceSink (ID 0x10) in the command list, 0 commands of 0x148 bytes`. +- **Cause:** SDK 4's command list has SDK 6's header, but its DeviceSink (`CommandBuffer::GenerateDeviceSinkCommand`, `sdk+0x19e690`) is ID 0xf and 0x130 bytes: device name +0x14, upsample buffer +0x118, input count +0x120, input indices as bytes from +0x124. +- **Fix:** `capture` takes the one 0x130-byte command with 1 to 6 inputs when there is no 0x148-byte one (`g_sink4` in `hle_audren.c`). +- **Found on:** [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md) (4.0.0, SDK 4.4.0; one boot). +- **Note:** its `AudioRenderSystem::Update` (`sdk+0x19b77c`) locks an `nn::os::MutexType` at +0x1d0, not SDK 16's `SdkMutexType` at +0x1f0; see [The renderer's update lock (`update_lock`)](#the-renderers-update-lock-update_lock). + +### SDK 6: the DSP command header layout + +- **Symptom:** no sound; red `audren: command 1 in the command list is malformed (magic 0 size 0)`. +- **Cause:** SDK 6's DSP command header is +0x04 size (s16), +0x06 command ID (s16), +0x10 enabled, where SDK 8–20 have +0x04 enabled, +0x05 ID, +0x06 size (`CommandListProcessor::Process`, `sdk+0xaaa00`); its DeviceSink is ID 0x10. +- **Fix:** `capture` in `hle_audren.c` takes the layout whose walk finds the 0xcafebabe magic on every command of the first full list. +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md) (SDK 6.5.0). + +### SDK 8: command IDs numbered differently + +- **Symptom:** no music and no sound effects, only the movies' sound (the log repeats `audren: first-frame commands …` every frame and never prints `DeviceSink inputs`). +- **Cause:** SDK 8 numbers the DSP commands differently: its command list goes 4×6, 7×24, 11×2, 16×1, 21×1 where SDK 20's goes 7×6, 11×24, 15×2, 20×1, 25×1, and the output command (DeviceSink, the same 0x148-byte layout with device name `MainAudioOut`, six inputs) is 0x10, not SDK 16's 0x13. The host never found it and played silence. +- **Fix:** when the expected ID is missing from the first full command list, `capture` in `hle_audren.c` now takes the one 0x148-byte command with 1 to 6 inputs as the sink and logs the ID it adopted. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0). + +### SDK 20's `AudioRenderSystem` layout + +- **Cause:** in SDK 20 the getters the host used (`GetExecutionMode`, `GetRenderingDevice`, `GetCommandBufferWorkBufferSize`) are inlined away, `GenerateCommand` takes and returns a `Span<std::byte>` (the generated list comes back in x0 / x1), the device and mode are words at +0x48 / +0x4c, the command buffer is at +0xf8 / +0x100, and `UpdateImpl` locks the mutex at +0x2d0 instead of +0x1f0. The DeviceSink command became ID 0x14 (SDK 20 inserted a command before it) with the same layout. +- **Fix:** `render_once` picks the layout by whether the Span version of `GenerateCommand` exists (`hle_audren.c`). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md). +- **Note:** the render thread's call takes another route in this SDK too: see [The render thread's route through the public `ExecuteAudioRendererRendering` (SDK 20)](#the-render-threads-route-through-the-public-executeaudiorendererrendering-sdk-20); the update lock at +0x2d0 is in [The renderer's update lock (`update_lock`)](#the-renderers-update-lock-update_lock). + +### SDK 21 DSP command header without a magic + +- **Symptom:** no sound; red `audren: command 0 in the command list is malformed (first word 0x181901 size 0)`. +- **Cause:** SDK 21's DSP command header has no magic: +0x00 enabled (bit 0), +0x01 command ID, +0x02 size (s16), +0x04 estimated time, +0x08 node ID (`CommandListProcessor::Process`, `sdk+0x24fd50`), and every command body moved 8 bytes forward. DeviceSink is still ID 0x14 but 0x140 bytes: input count +0x128, input indices +0x12c, upsample buffer +0x120 (written at `sdk+0x23a7b0`). `AudioRenderSystem` keeps SDK 20's offsets (update lock +0x2d0). +- **Fix:** `capture` (`hle_audren.c`) tries this layout when neither magic walk works, bounded by the generated list's length. Its first lists hold 4 commands or fewer, so the layout is now chosen on the first non-empty list. +- **Found on:** [Rhythm Heaven Groove](../../dm/games/0100D9F01D474000/README.md) (SDK 21.4.0, one boot). + +## AudioOut and devices + +### RemoteIO's default I/O buffer duration + +- **Symptom:** crackling on iOS only. +- **Cause:** RemoteIO asks for as many frames per callback as the audio session's I/O buffer duration, about 21 ms by default (1024 frames at 48 kHz), while the renderer keeps 4 blocks × 5 ms = 20 ms in flight. Every callback came up 1.3 ms short and was padded with zeros, about 47 gaps per second. Reproduced on the Mac by forcing 1024-frame callbacks (`DM_DBG_AU_FRAMES=1024` with `DM_AUDIO=1`: 1999 underruns in 2000 callbacks). +- **Fix:** `AVAudioSession` `setPreferredIOBufferDuration:0.005` (and 48 kHz) before the game opens audio and again on return to the foreground (`front_ios_audio_session` in `front_ios.m`); the device then calls back with 240 frames and nothing underruns. +- **Check:** when checking a new device, read the host's `audio #N device callbacks` line, logged every 2000 callbacks: frames per callback, queued blocks, underruns. + +### Interruptions that stop RemoteIO callbacks + +- **Symptom:** sound stays silent after Siri, an alarm, a call or another app's audio; a game whose audio thread waits for its buffers back (Wwise's poll-only output, FMOD) then waits forever. +- **Cause:** iOS stops calling the RemoteIO unit back without stopping it, so the host still thinks the output plays. +- **Fix:** three layers handle it: + - `front_ios.m` observes `AVAudioSessionInterruptionNotification` (began → the silent sink stands in, ended → session up again and resume; becoming active also ends an interruption whose "ended" never came); + - `front_ios.m` also observes `AVAudioSessionMediaServicesWereResetNotification` (`dm_audio_reset` recreates every AudioUnit); + - a watchdog in `hle_audio.c` switches any started output whose device has not called back for 200 ms in focus to the silent sink, retrying the AudioUnit after 1, 2, 4, 8 s. A start that fails (session still held by a call) falls back the same way instead of aborting. +- **Check:** the log shows `audio session: interrupted`, `the device has not called back for N ms` and `AudioUnit restarted`. `tests/t_audio.c` (in `dm/host`) scenario 3 stops a playing AudioUnit behind the host's back. + +### Audio across a background round trip on the Mac + +`DM_AUDIO=1 DM_AUDIO_MUTE=1` takes the real Core Audio path with the output silenced, and `DM_OE_SCRIPT="bg=1@60,bg=0@66"` runs the same sequence as the iOS lifecycle callbacks (focus, audio take-over, guest clock freeze, graphics suspend). Send `dmhost` `SIGSTOP` / `SIGCONT` between the two to imitate iOS suspending the process. The silent sink that stands in while in the background does not make up the time the process was suspended (it used to pull seconds' worth at once and log hundreds of underruns). + +### Several outputs at once (`audio #N`) + +- **Symptom:** combined numbers mislead: one fully fed output plus one fed by nobody look like "every other callback underruns, 2.0 blocks queued". +- **Cause:** a game may open several outputs. Engage opens one AudioOut from Wwise (`main+0x177d924`, no release event, the game polls); on the phone it opened a second one later, and the Mac runs never did. Unity's FMOD also has an output (`main+0x4aec0c` / `0x4aec4c`). +- **Fix:** the host logs each output as `audio #N`: the open with its call site, Start / Stop / Close (the first 32 per output), and the callback line every 2000 of that output's callbacks. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md). + +### Checking audio without listening + +`DM_AUDIO_DUMP=file` writes the PCM the host plays (16-bit, 48 kHz, stereo). RMS per second, clipping share, spectral flatness (music ≈ 0.1–0.5, white noise ≈ 0.85) and the strongest FFT peaks (they should land on musical notes) tell real music from noise or silence. + +### AudioOut volume kept per output + +- **Symptom:** a null read at `sdk+0x1f2314` as the first movie starts (the title menu's background, right after the first press), in `nn::audio::SetAudioOutVolume`, called from `main+0x1c4fbc`. +- **Cause:** the call is on the AudioOut the game opens for the movie's AAC sound (`OpenDefaultAudioOut` without an event, `main+0x1c48c0`). The SDK's version calls the audout session's virtual method, and the host never made the session. The game sets the volume every frame while a movie plays. +- **Fix:** the host keeps the volume per output and scales the PCM16 it plays, saturating; `GetAudioOutVolume` reads it back (`hle_audio.c`). +- **Found on:** [STEINS;GATE RE:BOOT](../../dm/games/01008BC024FDE000/README.md) (SDK 20.5.17, M2's ADV engine; the one call it needed). + +### AudioOut buffer count from the host queue + +- **Symptom:** a null read in `audio::GetAudioOutBufferCount` on Wwise's sink wakeup thread (`sdk+0x1e7ed0`, from `main+0x30faae8`). +- **Cause:** after every release event Wwise asks how many buffers are still queued and marks the sink starved at 0; the SDK's version read the audout session the host never made. +- **Fix:** it answers from the host queue: appended and not yet played to the end (`hle_audio.c`). +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11; one boot). + +### Audio device output volume and names + +- **Symptom:** an abort in `sm::Initialize` under `audio::GetAudioDeviceOutputVolume` (the game thread, `main+0xee8edc`). +- **Fix:** the volume `SetAudioDeviceOutputVolume` sets is now kept and read back (`hle_audren.c`). `audio::GetAudioOutName` is hooked too. +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (SDK 19.3.1, Unreal Engine 4.27: `GetAudioDeviceOutputVolume`); [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (SDK 11: `GetAudioOutName`). +- **Note:** older SDKs name some device calls differently (`ListAudioDeviceName` / `GetActiveAudioDeviceName`, `OpenDefaultAudioOut` in SDK 12; [Older SDK names for the same calls](services.md#older-sdk-names-for-the-same-calls)), and [Super Mario Odyssey](../../dm/games/0100000000010000/README.md) (SDK 7) aborted in `audio::AcquireAudioDeviceSwitchNotification` ([SDK calls that open a system service through sm](services.md#sdk-calls-that-open-a-system-service-through-sm)). + +### No audio input devices + +- **Symptom:** SDL3's audio driver lists the input devices (AudioIn) when its audio starts. +- **Cause:** the host has no microphone. +- **Fix:** `ListAudioIns` reports none, both opens fail, and the calls that need an open AudioIn answer without touching the game's object (`hle_audio.c`). +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) (SDK 18.3.2, FNA on .NET NativeAOT with SDL3; one boot each). + +## Codecs + +### Hardware Opus decoders built on the SDK's software decoders + +- **Symptom:** an abort in `sm::Initialize` under the hardware Opus decoder's C interface; for the multi-stream one, under `nn::codec::GetHardwareOpusMultiStreamDecoderWorkBufferSize(int, int, int, int, int)` (at `nnMain`, CRI's Opus streams). An abort in `sf::hipc::InitializeHipcServiceResolution` under `codec::GetHardwareOpusDecoderWorkBufferSize(int, int)` (`main+0x31f8e8`, early in `nnMain`). A game that builds `HardwareOpusMultiStreamDecoder` with the imported constructor and then calls it through its vtable. +- **Cause:** the hardware decoders are a service. The `HardwareOpusMultiStreamDecoder` class's 12 vtable slots match the SDK's software `OpusMultiStreamDecoder` one by one. SDK 11 has a work buffer size form without the option argument; the host had only the three-argument one. +- **Fix** (`hle_codec.c`): + - the classes: the constructor hook builds the software decoder in place (0x38 of the 0x48 bytes), as for the single-stream `HardwareOpusDecoder`; `GetHardwareOpusMultiStreamDecoderWorkBufferSize` returns the software size. Both destructors are a bare `ret`. + - the single-stream C interface (`GetHardwareOpusDecoderWorkBufferSize`, `InitializeHardwareOpusDecoder`, `DecodeOpusInterleavedWithHardware`, …) is forwarded to the `sdk`'s software `OpusDecoder` C interface with a host-held decoder state per game object. + - the multi-stream C interface, including the SDK 12+ overload with an option argument, goes to the `sdk`'s software `OpusMultiStreamDecoder` C interface, with a host-held state per game object as for the single-stream decoder. `GetHardwareOpusMultiStreamDecoderProcessingTime` reports 0. + - the two-argument `GetHardwareOpusDecoderWorkBufferSize` is hooked beside the three-argument one. +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md) (SDK 9.6: only the `HardwareOpusMultiStreamDecoder` class and the four-argument size); [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) (1.4.3, the single-stream C interface); [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3, the two-argument size); [Unicorn Overlord](../../dm/games/010069401ADB8000/README.md) (SDK 16.2.4, the multi-stream C interface; one boot). + +## Timing and catch-up + +### At least one rendered frame between two updates + +- **Symptom:** the light limiter crashed on a null state pointer (`ApplyLightLimiterEffect`, `sdk+0x228494`). +- **Cause:** an effect's "needs init" parameter state is set only in the update where the game flags it new; the next update copies the game's value over it. Mario updates twice in a row at start-up; with a real audio device the first frame came after both. +- **Fix:** the `AudioRenderSystem::Update` hook renders a catch-up frame first when nothing was rendered since the last update. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). +- **Check:** `DM_AUDREN_TRACE=1` logs every update with the number of frames rendered since the previous one. + +### A sound driver that never waits on the renderer event + +- **Symptom:** the sound runs about 50 times faster than the output; the loop runs 11,000 updates a second on one core. +- **Cause:** Square Enix's `SQEX::Sd::Driver::Core::CoreAudioOut` (its loop at `main+0x442c5c0`) appends a wave buffer, mixes the next block on the same thread, calls `RequestUpdate` and takes the next released buffer. It waits on the renderer event only when no buffer is released. Every update there finds no frame rendered since the last one and renders a catch-up frame, which releases the buffer just appended, so the loop never waits. When a full catch-up queue discarded the frame, the loop ran unchecked. +- **Fix:** an update with a full queue waits (at most 50 ms) for the render thread to play one queued frame, so catch-up frames keep real time (`hle_audren.c`). The render thread then only plays queued frames, and the driver's updates render every frame. A queued frame does not signal the renderer event (the SDK signals it in each render): signalled, an ordinary game woke, updated with nothing rendered since and rendered the next catch-up frame, so every frame became one ([Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md), about 38,000 catch-up renders instead of 37). +- **Found on:** [DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition](../../dm/games/01006C300E9F0000/README.md). +- **Check:** `audren: rendered 12000 frames, N of them not silent` shows whether a game's sound reaches the output. diff --git a/docs/reference/frame-rate.md b/docs/reference/frame-rate.md new file mode 100644 index 0000000..902ee47 --- /dev/null +++ b/docs/reference/frame-rate.md @@ -0,0 +1,99 @@ +# Frame rate + +Frame pacing: the host's frame step, engine frame limiters, present intervals and display rates. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Frame step + +### sead's dynamic present interval, `ksys::VFR` and the host's frame step + +- **Symptom:** a 30 fps game to run at 60; after the floor was lowered, slow motion on a GPU-bound device (GitHub issue #9). +- **Cause:** find where the present interval comes from and how the logic time step follows it before patching anything. BotW's `sead::GameFrameworkNx` main loop (`main+0x112660c`) runs a dynamic interval: from the last frame's time it computes how many vertical syncs were needed, clamps that to [2, 3] (the maximum from the interval controller built at `main+0x1126c04`), steps up after three overrunning frames and down one level every six frames, and sets the result through `setWaitVsyncInterval` (vtable +0x130, `main+0x11269b8` → `nvnWindowSetPresentInterval`). The game logic follows on its own: `ksys::VFR` is initialized with a constant base interval of 2 (`main+0x127e740`), and every frame `VFR::updateInterval` (`main+0x151d3d0`) takes the framework's current interval, giving a frame ratio of interval / 2, so the engine's variable frame rate scales its time step by the present interval and the speed is right as long as the interval matches the frames. Then check what the controller measures: BotW's measured only the main thread's CPU time (about 3 ms a frame), so a GPU-bound device stayed at interval 1 and played in slow motion. +- **Fix:** now the host's frame step in whole vsyncs: a stub at the framework's `setWaitVsyncInterval` hands the game the host's step, and frames stay on screen exactly that long. The method before it, which `DM_FRAME_STEP=legacy` still runs, cuts the game's frame-interval logic: five rewriter cuts lower the floor from 2 to 1 (its `patches.py`, build `cd57b23f…`): the candidate floor (two places) and the hysteresis targets. Two more cuts make the controller always take its GPU-time path, and the host's timestamp reports had to carry real GPU times for that path to see anything (profile key `gpu_timestamps`, [GPU timestamps per submission segment](#gpu-timestamps-per-submission-segment)). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (1.9.0; its note has the interval controller and the legacy cuts). +- **Check:** probes confirmed the legacy result: the framework presents at interval 1, VFR reports interval 1 with ratio 0.5, and when a frame runs long the game steps back to 2 with ratio 1, so the game speed does not change. +- **Note:** the zeldaret/botw decompilation (`VFR.cpp`) gave the semantics; 1.9.0's addresses were found by the constants (`60 / interval`, the `fmov s11, #2.0` floor). + +### GPU timestamps per submission segment + +- **Symptom:** slow motion when the GPU is the bottleneck: a game's interval controller that times the GPU never steps up (GitHub issue #9). +- **Cause:** a game that times its GPU work as the difference of two timestamp reports in one submission (`TIMESTAMP_TOP` at the start of the frame's command buffer, `TIMESTAMP` at its end) read 0–6 ticks, because the host wrote all the reports of a submission with one time in its completion handler. +- **Fix:** the profile key `gpu_timestamps segment` makes each report take the GPU start (`TIMESTAMP_TOP`) or end (other types) of the `sub_cut` segment ([Submissions handed to the GPU in segments](#submissions-handed-to-the-gpu-in-segments)) it was encoded in (`rep_ticks` in `nvn_mtl.m`: MTL4CommitFeedback times converted to the guest clock, written in the last segment's handler as before). `DM_NVN_GPU_TS=submission` gives the old timestamps. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (legacy frame-step method): BotW's frame has 5 reports in one submission (`main+0xe94584`, `main+0xe94664`); with two more cuts the controller always takes the GPU path, and the GPU time now reads the frame's GPU span, gaps where the GPU waits for the host's translation included. Measured on the Mac with two BotW runs sharing the GPU under a heavy CPU load (load average about 75, a shader rebuild running): GPU time p50 16–23 ms; the fixed run stayed at interval 2 (3 while the GPU time passed 1.9 vsyncs, 1 for under 2% of the frames) at 0.93–0.98 of real time, while a run with `DM_NVN_GPU_TS=submission` spent 10–70% of its frames at interval 1 and ran at 0.78–0.95. The same GPU time drives the game's dynamic resolution. + +## Engine frame limiters + +### A render-thread limiter with a constant and a fixed time step + +- **Symptom:** the game presents at interval 1, yet every frame takes exactly 33.3 ms of "frame work"; after only the limiter was raised, everything ran twice as fast on the iPhone. +- **Cause:** the cap is the render thread's own limiter, `main+0x707090(this, fps)`, which sleeps out 1000 / fps ms since its previous call, called once with the constant 30 (`main+0x706fa4`). Found from the render thread's backtrace in the thread dump (`HGCommandDispatcher`), not from the NVN calls. The frame update takes a fixed step, the float 1/30 at `main+0x167a6ec`, loaded at `main+0x667904` and passed to `main+0x6a337c`. +- **Fix:** two cuts: the limiter's constant, and a second cut that loads the 1/60 at `main+0x167a834` (the community's 60 fps cheat for this build changes the same two words). Some 20 other loads of that 1/30 turn parameters counted in 30 fps frames into seconds or step single systems; they stay. +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (30 on the Switch, 60 now). +- **Check:** before changing the constant, check how the game steps time, and not by eye: a 0.8 s walk that looked the same at 30 and 60 fps on the Mac was taken for a time-based game, and on the iPhone everything ran twice as fast. Probing every site that loads such a constant (`DM_PROBE`, 16 per run) tells per-frame steps from one-off conversions, but which of them sets the game's speed only shows by timing the game. + +### A frame-end wait polled in 2 ms sleeps + +- **Symptom:** 55 fps with every frame 18.2 ms, the main thread asleep most of the time; every 11th frame stays on screen two vsyncs. +- **Cause:** KTGL's frame end (`main+0x443e8c`) waits until 1/60 s has passed since the frame's start tick, polling `GetSystemTick` and sleeping 2 ms a round (`main+0x4c9f24`), so with about 2 ms of work the rounds end at 18.2 ms, never at 16.7. Found by sampling the host (`sample <pid>`: 90% of the main thread in `dm_guest_sleep_ns` under a guest return address of `main+0x443d74`) and a probe on the target's multiply (x20 = 1 frame, x0 = 19.2 MHz). +- **Fix:** the game presents at interval 1, so a build cut makes the check jump to the end, and the host's present pacing (the acquire waits) holds 60 (its `patches.py`): frame time p50 16.66 / p95 16.85 ms, wakeups 2,050 → 320 a second. +- **Found on:** [Taiko Risshiden V DX](../../dm/games/0100346017304000/README.md), in its menus and town; its movies, paced by their own player, ran at 60. + +## Present interval and display rate + +### Frames of a dynamic present interval held on screen and at the acquire + +- **Symptom:** "sometimes the speed increases when jumping" (Mario 1.2.1 on the iPhone): in `display` mode interval-2 frames were shown for 16.7 ms and the game visibly jumped ahead. "When pressing B, Link's move pace is speeding up" (Breath of the Wild on the iPhone); `old frames dropped by merging`. +- **Cause:** Mario times every frame (`main+0xcfd24`); after a frame that ran past one vertical sync it sets present interval 2 for the next frame *and* sets its own time step to the interval (a float at `main+0x36ceff0`), so that frame catches up 2/60 s. A user's 2-minute iPhone session switched 123 times, mostly for one frame. On the game side, the acquire waited for one display-link tick per frame whatever the interval, so the game kept producing frames every 16.7 ms while the screen held each interval-2 frame for 33 ms and dropped the one in between. BotW sprinting flaps between interval 1 and 2 for runs of 6 or more frames, and its VFR time step at interval 2 is a whole 30 fps tick, so those runs went at double speed. +- **Fix:** both sides, fixed: + - the screen: in `game` mode (the default since 2026-10-15) the display-link callback keeps a frame whose period is over 25 ms on screen for its whole period before taking the next one (`g_link_hold_until` in `nvn_mtl.m`; the `display link up to tick` log line counts these ticks as `held for the present interval`), and the 30 fps target that drops the display link to 30 Hz (`nvn_game_fps` in `hle_nvn.c`) waits until interval 2 has held for a full second, so one-frame switches no longer flip the display-link mode; + - the game: `mtl_link_wait_tick` now waits for as many ticks as the game's present interval (times 2 on the 120 Hz link, half rounded up on the 30 Hz link), as the Switch blocks the next acquire for N vertical syncs. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (1.2.1, iPhone: the display-link hold); [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (iPhone: the acquire). +- **Check:** `DM_TEST_PRESENT_LAG=90,30` (sleep 30 ms before every 90th present) makes Mario do this on the Mac; run it with `DM_WINDOW=1` to get the display link. `DM_NVN_TSV` writes each frame's interval as `@itv`. +- **Note:** the windowless Mac host was never affected by the acquire: there the acquire sleeps until the interval's slot. + +### Vulkan acquire and vsync event follow the display link + +- **Symptom:** a steady game whose frame-rate counter wanders between 50 and 60 on the iPhone only; the `display link up to tick` lines show about as many empty ticks as frames dropped by merging: frames arrive two in one tick and none in the next. +- **Cause:** the Vulkan layer's acquire and the `vi::GetDisplayVsyncEvent` thread each kept a 60 Hz clock of their own, and Galaxy runs the Wii's VI retrace on that event, so its frames drifted against the screen. +- **Fix:** while the display link presents, both follow its ticks (`hle_vk.m` acquire → `mtl_link_wait_tick`, as NVN's; `hle_sdk16.c` vsync thread → `mtl_link_wait_vsync`), with the old clocks as the fallback. +- **Found on:** [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md) (Vulkan). +- **Note:** a Mac window measures nothing here unless it is on screen: macOS fires an occluded window's display link about once a second (`display link recovered: stalled 1017 ms`). + +### Virtual 60 Hz ticks on screens that are not a multiple of 60 + +- **Symptom:** asked for 60 Hz, the system fires the display link on every third refresh of a 165 Hz screen, 55 Hz, so games paced by it ran at 55 fps (27.5 at 30). +- **Fix:** on 144 or 165 Hz Mac displays `link_range` (`nvn_mtl.m`) runs the link at the screen's rate and counts as ticks only the refreshes nearest a steady 1/60 s clock: 60 a second on average, each frame shown 12.1 or 18.2 ms at 165 Hz. A tick lies up to half a refresh off its slot, so two ticks can be one refresh nearer each other than their slots: the hold of frames over 25 ms ([Frames of a dynamic present interval held on screen and at the acquire](#frames-of-a-dynamic-present-interval-held-on-screen-and-at-the-acquire)) tolerates one refresh more (in a simulation of the callback without it, 30 fps frames held a third tick: 22.5 fps at 165 Hz, 24 at 144), and the frame step lets a frame on time show one refresh longer than its hold (`shown_slack`, `nvn_fstep.h`; a cheap 50 ms frame shows 54.5 ms at 165 Hz, and counted as late it kept Breath of the Wild at 20 fps for good). `DM_LINK_VTICK=0` keeps the system's choice. +- **Note:** iOS screens run at multiples of 60 and are not affected. +- **Found on:** PR #27 (its author measured at 165 Hz); [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (the 20 fps without `shown_slack`). +- **Not checked:** not yet run on such a screen here. + +### Fullscreen switches the screen to a rate the frame rate divides + +- **Symptom:** virtual ticks keep the average, but a frame still stays up two or three refreshes in turn (13.9 / 20.8 ms at 144 Hz), and Apple's HUD showed the frame interval jumping between them. +- **Fix:** `front_mac.m` `front_screen_match`, called from `link_sync` with the link's rate (60, 30, or 120 with `DM_PRESENT_LINK=2`): when the window is fullscreen on a screen whose mode is not a multiple of the link's rate, the host switches that screen to the mode of the same size with the highest rate that is (an XB273U at 2560x1440: 144 → 120 Hz for 60 fps), with `kCGConfigureForAppOnly`, so macOS switches it back when the host quits, crashes included (the PR's author checked: 144 Hz again after quitting); leaving fullscreen, or the screen, switches it back at once. A window keeps the screen's rate and the virtual ticks: the switch changes the whole screen for every app, and most external screens go black for a second or more. The link sets its range again whenever the screen's rate changes (`link_sync`; before, a move to another screen kept the first screen's range), with no virtual ticks after a switch. A screen with no such mode keeps the virtual ticks; `DM_SCREEN_MATCH=0` keeps the screen's rate. +- **Found on:** PR #79: the PR's author ran [Hades](../../dm/games/0100535012974000/README.md) at 144 Hz (switched in a window then): the HUD's frame interval line steady at 16.7 ms, where it jumped before. +- **Check:** here (a built-in 120 Hz screen, a multiple of every link rate) a harness drove `front_mac.m` asked for 50 fps: no switch in the window, 120 → 50 Hz in fullscreen, 120 Hz again on leaving it, and 120 Hz after quitting in fullscreen; at 60 fps and with `DM_SCREEN_MATCH=0` nothing switched. +- **Not checked:** the link's new range after a switch has not run here (Hades windowed and fullscreen paced at 60 as before). + +## CPU cost and pacing + +### Submissions handed to the GPU in segments + +- **Symptom:** the GPU sits idle each frame while the host translates the submission (a game that does not submit frame N+1 until frame N's GPU work is done). +- **Cause:** on the Switch the submit costs nothing; here it is the host's translation, and the GPU got nothing until the whole submission was encoded. +- **Fix:** the replay commits the command buffer encoded so far every 64 draws, between two passes (`sub_cut` in `nvn_mtl.m`, called after `rp_flush` in `OP_SET_RT`; `DM_NVN_CUT=<draws>`, 0 = off). It cuts only with no open encoder and nothing deferred, merged or hoisted, commits the pending upload batch first, and continues on the same command allocator. The fence signal, the completion work (`post` ops, `g_done`) and the allocator's fence stay on the last segment: the queue runs command buffers in commit order, and every encoder starts with a queue barrier. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): a stack sample in the shrine had the main thread in `nvnSyncWait` (the fence of the frame just submitted) for 59% of the time and the render thread in `nvnQueueSubmitCommands` for 40%; the translation was about 3,600 draws and 347 passes, 7 ms, so per frame the GPU ran 13–14 ms and then sat idle 7.7 ms (`DM_NVN_TSV` rows `@gpu_beg_ms` / `@gpu_end_ms` against `@call_ms`). GPU idle per frame went from 7.7 to 3.8 ms; most of the rest is the game's own work between its fence wait and the next submit (about 2.5 ms). The other five games' frame times were unchanged; Metroid Dread's p95 dropped from 19.6 to 16.9 ms. +- **Note:** after this, BotW's docked frame p50 was 16.7 ms (was 20.8 ms before segmenting). Host translation is 8 ms per frame (22,000 instructions per draw), spread over Metal's encoder creation and draw calls, argument encoding, Objective-C retain / release and pool lookups. The per-pass GPU timer (`DM_NVN_PASSGPU`) is switched off under Metal 4 (`pg_render` returns at once), so GPU time cannot yet be attributed to passes. Render-target traffic per frame is in `DM_NVN_TSV`'s `@rt_load_mb` / `@rt_store_mb`. + +### Address to memory pool lookups by binary search + +- **Symptom:** the submit thread's CPU samples dominated by `mtl_addr` / `mtl_gpu_addr` (vertex buffers in `draw`, uniform buffers in `encode_args`). +- **Cause:** with many memory pools alive, the lookups scanned them all for every buffer binding of every draw. +- **Fix:** they try the last hit, then a binary search over pools sorted by start address (`pool_find`, rebuilt when pools are added or removed). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): about 1,760 memory pools alive; a CPU sample of the submit thread had about 1,000 of its samples under those scans. Host submit time in the shrine went from 21 ms to 7 ms per frame (p50), the median frame from 38 ms to 21 ms. + +### Time-constraint computation declared below the thread's peaks + +- **Symptom:** stalls of 30–60 ms about once a second, at 30 fps and at 60 (99th percentile 63–127 ms); the main thread waits 20–34 ms for the render thread once a second. +- **Cause:** the thread that acquires the window (here the render thread) runs under a time-constraint policy whose computation the host re-declares every 120 frames; it was 1.25 × the 90th percentile of that thread's CPU time per frame. This render thread needs 1.9–2.0 ms a frame with one or two frames in 120 near 4.4 ms, against 2.5 ms declared, and each frame past the declaration stalled the thread. Probes on both threads' waits showed the main thread's waits; adding probes slowed the render thread and the stalls vanished, and `DM_RT=0` or a fixed `DM_RT_COMP=5` / `10` removed them too (99th percentile 18–21 ms) at the same CPU energy. +- **Fix:** the declaration also covers 1.1 × the window's peak (`rt_comp_tick` in `hle_nvn.c`): 5.6 ms here, 60.0 fps, 99th percentile 21 ms. +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md), in the Asylum cell. diff --git a/docs/reference/game-state.md b/docs/reference/game-state.md new file mode 100644 index 0000000..b1a80a8 --- /dev/null +++ b/docs/reference/game-state.md @@ -0,0 +1,50 @@ +# Game state + +The games' own state: saves, scripted state, and behaviour that looks like a host bug but is not. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Game behaviour that is not a host bug + +### Most draws are in command buffers replayed later + +- **Symptom:** "stuck" at a screen with one draw per frame. +- **Cause:** usually not stuck: most draws are in recorded command buffers replayed later. +- **Fix:** dump the passes (`DM_NVN_PASSDUMP`, [04, Diagnostic switches](../agents/04-graphics.md#7-diagnostic-switches-host)) before assuming the game logic is blocked. + +### Buttons mapped by position: B confirms + +- **Symptom:** menus ignore A. +- **Cause:** not a host fault: the game maps its buttons by position, B confirms and A goes back (its prompts say so). +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (SDK 4.5.0). The first boot shows two notices before the title menu ("Unable to connect to the network", then "Game will start in offline mode"), and a new game first walks through Brightness, Options and Button settings. + +### Any Lua error quits a LÖVE game + +- **Symptom:** a crash in `main+0x339c00` a second or two into the title, in love-nx's video module teardown (`love::video::theora::Video` → `~Worker`; its thread released the worker that the closing module deletes too). +- **Cause:** the Lua state was closing: on the Switch, Balatro's `love.errhand` returns at once, so any Lua error quits. A `print` at the top of `love.errhand` in a copy of the RomFS (`DM_ROMFS=<copy>`) gave the error: `Shader uniform 'distortion_fac' does not exist` (the capture run had no shader library, so every program linked with no uniforms), then `bad argument #2 to 'send' (number expected, got boolean)` (a `bool` uniform reported as `GL_UNSIGNED_INT`). +- **Fix:** look for the Lua error first; the two errors here were host faults in the GL layer's shaders and uniforms (see [GL shaders put together at run time and captured](rendering.md#gl-shaders-put-together-at-run-time-and-captured) and [GL uniforms, pipelines and attributes](rendering.md#gl-uniforms-pipelines-and-attributes)). +- **Found on:** [Balatro](../../dm/games/0100CD801CE5E000/README.md) (SDK 18.3.2, LÖVE 11.4; one boot). +- **Check:** probing the destructors (`DM_PROBE`) and logging every object the Lua GC released (`DM_PROBE=main:0x18b920@x0+0`: the module proxies all at once) showed it was a shutdown, not a premature collection. + +### TV latency calibration's late, beat-timed listening window + +- **Symptom:** the TV latency calibration ("press A when you hear the fourth beat") ignores A. +- **Cause (not a host bug):** docked, the first boot runs it (scene script `main+0x4c0820`). It starts listening only `0x3c0` frames (16 s) after the prompt, through the active controller (`main+0x22b0` id `0x3ed`, virtual button 4 = A), and passes when about ten presses keep a steady offset within −5 to +13 frames of the cue (evaluation `main+0x2948c0`; offset = cue ticks at `+0x34` / about 17 ticks per frame, about 480 per beat). +- **Testing it:** scripted presses at fixed seconds drift off the beat; use frame-timed `DM_KEYS` at the bar period (117.12 frames at its 123 BPM) and read each press's offset with `DM_PROBE=main:2951f0@34`. A passed calibration is saved per TV panel identity (the host's all-zero one included, [Default display panel identifier](services.md#default-display-panel-identifier)) and not asked again. +- **Profile:** it plays handheld with Boost (`op_mode handheld`, `perf_mode boost`), which skips the TV setup and keeps the 1080p draw size. +- **Found on:** [Rhythm Heaven Groove](../../dm/games/0100D9F01D474000/README.md) (SDK 21.4.0). + +## Scripted state + +### The software keyboard in scripted runs (`DM_KBD`) + +- **Symptom:** in a scripted run, a screen goes back by itself right after a confirm; the log shows `swkbd: … → no window, treated as cancel`. +- **Cause:** the confirm opened the system keyboard (`swkbd`, name entry). Without a window the host answers "cancel" by design. +- **Fix:** pass `DM_KBD=<text>` to answer it. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) (its appearance choice). + +### Player join screens (`hid::StartLrAssignmentMode`) + +- **Symptom:** a player join screen that ignores A (a character select with four "L + R to join" slots after the title). +- **Cause:** the game calls `hid::StartLrAssignmentMode` and waits for L and R pressed together on the join screen itself, then a character choice with A. +- **Fix:** in `DM_KEYS` that is `L+R~0.5` while the screen is up (presses on the title before it are lost). +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md). +- **Note:** on the game select, holding the stick right for a second (`7:LRight~1`) switches to Bowser's Fury, which also uses `nvnTextureGetSwizzle` and `nvnTextureGetStorageClass`. Map and level movement read the left stick only: `DM_KEYS` takes `LLeft` / `LUp` / `LRight` / `LDown`. A working route into World 1-1 on a fresh save: `DM_KEYS=6:A,10:A,14:L+R~0.5,17:A*3/2,24:Plus*6/3,45:A*6/3,66:LUp~1,70:A,74:LRight~1,78:A,82:LLeft~1,86:A` (World 1-1 by frame ~6000). diff --git a/docs/reference/memory.md b/docs/reference/memory.md new file mode 100644 index 0000000..a37322b --- /dev/null +++ b/docs/reference/memory.md @@ -0,0 +1,264 @@ +# Memory and address space + +The guest address space, heaps, pools, virtual address memory and the allocators games bring. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Guest layout + +### Partitioning the guest block between alias region, stacks and heap + +- **Symptom:** large resources silently skipped, later a null deref where the resource is used; also visible as `SetHeapSize` asking for much less than the pool. +- **Cause:** the game reserves "virtual address memory" (Mario: 1.75 GB in `HeapDef.byml`) and maps it into the alias region with `MapPhysicalMemory` on demand; the alias region left after the node area ([The SDK's address-region node area](#the-sdks-address-region-node-area)) was too small. +- **Fix:** repartition the guest block (`hle_svc.c` `mem_init`): Mario uses alias 2.875 GB, stacks 256 MB, heap 1.625 GB (it asks for 1.47 GB), 5 GB in total. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). +- **Check:** measure before sizing (`DM_SVC=1`: the `SetHeapSize` request and the highest `MapPhysicalMemory` address). +- **Note:** on iOS without the extended-virtual-addressing entitlement a single 6 GB reservation succeeds and 7 GB fails, but Metal and the rest of the process need address space too; 5 GB is the total proven on a phone (D2R). + +## Heap and application pool + +### Heap sized from the application pool less used memory + +- **Symptom:** `nnMain` returns right after `nv::InitializeGraphics`, with no error (Hyrule Warriors: Age of Calamity); `[mem] SetHeapSize … + modules … > app pool` (Monster Hunter Rise), `[mem] SetHeapSize … + modules 0x5c00000 > app pool` (Catherine: Full Body); `SetHeapSize` asks for almost the whole application pool and fails at boot (Metroid Dread: 0xc9400000, larger than the heap region); the memory pool for the game's 96 MB GPU heap has no storage (`MemoryPoolInitialize` red line), then a null read at `main+0x2eb954` (Catherine: Full Body); heap allocations fail late in a busy scene. +- **Cause:** the SDK sizes the heap as total − used memory (`QueryMemoryInfo`), so the reported application pool and used memory decide the game's heap size. Age of Calamity's `nninitStartup` does so, calls `SetMemoryHeapSize` and hands the block to its own malloc. The host reported 64 MB used beside the heap ("used" = heap + 64 MB), but `SetHeapSize` checks a request against the real size of the loaded modules (66.6 MB in Age of Calamity), so `SetHeapSize(pool − 64 MB)` failed with ResultLimitReached, `AllocateMemoryBlock` returned 0x1003, malloc was never set up, the 866 MB graphics heap allocation came back null and `nnMain` returned. Monster Hunter Rise's modules take 320 MB (`main` alone 314 MB), so its `SetHeapSize(pool − 64 MB − bss)` failed the same way. Catherine: Full Body's rewritten modules take 92 MB, so `SetHeapSize(pool − 64 MB)` failed and the game ran on a 32 MB heap. The 64 MB covers ordinary modules (Mario's two are about 74 MB, and its heap request matches hardware), but Metroid Dread's `main` carries a 1.8 GB bss that the Switch counts as used. +- **Fix:** report Switch-accurate numbers. The application pool was a host constant (`APP_POOL`, 3285 MB); it is now the profile key `app_pool` (`hle_svc.c`: `#define APP_POOL DM_GAME_APP_POOL`, Mario 0xCD500000, the same 3285 MB). The profile key `mem_used` sets the used-memory figure: 72 MB for Age of Calamity (the image plus stacks); 328 MB for Monster Hunter Rise, which gives a 2.88 GB heap; 96 MB for Catherine: Full Body, the NSP script's guess after the rewrite. Used memory now also adds the bss of modules in the front-patch layout (`big_bss_bytes` in `hle_svc.c`; the layout is in [Patch area before the module base for a large bss](rewriter.md#patch-area-before-the-module-base-for-a-large-bss)); Dread then asks for 1.38 GB. +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md); [Metroid Dread](../../dm/games/010093801237C000/README.md); [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md) (SDK 8.2.1; one boot). The source names no game for the late failures in a busy scene. +- **Check:** probe `DM_SVC=1`'s `svc 0x1` request and the `AllocateMemoryBlock` result. + +### `nninitStartup`'s heap only without virtual address memory + +- Not needed in the end for [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) (1.0.0.2, SDK 17.5.3): the process heap of pull request #16. `nninitStartup` calls `SetMemoryHeapSize` / `AllocateMemoryBlock` only without virtual address memory, and nothing else in `main` calls them ([Virtual address memory follows the NPDM's system resource](#virtual-address-memory-follows-the-npdms-system-resource)). + +### Sizing the pool and alias region for virtual address memory + +- A title that uses virtual address memory may never call `SetHeapSize`: everything comes from `MapPhysicalMemory` in the alias region. The `[mem]` log lines show what it maps ([Measuring address regions and commits](#measuring-address-regions-and-commits)). +- What fills the alias region first is address space, not memory. StandardAllocator reserves the whole free pool (pool − 64 MB, 1 GB aligned) but Engage commits only 0.58 GB of it, and Unity's own regions commit 0.97 GB. So Engage's profile reports a 2.5 GB pool (`app_pool`) and uses a 4.375 GB alias region in a 5 GB block (reserved address space peaks at 3.47 GB of 4.31 GB). +- iOS without the extended-virtual-addressing entitlement allows a single 6 GB reservation, but Metal needs address space too; 5 GB is the total proven on a phone ([Partitioning the guest block between alias region, stacks and heap](#partitioning-the-guest-block-between-alias-region-stacks-and-heap)). +- The pool a game is told decides how much StandardAllocator reserves (`Initialize(nullptr, pool − used)`), and the host does not enforce the pool (only `nninitStartup` reads it), so a pool a little above what the game maps leaves the most address space: + - Ace Attorney Investigations Collection: with Engage's 2.5 GB pool the game ran, but StandardAllocator's 2.44 GB reservation beside Unity's pools (1.06 GB live, from `main+0x40f1d4`) ended 4.37 GB into the 4.375 GB alias region after 140 s, while the `MapPhysicalMemory` log showed StandardAllocator committing 0.19 GB. The profile reports a 1 GB pool, a little less than the 1.02 GB the game maps at its peak; ten minutes then peak at +2.85 GB of address space, against +3.91 GB with a 1.5 GB pool. + - Brain Memory Training Academy: with Ace Attorney's 1 GB pool and 5 GB block the game boots to its title in 5 s with no red line; StandardAllocator ends at +1.0 GB, and five minutes of level loads take the alias high-water mark to +2.25 GB with at most 0.5 GB mapped. + - Patrick's Parabox: with Ace Attorney's 1 GB pool and 5 GB block (in place of the provisional profile's hardware pool) it boots to its title with no red line, and the alias region is mapped up to +1.5 GB with 0.27 GB in use. + - Advance Wars 1+2: Re-Boot Camp: the hardware pool used the alias region up to +4.25 GB; the heap check's 1 GB pool starved its malloc ([Unity's GPU pool block allocation spins at size 0](#unitys-gpu-pool-block-allocation-spins-at-size-0)); with a 2 GB pool the game maps up to 2.26 GB of the alias region through its menus and first battle. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md); [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) (Unity 2022.3, SDK 17.5.3); [Brain Memory Training Academy](../../dm/games/0100F5D01DA3A000/README.md); [Patrick's Parabox](../../dm/games/0100E44016C22000/README.md); [Advance Wars 1+2: Re-Boot Camp](../../dm/games/0100300012F2A000/README.md). +- **Note:** to measure regions and commits, see [Measuring address regions and commits](#measuring-address-regions-and-commits). + +## Virtual address memory and the alias region + +### Virtual address memory follows the NPDM's system resource + +- **Symptom:** `StandardAllocator::Initialize` aborts in `nninitStartup` (Fire Emblem Engage); `nn::diag` abort in `StandardAllocator::Initialize(void *, size_t, bool)` from `nninitStartup` (`sdk+0x2f98e4` ← `main+0x5b69e0`, Big Brain Academy: Brain vs. Brain). +- **Cause:** the SDK's `IsVirtualAddressMemoryEnabled` is `GetInfo(TotalSystemResourceSize) != 0` (`VammManager::IsVirtualAddressMemoryEnabled`, `sdk+0x34e074`), the system resource the NPDM declares (offset 0x14 of `META`). With it the game's allocator takes its memory as address regions in the alias region. Engage's NPDM declares one, so `IsVirtualAddressMemoryEnabled` is true and the allocator reserves all free memory (about 3.2 GB for Engage) as one region in the alias region, after the ~1 GB node area ([The SDK's address-region node area](#the-sdks-address-region-node-area)). Big Brain Academy's `nninitStartup` (`main+0x5b6984`) asks `os::IsVirtualAddressMemoryEnabled()`: when true it gives StandardAllocator a 60 GB address region (`0xf00000000`), when false it sets a heap of pool − used (2 MB aligned) and hands that over. The host reported a 32 MB system resource to every game, while this game's NPDM declares none. +- **Fix:** the profile key `sys_resource` reports the NPDM's size (0 for Big Brain Academy, so the heap path, as on hardware; `hle_svc.c`); the survey writes it for every new game, and profiles without it keep the 32 MB answer. A game with virtual address memory needs a profile `mem_alias` / `mem_reserve` that holds its regions: Engage first used 5.5 / 8 GB on the Mac; its profile now has `mem_reserve 0x140000000` / `mem_alias 0x118000000` (5 GB / 4.375 GB) with the node area cut and a 2.5 GB `app_pool` ([Sizing the pool and alias region for virtual address memory](#sizing-the-pool-and-alias-region-for-virtual-address-memory)). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md); [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3, NintendoWare Bezel Engine). + +### The SDK's address-region node area + +- **Symptom:** the first `AllocateAddressRegion` fails with 0x1803; `StandardAllocator::Initialize` aborts in `nninitStartup` (an `nn::diag` abort; the NSP script's heap check reports "the host stopped before nnMain"); the alias region fills in minutes; regions a game reserves later fail ([Size-aligned regions exhaust the alias address space](#size-aligned-regions-exhaust-the-alias-address-space), [The movie player's own StandardAllocator](#the-movie-players-own-standardallocator)). +- **Cause:** SDK 16's AddressRegionManager takes 0x3fe00000 at the start of the alias region as a node area, so the alias region must be larger than 1 GB; SDK 8 has the same 1 GB node area as the newer SDKs, and SDK 7 / 8 builds load it in another form (below). On hardware the alias region is 64 GB. On Ace Attorney Investigations Collection's SDK 17 build, `CentralHeap::Init(nullptr, pool − used)` got no 1 GB-aligned region for 3.14 GB after the 1 GB node area. +- **Fix:** shrink the node area with a rewriter cut for that SDK build (`dm/tools/sdk_patches.py`): the two `mov w8/w9, #0x3fe00000` in the AddressRegionManager constructor → 62 MB. +- **Finding the constructor on a new `sdk` build:** search its text for the two `mov w8 / w9, #0x3fe00000` words (`08fca752`, `09fca752`); `sdis.py --fn` names the function `AddressRegionManagerC2Emm` from its dynamic symbols. SDK 7 / 8 builds load the node area with `orr w8 / w9, wzr, #0x3fe00000` (`e8230b32`, `e9230b32`), so the `mov` search finds nothing there. +- **Found on** (the builds that take the cut): + - [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): the two `mov w8/w9, #0x3fe00000` in the constructor → 62 MB; without it the alias region ran out of address space minutes into play ([Size-aligned regions exhaust the alias address space](#size-aligned-regions-exhaust-the-alias-address-space)). + - [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) (Unity 2022.3, SDK 17.5.3): `StandardAllocator::Initialize` aborts in `nninitStartup` on SDK 17, then the alias region fills in minutes; the same node area as Engage's SDK; the constructor is at `sdk+0x139458` on that build and takes the same cut. + - [Brain Memory Training Academy](../../dm/games/0100F5D01DA3A000/README.md) (Unity 2022.3.5f1, SDK 16.2.4): `nn::diag` abort at `sdk+0x126490`, `StandardAllocator::Initialize`, called from `nninitStartup` at `main+0x73adcc`. Its `nninitStartup` is Ace Attorney's: `Initialize(nullptr, pool − used)` with virtual address memory and no `SetMemoryHeapSize`. The `sdk` build is Unicorn Overlord's, whose NPDM declares no system resource, so it never needed the cut; the constructor (`AddressRegionManagerC2`, `sdk+0x134768`) takes it at `+0x134778` / `+0x13480c`. + - [Patrick's Parabox](../../dm/games/0100E44016C22000/README.md) (Unity 2020.3.30f1, SDK 13.3.7): `StandardAllocator::Initialize` at `sdk+0x324dec`, called from `nninitStartup` at `main+0x1cd080`; the same `nninitStartup` and the same cut, on this build's constructor (`sdk+0x37c3a0`: `+0x37c3b0` / `+0x37c44c`). + - [Advance Wars 1+2: Re-Boot Camp](../../dm/games/0100300012F2A000/README.md) (Unity 2020.2.1f3, SDK 10.4.1): `StandardAllocator::Initialize` at `sdk+0x2edf90`, called from `nninitStartup` at `main+0x365310`; the heap check reports "the host stopped before nnMain"; the same `nninitStartup`. The `sdk` build is The Great Ace Attorney Chronicles', which has no virtual address memory; its constructor (`sdk+0x340e90`) has the `orr` form (`e8230b32` at `+0x340ea0`, `e9230b32` at `+0x340f3c`). + - [Pikmin 4](../../dm/games/0100B7C00933A000/README.md) (SDK 10.4.8): the build's AddressRegionManager is the 10.4.1 build's at the same offsets and takes the same three cuts (node area and alignment). + - [Super Smash Bros. Ultimate](../../dm/games/01006A800016E000/README.md) (SDK 8.2.1): constructor `sdk+0x336b40`, `orr w9, wzr, #0x3fe00000` at `+0x336b50` / `+0x336bf0` ([A 62 GB page allocator behind `nu::VirtualAllocHook`](#a-62-gb-page-allocator-behind-nuvirtualallochook)). + - [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (SDK 7.3.2): the node area cut on that SDK build (`sdk_patches.py`). + - [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (SDK 15.3.2, Celeste's `sdk` build, which has no virtual address memory): constructor `sdk+0x12e0e0`: `+0x12e0f0` / `+0x12e184`. + - [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md) (SDK 18.3.5): `sdk+0x13be4c` / `+0x13bee0` ([The movie player's own StandardAllocator](#the-movie-players-own-standardallocator)). + - [Absolum](../../dm/games/0100A83022A22000/README.md) (SDK 18.3.2): the node-area cut beside the alignment cut. + - [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (SDK 19): SDK 19's AddressRegionManager node area is cut to 62 MB as Engage's (`sdk+0x1423fc` / `+0x142468`). + - [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (SDK 19.3.11): the same constructor offsets, `sdk+0x1423fc` and `+0x142468`, beside a cut on `main` ([StandardAllocators over 12 to 62 GB of address space](#standardallocators-over-12-to-62-gb-of-address-space)). + - [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) (SDK 20.5.17): the `sdk` is [STEINS;GATE RE:BOOT](../../dm/games/01008BC024FDE000/README.md)'s build; its node area takes the same cut (`sdk+0x145cbc` / `+0x145d28`). + - [Hollow Knight: Silksong](../../dm/games/010013C00E930000/README.md) (SDK 20.5.8): `sdk+0x145bfc` / `+0x145c68` ([Unity 6 takes its memory with no StandardAllocator and no heap](#unity-6-takes-its-memory-with-no-standardallocator-and-no-heap)). + - [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) (SDK 20.5.22, the same `sdk` build as [Breath of the Wild](../../dm/games/01007EF00011E000/README.md) 1.9.0): constructor `sdk+0x145cfc`, `+0x145d0c` / `+0x145d78`; no alignment cut (`VammManager::AllocateAddressRegion`, `sdk+0x145df0`, aligns regions to 64 KB or 2 MB only). Its resource heaps live in address regions after the node area: at the first descent from the Great Sky Island the alias region's high-water mark was +2.25 GB of its 2.375 GB with 1.00 GB mapped, and the game took a memory shortage (its frame check at `main+0x1fd32ec` logs `MSF` / `R` / `Rf` and plays the memory-shortage Blood Moon, which puts Link back at the start; GitHub issue #70, 7 of 8 Mac runs from the issue's save). With an 8 GB alias region the mark went on to +2.50 GB; with the cut it stays at +1.75 GB, and neither showed the shortage. + - [BALL x PIT](../../dm/games/010086A022444000/README.md) (SDK 20.5.19): `sdk+0x145d0c` / `+0x145d78`, the SDK 20.5.22 build's offsets. + +### Size-aligned regions exhaust the alias address space + +- **Symptom:** a null-based write minutes into play, at the pool size minus 0x20 (Engage: a write to `NULL + 0x1fffe0`, `addr=0x1fffe0` at `main+0x9a4664`); NULL index pointers written on several job threads at once (`main+0xdb0ad4`, `0xdaea58`, `0xdadb5c`), with 32-bit mesh indices turning up inside texture objects. +- **Cause:** the alias region ran out of *address space*, not memory (the SDK's node area took about 1 GB of it), so new address regions came back empty. Unity's TLSF heap allocator (`main+0x9a45d0`) added a pool (a new 2 MB pool), and its low-level virtual allocator (vtable at `main+0x61ffeb0`; reserve `main+0x9aa0f4` → `nn::os::AllocateAddressRegion`) got no address region and returned 0. Buffers mapped fresh past 2 MB got no mapping: Unity's particle geometry scheduler (`main+0xd91e0c`) takes its vertex and index chunks from a fallback that creates and maps a fresh buffer each time (`main+0x7b9c64`; Unity's Switch renderer always takes it, as on hardware). The AddressRegionManager aligns each region by its size (64 KB under 2 MB, 2 MB under 4 MB, 4 MB under 32 MB, 32 MB under 1 GB, else 1 GB) and Unity asks for its block plus alignment slack, so a 2.25 MB block takes a 4 MB slot and a 16.25 MB block a 20 MB one. At the crash Engage held 4.17 of the 4.375 GB left after the node area (StandardAllocator 3.14 GB, Unity about 1.05 GB, the rest fragments). On hardware the alias region is 64 GB. +- **Fix:** shrink the node area with a rewriter cut for the game's SDK build (`dm/tools/sdk_patches.py`, Engage: the two `mov w8/w9, #0x3fe00000` in the constructor → 62 MB; [The SDK's address-region node area](#the-sdks-address-region-node-area)). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): the prologue battle crash (scripted run, about 357 s; a 12-minute scripted run now plays through the battle) and the idle attract demo crash after about 220 s (from about 180 s 32-bit mesh indices also turned up inside Unity's texture objects). With the cut an 850 s idle run is clean, with no corrupted texture objects; with the old SDK the same run dies at 159 s in Unity's heap allocator. +- **Check:** probe the SDK's `VammManager::AllocateAddressRegion` at its success path and `FreeAddressRegion` after its prologue, and sum the live regions by caller ([Measuring address regions and commits](#measuring-address-regions-and-commits)). +- **Note:** to reproduce the old state: `boot.sh <log> 245` with no input on an SDK rewritten without the cut. + +### 1 GB alignment of regions of 1 GB and more + +- **Symptom:** a cut engine region still does not fit; two large regions lose up to 1 GB each to alignment, depending on where `mmap` put the guest block (Absolum); the region comes 0.69 GB past the 1 GB node area, at the next 1 GB boundary (Paper Mario: The Thousand-Year Door). +- **Cause:** `AllocateAddressRegion` aligns a region of 1 GB or more to 1 GB in absolute address (SDK 18: `sdk+0x13c03c`; 32 MB from 32 MB up). +- **Fix:** a cut on that `sdk` build aligns them to 32 MB as well, beside the node-area cut (`sdk_patches.py`). +- **Found on:** + - [Absolum](../../dm/games/0100A83022A22000/README.md) (SDK 18.3.2: `sdk+0x13c03c`): the two large regions, with a 4 KB one between them, lost up to 1 GB each. + - [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (SDK 7.3.2): 32 MB alignment for large regions on that SDK build. + - [Advance Wars 1+2: Re-Boot Camp](../../dm/games/0100300012F2A000/README.md) (SDK 10.4.1): `AllocateAddressRegion` (`sdk+0x341030`) aligns a region of 1 GB or more to 1 GB (`+0x34109c`), as DRAGON QUEST XI S's SDK 7.3.2 build, and takes the same 32 MB cut. + - [Super Smash Bros. Ultimate](../../dm/games/01006A800016E000/README.md) (SDK 8.2.1): alignment at `sdk+0x336d5c`, cut as DRAGON QUEST XI S's SDK 7.3.2 build. + - [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (SDK 15.3.2, Celeste's `sdk` build): `AllocateAddressRegion`'s alignment at `sdk+0x12e2e0`, as Absolum's did. + - [Pikmin 4](../../dm/games/0100B7C00933A000/README.md) (SDK 10.4.8): the 10.4.1 build's offsets. +- **Note:** FINAL FANTASY RESONANCE's SDK 20.5.17 build aligns regions of 2 MB and more to 2 MB only (`VammManager::AllocateAddressRegion`, `sdk+0x145e00`), so it needs no alignment cut. + +### Unity 6 takes its memory with no StandardAllocator and no heap + +- **Symptom:** nothing aborts at startup as with Unity 2022.3, but the provisional profile gave a 1 GB block and a 256 MB alias region while Unity maps 1.5 GB in the first room. +- **Cause:** `nninitStartup` (`main+0x118cb70`) only calls `QueryMemoryInfo` and `SetMemoryAllocatorForThreadLocal`; Unity's own allocator reserves address regions with `nn::os::AllocateAddressRegion` and commits pages with `AllocateMemoryPages`. `main` imports no `SetMemoryHeapSize`, and the provisional profile took that for a game living in its bss. +- **Fix:** `profile_guess.py` now gives every game with an NPDM system resource the 5 GB block (Absolum and Pokémon Legends: Z-A, heapless too, came out right with it). The node area still sits at the start of the alias region: without the cut the high-water mark was +1.00 GB at `nninitStartup` and +3.00 GB in the first room, with it (`sdk+0x145bfc` / `+0x145c68`) +2.26 GB. +- **Found on:** [Hollow Knight: Silksong](../../dm/games/010013C00E930000/README.md) (Unity 6000.0.50f1, SDK 20.5.8); [BALL x PIT](../../dm/games/010086A022444000/README.md) (Unity 6000.0.62f1, SDK 20.5.19) is the same: the provisional profile was right as generated, and its `sdk` build only needed the same cut (`sdk+0x145d0c` / `+0x145d78`, the SDK 20.5.22 build's offsets): +2.00 GB in the base. Also [Absolum](../../dm/games/0100A83022A22000/README.md) and [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (the 5 GB rule). + +### The movie player's own StandardAllocator + +- **Symptom:** `nn::diag` abort from `StandardAllocator::Initialize` long after `nninitStartup`, right after the first Vulkan query pool (`sdk+0x12dad0` ← `main+0x62c52c`). +- **Cause:** the movie player initializes its own allocator with virtual address memory, `Initialize(nullptr, 384 MB)` (`main+0x62c280`); Super Mario Galaxy, on the same SDK, allocates that block from its main allocator first. The guessed 256 MB alias region holds no region after the 1 GB node area. +- **Fix:** the same node area cut for that `sdk` build (`sdk+0x13be4c` / `+0x13bee0`) and a 512 MB alias region: the region is 32 MB aligned, so it starts 64 MB in. +- **Found on:** [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md) (SDK 18.3.5); compare [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Measuring address regions and commits + +- **What a game really needs:** boot it once on the Mac with a 64 GB alias region (`mem_reserve 0x1030000000`, `mem_alias 0x1008000000`) with `DM_SVC=1`'s map log (Absolum, Super Smash Bros. Ultimate, Paper Mario: The Thousand-Year Door were sized this way). +- **Regions:** probe the SDK's `VammManager::AllocateAddressRegion` at its success path (`ReleaseWriteLock` return: x21 = size, x22 = address) and `FreeAddressRegion` after its prologue (x1 = address), and sum the live regions by caller. Offsets seen: Ace Attorney Investigations Collection's SDK 17.5.3 build `sdk+0x1395a8` (success path) and `sdk+0x139df0` (`FreeAddressRegion`); Absolum's SDK 18.3.2 build `sdk+0x13bf8c` (success path). +- **Commits:** probe `VammManager::AllocateMemoryPages` / `FreeMemoryPages` (x1 address, x2 size) and split them by region. On DRAGON QUEST VII Reimagined, `nn::os::AllocateMemoryPages` was probed past its `adrp` (`sdk+0x140f84`). +- **Windows:** `DM_ALIAS_WINDOWS=<window MB>` (for example `DM_ALIAS_WINDOWS=512`) logs, every 30 s, the mapped memory of each window of that size in the alias region (`hle_svc.c`); it measured Octopath Traveler 0's allocator pools and Super Smash Bros. Ultimate's battle commits. +- **Log:** the `[mem]` lines report the alias high-water mark (one line per 256 MB), the total actually mapped (page bitmap) and every heap growth. + +## Engine allocators + +### StandardAllocators over 12 to 62 GB of address space + +- **Symptom:** `StandardAllocator::Initialize` aborts in `nninitStartup` (Pokémon Legends: Z-A: `main+0x1b2e690`); `nn::diag` abort at `sdk+0x11ff34` (`StandardAllocator::Initialize`) in `nninitStartup`, SDK 15.3.2, called from `main+0x31370c` (Paper Mario: The Thousand-Year Door). +- **Cause:** with virtual address memory the game initializes a StandardAllocator over more address space than a phone gives a process, as the Switch's 64 GB alias region allows. + - Pokémon Legends: Z-A (a 16 MB system resource) initializes its allocator over 12 GB, `StandardAllocator::Initialize(nullptr, 0x300000000)` (`main+0x1b3e3ec`). The alias region must hold the SDK's 1 GB node area and that region, 13 GB. + - Paper Mario: The Thousand-Year Door's malloc is a StandardAllocator over 62 GB of address space (`Initialize(nullptr, 0xf80000000, false)`, `main+0x3136e8`), out of which it cuts 256 MB, 16 MB and 1 MB allocators. Booted once with a 64 GB alias region, the region came 0.69 GB past the 1 GB node area, at the next 1 GB boundary: this `sdk` build (Celeste's, which has no virtual address memory) aligns regions of 1 GB and more to 1 GB, as Absolum's did ([1 GB alignment of regions of 1 GB and more](#1-gb-alignment-of-regions-of-1-gb-and-more)). +- **Fix:** a cut on `main` to 4 GB of address space, with the node-area cut on the `sdk` build ([The SDK's address-region node area](#the-sdks-address-region-node-area)) and, where that build aligns large regions to 1 GB, the alignment cut: + - Pokémon Legends: Z-A: two cuts for 2.0.2 (its `patches.py` and `sdk_patches.py`) make it fit the 5 GB block Tears of the Kingdom runs in: the allocator's address space 12 GB → 4 GB (`main+0x1b3e3e4`) and the node area 1 GB → 62 MB (the AddressRegionManager constructor, `sdk+0x1423fc` and `+0x142468`, Engage's cut). SDK 19's AddressRegionManager puts the region right after the node area (+0x3fe2c000 before the cut), not 1 GB aligned as Engage's SDK did, so a 4.25 GB alias region holds both (`mem_alias 0x110000000`). + - Paper Mario: The Thousand-Year Door: Absolum's three cuts on the `sdk` build (the AddressRegionManager constructor `sdk+0x12e0e0`: `+0x12e0f0` / `+0x12e184`; `AllocateAddressRegion`'s alignment at `+0x12e2e0`) and a cut on `main` to 4 GB of address space, as Pokémon Legends: Z-A's, put the region at alias +62 MB to +4.06 GB in Engage's 5 GB block. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (2.0.2, SDK 19.3.11; one boot); [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md): the allocator maps 1.5 GB at boot and 2.0 GB by the arrival in Rogueport. The `sdk` build is [Celeste](../../dm/games/01002B30028F6000/README.md)'s. Absolum's malloc StandardAllocator (40 GB, cut to 2.5 GB) is in [.NET NativeAOT malloc and the GC region range](#net-nativeaot-malloc-and-the-gc-region-range). +- **Note:** on Pokémon Legends: Z-A, `QueryMemoryInfo` reporting more than 4 GB would take the game's big-memory path (heaps of allocatable − 4.27 GB instead of − 2.76 GB). + +### .NET NativeAOT malloc and the GC region range + +- **Symptom:** `nn::diag` abort at `sdk+0x12dad0` in `nninitStartup`, called from `main+0x756310`. +- **Cause:** malloc's `StandardAllocator::Initialize(nullptr, 0xa00000000)`: 40 GB of address space (`main+0x28bff4`). Absolum's regions (measured on a 64 GB alias region): malloc's (2.19 GB used into the first level, 1.75 GB of it FNA3D's GPU heap, allocated whole at `FNA3D_CreateDevice`), the .NET GC's region range (3.75 GB) and a few small ones. The GC sizes its range as 5 × its hard limit (768 MB = 75% of the 1 GB its `GetPhysicalMemoryLimit`, `main+0x28bffc`, reports), and it reads `DOTNET_GCRegionRange` through `getenv` first. +- **Fix:** the host now answers `getenv` from the profile key `game_env` (`hle_os.c`; every other name goes to the SDK's own `getenv`, an empty environment), which sets 1.5 GB. The GC then uses its range from both ends, 756 MB from the start and 748 MB from the end, where it used only the first 1.14 GB of 3.75. A cut on `main` gives malloc 2.5 GB. One more thing kept it from fitting, the 1 GB alignment of large regions ([1 GB alignment of regions of 1 GB and more](#1-gb-alignment-of-regions-of-1-gb-and-more)), cut beside the node-area cut. The regions then end 4.13 GB into the 4.375 GB alias region of Engage's 5 GB block. +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) (.NET NativeAOT on SDK 18.3.2). +- **Check:** boot once on the Mac with a 64 GB alias region (`mem_reserve 0x1030000000`, `mem_alias 0x1008000000`) and probe `VammManager::AllocateAddressRegion`'s success path (`sdk+0x13bf8c`: x21 size, x22 address) with `DM_SVC=1`'s map log. + +### A 62 GB page allocator behind `nu::VirtualAllocHook` + +- **Symptom:** the game aborts in its static initialisers, before `nnMain`, when its own 62 GB address region fails (`main+0x37c9bd0`, `AllocateAddressRegion(&p, 0xf80000000)` at `main+0x37c9a5c`). +- **Cause:** the allocator behind `nu::VirtualAllocHook` is a 64 KB page allocator whose own tables fill the region's last 14 MB, so the size, the header offset, the page count and three region-end bounds all follow from 62 GB. +- **Fix:** the profile's `patches.py` cuts all 11 instructions to a 4 GB region. The cut region still did not fit: SDK 8 has the same 1 GB node area and 1 GB alignment of regions of 1 GB and more as the newer SDKs (constructor `sdk+0x336b40`, `orr w9, wzr, #0x3fe00000` at `+0x336b50` / `+0x336bf0`; alignment at `+0x336d5c`), cut in `sdk_patches.py` as DRAGON QUEST XI S's SDK 7.3.2 build; the region then starts 64 MB into the 4.375 GB alias region of the 5 GB block. +- **Found on:** [Super Smash Bros. Ultimate](../../dm/games/01006A800016E000/README.md) (13.0.5, SDK 8.2.1). Booted once with a 64 GB alias region (`mem_reserve 0x1030000000`, `mem_alias 0x1008000000`) and `DM_SVC=1`'s map log, the game commits from the region's start upward (1.73 GB contiguous at the title) and its page tables from the top; a battle commits up to 2.5 GB (`DM_ALIAS_WINDOWS`). + +### Unreal's engine address region and its options + +- **Symptom:** the game calls `exit(1)` in `nninitStartup` (Octopath Traveler 0: `main+0x12e74`); `nn::diag::Abort` from `main+0x352fbec` (Unreal's fatal error) after the first commit, `AllocateMemoryPages(0, 2 MB)` while the RHI starts (`main+0x353ff8c`, result 0xe03) (Pikmin 4); a crash in `NumberLineAllocator::Compaction` at boot (DRAGON QUEST XI S, region option 4224). +- **Cause:** Unreal's `nninitStartup` needs virtual address memory and reserves one address region for the engine with `AllocateAddressRegion`, 62 GB by default (`-SwitchVirtualAddressSpaceSize=` in MB, or in bytes above 4095). The other options it reads are `-SwitchMaxMemorySize=`, `-NVNCommandBufferGranularity=`, `-NVNControlBufferSize=`, `-SwitchMainThreadStackSize=`, `-NVNTempMemoryPageSize=`, `-ResX=` / `-ResY=` and `-SwitchDisableVirtualAllocator`, which also exits. It reads them from the game arguments, which the host used to set only after `nninitStartup`. +- **Fix:** the game arguments are set before `nninitStartup` now, as on hardware, and the profile key `game_args` supplies them on iOS. +- **Found on:** + - [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (Unreal Engine 5): with the host allocator ([Unreal's FMallocBinned3 replaced by the host allocator](#unreals-fmallocbinned3-replaced-by-the-host-allocator)) the region is capped at 2 GB; with SDK 19's node area cut the guest block is 3.75 GB. + - [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (Unreal Engine 4.27, SDK 19.3.1): the same 62 GB region from `nninitStartup` (`main+0xe8a2c0`). With `-SwitchVirtualAddressSpaceSize=2048` the region is mapped up to +0.75 GB through the opening and the new-game menus. + - [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) (Unreal Engine 5.6.1, SDK 20.5.17): `nninitStartup` (`main+0x13370`) reserves the same 62 GB region; the profile gives it 4064 MB in Engage's 5 GB block, as DRAGON QUEST XI S's ([Unreal 5.6's page allocator never gives address space back](#unreal-56s-page-allocator-never-gives-address-space-back)). + - [Pikmin 4](../../dm/games/0100B7C00933A000/README.md) (Unreal Engine 4.26.2, 1.1.1, SDK 10.4.8): the region option is the trap. This `nninitStartup` reserves the region only when `-SwitchVirtualAddressSpaceSize=` is absent (`ldr x8,[x22,#0x130]; cbnz x8` past the default 62 GB and its `AllocateAddressRegion`, `main+0x353f564`), so with the argument the base stayed 0 and the first commit went into Unreal's fatal error. The default is cut to 4064 MB instead (two `orr #0xf80000000` → `mov #0xfe000000`, its `patches.py`) and the profile has no `game_args`. With Engage's 5 GB block, from the boot into the Olimar prologue: up to +1.5 GB of the alias region with 1.0 GB mapped, an Unreal heap of 1 GB. + - [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (Unreal Engine 4.18, SDK 7.3.2) reads the same option and the others from `argv[1]` on (`main+0x177b5a4`; `main+0x177c440` parses each one: `atoi`, MB up to 4095, bytes above). Its region is 16 GB by default. A value of 4224 was taken as bytes: a 4 KB region, and the first 2 MB request crashed in `NumberLineAllocator::Compaction` at boot. With the node area cut and 32 MB alignment for large regions on that SDK build (`sdk_patches.py`), a 4064 MB region starts 64 MB into Engage's 4.375 GB alias region. On the Mac the mapped total stays at 1.5 GB from the title to the first field (Cobblestone Tor), used up to +1.75 GB of the alias region; the process holds about 2.1 GB, 1.1 GB of it Metal allocations. + +### Unreal's FMallocBinned3 replaced by the host allocator + +- **Symptom:** a null read in the out-of-memory report at `main+0x1005824` when the engine region is smaller than the allocator's pools (Octopath Traveler 0). +- **Cause:** Unreal's allocator, an FMallocBinned3, reserves 37 GB of the engine region (74 pools of 512 MB, pool index = offset >> 29). The pools are uneven (at character creation the largest holds 68 MB, the 74 together 732 MB; measured with `DM_ALIAS_WINDOWS=512`), so no smaller window fits a phone's ~5 GB. +- **Fix:** the constructor call is cut (the profile's `patches.py`) and the host puts its own allocator in the object (`hle_uemalloc.c`: the FMalloc vtable on a host malloc zone, the trivial slots kept). With it the region holds only what the engine maps itself (709 MB at Octopath Traveler 0's character creation). +- **Found on:** + - [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (Unreal Engine 5): the allocator is constructed in place in the bss. + - [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (Unreal Engine 4.27, SDK 19.3.1): an FMallocBinned3 whose 37 GB of pools are cut out of the 62 GB region (the mapped high-water mark reached +38 GB of a 64 GB alias region at boot), so a capped region would starve it. Here the allocator is not in the bss: `GCreateMalloc` (`main+0x32758b0`) mallocs 0x1c38 bytes on the first allocation, during the static constructors, and constructs it there. Found from the commit calls (probe `nn::os::AllocateMemoryPages` past its `adrp`, `sdk+0x140f84`: every backtrace runs through the allocator's methods around `main+0x3270000`), the relative relocations that point at them (the vtable `main+0x5e4c0e8`; this build has no RTTI) and the one store of that vtable outside the destructors. The constructor call is cut and an observation gate right after it (`OBS_UE_MALLOC`, x19 = the new object) puts the host allocator's vtable in (`hle_uemalloc.c`); no code outside the allocator reads its pool base or calls its methods directly. UE 4.27's FMalloc has one FExec slot, not four, so the slot numbers differ from UE5's (Malloc 3 … GetAllocationSize 9; `Free` is `main+0x10`, right after the 16-byte module header). + - [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) (Unreal Engine 5.6.1, SDK 20.5.17): `nninitStartup` constructs its FMallocBinned3 in place in the bss, as Octopath Traveler 0's (`main+0x77c6210`; the call at `main+0x137f0` is cut, and its constructor would reserve 38 GB). UE 5.5 and later add `MallocZeroed` / `TryMallocZeroed` after `Free`, two TLS cache calls (`MarkTLSCachesAsUsed` / `Unused`) and two cached-memory queries, so the FMalloc vtable has 32 slots (layout `UE5_6` in `hle_uemalloc.c`: Malloc 6 … Free 10, QuantizeSize 13, GetAllocationSize 14). The base vtable shows it: its three pure slots are 0, and its slots 11 and 12 are one function that calls slot 6, then `memset`. + - [Pikmin 4](../../dm/games/0100B7C00933A000/README.md) (Unreal Engine 4.26.2, 1.1.1, SDK 10.4.8): the allocator is made as in 4.27, with the same 21-slot FMalloc vtable (`GCreateMalloc` `main+0x353f770` mallocs 0x1c38 bytes and calls the constructor `main+0x35ba610`, the one store of the vtable `main+0x8da4758` outside its two destructors; found from the FMallocBinned3 error strings through the relocated slots). The cut and the gate are as DRAGON QUEST VII Reimagined's, the gate on the `ldp x29, x30` after the cut call (the `ret` case of [Observation gates where x9–x17 are dead](rewriter.md#observation-gates-where-x9x17-are-dead)). +- **Note:** Unreal Engine 4.18 (DRAGON QUEST XI S) needs no allocator cut: the SDK's `NumberLineAllocator` hands its region out in 64 KB units (`main+0x177c4d0`), and the engine maps only what it uses. + +### Unreal 5.6's page allocator never gives address space back + +- **Symptom:** `nn::diag` abort from `main+0x11bc`, no message, in the first battle, with about 1 GB mapped (FINAL FANTASY RESONANCE with DRAGON QUEST VII Reimagined's 2 GB region). +- **Cause:** the engine hands its region out with a page allocator of its own (`main+0x13ef0`), a bump pointer plus free lists per size class, so the address space it has used only grows, and past the region's end it calls `OnOutOfMemory` (`main+0x13c74`). +- **Fix:** the profile gives the region 4064 MB in Engage's 5 GB block, as DRAGON QUEST XI S's (profile key `game_args`). This SDK build aligns regions of 2 MB and more to 2 MB only (`VammManager::AllocateAddressRegion`, `sdk+0x145e00`), so it needs no alignment cut. +- **Found on:** [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) (Unreal Engine 5.6.1, SDK 20.5.17). Probed where each new block comes back (`DM_PROBE=main:13fcc`: x0 the block, x21 its size, x8 the region's end): a 1 GB block at boot from a static constructor, +1.5 GB 15 s in, +2.0 GB in the battle, level at +2.03 GB over seven minutes. + +### Ring heaps cut from a pool at 64 KB alignment + +- **Symptom:** write to `0x268` right after the title screen (`main+0x5d6708`, a constructor whose `this` is `0x200`). +- **Cause:** `operator new` (`main+0x5a55d4`) takes the thread's ring heap, a bump allocator whose cursor starts at its base. One ring heap kept base 0, as in 13 Sentinels: the engine cuts ring heaps out of a memory pool at 64 KB GPU alignment, and the host's pool storage from malloc is only 4 KB aligned. The first allocation returned 0, which `operator new`'s null check skipped, and the next returned `0x200`. +- **Fix:** the profile key `alloc_big_align` aligns allocations of 64 KB or more to 64 KB ([dm/games/README.md](../../dm/games/README.md)); the game then plays from the title screen into its opening story scene. +- **Found on:** [Unicorn Overlord](../../dm/games/010069401ADB8000/README.md) (SDK 16.2.4, Vanillaware's engine as in [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md); one boot). + +## Mapping SVCs + +### `svcQueryMemory` must report each module's real layout + +- **Symptom:** C++ exception aborts in `nn::ro::detail::GetExceptionInfo`. +- **Cause:** the SDK's unwinder finds a module from `svcQueryMemory`: it walks back over Code+RX regions to the module start, forward over Code then CodeData, and reads `MOD0` for `.eh_frame_hdr`. +- **Fix:** the host reports each loaded module's real layout and reports other host memory as free (`hle_svc.c`). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) (its note lists it among the host changes first needed there). IL2CPP throws a C++ exception for every managed exception, so Unity games need this; Mario ([Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)) never threw. + +### `MapPhysicalMemory` in 16 KB host pages + +- **Symptom:** `MapPhysicalMemory` faults on part of a page. +- **Fix:** round to 16 KB host pages. +- **Found on:** the source names no game. + +### Maps past the application pool fail as on Horizon + +- **Symptom:** the alias region's mapped total runs past the application pool in the first minute. +- **Cause:** `nninitStartup` allocates and frees everything allocatable (pool − used, 3.15 GB, mapped at the top of the region) and the SDK keeps the freed pages mapped; later allocations map new pages at the bottom until a map fails, and only then does it unmap the cached block and retry. The host never refused a map, so the block stayed mapped beside the 0.4 GB the game used. +- **Fix:** profile key `mem_map_limit` (`hle_svc.c`): a map past the pool fails with ResultLimitReached, and the used memory `GetInfo` reports counts the mapped pages, as on Horizon (without that, the startup probe itself, sized as total − used, was refused). +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (2.0.2, SDK 19.3.11; one boot). Measured: one refused 4 MB map, the 3.15 GB unmap, the same map again. + +### `GetInfo(UserExceptionContextAddress)` returns a real buffer + +- **Symptom:** a write through a null pointer at +0x1c0 during startup. +- **Fix:** `GetInfo(UserExceptionContextAddress)` must return a real buffer. +- **Found on:** the source names no game. + +## Running out of memory + +### A null-pool buffer from a game heap that ran out + +- **Symptom:** a null-pool buffer even with virtual pools working ([Virtual memory pools mapped onto one physical pool](nvn-objects.md#virtual-memory-pools-mapped-onto-one-physical-pool)). +- **Cause:** the game's own heap ran out. +- **Tracing it with probes** ([05](../agents/05-debugging.md#probes-without-a-debugger-dm_probe)): the gsys GPU allocator at `main+0x4290c0` and the heap's `tryAlloc` (vtable slot +0x30) show which heap, its size and every allocation into it; the `!xN` heap walk shows the free and used blocks at the moment of failure. +- **Note:** read the probe log carefully: `tryAlloc` probes show *requests*, not successes, so a running total that already exceeds the heap means an earlier request failed. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md), [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (the two entries below). + +### Per-resource heaps sized from Nintendo's measured peaks + +- **Symptom:** a null-pool buffer for `Model/NpcCommonerA.bfres` in the opening course. +- **Cause:** each resource gets its own `sead::ExpHeap` sized *resource-size-table value + 0x1008* (`main+0x1e603c` → `ExpHeap::create` `main+0x4aac38`). The table values are Nintendo's measured peaks with almost no margin (Yoshi's animation file uses 99 % of its budget). `Model/NpcCommonerA.bfres` came up about 0x70 bytes short: its last GPU chunk (0x9000 bytes, 4 KB aligned) fit, but the chunk's 0x598-byte descriptor did not. +- **Fix:** a registered code cut gives every resource heap +16 KB (`CUTS` in the profile's `patches.py`, keyed by the module build ID, original bytes verified). Per-title code patches go there, with the reason; those of Nintendo's `sdk` modules in `dm/tools/sdk_patches.py`. +- **Note:** Breath of the Wild's identical failure came from sizes the game takes from the host (next entry); whether Mario's chunk is sized the same way is not checked yet. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Model heaps sized by the host's command record sizes + +- **Symptom:** `main+0x155a7a4` writes through a null `operator new` at the world load, about 1 run in 3. +- **Cause:** each model's heap is its table entry + 0x1008, the file goes in 4 KB aligned, and the setup (`main+0x155a2ac`) takes the rest as a frame heap (vtable `main+0x1d4f950`, head `+0xe0`, tail `+0xe8`). Its use is the same in every run; only the file's alignment pad varies (0 to 0xff8), and the entries leave room for any pad, about 0xd00 to spare on most models. Three models were a 4 KB page short, so the loads with a large pad failed. The game sizes the model's prerecorded commands by recording each mesh's draw and each shape's per-pass state into a scratch command buffer and summing `GetCommandMemoryUsed` (`main+0xf9aeb4`, `main+0xf9bb54`), and our records were larger than the driver's (draws carried the indirect and multi-draw fields, every recording began with a BEGIN record). +- **Fix:** plain draws now leave those fields out and recordings start with their first command (`hle_nvn.c`; [What `GetCommandMemoryUsed` counts](nvn-objects.md#what-getcommandmemoryused-counts)). +- **Check:** to check a whole scene, probe where the setup heap is cut (`main+0x155a520` with `!x21`, the resource heap) and the end of the setup (`main+0x155a5f4` with `!x2`, the frame heap), with `DM_PROBE_HEAP=28,38,40,e0,f8 DM_PROBE_HEAPFRAME=1d4f950 DM_PROBE_HEAPWORDS=1` ([05](../agents/05-debugging.md#probes-without-a-debugger-dm_probe)). A model fits any placement when entry + 0x1008 covers the resource heap's header and object (0x310), the file, the frame heap's block header and its peak use, plus 0xff8 of pad. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md). + +### Unity's GPU pool block allocation spins at size 0 + +- **Symptom:** the frames stop at the first loading screen, a graphics worker at 100% CPU (17 s in, `UnityGfxDeviceWorker` running, the main thread waiting on a condition variable, no red line). +- **Cause:** the heap check's 1 GB pool (the hardware pool had used the alias region up to +4.25 GB) left the game's malloc too little: Unity's GPU memory pool block allocation (`main+0x96f430`) calls `aligned_alloc` through `main+0x866810`, halves the request after each failure, and once the request rounds to 0 it calls on for ever with a size of 0. +- **Fix:** a 2 GB pool: the game then maps up to 2.26 GB of the alias region through its menus and first battle. +- **Found on:** [Advance Wars 1+2: Re-Boot Camp](../../dm/games/0100300012F2A000/README.md) (Unity 2020.2.1f3, SDK 10.4.1), after the node-area and alignment cuts. +- **Check:** `sample <pid>` on the Mac host shows such a spin (the guest frames are `???` in the host's image list; subtract the module base from the log's `base` lines); it worked here where lldb does not. + +### Host calls on small guest stacks + +- **Symptom:** SIGBUS inside a host hook (`snprintf` under `mount_ex` / `h_fs_MountSave`), then the process hangs (Super Mario 3D World + Bowser's Fury); SIGBUS inside Metal (`AGCDeserializedReply`, an AGX compiler block) at an address in the guest stack region (Breath of the Wild). +- **Cause:** host hooks run on the calling guest thread's stack. Super Mario 3D World's save thread has an 8 KB stack (CreateThread stack top `0x…9fe0`, fault `0x21a0` below it), and the file system hooks size their path buffers by PATH_MAX: one file open takes 6–7 KB (`-fstack-usage`: `OpenFile` 1.3 KB → `host_path_ex` 1.9 KB → `path_component` 3.2 KB); on hardware the same call is one IPC. In Breath of the Wild a host NVN call did Metal work on a game thread with a small stack: `nvnQueueSubmitCommands` replays the frame, and ending a render pass made the driver compile its fast-clear blit program on the calling thread. +- **Fix:** every hook in `hle_fs.c` now registers through `HLE_FORCE_BIG` (`hle.h`): a compile-time stub loads the implementation into x16 and jumps to `dm_bigstack_enter` (`hle_bigstack.c`), which moves the call to a per-thread 512 KB host stack (with a guard page) when sp lies in the guest address block, passing x0–x8 and d0–d7 through. Only for functions with register arguments only (no variadic or stack arguments). iOS forbids generating code at run time, so the stubs are static. NVN functions are not registered through `HLE_FORCE`, so they got their own big-stack entry: every implemented NVN function except the `nvnCommandBuffer*` recording calls goes through `nvn_big_entry` (`nvn_entry.S`), which finds the implementation by slot and hands it to `dm_bigstack_enter` ([NVN entry points on the big host stack](nvn-objects.md#nvn-entry-points-on-the-big-host-stack)). +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (file system hooks); [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (NVN entry points). + +### Deferred-release drain without an autorelease pool + +- **Symptom:** the iPhone app is killed for memory; resident memory grows steadily (about 0.2 MB a frame) while the residency set and the texture counts stay flat. +- **Cause:** the deferred-release drain ran on the guest submit thread with no autorelease pool, so autoreleased Metal objects were never freed (the Metroid Dread leak). +- **Fix:** `defer_drain` has its own `@autoreleasepool`. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): GitHub issue #9 (iPhone 16 Pro Max, handheld), killed in the first shrine. An 8 GB iPhone gives this app about 3.4 GB (resident plus `headroom to the system memory limit` stayed at 3,375 MB); resident memory grew from 2.6 GB at the shrine load until the kill. On the Mac a host without that pool grew 1 MB a frame in the shrine (GPU 3.0 → 4.4 GB in 2,400 frames); with it the shrine stays at 2.7–2.9 GB resident and 1.5–1.7 GB GPU, idle or walking (vmmap at 2.8 GB: about 1.6 GB guest memory, 0.75 GB private textures, 0.17 GB IOAccelerator). +- **Note:** found on the way: dead texture serials and the fixed-size pool tracking table ([Dead texture serials kept in a ghost table](nvn-objects.md#dead-texture-serials-kept-in-a-ghost-table), [Memory-pool tracking table grows in blocks](nvn-objects.md#memory-pool-tracking-table-grows-in-blocks)). diff --git a/docs/reference/movies.md b/docs/reference/movies.md new file mode 100644 index 0000000..1dc2438 --- /dev/null +++ b/docs/reference/movies.md @@ -0,0 +1,188 @@ +# Movies + +The SDK's movie library (`movie::Decoder`, `movie::Extractor`), which the host takes over in `dm/host/hle_movie.c`, and the games' movie players. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Decoder lifecycle + +### The SDK's hardware decoder opens nvdrv + +- **Symptom:** abort in NVIDIA's video path (`NvRmChannelOpen` → `NvOsDrvOpen`). +- **Cause:** the SDK movie library's hardware decoder opens nvdrv directly. +- **Fix:** the host takes over `movie::Decoder` (`hle_movie.c`, [`movie::Decoder` taken over as a MediaCodec-style protocol](#moviedecoder-taken-over-as-a-mediacodec-style-protocol)); nvdrv opens fail (`hle_nvdrv.c`). + +### `movie::Decoder` taken over as a MediaCodec-style protocol + +- **Symptom:** the game waits forever at a movie (Engage's 94 s opening `S01F.webm`; frames keep presenting, nothing happens). +- **Cause:** Unity demuxes the WebM itself and feeds VP9 packets to `movie::Decoder`; with no working decoder the movie never ends. +- **Fix:** `movie::Decoder` is a thin wrapper over Android's MediaCodec, and its vtable entries are symbol relocations, so `HLE_FORCE` on each method also catches Unity's virtual calls. The host implements the protocol: four `nn::os` events (`DecoderEvents` = input available, output ready, format changed, error), input indices (`AcquireInputBufferIndex`, none = 0x1011; `GetInputBuffer` fills the client's `movie::Buffer`; `SendInputBufferForDecode(index, offset, size, µs, flags)`), output indices (`AcquireOutputBufferIndex`, none = 0x100e; one NV12 frame per packet copied into the buffer Unity allocated, w × h × 1.5; `ReleaseOutputBufferIndex`). `Buffer`, `MediaData` and events are handled through the SDK's own exported functions (`SetDataAndCapacity`, `SetRange`, `SetInt32`, `SetRect`, `SignalEvent`). Unity reads `width`, `height`, `crop`, `nv12-y-stride`, `nv12-y-offset`, `nv12-uv-offset` and `slice-height`. The guest thread that sends a packet or returns an output index waits for the decode and signals the events itself, because `SignalEvent` is guest code and cannot run on a host thread. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): when the movie ends the game continues to its title screen. +- **Note:** output slots, VP9 decoding and the output format keys are in their own entries ([Packets wait while every output slot is taken](#packets-wait-while-every-output-slot-is-taken), [VP9 on host threads with libvpx](#vp9-on-host-threads-with-libvpx), [Output format keys (`md_format`)](#output-format-keys-md_format)). + +### Output format keys (`md_format`) + +- **Symptom:** a movie in grey with vertical bands of colour, its luma sharp; or a read far outside guest memory at `main+0x344f74` on the first movie picture (CRI Sofdec2's NV12 → RGB conversion). +- **Cause:** the host never set `nv12-uv-stride` in the output format (Unity reads `nv12-uv-offset` only). CRIWARE's Sofdec hands its VP9 video to `movie::Decoder` in mode 1, NV12 into its own buffers, and draws the planes with its shaders; it reads `nv12-uv-stride` from the output format, and the bands were the top row's chroma: every chroma row was read from the first one. Catherine's Sofdec2 sends its H.264 through `movie::Decoder`, mode 1, output format 4; the player reads `nv12-y-stride`, `nv12-y-offset`, `nv12-uv-stride` and `nv12-uv-offset` from the output format and checks none of the results (`main+0x34e694`), so the chroma stride was stack garbage. +- **Fix:** the host now sets everything the SDK's `DecoderImpl::GetOutputBufferFormatFromDecoder` (SDK 19 `subsdk0+0x121f50`) does: also `nv12-uv-stride`, the luma (Y) stride as for any NV12 picture, `nv12-width` and `nv12-height` (`md_format` in `hle_movie.c`). +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (the grey picture); [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md) (SDK 8.2.1, CRI middleware; one boot: the crash). + +### SDK 4's `movie::Decoder` constructor without a priority + +- **Symptom:** a call through a null pointer in the host's `movie::Decoder::Configure` right after `nnMain` starts the logos. +- **Cause:** SDK 4's `Decoder` constructor has no priority argument (`_ZN5movie7DecoderC1ENS_11DecoderTypeENS_19DecoderOutputFormatENS_11DecoderModeE`). The host took over only the four-argument one, so the SDK built its own decoder and the host's `Configure` read that object as its own. +- **Fix:** both forms are taken over now (`hle_movie.c`). +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (SDK 4.5.0; one boot). + +### The complete-object destructor + +- **Cause:** the game calls the complete-object destructor `movie::Decoder::~Decoder` (`D1`); the SDK's own one destroyed the host object as a `DecoderImpl`. +- **Fix:** the host now takes it over too. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). + +### Client events read through the registered pointer + +- **Symptom:** the first deferred signal crashed in `InternalCriticalSectionImplByHorizon::Enter` (DRAGON QUEST XI S). +- **Cause:** `RegisterClientEvents` copied the four event pointers, because Dread's struct had been overwritten by its second movie. DRAGON QUEST XI S (Unreal's Switch movie player, VP9 with 6-channel Vorbis) does the opposite: it registers its struct right after constructing the decoder (`main+0x39edab0`, through the decoder's vtable) and fills it later, so the copy held leftovers (`0xe00000010`). The SDK keeps only the pointer (`DecoderImpl::RegisterClientEvents` stores it at +0xd0) and reads the entry each time it signals, skipping a null one (`SendOutputBufferEventAvailableToClient` and the others, `subsdk0+0x1feb0`). +- **Fix:** the host now does the same (`md_event` in `hle_movie.c`). When the client's entry is not a readable, 8-byte aligned pointer, it falls back to the entry's value at registration if that was one (Dread's case), and otherwise logs a red line. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md). + +### `Flush`, `Stop` and `Start` + +- **Fix:** as with MediaCodec, `Flush` announces nothing, and no input or output index is handed out while the decoder is stopped. The "input available" after `Start` is signalled about 2 ms later from a guest thread, as the SDK's codec thread would. Pending signals are dropped at `Stop`, `Flush` and destruction. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md): the game ends a movie with `Flush`, `Flush`, `Stop`, destroy, and calls `Start` after a `Flush` when it loops. + +### Input indices returned by a flush while a client holds them + +- **Symptom:** the process dies when an activity starts, 2 runs in 5, with no `CRASH` line: the run ends with exit 137 (SIGKILL), and `~/Library/Logs/DiagnosticReports/dmhost-*.ips` has `EXC_BREAKPOINT` in `_os_unfair_lock_corruption_abort` ← `pthread_mutex_destroy` ← `md_free` (the movie decoder's destructor). +- **Cause:** each activity's preview is an H.264 movie (`movie/Movie/*.mp4`) the game loops with Flush and Start, from four threads at once (`DM_MOVIE_TRACE=1` logs every decoder call with its thread). Its event thread acquired an input index while the control thread's Flush waited for the packet being decoded; the flush then returned that index to the free list, it was handed out a second time, two packets shared one buffer (VideoToolbox -12909 on the overwritten one, in exactly the runs that died), and a fifth pending packet wrote past `pend[]` into the decoder's `npend` and `dmu`. +- **Fix** (`hle_movie.c`): a send now claims an index a flush returned, and `pend[]` cannot overflow. Handing such an index out last was not enough (with three hosts running at once one run still sent an index twice): an index the client held across a flush is now withheld for 250 ms unless the client sends it first, and an acquire refused only for that announces input when the first one comes back, so a client that abandons its indices, as MediaCodec's rules expect, loses none. +- **Found on:** [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3, its activity previews). + +### End of stream is the last input until `Flush` or `Stop` + +- **Symptom:** when the prologue movie was skipped, 2.25 GB were mapped within seconds, past the application pool, and the game crashed in its allocator (DRAGON QUEST XI S). +- **Cause:** DRAGON QUEST XI S's movie player sends an end-of-stream packet (length 0, time 0) for every input it gets once its extractor is done, until the game stops the movie. Each one became an end-of-stream output that the player turned into an audio sample. +- **Fix:** also as with MediaCodec, an input marked end-of-stream is the last until `Flush` / `Stop`: no input index is handed out or announced after it (`in_eos` in `hle_movie.c`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md) (movie sub-list); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md). + +### End-of-stream packets with no data, and deleted decoders + +- **Symptom:** decoding the end-of-stream packet gave VideoToolbox −12909 and AudioToolbox −50. +- **Cause:** the game ends a stream with time −1 and the whole 4 MB input buffer as its length: that packet carries no data. It also deletes its decoders (`D0`). +- **Fix:** for `D0` the host frees the object with the game's `operator delete` after its own state. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md). + +### H.264 drain at the end of a stream goes on with released slots + +- **Symptom:** the game never leaves a movie that has played to its end; the last picture stays on screen. +- **Cause:** `DM_MOVIE_TRACE=1`: the end-of-stream packet came with H.264 pictures still in the reorder queue, its output went out as a black picture without the end-of-stream flag, and the queued pictures were to follow one per free output slot (the drain). Unity held all 16 video slots by then. `ReleaseOutputBufferIndex` decoded again only when packets were queued, and none were, so the drain never went on: no flag, and the game waited for the movie's end for good. +- **Fix:** a released slot now goes on with a drain too, and the end-of-stream packet's own output takes the first queued picture (`jflush`), so no black picture shows before a movie's last frames (`hle_movie.c`). +- **Found on:** [AI: THE SOMNIUM FILES](../../dm/games/010089B00D09C000/README.md) (Unity 2017.4: the 6 s Spike Chunsoft logo `StreamingAssets/Movie/al_titel_cl_sc_1k_30.mp4`, H.264 High, before the title). +- **Note:** the audio decoder was never affected: AAC has no reorder queue. + +## Output slots + +### Output slot counts per decoder kind + +A decoder hands out as many outputs as it has slots, and while every slot is taken it takes no input ([Packets wait while every output slot is taken](#packets-wait-while-every-output-slot-is-taken)). The counts now (`hle_movie.c`): `MD_NOUT` 64 for a Vorbis decoder (the most any decoder has), `MD_NOUT_AAC` 32 for AAC and `MD_NOUT_VIDEO` 16 for video; a slot is allocated when first used. Every decoder had 8 at first. The entries below raised them one game at a time, and each gives the counts of its time: audio 32 ([Audio decoders keep more outputs than a pre-decoding player fills](#audio-decoders-keep-more-outputs-than-a-pre-decoding-player-fills)), audio 64 ([Audio output slots for players that hold every output](#audio-output-slots-for-players-that-hold-every-output)), video 16 ([Video output slots: how far the extractor gets ahead](#video-output-slots-how-far-the-extractor-gets-ahead)) and AAC 32 ([AAC decoders offer at most 32 outputs](#aac-decoders-offer-at-most-32-outputs)). + +### Packets wait while every output slot is taken + +- **Fix:** while every output slot is taken, packets wait and their input indices are not returned, as with MediaCodec; dropping them instead lost frames. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md). + +### Audio decoders keep more outputs than a pre-decoding player fills + +- **Symptom:** the opening movie never starts, in about half the boots (DRAGON QUEST XI S: a black screen and no sound from boot on). +- **Cause:** Square Enix's `UMediaPlayer` has two natives of its own, `StartPreDecoding` and `IsPreDecodingCompleted`. The opening's Blueprint polls the second once a frame and only then plays its Matinee, whose event key calls `Play`. Its Switch player (vtable `main+0x77921b8`, `IsPreDecodingCompleted` at `main+0x42d4d20`) is complete when every selected track has at least one decoded output queued. The player starts its clock and its decoders on the first `TickInput` after `StartPreDecoding`. The render thread's first fetch (`main+0x39f07d0`, the `+0x5c7` flag) takes every decoded picture and shows the newest. The host decoded all eight video outputs (to 236 ms) within 36 ms. Each Vorbis decoder's 8 outputs were full by then, because nothing consumes audio before `Play`, so the extractor waited for an audio input and no picture followed the ones the render thread took. The movie started only when the game thread's poll came between the first decoded picture and that fetch. +- **Fix:** audio decoders then kept 32 outputs (`MD_NOUT`) and video decoders 8, so video keeps flowing after the first fetch. Since then audio decoders have 64 (Monster Train 2), AAC ones 32 (`MD_NOUT_AAC`, Mario vs. Donkey Kong) and video ones 16 (`MD_NOUT_VIDEO`, Animal Crossing: New Horizons); see [Output slot counts per decoder kind](#output-slot-counts-per-decoder-kind). +- **Found on:** [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md): 8 boots out of 8 start; [Monster Train 2](../../dm/games/010051701FB46000/README.md) (64). +- **Note:** to find such a gate: probe the player's `SetRate` (`DM_PROBE=main:<off>`) to see whether the game ever asks to play, then follow the Blueprint native that calls it (the `FNameNativePtrPair` tables give the names). + +### Audio output slots for players that hold every output + +- **Symptom:** a movie stays black after its first picture, the sound never starts, and the game keeps presenting frames. +- **Cause:** the player takes every output a decoder offers and starts playing only once it holds about half a second of audio. The host then gave each decoder 8 output slots (before DRAGON QUEST XI S's 32 for audio) and takes input only while one is free (as MediaCodec does). `DM_MOVIE_TRACE=1` (each line now names the decoder): Unity held all 8 audio outputs, the audio decoder kept the next 4 packets and handed out no input index, the video packets behind them in the file never came, and the video decoder had given out one picture (the host releases H.264 pictures once more than 4 are queued). +- **Fix:** slot counts per decoder kind, measured: 8 audio / 8 video and 8 audio / 32 video hang; 64 audio / 8 video plays, starting with 27 audio and 8 video outputs held (peak 30 audio). 32 audio slots hold that peak with two to spare, so audio decoders got 64 (`MD_NOUT`; a PCM slot is 128 KB, allocated when first used), and video kept 8 (`MD_NOUT_VIDEO`: a 1080p picture is 3 MB, 8 MB as RGBA). +- **Found on:** [Monster Train 2](../../dm/games/010051701FB46000/README.md) (its 90 s intro, `StreamingAssets/intro_fmv.mp4`: H.264 High with 2 reorder frames and AAC interleaved packet by packet, played by Unity 2022.3's movie player: `movie::Extractor` in `subsdk0` feeds both decoders from one thread in file order). The game's A, B and + do not skip this intro. +- **Note:** later changes: video decoders have 16 slots ([Video output slots: how far the extractor gets ahead](#video-output-slots-how-far-the-extractor-gets-ahead)) and AAC decoders 32 ([AAC decoders offer at most 32 outputs](#aac-decoders-offer-at-most-32-outputs)); Vorbis keeps 64. `hle_movie.c` now has `MD_NOUT` 64, `MD_NOUT_AAC` 32, `MD_NOUT_VIDEO` 16. + +### Video output slots: how far the extractor gets ahead + +- **Symptom:** a movie shows as a black screen with the loading icon, one picture every few seconds, while its sound decoder keeps up. +- **Cause:** this player also holds every output it is offered (16 of 16 video, 59 of 64 audio), so how far its extractor can get ahead of the picture on screen is the video slots plus the 4 queued packets. With 8 video slots that was 0.4 s, and the audio behind the next video packet, whose clock the player follows, ran dry: 86 pictures in 25 s. With 16 or 32 slots all 1,960 play at 30 fps. +- **Fix:** a video decoder now has 16 (`MD_NOUT_VIDEO`); a slot is allocated when first used, so the 8 more cost 24 MB of NV12 at 1080p only in a player that holds them all. +- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) (the in-flight movie `Movie/OpeningMovie.webm`, VP9 1080p and Vorbis through `movie::Extractor`). + +### AAC decoders offer at most 32 outputs + +- **Symptom:** a crash in the game's heap free (`sead::ExpHeap`) on a movie thread, a second into the first movie, at the same address every boot (`pc=main+0x42f980`, a free-list unlink through a link of 0x3f, `addr=0x47`). +- **Cause:** the game's movie player keeps the outputs it holds in a ring of 32 entries {u32 output index, u32 flags, u64 time} per track (`main+0xe93d4c`: 0x200 bytes) and pushes with no bounds check (`main+0xe94140`), and the host's audio decoder offered 64 outputs, so the 33rd outstanding one, index 63, went past the ring. The console's AAC decoder cannot offer more than 32, or the game would overflow there too. +- **Fix:** AAC decoders now have 32 outputs (`MD_NOUT_AAC`; Monster Train 2 peaked at 30 held), Vorbis keeps 64. +- **Found on:** [Mario vs. Donkey Kong](../../dm/games/0100B99019412000/README.md) (1.0.1, SDK 15.3.0: its opening movie `Movie/Cutscene01.mp4`, H.264 and AAC through `movie::Extractor`). +- **Check:** `DM_PROBE_WATCH` on the link broke the movie's file reads (the page also held a 64 KB read buffer, and `read()` into a protected page fails with EFAULT), so the movie stopped and nothing crashed; `DM_PROBE_POLL=<address>+8=3f` (a snapshot only at the change to that value) showed it being written right after the block was relinked, and the values around it were the ring's pairs. + +### AAC output buffer size is one frame + +- **Symptom:** the movie then froze on its first second, its sound decoder full and the game taking no more audio. +- **Cause:** the player opens an AudioOut of its own (`main+0xe92c2c`), sizes its buffers by the audio decoder's `GetOutputBufferSize`, decodes straight into them (`SetDataAndCapacity` with that size) and appends each whole, advancing its audio clock by size / 4 samples (`main+0xe92a74`). The host answered 128 KB, a bound for any Vorbis packet, so every 4 KB AAC frame played as 0.68 s and the clock crawled (a temporary log of `AppendAudioOutBuffer` showed 131,072-byte buffers). +- **Fix:** an AAC decoder now reports one frame: 1024 samples per channel, 2048 with SBR (object type 5 or 29 in the AudioSpecificConfig), which is what each output holds; Vorbis keeps the bound, its packets vary in length. +- **Found on:** [Mario vs. Donkey Kong](../../dm/games/0100B99019412000/README.md) (1.0.1, the same movie as in [AAC decoders offer at most 32 outputs](#aac-decoders-offer-at-most-32-outputs)). + +## Native-texture output + +### Decoding into the client's NVN texture + +- **Cause:** the decoder is created with output format 3 and mode 2 (`DecoderMode_NativeTexture`). The client's `movie::Buffer` then carries the `NVNtexture` object itself as its data, 8 bytes of capacity (an `RGBX8_SRGB` texture embedded in the game's texture pool entry, `main+0x226c98`), plus `SetNvnDev` / `SetNvnSync`. +- **Fix:** the host converts the VP9 picture to RGBA (BT.709 / BT.601 and range from the stream) and writes it into the texture's storage like `WriteTexels`, then marks the sync as signalled (`nvn_host_write_texture` / `nvn_host_signal_sync` in `hle_nvn.c`). The host writes the texture's block-linear storage on the CPU and the replay layer re-uploads the dirtied pages, as for `WriteTexels` ([nvn-objects.md](nvn-objects.md)). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). + +### SDK 4's `movie::Buffer` has no sync object + +- **Symptom:** `FATAL movie: missing SDK symbol _ZNK5movie6Buffer10GetNvnSyncEv` at the first frame of the native-texture movie (H.264 into the game's RGBA texture, as in Monster Hunter Rise). +- **Cause:** SDK 4's `movie::Buffer` carries the texture object but no sync object. +- **Fix:** the lookup is optional now; the game reads the texture once `GetOutputBuffer` returns, which the CPU write satisfies. +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (SDK 4.5.0; one boot). + +### Parameter sets in `Configure` made the H.264 decoder without RGBA + +- **Symptom:** a native-texture movie shows as a strip of small grey copies at the top of the screen. +- **Cause:** the texture held NV12 bytes. In RGBA8, 405 rows of 1920 pixels are exactly a 1920×1080 NV12 frame, rows 0–270 the luma at video black (16) and rows 270–405 the chroma (128). This game's MP4 demuxer hands the parameter sets to `Configure` (`csd-0` / `csd-1`), which made the H.264 decoder there without switching it to RGBA, and the first packet then found it made. +- **Fix:** both paths now go through `md_avc_open` (`hle_movie.c`). +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) (H.264 logos, FNA3D's `nx_video_player.cpp`: type 1, output 3, mode 2). Monster Hunter Rise's movies never brought parameter sets to `Configure`. + +### Last picture kept across a flush and at end of stream + +- **Symptom:** looping movie previews flash black at the end of every loop. +- **Cause:** the game reads the buffers of the end-of-stream output (no picture) and of an output it acquired before the flush, and the host painted its texture black for both. +- **Fix** (`hle_movie.c`): a flush now keeps each slot's last picture, and in native-texture mode an output with no picture leaves the texture as it was, as the SDK's GPU consumer has nothing to draw. +- **Found on:** [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (the same looped activity previews, Flush and Start). + +## Demuxers and codecs + +### `movie::Extractor` runs natively + +- **Symptom:** the game crashed on a null right after the failed open. +- **Cause:** `movie::Extractor` (libwebm in `subsdk0`) runs natively. The host used to force `SetDataSource` to fail because Engage's Unity demuxes by itself. +- **Fix:** the host no longer forces `SetDataSource` to fail. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). + +### VP9 on host threads with libvpx + +- **Fix:** VP9 packets are decoded with libvpx (`dm/third_party/libvpx`) on one host thread per decoder: guest threads run on the stacks the SDK gives them, too small for libvpx. Packets are decoded in arrival order from one queue. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md). + +### H.264 into an RGBA texture and AAC through AudioToolbox + +- H.264 (type 1) into the client's NVN texture (output 3 = RGBA, mode 2), with AAC 5.1 sound (type 4, PCM16). VideoToolbox decodes to BGRA for that mode (`mavc_set_rgba`), and AAC goes through AudioToolbox's converter with an MPEG-4 ES descriptor built from the AudioSpecificConfig in `csd-0` (as FFmpeg's AudioToolbox decoder does). Found on [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md); its end-of-stream packets and decoder deletes are in [End-of-stream packets with no data, and deleted decoders](#end-of-stream-packets-with-no-data-and-deleted-decoders). + +### Vorbis movie sound + +- Movie sound in [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0): the decoder is asked for type 5 (Vorbis) with output format 1 (PCM16); see the other entries in this file and `hle_movie.c`. + +## Movie players + +### Unity's video planes and checking the movie path + +- Unity's `Hidden/VideoDecode` samples luma as `.w` of an `R8` texture with the swizzle `(0, 0, 0, R)` (Unity's Alpha8) and chroma from an `RG8` texture at half size; both are CPU-written block-linear textures ([nvn-objects.md](nvn-objects.md)). +- **Check:** to check the movie path apart from the decoder, fill the frames with a gradient in `hle_movie.c`; it should appear on screen. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md). diff --git a/docs/reference/nvn-objects.md b/docs/reference/nvn-objects.md new file mode 100644 index 0000000..48229b7 --- /dev/null +++ b/docs/reference/nvn-objects.md @@ -0,0 +1,361 @@ +# NVN objects + +NVN object layouts, memory pools, command memory, queues, syncs and entry points (`dm/gfx/nvn/hle_nvn.c`). Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Object sizes + +### Host records must fit the game's SDK object sizes + +The game allocates NVN objects with the sizes of *its* SDK's headers, which shrank in newer SDKs. A host record larger than the game's allocation overwrites neighbouring memory (memory corruption): + +- **Symptom:** heap corruption; a stack smash where a function returned to `0x100000000`; a crash in SHA-256 on the code address 0x16300. +- **Cause:** + - `NVNtexture` is 0x80 bytes in SDK 16 (heap corruption: the old 0x88 record overwrote the next heap block's links). + - `NVNtextureView` is 16 bytes in SDK 16 (stack smash: a 16-byte local next to the saved frame pointer; the function returned to `0x100000000`). + - `NVNprogram` is smaller in Mario than the 0xC0 bytes in `nvn.h`: growing the host record from 112 to 168 bytes corrupted the next object (a crash in SHA-256 on the code address 0x16300). +- **Fix:** keep `prog_t` at 112 bytes; per-program host data goes in a side table (`prog_key`). +- **Found on:** SDK 16 games (`NVNtexture`, `NVNtextureView`); [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (`NVNprogram`). +- **Check:** check every host object against the new SDK: `_Static_assert(sizeof(host) <= N)` next to the struct, and when a crash returns to a nonsense address, look at the 0x100 bytes below `sp` in the crash report for a zeroed frame. + +### Inlined SDK code writes into `NVNcommandBuffer` +0x10 + +The game may also write into an NVN object directly, without an NVN call, through code inlined from the SDK headers. Host records stored in place must leave those bytes alone (the same holds for recorded command memory: [Recorded commands patched in place by the game (`OP_BIND_TEX1`)](#recorded-commands-patched-in-place-by-the-game-op_bind_tex1)). + +- **Symptom:** out-of-range vertices, NaN, a black band and flicker; seen on the iPhone, while Mac runs of the same scene looked clean. +- **Cause:** Mario writes the byte 1 at `NVNcommandBuffer` +0x10 (`main+0x4da5c`, `main+0xb8dec`). In the host record it was the low byte of `end`. Each write shrank the game's 512-byte material setup blocks, so their recordings overflowed into host segments; `GetCommandMemoryUsed` (then `cur − begin`) returned 0, the game marked the setups empty and skipped calling them, and the draws ran with the previous draw's vertex buffers ([What `GetCommandMemoryUsed` counts](#what-getcommandmemoryused-counts)). +- **Fix:** `cmd_t` reserves the byte (`nvn_flag10` in `nvn_obj.h`). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). +- **Check:** to find such writes, put the field you suspect behind a guard value and log when it changes outside the host's own code, or force overflows on the Mac by shrinking the command memory the game gets. + +### Sizes the hardware never reports as 0 (ZCull save area) + +- **Symptom:** sprites broke apart in some runs only, depending on the allocation order; buffers overlapped. +- **Cause:** host answers that size the game's own allocations count too. A size the hardware never reports as 0 must not be 0 on the host, even where the host needs no memory: Cuphead (Unity 2017.4) allocates each depth target's ZCull save area from its NVN pool allocator, which gives a zero-size request the offset of a live block. Its map of allocations then lost that block, and freeing it later released a garbage-sized range. +- **Fix:** the host never reports a ZCull storage size of 0 (`hle_nvn.c` `zcull_size`). +- **Found on:** [Cuphead](../../dm/games/0100A5C00D162000/README.md). Found by tracking the allocator from observation gates (`OBS` in its `patches.py`) on its free-list push and after its map insert, checking at every event that each block allocated so far was still in the game's map. + +## Memory pools + +### Heaps cut from a pool at alignment relative to its GPU address + +- **Symptom:** a null write as soon as the main thread starts recording a command buffer (13 Sentinels: a constructor at `main+0x2fa9a0` runs on `this = 0` when the prologue enters its first battle). +- **Cause:** the object came from one of the engine's ring heaps (`main+0x2d83ec`), and that heap had base 0. At startup the engine takes one GPU memory pool, sized as exactly the sum of three ring heaps, and cuts the heaps out at 64 KB alignment *relative to the pool's GPU address* (`main+0x30c118`). On the Switch a pool's GPU address is big-page aligned; the host's GPU address is the CPU address, and the storage came from the game's `malloc` rounded only to 4 KB. Up to 60 KB was lost at the front, the third heap (16 MB) did not fit, and the sub-allocator returned 0. +- **Fix:** the game's `malloc` is `nn::mem::StandardAllocator::Allocate(size)`; on this build the host first routed allocations of 64 KB or more to `Allocate(size, 0x10000)` (`hle_os.c`). The routing is now the profile key `alloc_big_align` (`hle_os.c` `h_mem_SAAllocate`), which [Unicorn Overlord](../../dm/games/010069401ADB8000/README.md) (the same engine) sets too. +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md). Probing the heap object at the allocation (`DM_PROBE=main:2bec60@x0+8`) showed the zero base directly. + +### `newBufferWithBytesNoCopy` retried under memory pressure + +- **Symptom:** `[RED] memory pool … created at retry N`. +- **Cause:** a memory pool created while the game maps and unmaps a large region was sometimes refused once by `newBufferWithBytesNoCopy`, then accepted. +- **Fix:** `mtl_pool_add` retries up to 12 times with a short back-off and prints the red line; on a final failure it logs the VM regions around the pool. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (its 4 GB region). + +### Virtual memory pools mapped onto one physical pool + +- **Symptom:** a buffer is created with a null pool right after a scene change. +- **Cause:** NVN *virtual* memory pools (flag 0x200) have no storage until `nvnMemoryPoolMapVirtual` maps physical pool pages in; if the host returns failure, the game treats the pool as unusable. +- **Fix:** a virtual pool is aliased onto the physical pool named by `MapVirtual` when all requests share one base (`hle_nvn.c`): Mario maps its single 128 MB virtual pool onto one physical pool in one request, so the host aliases it. Other mappings (true sparse) were then unimplemented; they are now [sparse virtual pools](#sparse-virtual-pools-built-from-separate-physical-pools) (next entry). +- **Note:** a null-pool buffer even with virtual pools working means the game's own heap ran out ([A null-pool buffer from a game heap that ran out](memory.md#a-null-pool-buffer-from-a-game-heap-that-ran-out)). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (all requests share one base). + +### Sparse virtual pools built from separate physical pools + +- **Symptom:** `[RED] MapVirtual: non-contiguous mapping (true sparse) not implemented`. +- **Cause:** RE Engine's GPU heap is one 64 GB virtual pool built from separately allocated 4 MB physical pools, each mapped at a virtual offset of its own (45 at boot, over the first 180 MB). A virtual pool larger than the application pool cannot be one alias of a physical pool. +- **Fix:** such a pool gets an address range of its own at `MemoryPoolInitialize` (reserved, inaccessible: 64 GB on the Mac, at most 1 GB on iOS), and each mapping request `vm_remap`s the physical pages over its part of the range, so virtual addresses read and write the same memory, as the GPU's page table does. Each run of mapped pages is registered with the replay like a pool of its own and rebuilt when a request joins or splits runs (`vpool_*` in `hle_nvn.c`). Mario's 128 MB pool, mapped whole onto one physical pool, keeps the alias. The log reports the mapped high-water mark per 128 MB. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md). + +### Textures read through a 4 KB page table of a sparse pool + +- **Symptom:** `CRASH sig=10` in `memmove` from `pkg_upload` → `nvn_deswizzle`, at an address 32 GB into a sparse virtual pool; textures with unaliased tails came out magenta. +- **Cause:** the game's streamed textures (flag 0x20, packaged layout) live 32 GB into its virtual pool; it writes their tiles on the CPU into 4 MB physical pools and then maps them, with no GPU copy. NVN maps at 4 KB and the host aliases at its 16 KB page, so a request whose offsets differ modulo 16 KB cannot be aliased: the ASTC 6x6 textures' mip tail starts 0x81000 into the texture. +- **Fix:** every request also goes into a 4 KB page table (`vpt_*` in `hle_nvn.c`: the physical page behind each virtual page and the generation of the request that set it). The replay reads a sparse pool's textures through it (`nvn_sparse_copy`, unmapped pages as 0) and uploads one again only when a mapping in its range changed or the CPU wrote a physical page behind it (`nvn_sparse_since`; the physical pools are dirty-tracked). Before that the untracked alias counted as written on every look: 32 textures, 18 MB, were uploaded again every frame, and those with unaliased tails came out magenta. Plain sampled textures in the pool go the same way: [Sampled textures in a sparse pool uploaded through the page table](#sampled-textures-in-a-sparse-pool-uploaded-through-the-page-table). +- **Note:** requests beyond the reserved range (iOS reserves 1 GB: the buffers stay under 0x20100000 through the opening) are kept in the table only, like unaligned ones: textures can read them, the GPU cannot address them. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md). + +### Finalized pools kept alive by residency and the sharing table + +- **Symptom:** GPU memory grows with every scene change (Engage's idle title demo reached 5.4 GB of GPU memory and 3.6 GB resident in 850 s). +- **Cause:** a game that frees memory pools on scene changes and builds new ones (Unity does; Mario never did in its first course) hit two host structures that kept freed pools alive: the residency set still held each pool's no-copy `MTLBuffer` after `nvnMemoryPoolFinalize`, and the storage-sharing texture table (`g_alias`, keyed by pool object, offset and description) never evicted, so the old pools' render targets stayed, and linear textures built on a pool buffer kept the whole buffer alive. +- **Fix:** pool buffers are retired through deferred release, and finalizing a pool evicts its sharing-table entries (`mtl_alias_drop_pool`). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): it now stays at 1.2 to 1.4 GB. +- **Note:** the `memory up to frame` line also reports `serial objects` (host objects by serial), `freed but alive` (finalized pool buffers still alive) and `shared textures` (sharing table size and evictions); growth in GPU memory with those counts flat points to an allocation outside them. + +### Memory-pool tracking table grows in blocks + +- **Cause:** the memory-pool tracking table had a fixed 4,096 slots; a pool past the end would have its textures re-uploaded at every use. +- **Fix:** it grows in blocks of 1,024 (`nvn_dirty.c`, up to 262,144). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): the phone reached 4,006 pools (found while fixing GitHub issue #9: [Deferred-release drain without an autorelease pool](memory.md#deferred-release-drain-without-an-autorelease-pool)). + +### Dead texture serials kept in a ghost table + +- **Symptom:** the serial table grows by about 10 entries a frame (276,000 after 25,000 frames). +- **Cause:** a game that records transient render targets and finalizes them before it submits: replay put the dead serials back into `g_by_serial`. +- **Fix:** dead serials go to a 512-entry table (`tex_put`, `g_ghost`; the memory line's `finalized textures still referenced`). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (the HUD captures, the 640×360 buffers; found while fixing GitHub issue #9). + +## Command memory + +### Host-lent command memory segments + +- **Symptom:** `FATAL NVN command memory: the game's pool is empty…`. +- **Cause:** our command records are larger than real NVN commands, so command buffers the game sizes for hardware overflow; when the game's out-of-memory callback declines to add memory, the host lends a segment. Many small command buffers each overflowing once per frame exhausted a fixed pool of 64 × 1 MB. +- **Fix** (`hle_nvn.c`): the pool now grows on demand (256 KB segments, abort only beyond 1 GB; `hostseg_take`). A segment is released only when the game re-grants memory overlapping the guest bytes of the recording that jumped into it (`hostseg_release` in `AddCommandMemory`): the game records setup blocks once and keeps `CallCommands`-ing them for many frames, so recycling segments after a fixed number of frames would overwrite live recordings. An empty-pool grant of at most 4 KB outside a recording gets a 4 KB segment, a record larger than 256 KB gets a segment that holds it, and the cap counts every segment's bytes (`hostseg_size`). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (the many small command buffers: its opening course, per the comment at `HOSTSEG_SZ` in `hle_nvn.c`), [Super Mario Odyssey](../../dm/games/0100000000010000/README.md) (it hands out many 512-byte grants before recording, and with no recording to reclaim them by, 256 KB each reached the cap entering the Wooded Kingdom: the 4 KB segments). + +### What `GetCommandMemoryUsed` counts + +Games read this count as a position in their own command memory and size their heaps and slots from it, so it must be what NVN counts. + +- **Symptom:** frames stop after about 15 s; a game thread spins in its command-memory callback (a ring allocator at `main+0x9cc120` waiting for fences that have all signalled). +- **Cause:** + - NVN counts used bytes from the last `AddCommandMemory`; the host counted from `BeginRecording`. KTGL gives its main command buffer one 16 MB chunk of a ring and, in its callback (`main+0x9ca960`), takes chunk start + `GetCommandMemoryUsed` as the position the GPU has reached, so the position stayed at one frame's size and the ring never freed. + - `GetCommandMemoryUsed` must count everything the recording used, including the parts that went into host segments (plus 16 bytes for the jump). Mario marks a recorded block as empty when the count is 0 and then skips calling it (how that happened: [Inlined SDK code writes into `NVNcommandBuffer` +0x10](#inlined-sdk-code-writes-into-nvncommandbuffer-0x10)). + - Games also size their own heaps and slots from this count: [Model heaps sized by the host's command record sizes](memory.md#model-heaps-sized-by-the-hosts-command-record-sizes) (Breath of the Wild's model heaps) and [Prerecorded lists in slots of their measured size](#prerecorded-lists-in-slots-of-their-measured-size) (next entry). +- **Fix:** the count now resets at `AddCommandMemory` (`hle_nvn.c`). +- **Check:** Super Mario Bros. Wonder and Super Mario Odyssey checked unchanged after the reset. +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md) (the count from `AddCommandMemory`); [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (host segments counted). + +### Prerecorded lists in slots of their measured size + +- **Symptom:** `CRASH sig=10` in `mtl_exec` after minutes of play, the fault address equal to `x19` and shaped like `0x1000000003`. +- **Cause:** the fault is the record-header load at the top of `mtl_exec`'s loop, `x19` is the replay pointer, and the address is itself a record header (`{op 3 = OP_CALL, size 16}`): a JUMP's target had been overwritten by the next recording. Tears of the Kingdom gives each model's prerecorded lists fixed slots of the size a recording of that list measured (`GetCommandMemoryUsed`), slot k at base + size × k, and passes `AddCommandMemory` max(size, 512) (`main+0xbb120`); its memory callback is empty. The host kept 24 bytes free after every record for "a JUMP or an END", so a 496-byte list's END went to a host segment and the 16-byte JUMP at 488 reached 8 bytes into the next slot. +- **Fix:** the host now keeps 16 bytes (one JUMP), lets the END take a segment's last bytes, and links only recordings that have records, so a JUMP only stands where a record of 16 bytes or more was to go and never writes past what `GetCommandMemoryUsed` reported (`OP_RESERVE` in `hle_nvn.c`; `t_nvn` "slotted lists"). +- **Note:** to read such a crash, `x8` holds the jump-table target of the previous record's case, and the registers a callee saved below `sp` give the previous replay pointer (`x19`) and payload (`x26`). +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) (1.4.3, GitHub issue #61). + +### Recorded commands patched in place by the game (`OP_BIND_TEX1`) + +- **Symptom:** the title showed one stretched 64×64 icon over the 3D scene, a new game a white screen; post-process passes with stale texture handles; `[RED] bind record … overwritten after recording`. +- **Cause:** the game may write into recorded command memory directly, and host records stored in place must leave those bytes alone. The engine records a command buffer, then rewrites texture handles in it before submitting, as the hardware lays the command out: before `nvnCommandBufferBindTexture` it takes the command memory base + `GetCommandMemoryUsed` (`main+0x7c8964`, the only reader of that NVN entry point), and every frame it stores a 32-bit handle at that address + 0x10 (`main+0xb7d0d4`, on a job worker). In a host bind record that word was the count, and the replay read 535 million handles from whatever followed, so the post-process passes kept stale texture handles in their slots. +- **Fix:** a single texture bind is recorded as `OP_BIND_TEX1`, its handle 16 bytes in, in the hardware's layout (`nvn_obj.h` `p_bind_tex1`, read through `bind_rec`); every game. +- **Found on:** [Luigi's Mansion 3](../../dm/games/0100DCA0064A6000/README.md). Found with a check that reports a bind record holding more than 32 handles (`[RED] bind record … overwritten after recording`), then `DM_PROBE_WATCH` on that word. +- **Note:** any other command a game patches this way needs the same: find the readers of the `GetCommandMemoryUsed` slot first. + +## Queues and syncs + +### Fence syncs recorded in command buffers + +- **Symptom:** `nvnCommandBufferFenceSync` unimplemented, every `SyncWait` timed out at once. +- **Cause:** Pokémon Legends: Z-A fences once per frame from a command buffer and waits on it; Unreal's RHI records the call in its command buffers too. +- **Fix:** a record (`OP_FENCE_SYNC`) fences the sync at replay with the fence value of the submission segment being encoded, so it signals once everything before it has completed (the replay fences the sync with the submission that reaches it). +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11; one boot); [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (new with Unreal). + +### Submissions from two queues on two threads + +- **Symptom:** `FATAL submit re-entered` in the first frames. +- **Cause:** the game initializes two NVN queues and submits on both from two threads, while the Metal replay keeps one command buffer for the submission in progress. +- **Fix:** `nvnQueueSubmitCommands` now holds a lock across the replay (`g_sub_mu` in `hle_nvn.c`). +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md). + +### A GPU command buffer that never completes + +- **Symptom:** frames stop, no crash, the main thread sits in `nvnSyncWait(sync, -1)`, and the watchdog's wait-state line shows the main fence one short (`main fence submitted N / completed N-1`). +- **Cause:** a GPU command buffer never completes. +- **Check:** GPU load from outside: `ioreg -r -d 1 -c IOAccelerator | grep 'Device Utilization'` at 100 % means a shader is looping forever. Find the shaders first used in the last frames (`DM_NVN_PASSDUMP`, compare the `K` keys) and look at their `.metal` for an empty `do {} while (true);` with no `return` (one such case: [Exact program start for the decoder](shaders.md#exact-program-start-for-the-decoder)). +- **Found on:** no game named in the source (general). + +### `nvnQueueFinish` waits for the GPU, not the display + +- **Symptom:** loading screens many times slower on the iPhone than on the Mac (Three Houses' first loading screen 78 s on the iPhone, 5 s on the Mac). +- **Cause:** `mtl_finish` called `mtl_present_flush` first, and with a window that waits until the display link has taken every queued present: one to two display frames per call. A loader that follows each upload copy with `nvnQueueFinish` pays that hundreds of times per loading screen. The windowless Mac runs present at once, so they never showed it; `DM_WINDOW=1` does (one step took 20 s instead of 1.7 s). +- **Fix:** `nvnQueueFinish` waits only for the GPU work submitted so far, as on the Switch, where it does not wait for the display. A swapchain texture is still not handed back before its previous present is processed (`AcquireTexture` waits for that ticket), and `mtl_drain` keeps its flush. +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md): KTGL's loader (effect banks, face models). Windowed Mac run after the fix: first load 6.1 s, frame pacing unchanged. +- **Note:** how it was found on the device: `Documents/env.txt` with `DM_WATCH=50` (thread table at 50 s: only the yield-polling main thread used CPU) and `DM_FSLOG=<absolute path>` (the reads came in bursts, with stalls of 4 / 24 / 13 s right after the effect banks). + +## CPU-written textures + +### Linear textures in CPU-visible pools + +- **Symptom:** menus and dialogue boxes draw but contain no text, while images and pictogram digits are fine; the sky is black (CPU-written linear colour ramps). +- **Cause:** CPU writes into a linear texture in a CPU-visible pool must be visible to the GPU, and a private Metal texture never sees them. Mario rasterizes glyphs on the CPU into a *linear* texture in a CPU-visible pool (2048×8192 R8, flags IMAGE|LINEAR), and its sky reads four 64×1 RGBA16F colour ramps written the same way. +- **Fix:** a texture with `NVN_TEXTURE_FLAGS_LINEAR` or `LINEAR_RENDER_TARGET` in a CPU-visible pool is created on the pool's `MTLBuffer` (`tex_linear` in `nvn_mtl.m`), so CPU writes (Mario's glyph cache) are visible to the GPU. Since Rhythm Heaven Groove a linear texture is made on its pool's buffer in any pool ([A pool's storage at creation is its textures' content](#a-pools-storage-at-creation-is-its-textures-content)). +- **Note:** block-linear CPU-written textures still need the packaged-texture re-upload path (next entry). +- **Check:** `DM_NVN_TEXCPU=1` lists non-packaged textures in CPU-visible pools. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md): its glyph cache (the text texture) and its four 64×1 RGBA16F sky colour ramps. + +### Textures written with `nvnTextureWriteTexels` + +- **Symptom:** video frames, or a game's built-in default textures, are all zero. +- **Cause:** the texture is block-linear, has no packaged layout and is written from the CPU with `nvnTextureWriteTexels`, so its content lives in guest memory, not in a GPU texture (Engage's video frames, Unity's movie frames: Y plane `R8` + UV plane `RG8`, every frame; Mario's and Unity's 4×4 fallback textures, which hold black, white, transparent and red texels). The host uploaded only packaged textures from guest memory and made everything else a GPU-only texture. +- **Fix:** `WriteTexels` now records the texture, and it resolves like a packaged one: uploaded from guest memory, re-uploaded when its pages are dirtied (`mtl_tex_cpu_written`). +- **Note:** Mario's frames got slightly brighter where materials sample the default white texture. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) (video frames and default textures), [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (fallback textures), Unity games (fallback textures; the source names no other). + +### `WriteTexels` regions written in place by block-linear address + +- **Symptom:** after a few minutes of play the main thread spends whole frames in `WriteTexels` (Metroid Dread's frames stopped at about 175 s). +- **Cause:** a game that writes a 1×1 region per call (Metroid Dread's set-pixel helper `main+0x160e80`, probably the map / minimap mask); the host decoded the whole level, patched it and encoded it again for every call. +- **Fix:** non-3D regions are written in place by block-linear address (`write_region_bl` in `hle_nvn.c`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). +- **Check:** `t_nvn` checks 15 regions in RGBA8, BC1 and a two-layer RGBA16F array, from 1×1 to the whole level, byte for byte against the old whole-level path. + +### Packaged texture data copied at `TextureInitialize` + +- **Cause:** a texture created with `TextureBuilderSetPackagedTextureData` gets that data copied into its storage by NVN at `TextureInitialize` (a compressible pool cannot be written by the CPU). +- **Fix:** the host copies it and marks the texture CPU-written, so it is de-laid-out and uploaded like a packaged texture. Linear textures are excluded: in a CPU-visible pool they are read straight from the pool buffer. +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (every texture is created this way; the linear ones are the save thumbnails). + +### A pool's storage at creation is its textures' content + +- **Symptom:** black screen with every geometry right (`DM_NVN_HILITE` shows each sprite and glyph in place); the uniform buffer right (a `CBDUMP`), so every texture read 0. +- **Cause:** NVN keeps what a pool's storage held at `nvnMemoryPoolInitialize` as its content, but the host uploaded from guest memory only packaged, `WriteTexels` or flushed textures, and made a linear texture on its pool's buffer only in a pool the CPU can see. Rhythm Heaven Groove reads each texture file into memory and makes a `CPU_NO_ACCESS | GPU_CACHED` pool (flags 0x21) on it, so its textures sit in `CPU_NO_ACCESS` pools made on memory the files were read into, whose initial content the host did not take; its sprite shader discards a fragment whose alpha is 0. +- **Fix:** a new texture in a `CPU_NO_ACCESS` pool whose storage holds data starts with that content, uploaded once (`tex_init_fill` in `nvn_mtl.m`); it stays a GPU texture that can be rendered into, with no re-upload on page writes. A linear one is made on the pool's buffer in any pool (`tex_linear`). Left out: storage of zeros, `COMPRESSIBLE` textures (render targets in Rhythm Heaven Groove, in their own 0xa1 pools; their memory means nothing without the compression tags), multisample, depth / stencil and sparse ones. +- **Found on:** [Rhythm Heaven Groove](../../dm/games/0100D9F01D474000/README.md) (SDK 21.4.0): the sprite shader (`40adb3021c79b155`) computes texture × colour multiply + colour add and discards a fragment whose alpha is 0 (the `CBDUMP`: multiply 1, add 0); 245 pools by the first screens, each holding one texture at offset 0, 17 block-linear BC7 (flag `IMAGE`, no packaged layout) and 228 linear R8 (`IMAGE | LINEAR`). +- **Check:** found by a census of every texture's pool flags, flags, format and whether its storage held data at creation (a temporary line under `DM_NVN_LOG`). + +### First-use fill of storage the game flushed + +- **Symptom:** a texture the game filled with `memcpy` and flushed reads as zeros (BotW's sky brownish grey, the distant haze brown). +- **Cause:** a game that makes a CPU-cached pool, creates the textures, then copies data into the pool and flushes it: the host created the texture as a GPU texture of zeros at its first use. +- **Fix:** every game now fills a new GPU texture from its storage once when the game flushed that storage before the texture's first use, as it already did for a pool the CPU cannot access (`nvn_flushed` in `hle_nvn.c`, `tex_init_fill` in `nvn_mtl.m`; [previous entry](#a-pools-storage-at-creation-is-its-textures-content)): the GPU has not written such a texture yet, so its content is what the CPU put there. Unlike the profile's `tex_flush_upload` ([next entry](#mapped-range-flushes-taken-as-uploads-tex_flush_upload)) it never re-uploads, so a texture the game renders into afterwards stays a GPU texture. `DM_NVN_FLUSH_FILL=0` turns it off. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md), GitHub issue #52 (Mac and iPhone). The sky is precomputed atmospheric scattering: each frame a pass (`08fe25061d583637`, `uking_pass_shader.sharcb`) turns a 256×32×16 RGBA16F 3D table of in-scattered light into a 256×256 sky map, which the sky draw (`8110c74078548172`) and the materials' aerial perspective sample. The table and two 2D ones (transmittance 256×64, irradiance 64×64) come from `System/KSys/sky.skybin` in `Pack/Bootup_Graphics.pack` (1,212,416 bytes, the three textures' storage back to back). The game makes a `CPU_CACHED | GPU_CACHED` pool for each texture, creates the textures, copies the file's data in with `memcpy` and flushes (`main+0x1257d50`, once, at the sky's setup; the flush goes through `main+0xebf4c4`, which flushes only CPU-cached pools). With zeros the sky had no scattered light: only the clouds and the sun's colour were left. In the Great Plateau save the fill covers that one texture; the flushes (about 15 a frame) merge into about 530 ranges, and frame times did not change. DRAGON QUEST XI S (title, free play, menus; 300 s) filled none. +- **Note:** `DM_NVN_TEXRAW=212,214` (all zero) and `DM_NVN_TEXINFO=256x32` (`storage 1048576 bytes non-zero 781567` at first use, `swizzled=0`) showed it. + +### Mapped-range flushes taken as uploads (`tex_flush_upload`) + +- **Symptom:** a movie plays black (Age of Calamity: its intro movie under the "+ Skip" prompt). +- **Cause:** the decoder's NV12 output is copied by the game into textures in a CPU-visible pool and published with `nvnMemoryPoolFlushMappedRange`, which the host did not take as an upload. +- **Fix:** the profile key `tex_flush_upload`. Buffer textures stay out of that path: they are views onto the pool buffer, and an upload would have sent them past Metal's 2D size limit. +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md): a 1280×720 R8 and a 640×360 RG8 texture; the buffer texture kept out is a 1,048,576-texel R32F one. + +### CPU writes with no flush, caught by page protection (`tex_dirty_upload`) + +- **Symptom:** magenta surfaces (Metal decodes an all-zero ASTC block as the error colour (1, 0, 1)); a zero BC block decodes black, which looks like missing light; black screen while the UI (or a dialog box) draws in the right places under `HILITE` or in the pass dump; `packaged textures: first uploads 0`. +- **Cause:** the game writes texels into a texture's storage after `nvnTextureInitialize` with no `nvnMemoryPoolFlushMappedRange` or `FlushTexels` over them, so neither the flush path (`tex_flush_upload` never marks them) nor `tex_init_fill` (which only takes `CPU_NO_ACCESS` pools) sees them, and they stay GPU-only with content 0. Tears of the Kingdom creates textures on zeroed memory and then fills them from the CPU (its cache maintenance runs natively). Writes to an uncached pool need no flush (FINAL FANTASY TACTICS); a game with no cache maintenance instruction at all gives the host no flush to take as the signal (The Great Ace Attorney Chronicles). +- **Fix:** profile key `tex_dirty_upload`: `ft_add` write-protects a registered texture's pages, and on first use (submit thread) a write since then makes it CPU-written, so a texture whose pages the CPU writes after creation is uploaded at its first use. FINAL FANTASY TACTICS and The Great Ace Attorney Chronicles set it together with `tex_flush_upload`; with `tex_flush_upload` alone The Great Ace Attorney Chronicles stayed black. Textures written after their first use and `ReadTexels` before any draw: next entry. +- **Check:** `DM_NVN_TEXINFO=WxH` tells the cases apart: it prints how many bytes of the texture's guest memory are non-zero at first use. +- **Found on:** + - [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md): the cave was magenta; the terrain draws sample two 121-layer ASTC 8×8 arrays (42 MB each) filled this way. + - [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md): the UI reads its textures from `.tex` files the engine loads straight into a `CPU_UNCACHED | GPU_CACHED | COMPRESSIBLE` pool (flags 0xa2): 231 textures by the title screen, no packaged layout, no `WriteTexels`, no copies. + - [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md): a dialog box sampling 256x256 and 1024x256 textures of format 0x4d; `DM_NVN_TEXINFO=1024x256` showed storage that already held data at first use (226,555 of 262,144 bytes non-zero): no packaged layout, block-linear, in a 1000 MB `CPU_CACHED | GPU_CACHED | COMPRESSIBLE` pool (flags 0xa4). A log of the pool flushes, texture creations and file reads (temporary lines under `DM_NVN_LOG`) showed every `.tex` read into staging buffers (512 KB `ReadFile`s, none into the pool) and the texels copied by the CPU into the texture's storage; only the four linear 1024x64 text textures are flushed, every frame. `main` has no cache maintenance instruction at all. + +### `ReadTexels` from guest memory and textures written after first use + +- **Symptom:** a texture the game builds from CPU-filled textures read back with `nvnTextureReadTexels` comes out zero. +- **Cause:** `tex_dirty_upload` only marked such a texture CPU-written when the submit thread first resolved it, so `ReadTexels` on another thread before any draw returned the never-uploaded host texture: zeros. A texture filled after it was first bound was never marked at all. +- **Fix:** `mtl_read_texels` reads any texture whose content is in guest memory there (a packaged layout, a CPU-written texture, or a `tex_dirty_upload` texture written since registration), on any thread, without an upload; and `nvn_ft_poll` (`hle_nvn.c`, at each submission after guest memory changed, at most 128 MB of textures per look, going on where it stopped) marks registered textures written after their first use. Textures the GPU renders into or writes (`mtl_tex_gpu_owned`) leave that list without an upload. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) (Mac and iPhone): the stone blocks and steps of the opening cave got no torch light (on the Switch they are grey stone lit warm on the side facing the torch; here they kept their texture and the ambient light but stayed black). Their fragment shader `54e8c08998050e79` takes its roughness from a 5120×5120 BC4 atlas (roughness = the square root of its red, packed into the upper six bits of blue in colour target 3, which the tile-light shaders read); the atlas was all zero, and roughness 0 gives no point light. TOTK fills 1024×1024 BC4 material textures from the CPU with no NVN flush, reads them back with `nvnTextureReadTexels` on its own threads before any draw uses them, and copies the blocks into the atlas, which itself is filled after it was first bound. +- **Note:** found by running MacNativeGames' build of the same game on the same save: their atlas held data and their blocks were lit (their `ReadTexels` reads guest memory). Traced with `DM_NVN_TEXDUMP` (the blocks' roughness bits in colour target 3), the shader's MSL, `DM_NVN_TEXRAW` on the atlas and `DM_NVN_TEXINFO` (guest bytes), then lldb reading the atlas memory inside their process. Fast loop: the in-cave autosave (`run/save`, keys `f2500:A,f2800:A,f3100:A,f3600:A`, shot at frame 4500); `DM_DETCLOCK` cannot be used (TOTK stops at frame 1 waiting for guest time). + +### Sampled textures in a sparse pool uploaded through the page table + +- **Symptom:** textures that should hold CPU-written data come out magenta (ASTC's error colour) or black. +- **Cause:** textures in the sparse virtual pool hold what the CPU wrote into physical pages before mapping them: the streamed textures (flag 0x20, packaged layout) and plain sampled ones with no packaged layout. The replay took the plain ones for GPU textures, so their Metal copies stayed 0. +- **Fix:** a sampled texture in a sparse pool (not a render target, image, linear or buffer texture) is now uploaded like a packaged one, through the pool's page table ([Textures read through a 4 KB page table of a sparse pool](#textures-read-through-a-4-kb-page-table-of-a-sparse-pool)). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md): the hunter's innerwear (ASTC 6x6) came out magenta and the face black. + +### Dirty-page ranges of pools that share a host page + +- **Symptom:** some textures are re-uploaded every frame (several MB per frame). +- **Cause:** the dirty-page tracker registers each memory pool rounded to 16 KB host pages, but NVN pools are only 4 KB aligned, so neighbouring pools share their first / last host page and one address can lie in several registered ranges. Taking the first range that contains an address judged textures in the second pool untracked (always dirty), released only one range on a write fault, and could unregister the neighbour when a pool was destroyed. +- **Fix:** `nvn_dirty.c` now uses the range that holds the whole span, releases every range covering a faulting page, and removes a range by the pool's exact start. +- **Found on:** the source names no game. +- **Check:** `t_dirty_slots` covers it. + +### Writes beside a packaged texture in a shared host page + +- **Symptom:** packaged textures re-uploaded at every use with unchanged content; upload staging and frame times rising. +- **Cause:** the game keeps each texture of its models in a memory pool of its own, the pool's storage 4 KB after its own object, in the same 16 KB host page, and writes there every frame. The dirty-page tracker marks whole host pages, so every such texture counted as changed at each use; hashing showed every re-uploaded content unchanged. +- **Fix:** a packaged texture keeps the bytes of its storage that lie in its first and last host pages when they are shared (`pkg_edges_take`); when only those pages were written and the bytes are as uploaded, it is protected again instead of re-uploaded (`pkg_edges_clean`, counted in the `packaged textures` line as `writes beside a texture only`). +- **Found on:** [Pokémon Sword](../../dm/games/0100ABF008968000/README.md): 73,232 re-uploads in 70 s from a save, 12 GB of upload staging in 8 minutes, frame times rising to 75 ms; the same 70 s with the fix: 0 re-uploads and 72,584 such checks. +- **Check:** `t_nvn` 6c1 covers it. + +### Dirty-page tracker race and early stalls + +- **Symptom:** about one run in four stalled early (around the opening movie or the title), with no log output, not even the time-limit report. +- **Cause:** two candidates. First, the stalls happened before the real thread pause for IL2CPP's GC ([Pausing threads and reading their context for garbage collectors](threads.md#pausing-threads-and-reading-their-context-for-garbage-collectors)): without the pause the collector ran while other threads kept changing the heap. Second, fixed later: a race in the dirty-page tracker (`nvn_dirty.c`, ported from d2r-mobile) could leave a page read-only while marked writable, so a game thread writing it spun forever in the fault handler; Engage's movie decoder writes frames into texture storage from a host thread while the replay thread protects pages. +- **Fix:** the thread pause for the GC, and the tracker race fixed; `tests/t_dirty_race.c` reproduces the race. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): probably fixed; since the pause, 12 runs (eight of them 200 s runs through the movie and title, four of 330 to 850 s) have not stalled once, where one in four used to. +- **Note:** if a stall comes back, sample the process when its log stops growing (`sample <pid> 3`) to see where every thread waits. The crash handler writes one line with `write()` and exits if the stderr lock stays taken for 1 s, so a stall that holds that lock shows up as such. + +## NVN entry points + +### Version and capability checks that fail quietly + +- **Symptom:** device / subsystem setup returns quietly and the next stage never starts. +- **Cause:** a version or capability check failed (NVN API version, a feature query). +- **Check:** compare what the game queries with what the host answers. Two cases follow: [The device's API version](#the-devices-api-version) and [Newest GPU code version reported by the driver](#newest-gpu-code-version-reported-by-the-driver). +- **Found on:** every game (general). + +### The device's API version + +- **Symptom:** a null-pointer crash early in `nnMain`, right after NVN setup should have happened (Super Mario 3D World + Bowser's Fury, `main+0x759e08`: the device singleton `main+0x1a28ea0` is still null). +- **Cause:** the game checks `nvnDeviceGetInteger(API_MAJOR / API_MINOR)` for major == 53 and minor ≥ 313 (`main+0x711c88`) and quietly skips creating its device otherwise. +- **Fix:** the reported version follows the SDK (`hle_nvn.c` `nvn_api`: 53.313 for SDK 10 and older, 55.15 for later ones, Mario's requirement); the profile key `device_api` overrides it. NVN 53 also brings `nvnQueueAcquireTexture` (the older window acquire: the queue waits instead of a sync object), `nvnSamplerCompare` and the debug group calls (no-ops). +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10); [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (needs 55.15). + +### Newest GPU code version reported by the driver + +- **Symptom:** null read at `main+0xb8462c` right after the first two textures, called from `main+0x670eb0`. +- **Cause:** not the fault itself: `nnMain`'s start-up (`main+0x6705c0`) returns false when any of its 30 steps fails, the game then skips its main loop and goes straight to shutdown, and the shutdown walks a pool (`main+0x4efce70`) whose initialization (`main+0xb840d0`) came after the failed step. `DM_PROBE` on each step's entry (lldb was unavailable: developer mode off) showed the last one entered, `main+0x69c698`, the built-in shaders of `shader2.dat` (`hwfillcolor`, `hwshaderpf`, …), and probes inside it the step that failed: creating a shader from its object (`main+0xd0d998`). The engine checks each shader object's GPU code version (byte 6 major, byte 7 minor of the object) against the driver's range (`nvnDeviceGetInteger` 0x2E–0x31: newest major, oldest major, newest minor, oldest minor) and refuses newer code; the host reported 1.14, the table measured on Super Mario Odyssey's SDK 7 driver, and `hwfillcolor.vso` is 1.15 (the game's objects run from 1.9 to 1.16). +- **Fix:** a driver runs the code of its own SDK's GLSLC, so the newest version the host reports now follows the SDK: 1.15 from SDK 8 (Catherine: Full Body's GLSLC 8.2.1), 1.16 from SDK 13 (Bayonetta 3's newest objects), and at least the version of a GLSLC the game carries, as before (`hle_nvn.c`; that GLSLC check: [GLSLC GPU code version against the driver's range](shaders.md#glslc-gpu-code-version-against-the-drivers-range)). For every game. +- **Note:** a wider range only lets more shaders pass such checks. +- **Found on:** [Bayonetta 3](../../dm/games/01004A4010FEA000/README.md) (SDK 13.3.2, PlatinumGames' engine); [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md) (its GLSLC 8.2.1 set the SDK 8 value); [Super Mario Odyssey](../../dm/games/0100000000010000/README.md) (the table's 1.14). + +### Extensions fetched by name and called unchecked + +- **Symptom:** a jump to pc=0 right after the NVN bootstrap loader (`lr=main+0x37fe930`, `[RED] NVN address lookup of an unknown name: nvnSetExternalDebuggerEnabledNVX`). +- **Cause:** the game fetches an NVX extension of the SDK 8 driver by name and calls it without checking the pointer. Unknown names return null, as the real driver does for a name it lacks; this one the driver has. +- **Fix:** it joins `dm/gfx/nvn/names.txt` (thunks regenerated with `gen_protos.py`; no prototype in `nvn.h`, so `NVN_NOPROTO`) as a no-op. +- **Found on:** [Super Smash Bros. Ultimate](../../dm/games/01006A800016E000/README.md) (13.0.5). + +### NVN entry points on the big host stack + +- **Symptom:** SIGBUS in `AGCDeserializedReply`, address in the guest stack region. +- **Cause:** the render thread's stack is small, and Metal's driver compiles internal programs (a fast-clear blit) on the calling thread while the host ends a render pass inside `nvnQueueSubmitCommands`, which overflowed the guest stack. +- **Fix:** all implemented NVN functions except the `nvnCommandBuffer*` recording calls run on the per-thread 512 KB host stack (`nvn_big_entry` in `nvn_entry.S`). The general mechanism (`dm_bigstack_enter`, and `HLE_FORCE_BIG` for the file system hooks): [Host calls on small guest stacks](memory.md#host-calls-on-small-guest-stacks). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md). + +### State read back through NVN getters + +- **Symptom:** red `NVN unimplemented function called` lines for `nvnColorStateGetBlendEnable`, `nvnBlendStateGetBlendFunc`, `nvnPolygonStateGetCullFace`, `nvnDepthStencilStateGetDepthTestEnable`, `nvnBufferBuilderGetSize`, `nvnTextureBuilderGetWidth`, …. +- **Cause:** Trinity builds its pipeline states from state objects it reads back; the red-reporting stub returned whatever was in x0, the object's address, so a blend enable or depth test read as on and a texture builder's width as an address. +- **Fix:** the getters read the packed state objects (`hle_nvn.c`); the multisample setters the host lacked (alpha-to-coverage dither, raster samples, coverage modulation and coverage-to-colour) are recorded in fields `ms_st` already had. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11, Game Freak's Trinity engine; one boot). + +### Device getters that were red stubs + +- **Fix:** `nvnDeviceGetWindowOriginMode` and `nvnDeviceGetDepthMode` were red stubs: the game sets neither mode and reads both. `nvnDeviceGetTexelFetchHandle` returns the texture ID with sampler 0. +- **Found on:** [Pokémon Sword](../../dm/games/0100ABF008968000/README.md) (SDK 7: the window origin and depth modes); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11: `nvnDeviceGetTexelFetchHandle`). + +### Pool size getters, texture address and command memory size + +- **Symptom:** an abort in device setup when `nvnTexturePoolGetSize` / `nvnSamplerPoolGetSize` returned 0. +- **Fix:** implemented: `nvnTexturePoolGetSize` / `nvnSamplerPoolGetSize` and their memory pool / offset getters; `nvnCommandBufferFenceSync`, which Unreal's RHI records in its command buffers ([Fence syncs recorded in command buffers](#fence-syncs-recorded-in-command-buffers)); `nvnTextureGetTextureAddress` (pool address + offset, as `nvnMemoryPoolGetBufferAddress`); `nvnCommandBufferGetCommandMemorySize` (used + free, as in NVN; `GetCommandMemoryFree` is what is left in the current segment). +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (the pool getters and `nvnCommandBufferFenceSync`, new with Unreal); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (Unreal Engine 4.18: `nvnTextureGetTextureAddress`, `nvnCommandBufferGetCommandMemorySize`). + +### NVN calls that were red stubs + +- **Symptom:** red stub lines for NVN calls. +- **Fix** (`hle_nvn.c`): + - `nvnDeviceSetIntermediateShaderCache`. + - `nvnCommandBufferDiscardColor`: a discarded target is undefined, so keeping its content is correct. + - `nvnWindowGetCrop`: read back every few frames; before any `SetCrop` it is the whole first window texture. + - `nvnSamplerBuilderGetLodClamp`. +- **Found on:** [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1: the first two), [Bayonetta 3](../../dm/games/01004A4010FEA000/README.md) (SDK 13.3.2: the two getters). + +### `nvnDeviceRegisterFastClearDepth` is a hint + +- **Fix:** `nvnDeviceRegisterFastClearDepth`, a hardware hint, returns success instead of reaching the red stub. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md). + +### SDK 8 window and copy-stride entry points + +- **Symptom:** `nvnWindowGetPresentInterval`, `nvnCommandBufferSetCopyImageStride` missing. +- **Fix:** implemented (the latter is the layer stride of texture ↔ buffer copies). +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0). + +### Images registered on their own + +- `nvnTexturePoolRegisterImage` writes the same table as `RegisterTexture` (image handles are pool indices). +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md). + +### Plural texture binds, builder-registered samplers, strided texel writes, device item 0x67 + +- `nvnCommandBufferBindTextures` (the plural of `BindTexture`), `nvnSamplerPoolRegisterSamplerBuilder` (a sampler registered from its builder), `nvnTextureWriteTexelsStrided` (`WriteTexels` with row and image strides) and device item 0x67, which the game reads before every program it loads and uses as the alignment of its copies of the shader (answered 256, not measured). +- **Found on:** [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md). + +### `nvnPrivate0`, an inlined command-buffer fast path flag + +- `nvnPrivate0`, a new SDK 20 import, needs nothing: it is a flag in `sdk`'s bss that enables an inlined command-buffer fast path when the NVN driver sets it (it stays 0). Found on [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (1.9.0, SDK 20.5.22). diff --git a/docs/reference/rendering.md b/docs/reference/rendering.md new file mode 100644 index 0000000..a7ddf41 --- /dev/null +++ b/docs/reference/rendering.md @@ -0,0 +1,686 @@ +# Rendering + +The replay on Metal (`dm/gfx/nvn/nvn_mtl.m`) and the Vulkan and OpenGL layers. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Textures and formats + +### Texture objects copied into command records + +- **Symptom:** Metal aborts on a texture descriptor with an absurd size (`height (1869367126) greater than … 16384`). +- **Cause:** a command record kept a pointer to the game's `NVNtexture` and read it at replay. NVN commands do not depend on the API object once recorded. +- **Fix:** the host now copies the texture object into the record (`REC_SNAP` / `tex_snap` in `hle_nvn.c`: render targets, texture copies and clears). Replay rejects objects without the texture magic and sizes beyond Metal's limits, with a red line instead of an abort. +- **Note:** the first red lines name the call site and the object's first bytes. In Engage those bytes were 32-bit triangle indices, i.e. another write had already overrun into the texture object; that stopped once the alias region had address space left ([Size-aligned regions exhaust the alias address space](memory.md#size-aligned-regions-exhaust-the-alias-address-space)). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md). + +### New textures start as zero + +- **Cause:** Metal leaves a new texture's content undefined; a freshly mapped Switch memory pool reads as 0, and feedback effects (such as Mario's fountain wave simulation, [Metal 4 does not order operations inside one encoder](#metal-4-does-not-order-operations-inside-one-encoder)) never clear their state texture. +- **Fix:** non-packaged colour textures are zero-filled from a shared zero buffer when created (`tex_zero_fill`, batched with the uploads). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Texture swizzle through swizzled Metal views + +- **Symptom:** glyphs render as white boxes (the font atlas is `(1, 1, 1, R)`). +- **Fix:** `TextureBuilderSetSwizzle` / `TextureViewSetSwizzle` are applied with Metal swizzled texture views (NVN and Metal channel enums have the same values). The swizzle names components of the NVN format, which the format's own mapping finds in Metal's (`nvn_fmt_to_mtl`: alpha 1 for RGBX-type formats; red and blue swapped for the RGB565 family, but only in textures uploaded from guest memory, since what Metal renders into one is in Metal's order). +- **Note:** Metal makes a swizzled view read-only (usage 0x1), so an image slot gets the same view without the mapping (`img_fit`), and the on-chip read merge leaves a swizzled sample alone. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (the font atlas); The Legend of Zelda: Echoes of Wisdom's hearts (GitHub PR #66, the RGB565 red / blue swap). + +### `nvnTextureCompare` compares the channel mapping + +- **Symptom:** dark boxes and cream cloud rectangles behind HUD prompts, a black stamina wheel and a black System menu background (layout captures). +- **Cause:** agl's layouts rebuild a capture texture on the same storage with a new channel mapping, and replace the registered descriptor only if `nvnTextureCompare` reports the two textures different. The host compared the description and the storage but not the mapping, so every rebuild was dropped. +- **Fix:** `nvnTextureCompare` compares the mapping too, an unset mapping counting as (R, G, B, A). `t_nvn` covers the comparison. From GitHub PR #6. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): panes tagged `CaptureOn` (about 140 layouts, e.g. `PaGuideKeySmoke_00`; user data `A8`, `CaptureScale` 0.5, `MultiFilter`) are drawn into a target, blurred at half size and copied into a capture texture that the layout's materials sample in place of the placeholder `FBLayout_00`. The game creates it with its per-format default mapping (one call site, `main+0xecb0bc`: R8 → (R, 0, 0, 1), RG8 → (R, G, 0, 1), and so on), then rebuilds it with the capture's own mapping: `main+0xf597dc` picks (1, 1, 1, R) for an `A8` capture (the material colour kept, the mask as alpha) and alpha 1 when the pane says `CaptureOutputAlpha` 255 (read at `main+0xf58e98`; an RGB10A2 capture then reads (R, G, B, 1)); `main+0xecacf0` stores the mapping and rebuilds the texture (`main+0xecabe8`), and `main+0xec33f0` replaces the descriptor only on a difference. The captures kept (R, 0, 0, 1), whose alpha is always 1: hard-edged dark boxes behind every prompt (the shadow pane `e6a88e12dbae9743`: black, alpha = capture alpha × vertex alpha 180/255) and rectangles filled with cream cloud (the glow `aa209f57b133e862`: alpha = min(cloud alpha, capture alpha)). Checked on the Mac by the PR's author (M4 Pro, unchanged shader libraries): green stamina, the purple noise meter and the game's own blurred System menu background. [Super Mario Odyssey](../../dm/games/0100000000010000/README.md): its layouts rebuild their captures through the same agl code (`main+0x982320` replaces the registered descriptor only if `nvnTextureCompare` reports a difference), so the fix covers them too. +- **Note:** before the cause was known, the profile key `alpha_from_red` listed the layout library's pixel shaders and bound such render targets through a view mapped (R, 0, 0, R) (`tex_alpha_red` in `nvn_mtl.m`). In BotW (all 26 pixel-shader variants, `main`'s BNSH at 0x1d83000) that fixed the alpha but zeroed green and blue, so tinted panes such as the stamina wheel came out black, and the System menu background (a capture with alpha forced to 1) stayed black; BotW's profile no longer lists it. In Odyssey (all 26 layout pixel shaders, `main`'s BNSH at 0x132c000) it had turned the coin counters' solid white cards into the right shapes; with the fix it no longer fires: on the Mac smoke route to frame 20,100 it converted no texture (main's host converted its first, a 640×80 R8 target, at frame 4129), and the HUD at frames 17,300 and 17,600 is pixel-identical to main's. Odyssey's list stays until the change has been seen on an iPhone. + +### Texture views cached by texture and view parameters + +- **Symptom:** `newTextureViewWithPixelFormat` takes a large share of the replay thread (45% in Age of Calamity) when a game registers textures into a ring of IDs before every bind. +- **Cause:** KTGL registers a texture into the next ID of a ring before every bind (`RegisterTexture`, `GetTextureHandle` and `BindTexture` counts are equal: 8,000 a frame in battle, up to 28,000 after the cutscene). Only 1 in 6 registrations repeated what the ID held, so the per-ID view (`treg_t.view`) was rebuilt for half of all binds. +- **Fix:** views are now also kept by texture serial and view parameters (4 per texture), checked against the current base texture on each hit, and dropped when the texture or its pool is finalized; a registration identical to what the ID holds keeps its view and the registration generation (`vc_get` in `nvn_mtl.m`). +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md): 2,036 views were made in a 15-minute Mac battle run, 211 million reused; the replay before the cutscene went from 14.3 to 10.8 ms a frame. + +### Compatible views for a mismatched declared texture type + +- **Symptom:** daylight scenes washed out to white (every read of a texture returned one fixed value); `texture type mismatch` red lines. +- **Cause:** on Maxwell the sample instruction carries its own dimension, and a non-array sample of an array texture reads layer 0. On Metal a 2D array bound to a `texture2d` argument is a type mismatch, and every read returned one fixed value. 3D World creates its deferred pre-lighting buffer (the G-buffer's fourth target, R11G11B10F) as a *one-layer 2D array* texture, and the lighting shader samples it as 2D, so the whole picture got one pale ambient colour (bloom then blew it out further). +- **Fix:** the host compares each texture binding's declared type (the reflection's `dtt`) with the texture it resolves to and binds a compatible view: 2D array ↔ 2D, 2D array → cube / cube array (`tex_fit` in `nvn_mtl.m`, cached per texture-pool entry); any other mismatch is reported once. +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md). +- **Note:** how it was found: a blit-based pass snapshot of that target showed a correct, colourful image while a lighting shader edited to output only that input showed a uniform colour, and both resolved to the same Metal texture. Earlier suspects that turned out correct: fog, bindings, the 1D lookup tables, clear colours, the depth convention, the final sRGB view, blend states, GPU luminance readback. There is no tone-mapping pass in this game: the final composite writes `scene + bloom` into an RGB10A2 target. + +### 1D textures as one-row 2D textures + +- **Symptom:** a type mismatch when a Metal 1D texture is bound to a shader's `texture2d` argument. +- **Cause:** translated shaders never declare `texture1d` (none of the five games' libraries did at the time). +- **Fix:** NVN 1D and 1D-array textures are created as one-row 2D / 2D-array Metal textures. +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md): the material shaders sample 64×1 R32F lookup tables created as NVN 1D textures (and filled with `WriteTexels`). + +### Cube textures held as 6-layer 2D arrays + +- **Symptom:** shaders that sample a cube texture as `texturecube_array` get a type mismatch. +- **Cause:** Metal can make a cube-array view only of a 2D-array base texture. +- **Fix:** NVN cube textures are created as 6-layer 2D arrays and bound through cube / cube-array views by `tex_fit`, which also maps a cube array read as a cube to its first cube. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): 1,512 shaders sample the environment probes as `texturecube_array` while a single cube texture is bound. + +### Cube map arrays count layer-faces + +- **Symptom:** an abort in `MTLTextureDescriptor` validation, `arrayLength (510) … greater than the max allowed size (341)`, when the prologue movie ends. +- **Cause:** as in OpenGL, an NVN cube map array's depth is its number of layer-faces (a multiple of 6: 510 for 85 cubes), while a Metal cube array's `arrayLength` counts cubes. The host took the depth as a cube count in three places: the Metal texture, the upload's layer count and the guest layout (`nvn_tex.c` `shape`), so `GetStorageSize` reported six times the storage. +- **Fix:** a cube array is now created like a cube, as a 2D array of all its faces, and `tex_fit` makes the cube-array view a shader asks for (`t_nvn` checks the storage size against a 2D array's). +- **Found on:** [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md). + +### Cube views count their layers in cubes + +- **Symptom:** an abort in Metal's validation: `newSliceRange.length(1) must be a multiple of (6)`. +- **Cause:** environment maps are 6-layer textures (target 2D array, depth 6) sampled through views with target CUBEMAP and layers 0 + 1. As in the hardware's texture header, a cube view counts its layers in cubes; Metal counts faces. +- **Fix:** a cube view whose layer count is not a multiple of 6 now takes its layers as cubes when the texture has that many faces (`tex_by_id`; else it is read as a 2D array with a red line). +- **Found on:** [Pokémon Sword](../../dm/games/0100ABF008968000/README.md) (as the player first walks out of the house). + +### A 2D texture in a cube slot binds the zero cube + +- **Symptom:** `texture type mismatch … declares slot 10 as Metal type 5, texture … is type 3` (undefined reads). +- **Cause:** a 2D texture bound where the shader declares a cube (a placeholder for an environment map) has no cube view. The same for a 2D array of fewer than six layers, which is how the host holds some NVN 2D textures. +- **Fix:** the zero cube is bound instead of a type mismatch with undefined reads (`tex_fit`). +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md); [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md) binds a 2D array of fewer than six layers (format 0x44) at slot 10 of two pixel shaders that declare a cube there as a race loads, and it was bound as is. + +### Integer views for images a shader declares uint or int + +- **Symptom:** translucent UI (frosted text bands, button guides, dialog headers) yellow and white. +- **Cause:** the composite shader loads the window texture's texel as `r32ui`, unpacks the four bytes and stores the blend through a float view of the same image. The host bound the RGBA8Unorm texture to the `texture2d<uint>` argument as it was, whose reads are undefined (Metal's validation layer does not check textures in argument tables). +- **Fix:** an image slot the shader declares as uint / int now gets a view of the same texel size in that class (`img_fit` in `nvn_mtl.m`). The format-view usage costs lossless compression, so a texture gets it only when a shader needs such a view: it is upgraded (copied into a texture with the usage) before the next draw pass opens, as the copy cannot run inside the draw's own pass; until then that one draw reads it as before. +- **Found on:** [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md): the last passes have no render target; fragment shaders composite each UI layer into the window texture with image stores, blending in software (`uirender.tec`, `4cf1975ed1e36590`). + +### Buffer textures read by 1D integer fetches (row wrap) + +- **Symptom:** `texture type mismatch: shader declares 2, texture is 9`; the slot read 0. +- **Cause:** Maxwell fetches a texture buffer and a 1D texture with the same instructions (TLD with integer coordinates only, TLDS `Texture1DLodZero`), and the bound descriptor decides which it is. Without PC reflection the translator has no type answer and reads such a slot as a 2D texture at `(x, 0)`, and the host could only bind a dummy when the game put a buffer texture there. +- **Fix:** the vendored translator (`InstEmitTexture.cs`, marked `nxnative:`) now addresses every such fetch as `(x & 0x3FFF, x >> 14)` and reports the slot (`fetch1d` in the reflection). The host binds a buffer texture in a flagged slot as a linear 2D texture of 16,384-texel rows over the same pool memory (`tbuf_as_2d` in `nvn_mtl.m`, `nvn_tbuf_rows` in `nvn_tex.c`), cached by texture serial beside the views. A real 1D texture, which the host creates as a single row at most 16,384 wide, reads as before. A mismatch no view can bridge is logged once per shader and slot, with its texture ID, NVN target and formats (`tex_mis_note`). +- **Note:** limits: a size query on a buffer of more than 16,384 elements returns 16,384 (no shader both size-queries and fetches such a slot in Age of Calamity; the build summary counts them), and the rows stop at the end of the Metal buffer. +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md): 16,438 such shaders (98,970 bindings; slots 16–21 hold KTGL's light and decal lists). +- **Check:** Age of Calamity's 15-minute Mac battle bound 15.7 million of them through 320 row textures, none failed; the scene is darker and more saturated than with the zeros (mean luma 163 against 178 at the start of the battle), GPU time unchanged. + +### Buffer textures at offsets below 16 bytes + +- **Cause:** Metal reports a 16-byte alignment for texture buffer views, and the host used to leave a view at a smaller offset as a dummy. +- **Fix:** on an Apple GPU a view at any multiple of the element size reads the right elements (measured on an M6 with R32Float views at offsets 4, 8 and 12), so the host now creates it there; below the element size it stays a dummy. +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): one at pool offset `0x168`, read by its particle shaders ([Texture slots picked at run time](shaders.md#texture-slots-picked-at-run-time), `shaders.md`). + +### Automatic block height truncates + +- **Symptom:** torn digits and a background in blue steps (Rhythm Heaven Groove's latency calibration). +- **Cause:** NVN chooses the automatic block height by truncating `height_in_blocks * 1.5 / 8`, while `nvn_tex.c` rounded the division up, so some sizes (a 160×160 BC3 texture: H3 instead of H2) were read with the wrong layout; Rhythm Heaven Groove's BC7 textures without a packaged layout were read with the wrong block height. +- **Fix:** truncating height in blocks × 1.5 / 8, as MacNativeGames found: `nvn_tex.c` now truncates for every game (`pick_tile(ah * 3 / 16)`). It changes layouts at 6–10, 17–21, 38–42 and 81–85 block rows, which matters only for data the game lays out itself (no packaged layout: Tears of the Kingdom's flushed textures, Monster Hunter Rise's sparse ones); those were not re-checked. +- **Found on:** found by MacNativeGames on [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) (listed there as open); confirmed by [Rhythm Heaven Groove](../../dm/games/0100D9F01D474000/README.md) on its two 40-block-row textures (two BC7 textures): the rule matches every size checked by eye: 672×160 and 1232×160 (40 block rows: 4 GOBs, where rounding up gives 8), and 112×112 (4), 288×208 (8) and 880×432 (16), where both roundings agree and `floor(rows / 8)`, the other rule that fits 40 rows, does not; and by Touchable (a d2r-mobile port of Tears of the Kingdom) on 16 of its texture files. +- **Check:** a dump of the guest bytes deswizzled in Python with 1 to 16 GOBs and decoded with `texture2ddecoder` shows the right one by eye; a smoothness score does not, because a block height too tall reads padding and looks smooth. +- **Not checked:** the games whose textures the game lays out itself (above). + +### Sparse textures laid out in 64 KB tiles + +- **Symptom:** magenta (the ASTC error colour) on streamed textures in a sparse pool; `DM_NVN_ZBLOCK` finds such textures (all-zero blocks). +- **Cause:** sparse textures have a layout of their own: they are mapped in 64 KB tiles, one row of blocks 128 GOBs / (block height × block depth) wide. The game's files hold its streamed textures pre-swizzled that way, and it places each level at the offset the host reports; the host laid them out unpadded. +- **Fix:** `nvn_tex.c` `level_walk`: a level at least one tile wide, high and deep is laid out with its width padded to whole tiles, the levels from the first smaller one on form the mip tail (packed as usual), and the storage is whole tiles, 64 KB aligned. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md): laid out unpadded, the 1024×1024 ASTC 6×6 clothes of Hinoa and Minoto (43 GOBs wide instead of 48, level 1 at 0x81000 instead of 0x90000) read 2,183 all-zero blocks in level 0 and 3,043 in level 1, which ASTC decodes as its error colour: their trousers, sleeves and gloves were magenta, worst in Kamura Village. + +### Sparse textures bound from their first mapped level + +- **Symptom:** streamed textures read 0 (magenta on Monster Hunter Rise's title screen's clothes). +- **Cause:** the translator drops TEX's LOD clamp and its residency predicate, so streamed textures were sampled at levels the game had not mapped yet. +- **Fix:** a sparse texture is bound as a view starting at its first level whose pages, and those of every smaller level, are all mapped (`sparse_view` in `nvn_mtl.m`). That samples exactly as a minimum LOD would; the view follows each upload. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md). + +### Sampler reduction (min / max) + +- **Cause:** a sampler built with `SamplerBuilderSetReductionFilter` (min / max) was averaged by the host. Metal 26 has the same thing (`MTLSamplerDescriptor.reductionMode`), but ignores it with a nearest or no mip filter. +- **Fix:** reduction samplers filter mips linearly (`samp_by_id`): at the integer LODs reductions are used with, that is the same level. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): its depth pyramid is built with a min-reduction sampler (`SamplerBuilderSetReductionFilter 1`; `151607c7a92386cc` samples the level above at an explicit LOD). + +## Render targets, passes and clears + +### Transient render targets shared by storage + +- **Symptom:** ≈13 MB/frame leak. +- **Cause:** the game builds new texture objects every frame over the same storage and never finalizes them; each new object allocated a new private Metal texture. +- **Fix:** host textures are shared by `(pool, offset, description)`. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Shared-storage entries taken over by a new texture + +- **Symptom:** GPU memory grows with every image shown; the memory line's `shared textures` count keeps rising. +- **Cause:** the game decodes each picture it shows into one scratch region of a pool that lives for good, finalizes that texture when it is done with it and builds the next one in the same region with another size or format; during some transitions it builds a new render target every frame, one pixel wider each time (614x720, 615x720, …). The storage-sharing table (`g_alias`, keyed by pool, offset and description; [Transient render targets shared by storage](#transient-render-targets-shared-by-storage)) kept every one of them until the pool was finalized. +- **Fix:** a new shared texture takes over the entries of its pool whose storage it overlaps under another description (`alias_take_over` in `nvn_mtl.m`), unless a live texture object still resolves to one (Wonder's transient render targets, rebuilt on new objects every frame and never finalized) or a lookup found it in the last 16 submissions (Breath of the Wild finalizes its transient render targets before every submit and builds them again the next frame; targets sharing storage would otherwise evict each other every frame). An entry that only a finalized serial in `g_ghost` still holds loses the table's reference and is retired with that serial. The memory line reports `shared textures N (M of them unreferenced; evicted with pools …, on storage takeover …)`: the unreferenced ones wait for their storage to be reused or their pool to go. +- **Found on:** [STEINS;GATE RE:BOOT](../../dm/games/01008BC024FDE000/README.md) (ASTC 4x4 images up to 4096 wide, RGBA8 render targets, R8 masks): from a quick save, 297 entries and 1.29 GB of GPU memory at frame 6,600; 468 entries and 1.58 GB 24,000 frames into a new game. With the fix: 97 entries and 0.93 GB at frame 6,600; 15 minutes of story from the quick save (53,700 frames) stayed at 1.08–1.15 GB of GPU memory and 1.02–1.13 GB resident, and a new game from boot stays at 0.97–1.03 GB from its first story scene to frame 27,300 (with the 16-submission guard). +- **Check:** Captain Toad: Treasure Tracker's opening course, rendered with agl's transient targets, keeps its 26 entries and takes none over, with the same memory and frame times as before. + +### Render-target views that change the format + +- **Symptom:** the whole picture came out darker and more saturated on the Mac and the iPhone: comparing with a Switch screenshot, every channel at every level equalled the Switch value decoded from sRGB once more (0.55 → 0.26, 0.75 → 0.52). +- **Cause:** Mario's last pass writes the `RGBA8` swapchain texture through an `RGBA8_SRGB` view (`TextureViewSetFormat` 0x38 on a 0x25 texture), so the GPU encodes the linear shader output to sRGB on write. The host used to attach the base texture, which stored the linear values unencoded. +- **Fix:** `attach` now attaches a Metal view of the view's format, so the pipelines get that format too; the view is kept until the submission completes. A format change that is not an sRGB / linear pair needs `MTLTextureUsagePixelFormatView` on the base texture; without it the host draws in the base format and reports `render target view format … cannot be attached`. +- **Present step:** the matching rule: the Switch scans out the stored bytes, so an sRGB-typed swapchain texture (`RGBA8_SRGB` / `BGRA8_SRGB`) is sampled through a non-sRGB view (`present_raw_src`); sampling it directly would decode it once more on screen only, which a `DM_NVN_SHOT` capture (raw bytes) does not show. Mario's swapchain is `RGBA8`, so this path is for other games (`t_present_crop` covers it). +- **Check:** `DM_NVN_PASSDUMP` prints these views as `V <pass> c<i> view format …`; Mario uses only this one (1 of 80 passes in a course frame). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Render-target use added on demand + +- **Symptom:** a clear of, or a format-converting copy into, a texture made without render-target use is declined. +- **Cause:** a whole-level clear and `copy_t2t_draw` (which renders into the destination, [Copies a blit cannot do, drawn](#copies-a-blit-cannot-do-drawn)) need a texture with render-target use. +- **Fix:** the texture is upgraded to render-target use first, as format views are upgraded (`tex_upgrade`; [Integer views for images a shader declares uint or int](#integer-views-for-images-a-shader-declares-uint-or-int)). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (the clear of a 32x80 packaged texture); [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (a format-converting copy of a 128x128 RGBA8 render target into a packaged RG8 texture, mirrored in Y, as the boat arrives in Rogueport, about frame 10,950: `copy_t2t_draw` declined a destination made without render-target use). + +### Compressed textures as render targets through an uncompressed shadow + +- **Symptom:** `render target is a compressed-format texture …`; the pass's draws did nothing (the attachment was left out), or the pipeline-descriptor validation aborts the process. +- **Cause:** a GPU block encoder writes a BC texture through an uncompressed integer view (one texel per block). Metal can neither attach a compressed format nor make an uncompressed view of one. +- **Fix:** such a pass draws into an uncompressed shadow of blocks × blocks in the view's format: the level's (or layer's, or depth plane's) blocks are copied into it through a buffer before the pass and back after it (`bcrt_attach` / `bcrt_flush` in `nvn_mtl.m`; the method comes from PR #11). The shadows stay resident and are reused, and they are kept out of the load-skip and store-elision bookkeeping. `attach`, the pipeline prediction and the sampling views go by the NVN format, so BC textures are never attached or viewed raw. The encoder's output must be declared `uint` ([Integer fragment outputs declared by hand](shaders.md#integer-fragment-outputs-declared-by-hand)). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): agl's BC4 block encoder (`8e2cc2671cae6020`) writes a 32×32 RGTC1 (BC4) texture through an `RGBA16UI` view (one texel per 8-byte block); its twin for a 2D source (`93aa5e22a8202409`) failed in an iPhone run; GitHub issue #4 (reported from an iPhone 14 Pro / A16). [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md): agl's BC4 block encoder (`0544282c162305db`, the same shader Mario Wonder ships) encodes a 32×32×32 BC4 3D texture one depth plane per pass through an `RGBA16UI` view; a raw dump at frame 1500 of the new-game route: the texture was all zeros before, now all 32 planes hold encoded blocks. +- **Note:** the A16 has no BC formats; the host used to store BC textures decompressed on such GPUs (about 3.9 GB instead of 2.8 GB for BotW, over iOS's limit on a 6 GB iPhone), and that path has since been removed: only GPUs with BC are supported. + +### A depth texture drawn through a colour view + +- **Symptom:** the boot aborts in Metal's pipeline validation: `MTLPixelFormatDepth16Unorm is not color renderable`. +- **Cause:** a depth texture drawn through a colour view (a 1024×1024 D16 texture through an R16 view); Metal has no colour view of a depth texture. +- **Fix:** such a pass draws into a colour shadow of the same pixel size, copied in before and back after the pass through a buffer, as a compressed render target is ([Compressed textures as render targets through an uncompressed shadow](#compressed-textures-as-render-targets-through-an-uncompressed-shadow); `shrt_block`; D16 and D32F, whose Metal pixel is the NVN pixel; not D24S8). +- **Found on:** [Luigi's Mansion 3](../../dm/games/0100DCA0064A6000/README.md). + +### The same texture attached at two colour slots + +- **Symptom:** a G-buffer target holds only scattered squares (a different order for each 32×32 tile). +- **Cause:** on Maxwell the ROP writes only masked channels, so two slots holding the same texture with one at write mask 0 do not interfere. On Apple's TBDR each attachment has its own tile memory and the whole tile is stored at the end of the pass, so the copy stored later (loaded, never written) overwrote the other one's writes. +- **Fix:** a slot whose texture, level and layer repeat an earlier slot is left unattached (`rt_dup_slot`), also in pipeline prediction; a draw that really writes such a slot is reported once (`color slot … holds the same texture as an earlier slot`). Records hold copies of texture objects (`tex_snap`), so slots are compared by serial, not by pointer. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): the G-buffer passes attach the albedo and another buffer twice each (c1 = c2, c3 = c4), the duplicate slots with write mask 0; the albedo held only scattered squares. + +### Passes with no render target + +- **Symptom:** `no render target could get an MTLTexture` (the pass skipped). +- **Cause:** a fragment shader that only stores to a storage buffer draws into a viewport with no colour or depth target set. +- **Fix:** the host opens an attachment-less pass at the viewport's size (`bare_dims` in `nvn_mtl.m`, from PR #11). +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md): an agl fragment shader draws one triangle into a 1×1 viewport (it samples a 64×64 cube map, and the game reads the result back from the buffer). Compared at the same frames of the new-game route, the cave's rock and stone blocks now get ambient light they lacked, and Zelda's bloom halo in the descent cutscene is gone. + +### Layered passes for vertex shaders that write the layer + +- **Symptom:** a black world: everything rendered through a 3D LUT built by layered draws comes out black (every plane went to plane 0); `DM_NVN_PASSDUMP` shows no `L` line. +- **Cause:** a Metal vertex shader selects the attachment layer per primitive only when the render pass has `renderTargetArrayLength`. +- **Fix:** a draw whose vertex shader writes the layer opens a layered pass (`g_rp_layered` in `draw`, `g_pass.layered` in `rp_begin_ex`): the pass gets every layer the attachments have from their attached one on (3D textures: depth planes), and it is split when the next draw's shader differs. `DM_NVN_PASSDUMP` prints `L … layered pass: n layers`. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): the 3D scene came out black once the opening reached Lumiose. The game builds its 32×32×32 colour grading LUT with `b29f08619f379a83`, one instance per depth plane, each writing its plane through `render_target_array_index`, and tone-maps the frame through that LUT. + +### Lazy clears folded into the next pass + +- **Fix:** lazy clears (`DM_NVN_LAZY_CLEAR`) fold clear-only passes into the pass that follows. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): the four clear-only passes before the G-buffer fold into its first pass (about 30 MB less loaded and stored a frame at 1600x900). + +### A channel mask that covers every channel is a whole clear + +- **Symptom:** `texture clear case not implemented … mask 0x1`. +- **Cause:** an R8 render target cleared with channel mask 0x1. +- **Fix:** a mask that covers every channel the format has is a whole clear (`mtl_pf_chmask`). +- **Found on:** [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md). + +### Scissored colour clears + +- **Cause:** NVN `ClearColor` honours the scissor. Clears are normally folded into the next pass's load action, which only works for a full-target clear; Mario clears sub-rectangles of small targets to zero (pyramid levels packed into 480×272, 256×256 and 64×36 targets). +- **Fix:** a zero clear outside a pass with a partial scissor is now a copy from the shared zero buffer into that rectangle of the attachment's level / slice (`clear_rect_zero`). +- **Check:** `DM_NVN_CLEARLOG=1` logs clears that are still skipped (mid-pass, partial channel mask or non-zero colour with a partial scissor). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Clears inside a pass, with a partial mask or a partial scissor + +- **Symptom:** red lines for unimplemented mid-pass, partial-mask and partial-scissor clears (the clears were dropped); `depth-stencil clear with a scissor or partial stencil mask, cleared whole` (logged once). +- **Cause:** NVN clears obey the scissor and the clear's own channel or stencil write mask, and can come between draws, while a Metal load action clears a whole attachment at the start of a pass. +- **Fix:** such clears are drawn as one triangle over the pass in the open render encoder (`clear_color_tri` / `clear_ds_tri` in `nvn_mtl.m`, adapted from MacNativeGames, MIT). The pipeline has the pass's formats, the clear's mask as the write mask of its slot and no writes to the other slots. Depth is `ALWAYS` (and written for a depth clear), stencil `REPLACE` through the clear's write mask, the scissor is clamped as for draws, and visibility counting is off for the triangle. The value comes from the frame arena through the triangle's own argument tables, and the draws' tables are put back after it. A clear outside a pass opens the pass as a draw would. Clears a load action can express keep it. Two cases are still cleared whole, logged once: a depth / stencil clear with a scissor outside a pass, and a partial stencil write mask. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md): it issues the first whole-cleared case (a depth clear scissored to 8×8 of a 64×64 target), and so does Wonder ([Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)). With `render_cond` ([Conditional rendering evaluated at replay](#conditional-rendering-evaluated-at-replay)) TOTK's scissored colour clears are all skipped; without it they are 50×50 tiles of a 500×400 target cleared to opaque black, about five a frame. [Metroid Dread](../../dm/games/010093801237C000/README.md) logged the red line once while loading on its smoke route. + +### Integer colour clears + +- **Cause:** `nvnCommandBufferClearColori` / `ClearColorui` clear an integer target. +- **Fix:** recorded as a float clear; both replay paths convert the value to the attachment's integer type. +- **Found on:** [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (an integer target cleared once Mario stands on Rogueport's docks, about frame 12,300 of a new file). + +### Same-pass feedback read through a snapshot + +- **Symptom:** solid magenta blocks, exactly (255, 0, 255), hard-edged and aligned to the 16-pixel grid (16×16, sometimes two stacked): Super Mario 3D World showed them only on the iPhone; Captain Toad on a Mac, Apple M6, in about half the frames of the opening course, moving with the falling leaves. +- **Cause:** a program writes an attachment while sampling it (agl's post-effect program writes the 1920x1080 RGB10A2 image as an attachment while sampling it). Writing and sampling one attachment within a render pass is undefined in Metal: the M4 Pro happens to return the old content, while the A20 Pro now and then returns whole 16×16 blocks as (1, 0, 1), probably blocks of the compressed render target caught mid-write. On the Switch the read returns the content from before the draw. +- **Fix:** for the fragment shaders the profile's `feedback_snap` key lists, the host ends the pass before such a draw, copies the attachment's level and layer into a snapshot texture in the replay stream, and binds the snapshot for that draw (`fb_prepare` in `nvn_mtl.m`). A sampled view that spans more levels than the attachment's one is left as before with a red line. +- **Found on:** + - [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md): `DM_NVN_MAGSCAN` on the A20 Pro found the blocks first in the 1920×1080 RGB10A2 composite (`ser=30`) in World 1-1, in the pass whose shader `66a71a3081b91b9a` renders into that texture while sampling it. They kept appearing with the sparkle particle `5e4de7561c670fee` dropped, so that earlier suspect (picked by position) is not the cause. `feedback_snap` lists `66a71a3081b91b9a` and `30e3ccb0bd304ead` (which samples a G-buffer target of its own pass on the title screen); in 1-1 that is one 8 MB copy per frame. `3e8567da37eb28a1` renders level 1 of a texture while its view covers levels 0–4 (mip-chain passes) and keeps the red line. The source said the fix was in place and waiting for a device run when written; the README lists the game as tested and playable on the iPhone. + - [Captain Toad: Treasure Tracker](../../dm/games/01009BF0072D4000/README.md): agl's post-effect program 987 (`54f58a7cab608766`, the same program as 3D World's `66a71a3081b91b9a` in another build); `DM_NVN_FB_SNAP=all` cleared every frame and named this one shader, now in the profile's `feedback_snap`. +- **Note:** doing it for every shader is too expensive: Breath of the Wild has over a thousand draws a frame that sample the G-buffer of their own pass (Mac frame time 16.6 → 36.6 ms), and Mario Wonder's submit time went from 1.4 to 4.0 ms. Those games' feedback draws (Mario Wonder `a7e5c6fd501cf21f` and `22a61cb1ec732aef` on its RG11B10F scene; BotW `4d3d5a4408c2e8d9`, `753d0169ae3ce4f0` and others on its 1600×900 G-buffer) had shown no blocks on the iPhone; if they do, `DM_NVN_MAGSCAN` names the pass and the shader goes into that profile's `feedback_snap`. `DM_NVN_FB_SNAP=all` snapshots every such draw. + +### Lower-left window origin + +- **Symptom:** text a game renders into textures for the title and the menus comes out upside down, while the scene looks right. +- **Cause:** a game that never calls `nvnDeviceSetWindowOriginMode` keeps NVN's default, lower-left (OpenGL), where NDC y = -1 lands on a render target's first memory row. Read as upper-left, every texture the game renders and samples came out flipped and cancelled out, but not the text, and the game copies the frame with `MIRROR_Y`. +- **Fix:** viewport and scissor rectangles, fragment coordinates, copies and texture coordinates address the same memory rows in both conventions, so the host only flips the direction inside the viewport (negative viewport height), reverses the front-face winding, and flips the window image at present and in raw screenshots (`g_origin_ll` in `nvn_mtl.m`). +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md); the six games before it all switch to upper-left. +- **Note:** the window's Y-direction system register under this origin is open ([Open: the window Y-direction register `SR_Y_DIRECTION`](shaders.md#open-the-window-y-direction-register-sr_y_direction)); the OpenGL layer renders through this origin ([GL orientation through the lower-left origin](#gl-orientation-through-the-lower-left-origin)), and Absolum combines it with a viewport swizzle ([Viewport swizzles applied as a flipped viewport](#viewport-swizzles-applied-as-a-flipped-viewport)). + +### The residency set committed before a present + +- **Symptom:** a pink window with every command buffer ignored while headless runs look right: the first presents fail on the display command buffer with `kIOGPUCommandBufferCallbackErrorPageFault`, and after a few of those macOS ignores every submission of the process ("Ignored (for causing prior/excessive GPU errors)"). +- **Cause:** the game presents its window texture in its first frames before drawing into it; the texture was made on that first resolution in the present path and added to the residency set (`res_add`), but only a submission commits the set, so the display command buffer sampled a texture outside the committed set. Headless runs never present, which is why they looked right. +- **Fix:** `present_now` commits the residency set after resolving the presented texture (`res_commit`, free when nothing changed), for every game. +- **Found on:** [Super Smash Bros. Ultimate](../../dm/games/01006A800016E000/README.md) (13.0.5, `DM_WINDOW=1`). +- **Check:** Breath of the Wild windowed with the change: its title, 60 fps, no GPU error. + +## Blend, depth and stencil + +### No writes to colour outputs the shader does not write + +- **Symptom:** undefined values blended into attachments the fragment shader leaves alone. +- **Cause:** Maxwell leaves targets outside the shader's output map untouched; on Metal such an attachment gets undefined values blended in. +- **Fix:** pipelines get a zero write mask and no blending for colour outputs the shader does not write (reflection `colors`, `color_known` in `nvn_refl.m`). +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md): volume draws after the G-buffer (`50d813761ca3f1f9` and others) write only outputs 0 and 2, with blending and all channels enabled on targets 1 and 3. +- **Check:** all five games' regressions were unchanged by it. + +### Advanced blend modes as Porter-Duff factors + +- **Symptom:** `nvnBlendStateSetAdvancedMode 8` (SRC_OVER) logged and blended with the state's own functions. +- **Cause:** the hardware ignores the blend state's own functions while an advanced mode is set. +- **Fix:** the blend state keeps the mode, the premultiplied-source flag and the overlap (`blend_st` in `nvn_obj.h`), and the pipeline (`adv_blend` in `nvn_mtl.m`) blends the Porter-Duff modes (ZERO, SRC, DST, SRC/DST_OVER, _IN, _OUT, _ATOP, XOR, the PLUS modes) with the fixed-function factors they amount to under uncorrelated overlap, with premultiplied colours as NV_blend_equation_advanced defines them; a source that is not premultiplied is multiplied by As where the source factor is ONE. Multiply, screen, overlay, the HSL modes and the other overlaps need the destination colour in the shader and keep the state's functions, with a red line naming the mode. +- **Found on:** [Mario vs. Donkey Kong](../../dm/games/0100B99019412000/README.md) (SRC_OVER, on one blend state in the levels). + +### Depth clamp + +- **Fix:** `SetDepthClamp` is recorded (`OP_DEPTH_CLAMP`) and replayed as Metal's depth clip mode (it used to be ignored). +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md) (the game turns it on). + +### Telling depth rejection from no rasterization + +- `DM_NVN_DEPTHOVR` (depth test forced to `ALWAYS`) with `HILITE` tells "rejected by depth" from "never rasterized". `TEXSNAP` copies used to run before the pass they were meant to show (classic-queue command buffer committed ahead of the Metal 4 one); they now go through the replay's copy encoder. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md), tracing the tile lighting: the G-buffer class byte selects per-class half-resolution passes through a depth buffer of class values (`gl_FragDepth = mix(0.5, 1, nibble-swapped class / 255)`, then full-screen or per-tile quads with depth `EQUAL`). + +### Stencil-mode texture views read through `X32_Stencil8` + +- **Symptom:** striped lighting: black and white bands that follow depth on every lit surface, as if every pixel were in or out of shadow at random. +- **Cause:** a shader that reads a depth-stencil texture through a view whose `nvnTextureViewSetDepthStencilMode` is `STENCIL`: the host ignored the mode and bound depth, so a stencil test read bits of the depth value's float bits. +- **Fix:** `tex_by_id` binds an `X32_Stencil8` view for such a view. The shader declares a float texture (the translator has no integer texture type) and takes the raw bits with `as_type<uint>`, and a read of the integer view gives the stencil value in those bits. Metal allows such a view only of a texture with the pixel-format-view usage: Apple GPUs read it correctly without, but the API validation layer aborts at the first one (PR #106, Luigi's Mansion 3). Upgrading the texture when the view appears recreated the depth buffer while it was attached, and the next occlusion-counted draw crashed inside `setVisibilityResultMode`, so every depth-stencil texture has that usage from the start (`tex_desc`; Vulkan depth-stencil images too, `vkCreateImage`), which costs nothing measured (on an M4 Pro the same allocated size and GPU time); `t_stencil_view` checks it under validation. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): the deferred lighting pixel shader (`ead6e2a32540b19a`, 31 bindings) first reads the depth-stencil buffer through such a view, tests stencil bit 6 and discards. [Luigi's Mansion 3](../../dm/games/0100DCA0064A6000/README.md) (PR #106, the validation abort). +- **Note:** inputs ruled out on the way in Z-A: the G-buffer albedo, the normals (12-bit pairs packed into 3 bytes, noisy by design), the screen-space shadow mask, the scaled depth copies and conditional rendering (`render_cond` changes nothing in this game and stays off). + +### Stencil-mode textures from the texture builder + +- **Symptom:** a character drawn as a dithered silhouette inside a dithered square (DRAGON QUEST VII Reimagined's hero). +- **Cause:** Unreal's stencil shader resources are second textures on the depth buffer's storage built with `nvnTextureBuilderSetDepthStencilMode(STENCIL)`, which the host only logged, so the character occlusion mask read depth where it tests stencil bits. +- **Fix:** the builder's mode now goes to the texture (`tex_t.stencil`), whose sampling view is the `X32_Stencil8` one a stencil-mode texture view already got. +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (the character occlusion mask, `edbfc7a088be545e`). + +## Vertex input and draws + +### Integer vertex inputs + +- **Symptom:** characters (skinned models) invisible or stretched into streaks while the level renders; `vertex attribute format … does not match shader input type …` for a remaining mismatch. +- **Cause:** integer vertex inputs (bone indices) translated as float: without PC reflection the translator declares every vertex input `float4`. Skinned models read their bone index (`R8UI` / `RG8UI` / `RGBA8UI`) as raw integer bits (`as_type<int>(in.m_N.x) * 48`); Metal converted the integers to floats, and every bone index was garbage. +- **Fix:** `dmsh_build.py` (`int_inputs`) now re-translates a vertex shader with `uint` inputs when every use of an input is an integer bit cast (529 of 4,854 in Mario), and the host makes the Metal vertex format agree with the declared type (`vfmt_for_ct`; signedness swaps of 32-bit components and 32-bit float→int reinterpretation keep the bits, while float-type formats and 8- and 16-bit integers of the other sign go through [shadow vertex inputs](#shadow-vertex-inputs), next entry). A remaining mismatch is reported as `vertex attribute format … does not match shader input type …`. +- **Check:** finding the draws: log model file ranges with `DM_PROBE=main:480f0` (x1 = file data, x2 = size), match the vertex/index buffer addresses in `DM_NVN_PASSDUMP_ALL` against them, then `DM_NVN_HILITE` their fragment shader. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (skinned characters). + +### Shadow vertex inputs + +- **Symptom:** red lines 89 and 96 (`vconv_bind`, the CPU conversion every draw); about a quarter of Age of Calamity's replay thread in battle. +- **Cause:** an input that `int_inputs` declares an integer may still be fed a format of another class. A float-type format (half, unorm / snorm, 10:10:10:2, 11:11:10) leaves the converted float32 bits in the register; an 8- or 16-bit integer of the other signedness is widened by the format's own sign (Fire Emblem: Three Houses, [CPU conversion of vertex attributes the input type cannot take](#cpu-conversion-of-vertex-attributes-the-input-type-cannot-take)). Metal reads a format only into an input of its own class, so the replay used to convert those attributes on the CPU every draw (`vconv_bind`). +- **Fix:** shadow inputs (`<key>_bv`, `dm/gfx/tools/dmsh_vin.py`): the build also writes a side library for each such vertex shader. In it, every integer input is declared again as a `float4` and as the integer of the other sign, at Metal attribute indices 16–30 (NVN uses 0–15), and the body starts with `in.m_N = dm_in.m_N | as_type<T>(dm_in.m_N_vf) | as_type<T>(dm_in.m_N_vo)`. The host sends each attribute to the index of its format's class and the zero buffer to the others (`vin_ct`, `vin_on` in `nvn_mtl.m`; pipeline keys unchanged), so a pipeline with any mix of formats reads Maxwell's register bits. The float shadows are assigned first: with more than 7 integer inputs the last ones get only the float shadow (169 of 8,201 inputs in Age of Calamity). +- **Note:** a format with no shadow keeps the CPU conversion, as do transform feedback draws (`main0_tf` has no variant) and `DM_NVN_VCONV_CPU=1`. The host loads a side library only for a pipeline that routes an attribute to a shadow, but the build cannot tell which shaders will need one (an input used only as an integer may hold bone indices or float bits), so it writes them all. Shadows cover only integer inputs, so 8- and 16-bit integer formats feeding float inputs still go through the CPU. +- **Found on:** + - [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md): 2,259 libraries, 77 MB each for Mac and iOS. Its Mac battle at 60 fps against the CPU conversion: 0 vertices converted instead of 545,000 per submission, submit + present 10.0 → 5.7 ms a frame, CPU 3.97 → 2.44 W, GPU unchanged, the same picture. + - Other games have 0–2,000 libraries (at most 31 MB on iOS), except [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md): 16,781 libraries, 344 MB on iOS (21% of its shader libraries, 1.5% of its IPA), and its Mac and iPhone runs so far load none. Its 8- and 16-bit integer formats feeding float inputs (red line 99, about 60,000 vertices per submission on its Mac route into the opening) still go through the CPU. + +### CPU conversion of vertex attributes the input type cannot take + +- **Symptom:** `Vertex attribute m_36(5) of type uint4 cannot be read using MTLAttributeFormatHalf2` (draws dropped: black silhouettes); UVs collapse onto one texel (flat tan characters); vertices far out of view (negative 16-bit values widened with zeros). +- **Cause:** `int_inputs` declares an input `uint` when its first translation only bit-casts it, and the shader then takes it back with `as_type<float>`. Maxwell's fetch converts by the format and the register holds the float32 bits, and widens integers by the format's signedness, whatever type the shader gives the register. Metal refuses a float-type format for an integer input; reading the raw bytes as integers gives denormals. For a signed format on an unsigned input `vfmt_for_ct` swapped `Short4` for `UShort4`, which keeps the 16 bits but widens them with zeros, so every negative value read as 32,768–65,535. +- **Fix:** the replay converts such attributes (half, unorm / snorm 8 and 16, 10:10:10:2, 11:11:10, BGRA8 into an int / uint input) on the CPU into float32 bits, four components with the fetch's (0, 0, 0, 1.0) defaults, in the frame arena, and binds them as UInt4 / Int4 on buffer slots 1–12 (`vconv_bind`, once per range and layout per submission; red line 89). 8- and 16-bit integer formats whose signedness differs from the input's go through the same conversion (`vconv_int` in `nvn_mtl.m`, red line 96 once): 32-bit values extended by the format's own sign, missing components (0, 0, 0, 1) as integers. +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md): KTGL's character and soldier shaders, fed a Half2 (UVs); also [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md)'s soldiers (flat tan). The ground under close-up battle cameras was flat steel blue: the terrain's vertex shader (`56778feb1cfdee7e`) reads its grid position from an `RGBA16I` attribute (x and z the grid cell, y the height, which spans the whole int16 range), and so does the side pass that draws the river surface into targets of its own (`db725c85ca060403`). +- **Note:** [shadow vertex inputs](#shadow-vertex-inputs) (`dmsh_vin.py`) have since replaced both conversions; they remain for the formats shadow inputs cannot take (and for transform feedback draws and `DM_NVN_VCONV_CPU=1`), and the profile-side answer for one shader is [Integer-typed vertex inputs fed normalized formats](shaders.md#integer-typed-vertex-inputs-fed-normalized-formats). +- **Note:** how the steel-blue ground was traced, since two steps misled: where the ground was missing nothing wrote the depth (reversed-Z, cleared to 0), the lighting takes depth 0 for sky and paints the fog colour, and the never-cleared G-buffer colour targets kept an old image there (cobblestones of a mesh under the terrain, which from the map camera showed through the gaps like a road). A `HILITE` colour over the gap was stale too (the never-cleared albedo), and the end-of-frame depth showed 0 there. Recomputing every draw's clip positions from `CBDUMP` data showed that no draw of the shader reached the gap; then the side pass's layer, which showed only scattered tiles of the river, led to the vertex format. Ruled out on the way: depth clamp (Metal's clip and clamp modes gave the same depth), the pass without a depth attachment (142595 is a screen-space decal pass: it samples the depth and writes albedo and normals), polygon offset (none), dropped draws. +- **Note:** to confirm in Three Houses: 64-pixel rectangles in the albedo where decals land. The decal pass (`417a523053457352`, slots 16 and 18) and the lighting (`094a2d11253f126c`, `f54830a75e07e14a`, slots 18–20) read their decal and light lists from texture buffers that the translation declared `texture2d`, because those handles have no PC type (`texture type mismatch: shader declares 2, texture is 9`); skipping the decal pass removed the rectangles. Since row wrap ([Buffer textures read by 1D integer fetches (row wrap)](#buffer-textures-read-by-1d-integer-fetches-row-wrap), general) they read the real lists: slot 18 holds 1,052,640 elements, 65 rows. On the Mac prologue route the torches and burning houses now light the battle map and the close-up ground (they lit nothing before), and the blood decals take the scene's light instead of showing as dark blue patches. The rectangles did not appear on that route in either build (no 64-pixel structure in the close-up frames), so the scene where they were seen still has to be checked. + +### Vertex formats Metal lacks + +- **Symptom:** `vertex attribute format … not mapped` (printed once). +- **Cause / Fix:** Mario uses `RGB10A2SN` (0x69) → `MTLVertexFormatInt1010102Normalized`. Unmapped formats fall back to a 4-component read and print the line once. The integer-read-as-float formats (`*_UI2F` / `*_I2F`, 0x51–0x68), which Metal has no vertex format for, are converted on the CPU at replay like the other conversions (`vconv_i2f`). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (`RGB10A2SN`); [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) has an `RG8_UI2F` attribute. + +### Vertex streams 14 and 15 at Metal slots 14 and 15 + +- **Symptom:** the assertion `buffer index (31) must be < 31`. +- **Cause:** the host bound NVN stream *n* to Metal vertex buffer 16 + *n*, but Metal has slots 0–30 only (30 is the host's constant fallback), so stream 15 went to 31. +- **Fix:** translated shaders use only `buffer(0)`, so streams 14 and 15 now go to slots 14 and 15 (`VB_SLOT`). +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (a minute and a half into the prologue). + +### Programs bound per stage + +- **Symptom:** scenes and characters black, only foliage, light shafts and the HUD drawn; pass dumps full of `Z … draw skipped: no program bound`. +- **Cause:** NVN's `BindProgram(program, stageBits)` binds per stage, and a null program unbinds just those stages. Metroid Dread's depth prepass calls `BindProgram(program, vertex | fragment)`, then `BindProgram(null, fragment)`, and draws depth with the vertex shader alone. The host replaced its single bound program on every bind, so a null bind meant "no program bound", and all of the prepass's ~140 draws per frame were dropped. The depth buffer stayed empty, and the G-buffer passes, which test depth `EQUAL` without writing, passed no pixel. +- **Fix:** the replay tracks the program per stage (`bind_stages` in `nvn_mtl.m`). When the stages come from different programs, or the fragment stage is unbound, it builds a composite program, which gives a vertex-only pipeline for the prepass. Games that always bind every stage from one program (Mario, Engage) are unchanged. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md): the combinations it uses are (program, 0x3), (null, 0x2), (null, 0x7). [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) builds one program per GLSLC output and pairs them only through these composites ([Single-stage programs paired with a cover](shaders.md#single-stage-programs-paired-with-a-cover)). +- **Note:** pass dumps print a `Z` line wherever a draw is dropped (no program, no pipeline, empty scissor, both faces culled, no render pass). That is how this one was found: 144 `Z … draw skipped: no program bound` lines in a frame whose G-buffer pass had draws but stayed black. + +### Composite program taken again after a replay state restore + +- **Symptom:** `[RED] failed to create render pipeline (vertex 790573a52b9c5053_b fragment f39fb42850f50ecd_b): Fragment input(s) user(locn0),user(locn1) … not written by vertex shader`, once on entering the first investigation; `texture type mismatch: shader declares a cube, texture is 2D` (a pixel shader run with another pair's bindings). +- **Cause:** a draw ran with the vertex shader of another Shader object (`Legacy Shaders/VertexLit`'s shadow caster with the pixel shader of `PSYNC/BG_AlphaTest`, whose own pass pairs it with `00b462fe0dec6e06`). Unity binds the vertex and fragment programs apart, so every draw's pair is a composite of `bind_stages`, one of a ring of 64. The precompile prediction (`pso_predict_rest`, at a submission's first pipeline miss) saves the replay state, replays the rest of the submission's binds and restores it; in a 280-draw submission it made more than 64 composites, reused the slot the saved `S.prog` named, and the draws after it up to the next bind ran with that slot's new pair. The fusion probe and redraw and the deferred-draw replay restore `S` from copies the same way. +- **Fix:** each restore takes the composite again from the per-stage bindings it restored (`prog_resolve`, logged as `combined program taken again after a state restore`); a draw already under way keeps its `NvProg`, whose release is deferred. Every game. +- **Found on:** [AI: THE SOMNIUM FILES](../../dm/games/010089B00D09C000/README.md) (Unity 2017.4). +- **Check:** two runs each from one save with one key script, prewarm off: the old host logged the `texture type mismatch` line both times, the fixed host no red line, with the slot taken again twice. + +### Compute programs bound apart from graphics programs + +- **Symptom:** `no program bound before the draw`, `program has no vertex shader` (a draw after a dispatch is dropped); `the dispatched program has no compute shader` (a dispatch after a graphics bind). +- **Cause:** `BindProgram(program, stageBits)` with the compute bit replaced the one bound program, so a draw after a dispatch found the compute program (or none), and a dispatch after a graphics bind found no compute shader. +- **Fix:** the replay keeps the compute program apart (`S.cprog` in `nvn_mtl.m`, and `g_pp_cprog` in the pass analysis). +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md): the game interleaves dispatches and draws; its intro movie's NV12 → RGB draw was dropped and the movie played black under its "+ Skip" prompt. + +### Instanced array draws + +- Instanced array draws (`DrawArraysInstanced`) carry instance count and base instance in the `OP_DRAW` record. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### 8-bit index buffers + +- **Cause:** Metal has only 16/32-bit indices. +- **Fix:** 8-bit indices are widened to 16-bit at replay into the frame arena. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md): its full-screen passes and its final copy use them. + +### 8-bit indices widened on the GPU after a write in the same submission + +- **Symptom:** a red line: the 8-bit index buffer had GPU-side writes in the submission (indices read stale). +- **Cause:** Metal has no 8-bit index type, so replay widens 8-bit indices to 16-bit on the CPU. When a copy, update or fill encoded earlier in the submission covers them (`pend_hits`), the CPU reads the bytes from before that write. +- **Fix:** those draws are widened on the GPU (`wide_run`): a compute pass reads the indices from a 16-byte-aligned address, with the byte shift as a constant, and writes 16-bit ones into the frame arena. As in `ixc_run`, the first such draw widens every such draw after it up to the next buffer write outside a render pass, so only a draw met inside an open render pass with no widening yet would split it. Draws whose indices no write covers keep the CPU widening. Log line `8-bit index GPU expansion`. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md): about one draw per widening, up to a dozen widenings a frame on the title and load screens, no pass split. + +### Triangle fans and indexed quads + +- **Cause:** Metal has neither triangle fans nor quads. +- **Fix:** non-indexed fans draw through a shared `(0, i, i+1)` index buffer with `first` as the base vertex; indexed fans and quads are expanded on the CPU into a 32-bit triangle list in the frame arena at replay (`nvn_mtl.m`, as for 8-bit indices). +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md). + +### Indirect draws and dispatches read by the GPU + +- **Symptom:** indirect draws whose record the GPU writes earlier in the frame cannot be read on the CPU at replay; GPU-driven effects drawn as indirect quads, which Metal has no primitive for. +- **Cause:** `nvnCommandBufferDrawArraysIndirect`'s record `{count, instances, first, base instance}` has the layout of `MTLDrawPrimitivesIndirectArguments` (and `DrawElementsIndirect`'s `{count, instances, first index, base vertex, base instance}` that of `MTLDrawIndexedPrimitivesIndirectArguments`), but quads, fans and indexed draws with 8-bit indices have no Metal form. +- **Fix:** the replay issues Metal indirect draws from the pool buffer. `ixc_run` in `nvn_mtl.m` converts records of quads, fans and indexed ones with 8-bit indices with a compute pass into indexed indirect arguments in the frame arena, and the draw indexes the shared quad / fan index buffers (indexed ones get a 32-bit list expanded on the GPU). A compute pass cannot run inside a render pass, so when the first such draw is met it converts every such draw up to the next buffer write outside a render pass (copy, update, fill, texture → buffer, dispatch; they bump `g_wr_epoch`). Each record is then read after its last writer, and only a draw met inside an already open render pass splits it. `DispatchComputeIndirect` → `dispatchThreadgroupsWithIndirectBuffer`; when the shader reads the driver segment, the GPU copies the counts into c0 + 0x200 first. +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md) (its GPU-driven effects are indirect quads). +- **Check:** `DM_NVN_IXCLOG=<n>` logs the first n conversions (record → arguments) once the GPU has run them. + +### Indirect-count multi-draws + +- **Symptom:** an empty G-buffer pass (the title screen showed only the UI); later, the submitting thread waiting for the GPU before each such draw (36–54 fps in Kamura Village on an iPhone). +- **Cause:** `nvnCommandBufferMultiDrawElementsIndirectCount` / `MultiDrawArraysIndirectCount` were empty. The scene is GPU-driven: compute culling writes the draw records (stride 20) and a u32 count, then up to `max` draws (1–49 per call) are made from them. Metal has no draw count read by the GPU. +- **Fix:** each call is recorded as `max` ordinary indirect draws that carry the count's address and their index (capped at 1,024). A compute pass copies each such draw's record into the frame arena, or zeros past the count, and the draw reads the copy, so a draw past the count draws nothing (`mdx_run` in `nvn_mtl.m`). Like `ixc_run`, it runs at the first such draw with no copy and takes every later one up to the next write that could change a record or count; copies are keyed by record and count address, since the game draws the same records with two counts. Draws of one multi-draw replayed back to back are encoded without the rest of `draw` (`mdx_next`). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md). +- **Note:** the replay used to commit its work and wait for the GPU before the first such draw after each GPU write, then read the count on the CPU (`mdi_wait`, now only for quads, fans and 8-bit indices). In Kamura Village that wait held the submitting thread 14 of its 20 ms a frame on an iPhone (36–54 fps). On the Mac the same scene's submit took 20 ms of wall time for 6.1 ms of work, and now 5.1 ms, all of it work. + +### Transform feedback captured by a vertex entry point + +- **Symptom:** characters collapsed into one thin spike (vertex streams read from feedback buffers that were never written); a pass dump shows a pass of draws with a vertex shader and no fragment shader (`fs=-`) before the pass that reads them. +- **Cause:** a game that skins on the GPU with transform feedback: a vertex-only program runs under `SetRasterizerDiscard` between `BeginTransformFeedback` / `EndTransformFeedback` and writes into feedback buffers that later draws read as vertex streams. The host ignored all of it, so the streams kept whatever was there. Metal has no feedback stage. +- **Fix:** + - the layout comes from the program's control section, where the NVN driver would read it (`tf_layout` in `dmsh_build.py`): +0x6dc u16 strides, +0x6e8 per-buffer enables, +0x6ec buffer count, and per buffer b the captured output (0x20 + n) at +0xdc + 0x80 b and its component count at +0x4dc + 0x80 b. Any form other than one output per buffer from x on gets a red line and no capture entry point, so one shader cannot stop a game's build; + - such a vertex shader gets a third entry point, `main0_tf` (`vs_tf`): the same body, returning nothing, storing the captured components as their bits into `device uint *` buffers at Metal vertex buffer slots 8–11 (`TF_SLOT`), indexed by vertex ID; the reflection lists `entry_tf` and `tf_strides`; + - the binds, begin / end / pause / resume and rasterizer discard are records (`OP_TF_BIND`, `OP_TF_STATE`, `OP_RAST_DISCARD`). At replay a draw under rasterizer discard runs `tf_draw` in `nvn_mtl.m`: with feedback active and a capture entry point, an encoder of its own with no attachments, a pipeline with rasterization off, the buffers bound at their current offset less first × stride, the offsets advanced and the written ranges queued as GPU writes (`pend_add` kind 4); without feedback the draw rasterizes nothing, as on the GPU (it used to draw, depth writes included). Only plain array draws are captured; indexed, indirect or instanced ones give a red line. `DM_NVN_PASSDUMP` prints the records as `Y tf …` and each capture as a `T … tf capture` line. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): the character preview. 28 such shaders (e.g. `681cc42e48c0bbc0` in the `special/imgui.trsha` pack; skin indices in attribute 12, weights in 13) write positions (location 0 xyz, 12 bytes a vertex) and packed normal and tangent bits (locations 1 and 2, x only, 4 bytes) into three feedback buffers; the character draws read them as vertex streams 0–2 (`RGB32F`, `RGB10A2SN`, `RGB10A2SN`). Every buffer in this game holds one output from x on (strides 12/4/4, 12/12/12, 12/4, 12). The game draws these while its frame target is bound. The pass dump showed a pass of 19 draws with `fs=-` before the character pass. Tears of the Kingdom's 14,766 and Ace Attorney's 2,475 vertex shaders have no feedback at all. + +### Helper encoders end the open copy encoder first + +- **Symptom:** `SIGSEGV` in `-[AGXG18GFamilyComputeContext_mtlnext performEndEncoding]` (from `rp_begin_ex_`, or in its `dealloc` on the encoding thread); Metal validation (`MTL_DEBUG_LAYER=1 MTL_DEBUG_LAYER_ERROR_MODE=nslog`) names it: "Command buffer already has an open command encoder". +- **Cause:** `tf_draw` opened its render encoder while the copy encoder (`g_blit`) was open, which the inline updates `ubd_land` copies there leave behind, as does a copy or a dispatch just before it. +- **Fix:** every other helper pass ends `g_blit` before it opens its own encoder, and now `tf_draw` does too (GitHub issue #101). +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): two minutes into a new game, about frame 3390 on the Mac, with the encoding thread on and off. + +### Viewport swizzles applied as a flipped viewport + +- **Symptom:** the whole picture upside down (Absolum's title logo and "Press Any Button" mirrored top to bottom); a red-reporting stub for `nvnCommandBufferSetViewportSwizzles`. +- **Cause:** the game keeps NVN's lower-left window origin (handled as for Link's Awakening, [Lower-left window origin](#lower-left-window-origin)) and, twice a frame, sets viewport 0's swizzle with `nvnCommandBufferSetViewportSwizzles` to (x, −y, z, w), which negates clip-space y: how a renderer with OpenGL heritage flips its picture. +- **Fix:** the swizzle is recorded (`OP_VP_SWIZZLE`) and applied at the draw as a viewport flipped about its centre, the same mapping. Facing is decided after the swizzle on the GPU as in Metal, so the winding stays. Other swizzles (axis exchanges, negated x / z / w) give a red line and are drawn unswizzled. +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) (FNA3D's NVN driver, `FNA3D_Driver_NVN.cpp`). + +## Uniforms, copies and ordering + +### Inline uniform updates as ordered GPU copies + +- **Cause:** reading the uniform buffer from the CPU at encode time gives the frame's *last* value, not the draw's. +- **Fix:** inline uniform updates (`CommandBufferUpdateUniformBuffer`) are replayed as in-order GPU copies (their order inside one encoder: [Metal 4 does not order operations inside one encoder](#metal-4-does-not-order-operations-inside-one-encoder); inside a pass: [Inline uniform updates inside a render pass](#inline-uniform-updates-inside-a-render-pass)). They are snapshotted at encode time (`mtl_sub_update`): the data is copied into a host-owned ring instead of being read from command memory when the GPU runs. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Metal 4 does not order operations inside one encoder + +- **Symptom:** tall blue / textured panels that flickered frame to frame: the fountain's vertex shader pushed the water surface to the top of the screen. +- **Cause:** the host batches consecutive copies, fills and inline uniform updates (`UpdateUniformBuffer`) into one compute encoder, and Metal 4 does not order operations inside one encoder. Mario writes the same uniform slot twice in a row each frame (4 bytes of −1, then 16 bytes of wave coefficients); without an intra-encoder barrier the writes occasionally landed in reverse order, the fountain's GPU wave simulation (a 512×1 RGBA16F texture updated by render → copy back → splash pass every frame) ran with −1 as its self coefficient and diverged to ±65504. +- **Fix:** `blit_buf` now inserts `barrierAfterEncoderStages` when an operation reads or writes a guest range an earlier operation in the same encoder wrote; texture copies always get one. Render passes have their own encoder with full barriers, so they were not affected; compute dispatches share the copy encoder and always get a barrier, as texture copies do (Hyrule Warriors: Age of Calamity, [Compute dispatches share the copy encoder](#compute-dispatches-share-the-copy-encoder)). +- **Check:** `DM_NVN_BLIT_ORDER=0` turns the barriers off to reproduce. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Inline uniform updates inside a render pass + +- **Symptom:** many render passes with one draw each and high submit time: Metroid Dread's cutscene frames had about 180 passes with one draw each (about 5 ms of submit time per frame on the Mac). +- **Cause:** the game updates uniforms inline (`CommandBufferUpdateUniformBuffer`) between draws, and every copy or update ended the host's render pass. This was correct but cost submit time. +- **Fix:** an inline uniform update inside a pass no longer ends it: it is deferred to the pass end (`DM_NVN_UB_DEFER=0` restores the old behaviour), and the draws after it read a shadow ([Partial-cover uniform shadows keep passes open](#partial-cover-uniform-shadows-keep-passes-open)); see [development.md, Power](../development.md#power). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). +- **Note:** 04's Metroid Dread section (at `6a06877`) still described the old behaviour ("every copy or update ends the host's render pass"); the deferral came later (development.md's Power section), and it is now the default (`ubd_on` in `nvn_mtl.m`, off with `DM_NVN_UB_DEFER=0` or `DM_NVN_TBDR=1`). + +### Partial-cover uniform shadows keep passes open + +- **Symptom:** hundreds of render passes a frame and heavy attachment traffic when a game updates a small part of a large uniform block before each draw group. +- **Cause:** a draw whose uniform binding overlaps an update deferred in the open pass reads a shadow; when the deferred updates cover only part of what the shader can read, the rest must come from somewhere, and the pass used to end there (`nvn_mtl.m`, the comment above `ubd_on`). +- **Fix:** the shadows take the rest from memory (`ubp_get`, `DM_NVN_UB_PARTIAL`) and the per-shader read extents of `cb_extent.tsv` (written by a full `dmsh_build.py` run) keep those passes open. `DM_NVN_UBP_CHECK=1` compares every shadow with the GPU's memory. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (2026-10-21, the user's Great Plateau save, Mac M4 Pro): a frame had about 386 render passes and loaded and stored about 900 MB of attachments (on an iPhone at 60 fps about 55 GB/s, near the memory's bandwidth); the GPU waits for each pass to finish before the next. 220 of the passes ended only because the game updates the first 64 bytes of a 64 KB vertex uniform block (`0x70c38c0000`) before each draw group while its shaders read up to 544 bytes of it (the terrain pixel shader `eb0afe555b86da79`, which indexes the block with an integer made from an interpolated value, reads the whole block). With the fix: 386 → about 165 passes, GPU p50 about 9.3 → 6.5 ms, submit 5.7 → 4.9 ms; `DM_NVN_UBP_CHECK=1` found every shadow equal to the GPU's memory (968,000 checked). +- **Note:** what still splits BotW's big passes (a `DM_NVN_PASSDUMP` frame dump): the NintendoWare VFX GPU particles run one compute dispatch per emitter between draws (ten 800x450 half-resolution passes and two G-buffer passes, about 190 MB a frame; the G-buffer ones read the depth being drawn, so they cannot move ahead of the pass), and decals that sample the G-buffer right after the game sets it again. The decals sample targets of their own pass (Metal leaves that undefined; the Mac reads the stored content); `DM_NVN_FB_SNAP=all` would snapshot them at a large cost. Tried and dropped: `DM_NVN_TBDR=1` (10–15% fewer stores, more CPU, red ledger lines, and it turns the deferred uniform updates off), `DM_NVN_BARRIER=game` (no gain), same-target continuation outside TBDR mode (the decals end the pass anyway). + +### An unbound uniform slot reads the zero bank + +- **Symptom:** `stage 0 cb5 has no uniform buffer bound`. +- **Cause:** the argument buffer kept a null pointer for the unbound slot. +- **Fix:** the slot now reads the all-zero bank, as an unbound texture slot reads the zero texture. +- **Found on:** [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (at the title). + +### Compute dispatches share the copy encoder + +- **Symptom:** GPU time per frame doubles with many small dispatches (Age of Calamity: GPU 9 → 18 ms a frame on the Mac; the iPhone's 15 → 51 ms, 17 fps). +- **Cause:** each dispatch had a compute encoder of its own with full queue barriers on both sides, a GPU drain per dispatch: about 10 µs of GPU each on the Mac. +- **Fix:** dispatches now go into the open copy encoder, each after a barrier inside it (as texture copies are, `blit`), with the compute argument table set once per encoder (`dispatch_enc`): about 1.3 µs each. +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md): after its first cutscene the first battle runs about 900–1,500 small dispatches a frame on the Mac (an iPhone log's call counts suggest more), each after a game barrier, at the same draws and passes; GPU 7.3 → 8.1 ms across the cutscene with the change. +- **Note:** `DM_NVN_DISPATCH_OWN=1` (and `DM_NVN_PASSGPU`, which times each dispatch) gives each dispatch its own encoder again. + +### Fragment-ordering barriers between draws that write images + +- **Symptom:** dialog boxes with holes the size of their text, and white specks where text faded. +- **Cause:** each UI element is drawn into its own layers and then composited into the window image by one draw per layer, all in one attachment-less pass, each a read-modify-write of the image, with `nvnCommandBufferBarrier(ORDER_FRAGMENTS | INVALIDATE_TEXTURE)` (bits 0x12) between them. The host took a barrier only as an order for the next pass, and Metal orders nothing between draws of one render pass for shader writes: the text element's composite read the window before the parchment had landed and wrote the old pixels back. +- **Fix:** a barrier with `ORDER_FRAGMENTS` or `ORDER_FRAGMENTS_TILED` now ends the open pass when a draw in it wrote an image or a storage buffer (`swrite_note` in `nvn_mtl.m`), so the next draw opens a pass behind the full consumer barrier; reflection marks an image binding written when its type has write access (`nvn_refl.m`, which left `rw` unset for images). +- **Found on:** [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md). + +### A segment committed after a buffer → texture copy + +- **Symptom:** a texture copied from a staging buffer shows another texture's blocks (Catherine: Full Body's cutscene subtitle band a hard-edged dark box with a magenta stroke through it). +- **Cause:** the GPU copy read the next texture's blocks: the source was right when recorded and when encoded, but the game's resource thread wrote the next texture into the same staging block before that first submission had been committed, because the host spent a long time building pipelines inside it. On the Switch the GPU runs a submission's copies as soon as the submit returns. +- **Fix:** a segment is committed at the first pass boundary after any buffer → texture copy (`g_cut_copy`, `sub_cut` in `nvn_mtl.m`), so the copies run while the host encodes the rest; the `residency set` line counts these cuts (`N after buffer → texture copies`). This cut always happens, whatever `DM_NVN_CUT` says. +- **Found on:** [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md): the band is a 32x512 BC3 gradient copied with `nvnCommandBufferCopyBufferToTexture` (recorded at the start of a frame, `main+0x305e64`); the resource thread wrote the next texture into the staging block 64 ms later, and the host spent 300 ms building pipelines inside the submission; a few cuts per 300 frames in this game, only while it loads. +- **Check:** found with `DM_NVN_HILITE` on the band's pixel shader, `DM_NVN_TEXRAW` of the texture (decoded: opaque black and magenta blocks, not a fade) and `DM_PROBE_WATCH` on the staging address beside a log of the copy's submission completing. + +### Copies a blit cannot do, drawn + +- **Symptom:** a scaled texture copy skipped with a red line (Metal blits cannot scale); format-changing, depth-stencil and mirrored copies. +- **Fix:** such copies (a format change, a scaled region or a mirrored one) are drawn (`copy_t2t_draw`), one render pass per layer; a destination made without render-target use is upgraded first ([Render-target use added on demand](#render-target-use-added-on-demand)); depth and stencil are written from a fragment shader, stencil through `[[stencil]]` export from an `X32_Stencil8` view of the source. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md): RG11B10F → RGBA16F every frame, half-size depth-stencil copies, and a scaled `MIRROR_Y` copy of the frame to 1280×720 RGBA8. [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md): one scaled copy when leaving the title (1920×1080 → 1280×720, RGB10A2, flags `ENGINE_2D`; probably the captured frame for the transition): 04 listed it as open (skipped with a red line); `copy_t2t_draw` now draws scaled copies too (its comment in `nvn_mtl.m` names this one), so it is no longer skipped. + +### Depth-stencil copies carry the stencil + +- **Symptom:** no ambient occlusion (Mac); on a device the result depends on whatever that memory holds. +- **Cause:** the depth downsample copy left the stencil as the new texture's undefined content (new depth-stencil textures are not zero-filled). +- **Fix:** the copy now carries the stencil. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md): the occlusion pass (`aa05724ffa62cb82`, a 480×270 R8 target cleared to 1.0) only writes where stencil bit 1 of the quarter-size depth-stencil copy is clear, so on the Mac the mask kept 1.0 everywhere. + +## Conditional rendering and queries + +### Conditional rendering evaluated at replay + +- **Symptom:** the 3D scene black (targets cleared again before they are read); `[RED] SetRenderEnableConditional: the host draws anyway`; with a game that skips clears under render conditions, scenes black behind the HUD (Mario vs. Donkey Kong's levels, 1.0.1). +- **Cause:** by default the host draws everything a `SetRenderEnableConditional` would skip. A game that wraps clears in `SetRenderEnableConditional(RENDER_IF_NOT_EQUAL, rec)` … `SetRenderEnable(true)`: the condition compares the 64-bit counters at `rec` and `rec + 16`. A timestamp report puts 0 in the counter field (the time goes in the second field). The host ignored the condition and wrote the time into both fields, so every clear ran. +- **Fix:** profile key `render_cond`: `OP_RENDER_COND` is evaluated at replay from the records in memory. A timestamp report replayed earlier in the same submission counts as 0, and a condition on a samples-passed result (an occlusion query, known only when the GPU completes) keeps drawing. A pass dump shows the switches as `E` lines (`conditional render … → not drawn`) and a skipped clear as `c … skipped by conditional rendering`. +- **Found on:** + - [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md): TOTK wraps nearly every `ClearColor` / `ClearDepthStencil` this way; each clear is followed by `ReportCounter(TIMESTAMP_TOP, rec)`, the CPU sets `rec` to 1 when a target needs a clear, and `rec + 16` is never written, so after one clear the two are equal and later conditional clears of that target are skipped. The first tile-lighting round, the half-resolution scene depth the light volumes test against (back faces `GREATER` against a buffer just cleared to 1.0 can never pass) and the light accumulation were all cleared before the composite read them. + - [Mario vs. Donkey Kong](../../dm/games/0100B99019412000/README.md) (it skips clears under render conditions, as Tears of the Kingdom and Animal Crossing: New Horizons do). + - Drawn unconditionally with no `render_cond`, nothing looked wrong in [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (no screen played looked wrong) or [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (nothing seen). + +## Vulkan layer + +The Vulkan layer (`dm/gfx/vk/hle_vk.m`) was written for Hades; Super Mario Galaxy needed the additions below, each found from a red line or a screenshot on the route to the Star Festival. + +### Vulkan shaders generated at run time + +- **Symptom:** no SPIR-V file in the RomFS; SPIRV-Cross refuses a module with "Struct member does not have ArrayStride set". +- **Cause:** Hagi emits SPIR-V for each GX configuration and creates every pipeline of its pipeline database at boot. Its SPIR-V puts `ArrayStride` on the struct member (`OpMemberDecorate`) instead of the array type, which NVIDIA's driver accepts and SPIRV-Cross refuses. +- **Fix:** `DM_VK_CAPTURE=<dir>` writes the modules; `vk_shaders.py` reads capture folders like a RomFS; `fix_strides` copies the stride onto the type before translating. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md) (3,089 modules, 4,535 pipelines). + +### Push constants in the next free buffer slot + +- **Cause:** SPIRV-Cross binds the push constant block at `buffer(0)`, where the first renumbered uniform buffer goes. +- **Fix:** the translator moves it to the next free buffer slot (`p:0.0=<slot>` in the table) and the replay sets the recorded bytes there (`setVertexBytes` / `setFragmentBytes`, the pipeline layout's range). +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Descriptor arrays bound to consecutive slots + +- **Cause:** SPIRV-Cross gives an array of n descriptors n consecutive slots. +- **Fix:** the table says `slot*n` and the replay binds element k at slot + k. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md): the vertex shaders read the Wii's transform memory as arrays of uniform buffers (4 position-matrix blocks, 2 normal, 2 dual-texture) and the fragment shaders 8 combined image samplers. + +### Descriptors hold the Metal objects they were written with + +- **Symptom:** a crash in `bind_res` reading a freed view's texture (Super Mario Galaxy 2 on an iPhone, element 1 of the 8-texture array, when the ship took off after the first galaxy; GitHub issue 68). +- **Cause:** the replay binds every element of an array the shader takes, and a set may still name an image view the game has destroyed (the hardware reads a descriptor only when a draw samples it). The layer's descriptors were pointers to its own view, sampler and buffer structs. +- **Fix:** a descriptor now holds the texture, sampler state and buffer it was written with (retained), as a hardware descriptor holds a copy of the handle. +- **Found on:** [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md) (issue 68). Through the opening and the first storybook stage the game allocates 22,785 descriptor sets and frees none, and destroys only 5 image views (and 1,244 images), so a stale element is rare: a Mac run that kept destroyed objects to catch their binds found none there. + +### Dual-source blending from output location 1 + +- **Symptom:** Metal refuses the pipeline: "Fragment shader does not write to render target color(0), index(1)". +- **Cause:** the GX shaders write the second blend source as an ordinary output at location 1 (no `Index` decoration), and pipelines blend with `SRC1` factors; NVIDIA's hardware feeds MRT 1 to source 1. +- **Fix:** `vk_shaders.py` builds a variant `vk_<key>_d` with that output at `[[color(0), index(1)]]`, which the host takes for pipelines whose blend reads source 1. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Scaled and three-component vertex formats + +- **Symptom:** Metal's validation aborts on a buffer layout no mapped attribute uses. +- **Cause:** GX positions and colours arrive as integers read as unnormalized floats (`R16G16B16_SSCALED`, `R8G8B8A8_USCALED`, …) and in three-component formats. +- **Fix:** Metal's integer vertex formats convert to the shader's float type; a buffer layout no mapped attribute uses is left out. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Viewport and scissor persist across render passes + +- **Symptom:** menus 1.5x too big in handheld mode (the file-creation dialog cut off at the right edge). +- **Cause:** Vulkan keeps viewport and scissor across render passes within a command buffer; the layer reset them to the render target at every pass. In handheld mode the game sets a 1280x720 viewport once and draws through several passes into its 1920x1080 swapchain. +- **Fix:** the layer keeps them across passes, as Vulkan says. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Copies inside a render pass + +- **Symptom:** `draw outside a render pass`; the rest of the pass was dropped. +- **Cause:** the game copies from its EFB with `vkCmdCopyImage` between `vkCmdBeginRenderPass` and `vkCmdEndRenderPass`, which NVIDIA's driver runs. +- **Fix:** render passes now start with deferred store actions; a copy stores every attachment and ends the encoder, the next draw resumes the pass with load actions, and the pass's own store actions are set when it really ends. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Swapchain pre-transform and layer crop + +- **Symptom:** an assert (a stripped trap, `main+0x4d26e0`) when the surface does not list `preTransform` INHERIT. +- **Cause:** the game asks for `preTransform` INHERIT. In handheld mode it keeps a 1920x1080 swapchain, draws into the top-left 1280x720 and calls `vi::SetLayerCrop` every frame. +- **Fix:** the surface lists INHERIT; the host keeps the crop (`hle_vi.c`) and the Vulkan present shows only that rectangle. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Linear images alias their memory + +- **Cause:** staging images (`VK_IMAGE_TILING_LINEAR`, filled by the CPU and copied with `vkCmdCopyImage`) and one sampled 4x4 image alias their memory as on the Switch. +- **Fix:** the texture is made over the bound `MTLBuffer` with Metal's linear row alignment, which `vkGetImageSubresourceLayout` reports. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +### Timestamp and occlusion queries + +- Timestamps are their submit's completion time; occlusion queries count into a per-submit visibility buffer, copied to the pool when the submit completes. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md): two timestamp pools and a 3,072-entry occlusion pool. + +### Not translated in the Vulkan layer: geometry shaders and LOD bias + +- **Symptom:** GX points and lines drawn as one-pixel points and lines; a red line for the sampler LOD bias. +- **Cause:** two geometry shaders (GX points and lines expanded to quads) are not translated (Metal has no geometry stage); GX's sampler LOD bias has no Metal sampler equivalent. +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md). + +## OpenGL layer + +Digimon Story Cyber Sleuth: Complete Edition (Media.Vision's MVGL engine, SDK 6.4.0) and Balatro (LÖVE 11.4, "love-nx", with SDL and LuaJIT) render with **NVIDIA's desktop OpenGL 4.5 core profile through EGL**, not NVN. The host implements GL itself on Metal (`dm/gfx/gl/hle_gl.m`), as it does Vulkan for Hades; NVIDIA's GL driver (`subsdk0`) is not loaded. Balatro's graphics module is LÖVE's own OpenGL backend, so its choices follow what the context reports. + +### Games that render with desktop OpenGL through EGL + +- **Symptom:** `UNIMPL eglGetDisplay`, then `Break` right after `nv::InitializeGraphics` and `vi::OpenDefaultDisplay` (`main+0x5ed9d0`). +- **Cause:** the game renders with NVIDIA's desktop OpenGL through EGL; its NVN device (made just before) serves only a helper beside the movie player. +- **Fix:** the host's GL layer (`dm/gfx/gl/hle_gl.m`) answers the EGL imports and `eglGetProcAddress` (the rest of this section). +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md) (SDK 6.4.0, Media.Vision's MVGL engine; one boot). + +### Telling a GL game from an NVN or Vulkan one + +- **Symptom:** the NSP script's provisional profile said Vulkan, and the capture boot crashed in the GL layer. +- **Cause:** `main` imports `vkGetInstanceProcAddr` for SDL's Vulkan loader, which nothing uses; the game draws with EGL and GL. +- **Fix:** `profile_guess.py` decides `graphics gl` when `main` imports `eglCreateWindowSurface` and names `glShaderSource`, and now leaves `vkGetInstanceProcAddr` out of its Vulkan count. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md): `main` imports `nvnBootstrapLoader` and 13 EGL functions. Its graphics setup (`main+0x5ed8f0`) makes an NVN device first, which only a helper beside the movie player uses (`main+0x528000`–`+0x52c400`: a memory pool and four textures next to the movie's GL textures). Then `eglGetDisplay`, `eglBindAPI(EGL_OPENGL_API)`, a 4.5 core context, a window surface on `vi::CreateLayer(w, h)` and a glad loader (`main+0x52c460`). Of the 22 games extracted here (at the time), only Digimon matches the rule. [Balatro](../../dm/games/0100CD801CE5E000/README.md) (SDK 18.3.2, LÖVE 11.4 on OpenGL with SDL and LuaJIT; one boot): `main` imports the EGL functions and `vkGetInstanceProcAddr`, SDL's Vulkan loader, which nothing calls (`subsdk1` is NVIDIA's Vulkan driver); Hades and both Galaxies import 99 or 100 Vulkan functions, `vkCreateInstance` among them. +- **Note:** [Absolum](../../dm/games/0100A83022A22000/README.md) imports the same EGL functions (SDL3) but renders with NVN and names no GL function. + +### Which GL functions a game calls + +- **Symptom:** a jump to address 0 right after `glGetFloatv(0xb12)` (`lr=main+0x2d58c4`). +- **Cause:** a GL function the layer did not have (here `glVertexAttrib4fv`, LÖVE's default colours). glad asks `eglGetProcAddress` for every name it knows; the functions the game really calls come from a census of the code's loads outside the loader (`dm/tools/loader_census.py`). Every other name gets a null pointer, and a jump to address 0 names nothing. +- **Fix:** `DM_GL_STUBS=1` gives every name the layer lacks a stub of its own that reports its first call instead; `loader_census.py` lists what the code calls before the first boot. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md): glad asks for 2,956 names and stores each in a global its code reaches through the GOT; a census of the GOT slots the code loads outside the loader gives the 79 functions the game calls: buffers, vertex arrays, textures (2D and cube, S3TC), framebuffers and renderbuffers, programs with plain `glUniform*`, `glDrawElements` / `glDrawArrays`, and three compatibility calls (`glTexEnvi`, `glColorTable`, `glPixelTransferi`). No uniform buffers, instancing or buffer mapping. [Balatro](../../dm/games/0100CD801CE5E000/README.md) (SDK 18.3.2; one boot; the missing `glVertexAttrib4fv` above): LÖVE's glad stores each pointer straight into a global of its own (no GOT slot per name), and the code reaches those globals through GOT entries that hold their addresses, which it often keeps in callee-saved registers across a loop, or picks between with `csel` (`glEnableVertexAttribArray` or `glDisableVertexAttribArray`). `loader_census.py` follows registers in a straight line for that (adrp, add, loads through relocated GOT entries, `mov`, `csel`): 2,133 names asked, about 150 called. + +### GLSL source translated offline and found by hash + +- Shaders are GLSL source compiled at run time with `glShaderSource` / `glCompileShader`. `gl_shaders.py` translates them offline: glslang (`-V -R`: relaxed Vulkan rules gather the loose uniforms into one `gl_DefaultUniformBlock`, whose member offsets the table lists) → SPIR-V → SPIRV-Cross. The host finds each shader by the hash of its text and its stage. Varyings are matched by name, as GL links them: `[[user(locnN)]]` becomes `[[user(<name>)]]`, so a vertex and a fragment shader translated apart still meet. `--fixup-clipspace` maps GL's clip z to Metal's. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md): GLSL ES 3.20 source, 1,682 `.shad` files in the MVGL archives (615 distinct) and 37 strings in `main` (post effects, FXAA, shadows, the movie's YUV). + +### GLSL that NVIDIA's compiler accepts and glslang does not + +- Fix-ups on the preprocessed text: the GLSL ES 1.0 functions `textureCube` / `texture2D` in ES 3.20 code (with `-V` glslang reads `textureCube` as Vulkan's texture type, so a macro cannot rename it), `out vec4 gl_Position` redeclared through a macro, and `textureOffset` with an offset that is not a constant, which becomes a plain sample at P + offset ÷ level 0 size. A string in `main` says nothing of its stage, so it is translated as each stage that compiles. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md): `textureCube` / `texture2D` in 89 fragment shaders; the redeclared `gl_Position` in the built-in sprite shader; the non-constant `textureOffset` in FXAA; one fragment shader string in `main` defines `vPosition` as `gl_Position`. + +### GL shaders put together at run time and captured + +- **Symptom:** in a capture run every program links with no uniforms; the game's first `Shader:send` raises a Lua error and, on the Switch, its error handler quits ([Any Lua error quits a LÖVE game](game-state.md#any-lua-error-quits-a-löve-game)). +- LÖVE builds each source in Lua from its header, its built-in code and the game's `.fs` file. The NSP script captures them on the Mac (`gl_capture.tsv`). A capture run has no library. +- **Found on:** [Balatro](../../dm/games/0100CD801CE5E000/README.md): the 35 shaders are all made before the first frame. + +### The GL core profile and persistently mapped buffers + +- LÖVE reads `GL_CONTEXT_PROFILE_MASK`; on a core profile it streams vertices through persistently mapped buffers (`glBufferStorage` + `glMapBufferRange` of a 3-frame ring, a `glFenceSync` per frame and `glClientWaitSync` three frames later), on a compatibility one from client memory, which Metal cannot draw from. The layer answers core: the mapping is the shared `MTLBuffer`'s memory, a fence is the serial of the command buffer being recorded, and a wait on a fence of that command buffer commits it first. The shaders then come as `#version 330 core`. +- **Found on:** [Balatro](../../dm/games/0100CD801CE5E000/README.md). + +### GL uniforms, pipelines and attributes + +- Uniforms are CPU copies per stage, set as bytes at a draw after they change; pipelines are cached per program by vertex layout, attachment formats, blend and colour mask; disabled vertex arrays read the attribute's current value (GL's default (0, 0, 0, 1) until `glVertexAttrib4f`) through a constant-step buffer. Without `glBindAttribLocation` the game's attribute index is the Metal one: `glGetAttribLocation` returns the shader's own attribute numbers. With it, the layer maps each Metal input to the game's index at the link. +- A GLSL `bool` is a `uint` in the SPIR-V block; `gl_shaders.py` keeps the declared type, as LÖVE's `Shader:send` takes only a boolean for a bool. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md) (no `glBindAttribLocation`); [Balatro](../../dm/games/0100CD801CE5E000/README.md): LÖVE binds VertexPosition, VertexTexCoord, VertexColor and ConstantColor to 0–3 with `glBindAttribLocation` before each link, keeps VertexColor and ConstantColor as disabled arrays and sets their values with `glVertexAttrib4f` (the draw colour); it lists uniforms with `glGetActiveUniform` (types as GL enums, arrays as `name[0]`) and reads their initial values back with `glGetUniformfv` / `iv`. + +### GL textures, swizzles and uploads + +- Textures are made on first use from the levels given so far (Digimon's loader defines them level by level). An upload into a texture a pending command buffer may still read is a blit in command order; any other is a CPU write. Metal reads a channel a format lacks as 0 (alpha 1) as GL does, so only luminance / alpha formats get a swizzle: Metal refuses a swizzle on a texture that can be a render target (`Texture swizzling is not compatable with MTLTextureUsageRenderTarget`, Digimon's first story scene). +- `glTexStorage2D` makes every level up front (mipmaps filled by `glGenerateMipmap` as a blit), `glTexStorage3D` 3D and 2D array textures; the layer's units hold all four targets (96 of them, as `GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS` says). A `GL_TEXTURE_SWIZZLE_*` is applied when the texture is made (a swizzled texture cannot be a render target). +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md); [Balatro](../../dm/games/0100CD801CE5E000/README.md): the game's images are mipmapped; LÖVE makes a 1x1 default texture of each type, 3D and 2D array among them; font glyphs are RG8 with a swizzle of R, R, R, G. + +### GL buffers orphaned, clears as load actions + +- `glBufferData` on a buffer the GPU may still read gets a new `MTLBuffer`, as GL drivers orphan it. A whole-framebuffer clear becomes the next pass's load action; a scissored or masked one is a quad. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md). + +### GL draws and framebuffers + +- Batched quads share one 16-bit index buffer through `glDrawElementsBaseVertex`; canvases bind GL_FRAMEBUFFER (read and draw), check completeness, discard depth and stencil after use (`glInvalidateFramebuffer`) and clear with `glClearBufferfv`. `glBlitFramebuffer` (colour, same size) and `glReadPixels` (RGBA8) are there for multisample resolves and `Canvas:newImageData`, which the game does not use. +- **Found on:** [Balatro](../../dm/games/0100CD801CE5E000/README.md). + +### GL orientation through the lower-left origin + +- The layer renders as GL does, bottom row first, through the NVN layer's lower-left origin (`mtl_set_origin_lower_left`): flipped viewports, reversed winding, a flipped present. Scissor rectangles and `gl_FragCoord` need no change, and render targets read back with GL's texture coordinates. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md). + +### GL present paced by the swap interval + +- Three window textures go through the NVN layer's present path (`mtl_ext_tex`, as the Vulkan swapchain), paced by the swap interval in display-link ticks. +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md): the game sets the interval to 60 ÷ its frame rate before each swap: 1 at the title and in the DigiLab, 2 from the story's opening on. + +### GL movies decoded by the SDK's movie library + +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md): the USM movies' H.264 is decoded through the SDK's movie library (the host's decoder) into NV12 buffers (3,110,400 bytes a 1920x1080 frame); the game uploads the planes into a `GL_RED` and a `GL_RG` texture (two `glTexSubImage2D` a frame) and converts YUV in its own shader (in `main`, `SamplerY` / `SamplerUV`). diff --git a/docs/reference/rewriter.md b/docs/reference/rewriter.md new file mode 100644 index 0000000..e1ae734 --- /dev/null +++ b/docs/reference/rewriter.md @@ -0,0 +1,219 @@ +# Rewriter + +The NSO rewriter (`dm/tools/patch_nso.py`) and its per-build tables (`dm/tools/sdk_patches.py`, a profile's `patches.py`). Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Jump tables + +### Offset jump tables through x18 read at rewrite time + +- **Symptom:** `jump table <addr>` assertion in `patch_nso.py`, in a Unity / IL2CPP `main`. +- **Cause:** the compiler used `x18` for jump-table shapes other than the ones the rewriter knew. One of them is an offset table: `ldrsw x18, [xB, x18, lsl #2]; add x18, x18, xB; br x18`. (The same `main` also uses `x18` for an indirect call and an `alloca`: see [Indirect calls through x18](#indirect-calls-through-x18) and [Stack pointer writes through x18](#stack-pointer-writes-through-x18).) +- **Fix:** the rewriter reads the offset table's targets at rewrite time (`site_br_tab` in `patch_nso.py`). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) `main`: 7 jump tables (beside 1 call and 1 `sp` write). +- **Check:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s output stays byte-identical. + +### Offset jump tables with the bounds check on another register + +- **Symptom:** `jump table <addr>` assertion in `patch_nso.py`, in a `subsdk`. +- **Cause:** the offset-table form `ldrsw x18, [xB, xI, lsl #2]; add x18, x18, xB; br x18` with the bounds check on another register: the index in `w0` (`sub w0, w18, #0xd1; cmp w0, #0x31`), a copy (`cmp w21, #7; …; mov w18, w21`) or a masked copy (`and w16, w18, #0x3f; cmp w16, #0xe; …; and x18, x18, #0x3f`). The entry may also be loaded into another register, with only the `add` writing `x18` (Cuphead, Unity 2017.4: `cmp w17, #7; ldrsw x17, [x15, x17, lsl #2]; add x18, x17, x15`). +- **Fix:** `jump_offsets` follows `mov` copies back to the `cmp` and accepts a check on the same `and` of the same, unchanged source. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (GLSLC and movie modules); [Cuphead](../../dm/games/0100A5C00D162000/README.md) (the entry loaded into another register). +- **Check:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md)'s output stayed byte-identical. + +### Index copied before the bounds check + +- **Symptom:** `jump table <addr>` assertion where the index is a copy made before the check (`mov w0,w18`, then 41 instructions holding two other jump tables on `x0`, then `cmp w18,#4; b.hi; ldrsw x18,[x11,x0,lsl #2]; add x18,x18,x11; br x18`). +- **Cause:** every other form follows copies made after the bounds check. +- **Fix:** `check_before_copy` accepts the `cmp` and `b.hi` right before the load when a `mov wI, wS` up to 64 instructions back is the last writer of both registers; capstone lists a `cmp`'s first operand as written, which the search skips. It runs only when every other form has failed, so modules that rewrote before come out the same. +- **Found on:** [DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition](../../dm/games/01006C300E9F0000/README.md) `main+0x2dd8cd0`. +- **Note:** before fixing one site, a census of all of them saves a run per site: a copy of `patch_nso.py` whose assert prints and continues found this one alone among the 25,508 `x18` sites. That `main` has no unwind entries for its first 70 MB, so the rewriter disassembles them as one function and takes 6 minutes (see [Long functions analysed in pieces](#long-functions-analysed-in-pieces)). + +### Offset table base set far back in the function + +- **Symptom:** `jump table <addr>` assertion on an offset table whose base was set far back (`cmp w18,#5; b.hi; ldrsw x18,[x24,x18,lsl #2]; add x18,x18,x24; br x18`). +- **Cause:** the function's first block loads six table bases with `adrp` + `add` into registers that its loop keeps (`x24` 441 instructions before this load), and a later block one more (`x3`, 549 before the load at `+0x6db70c`); `jump_offsets` follows the base's writer chain 256 instructions back. The function-wide search that `jump_adr_offsets` uses for a hoisted base (the one `adrp` + `add` pair that fits) found two for `x3`, the other a local table of a `br x17` further on. +- **Fix:** where nothing writes the base in those 256, its last attempt (`far_base`) follows the chain to the function's start (`table_base` `far`), through no call when the base is caller-saved, and accepts the table only when every target is an instruction of the function. It runs only where every other form has failed. +- **Found on:** ASTRAL CHAIN 1.0.1 (SDK 6) `main+0x6db4a0`: three of that `main`'s five `br x18` sites need it. +- **Check:** modules that rewrote before come out the same (Tears of the Kingdom, Age of Calamity, Cuphead, DARK SOULS: REMASTERED, Luigi's Mansion 3 and Metroid Dread `main`s: byte-identical). + +### Far table base passed through other registers + +- **Symptom:** `jump table <addr>` assertion on an offset table whose base was set far back and moved through other registers (`ldrb w18,[x14]; sub w18,w18,#1; cmp w18,#6; b.hi; ldrsw x18,[x7,x18,lsl #2]; add x18,x18,x7; br x18`). +- **Cause:** `x7` is built by `adrp` + `add` 688 instructions before, then parked in `x30` around a loop (`mov x30,x7` … `mov x7,x30`), so the far chain meets a `mov` and stops; the function-wide search finds a second `adrp x7` + `add` pair (another table further on) whose seven targets also lie in the function. +- **Fix:** the last attempt now follows register copies back to the `adrp` + `add` (`copied_base`, through no call while the register followed is caller-saved; the same target check). It runs only where the rewriter asserted before. +- **Found on:** [Super Smash Bros. Ultimate](../../dm/games/01006A800016E000/README.md) 13.0.5 (SDK 8.2.1) `main+0x2b74458`: that `main`'s only unhandled site. +- **Check:** [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md)'s three modules and Smash's `sdk` and movie module come out byte-identical. + +### Offset tables relative to an adr + +- **Symptom:** `jump table <addr>` assertion, entries relative to an `adr`. +- **Cause:** `cmp wI, #N; b.hi; …; adr x18, #L; ldrsw xE, [xT, xI, lsl #2]; add x18, x18, xE; br x18`: target = L + entry, not table + entry. +- **Fix:** `jump_adr_offsets`, tried only where the table-relative form is not found. The bounds check can be on a register copied into the index (`mov w16, w15`), and the table base can be built far from the dispatch; then every `adrp` + `add` pair in the function that builds that register is tried, and a table is accepted only if exactly one makes every target an instruction of the function. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) `main+0x393bb0` and others. + +### Byte tables indexed by x30 (capstone register names) + +- **Symptom:** `jump table <addr>` assertion on a byte table indexed by x30 (`adr x18, #L; ldrb w30, [x14, x12]; add x18, x18, x30, lsl #2; br x18`). +- **Cause:** capstone names x30 `lr` (and x29 `fp`), so the index register looked like no register and the `ldrb` never matched. +- **Fix:** the byte / halfword table check maps those names back to 30 / 29. +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md) `main+0x138216c`. + +### Byte tables whose add is far from the br + +- **Symptom:** `jump table <addr>` assertion on a byte table whose `add` is far from the `br` (`adr x18,#L; ldrb w0,[x17,x15]; add x18,x18,x0,lsl #2`, then 17 register moves before `br x18`). +- **Cause:** the byte / halfword search looked 16 instructions back. +- **Fix:** `jump_bytes_far` looks 32 back, requires that nothing writes x18 between the `add` and the `br`, and runs only when every other form has failed, so sites that rewrote before come out the same. +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) `main+0x37f9bc8`: one of the 12 `br x18` sites in this `main`. +- **Note:** a census of all the `br x18` sites first (each shape within 16 / 32 / 48 instructions) saves a 12-minute rewrite per site. + +### Indirect calls through x18 + +- **Symptom:** `jump table <addr>` assertion in `patch_nso.py`, in a Unity / IL2CPP `main`, at a `blr x18`. +- **Cause:** the compiler made an indirect call through `x18` (`blr x18`). +- **Fix:** the rewriter calls through `x16` with the original return address, so C++ unwinding still finds its frames (`site_blr`). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) `main`: 1 call (beside 7 jump tables and 1 `sp` write). +- **Check:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s output stays byte-identical. + +### Indirect tail calls through x18 in IL2CPP thunks + +- **Symptom:** `jump table <addr>` assertion on a `br x18` with no table in front of it. +- **Cause:** not a jump table but an indirect tail call: an IL2CPP thunk loads the method pointer with `ldr x18, [x0, #0x18]`, moves its stack arguments with every other scratch register (`ldr x16, [sp]`, `ldp x17, x15, [sp, #8]`, …) and ends with `br x18`. +- **Fix:** when neither table form is found, the last write of `x18` is a plain load and the `br` ends the function, it becomes `x16 = guest x18; br x16` (`tail_call` / `site_br_call`; at a call `x16` / `x17` are free, as for `blr x18`). It is tried only where the rewriter used to stop. +- **Found on:** [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) (Unity 2022.3 IL2CPP): `main+0x1d271bc` and `+0x204ebec`. +- **Check:** every module the rewriter accepted before comes out byte-identical. + +## Exclusive windows + +### Exit stub order in exclusive windows + +- **Symptom:** job system hangs; a queue claims thousands of tasks but every slot is empty. +- **Cause:** in an `ldxr … cmp … b.ne out … stxr … cbnz` window, the rewritten stub emitted its exits in the wrong order, so successful stores took the failure path. +- **Fix:** emit the fall-through exit first (`patch_nso.py` `site_window`). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md): 82 such windows. D2R had no such windows. + +### Out-of-line retry blocks of atomic updates + +- **Symptom:** `branch into window <addr>` assertion. +- **Cause:** the compiler moved an atomic update's retry path out of line: the failed `stxr` branches to a block `ldxr; cmp; b.eq <the stxr>` that re-enters the window at its store. That block cannot jump into the window's patch copy (the `x18` stand-in would be loaded between its exclusive load and the store). +- **Fix:** the retry block gets a copy of its own: the retry block and the window's tail from the store on, under one prologue (`retry_block` / `site_retry_window`). +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) 1.4.3 `main`, 8 sites. + +### A window never spans a second exclusive load + +- **Symptom:** branches to an `ldxr … stxr` loop's head landed mid-window. +- **Cause:** before this rule, a retry block's `ldxr` opened a window that ran on into the next, unrelated `ldxr … stxr` loop (`main+0x5a4674`). +- **Fix:** a window never spans a second exclusive load. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) 1.4.3 `main` (with [Out-of-line retry blocks of atomic updates](#out-of-line-retry-blocks-of-atomic-updates)). + +### adrp inside an exclusive window + +- **Symptom:** `AssertionError: 0x…` in `site_window` on an `adrp` inside an exclusive window (`ldaxr x9,[x9]; cmp x9,x25; b.ne out; adrp x10 + add x10` (the same address again, for the store); `mov x18,x26; stlxr w9,x8,[x10]`, then loads and `cbnz w9` back). +- **Cause:** a window that uses x18 is copied into the patch area, and an `adrp` there would build another page. +- **Fix:** it is re-encoded for the copy's address with the same target page (x18 as its stand-in), in both window copiers (`window_adrp`). +- **Found on:** [Pikmin 4](../../dm/games/0100B7C00933A000/README.md) 1.1.1 (Unreal Engine 4.26.2) `main+0x43da14c`. +- **Check:** only windows that stopped the rewrite change, so every other module rewrites byte for byte as before. + +## Patch area reach + +### Halfword jump table landing pads sized to the largest entry + +- **Symptom:** `B()` assertion at a spin wait: a branch into the patch area beyond the ±128 MB a `b` reaches. +- **Cause:** each halfword jump table (`ldrh wE, [xT, xI, lsl #1]; …; adr x18, #L; add x18, x18, xE, lsl #2; br x18`) got a landing pad for every one of the 65,536 values an entry can hold (768 KB a table), whatever the switch's size: that `main` has 51 of them, 40 MB, and the spin-wait stubs placed after them were out of reach. +- **Fix:** `half_count` reads the table as `jump_adr_offsets` does (the index's bounds check, following register copies, then the base from `adrp` + `add`, or the one candidate in the function whose every target is an instruction of it) and pads only up to its largest entry; a table it cannot read keeps all 65,536 (7 of the 51). The patch area went from 42 MB to 7.9 MB. +- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) 3.0.3 `main`, 63 MB of text. +- **Check:** in the other titles' modules only such tables changed: one in the `sdk` of several SDK 11–20 builds (936 to 1,320 pads), 1 to 6 in some `main`s (Tears of the Kingdom, Breath of the Wild, Silksong, Monster Hunter Rise, Absolum, STEINS;GATE RE:BOOT) and one in some movie and middleware `subsdk`s. + +### Patch area before the module base for a large bss + +- **Symptom:** `B()` range assertion in `patch_nso.py` (`site_plain`), only for one module. +- **Cause:** its bss is 1.8 GB, the engine's static heap, so a patch area appended after the image starts 1.9 GB from the code and no `B` (±128 MB) reaches it. +- **Fix:** when an appended patch area would start beyond 112 MB, the rewriter puts it 32 MB *before* the module base (negative `PATCH_OFF`). `patched.bin` then holds only text + ro + data (`FILE_IMAGE_LEN`) followed by the patch bytes, and the loader maps the bss and the key page as zero pages (`IMAGE_SIZE`). The Mac loader, the iOS embedding (`gen_embed.py`: no bss in the executable) and `QueryMemory` handle both layouts. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md) `main`. +- **Check:** Mario's and Engage's output is byte-identical to before. + +### Patch area right before the base for a long text + +- **Symptom:** `B()` range assertion, then `patch area of … bytes does not fit in the … before the base`. +- **Cause:** with the patch area 32 MB before the base, sites near the end of the text are more than 128 MB away. +- **Fix:** when the text ends within 32 MB of the reach (`BIG_TEXT`), the patch area goes right before the base: the stubs follow the sites in address order, so each site's stub is about as far from it as the text is long. The first pass tries the gap that keeps every site in reach (2.6 MB here); when the patch area is larger (3.6 MB, 34,322 sites), the rewriter runs itself again with exactly that size (`DM_PATCH_FRONT`, set only by the rerun) and `B()` checks every branch. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) `main`: 125.3 MB of text. + +## Cuts and observation gates + +### Observation gates where x9–x17 are dead + +- **Symptom:** `observation gate neither at a function entry nor after a bl`; `observation gate neither at a function entry, after a bl nor before a call` on an epilogue. +- **Cause:** the host may change x9–x17 in a gate, so a gate sat only where they carry nothing (a function entry, after a `bl`). +- **Fix:** a site is also accepted when the straight-line code from it to the next `bl` reads none of x9–x17 before writing it (`scratch_dead`); it is tried only after the other two rules fail. `ret` ends the code ahead as a `bl` does: under AAPCS64 none of x9–x17 carries a result, so the straight-line scan (`scratch_dead`) accepts a `ret` too. +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md): the gate on `mov x0, x19` at `main+0x32758e4`, after a cut `bl` and an `adrp`. [Pikmin 4](../../dm/games/0100B7C00933A000/README.md): the gate on `ldp x29, x30, [sp, #0x10]` right after the cut allocator constructor call, `main+0x353f78c`; the rule checks the image after the cuts, so the `bl` before it is a `nop` by then (the `ret` case). +- **Note:** the gate table takes the original instruction as a word value (`aa1303e0`), the cut table as bytes (`e00313aa`). + +## Functions, unwind entries and data in text + +### Code before the first unwind entry is one function + +- **Symptom:** `jump table <addr>` assertion on an ordinary byte table near the start of the text (`adr x18; ldrb w0,[x17,x16]; add x18,x18,x0,lsl #2; br x18`). +- **Cause:** the module has no unwind entry before `main+0x6a75d4`, and `func_of` made every site before the first FDE a function of one instruction, so no form saw the `adr` in front of the `br`. +- **Fix:** the text before the first FDE is now one function, from the end of the data block before it (or the text's start), as a gap between two FDEs already was (`func_of`). +- **Found on:** [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) 1.0.1 `main` (Unreal Engine 5.6.1): `main+0x219234`. +- **Check:** of the 120 modules of 33 games on hand, only DRAGON QUEST VII Reimagined's `main` has other `x18` sites there (110 before `+0xb310`, none in a window or a `br x18`), and its rewrite is byte-identical. + +### Long functions analysed in pieces + +- **Symptom:** the rewrite never finishes, the Mac swaps, the rewriter's footprint reaches 100 GB. +- **Cause:** a stretch without FDEs is one function to `func_of`, and its detailed disassembly costs about 880 bytes an instruction once built (2.6 KB while capstone builds it): 21.6 million instructions. In FINAL FANTASY RESONANCE's `main`, 82 MB at `main+0x6c4f88`–`+0x59220c0` and the 6.65 MB before the first FDE have no unwind entries. Octopath Traveler 0 (71 MB), DRAGON QUEST XI S (68 MB) and DRAGON QUEST VII Reimagined (two of 23 MB) have such stretches too; they got through on a quieter Mac with the compressor and swap. +- **Fix:** a function longer than 256 KB is now analysed in pieces (`pieces` in `patch_nso.py`), cut only right after a `ret` / `br` / `b` where no conditional branch and no `b` within 64 KB crosses, so a window, a jump table's bounds check and table base, and a retry block stay in one piece. After all pieces, every farther `b` / `bl` and every jump table target is checked against the windows and retry blocks of all pieces. Counting `b` up to 1 MB left only 21,385 cut points in the 82 MB (tail calls between neighbours) and pieces of up to 6.5 MB; with 64 KB there are 185,228, and no piece is over 256 KB (the two stretches: 28 and 344 pieces). The rewrite then takes 13 minutes at about 2.7 GB resident on a loaded Mac. +- **Found on:** [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) 1.0.1 `main`; also [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md), [DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition](../../dm/games/01006C300E9F0000/README.md), [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md). +- **Check:** DRAGON QUEST VII Reimagined's `main` came out byte-identical to its rewrite by the previous `patch_nso.py`. FINAL FANTASY RESONANCE's came out the same as with `b` counted up to 1 MB (3 and 136 pieces), and DRAGON QUEST XI S's (292 pieces) and Octopath Traveler 0's the same as with 1 MB pieces and `b` within 256 KB (75 pieces for DQ XI S), so the analysis does not depend on where the cuts fall. + +### Disassembly goes on after undecodable words + +- **Symptom:** `KeyError` at `site_br` for a `br x18` that is an ordinary byte jump table. +- **Cause:** capstone stops at the first word it cannot decode, and the function held a trap word (`0xe7ffdefe`, `main+0x6b34540`) before the jump table, so the table was never analysed. +- **Fix:** function disassembly now goes on after undecodable words (`disasm_fn`). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) `main+0x6b346d0`. + +### Data blocks at the start of the text + +- **Symptom:** `RuntimeError: 788 ldr x18, #0xd1eb0` in `patch_nso.py`, or any rewrite site in data. +- **Cause:** the text segment starts with 1.7 MB of data (`main+0x0`–`+0x1b5000`, high-entropy, no FDE covers it, and no `adrp` or relocation points into it): 62% of its words do not decode, and 6,858 of the rest read as instructions that use `x18`, plus three "`svc`" words with numbers such as `#0x91e7`. The linear sweep rewrote them all. +- **Fix:** `dm/tools/nso_code.py` finds such data blocks: a stretch of at least 4 KB between the `.eh_frame` FDE ranges in which more than a quarter of the words do not decode. The sweep skips them (`DATA_BLOCKS`, logged as `data blocks left alone`), and so does `insn_census.py`, whose counts then match the rewriter's again (11,744 `x18` sites in Absolum's `main`). Code without unwind information is still swept: it decodes (Absolum's zlib at `main+0x75261c`, libunwind's register restore at `main+0x28dbd4`). +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) `main` (.NET NativeAOT). +- **Check:** no module of the 19 other games on hand has a data block, and their rewrites are byte-identical. + +## Stack and system registers + +### Stack pointer writes through x18 + +- **Symptom:** `jump table <addr>` assertion in `patch_nso.py`, in a Unity / IL2CPP `main`, at an `alloca` (`mov sp, x18`). +- **Cause:** the compiler wrote `sp` from `x18` for an `alloca` (`mov sp, x18`). +- **Fix:** the rewriter restores its scratch registers relative to the old `sp` (`site_sp`). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) `main`: 1 `sp` write (beside 7 jump tables and 1 call). +- **Check:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s output stays byte-identical. + +### Stack arguments reserved with sub sp, sp, xN + +- **Symptom:** `CRASH sig=10 addr=0` on an `stp` relative to `sp`, with `sp` not a multiple of 16 (`main+0x2bbba4`, `sub sp, sp, #0xc0` then `stp d15, d14, [sp, #0x60]` with sp ending in `…9dc`). +- **Cause:** Horizon leaves stack alignment checking off for user code; Apple's kernel faults on an sp-based access while sp is misaligned. The game's script engine calls native functions through six hand-written bridges (`main+0x17c140` and five more beside it): each reserves the callee's stack arguments with `sub sp, sp, x19`, x19 being their sizes added up (4 for one int, `[x0 + 0x238]`), and gives them back with `add sp, sp, x19` after the call, so the callee ran with sp 4 bytes off. +- **Fix:** the rewriter now sends every `sub sp, sp, xN` (extended-register form, no shift) through a stub that rounds xN up to 16 first (`site_sp_align`); the `add` then gives back the same amount. Compiled code keeps such sizes 16-aligned, so there it changes nothing. `insn_census.py` counts the sites as `sub sp, sp, xN`. +- **Found on:** [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) `main` (its AngelScript call bridges). +- **Check:** of the 22 games' modules on hand only this `main` has the form (6 sites), so every other rewrite is byte-identical. + +### System register reads that macOS traps + +- **Symptom:** SIGILL / trap on a system register read. +- **Cause:** a newer SDK reads a register macOS traps (`CTR_EL0` in SDK 16's `FlushDataCache`). +- **Fix:** rewrite to a constant (`patch_nso.py`, the `ctr` site kind). + +## Relocations and module layout + +### Packed relative relocations (DT_RELR) + +- **Symptom:** every pointer in `.data` / the GOT is a module offset, crashes everywhere. +- **Cause:** SDK 20 modules pack their relative relocations as `DT_RELR` (tag 36, size tag 35): `main` has 223,994 of them and its `DT_RELA` only symbol relocations. Each 8-byte entry is either an even offset (relocate it, move to the next slot) or an odd bitmap whose bits 1–63 stand for the following 63 slots; the addend is the slot's own value. +- **Fix:** `relr_apply` in `loader.c`; `DM_LOADHASH` and `nvn_slots_got.py` read it too. The loader's log line counts them as `relr=`. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) 1.9.0 (SDK 20). + +## Checking a rewriter change + +Rewrite every extracted module of every title with the old and the new `patch_nso.py` and compare all outputs; only `build_src.sha256` (the rewriter's own hash) may differ for titles the change is not meant for. diff --git a/docs/reference/services.md b/docs/reference/services.md new file mode 100644 index 0000000..979c615 --- /dev/null +++ b/docs/reference/services.md @@ -0,0 +1,595 @@ +# Services and SDK calls + +SDK calls and system services the host answers (`dm/host/hle_*.c`). Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Service connections + +### SDK calls that open a system service through sm + +- **Symptom:** `sm::Initialize` aborts right after `ConnectToNamedPort` (SVC 0x1F) and `SendSyncRequest` (0x21); abort inside `nn::sm::Initialize`, the log showing `SVC 0x1f` (ConnectToNamedPort, port name `sm:`) and `SVC 0x21` answered by placeholders, then an `nn::diag` abort; abort with no message at start-up in a service call the host has no hook for (`Break` from `sdk+…` with an `nn::diag` backtrace). +- **Cause:** an SDK path reached a system service the host doesn't replace: the SDK's IPC to the service fails and it calls `AbortImpl`. Often it is an SDK call the host does not hook, for example a newer overload of a hooked call. +- **Fix:** hook the public SDK entry point named in the backtrace (`HLE_FORCE` in the relevant `hle_*.c`). Symbolize the abort backtrace against the SDK's exports to find the caller (resolve the frames with `sdis.py --sym` on the decompressed module) and hook that name. List the game's imports of that service the host does not hook ([Listing a game's unhooked imports before they are met](#listing-a-games-unhooked-imports-before-they-are-met)) and add the ones that talk to a service; library helpers that only fill the game's own buffers need nothing. Look at how the game uses the result before choosing it (a `bcat` mount failure is the game's own "nothing delivered" path). +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md) (SDK 18): `nn::hid::InitializeTouchScreen(int, int)`, the SDK 18 overload of the hooked `InitializeTouchScreen()`; [Super Mario Odyssey](../../dm/games/0100000000010000/README.md) (SDK 7): `account::OpenPreselectedUser`, then `bcat::Initialize`, then `audio::AcquireAudioDeviceSwitchNotification`, and a null read in `account::GetLastOpenedUser`. +- **Note:** the same abort from other services: `ec::InitializeForShopServiceAccessors` ([Shop service accessors](#shop-service-accessors)), `pctl::IsFreeCommunicationAvailable` ([Parental controls and free communication](#parental-controls-and-free-communication)), `oe::FinishStartupLogo` ([The startup logo](#the-startup-logo)), `hid::VibrationNode::Update` ([Vibration values flushed by the SDK's vibration graph](#vibration-values-flushed-by-the-sdks-vibration-graph)), `audio::GetAudioDeviceOutputVolume` ([Audio device output volume and names](audio.md#audio-device-output-volume-and-names)), `capsrv::InitializeForApplication` ([Album calls fail with no album](#album-calls-fail-with-no-album)), `settings::LanguageCode` ([Language codes](#language-codes)), and the rest of this file. + +### Listing a game's unhooked imports before they are met + +Most service calls can be found before they crash, before the first boot or the next one: the calls that would abort in `sm::Initialize` (or read a null service object) are in the import list. + +- **Procedure:** list `main`'s `nn::` imports (its undefined imports) that no host hook names, by namespace: the survey's import table (`nso_syms.py`), then `main`'s undefined `nn::` symbols that no hook (`HLE` / `HLE_FORCE`) names ([02](../agents/02-survey.md#imports-vs-host-hooks)), and read each one's SDK code. This gives the calls a game will make before they are met. The import list itself says where to look: diff the undefined symbols against the closest title the host already runs (and every other game's imports), drop what the host hooks, and the rest are the candidates. To find which `sdk` function aborted, map the backtrace offsets to the nearest exported symbol ([The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) 1.4.3). +- **What needs nothing:** most `nn::os` / `nn::mem` imports run natively in `sdk`. Calls that are pure computations need nothing, nor do socket calls behind a `Socket()` that already fails offline, nor library helpers that only fill the game's own structures. +- **Hook names:** hooks are `HLE_FORCE`, `SSF`, `SKF`, `IKB_IGNORE` across `dm/host` and `dm/gfx`. `strings` misses names the symbol table stores as the tail of a longer one, so list them with `nso_syms.dynsyms`. +- **The survey table's blind spots:** it counts only names given to `HLE` / `HLE_FORCE` directly: hooks made through macros (`SKF` and `SSF` in `hle_net.c`, `TOUCH(N)` in `hle_hid.c`) show as missing. The touch-screen states (`GetTouchScreenState<16>`) were already hooked by such a macro, which the import table does not see. It once listed every vi call as unhooked: `nso_syms.py host_hooks` read only `dm/host`, and the vi hooks are in `dm/gfx` (it reads `dm/gfx` and `dm/gfx/nvn` now; found on [Absolum](../../dm/games/0100A83022A22000/README.md)). +- **What it found** (the calls themselves are in the entries of their service below): + - [Pokémon Sword](../../dm/games/0100ABF008968000/README.md) (SDK 7, one boot each): the import list against the hooks and every other game's imports named all of them beforehand: 33 `nn::` imports no host hook names and no other game imports. + - [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (SDK 11). + - [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11, one boot each): the import list against the hooks and the other SDK 11 games found most of them before the first boot. + - [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md). + - [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): the pure computations were the swkbd presets, the web argument constructors, `time::GetDayOfWeek` and the audio resampler. + - [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (SDK 19.3.1): the import list against the hooks gave the rest before they were met; the rest fill the game's own structures or sit behind a socket that fails offline. + - [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11): the import list against the hooks (the survey's `nso_syms.py` table, then `main`'s undefined `nn::` symbols that no `HLE_FORCE` names) left 86 candidates; most only fill the game's own structures (error and web argument builders, `prepo` report buffers, `swkbd` configuration, the audio resampler, `nn::async`). The ones that reach a service were hooked before they were met where the answer is clear: `time::ToPosixTime` with an explicit rule, `album::SaveCurrentScreenshot`, `err::ShowApplicationError` and the `ShowShopAddOnContentList(UserHandle)` overload. + - [STEINS;GATE RE:BOOT](../../dm/games/01008BC024FDE000/README.md) (SDK 20.5.17): `nn::audio::SetAudioOutVolume` ([AudioOut volume kept per output](audio.md#audioout-volume-kept-per-output)) was among `main`'s imports that no host hook names; the others there fill the game's own structures or run in `sdk` (the CPU audio renderer, the swkbd and hid argument builders). + - [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (SDK 15.3.2): `nn::settings::GetAvailableLanguageCodes` was the one import of `main` that no host hook names and that reaches a service; `fs::MountRamDisk` and the swkbd and hid helpers it also imports run in `sdk` or were never called. + - [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md) (SDK 8.2.1): besides `GetAvailableLanguageCodeCount`, the other imports no hook names run in `sdk` (the `LanguageCode` comparison, `ClockSnapshot`'s constructor and getters, fibers, `NumberLineAllocator`). + - [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) (SDK 20.5.4): the clock, region and album calls under their own entries below were found in `main`'s imports before the first boot. + - [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1): listed after its first boot crashed in `oe::DisableAlbumImageTakenNotification`; the calls found are in the entries under `## Display, window and operation mode`, `## File system and save data` and `## Accounts, network, shop and parental controls`. + - [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3). + +### A game update imports SDK calls the base game did not + +- **Symptom:** a game update aborts at boot where the base game ran. +- **Cause:** the update imports SDK functions the base game didn't. Mario 1.2.1 added 28, among them `nn::nfp` (amiibo, initialized at boot), `pl::IsRunningOnOunce` (is this a Switch 2), `ec::ShowShopApplicationInformationForUpgrade`, `account::PopOpenUsers` and `aoc::CountAddOnContent`. +- **Fix:** diff the undefined dynamic symbols of the two `main` modules before the first run, and read the call sites before choosing answers: Mario only reads the count from `PopOpenUsers` and falls back to `TryOpenPreselectedUser` on 0, and only counts add-on content when running on a Switch 2. The host answers as a Switch 1 with no amiibo and no add-on content (`hle_sdk16.c`, `hle_account.c`, `hle_misc.c`). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (update 1.2.1). + +### Game-linked HIPC clients of system services + +- **Symptom:** abort in `sm::Initialize` under `sf::hipc::InitializeHipcServiceResolution`, right after the first frames (from `main+0x5b8198`). +- **Cause:** `main` links its own clients of `olsc:u` (cloud save data backup settings) and `npns` (push notifications), which talk HIPC through `sm` and `svcSendSyncRequest`. For each user the game asks whether backup is on and switches it off where it is (`main+0x2809c8c`); a failed connect aborts as well. +- **Fix:** `hle_sf.c` takes over the SDK's HIPC client entry points (`InitializeHipcServiceResolution`, `ConnectToHipcService`, `SendSyncRequest`, `CloseClientSessionHandle` and the control requests `QueryPointerBufferSize`, `ConvertSessionToDomain`, `DuplicateSession(Ex)`): a service it serves (`olsc:u`) gets a host session, and every request on it succeeds with zeroed output, which reads as backup off for every user. Any other name is refused with a red line, as `sm` refuses a service nobody registered (`npns` was never connected). `DM_SF_TRACE=1` dumps each request. +- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) (SDK 20.5.4). + +## SDK names that changed between versions + +### Older SDK names for the same calls + +- **Symptom:** a call the host hooks under its newer name reaches the SDK's own code on an older game. +- **Cause:** older SDKs export other names for the same calls. +- **Fix:** hook both generations. SDK 12 names: `ListAudioDeviceName` / `GetActiveAudioDeviceName`, singular `hid::GetNpadState`, `OpenDefaultAudioOut`. SDK 11 names: `fs::CommitSaveData(mount)` is the older name of `fs::Commit`. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md) (SDK 11 names). +- **Note:** overloads change too: SDK 18's `nn::hid::InitializeTouchScreen(int, int)` ([Super Mario Galaxy](../../dm/games/010099C022B96000/README.md), [SDK calls that open a system service through sm](#sdk-calls-that-open-a-system-service-through-sm)), SDK 19's `LoadModule` with one more bool ([`nn::ro` and its port](#nnro-and-its-port)), and `vi::CreateLayer` with the layer's size ([Absolum](../../dm/games/0100A83022A22000/README.md), [`vi::CreateLayer` with the layer's size](#vicreatelayer-with-the-layers-size)). + +### SDK 4's `friends::GetBlockedUserList` without a `Uid` + +- **Symptom:** abort in `sm::Initialize` under `nn::friends::GetBlockedUserList` after the opening movie. +- **Cause:** SDK 4's overload without the `Uid` (`EPiPNS_7account23NetworkServiceAccountIdEii`), which the host did not take over. +- **Fix:** it reports an empty list with success: the game turns a failure into an error for the error applet (`main+0x8ff9f0`). The overload with a `Uid` keeps reporting a failure, which [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) takes as it is ([Account, friends, play data and Nintendo Account calls](#account-friends-play-data-and-nintendo-account-calls)). +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (SDK 4.5.0, the oldest so far; one boot). + +### SDK 6's `account::OpenPreselectedUser` returns a Result + +- **Symptom:** the first loading screen never ends; `DM_FS=1` shows `save:/system` "not found" and no `fs mount save`. +- **Cause:** in SDK 6 `account::OpenPreselectedUser` returns a Result, and the game skips its save mount when it is not 0 (`main+0x53b7e0`). The host's hook returned void and left the handle pointer in w0. +- **Fix:** the hook returns 0 now; SDK 7 and later callers ignore it. +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md) (SDK 6.5.0). + +## File system and save data + +### The SDK's own `fs::IsMounted` reads an empty mount table + +- **Symptom:** IL2CPP "Could not open …/global-metadata.dat" (then a crash in its logger). +- **Cause:** IL2CPP checks `nn::fs::IsMounted("rom")` before opening; the SDK's own `IsMounted` reads a mount table the host never fills. +- **Fix:** answer from the host's mount table (`hle_fs.c`). +- **Note:** the C library's path checks read the same empty table through `IsPathMounted` ([C library path checks ask the file system's limits](#c-library-path-checks-ask-the-file-systems-limits)). + +### C library path checks ask the file system's limits + +- **Symptom:** Addressables never start (no `StreamingAssets/aa/settings.json` in the `DM_FS=1` output at all), then a null asset crash (`main+0x31100ec`). +- **Cause:** Unity 2022's C# file API reaches the SDK's C library, not `nn::fs`: `File.Exists` → `SystemNative_Stat` → `stat` = `open` + `fstat` + `close` (`sdk+0x948b0`). nnmusl's path check (`sdk+0x904e0`, then the converter `sdk+0x8ef50`) asks `nn::fs::IsPathMounted`, `GetPathLengthMax`, `GetUtf16PathLengthMax`, `GetEntryNameLengthMax` and `GetUtf16EntryNameLengthMax` before it calls `nn::fs::GetEntryType` / `OpenFile`, and the SDK's own versions read the empty mount table (`IsPathMounted` false; the next one aborted with Result 0x35f202). +- **Fix:** the host answers them from its mount table, with Horizon's 0x300-byte limits and no UTF-16 limit (`hle_fs.c`). The path .NET builds is `/rom:/…`; nnmusl's converter turns it back into `rom:/…`. +- **Found on:** [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) (1.0.0.2, SDK 17.5.3, Unity 2022.3 IL2CPP). +- **Note:** armyja's fork (GitHub issue #18) patched the managed stat wrappers by address instead; these hooks need no per-build table. + +### Paths with `.` and `..` components + +- **Symptom:** an Atk task aborts in a `switch` default (`UnexpectedDefaultImpl`, then `AbortImpl`) while a stream sound loads. +- **Cause:** the stream path was `rom:/system/snd/../../sounds/streams/…`, and the host answered "not found" for any `..` component. Horizon's path normalizer resolves `.` and `..` lexically and fails only above the mount root. +- **Fix:** `host_path_ex` now does the same (`norm_rest` in `hle_fs.c`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md) (`main+0x1225a8c`). + +### RomFS lookups on large directories + +- **Symptom:** a long black screen at boot with nothing in the log (Engage: about 17 s on the Mac, 32 s on an iPhone); sampling the process (`sample <pid> 3`) shows `getxattr` under `path_component` at the top of stack. +- **Cause:** the game is walking RomFS, and every path component scanned its whole parent directory. That scan maps the game's case-sensitive names onto APFS, so it calls `getxattr` on each entry, and Engage has directories of 38,000 files. The rest of the time was the listing itself: `ReadDirectory` called `stat`, `getxattr` and a failing `fopen("<file>.holes")` for every entry. +- **Fix:** read-only mounts now use a per-directory name index built at the first lookup (`dir_index_get` in `hle_fs.c`), which also speeds up every file Wwise opens during play. In listings, now only `.dm-case-*` names (the only ones that carry the xattr) read it, directories take their type from `d_type`, and files use `fstatat` / `openat` relative to the directory. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): with the name index the first frame came at 5.7 s on the Mac; with the listing fix too it comes at 1.8 s on the Mac (9.9 s → expected about 3 s on an iPhone), and the opening movie starts at 7 s instead of 49 s. + +### ReadFile into write-protected texture pool pages + +- **Symptom:** `[RED] fs ReadFile into … failed at +0 (EFAULT): VM region … protection 1`, now and then. +- **Cause:** the destination was texture pool memory, which the dirty-page tracker write-protects after each upload (`nvn_dirty.c`). `ReadFile` lifts the protection first (`dm_guest_write_prepare`), but the graphics layer can protect the pages again before `pread` writes them, and a system call gets EFAULT where a plain write would fault into the tracker's handler; at 60 fps uploads come twice as often. +- **Fix:** on EFAULT the host now prepares the rest of the buffer again (unprotect, mark dirty, so the texture is uploaded again) and reads on, up to eight times (`h_fs_ReadFile`; the first case is logged, `[fs] ReadFile … prepared again`). +- **Found on:** [Monster Train 2](../../dm/games/010051701FB46000/README.md) at 60 fps: two runs in three as the route map loads, never at 30 fps. +- **Check:** four runs at 60 fps: one retry, no failed read. + +### File-not-found noise from optional paths + +- **Symptom:** `fs … not found` lines with `DM_FS=1`. +- **Cause:** games probe many optional paths (debug parameters, other-language voice files). +- **Note:** only a missing file followed by a failure matters. +- **Found on:** every game (general). + +### Directory, file data cache and save-thumbnail calls + +- **Symptom:** a missing `fs::DeleteDirectory`; undefined `nn::fs::` / `nn::account::` imports with no hook, found from the import list ([Listing a game's unhooked imports before they are met](#listing-a-games-unhooked-imports-before-they-are-met)) before they were met. +- **Fix:** hooked: `fs::DeleteDirectory` (an empty directory), `fs::CleanDirectoryRecursively`, `fs::RenameDirectory` (done as `RenameFile`); `fs::EnableGlobalFileDataCache`, `fs::DisableGlobalFileDataCache`, `fs::EnableIndividualFileDataCache` / `DisableIndividualFileDataCache`; `account::StoreSaveDataThumbnailImage` / `DeleteSaveDataThumbnailImage` (nothing kept) (`hle_fs.c`, `hle_account.c`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md) (SDK 11: `fs::DeleteDirectory` was missing); [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (SDK 11: `CleanDirectoryRecursively`, `EnableGlobalFileDataCache`); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (SDK 7.3.2: `DisableGlobalFileDataCache`); [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1: `RenameDirectory`, the thumbnail calls), [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3: the individual file data cache). + +### `fs::Commit` with a list of mounts + +- **Symptom:** the SDK's own code opened fsp-srv and aborted, leaving the save writes uncommitted. +- **Cause:** `fs::Commit(const char* const*, int)`, the mount-list overload, had no hook. +- **Fix:** the host commits the list as one transaction when every name is the same host save; cache and temporary storage in the list are skipped (their writes are already on disk, as for the single-name `Commit`), and a list of two separate saves is refused without committing either. +- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) (PR #25). +- **Note:** SDK 11's `fs::CommitSaveData(mount)` is the older name of `fs::Commit` ([Older SDK names for the same calls](#older-sdk-names-for-the-same-calls)); `fs::Commit` with a `CommitOption` is under [Device save data on the account save's directory](#device-save-data-on-the-account-saves-directory). + +### Save data size queries + +- **Symptom:** SIGSEGV at address 0 in `nn::fs::GetSaveDataSizeMax`, about 13 minutes into a new game. +- **Cause:** Unreal's save system asks before its first save; the SDK's version reads the fsp-srv object the host never made. +- **Fix:** it reports 256 MB of data and journal, as `GetSaveDataSize` does (`hle_fs.c`). `fs::GetCacheStorageSize` and `fs::GetFreeSpaceSize` give the same answers. Also hooked: `fs::GetSaveDataSize` / `ExtendSaveData` and the one-argument `fs::MountCacheStorage`. +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (`GetSaveDataSizeMax`, `GetCacheStorageSize`, `GetFreeSpaceSize`, found by the import list); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11: `GetSaveDataSize`, `ExtendSaveData`, `MountCacheStorage`). + +### Cache storage and other titles' save data + +- **Symptom:** each aborted in `sm::Initialize` or read a null applet proxy, one per boot. +- **Cause:** SDK 20 file system calls that reach a service. +- **Fix:** cache storage: `OpenCacheStorageList` / `ReadCacheStorageList` report none, `CreateCacheStorage` / `MountCacheStorage` use a host folder beside the temporary storage. `fs::IsSaveDataExisting(ApplicationId, Uid)` for another title (it asks for Breath of the Wild's save: none) and `MountSaveDataReadOnly`. +- **Found on:** [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) (1.4.3, the same `sdk` build as Breath of the Wild 1.9.0). + +### Device save data on the account save's directory + +- **Symptom:** the island lives in device save data, which the host did not mount. +- **Cause:** 3.0.3's `main` mounts no account save data; the island is in device save data (`MountDeviceSaveData`, `GetDeviceSaveDataSize`, `ExtendDeviceSaveData`). +- **Fix:** the host gives device save data the account save's host directory and transaction (one player). A second mount name over a save already mounted now shares that mount's transaction (`mount_ex`); it used to clone the committed directory again, over the first name's pending writes. Also `fs::Commit` with a `CommitOption` (its one flag clears the restore flag) and `GetSaveDataRestoreFlag` (never restored from a backup). +- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) (3.0.3, SDK 20.5.4). +- **Note:** a game whose `main` imports `MountSaveData` too gets its device save as a save of its own ([Device and account save data as two saves](#device-and-account-save-data-as-two-saves)). + +### Device and account save data as two saves + +- **Symptom:** every boot asks to pick a Mii again (`DM_FS=1`: `UserSave:/userdata.dat` not found although the host save has it). +- **Cause:** the game mounts `CommonSave` with `MountDeviceSaveData` and `UserSave` with `MountSaveData(uid)` (`main+0x8b0c84`), two saves on Horizon; the host made device save data the account save (for Animal Crossing: New Horizons, which mounts only device save data), so both names shared one directory. At boot the save manager deletes `CommonSave:/userdata.dat` (`main+0x8b1910`, through its file device's `DeleteFile`, `main+0x633afc`; a probe there with `@*x1+8` shows the name) and commits, and in the shared directory that was the account save's own file. +- **Fix:** a game whose `main` imports `MountSaveData` too gets its device save as `<DM_SAVE>_device`, a transaction of its own (`hle_fs.c`); of the games on hand only this one imports both, so Animal Crossing's islands stay where they are. +- **Found on:** [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md) (4.0.0, SDK 4.4.0; one boot). + +### System data stand-in for the MiiModel archive + +- **Symptom:** abort in `fs::detail::GetFileSystemProxyServiceObject` under `fs::QueryMountSystemDataCacheSize` (`main+0x8f23bc`). +- **Cause:** the game mounts system data 0100000000000802, MiiModel, as `MiiResource:` and reads `NXTexture{Low,Mid}{Linear,SRGB}.dat` and `Shape{Mid,High}.dat` whole; the SDK's Mii rendering is linked into `main` (`main+0x8f1400`–`+0x8fc000`). +- **Fix:** firmware content is not the host's to ship, so the host writes a stand-in of the same layout with every part empty (`dm_mii_standin` in `hle_mii.c`, mounted by `hle_fs.c` `MountSystemData`; any other system data answers "target not found"). Heads build and draw nothing; bodies and favourite colours show. +- **Found on:** [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md) (4.0.0, SDK 4.4.0; one boot). +- **Note:** the layout as the code reads it: texture file +0xc the largest decompressed size of each of 11 part kinds, +0x38 their alignments, from +0x64 one 0x1c-byte entry per part (offset, size, stored size, 4 zlib parameters, then width, height, format, mips); shape file +0xc / +0x3c the same for 12 kinds, from +0x6c one 0x58-byte entry per part (offset, size, stored size, parameters, five vertex attribute offsets and sizes, index offset and byte size, bounding box), then hair (132 × 0x48, +0x12704) and faceline (12 × 0x24, +0x14c24) transforms. A texture of size 0 is skipped (`main+0x8faaf4`); shapes cannot all be absent, because the faceline is drawn without a check (a null buffer read at `main+0xa690d0` under `main+0x8286c4`), so every shape entry points at one shared mesh of one vertex and the triangle (0, 0, 0), 32 bytes of zeros in a stored zlib block. The largest sizes must not be 0: they size the model's memory pool (`main+0x8f2e6c`), whose alignment is the first one. The Mii database itself: [Mii database answered by the host](#mii-database-answered-by-the-host). + +## Add-on content + +### Add-on content from `DM_AOC_DIR` + +- **Symptom:** DLC the player owns is not unlocked. +- **Fix:** the host reports add-on content from `DM_AOC_DIR`, one numbered folder per DLC index (title ID − base − 0x1000) holding that DLC's RomFS. `build_ipa_from_nsp.py` extracts the DLC NSPs in the folder with `nx_extract --aoc`, `build_ipa.sh` copies them to `<App>/aoc/`, and the app sets `DM_AOC_DIR`. `aoc::CountAddOnContent` / `ListAddOnContent` report those indices (`hle_misc.c`), and `fs::MountAddOnContent` mounts the folder read-only (`hle_fs.c`). D2R keeps its fixed `{1}` without `DM_AOC_DIR`; `DM_AOC_OWNED=0` reports none. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): its two DLC packages each hold one empty marker file (`e0`, `h0`), and the content is already in update 2.0.0. `DLCManager.RefreshHasList` marks a content owned when its mount succeeded (probe `main+0x29f3c88`). + +### Add-on content mount cache sized as RomFS working memory + +- **Symptom:** SIGSEGV in `crypto::detail::CbcModeAes128Helper::DecryptBlocks` with a null destination (`main+0x8737e8`, right after `aoc:` is mounted). +- **Cause:** the add-on content (0100152000023001: `contents.dat` and `version.txt`, the courses are in the update) is decrypted in `AocHeap`, a 0x3130-byte heap that also holds the mount cache. The host answered `QueryMountAddOnContentCacheSize` with 4 KB, and the 0x1400-byte decryption buffer then did not fit. +- **Fix:** on Horizon the cache is the RomFS's file table as `RomFsFileSystem::GetRequiredWorkingMemorySize` sizes it: both hash tables (4 bytes a bucket, the builder's bucket count) and all directory and file entries (0x18 / 0x20 bytes plus the name, padded to 4), 0x88 bytes for these two files. The host computes that from the folder it mounts (`romfs_work_size` in `hle_fs.c`); `rom` keeps its minimum answer, which is too small rather than too large. +- **Found on:** [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md) (4.0.0, SDK 4.4.0; one boot). + +### Add-on content queries on a small guest stack + +- **Symptom:** SIGBUS in `__vfprintf` under `aoc_load`, from `h_aoc_Count` (a store to `0x…67ff0`, right below a page of the guest thread's stack). +- **Cause:** the first add-on content query lists `DM_AOC_DIR` and logs, a few KB of stack, and the game asks from a thread with little left. +- **Fix:** `aoc::CountAddOnContent` and `ListAddOnContent` run on the big host stack (`HLE_FORCE_BIG`, `hle_misc.c`). +- **Found on:** [Digimon Story Cyber Sleuth: Complete Edition](../../dm/games/010014E00DB56000/README.md) (SDK 6.4.0; one boot). + +## Input + +### History reads fill every requested entry + +- **Symptom:** buttons and menus work, the sticks do nothing in play (Super Mario Odyssey: its first hint, "Look Around", waited forever); an `nn::diag` abort with Result 0x2aca in the first frames of `nnMain` (Hollow Knight: Silksong, Unity 6000.0.50f1: `bt: sdk+0x514580 main+0x1296c64 …`). +- **Cause:** games read whole histories and ignore the returned count. Odyssey reads all 16 entries of `GetNpadStates`; the host filled only the newest. It polls every controller slot each frame, which moves the shared sampling number on by about 16, so the stick samples it combined were mostly empty, while buttons and the stick's direction bits (which menus use) still came through. Silksong's `main+0x1296c60` calls `hid::GetDebugPadStates(states, 16)` every frame, and the SDK's own reads a debug pad resource nothing initialized (module 202, hid); the host had only the single-state `GetDebugPadState`. +- **Fix:** the host now fills every requested `GetNpadStates` entry with the current state under consecutive sampling numbers; the `GetDebugPadStates` history form answers every requested entry as not connected, newest first with consecutive sampling numbers (`hle_hid.c`). `DM_KEYS` takes `RLeft` / `RUp` / `RRight` / `RDown` for the right stick. +- **Found on:** [Super Mario Odyssey](../../dm/games/0100000000010000/README.md); [Hollow Knight: Silksong](../../dm/games/010013C00E930000/README.md). +- **Note:** Odyssey's six-axis history is read the same way ([Six-axis sensors](#six-axis-sensors)). + +### Npad style set update event + +- **Symptom:** the game never reads the controller (the title's "Press L+R" never sees a press; no `GetNpadState` call at all). +- **Cause:** it waits for the event bound by `hid::BindNpadStyleSetUpdateEvent` before reading. +- **Fix:** the host now signals it once when it is bound for a connected controller, and again when a player's controller connects or goes away. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0). + +### Six-axis sensors + +- **Symptom:** motion moves never happen; shaking does nothing (Super Mario Odyssey: cap throws by shaking). +- **Cause:** the host handed out six-axis handles but reported no sensor. +- **Fix:** it now reports one per connected controller: two for handheld or dual Joy-Cons, one otherwise, with the style, npad id and device in the handle. At rest the sensor lies flat: acceleration (0, 0, −1) G, no rotation, identity direction. Samples are 5 ms apart on the guest clock and numbered by time, so a history read returns the samples at those times. `DM_BTN_SHAKE` (`dm_input.h`: the iPhone shaken, Space on the Mac, `DM_KEYS` `Shake`) plays a side-to-side shake in whole cycles. +- **Found on:** [Super Mario Odyssey](../../dm/games/0100000000010000/README.md): it reads all 16 history entries per sensor and ignores the count, like its npad history ([History reads fill every requested entry](#history-reads-fill-every-requested-entry)). +- **Note:** find a game's swing detector by probing the accelerometer value it reads (`DM_PROBE=main:626248@x8+10` in Odyssey 1.4.1). + +### SDK 20 six-axis sensor and amiibo (`nfp`) calls + +- **Symptom:** SDK 20 calls that reach a service; `nfp`'s `AttachActivateEvent` / `AttachDeactivateEvent` crashed in the SDK at `sdk+0x3f2a78`. +- **Fix:** the console six-axis sensor (`hid::*SevenSixAxisSensor*`) reports zero states; more `nfp` calls are hooked: `AttachActivateEvent` / `AttachDeactivateEvent`, the two-argument `Mount`, `Unmount`, `GetCommonInfo`, `GetRegisterInfo`, `GetState`, `Finalize`. The host answers as a console with no amiibo. +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (1.9.0, SDK 20.5.22). [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) 1.2.1 already initialized `nn::nfp` at boot ([A game update imports SDK calls the base game did not](#a-game-update-imports-sdk-calls-the-base-game-did-not)). + +### Vibration values flushed by the SDK's vibration graph + +- **Symptom:** abort in `nn::sm::Initialize` under `nn::hid::VibrationNode::Update`. +- **Cause:** the SDK's vibration graph flushes queued values with `hid::detail::SendVibrationValues`, which opens the hid service. The game sets handles on its `VibrationTarget`s even when the host reports no vibration device. +- **Fix:** the flush calls it through the PLT, so an `HLE_FORCE` on that symbol catches it and returns success (`hle_hid.c`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). + +### Touch gestures recognized from the touch points + +- **Symptom:** taps in handheld mode do nothing, though the game reads the touch screen (the title's PRESS ANY BUTTON, the menus' cards). +- **Cause:** the game's touch handler (`main+0x42f1ac`) uses `GetTouchScreenState` only to track the finger and takes its taps, pans and swipes from `hid::GetGestureStates` (it acts on types 5–9), which the host answered with no entries. +- **Fix:** the host now recognizes gestures from the same touch points, by yuzu's rules plus an 8 px slop before a touch counts as moving (`hle_touch.c` `gesture_sample`, unit test `t_gesture`; only in handheld mode, as touches): Touch, Press after 0.5 s, Pan, Pinch / Rotate with two fingers, Cancel on a change in the finger count, then Tap (IsDoubleTap within 0.35 s), Swipe above 400 px/s or Complete, and Idle. For every game. +- **Check:** every menu, the cards, the map's drag and flick worked by touch with it. `DM_TOUCH_LOG=1` logs each gesture entry, and a `DM_CTL` line `drag <x0> <y0> <x1> <y1> [seconds]` moves a finger for testing pans and swipes on the Mac. +- **Found on:** [Taiko Risshiden V DX](../../dm/games/0100346017304000/README.md) (SDK 12.3.3, Koei Tecmo's KTGL). + +### Mouse, keyboard and key code maps + +- **Symptom:** the first boot aborted in `hid::GetMouseState` (the SDK's mouse resource was never initialized). +- **Fix:** hooked: `hid::GetMouseState`, `hid::GetKeyboardStates`, and `kpr::GetKeyCodeMap`, which asks the settings service (`settings::detail::GetKeyCodeMap`) for the keyboard layout's map; the host leaves the map zero. +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (SDK 11, `GetMouseState`); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11, `GetKeyboardStates`); [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (`kpr::GetKeyCodeMap`: the game zeroes a 0x1000-byte map and fetches it only once a keyboard reports keys, `main+0x1f0eb4`). + +### Poké Ball Plus (Palma) + +- **Symptom:** the first boot crashed in `hid::detail::SetPalmaBoostMode`, a null read at `sdk+0x238eb0` (the hid service never opened). +- **Fix:** the Poké Ball Plus calls (`hid::*Palma*`, 26 functions) answer that none is connected: no npad has the Palma style, so the game never gets a connection handle, and every call on a handle fails (`hle_hid.c`). +- **Found on:** [Pokémon Sword](../../dm/games/0100ABF008968000/README.md) (SDK 7). + +### IR camera reported unconnected + +- **Symptom:** undefined `irsensor` imports found from the import list ([Listing a game's unhooked imports before they are met](#listing-a-games-unhooked-imports-before-they-are-met)). +- **Cause:** Bezel Engine's input layer makes one (an IR camera) per controller and asks its status (`main+0x497f84`). +- **Fix:** the IR camera (`irsensor`) answers every camera unconnected, and processors cannot run. +- **Found on:** [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3, NintendoWare Bezel Engine). + +## Software keyboard and applets + +### System applets opened by `Show…` calls + +- **Symptom:** null read in `sdk` under `nn::applet::CreateLibraryApplet` (Engage: title menu → DLC → `nn::ec::ShowShopAddOnContentList`); `Show…` overloads found in the imports before they were met. +- **Cause:** every `Show…` call that opens a system applet (eShop, friend lists, web pages, error screens) goes through the applet proxy that `oe::Initialize` would have set up, and the host has none. +- **Fix:** hook each `Show…` overload the game imports: list `main`'s undefined symbols containing `Show` / `Applet` and grep the host sources for each mangled name; one overload missing is enough to crash. The shop calls return `void` and act as if the player closed the shop (`hle_misc.c`); the friends and web ones return failure (`hle_friends.c`, `hle_offline.c`). `web::ShowOfflineHtmlPage` is cut, as `ShowWebPage`. `err::ShowNetworkServiceError` and `err::ShowApplicationError` are logged. Also hooked, found in the imports before they were met: the `ShowShopAddOnContentList(UserHandle)` overload, and the `ec::ShowShop…` overloads without an `ApplicationId` (they show the calling application's own pages). `ec::ShowShopSubscriptionDetails` is under [Shop service accessors](#shop-service-accessors). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) (the shop calls); [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (`web::ShowOfflineHtmlPage`); [Pokémon Sword](../../dm/games/0100ABF008968000/README.md) (SDK 7, `err::ShowNetworkServiceError`); [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11: `ShowShopAddOnContentList(UserHandle)`, `err::ShowApplicationError`); [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1: the overloads without an `ApplicationId`). + +### The software keyboard blocks only its caller + +- **Symptom:** the sound stutters and the input box lags while the player types a name (Octopath Traveler 0 on an iPhone: one 4.7 s frame, 5,035 wakeups a second instead of about 1,800, and no audio underruns). +- **Cause:** the host froze the guest clock while `swkbd::ShowKeyboard` was up (d2r-mobile's choice, so that monsters did not attack while the player typed). Unreal calls it off its game thread, so the game kept running, and every thread that waits on the clock spun while its audio mixing starved. On the Switch the applet blocks only the caller while the clock runs. +- **Fix:** the host now does the same (`hle_swkbd.c`). The iOS input box froze the clock a second time around its own wait (`front_ios.m`), so the first fix changed nothing on the phone (4,977 ms and 2,577 ms frames, 5,181 wakeups a second): look for every caller of `guest_clock_freeze_for`, the front ends included. The inline keyboard keeps the freeze: its game (Monster Hunter Rise) does not block on hardware at all, and the host's input box blocks it. +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md). +- **Check:** on the Mac `DM_KBD=<text> DM_TEST_SWKBD_HOLD_MS=8000` holds the keyboard up: frozen, 8.8 fps and 7,936 wakeups a second in that window; running, 30 fps and 1,788. + +### The inline keyboard as a resident library applet + +- **Symptom:** null read in `applet::CreateLibraryApplet` under `swkbd::InlineKeyboard::Initialize` (3D World, entering the save select after the character select). +- **Cause:** the inline keyboard is a resident library applet whose results arrive through `Calc`. +- **Fix:** `hle_swkbd.c` takes over every entry that touches the applet: `Appear` arms a request, and the next `Calc` asks for the text the way `ShowKeyboard` does (`DM_KBD`, else the front end's input box) and hands it to the game's changed-string and enter callbacks, or calls its cancel callback; `GetImage` reports no picture. `GetImageMemoryRequirement` is not a pure computation: it asks vi, hence the `vi::GetIndirectImageMemoryRequirement` hook ([vi layer and display calls without vi service objects](#vi-layer-and-display-calls-without-vi-service-objects)). +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md). + +### Inline keyboard window size, setters and callbacks + +- **Symptom:** texture creation returns null right after the inline keyboard's `Initialize`, then a null read at `main+0x7713044`. +- **Cause:** the game makes three textures of `swkbd::InlineKeyboard::GetWindowSize`, and the host answered 0 × 0 (its creation function returns null for a size of 0, `main+0x77112b8`). +- **Fix:** it is 1280 × 720 as on hardware now. SDK 11 adds keyboard setters (window mode, key top alpha / scale / position, touch, hardware keyboard, user words, dictionaries; ignored) and two callbacks the applet makes from `Calc`: finished-initialize after `Initialize` and released-user-words after `UnsetUserWordInfo`. The host makes them on the next `Calc`. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md). + +### Inline keyboard text callbacks and `Calc` states + +- **Symptom:** the hunter's name box stays empty, then stays open with the name in it (Monster Hunter Rise, character creation). +- **Cause:** the inline keyboard used to count every `Appear` as a cancel. The game also finishes the entry only when `Calc` returns 1 (ready / hidden) after a non-zero value (`main+0x6fb3e6c`); the hook returned nothing. +- **Fix:** the text arrives through the enter callback, whose argument is the text inline (`char16_t[502]`, the length at +0x3EC; the changed-string argument adds three cursor positions at +0x3F0..+0x3F8: `ikb_str_arg`, read off the game's callbacks at `main+0x6fb3910` / `main+0x6fb3afc`). `Calc` now returns 0 until initialization finishes, 3 while shown and 1 otherwise. `SetFinishedKeyboardCallback` is called on the `Calc` after `Disappear`. AppearArg's `textMaxLength` (+0x20) limits the text. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md). + +### `nn::ngc::ProfanityFilter` vtable and by-name overloads + +- **Symptom:** `CRASH sig=10` at a heap address, called from `main+0x6bca160` right after the name was entered (Monster Hunter Rise); crash in `lmem::AllocateFromExpHeap` under `ngc::ProfanityFilter::MaskProfanityWordsInText` right after the character name is entered (DARK SOULS: REMASTERED); filter methods among the undefined imports with no hook, found from the import list before they were met (The Great Ace Attorney Chronicles). +- **Cause:** `nn::ngc::ProfanityFilter` is polymorphic and Monster Hunter Rise masks the name through its vtable (slot 8, `MaskProfanityWordsInText(int *, char16_t *, uint32_t)`). The constructor hook did nothing, so the object kept what the heap held. Other games call the methods by name: DARK SOULS: REMASTERED calls the UTF-16 overload (`EPiPtj`) by name, and the host had taken it over only in the filter's vtable, so the SDK's version ran on a filter whose word list the host never loaded; The Great Ace Attorney Chronicles calls `ngc::ProfanityFilter::MaskProfanityWordsInText(int *, char *, uint32_t)`, `SetMaskMode` and `Finalize` by name rather than through the vtable. +- **Fix:** the filter now gets a host vtable (`ngc_vtable` in `hle_misc.c`): each slot of the SDK's `_ZTVN2nn3ngc15ProfanityFilterE` is matched by address with the class's methods, and slots the loader already bound to forced take-overs keep them. The `unsigned short` and `char16_t` overloads are taken over by name too, as are the `char` overload, `SetMaskMode` and `Finalize` (`hle_misc.c`). Nothing is masked or reported. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (the vtable); [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (SDK 4.5.0; one boot: the UTF-16 overload by name); [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1: the `char` overload, `SetMaskMode`, `Finalize`). + +## Display, window and operation mode + +### `vi::CreateLayer` with the layer's size + +- **Symptom:** SIGSEGV in `vi::CreateLayer(Layer**, Display*, int, int)` (`sdk+0x3f9bc4`, from SDL3's Switch video driver at `main+0x239304`). +- **Cause:** the overload with the layer's size; the host had only the two-argument one. +- **Fix:** it returns the static layer (`hle_vi.c`). +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) (SDK 18.3.2, FNA on .NET NativeAOT with SDL3; one boot each). + +### vi layer and display calls without vi service objects + +- **Symptom:** SIGSEGV in `vi::SetLayerCrop`. +- **Cause:** the SDK's version dereferences vi service objects the host never made. +- **Fix:** `SetLayerCrop`, `DestroyLayer`, `CloseDisplay` and `vi::Finalize` are host no-ops. Also hooked: `vi::GetIndirectImageMemoryRequirement` (asked by the inline keyboard's `GetImageMemoryRequirement`, [The inline keyboard as a resident library applet](#the-inline-keyboard-as-a-resident-library-applet)); `vi::ListDisplays` (one display, "Default"; the game lists 10 into a stack buffer and does not read them, `main+0x808f08`), found from the import list before it was met. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0); [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10: `vi::GetIndirectImageMemoryRequirement`); [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1: `vi::ListDisplays`). + +### Layer frame numbers + +- **Symptom:** SIGSEGV in `android::Surface::getQueuedFrameNumber` under `nn::vi::GetLatestFrameNumber`. +- **Cause:** Unreal paces frames by the layer's frame number (`main+0x10140e8`); the SDK reads the vi service's surface, which the host never made. +- **Fix:** `hle_vi.c` answers with the frames presented so far; `ListFrameHistory` returns no entries. +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md). + +### Default display panel identifier + +- **Symptom:** SIGSEGV in `vi::GetDefaultDisplayPanelIdentifier` (`main+0xa9a4`, before the window). +- **Cause:** the SDK's version calls through the vi service object the host never made. The game reads a u32 kind and, from kind 2, 0x14 more bytes; anything else makes it use zeros. +- **Fix:** the host answers an all-zero identifier (`hle_vi.c`). +- **Note:** a passed TV latency calibration in this game is saved per TV panel identity, the host's all-zero one included ([TV latency calibration's late, beat-timed listening window](game-state.md#tv-latency-calibrations-late-beat-timed-listening-window)). +- **Found on:** [Rhythm Heaven Groove](../../dm/games/0100D9F01D474000/README.md) (SDK 21.4.0, one boot). + +### oe calls answered without an applet proxy + +- **Symptom:** the first boot crashed in `oe::DisableAlbumImageTakenNotification`: a null read at `sdk+0x338070`, the first call in `nnMain` (`main+0x6b188c`); other `oe` calls that reach am, found in the imports before they were met. +- **Cause:** the SDK asks am through the applet proxy the host never made. +- **Fix:** hooked (`hle_oe.c`): `oe::DisableAlbumImageTakenNotification`; `oe::IsInStoreDemoModeEnabled` (no); `oe::GetDefaultDisplayResolution` and its change event; `oe::SendServerMaintenanceOverlayNotification`; `oe::SetUserInactivityDetectionTimeExtendedUnsafe`; `oe::GetPerformanceConfiguration`; `oe::SetAlbumImageOrientation`; `oe::Begin` / `EndAutoSleepDisabledSection` and `IsAutoSleepDisabled` (counted); `oe::ExecuteProgram` (logged red: the host runs one program). The ones with a symptom of their own follow: [The startup logo](#the-startup-logo), [Copyright frame buffer overlay](#copyright-frame-buffer-overlay), [Applet resource user ID, launch parameters and VR mode](#applet-resource-user-id-launch-parameters-and-vr-mode), [CPU overclock calls without an apm session](#cpu-overclock-calls-without-an-apm-session), [Focus handling mode decides whether focus changes are notified](#focus-handling-mode-decides-whether-focus-changes-are-notified). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11: `GetDefaultDisplayResolution`, `SendServerMaintenanceOverlayNotification`); [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (SDK 11: `SetUserInactivityDetectionTimeExtendedUnsafe`); [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10: `GetPerformanceConfiguration`, `SetAlbumImageOrientation`); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (SDK 7.3.2: the auto-sleep calls); [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (`oe::ExecuteProgram`); [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1, Capcom's MT Framework: `DisableAlbumImageTakenNotification`, then `IsInStoreDemoModeEnabled` from the import list). After all of its fixes The Great Ace Attorney Chronicles runs its frame loop at 30 fps; the black screen it showed was its textures, never uploaded (CPU-written textures, the profile's `tex_flush_upload` and `tex_dirty_upload`: [Mapped-range flushes taken as uploads (`tex_flush_upload`)](nvn-objects.md#mapped-range-flushes-taken-as-uploads-tex_flush_upload), [CPU writes with no flush, caught by page protection (`tex_dirty_upload`)](nvn-objects.md#cpu-writes-with-no-flush-caught-by-page-protection-tex_dirty_upload)). + +### The startup logo + +- **Symptom:** abort in `nn::sm::Initialize` under `nn::oe::FinishStartupLogo`, right after the first frames. +- **Cause:** it reaches the applet proxy through `am::GetApplicationFunctions`. +- **Fix:** the host has no startup logo: a no-op hook (`hle_oe.c`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). + +### Copyright frame buffer overlay + +- **Symptom:** null read in `oe::InitializeCopyrightFrameBuffer`, right after the window setup. +- **Cause:** the copyright frame buffer is an overlay the system draws only into screenshots and recordings; the SDK hands it to am as transfer memory. +- **Fix:** `InitializeCopyrightFrameBuffer`, `SetCopyrightImage` and `SetCopyrightVisibility` are host no-ops (`hle_oe.c`). +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md) (SDK 9.6). + +### Applet resource user ID, launch parameters and VR mode + +- **Symptom:** SDK 20 calls that reach a service, each one boot; SDK 20's `OpenAudioRenderer` aborts on an ARUID of 0. +- **Cause:** the SDK's own `applet::GetAppletResourceUserId` returns 0 without am, and SDK 20's `OpenAudioRenderer` aborts on that. +- **Fix:** `applet::GetAppletResourceUserId` reports 0x80; the VR-mode calls (`pl::IsVrMode` / `BeginVrMode` / `EndVrMode`); `pctl::CheckStereoVisionPermission` (bool, permitted); `oe::TryPopLaunchParameter` (none) ([The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) 1.9.0, SDK 20.5.22). `oe::GetPreviousProgramIndex` reports −1: launched from the home menu ([The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) 1.4.3). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md), [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md). + +### CPU overclock calls without an apm session + +- **Symptom:** a null read in `sdk+0x35258` (apm's session) in `oe::IsCpuOverclockEnabled`, the first call in `nnMain`; null read at `sdk+0x362b8` (`apm::SetCpuOverclockEnabled`) in the first calls of `nnMain` (`main+0x2e1e4c`). +- **Cause:** `oe::IsCpuOverclockEnabled` and `oe::SetCpuOverclockEnabled(bool)` go through the apm session the host never opened. +- **Fix:** `oe::IsCpuOverclockEnabled` is hooked; for `SetCpuOverclockEnabled` the host keeps the flag and the query reports it (`hle_oe.c`); it does not scale clocks. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11: `IsCpuOverclockEnabled`); [Taiko Risshiden V DX](../../dm/games/0100346017304000/README.md) (SDK 12.3.3: `SetCpuOverclockEnabled`). + +### Focus handling mode decides whether focus changes are notified + +- **Symptom:** the game freezes after switching apps and back, picture still, sound on; the main loop waits in its pause handshake (`main+0x5d10d4` / `main+0x5cea68`). Mac repro `DM_OE_SCRIPT=bg=1@20,bg=0@25`. +- **Cause:** the game's own pause deadlocks. The game set focus handling mode 3. The SDK (`oe::SetFocusHandlingMode`, SDK 18 `sdk+0x3c5770`) turns modes 0 and 3 into "no focus notifications, the system suspends the game out of focus"; only 1 and 2 notify. The host sent message 15 anyway, and Galaxy 2's Wii layer ran a pause the console never triggers. +- **Fix:** `hle_oe.c` now queues 15 only under modes 1 and 2 (or when no mode was set). +- **Found on:** [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md) (one of two causes of the freeze; the other is [Guest clock reads wait while the app is in the background](#guest-clock-reads-wait-while-the-app-is-in-the-background)). +- **Note:** modes seen: 1 Monster Hunter Rise; 2 Wonder, Breath of the Wild, Tears of the Kingdom, Link's Awakening, Cuphead; 3 Galaxy, Galaxy 2, 3D World, Odyssey. + +### Album calls fail with no album + +- **Symptom:** abort in `sm::Initialize` under `capsrv::InitializeForApplication` (`album::Initialize`); calls that reach the album service, found in the imports before they were met. +- **Fix:** with no album they fail: `album::Initialize` (saving to the album now fails, `hle_offline.c`); SDK 7's `album::SaveScreenshot` without an orientation (fails); SDK 20's `SaveAndShareScreenshot` with a share ID and `GetSharePostedLastUrl`; `album::SaveCurrentScreenshot`. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11: `album::Initialize`); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (SDK 7.3.2: `album::SaveScreenshot`); [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) (SDK 20.5.4: the sharing calls); [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11: `SaveCurrentScreenshot`). + +## Accounts, network, shop and parental controls + +### Account, friends, play data and Nintendo Account calls + +- **Symptom:** a null read at `sdk+0x336ccc` in `pdm::QueryApplicationPlayStatistics` (Pokémon Sword, the second boot); undefined `nn::account::` / `nn::friends::` imports with no hook, found from the import list ([Listing a game's unhooked imports before they are met](#listing-a-games-unhooked-imports-before-they-are-met)) before they were met. +- **Fix:** hooked (`hle_account.c`, `hle_friends.c`, `hle_offline.c`): `account::ListAllUsers`; `account::PushOpenUsers` (nothing to hand over); `account::LoadCachedNintendoAccountInfo` (fails: no Nintendo Account; Big Brain Academy then skips the birthday and country getters); `friends::GetBlockedUserList`, `DeclareOpen` / `DeclareCloseOnlinePlaySession` (unavailable); `friends::StartFacedFriendRequest` / `StartSendingFriendRequest` (fail); the Nintendo Account request's `GetAuthorizationCode` and `GetState` (both fail, as `GetIdToken`); `pdm::QueryApplicationPlayStatistics`, which asks how often this user launched two other games and reads only the launch count, so each is never played. +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10, `ListAllUsers`); [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (`PushOpenUsers`); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (the friends calls); [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (the Nintendo Account request); [Pokémon Sword](../../dm/games/0100ABF008968000/README.md) (SDK 7, `pdm`); [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (SDK 10.4.1: the friend requests); [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3: `LoadCachedNintendoAccountInfo`). +- **Note:** [Super Mario Odyssey](../../dm/games/0100000000010000/README.md)'s `account::OpenPreselectedUser` and `account::GetLastOpenedUser` are in [SDK calls that open a system service through sm](#sdk-calls-that-open-a-system-service-through-sm); SDK 6's `OpenPreselectedUser` in [SDK 6's `account::OpenPreselectedUser` returns a Result](#sdk-6s-accountopenpreselecteduser-returns-a-result); SDK 4's `GetBlockedUserList` without a `Uid` in [SDK 4's `friends::GetBlockedUserList` without a `Uid`](#sdk-4s-friendsgetblockeduserlist-without-a-uid); the save thumbnail calls in [Directory, file data cache and save-thumbnail calls](#directory-file-data-cache-and-save-thumbnail-calls). + +### Mii database answered by the host + +- **Symptom:** `[RED] sf: the game connects to service mii:e … mii:u itself`, then an abort. +- **Cause:** `nn::mii::Database` keeps an `IDatabaseService` of the mii service at +0 (`Database::Initialize`, `sdk+0x1c5664`), and `AppendMii` / `ShowMiiEdit` start the MiiEdit applet. +- **Fix:** `hle_mii.c` takes over the public methods: a console with no Mii of its own (an empty database: no index, nothing to update) that still has the six default Miis every console has. A CharInfo the host builds is the default Mii's parameters with Switch common colours, a create ID with version 4 and variant bits (fixed per default Mii, so a saved choice finds the same Mii; fresh for `BuildRandom`, which the title's demo race calls for 12 racers) and the nickname "no name"; every field is within `CharInfoRaw::Verify`'s ranges (`sdk+0x1b68c8`). The Mii editor is cancelled (0xE7E, which `ShowMiiEdit` itself treats as success). +- **Found on:** [Mario Kart 8 Deluxe](../../dm/games/0100152000022000/README.md) (4.0.0, SDK 4.4.0; one boot). The game asks for the default Miis with `Get(…, 6, SourceFlag 2)` and never reads the count (`main+0x81e954`): its boot-time list of Mii icons (`main+0x419dc8`) builds a head model for each of the six, and zeroed entries failed the model's `CharInfoAccessor::IsValid` (`CharModel::Initialize` returned 0xd67e, `main+0x8f15b0`) and crashed at `main+0x8f152c`. +- **Note:** the heads' model data is a stand-in with every part empty ([System data stand-in for the MiiModel archive](#system-data-stand-in-for-the-miimodel-archive)). + +### Shop service accessors + +- **Symptom:** the abort with no message (`SVC 0x15` / `0x1f` / `0x21` placeholders, then `Break` with `sm::Finalize` under `ShopServiceAccessServer::Proxy::Initialize`) from `ec::InitializeForShopServiceAccessors`, called from Unreal's `FOnlineSubsystemSwitch::Init` at the start of `nnMain`. +- **Cause:** the shop service accessor library reaches the shop service. +- **Fix:** the library init succeeds with nothing to set up, an accessor never initializes, and its async response is complete at once with no data (`hle_misc.c`); the accessor and response are stubbed whole because the SDK's destructor aborts unless `Finalize` succeeds. Also hooked: `ec::FinalizeForShopServiceAccessors`, the shop response's `Cancel` and `GetErrorCode`, and `ec::ShowShopSubscriptionDetails`. +- **Found on:** [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (SDK 7.3.2: the game then runs its opening movie at 30 fps); [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (SDK 19.3.1: `FinalizeForShopServiceAccessors`, `Cancel`, `GetErrorCode`, `ShowShopSubscriptionDetails`, found by the import list). +- **Note:** the shop's `Show…` pages are under [System applets opened by `Show…` calls](#system-applets-opened-by-show-calls). + +### Parental controls and free communication + +- **Symptom:** abort in `nn::sm::Initialize` under `nn::pctl::IsFreeCommunicationAvailable` (Engage, entering its first battle map). +- **Cause:** the SDK connects to the parental-control service through `sm`. +- **Fix:** the host answers it like the other free-communication calls: not available (`hle_misc.c`). `pctl::CheckFreeCommunicationPermission`: never. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md); [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md) (`CheckFreeCommunicationPermission`). +- **Note:** `pctl::CheckStereoVisionPermission` is under [Applet resource user ID, launch parameters and VR mode](#applet-resource-user-id-launch-parameters-and-vr-mode). + +### Network interface, LDN, socket and play report calls + +- **Symptom:** play report calls met entering the save select (3D World); undefined `nn::ldn::` / `nn::socket::` imports with no hook, found from the import list before they were met (Big Brain Academy). +- **Fix:** hooked (`hle_net.c`, `hle_offline.c`): the `nifm` IP queries with SDK 10's `socket::InAddr` / `in_addr` mangling; `prepo::AddSessionId` / `AddInternetConnectionStatus`; `ldn::GetNetworkConfig`; SDK 11's socket overloads with plain `sockaddr` / `int` arguments (`Connect`, `Accept`, `GetSockName`, `Get` / `SetSockOpt`, `Fcntl`, `Recv`, `Send`). +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10: `nifm`, `prepo`); [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (SDK 11.4.3: LDN, sockets). + +### Network requests stay on hold before they settle + +- **Symptom:** the boot sits at a black loading screen for good, CPU nearly idle, every thread waiting, about 1 boot in 6 (3 in 4 with four Mac hosts at once); the title movie is never opened, and the save files are never probed. +- **Cause:** a race the host's instant offline answers lost. The game's boot state machine (`main+0x1f2f0e0`, state at `this+0xd4`, found by probing the save probe's call stack and dumping the state each tick) stayed in state 7, which polls the status word (+0x14) of the game's BCAT news runnable ("OlimarMemo") for 0, 4, 5, 6 or 7; it read 1. The runnable's constructor (`main+0x203f3b0`) clears the status, starts `FBCATThread` and only then stores 1; the thread's `Run` (`main+0x203f510`) stores 2, calls `nn::nifm::SubmitNetworkRequest`, sleeps while `IsNetworkRequestOnHold`, and stores 6 when `IsNetworkAvailable` is false. On a Switch the request stays on hold while the nifm service evaluates it, so the constructor's 1 always comes first; the host answered "not on hold" at once, and a thread that ran before the constructor's store finished at 6, which the 1 then overwrote. +- **Fix:** the host now keeps a submitted request on hold for 50 ms, then settles it as unavailable, as Horizon's asynchronous request does (`hle_net.c`). +- **Found on:** [Pikmin 4](../../dm/games/0100B7C00933A000/README.md) (Unreal Engine 4.26.2). +- **Check:** 12 of 12 boots with four hosts at once reached the title. +- **Note:** how it was found: the hung boots' logs diverged from good ones at the save probe (`DM_FS=1`), the thread dumps showed nothing but idle waits, and `DM_PROBE` on the state machine, then `peek` lines in a hung run's `DM_CTL` file, followed the pointers to the status word (a probe on the hot path slowed the boot enough to hide the race: 8 of 8 good). + +### Socket `Poll` and `Select` with nothing to wait on + +- **Symptom:** one thread at 100% CPU from boot, with one SVC in 99 s (`NexusRevolution Socket`, `main+0x92c7c4`, the game's network thread). +- **Cause:** it calls `nn::socket::Poll(fds, 0, 33)`: no descriptors and a 33 ms timeout, which on hardware is a 33 ms sleep. The host failed every socket call at once, so the thread spun 11,000 times a second (88,000 hits of a probe in 8 s). +- **Fix:** `Poll` and `Select` with nothing to wait on (no descriptors, or only negative ones, which POSIX ignores) now wait out their timeout and return 0, in every mangling the host takes over, including SDK 4's `Select(int, fd_set*, …, timeval*)`; an endless timeout returns 0 after 100 ms, since a thread inside a host hook pauses for IL2CPP's collector only once the hook returns (`hle_net.c`, `dm_guest_sleep_ns` in `hle_thread.c`). +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (SDK 4.5.0; one boot). Guest CPU fell from 1,030 to 60 ms per second. + +## Settings, languages and fonts + +### Language codes + +- **Symptom:** the first comparison in `nnMain` aborted in `sm`. +- **Fix:** hooked: `settings::LanguageCode::Make` and the reversed `operator==(LanguageCode, Language)`. +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10). + +### Available language codes and their count + +- **Symptom:** abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodes` (early in `nnMain`, `main+0x327888`); abort in `sm::Initialize` under `nn::settings::GetAvailableLanguageCodeCount()` (`main+0x117590`). +- **Cause:** Paper Mario asks for up to 32 codes and takes the index of `oe::GetDesiredLanguage`'s code in the list as its language. Catherine compares `LanguageCode::Make(0 … count − 1)` with `GetDesiredLanguage`'s code. +- **Fix:** the host lists the system's codes in `nn::settings::Language` order, the table `LanguageCode::Make` uses, and reports the count of that table (`hle_sdk16.c`). +- **Found on:** [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (SDK 15.3.2, the codes), [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md) (SDK 8.2.1, the count; one boot). + +### Region code from the language + +- **Fix:** `oe::GetRegionCode` answers from the language: Japan, the Americas, Europe or Hong Kong / Taiwan / Korea. +- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) (SDK 20.5.4; found in `main`'s imports before the first boot). + +### Shared font address: BFTTF header or bare TTF + +- **Symptom:** the font engine runs on garbage; crash after the shared fonts load, looking up the "?" glyph (`main+0x6c73c0`, a font cache lookup returning null), with SDK 10's `ScalableFontEngine::LoadFont` failing with error 403 and `SetFont` with 301 on both stand-in fonts; or Japanese, Korean or Chinese text draws nothing in a Unity game while Latin text is fine (Patrick's Parabox's language menu showed three blank lines). +- **Cause:** consumers read the shared font differently. + - The SDK's font engine calls `ScalableFontEngineHelper::Decode(ptr, size)`, which XOR-decodes in place unless the first word is the "already decoded" marker (LE `0x18029a7f`). So for a game that decodes it the system shared font must look like BFTTF: serve `marker + bswap(size ^ 0x49621806) + TTF` (`hle_sdk16.c`). Mario decodes the pointer and needs the 8-byte BFTTF header in front. + - 3D World records shared fonts as type 0 (`main+0x975658`), calls no `ScalableFontEngineHelper::Decode` and hands the `GetSharedFontAddress` pointer straight to `LoadFont`, so the pointer must be the plain TTF. SDK 10 and older games all take the plain TTF. + - Unity's Switch player hands the shared fonts straight to FreeType. In Patrick's Parabox (SDK 13) the game's one font (Jost, a Unity dynamic font) falls back to the OS fonts for those glyphs, and the player loads them from the shared fonts: `main+0x2390b0` maps the names `Switch_Font_STD`, `Switch_Font_CN_Simplified`, `Switch_Font_CN_Extended`, `Switch_Font_CN_Traditional`, `Switch_Font_KR` and `Switch_Font_NIN_Extended` to types 0–5 and hands `GetSharedFontAddress` / `GetSharedFontSize` straight to FreeType's `FT_New_Memory_Face` (`main+0x23b2e0`), as libnx homebrew does with the same pl offsets. The host gave every SDK 11+ game the 8-byte BFTTF header in front, so every face failed to open. + - The SDK's `GetSharedFontAddress` is the shared memory base plus the pl service's offset, with no adjustment, and that offset points past the header (yuzu's pl service: "font offset and size do not account for the header"); Captain Toad (SDK 8) records its shared fonts as plain TTFs and calls `Decode` only for its BFTTF formats. + - A game on a newer SDK can import `Decode` (and so get the header) yet hand the address straight to `LoadFont` the same way (The Legend of Zelda: Echoes of Wisdom 1.1.0, SDK 20.5.1: `main+0x8b37c` crashed at address 0x38). +- **Fix:** the host first decided by the SDK version alone (SDK 10 and older the TTF, SDK 11+ the header). Now the address is the header (marker form above) only for a game on SDK 11 or later whose `main` imports `nn::fontll::ScalableFontEngineHelper::Decode` (Wonder, Breath of the Wild, Tears of the Kingdom, Pokémon Legends: Z-A); every other game gets the TTF itself (`font_skip` in `hle_sdk16.c`, `dm_main_imports` in `loader.c`; the profile key `shared_font` overrides). The host's `LoadFont` takeover passes the TTF past the header whenever it is given the host's own BFTTF base, so the address form `font_skip` picks only matters to `Decode` and other readers (`hle_sdk16.c` `h_font_LoadFont`). +- **Check:** 3D World's case was found by probing the engine's error code after each call (`DM_PROBE` on the instruction after `GetError`). +- **Note:** of the games on hand, the ones that call `GetSharedFontAddress` without importing `Decode` are all Unity players (Engage, Ace Attorney Investigations Collection, Brain Memory Training Academy, Patrick's Parabox). To see which fonts a Unity game asks for, probe its name mapper's entry (`DM_PROBE=main:2390b0` on Patrick's Parabox 1.0.1; the function is the one `main` caller of `GetSharedFontLoadState`). +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10, the plain TTF); [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md), [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md), [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md), [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (the `Decode` importers); [Patrick's Parabox](../../dm/games/0100E44016C22000/README.md) (the blank CJK text, 1.0.1); [Captain Toad: Treasure Tracker](../../dm/games/01009BF0072D4000/README.md) (SDK 8, plain TTFs); [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md), [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md), [Brain Memory Training Academy](../../dm/games/0100F5D01DA3A000/README.md) (the other Unity players); The Legend of Zelda: Echoes of Wisdom 1.1.0 (no profile). + +### Shared fonts requested all at once + +- **Symptom:** abort in `nn::sm::Initialize` under `nn::pl::sharedresource::GetPlatformSharedResourceManager` (first thing in `nnMain`). +- **Cause:** the game asks for every shared font at once with `pl::GetSharedFont(int *, SharedFontData *, int, LanguageCode)` (`sdk+0x40f270`), not by type. `SharedFontData` is 0x18 bytes {u32 type, address, size}; the SDK adds the pl service's offsets, which point past each font's 8-byte BFTTF header, to the shared memory base, and the game hands each address straight to `ScalableFontEngine::LoadFont` (`main+0x501ff0`). +- **Fix:** the hook returns the six types in order with the stand-in TTFs past the header, whatever the profile's `shared_font` (`hle_sdk16.c`). +- **Found on:** [Rhythm Heaven Groove](../../dm/games/0100D9F01D474000/README.md) (SDK 21.4.0, one boot). + +### Shared font load request + +- **Fix:** `pl::RequestSharedFontLoad` is hooked (an SDK 20 call that reaches a service). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (1.9.0, SDK 20.5.22). + +### Nintendo-extension glyphs for button icons + +- **Symptom:** controller-button icons in text show as `?`. +- **Cause:** those glyphs come from the system shared font's Nintendo-extension set (private-use code points; Mario's `GameMsg/Key.msbt` uses U+E0E0–E0F0 for A/B/X/Y/L/R/ZL/ZR/SL/SR/D-pad/+/−, U+E0AB–E0AE for D-pad directions, U+E090–E097 for stick arrows, U+E100/E101 for sticks). +- **Fix:** `dm/tools/make_ext_font.py` generates a stand-in `nintendo_ext.ttf` with those icons; the host serves it for shared-font type 5 (`DM_FONT_DIR` or `DM_SHARED_FONT_EXT`) and a desktop font for the other types. +- **Note:** to find which code points a title uses, scan the `TXT2` sections of its MSBT files for U+E000–F8FF. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Shared fonts on iOS + +- **Symptom:** on a phone, `GetSharedFontLoadState` would report "not loaded". +- **Cause:** the macOS stand-in path (`/System/Library/Fonts/Supplemental/Arial Unicode.ttf`) does not exist on a phone. +- **Fix:** the iOS app ships `dm/font/standard.ttf` and `dm/font/nintendo_ext.ttf` (copied / generated by `build_ipa.sh`) and sets `DM_FONT_DIR`. +- **Found on:** every game on iOS (general). + +## C library, time and random numbers + +### Time zone rules, calendar conversion and the network clock + +- **Fix:** hooked: `time::ToPosixTime` (host `mktime`), and `time::ToPosixTime` with an explicit rule (the host's zone, as for `ToCalendarTime`); `time::LoadTimeZoneRule` and the `ToCalendarTime` overload with a rule (the host's zone is used); `time::StandardNetworkSystemClock::IsAccuracySufficient` (offline: no). `time::GetDayOfWeek` is a pure computation and needs nothing. +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md) (SDK 11, `ToPosixTime`); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (SDK 11, the rest); [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (`GetDayOfWeek`); [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11, `ToPosixTime` with an explicit rule, found in the imports before it was met). + +### Steady clock, clock snapshots and user clocks + +- **Symptom:** SDK calls that reach the time service, each aborting in `sm::Initialize` (or found in the imports before they were met). +- **Fix:** + - `time::StandardSteadyClock::GetCurrentTimePoint`: host monotonic seconds, fixed source ID; `GetSpanBetween` runs in `sdk` ([The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) 1.9.0, SDK 20.5.22). + - `time::ClockSnapshot::CreateWithStandardSystemClock`: 0xD0 bytes, filled on the host from the steady clock, the user clock and the calendar; the network fields stay 0 ([The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) 1.4.3). + - The adjustable user clock (`time::AdjustableUserSystemClock::GetCurrentTime`, which the game's days follow) reads the time service's shared memory in the SDK (`MonotonicSystemClock::GetCurrentTime`, `sdk+0x4421cc`), which the host never maps, so it is the host's clock, as the user clock; `time::GetStandardSteadyClockCurrentTimePoint`, the free-function form of the steady clock ([Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md) SDK 20.5.4, found in `main`'s imports before the first boot). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md), [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md), [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md). + +### `clock_gettime(CLOCK_REALTIME)` through the time source pointer + +- **Symptom:** `std::terminate` (`Break` from `sdk+0x5bfa48`) at the first real-time read (right after the first-boot dialog, `main+0x3f3f68` → `system_clock::now()`). +- **Cause:** `clock_gettime(CLOCK_REALTIME)` calls through a source pointer (`sdk+0xb05010`) that `nntimeStartup` sets once `time::Initialize` succeeds: 1 = `nn::time::GetStandardUserSystemClockCurrentTimespec`, 2 = the adjustable clock's (`sdk+0x8e414`). Both asked the time service. +- **Fix:** they are host take-overs now (`hle_time.c`), and `system_clock::now()` works on any build. +- **Found on:** [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) (1.0.0.2, SDK 17.5.3). + +### Guest clock reads wait while the app is in the background + +- **Symptom:** after switching apps and back, one guest thread at 100% in a tight loop (`main+0x9abd0`, `x10 = 0`); the game freezes, picture still, sound on. Mac repro `DM_OE_SCRIPT=bg=1@20,bg=0@25`. +- **Cause:** the frame pacing measured a frame period of 0 (`main+0x5926a8`: the clock difference between two frames). The host freezes the guest clock in the background but its threads went on, so two frames read the same time. +- **Fix:** guest reads of the system counter (the `cntpct` gate, `svc GetSystemTick`) now wait while the app is in the background (`hle_os.c` `guest_clock_read_wait`); host work and the keyboard freeze ([The software keyboard blocks only its caller](#the-software-keyboard-blocks-only-its-caller)) keep the frozen clock. +- **Found on:** [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md) (one of two causes; the other is [Focus handling mode decides whether focus changes are notified](#focus-handling-mode-decides-whether-focus-changes-are-notified)). + +### Cryptographically random bytes + +- **Symptom:** abort in `sm::Initialize` under `nn::crypto::GenerateCryptographicallyRandomBytes` (Unicorn Overlord: `std::random_device` → the `sdk`'s own `getentropy` → `InitializeCsrng`; Z-A: the first call in `nnMain`, `main+0x1734b4`). +- **Cause:** the bytes come from the csrng service. The `sdk` defines `getentropy`, so the host's `getentropy` hook never sees the call, but the call into `nn::crypto` goes through the `sdk`'s PLT. +- **Fix:** an `HLE_FORCE` on it fills from the host's entropy stream, the same stream as `getentropy` (`hle_os.c`). +- **Found on:** [Unicorn Overlord](../../dm/games/010069401ADB8000/README.md) (SDK 16.2.4; one boot), [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (19.3.11; one boot). + +## Modules and symbols + +### Global symbol resolution in load order + +- **Symptom:** a library's `free` rejects a pointer (`CentralHeap::Free` → `NotNMallocPtrError`). +- **Cause:** the game's `main` defines its own `operator new` / `delete` (Unity's memory manager) and `sdk` has a copy forwarding to `malloc` / `free`. +- **Fix:** symbols are resolved globally in load order (the first module's definition wins, also for a module's references to its own exports); only local / non-default-visibility symbols bind to the module itself (`loader.c`). The loader logs how many references bind to an earlier module. + +### Undefined weak imports bind to 0 + +- **Symptom:** a weakly imported function the game calls although nothing defines it: an `UNIMPL` line, then an abort. +- **Cause:** the loader bound every undefined import to a named stub, so the address test the compiler emits for a weak symbol saw a function. +- **Fix:** an undefined weak symbol that no module defines and the host does not implement is now 0, as Horizon's rtld binds it (`loader.c`, logged as `undefined weak … bound to 0`). +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md) (the `thread_local` init function `_ZTHN13LowLevelTasks5FTask10ActiveTaskE`). + +### `nn::ro` and its port + +- **Symptom:** crash in `nn::ro::InitializeWithPortName`. +- **Cause:** `nn::ro` connects to `ldr:ro`. +- **Fix:** take over `Initialize` and answer `LookupSymbol` from the loaded modules (`hle_ro.c`). Check the mangled names against the new title's imports: SDK 19's `LoadModule` takes one more bool (`_ZN2nn2ro10LoadModuleEPNS0_6ModuleEPKvPvmib`), so the hook for the older form did not catch it and the SDK's own code would have run. +- **Found on:** [Hades II](../../dm/games/0100A00019DE0000/README.md) (SDK 19's `LoadModule`). + +### `ro::ProtectRelro` runs natively + +- `nn::ro::ProtectRelro` (called from `main`'s DSO init), a new import of SDK 20, needs nothing: it runs natively on the host's no-op `SetMemoryPermission` / `SetMemoryAttribute`. Found on [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (1.9.0, SDK 20.5.22). + +### Loading NROs with `ro::LoadModule` + +- **Symptom:** the frame loop runs, but the game waits for code it loads itself (`ro::RegisterModuleInfo` / `ro::GetBufferSize` red lines; `main` imports 1,045 `lua2cpp::create_agent_*` functions, global, that no ExeFS module defines, bound to stubs at start). +- **Cause:** the fighters' code is 98 NROs inside `data.arc`, registered by `.nrr/cross2_*.nrr`. The SDK's own `LoadModule` (SDK 8.2.1 `LoadModuleInternal`, `sdk+0x347460`) uses the service only to map the image, then zeroes bss, relocates through rtld's module lists, calls the module's init and binds the variables of the modules loaded before; the host replaces rtld, so it takes over the whole call. +- **Fix:** the NROs are collected at pack time by the SHA-256s the NRRs list (`dm/tools/nro_collect.py`: loose files, the current files of an indexed archive, or a scan for stored and zstd-framed NROs; Smash's NRRs of older releases also register 666 superseded copies, which `data.arc` still holds and which are left out), rewritten like any module (`patch_nso.py` reads NROs: the flat image, segments at +0x20, build ID at +0x40) and listed in `<patched dir>/nro.tsv`. The host maps them all at start and keeps them out of symbol lookups; `LoadModule` finds one by the image's SHA-256, restores its data and bss (after the first load), relocates it against the modules loaded then in load order, binds the earlier modules' symbol relocations it defines (main's `lua2cpp` stubs), fills `nn::ro::Module` as the SDK does (+0 module, +8 state 1, +0x10 image, +0x18 buffer) and calls its `DT_INIT`; `UnloadModule` calls `DT_FINI` and binds every relocation that pointed into it again (`loader.c` "NRO modules", `hle_ro.c`). `GetBufferSize` returns the bss size from header +0x38 as the SDK does. The startup init pass must skip NROs (their init ran unrelocated and crashed). Only data is written at a load, so it works with code embedded in the iOS executable: `gen_embed.py` packs the NROs into three sections (`__dm_nro_x0`, `__DM_NRO_W`, `__dm_nro_x1`) and a table of their pieces (`__dm_nro_tab`), since a Mach-O file has at most 255 sections and three per NRO would pass it, and `map_image` maps an NRO from there (`loader.c` `nro_pieces`). +- **Found on:** [Super Smash Bros. Ultimate](../../dm/games/01006A800016E000/README.md) (13.0.5): it loads six NROs in its first 30 s and calls none of the stubs. Played on an iPhone. + +### Online SDKs in NROs are never loaded + +- **Symptom:** `[EOS] Failed to create platform` (the game plays offline); `refused …: an online SDK (Epic Online Services), not loaded`; `refused an NRR that registers only online SDKs (1 hashes)`. +- **Cause:** Epic Online Services in an NRO (`rom:/nro/EOSSDK-Switch-Shipping.nro`, loaded with `ro::LoadModule`; `main` imports the `EOS_*` functions). Before the host loaded NROs, the imports stayed `UNIMPL` stubs and `EOS_Platform_Create` handed back a garbage handle; the game checks it and plays offline. +- **Fix:** the host now loads NROs ([Loading NROs with `ro::LoadModule`](#loading-nros-with-roloadmodule)), but never an online SDK, whatever the game: an NRO that exports `EOS_Platform_Create` is left out of the pack (`dm/tools/nro_collect.py` `ONLINE_SDKS`, a `refused` row in `nro.tsv`) and `ro::LoadModule` refuses it with one log line (`refused …: an online SDK (Epic Online Services), not loaded`), also by the export when an older `nro.tsv` has it. An NRR that registers only such modules fails `RegisterModuleInfo`, as every registration did before the host loaded NROs, so the game takes the path it took then. So both games behave as before: offline, local saves. +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) (SDK 18.3.2, one boot each); [Hades](../../dm/games/0100535012974000/README.md): it never asks for the module (checked on 1.0.38233: `refused an NRR that registers only online SDKs (1 hashes)`, no `LoadModule`, its frames as before). Hades' `main` imports only EOS Auth, Connect and PlayerDataStorage, Epic-account cross-save the host cannot serve; the real SDK would at best fail its login and at worst wait on time-outs or contact Epic's servers, with an unknown risk to the player's account. +- **Note:** open question for the maintainer: always load, refuse online SDKs (as now), or decide per profile. + +### GPU driver modules that are never loaded + +- **Symptom:** `main` imports EGL (SDL3's GL support) and `vkGetInstanceProcAddr` (its Vulkan support), but the game renders with NVN (`FNA3D_Driver_NVN.cpp`). +- **Cause:** NVIDIA's OpenGL and Vulkan drivers (`subsdk1`, `subsdk0`) program the GPU through NvRm, which the host does not have. +- **Fix:** they are never loaded (`profile_guess.py` `GPU_DRIVERS`); the EGL imports stay unbound stubs that nothing calls. +- **Found on:** [Absolum](../../dm/games/0100A83022A22000/README.md) (FNA3D). + +### IL2CPP common string table intact + +- Not needed in the end: the tolerant common string table of pull request #15. [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) 1.0.0.2's `main` has the table intact at `main+0x46169ac`, 109 entries ending at `Hash128`. + +### Code inherited from another title at fixed addresses + +- **Symptom:** harmless on the build it was written for, silent memory corruption on any other. +- **Cause:** code that calls fixed module addresses. d2r-mobile's 3D render scale read a renderer pointer at `main+0x31e0848` and called `main+0x8582f0`; in Mario v0 that is the middle of a static initializer and only did nothing because the pointer happened to read 0. +- **Fix:** such code checks `module_t.build_id` now (written into `fixups.tsv` by the rewriter) and stays off on other builds. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (v0, code inherited from d2r-mobile). diff --git a/docs/reference/shaders.md b/docs/reference/shaders.md new file mode 100644 index 0000000..ac9ec01 --- /dev/null +++ b/docs/reference/shaders.md @@ -0,0 +1,557 @@ +# Shaders + +Finding, keying and translating shaders (`dm/gfx/tools/shader_split.py`, `dmsh_build.py`, `dmsh_xlate`, `Program.cs`) and their pipelines. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Containers by engine + +### BNSH in NintendoWare archives and in main + +- BNSH (`BNSH\0\0\0\0` + `grsc` block at +0x60) is found inside `bfsha`, particle effects (`esetb`, VFXB), UI layouts (`blarc`), and `main` itself. +- Variation table at +0x80 (count at +0x7c); binary program at variation +0x10; six stage pointers from program +0x08 (vs, hs, ds, gs, ps, cs). +- Stage code descriptor: control ptr +0x08, code ptr +0x10, code size +0x18, control size +0x1c. All pointers are relative to the BNSH start. + +### Nintendo EPD (sead / agl / gsys): BNSH, agl sharcb and resource packs + +- [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md): bfsha (BNSH) and agl sharcb (BAHS version 9, [next entry](#bahs-in-agl-sharcb)) in `ShaderData/*.szs`, particle effects (BNSH, including 25 compute shaders) in `EffectData`, six layouts and `SystemData/AglResource.szs` (BAHS), and two BNSH in `main`: 11,080 shaders. All bind with Mario's texture handle layout (`0x20 + 8 × slot`); the one failure is a geometry shader (fur fins, `RenderFinFur`), which Metal cannot express ([Geometry shaders with no Metal stage](#geometry-shaders-with-no-metal-stage)). +- [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): 19,592 shaders: `Shader/*.sbfsha` (BNSH) and the gsys / ksys / terrain resource packs inside `Pack/Bootup_Graphics.pack` and `Pack/TitleBG.pack`, the particle effects in `Effect/*.sesetlist` (11,373 shaders, 101 compute), layouts (`.sblarc`) and 37 in `main`. All bind with Mario's texture handle layout; the full build had no failure and no unresolved binding. + +### BAHS in agl sharcb + +- **How it is stored:** agl `sharcb`, the engine utility shaders: code region at header +0x1c (size +0x18). Binary entries in the table area are 16 bytes `{code offset, 0, control size (0x880/0x88c; 0x848 in BAHS version 9, Super Mario 3D World), signed offset from this entry to the control section}`. +- **Pairing:** control sections are shared between programs, so pair by entry, never by searching for code near a control. +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (BAHS version 9). + +### Zstd files recognised by their magic + +- **Symptom:** `shader_split.py` finds 0 shaders in a RomFS of zstd files named `.zst`. +- **Cause:** the split decompressed only `.zs` files. +- **Fix:** it now goes by the zstd magic (`28 b5 2f fd`) whatever the name. +- **Found on:** [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md) (8,726 of the RomFS's 9,959 files, the shader archives among them). Of the other 21 games' RomFS on hand only Super Mario Galaxy's `GDDR3.zst` (a Vulkan game, no NVN split) has such a file. + +### BNSH in Bezel Engine archives + +- **How it is stored:** the shaders are BNSH in the `.bfsha` entries of the `.bea` archives (zstd entries, [01](../agents/01-extract.md)), most of them the engine's own in `System.nx.bea` (`_BezelSystemResources/<system>/shader/*.bfsha`: PBR shadow maps 846, post effects 770, …), the rest in the layouts' `__ArchiveShader.bnsh` and the per-archive shader packs (`.bnbshpk`), plus one BNSH in `main`. +- **Reader:** `shader_split.py` scans each archive entry as a file of its own (`bea_members`). +- **Found on:** [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (NintendoWare Bezel Engine 1.56): 2,125 shaders in about 90 s. + +### BNSH entries of GFPAK archives + +- **Symptom:** a provisional split finds only a fraction of the shaders (3,126 of Pokémon Sword's 41,516). +- **Fix:** `shader_split.py` `gfpak_members` reads the BNSH entries of GFPAK archives (LZ4 entries, [01](../agents/01-extract.md)). +- **Found on:** [Pokémon Sword](../../dm/games/0100ABF008968000/README.md): 41,516 shaders, nearly all BNSH entries of the 7,699 GFPAK archives under `bin/archive` (the models' and effects' shader archives), beside loose BNSH files (`bin/battle/waza/model/**/*.bnsh_vsh` / `_fsh`, `bin/graphics`, `system_resource/shader`) and two in `main`. The YEBIS shaders it also draws are in `main` ([Shaders in a zlib stream in main](#shaders-in-a-zlib-stream-in-main-yebis-tfxo)). + +### BNSH in Trinity packs + +- **How it is stored:** every shader is a BNSH inside the Oodle-compressed files of `arc/data.trpfs` ([01](../agents/01-extract.md)). +- **Reader:** for any game with Trinity packs (`ONEPACK` files), `trpfs_jobs` / `trinity_shaders` in `shader_split.py` name the packs from `data.trpfd`, decode every file with the game's own Oodle in the Mac host (`DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script) and scan it like a loose file. +- **Copies that differ:** the layout archives' control sections are 104 KB long, and their stage descriptors run 456 bytes past the control's own length (control +0x20), with bytes that differ between copies of one shader; `same_shader` compares copies only up to the control's own length. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (Game Freak's engine): 28,603 shaders (11,633 vertex, 16,845 pixel, 125 compute) and 26,051 variation pairs: the material shaders in `system_resource/gfx2/shader/NX64/env_ikkaku` (`material/standard.bnsh`, `material/transparent.bnsh`, `system/light.bnsh`, and the `special/imgui.trsha` pack with the engine's system shaders), particle effects (`.ptcl` / `.trtml` in `ik_effectdata`, 19,842 of the shaders), UI layouts (`__ArchiveShader.bnsh` in the `.truiv` packs of `uidata`) and the 40 BNSH in `main`. + +### GLSLC outputs in main and in loose files + +- **How it is stored:** 13 Sentinels (Vanillaware's engine) has no shader file in RomFS, whose six CRI CPK archives hold textures, scripts, CRI audio and USM movies: 838 GLSLC compiler outputs sit in `main`'s read-only segment, each referenced from a `{pointer, size, 0}` table in `.data`. Other engines keep whole outputs in loose RomFS files of their own formats (below). +- **Layout:** an output starts with the magic `0x19866891`, its total size is at +0x44, and it holds exactly one control section with the code area right after it (control +0x20 = control length); a debug section with the NV assembly and the GLSL source follows. The stage comes from SPH word 0, as for BAHS. Outputs that hold several stages, and SDK 4's outputs, are read through their section headers ([GLSLC outputs holding several stages](#glslc-outputs-holding-several-stages), [GLSLC outputs read through their section headers](#glslc-outputs-read-through-their-section-headers)). +- **Reader:** `shader_split.py` reads them in every game's `main`, accepting only outputs that pass the control / code / length check. Loose RomFS files are searched the same way: it reads GLSLC outputs in any RomFS file no other reader takes. When the reader was written, of the other profiles' files only Metroid Dread's MSHD files held any, which `mshd_shaders` reads ([MSHD in MercurySteam's bshdat files](#mshd-in-mercurysteams-bshdat-files)). +- **Found on:** + - [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md): 838 in `main`. + - [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): 11 in `main`, Unity's own. + - [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md): 257 whole GLSLC outputs in the engine's technique files (`*.tec`: `resources_nx/release/master/shader/*.tec`, magic `TEC\0`, each output whole after a table of parameter names) beside 49 in `main`. + - [Absolum](../../dm/games/0100A83022A22000/README.md): 163 GLSLC outputs in `shaders.nx.bin` (`rom:/shaders.nx.bin`: a u32 count, then per entry a u32 hash, a u32 size and one whole output): 162 unique shaders, 83 pixel and 79 vertex, every vertex shader paired with the single-stage cover ([Single-stage programs paired with a cover](#single-stage-programs-paired-with-a-cover)). Its `.xnb` effects are XNA's D3D9 effects, MOJOSHADER linked in, which the GPU does not run. + - [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md): 3,050 loose files in `data/shaders_binary` (`*.vs` / `*.ps`, one stage each, one output per file), read like FINAL FANTASY TACTICS' technique files; its vertex shaders are in two parts ([VertexA and VertexB parts merged into one vertex shader](#vertexa-and-vertexb-parts-merged-into-one-vertex-shader)). + +### GLSLC outputs holding several stages + +- **Symptom:** the split found no shader in the game (0 shaders). +- **Cause:** an output may hold several stages compiled together as one program (Baba Is You: the 120 outputs in `Assets.dat`, each a pixel and a vertex shader, then a reflection section), which the single-stage reader does not take. +- **Fix:** such an output is read by its section headers (+0x4c section count, section *i*'s header at +0x90 + 0x90 *i*, type 0 = GPU code; `glslc_program`; the same headers SDK 4's outputs are read through, [GLSLC outputs read through their section headers](#glslc-outputs-read-through-their-section-headers)), and its stages are paired with each other, not as single-stage programs. The NSP script now stops at the split when it finds none. +- **Found on:** Baba Is You (no profile). + +### G2S containers in KTGL RDB archives + +- **How it is stored:** each material's shaders are a G2S container (`_S2G3000`, type KTID `0x7bcd279f`; 23,108 entries in `asset/system.rdb.bin*`, 2.3 GB inflated) of GLSLC outputs, as in 13 Sentinels' `main`, inside zlib-chunked IDRK entries ([01](../agents/01-extract.md)). One `_DOK0000` container (type `0x20a6a0bb`, the engine's system resources) holds 151 more bare GLSLC outputs, 21 of them drawn in the first battle. +- **Reader:** `shader_split.py` maps the RDB files and inflates only entries that start with those two magics (`idrk_shaders`), and the 117 outputs in `main` as in every game. Many outputs carry 0x850-byte control sections and the compute ones have no code header, so `glslc_shaders` accepts controls from 0x760 bytes and takes the stage of headerless code from the thread group size at control +0x748. Pairs come from the single-stage cover (`separable_pairs`). +- **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md) (Koei Tecmo's KTGL): 40,726 shaders (37,043 pixel, 3,440 vertex, 241 compute, 2 geometry), Mario's texture handle layout (`0x20 + 8 × slot`). [ROMANCE OF THE THREE KINGDOMS 8 REMAKE](../../dm/games/0100C2A01A730000/README.md)'s newer KTGL keeps them (`_S2G5000` / `6000` / `7000`, and `_DOK0000`) in `.fdata` files whose IDRK entries are LZ4 or zlib in chunks of another form ([01](../agents/01-extract.md)): 59,214 outputs there and 149 in `main`, all of them built. +- **Note:** the older KTGL of Three Houses packs the same G2S containers differently ([SPKG packages in KTGL LINKDATA](#spkg-packages-in-ktgl-linkdata)). + +### SPKG packages in KTGL LINKDATA + +- **How it is stored:** shader packages (`SPKG`: `patch*/nx/shader/*.spkg` and 132 entries of `DATA1.bin`) of G2S containers as in the [RDB entry](#g2s-containers-in-ktgl-rdb-archives), one `_SPK4100` package without G2S (the post effects, `DATA1.bin` entry 549), and map and model archives carrying packages of their own. +- **Reader:** `kt_shaders` inflates every LINKDATA entry and loose `*.gz` / `*.spkg` file ([01](../agents/01-extract.md)), takes every GLSLC output that passes the triple check, and scans its SARCs for BNSH; `DATA1.bin` is split into 64 scan jobs by entry range (`linkdata_jobs`). +- **Symptom:** 24 shaders before the control-length fix, 1,355 after. +- **Cause:** its GLSLC (output version 0x1d.2) writes the 0x848 / 0x850-byte controls and headerless compute code described in the RDB entry, which `glslc_shaders` used to drop. A few outputs of version 0x1d.0 keep something else at control +0x20 (odd values such as 0x136b), so a control length must be a multiple of 4; the section-header reading now takes 8 of them ([GLSLC outputs read through their section headers](#glslc-outputs-read-through-their-section-headers)). +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md) (the older KTGL). +- **Note:** nine more shaders exist only at run time ([Built-in shaders compiled with the game's own GLSLC](#built-in-shaders-compiled-with-the-games-own-glslc)). + +### GLSLC outputs in FromSoftware's BND3 and dvdbnd archives + +- **How it is stored:** RomFS is two archives, `dvdbnd0.bdt` and `dvdbnd1.bdt` (3.6 GB), each indexed by the `.bhd5` beside it: `BHD5`, 0xff (little-endian), u32 1, u32 file size, u32 bucket count, u32 0, u64 bucket table offset; each 16-byte bucket `{u32 count, u32, u64 offset}` points at 16-byte entries `{u32 path hash, u32 size, u64 offset}` (paths only hashed). 5,184 entries are DCX containers (`DCX\0`, big-endian header, `DFLT` = a zlib stream at +0x4c), most of them BND3 archives (28-byte entries `{u8 flags, i32 size, u64 offset, i32 ID, u32 name offset, i32 unpacked size}` in formats 0x5c / 0x7c). Six BND3 archives in `dvdbnd0.bdt` hold the shaders, one GLSLC output per entry (`.vpo` vertex, `.fpo` pixel, named `Shader/FRPG_Sfx/NVNShaderBinary/ps/…` or by base name in the two FLVER material archives, 2,602 + 486 entries). +- **Reader:** `bhd5_entries`, `undcx`, `bnd3_members`, `bdt_shaders` in `shader_split.py`, in 64 jobs by entry range, about 10 s. Only archives that contain a GLSLC magic are parsed (the others use format 0x78). +- **Note:** SDK 4's GLSLC needs its section headers read ([GLSLC outputs read through their section headers](#glslc-outputs-read-through-their-section-headers)). +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md). + +### MSHD in MercurySteam's bshdat files + +- **How it is stored:** `system/shd/*.bshdat`, the only place the game's shaders live (nothing in `main` or the `.pkg` packs). Each stage is a control section, whose length is at control +0x20, sometimes followed by a small compiler-option table (`fastmath true`), then the code area at the next code header. Every file holds vertex / pixel stages in alternating order (1, 2 or 4 programs) and is paired two by two. +- The control section carries NV assembly text after its first 0x880 bytes, and the SASS disassembly follows each code area, so the binding layout can be read straight from the files (`TEX` / `TEXS` handle immediates). +- **Reader:** `mshd_shaders`. +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md) (MercurySteam's engine): 478 files, 1,342 stages, 1,010 unique keys (359 vertex, 651 pixel), 659 pairs. + +### SDF definitions in RE Engine PAKs + +- **How it is stored:** shader definitions (`SDF\0`, 2,685 files in the PAK view, [01](../agents/01-extract.md)): u16 version 1, u16 entry count, u64, then 0x108-byte entries from +0x10. Entry +0x10 holds six u64 offsets (vertex, hull, domain, geometry, pixel, compute; 0 = none), each to `{u32 control length, u32 code length}` followed by the control section (0x880 bytes) and the code area; compute code has no header. An entry's stages are one program, so its vertex / pixel stages are a pair. Empty definitions are 16 bytes of version 0. +- **Reader:** `kpka_jobs` maps the PAKs, takes the newest copy of each path, peeks at 4 decompressed bytes and inflates only SDF entries (`sdf_shaders`). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md): 63,111 shaders (30,233 vertex, 21,475 pixel, 11,271 compute, 53 geometry, 33 hull, 46 domain) and 102,345 pairs, in 5 s with 12 jobs. + +### SPK shader package in an MT Framework ARC + +- **How it is stored:** the engine's one shader package, `sc\NX_64\root` (`SPK\0`, type hash `0x02358e1a`) in `nativeNXx64/sa/NX_64/root.arc` ([01](../agents/01-extract.md)). After the package's tables every shader is a code area followed directly by its control section, back to back to the end of the file: 230 vertex shaders, then 230 pixel shaders. +- **Reader:** each control section names its code length (+0x704), so the code is the bytes right before it, and every block must start where the previous one ended (`mtspk_shaders`). The tables that say which vertex and pixel shaders the engine combines are not read: the vertex shaders are paired with the single-stage cover (one pixel shader reads every input any of them reads). +- **Note:** `CustomShaderPackage.mfx` (`MFX\0`, the effect descriptions) holds no shader code, and neither `main` nor any other ARC entry does. +- **Found on:** [The Great Ace Attorney Chronicles](../../dm/games/010036E00FB20000/README.md) (Capcom's MT Framework): all 460 build, with Mario's binding layout; every binding decodes. + +### PlatinumGames' shader objects + +- PlatinumGames' shader objects: `VSO\0` / `PSO\0` / `CSO\0` / `HSO\0` / `DSO\0` (the stage), u32 compiler version, u32 control length, u32 control offset (0x18), u32 code length, u32 code offset, and nothing else. The engine makes a program of each and binds them per stage at draw time, so they are single-stage programs, paired with a cover pixel shader (`shader_split.py` `pg_shader`; the archives: [01](../agents/01-extract.md)). It finds its built-in ones by name (`hwfillcolor.vso`, …) and checks each object's GPU code version against the driver's (the NVN layer's newest GPU code version follows the SDK: [Newest GPU code version reported by the driver](nvn-objects.md#newest-gpu-code-version-reported-by-the-driver), SDK 13.3.2). +- **Found on:** [Bayonetta 3](../../dm/games/01004A4010FEA000/README.md): 8,024 objects in `root_000.pkz` (`shader.dat`, `shader2.dat`, `shadereff*.dat`, `autoshadereff.dat`); 6,424 distinct keys (4,288 pixel, 2,047 vertex, 78 compute, 6 domain, 5 hull). + +### Shaders in a zlib stream in main (YEBIS .tfxo) + +- **Symptom:** shaders the game draws are missing from the map (`DM_NVN_SHDUMP`), in no RomFS file and no module, stored or LZ4, and not compiled at run time. +- **Cause:** Silicon Studio's YEBIS post-effect library (bloom, glare, depth of field, vignette; `C:/workspace/YEBIS_draft/src/GPU/…` in `main`) keeps its shaders in a zlib stream in `main`: YEBIS's `.tfxo file`, a text header (the shader count, `hlsl`, `TexUtil_NVN_NX`, the technique list) and the shaders, each a control section (its own length 0x850 on SDK 7) directly followed by its code area. +- **Fix:** `shader_split.py` inflates it and reads the shaders as single-stage programs (`tfxo_streams`, `tfxo_shaders`). +- **Found on:** [Pokémon Sword](../../dm/games/0100ABF008968000/README.md): after the [GFPAK reader](#bnsh-entries-of-gfpak-archives), 38 shaders the opening drew were still missing; none was a patched copy of a split shader (the nearest ones of the same length differ in hundreds of bytes). `main` imports `glslcCompile` from `subsdk1` but never calls it and holds no GLSL source. The only zlib stream in `main` that inflates (`main+0x1c98ed9`, 39.5 MB) holds 3,155 shaders (3,015 distinct, 3,112 pixel and 43 vertex); they covered all 38. +- **Check:** a temporary trace in `nvnProgramSetShaders` (the key's code and control addresses, a backtrace and the heap around them) showed control sections packed back to back with technique names such as `ps_BrightPassT1_THRESHOLD_LUMINANCE_S1` beside them; the names are in `main`, and so is the data. + +### Loose control sections and code areas in main (Bink 2) + +- **Symptom:** two `[RED] shader … is not in the shader map` lines and `program has no vertex shader` as the opening movie starts; every movie drew white. +- **Cause:** the movie player (Bink 2's Unity plugin) builds its programs from 32 control sections and code areas in `main`'s read-only segment that are in no GLSLC output and no BNSH file. 26 are 0x20-byte records in two tables in `.data` (`{code area pointer, control section pointer, u32 code length, u32 control length}`, both pointers relative relocations); the code takes the other 6 by address (`adrp` + `add` to the control section and, a few instructions away, to the code area it copies with `memcpy`), the code area right after the control section's own length (+0x20, 16-byte aligned) or right before it, and two control sections share one code area. The 8 compute shaders' code areas have no header (SDK 10). +- **Fix:** the reader (`raw_shaders`) takes them. Both neighbours can pass the header and length checks, so it takes only a code area the code builds the address of, the nearest one to the control section's. +- **Found on:** [Advance Wars 1+2: Re-Boot Camp](../../dm/games/0100300012F2A000/README.md) (Unity 2020.2 with Bink 2's Unity plugin). Of 25 `main`s on hand no other has such control sections. + +### MGFX effects in MonoGame XNB files + +- **How it is stored:** the 13 XNB files in `Content/Effects` (XNB platform `S`, uncompressed, read by `EffectReader`) each hold one MGFX effect, and `main` holds SpriteBatch's built-in one; no other file and no GLSLC output. A compressed XNB (MonoGame's LZX flag 0x80 or LZ4 flag 0x40, as in Stardew Valley) is decompressed first (`xnb_decompress`). +- **Layout:** MGFX version 8 is MonoGame's own layout: `MGFX`, u8 version, u8 profile, u32 effect key, the constant buffers, then each shader as u8 vertex (1) / pixel (0), u32 length, the bytecode (u32 control length, the control section, u32 code length, the code area: GLSLC's output without its header), the samplers, the constant buffer indices and an attribute list (as MonoGame's OpenGL backend writes it), then the parameters and the techniques with their passes (`mgfx_effect`). +- **Reader:** the reader parses every effect to its end, which checks the layout; in `main`, four bytes `MGFX` count only when version 8 and a control section follow. +- **Pairing:** a pass names its vertex and pixel shader by index, 255 for none, and 10 of the 13 effects have no vertex shader: SpriteBatch binds its own and the effect's pass only replaces the pixel shader. Vertex and pixel shaders therefore combine across effects, and they are paired as single-stage programs. +- **Found on:** [Celeste](../../dm/games/01002B30028F6000/README.md) (MonoGame for the Switch, its C# compiled to C++ by Sickhead's BRUTE): 26 shaders (20 pixel, 6 vertex), all with Mario's binding layout; every binding decodes. [Stardew Valley](../../dm/games/0100E65002BB8000/README.md) (compressed XNB). + +### Unity 2020.3 Shader objects in UnityFS bundles + +- **How it is stored:** Unity 2020.3 keeps each shader's compiled programs in its `Shader` object (class 48), in UnityFS bundles (`StreamingAssets/aa`) and in the player's own serialized files (`data.unity3d`, `Resources/unity default resources`). +- **Reader:** `dm/gfx/tools/unityfs.py` reads them: UnityFS blocks (LZ4 / LZMA), the serialized file, the object through its type tree, then `compressedBlob` per platform (LZ4 segments; each segment starts with a u32 count and 12-byte `{offset, length, segment}` entries). +- **Type trees:** the player's files have no type trees; they borrow the bundles' `Shader` tree with the same type hash (same Unity build, same layout). Type-tree strings with the high bit set index Unity's common string table, which `unityfs.common_strings` reads from the game's own `main` (`AABB\0AnimationClip\0…`). +- **Program code:** on Switch every subprogram is one whole NVN program. Its code is a 0x60-byte header of four `i32 ×6` arrays indexed by NVN stage (0 vertex, 1 fragment, 2 geometry, 3 tess control, 4 tess evaluation, 5 compute): an unused / scratch size, the block offset from +0x60 (−1 = no stage), the control size and the code size. Each block is the control section followed by the code area. The vertex and fragment stages of one program form a pair. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): 209 `Shader` objects in 188 bundles plus the engine files, 9,269 stage shaders (4,634 vertex / pixel pairs and one geometry shader), no compute shaders. + +### Unity 2022 Shader objects with parameter blobs + +- **How it is stored:** the program code keeps the 0x60-byte stage header ([Unity 2020.3 Shader objects in UnityFS bundles](#unity-20203-shader-objects-in-unityfs-bundles)), but two things changed. From Unity 2022 the blob also holds each program's parameter tables as entries of their own (the parsed form's `m_ParameterBlobIndices`; their bytes start `PGlobals`), and the player's programs are listed in `m_PlayerSubPrograms[tier][].m_BlobIndex` (`m_SubPrograms` is empty). +- **Reader:** `unity_code_blobs` keeps only those player programs. UnityPy's type trees for 2020.3 (Engage) and 2021.3 have neither field, so those read every entry as before. Subprogram version 202012090 (from Unity 2021.2) also has no local keyword list, so `unityfs.subprogram` reads no second keyword array (with one it read the code length as a keyword count). +- **Type trees:** the player's serialized files have no type trees and the Addressables bundles hold no `Shader`, except the `unitybuiltinshaders` bundle, whose tree has the same hash and is borrowed in the second pass. +- **Found on:** [Ace Attorney Investigations Collection](../../dm/games/010005501E68C000/README.md) (Unity 2022.3.17f1): 4,951 stage shaders: 2,475 vertex / pixel pairs (4,294 in the player's files, 517 built-ins, 140 in bundles) and one geometry shader, which Metal cannot express (its build fails). +- **Note:** armyja's issue #17 proposed UnityPy and a reader of its own; neither was needed. + +### Unity Shader layouts written out for players without bundles + +- **Symptom:** the split stopped with `no bundle has a Shader type tree of the same hash to borrow`. +- **Cause:** a Unity player with no bundle to borrow a tree from: its serialized files have no type trees, and no bundle carries a `Shader` tree of the same hash. +- **Fix:** `unityfs.py` carries the layouts written out (`KNOWN_TREES`; `tree_from_layout` builds the tree a file with type trees would have), and the split's second pass gives one to the stripped files when no bundle lent a tree of that hash. Only there: taken in the first pass, Ace Attorney Investigations Collection's player files came before its `unitybuiltinshaders` bundle and the built-ins' source column and package indices moved, while with the second pass its tables come out byte-identical. The structs both the 2020.3 and 2022.3 layouts share are written once (`_SHADER_COMMON`), and the 2022.3 layout comes out byte-identical. +- **Found on:** + - [Brain Memory Training Academy](../../dm/games/0100F5D01DA3A000/README.md) (Unity 2022.3.5f1: only `Data/*.assets`, `level*` and the two engine files in `Data/Resources`, none with type trees). The `Shader` type hash there (`ea424a5f…`) is the one in Ace Attorney Investigations Collection's bundles (2022.3.17f1): the same layout, about 30 structs. Checked against Ace Attorney's own tree: the same 2,324 nodes with the same alignment flags, and its 40 bundle `Shader` objects decode identically with either. 496 stage shaders: 439 engine built-ins, 46 of the game's, 11 of the player's in `main`, and the built-in VR blit's geometry shader, which does not build. + - [Patrick's Parabox](../../dm/games/0100E44016C22000/README.md) (Unity 2020.3.30f1, built-in shaders only, no bundles) stopped there with hash `a30e3dcf…`, which one 18 KB shader bundle of Fire Emblem Engage (2020.3.18f1) carries; its tree is now in the table too (`SHADER_2020_3`: per subprogram global and local keyword indices, every subprogram in `m_SubPrograms`, no parameter blobs, keyword names or stage counts), equal to Engage's node for node (2,179 nodes, the alignment flags included), and each of the game's 48 `Shader` objects reads to its exact end. + - [Raging Loop](../../dm/games/010020400E1C2000/README.md) (Unity 2018.3.10f1, no bundles: 48 `Shader` objects in `Data/*.assets` and `Data/Resources`) stopped there with hash `8db76007…`, and no game of that Unity version with a bundle was at hand, so `SHADER_2018_3` is written from AssetRipper's TypeTreeDumps (`StructsDump/release/2018.3.10f1.dump`), checked node for node against the dump (1,325 nodes, the alignment flags included). It is a layout of its own, older than the common structs: no conservative raster state, `BufferBinding` without `m_ArraySize`, `ConstantBuffer` without `m_IsPartialCB`, the parameters in each subprogram and no `m_CommonParameters`, one segment per platform (the pre-2019.3 form Cuphead's trees gave `shader_programs`). Each of the 48 objects reads to its exact end: 450 subprograms of version 201802150 (older than the local keyword list, which starts at 201806140). +- **Note:** a Unity version whose `Shader` hash is not in the table still stops with the same message; read its tree from any bundle of that Unity version and add it. To take one file out of an NSP without extracting the RomFS, filter `nx_extract`'s `Copy` by path. + +### Unity 6 Shader objects + +- Read with no change: Hollow Knight: Silksong's 2,068 Addressables bundles carry their type trees (they report the engine version `0.0.0`, Unity's stripped version string), so no layout had to be written out. +- BALL x PIT keeps most of its shaders in the player's loose serialized files (`resources.assets`, `sharedassets1.assets`, the built-in resources), which have no type trees; they borrow the `Shader` tree its 10 shader bundles carry (`shader_split.py` defers a file that raises `NoTree` to a second pass), so it needed no layout either. +- **Found on:** [Hollow Knight: Silksong](../../dm/games/010013C00E930000/README.md) (Unity 6000.0.50f1): 2,450 stage shaders: 1,189 vertex, 1,193 pixel, 67 compute and one geometry shader, which does not build; 11 of them are the player's own in `main`. [BALL x PIT](../../dm/games/010086A022444000/README.md) (6000.0.62f1): 38,099 shaders, most of them in the loose files. + +### Unity ComputeShader objects + +- **Symptom:** the 3D battle scene was black behind a correct UI, with `[RED] shader 854a8afc9d07f88d is not in the shader map` and `the dispatched program has no compute shader`. +- **Cause:** `DM_NVN_SHDUMP` saved the program; its code bytes were in `Data/sharedassets2.assets`, uncompressed, inside a `ComputeShader` (class 72): the post-processing stack's colour grading LUT baker (`Lut3DBaker#KGenLut3D_AcesTonemap`), so the LUT the scene is graded through stayed black. The split read only `Shader` objects. A `ComputeShader` holds per platform variant a list of kernels, each with its unique variants; a variant's `code` is the program code of a `Shader` subprogram (the 0x60-byte stage header, only the compute block present) and its `threadGroupSize` matches control +0x748. +- **Fix:** `unity_compute` in `shader_split.py` reads them beside the `Shader` objects of every serialized file. The game has no bundle with a class 72 tree, so `unityfs.py` carries Unity 2022.3.54f1's layout (`COMPUTE_2022_3`, written from AssetRipper's TypeTreeDumps for that version; all 12 objects decode, `resourcesResolved` true at each end). 85 compute shaders came in (86 with the player's own). +- **Found on:** [Monster Train 2](../../dm/games/010051701FB46000/README.md) (Unity 2022.3.54f1). +- **Note:** a file whose `ComputeShader` has no tree still goes to the second pass; if it has none there either, the split prints a `[RED]` line and leaves its kernels out instead of stopping, so profiles whose compute shaders were never read keep their tables: [Cuphead](../../dm/games/0100A5C00D162000/README.md) (2) and [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) (4) have `ComputeShader` objects only in files without type trees, of hashes no layout covers. So does [Raging Loop](../../dm/games/010020400E1C2000/README.md) (Unity 2018.3: the two built-ins in `unity default resources`, whose 2018 layout lists kernels with no unique variants), which dispatches no compute. + +### Unreal Engine 5 shader code archives (Oodle) + +- **How it is stored:** `ShaderArchive-Global-SF_GLSL_SWITCH_MOBILE-SP_GLSL_SWITCH_MOBILE.ushaderbytecode` (1,219 shaders) and `ShaderArchive-Octopath_Traveler0_Chunk0-…` (9,377), both stored in the pak ([01](../agents/01-extract.md)). Version 2 archives of shader map hashes, shader hashes, shader map entries, shader entries `{u64 offset, u32 size, u32 raw size, u8 frequency}` (17 bytes packed; frequency 0 vertex, 3 pixel, 5 compute), preload entries and shader indices, then the code. +- **Compression:** every shader is Oodle-compressed (Mermaid, block header `8C 0A`). There is no Oodle decoder in this repository, so the game's own `OodleLZ_Decompress` (statically linked into `main`; the host finds it as the one function that takes the address of the library's own name string "OodleLZ_Decompress", its profiler scope: Octopath Traveler 0 `main+0x4f77e10`, Pokemon Legends: Z-A `main+0x1a9c9e0`) decodes them in the Mac host: `DM_OODLE_JOB` (`dm/host/hle_oodle.c`), the NSP script's `decode_ue_shaders` for any game with Unreal paks. +- **Record:** a decoded shader is an `LSLG` record: `LSLG`, a stage tag (`SV`, `SP`, `SC`), a header with the source name (`/Engine/Private/…usf:MainPS`), then `u32` code length, the code area, `u32` control length (0x880), the control section and a 24- or 29-byte tail, all at unaligned offsets (`uepak.lslg`). +- **Check:** the host keys came out exactly: 37 of 37 shaders a first run reported missing were in the two archives. +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): 10,324 unique shaders (1,457 vertex, 6,346 pixel, 2,521 compute); vertex and pixel shaders combine per draw, so every vertex shader is paired with the single-stage cover. Bindings: separate textures and samplers, as Engage's (`0x168` / `0x568`); 61 vertex shaders pick a texture slot at run time (profile key `tex_dyn_slots`, [Texture slots picked at run time](#texture-slots-picked-at-run-time)), 2,084 shaders do atomics on buffer images ([Image atomics](#image-atomics)). +- **Note:** 16 compute shaders do atomics on a 3D `r32ui` image, which SPIRV-Cross's MSL 2.3 image-atomic emulation indexes with an `int3` (`array subscript is not an integer`): all are Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run), so they stay unbuilt. + +### Unreal Engine 4.27 shader code archives (LZ4) + +- **How it is stored:** the desktop renderer's `ShaderArchive-Global-SF_GLSL_SWITCH.ushaderbytecode` (1,764 shaders) and `ShaderArchive-DOLL_Chunk0-SF_GLSL_SWITCH.ushaderbytecode` (63,185), stored in the pak: the same version 2 archives and the same `LSLG` records as [Unreal Engine 5's](#unreal-engine-5-shader-code-archives-oodle), but every shader is an LZ4 block (the first bytes are LZ4 tokens followed by `LSLG`; LZ4 is Unreal's default shader compression since 4.26). +- **Reader:** `uepak.py` decodes them itself (`lz4_shaders`): every compressed shader must decode to its raw size as an `LSLG` record, else the archive goes to the Oodle job as before. +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (Unreal Engine 4.27): 62,901 unique shaders (10,114 vertex, 50,756 pixel, 1,968 compute, 63 geometry). +- **Note:** its fragment shaders carry no output types either: one global shader writes integers bit-cast into a float output to an `R8Uint` target, listed in the profile's `shader_facts_manual.tsv` ([Integer fragment outputs declared by hand](#integer-fragment-outputs-declared-by-hand)). + +### Unreal Engine 5.6 IoStore shader libraries + +- **How it is stored:** the desktop renderer (`SF_GLSL_SWITCH`) with the same `LSLG` records, but no archive in the pak: the two libraries are IoStore shader libraries in `FFRS-Switch.ucas`, whose shader groups are Oodle-compressed as a whole ([01](../agents/01-extract.md)). Each shader entry packs its frequency (bits 0–3), group (4–33) and offset in the group's decoded bytes (from bit 34), and a shader runs to the next one's offset. +- **Reader:** `uepak.py` writes them out as version 2 archives, so the split reads them as any other. +- **Found on:** [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) (Unreal Engine 5.6.1): 73,496 unique shaders (9,253 vertex, 61,305 pixel, 2,934 compute, 4 geometry). + +### Unreal Engine 4.18 shaders inline in cooked packages + +- **How it is stored:** legacy version 4 paks with LZ5-compressed blocks ([01](../agents/01-extract.md)) and no shader code archive. Each material's shader map is serialized in its `.uexp` (38 to 87 shaders in the ones looked at) and the engine's global shaders in `Engine/GlobalShaderCache-GLSL_SWITCH.bin` (945: 597 pixel, 104 vertex, 241 compute, 3 geometry). Each shader's code is a byte array (`i32` length, then the bytes) holding one zlib stream that inflates to the same `LSLG` record as Unreal Engine 5's. +- **Reader:** `shader_split.py` (`ue_legacy_jobs`, `ue_inline_shaders`) decodes every file of the newest copy of each path with the game's own LZ5 and takes each zlib stream that inflates whole from exactly the length in front of it and starts with `LSLG` (every such stream in the files looked at). Records stored uncompressed are taken too, but none were seen. +- **Found on:** [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md): 38,892 unique shaders (4,654 vertex, 33,981 pixel, 234 compute, 23 geometry) from 1,740 files; vertex and pixel shaders are paired as single-stage programs, as for Octopath Traveler 0. + +## Control sections and code + +### Code area and control section layout + +- **Code area:** graphics = 0x30-byte header (magic `0x12345678`) + 0x50 SPH + instructions (at +0x80); compute = 0x30-byte header (magic `0x12345679`) + **zero padding, instructions at +0x100** (all 71 of Mario's). +- **Control section:** magic `0x98761234`; +0x700 = c1 offset in the code area, +0x704 = code size, +0x748 = compute thread-group size (u32 ×3), +0x754 = compute shared memory bytes. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Control sections shorter than 0x880 bytes + +- **Cause:** the control section stores its own length at +0x20. Mario's, Engage's, Dread's and 13 Sentinels' are all at least 0x880 bytes; 3D World has 2,888 of 0x850 bytes (its ShaderData bfsha and six layouts) and 0x848 in BAHS version 9, each followed directly by other data. +- **Fix:** the key hashes the first `min(0x880, control +0x20)` bytes (`ctl_len` in `shader_split.py`, `keys_hash` in `hle_nvn.c`); the other games' tables came out byte-identical. +- **Found on:** [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (SDK 10). + +### Program keys computed inside `nvnProgramSetShaders` + +- **Symptom:** shaders reported missing at random, and their dumped control sections are garbage (Unity). +- **Cause:** NVN lets the caller free a program's control section once `nvnProgramSetShaders` returns; Unity does, and the host used to hash it at the first draw, so keys computed later hashed freed memory. +- **Fix:** the keys are now computed inside `SetShaders` (`hle_nvn.c` `prog_key`). Mario calls `SetShaders` for some programs with a code address that is not mapped yet, so a stage whose code is not in a registered memory pool at that moment is hashed at its first draw, as before. +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) and Unity games in general (the source names no other); [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (the unmapped code addresses). + +### Compute code with and without a header + +- **Cause:** compute code areas have a header in Mario (`0x12345679`), not in D2R. 3D World's particle compute shaders (SDK 10) start their instructions at code +0 (like D2R), not after a `0x12345679` header. +- **Fix:** `shader_split.py` accepts both; `dmsh_build.py` strips only a header that is there. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) (header); [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (no header, SDK 10). + +### Exact program start for the decoder + +- **Symptom:** 12 compute shaders (particles, post effects) translated into an endless loop that hung the GPU at the first gameplay scene; SPIRV-Cross failed on 7 shaders instead of 2 ([SPIRV-Cross parse failures](#spirv-cross-parse-failures)). +- **Cause:** Ryujinx's decoder treats every 32-byte boundary *relative to the start* as a scheduling word. Mario's compute code was first fed from +0x30: the groups were misaligned by 0x10, and instructions were dropped or misread. +- **Fix:** the program start must be exact (compute instructions at +0x100). `dmsh_build.py` now refuses compute code whose padding is not zero. +- **Check:** for a new title, the first non-zero word after the header must be a control word, and the trailing `EXIT` (`e30000000007000f`) / `BRA`-to-self (`e2400fffff87000f`) must sit in instruction slots of 32-byte groups counted from your chosen start. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### GLSLC outputs read through their section headers + +- **Symptom:** GLSLC outputs fail the reader's check and are dropped (44 of DARK SOULS: REMASTERED's `main`'s 72 outputs and most of its archives' pixel shaders). +- **Cause:** SDK 4's GLSLC outputs (version 0x20.4): pixel shaders' control sections store at +0x20 a value 12 bytes short of their length, so the code area is not right after it. +- **Fix:** an output's own section headers say where everything is: the output's +0x4c is the section count, section i's header is at +0x90 + 0x90 i (`{size, data offset from the output, type}`, type 0 = GPU code), and a GPU code header adds +0x30 control offset, +0x34 code offset (both from the section's data), +0x38 code length, +0x3c control length. `glslc_section` reads them for outputs the old rule rejects. +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md) (SDK 4.5.0). +- **Check:** on every output the old rule keeps (1,073 in 13 Sentinels, Three Houses, FINAL FANTASY TACTICS, Engage and Age of Calamity) both give the same key; the new reading also takes 8 outputs of [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md) and 3 of [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md) (GLSLC version 0x1d.0, whose control +0x20 holds odd values) that used to be dropped, so those two profiles' tables gained them. + +### VertexA and VertexB parts merged into one vertex shader + +- **Symptom:** `shader_split.py` finds no stage for vertex shaders and stops at the first captured output (`no GLSLC output`). Translated, every post pass and the window draw one texel's colour (a black screen with sound); every varying at (0, 0, 0, 1). +- **Cause:** shader type 0 in every vertex shader's SPH (word 0 = `0x0061`; every other game's vertex shaders have `0x0461`). Shader type 0 is Maxwell's first vertex slot (VERTEX_CULL_BEFORE_FETCH), and this GLSLC (GPU code version 1.15) writes every vertex shader as two programs in one code area: part A at code +0 (the type-0 SPH) loads the position attribute and writes only the position; part B, at control +0x700 with a 0x30-byte header of zeros and an SPH of type 1, reads the other attributes and writes the other outputs. Control +0x748 lists both as {SPH offset, length from the SPH, ?}. The hardware runs A, then B. Translated alone, A left every varying at (0, 0, 0, 1), so the full-screen passes sampled their sources at (0, 0) and the 2D draws their atlases' first texel. +- **Fix:** Ryujinx's `ShaderHeader` reads a graphics header's type 0 as a vertex shader; `sph_stage` does too. `dmsh_build.py` (`vertex_ab`) merges the two parts as Ryujinx's `TranslatorContext.Translate(VertexA)` does: A's instructions with each unconditional `EXIT` turned into a `BRA` to B's first instruction, then B's, under B's SPH with A's flags and input / output maps ORed in and the larger local memory sizes. c1 is what follows part B. A type-0 shader with no part B listed is translated as it is. +- **Found on:** [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md): in all 1,244 vertex shaders A writes no generic output, A ends in an `EXIT` predicated on CC FCSM_TR (which Ryujinx never takes; left as it is), an unconditional `EXIT` and a `BRA` to itself, and neither part has an absolute or table jump, so moving B keeps every branch; c1 (what follows part B) is zero padding in all of them. A census of the other games' packages and of every `main` on hand found no other type-0 vertex shader. + +## Translation + +### SPIRV-Cross parse failures + +- **Symptom:** SPIRV-Cross can't parse the SPIR-V (`nullptr` exception). +- **Cause:** 2 agl post-effect compute shaders still produce such SPIR-V. Before the compute start-offset fix ([Exact program start for the decoder](#exact-program-start-for-the-decoder)) it was 7: misdecoded code produced the bad SPIR-V. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md). + +### Pixel shader inputs the vertex shader does not write + +- **Symptom:** `Fragment input(s) user(locnN) mismatching vertex shader output type(s) or not written by vertex shader` and the draw is skipped; the host reports `[RED] failed to create render pipeline (vertex … fragment …)`. +- **Cause:** every pixel shader input must be a vertex shader output. Maxwell attributes are untyped slots: a pixel shader that reads a slot the vertex shader never wrote gets a default, but Metal refuses to build the pipeline. Translating one shader at a time cannot know what the partner reads. +- **Fix:** `shader_split.py` writes the pairs of each BNSH variation (`shader_pairs.tsv`) and `dmsh_build.py` gives each vertex shader the outputs its partners read but it doesn't write, as `float4` (0, 0, 0, 1) (`vs_pad`; vertex shaders with partners are finished after the pixel shaders, and the partner list is part of the shader's cache rule, [Shader cache key covering the pairs](#shader-cache-key-covering-the-pairs)). The host caches failed pipelines and reports each once with its vertex / fragment keys; before that, every draw retried the failing compile. +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md): 331 of 9,043 pairs failed; the opening course used none of them, later levels did (about 11,500 failed builds in one 11-minute session). +- **Check:** check all pairs offline after a build (recipe in [05](../agents/05-debugging.md)). + +### Single-stage programs paired with a cover + +- **Cause:** a game that builds one NVN program per GLSLC output and combines vertex and pixel programs per draw with `BindProgram` stage bits has its real pairs only at draw time, so the offline build cannot know which pixel shaders a vertex shader must feed. +- **Fix:** the composite-program path ([Programs bound per stage](rendering.md#programs-bound-per-stage)) does all the pairing at replay. When every pixel shader input is a plain `float4` at fixed locations, `shader_split.py` pairs every vertex shader with a *cover*: the fewest pixel shaders whose inputs together are every input any pixel shader reads. `vs_pad` then gives each vertex shader all of them, and any vertex / pixel combination links. +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md): 838 `ProgramSetShaders` at startup; every pixel shader input is a plain `float4` at locations 1–9 (the SPH input map, words 6–13, matches the translated MSL inputs for all 375 pixel shaders), so the cover is one shader. + +### Unwritten components of a written output start as (0, 0, 0, 1) + +- **Symptom:** outdoors dim and nothing casts a sun shadow (every shadow-map lookup fails). +- **Cause:** a vertex shader may write only part of an output (Link's Awakening writes a world position's x, y and z) while the pixel shader reads the rest (it divides by w). On the hardware the unwritten components read as (0, 0, 0, 1); Ryujinx gets the same by initializing the components the next stage reads when it links stages. The translator's MSL starts every vertex shader with `main0_out out = {};`, all zeros, so w was 0, and a fragment shader dividing by it got infinite compare values. +- **Fix:** general: `dmsh_build.py` (`vs_init`) sets every `float4` output to (0, 0, 0, 1) right after it, and the translated body overwrites what it writes. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (Mac and iPhone): 12,281 of 18,410 vertex outputs never write w. The sun's shadow map (2048×2048, depth 32-bit float) was correct; the surface shaders compute the lookup from a world position the vertex shader passes in `locn24`, divided by its w. Found with `DM_NVN_TEXRAW` on the shadow map, the light matrices from `DM_NVN_CBDUMP`, and a hand-edited copy of one surface shader that wrote the compare value and the stored depth as its colour. Compared with forcing the compare to pass (`compareFunction` always): the same brightness, plus shadows from the fences, the house and the trees. + +### Position invariance across shaders + +- **Symptom:** the ground kept stale 32×32 blocks where the G-buffer draw failed the depth test: blocks of a stale image where a depth pre-pass and a later pass with other vertex shaders meet (Odyssey: 32×32 blocks on every fourth frame). +- **Cause:** a game may draw the same geometry with two programs and rely on identical depth, for example a depth pre-pass and then a less-or-equal G-buffer pass. Maxwell evaluates identical position math identically; Metal only does for outputs marked `[[invariant]]` (the translation marks `gl_Position`) when compiled with `-fpreserve-invariance`. Without it, Metal compiled two vertex shaders' identical position math to slightly different depths, so fragments of the second pass failed a less-or-equal depth test against the pre-pass and those pixels kept the previous content. +- **Fix:** `dmsh_build.py` (Mac) and `ios_metal.py` (iOS) pass `-fpreserve-invariance` to every compile. +- **Found on:** [Super Mario Odyssey](../../dm/games/0100000000010000/README.md) (Mac): the Cap Kingdom ground. Presents 4k were brighter and blocky, 4k+1 and 4k+2 clean, 4k+3 partly blocked, repeating to the decimal across runs. The blocks were already in the HDR target when the G-buffer pass ended, and only with the ground's material (`b6384a85afd3ef94`) drawn. In some frames the game first draws a depth pre-pass (`alRenderDepthOnly`) for objects its occlusion queries report visible (the queries rotate through three slots, hence the cycle), then draws the G-buffer with other vertex shaders and depth test less-or-equal, without clearing the colour targets. +- **Note:** ruled out on the way: the TBDR optimizations, same-pass feedback, the present path. Evaluating the game's conditional rendering on the CPU from earlier frames' results made it worse (the condition belongs to the same frame on the GPU), so the host still ignores the condition there and draws. + +### iOS math mode (relaxed) + +- **Cause:** the translated shaders reproduce Maxwell's NaN behaviour with `isnan` selects (8,337 of 12,809 use them), and the default fast mode allows the GPU compiler to assume there is no NaN. +- **Fix:** the iOS libraries are compiled with `-fmetal-math-mode=relaxed` (`dm/ios/tools/ios_metal.py`; the Vulkan and GL libraries in `vk_shaders.py` and `gl_shaders.py` too). Relaxed keeps fast math's reassociation but not that assumption. +- **Found on:** listed among the mechanisms first met in [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md); the source gives the 8,337 of 12,809 count without naming the game. + +### min, max and clamp with a NaN operand + +- **Symptom:** the half of Mario facing the camera's left was white (GitHub issue 72, on an iPhone). +- **Cause:** spirv-cross writes `FMin` / `FMax` / `FClamp` as `fast::min` / `max` / `clamp`, which leave a NaN operand's result to the GPU. Maxwell's `FMNMX` returns the other operand and its saturation gives 0, as Metal's `precise::` functions (IEEE `fmin` / `fmax`) do. Plain `max` / `min` / `clamp` compile to the `fast_` intrinsics in every Metal math mode, relaxed included ([iOS math mode (relaxed)](#ios-math-mode-relaxed)). Galaxy's GX lighting normalizes light directions the game leaves at (0, 0, 0) (the key and fill lights of its characters) and takes `max(0, dot(ldir, NaN))` as the spot term: 0 on the M4 Pro with either function, so the attenuation is 1. The fill light sits at the camera's left in view space, so its terminator runs down the middle of a character facing the camera. +- **Fix:** `dmsh_build.py` (`precise`), `vk_shaders.py` and `gl_shaders.py` write the `precise::` functions instead (every game). +- **Found on:** [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md) (GitHub issue 72). The NVN chain had about 124,000 `fast::max`, 98,000 `fast::clamp` and 41,000 `fast::min` in [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md), none guarded by its `isnan` selects. + +### Integer fragment outputs declared by hand + +- **Symptom:** a pipeline red line for a float output meeting an integer attachment (the red line prints the row to add); a BC encoder's pipeline fails. +- **Cause:** with no PC version the translator has no output types and declares every output float, and Metal refuses a pipeline whose float output meets an integer attachment. A scan for outputs built from integer operations was not trusted: the translated code keeps float bits in integer variables (register reuse), so such a scan cannot tell an integer output from raw float bits stored into a float target. +- **Fix:** the profile's `shader_facts_manual.tsv` declares such outputs `uint`. When another one fails, the pipeline red line prints the row to add. Shaders that write an output from integer bits now also get integer entry points ([Integer entry points for integer colour outputs](#integer-entry-points-for-integer-colour-outputs)). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (agl's BC4 encoder `8e2cc2671cae6020` and its twin for a 2D source `93aa5e22a8202409`, whose failure showed up in an iPhone run); [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) (the BC4 encoder `0544282c162305db` and the Ultrahand mask `e3fb0bb58a691e10`, R8Uint, found by PR #11; the Mac route does not reach it); [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (two system pixel shaders write an ID into an `R8_UI` target on the character creation screen); [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (one global shader writing an `R8Uint` target, [Unreal Engine 4.27 shader code archives](#unreal-engine-427-shader-code-archives-lz4)). +- **Note:** the compressed render target side is in [Compressed textures as render targets through an uncompressed shadow](rendering.md#compressed-textures-as-render-targets-through-an-uncompressed-shadow). + +### Integer entry points for integer colour outputs + +- **Symptom:** `failed to create render pipeline … output of type float4 is not compatible with a MTLPixelFormatR32Uint color attachment` (or `MTLPixelFormatRGBA16Uint`); every draw into the target skipped. +- **Cause:** with no PC version the translator types every fragment output float, and the shaders write bit-packed integers through `as_type<float>`. A hand-written row per shader (`shader_facts_manual.tsv`, Breath of the Wild's way, [Integer fragment outputs declared by hand](#integer-fragment-outputs-declared-by-hand)) does not scale. +- **Fix:** `dmsh_build.py` `fs_uint` gives a pixel shader a second entry point, `main0_u`, with every colour output the unsigned integer vector of the same bits, when it writes an output from integer bits or nothing but constants (a clear drawn into an R32Uint target); the hardware writes the output's bits to an integer target whatever the shader computed, so the copy is exact. The host takes it for a pipeline whose written colour attachments are all unsigned integer formats (`pso_all_uint`). A shader whose outputs are of both kinds also gets a third entry point, `main0_m`, that returns only the outputs written from integer bits as unsigned integers, and records their colour indices (`uint_mask_m` in its reflection); the host takes it for a pipeline whose integer attachments are exactly those (`pso_mixed_uint`). Otherwise a pipeline with a float target as well keeps `main0` and the red line. +- **Found on:** [Luigi's Mansion 3](../../dm/games/0100DCA0064A6000/README.md): `main0_u`; about 1,400 pixel shaders writing its G-buffer of two integer targets (R32Uint, RGBA8Uint), so every scene draw was skipped. [Pokémon Sword](../../dm/games/0100ABF008968000/README.md): `main0_m`; 10 pipelines as the opening's stadium loads; the scene shaders write colour 0 and 2 as floats and colour 1 from integer bits into an RGBA16Uint target (colour 1 in all 336 of Sword's). + +### Integer textures and images through float declarations keep their bits + +- Integer textures and images read and written through float declarations (an `RGBA8_UINT` texture read by `texture3d<float>` and bit-cast, `R16_UINT` / `R32_UINT` buffers through `texture_buffer<float>`, integer stores into a float-declared `texture3d` image) were checked on an Apple GPU: all keep the integer bits, so they need no facts. +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md) (the volumetric lightmap's indirection texture, the light grid buffers). + +### Integer-typed vertex inputs fed normalized formats + +- **Symptom:** `failed to create render pipeline … Cannot convert attribute …`; draws missing (a pipeline that fails to build). +- **Cause:** the translation declares a vertex input `uint4` when the shader only moves its bits (x bit-cast to an integer, y copied as a float; [Integer vertex inputs](rendering.md#integer-vertex-inputs)), and a game that feeds it a 16-bit normalized format gets a pipeline Metal will not build (no integer input pairs with a normalized format). +- **Fix:** the profile's `shader_facts_manual.tsv` declares such inputs float (the bits Maxwell would see), which changes only that profile's chain version. The guess is not wrong in general, so overrides are added only for shaders seen failing with float formats. The general replay-side answers are the CPU conversion and the shadow vertex inputs ([CPU conversion of vertex attributes the input type cannot take](rendering.md#cpu-conversion-of-vertex-attributes-the-input-type-cannot-take), [Shadow vertex inputs](rendering.md#shadow-vertex-inputs)). +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md): missing palm tree and flowers (the pipeline `147c829e79fb322f` + `0ebba5ff6a44aa52`, `shadowmap_pbr.bfsha` variant 641, vertex inputs 1–5): the foliage's shadow-map and depth pre-pass draws; the main pass shades with depth test EQUAL, so without the pre-pass the plants never appeared. Four outdoor shaders with the same loop (`03c43da3e3f46da8`, `079700f3577e311f`, `15c02683a10f8e9d`, `2344874045b1e758`) failed the same way on the iPhone in Mabe Village (Half2, RG8 / RG16 normalized) and are listed too. 1,437 of the game's vertex shaders have integer-typed inputs, many of them bone indices the shader multiplies as integers. + +### Depth mode near = 0 needs the per-mode entry points + +- **Symptom:** outdoors the whole screen blurred (depth of field out of focus everywhere). +- **Cause:** a game that sets depth mode near = 0 had every vertex shader translated for GL, so all depths sat in 0.5..1. +- **Fix:** the per-mode entry points of [04, section 5](../agents/04-graphics.md#5-depth-convention). +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (Mac and iPhone): the depth-of-field composite (`52fcbb4ca2a78907`, `postprocess.bfsha`) blends the sharp frame with two blurred levels by a blur amount computed from the half-size depth copy: `pow(clamp((c5[2].y / (depth + c5[2].x) − c5[3].x) · c5[3].y), c5[3].z)` above a screen line, by screen position below it. Found with a save file from the iPhone (Mabe Village outside the house), `DM_NVN_PASSDUMP` / `DM_NVN_TEXDUMP` of that frame, and LASRecomp, which rewrites z only for the GL mode. + +### Open: the window Y-direction register `SR_Y_DIRECTION` + +- The register is −1 under a lower-left origin. The Ryujinx recompiler answers a constant +1 (a TODO in `InstEmitMove.cs`; LASRecomp, a static recompilation of Link's Awakening, answers −1). Answering it per profile means a translator change, i.e. every game's chain. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md): 2,487 of its pixel shaders read it; in the room shader the value only scales a depth slope under `abs()`; other shaders may use its sign (`dFdy`-based effects). + +### Subgroup operations rewritten for a one-lane subgroup + +- **Symptom:** `SPIRV-Cross threw an exception: Subgroup builtins are not available in this type of function`. +- **Cause:** Metal has no SIMD-group functions in vertex functions, and SPIRV-Cross's own subgroup emulation covers only `Elect`. RE Engine scalarizes uniform loads: ballot the active lanes, shuffle the first lane's value to all and take a fast path when all agree. +- **Fix:** with a one-lane subgroup that is exact, so `dmsh_build.py` rewrites the vertex shaders' SPIR-V that way before SPIRV-Cross (`spv_single_lane`: the ballot is the lane's own bit, shuffles and broadcasts return its own value, all / any are the predicate, all-equal and elect are true, lane ID 0, size 1). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (about 4 % of its vertex shaders). +- **Note:** the rewrite covers vertex shaders only: two of Bayonetta 3's domain shaders still fail this way ([Tessellation stages are not drawn](#tessellation-stages-are-not-drawn)). + +### Geometry shaders that only pick the layer + +- **Symptom:** the tone mapper read an empty LUT and the screen came out a flat blue. +- **Cause:** Unreal Engine 4.18 builds its 32×32×32 colour-grading LUT with `WriteToSliceMainVS` + `FWriteToSliceGS`, one slice per instance. Metal has no geometry stage. +- **Fix:** `dmsh_build.py` recognises such a shader in its SPIR-V (`gs_slice`: triangles in, at most 3 vertices out, positions and other inputs copied, `gl_Layer` taking the bits of one input component). It builds no library for it and records the component in its reflection. Each vertex shader that writes integer bits from the instance index into an output component gets two more entry points (`vs_layer_entry`: `main0_l` / `main0_lz` write that component as `[[render_target_array_index]]`). At bind time `prog_fold_gs` (`nvn_mtl.m`) swaps the vertex shader for its layer variant and drops the geometry stage when the components match. The pass and pipeline side of layer-writing vertex shaders: [Layered passes for vertex shaders that write the layer](rendering.md#layered-passes-for-vertex-shaders-that-write-the-layer), [Layer-writing vertex shaders get the triangle class](#layer-writing-vertex-shaders-get-the-triangle-class). +- **Note:** Unreal binds each stage from a program of its own, so a vertex shader is often paired for a moment with the previous draw's geometry shader. A stub that does not fit stays in the program, and only a draw that uses it prints a red line (`draw_gs_check`). The other 22 geometry shaders in DQ11 (one-pass cube shadows that emit transformed triangles) still do not build ([Geometry shaders with no Metal stage](#geometry-shaders-with-no-metal-stage)). +- **Found on:** [DRAGON QUEST XI S](../../dm/games/01006C300E9F0000/README.md). + +### Geometry shaders with no Metal stage + +- **Symptom:** a geometry shader fails to compile with `unknown main0_out`: SPIRV-Cross writes `unknown main0_out main0()` for the geometry stage, and the shader does not build. +- **Cause:** Metal has no geometry stage, so a geometry shader that does more than pick the layer has nothing to translate to. A geometry shader that only picks the layer is folded into the vertex shader instead ([Geometry shaders that only pick the layer](#geometry-shaders-that-only-pick-the-layer): `gs_slice`, `prog_fold_gs`; DRAGON QUEST XI S). +- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md) (one geometry shader, in `unity_builtin_extra`; nothing in the opening scenes uses it); [Super Mario 3D World + Bowser's Fury](../../dm/games/010028600EBDA000/README.md) (fur fins, `RenderFinFur`, its one failing shader); [Captain Toad: Treasure Tracker](../../dm/games/01009BF0072D4000/README.md) (the fin fur geometry shader, as in Super Mario 3D World, is the one shader of its 7,258 that does not build); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (53 geometry shaders); [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md) (one, `bahamuteffect.tec`); [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md) (Bezel's line renderer, below); [Luigi's Mansion 3](../../dm/games/0100DCA0064A6000/README.md) (10 geometry shaders, one expands a box from two corners; no draw used one from the title through the opening cutscene to the lobby). The Unity games' built-in geometry shaders do not build either (the container entries above). +- **Note:** a program whose geometry shader has no library used to draw silently without that stage, with the vertex shader's positions taken as clip coordinates; `draw with geometry shader … which has no library` now names it (Luigi's Mansion 3). +- **Note:** Big Brain Academy's 12 geometry shaders (`_BezelSystemResources/line/shader/line.bfsha`) do not build (`unknown main0_out`) and their draws are skipped (`[RED] primitive type 11 not implemented`). They take a line strip with adjacency (primitive type 11) and emit up to 12 vertices per segment as triangle strips: thick lines with joins and caps (28 emit sites). From the main menu on, every frame draws one such line (width 10 in its uniform block) into the top-left 512x512 of the frame and copies that region into a texture, so the menu's chart loses its outlines. Not done. +- **Note:** the vendored Ryujinx translator has its geometry-to-compute transforms (`VertexToCompute`, `GeometryToCompute`), which the chain does not use yet. + +### Tessellation stages are not drawn + +- **Symptom:** `primitive type 14 not implemented`; patch draws skipped. +- **Cause:** SPIRV-Cross already writes the hull shaders as kernels and the domain shaders as post-tessellation vertex functions, but Metal 4's render encoder and pipelines have no tessellation (no `drawPatches`, no factor buffer). The hull kernels also write quad factors (`MTLQuadTessellationFactorsHalf`) for triangle patches. +- **Fix:** not done. The draws would need a tessellator in a compute kernel and domain shaders rewritten as plain vertex functions. +- **Found on:** [Luigi's Mansion 3](../../dm/games/0100DCA0064A6000/README.md): 15 hull and 18 domain shaders. Two tessellated programs (vertex `28a39709d0adfcfa` / hull `51d2d156bb12c260` / domain `0b95742e863bec50`, and `d4d89931b5a9a514` / `8aa6feeb7d33e9f6` / `f254b4dce2d6c012`) draw twice a frame for about 30 frames, in the title scene behind the file select as it slides out before a new game. [Bayonetta 3](../../dm/games/01004A4010FEA000/README.md): two domain shaders do not build (`DS_Vanish05_*.dso`: `SPIRV-Cross threw an exception: Subgroup builtins are not available in this type of function`); the one-lane rewrite (`spv_single_lane`, [Subgroup operations rewritten for a one-lane subgroup](#subgroup-operations-rewritten-for-a-one-lane-subgroup)) covers vertex shaders only, and tessellated draws are skipped anyway. +- **Note:** the vertex shaders that feed a hull or geometry shader can be found from the shader headers (SPH): the generic attribute map at +0x18 of the consumer's header (inputs) equals the one at +0x36 of the vertex shader's (outputs). In Luigi's Mansion 3, 40 vertex shaders match a hull shader and 49 a geometry shader. + +## Bindings and handles + +### Driver-segment reads outside fragment shaders + +- **Symptom:** a vertex or compute shader reads zeros from NVN's driver segment `c0`. +- **Cause:** `dmsh_build.py`'s `drv_reads` scan matches only fragment-shader names (`fp_c0` / `fp_c2`), so vertex and compute shaders that read `c0` were never marked, and the host passed them an all-zero `c0`; these reads are not listed in the JSON. +- **Fix:** the host treats a declared driver-segment binding as read (`nvn_refl.m`). The words the host fills are in the entries below. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md), [Super Mario Odyssey](../../dm/games/0100000000010000/README.md), [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md). + +### Compute `c0`: dispatch size and uniform buffer table + +- **Symptom:** flickering hard-edged white tiles (tiled lighting with every tile row getting row 0's or other rows' lights). +- **Cause:** a compute shader that computes its tile index as workgroup y × `c0[0x200]` + workgroup x reads the dispatch's workgroup count from the driver segment, which the host left zero. +- **Fix:** for compute, `c0` carries the dispatch size at 0x200 (x, y, z) and the uniform buffer table at 0x210 + 16 r (address, size) in front of the storage buffer table at 0x310 (`encode_args_`). +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (Mac and iPhone): the step of Marin and Tarin's doorway. The room shaders use tiled forward lighting: the fragment position picks a 64×64-pixel tile (`trunc(y / 64) × ceil(width / 64) + trunc(x / 64)`), whose light count and light indices come from two storage buffers that a compute shader (`light_culling_compute.bfsha`, one thread per workgroup, one workgroup per tile) fills every frame. Every row's workgroups wrote row 0's entries, the last writer won (a different one each frame), and the bottom tile row, where the step is, got the lights of other rows; a dump of the tile lists showed only the first of 17 rows filled. After the fix every tile row has its lights and the doorway shows the game's soft light from outside. +- **Check:** Breath of the Wild (101 compute and 4 vertex shaders read `c0`) and Super Mario 3D World (11 and 1) render the same as before on the Mac. + +### Graphics `c0 + 0x610`: the largest point size + +- **Symptom:** point sprites of size 0 (Odyssey's Cap Kingdom fog never parted around Mario). +- **Cause:** a vertex shader that writes `gl_PointSize = min(size, c0[0x610])` read 0, because the host had nothing at that offset of the driver segment. +- **Fix:** `c0 + 0x610` holds the largest point size, and the host writes 2047 there for the graphics stages. Writing the `SetPointSize` value instead (always 1 in Odyssey) left one-pixel splats whose disc weight came out 0 (the fields held `-0.0`). +- **Found on:** [Super Mario Odyssey](../../dm/games/0100000000010000/README.md) (reported on an iPhone; fix checked on the Mac): on the Switch the ground fog opens around Mario as he walks through it. The fog's shading reads velocity fields that a small fluid simulation updates every frame: two 256×256 and two 128×128 RG16F targets, with a two-pass blur and a blended composite. Its only input is a point-sprite splat per mover, drawn with `alPointSpriteShader`: one vertex per mover, its size in the vertex data (Mario's main splat asks for about 18 px, others 3–5 px) and a soft disc in the fragment shader (`clamp(1 - 4r², 0, 1)` × velocity). The simulation fields stayed exactly zero (raw dumps with `DM_NVN_TEXDUMP` + `DM_NVN_SNAPRAW`; the 8-bit dump shows velocities as black either way). After the fix the fields hold velocities up to ±44, and the fog clears around Mario. + +### Graphics `c0 + 0x644`: the draw's base instance + +- **Symptom:** occlusion queries that always return 0 samples (Z-A: characters' legs folded back from the hips once they walked); every instanced draw's transform taken for instance 0. +- **Cause:** a shader that picks its transform with (instance index − base instance) + `c0[0x644]`: Maxwell's instance ID leaves the base instance out, and the NVN driver keeps it in its constant segment, where GLSLC reads it for `gl_InstanceID + gl_BaseInstance`. The host left `0x644` at 0. +- **Fix:** `encode_args` writes the draw's base instance there for graphics stages and does not reuse a stage's argument block across draws with different base instances. The word before it, `0x640`, is added to the vertex ID by 58 of Z-A's shaders and stays 0 until what the driver puts there is known. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): from the first step on, the player's legs stuck straight out behind her at hip height, and the rival's forward in a later cutscene; standing still beforehand looked right. The bone palette the game computes on the CPU was already bent. The cause was six occlusion queries a frame, one 36-vertex box each (`a793f436dcabd350`, base instance 61, 62, …), drawn against mip level 2 of the depth pyramid: every box was box 0, every query came back with 0 samples, and the game took its characters for hidden and posed their legs that way. Forcing every result to visible fixed the legs, and depth testing with ALWAYS did not. 793 of the game's vertex shaders read the word. +- **Note:** how it was found: the raycasts were logged with scratch observation gates (all correct), then `DM_HLECOUNT` showed nothing walking-specific, then the counter reports. + +### Texture and sampler pools with overlapping IDs + +- **Symptom:** a menu's labels drawn as plain bars, other menus empty except for the cursor (one pool's textures overwritten by another's); a black screen when IDs past a fixed table read as empty slots. +- **Cause:** a game that binds two texture pools and two sampler pools whose IDs overlap; the replay had one ID table. +- **Fix:** each pool has a base in the table, in creation order; `RegisterTexture` / `RegisterSampler` add it, and binds recorded on a command buffer add the base of the pool that command buffer last set (`bind_handles` in `hle_nvn.c`, the base kept in `cmd_t`, [next entry](#bases-of-bound-handles-kept-in-the-command-buffer); log line `pool #2 (…): numbered from N`). Handles read from uniform buffers take the base of the pool bound at replay (DARK SOULS: REMASTERED, `ubo_tex_id` / `ubo_samp_id`, [Handles from uniform buffers take the bound pools' bases](#handles-from-uniform-buffers-take-the-bound-pools-bases)). Deferred binds are not remapped (red line; [Deferred texture binds read at execution](#deferred-texture-binds-read-at-execution)). The table's capacity is the sum of the pools. +- **Found on:** [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md): the scene binds a 12,544-entry texture pool and a 3,856-entry sampler pool, the 2D UI's command buffer binds 4,096-entry ones, and both fill IDs from 256. The scene's default 8×8 white texture overwrote the UI's textures (the title menu's labels drew as plain bars). The scene registers each draw's textures into fresh IDs of a ring (8,729 and up a few seconds in), which used to read as empty slots and left the screen black. + +### Bases of bound handles kept in the command buffer + +- **Symptom:** red lines from the list of command buffers that bound a non-first pool (6,734 in one DARK SOULS: REMASTERED run). +- **Cause:** that list had 64 entries and was never cleared, and the game binds a non-first pool in more than 64. +- **Fix:** the bases of bound handles live in the command buffer (`cmd_t` `tex_base` / `samp_base`). +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md). + +### Handles from uniform buffers take the bound pools' bases + +- **Symptom:** distant walls and floors smeared sideways (DARK SOULS: REMASTERED): tiling textures took a clamp-to-edge sampler and repeated their edge column. +- **Cause:** the map materials read their combined texture / sampler handles from a uniform buffer (`fp_t_cb13_*`, resolved at replay by `ubo_handle`), and the game keeps several sampler pools, each with its own samplers under the same IDs, one per command buffer that records the scene. The replay resolved uniform-buffer handles in the first pool. +- **Fix:** the replay keeps the texture and sampler pools bound (`OP_SET_TEX_POOL` / `OP_SET_SAMP_POOL`, which it used to skip; a command buffer that binds none keeps the queue's, as on the GPU) and adds their bases to handles read from uniform buffers (`ubo_tex_id` / `ubo_samp_id` in `nvn_mtl.m`). +- **Found on:** [DARK SOULS: REMASTERED](../../dm/games/01004AB00A260000/README.md): the map materials `FRPG_Phn_*Lit*`; 12 sampler pools of 2,304 entries (ID 261 is clamp-to-edge with LOD 0..0 in one pool, repeat with mips in others); the walls' tiling base textures (u ≈ −3.45). +- **Check:** found by highlighting the six most common fragment shaders of the scene pass (`DM_NVN_HILITE`), then the next six, and reading the wall draw's binding lines (`DM_NVN_PASSDUMP_ALL`). + +### Deferred texture binds read at execution + +- **Symptom:** the 3D scene black, only particles and the HUD drawn; pass dump `B` lines all `tid=0 dummy`. +- **Cause:** `nvnCommandBufferBindTexturesDeferred` passes a pointer to the handle, and "deferred" means the GPU reads the handle from that address *when the command executes*. A game that records its command blocks once, submits them every frame and writes the handles afterwards got the handles the host copied at recording, mostly zeros, so draws read the dummy texture 0. +- **Fix:** the bind record keeps the source address (`p_bind_handles.defer / src`) and every replay-time reader refreshes the handles first (`bh_fresh` in `nvn_mtl.m`). +- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): the engine binds almost every texture this way (`main+0xec2e64`, one handle, the pointer being the handle field inside the texture-sampler object); the deferred-lighting composite (one full-screen triangle per material class, blending into the HDR target) read texture 0 for all of its inputs. +- **Note:** how it was found: a frame data-flow table built from `DM_NVN_PASSDUMP_ALL` (per pass: attachments, shader keys, sampled serials) showed the half-resolution lighting outputs never being read, and the composite pass's `B` lines were all `tid=0 dummy`. Without `_ALL` the dump lists only the first draw of each pass, which hid the composite. Deferred binds are not remapped by pool base ([Texture and sampler pools with overlapping IDs](#texture-and-sampler-pools-with-overlapping-ids)). + +### Handles in uniform buffers written earlier in the same submission + +- **Symptom:** bindless handles read from a uniform buffer name the wrong texture when the game writes them earlier in the same submission. +- **Cause:** the replay reads handles on the CPU, and a write queued earlier in the submission has not landed yet. +- **Fix:** inline uniform updates keep where their data sits, so the handle is read from the newest one covering it (`pend_read`). The old limit of 256 writes per submission, after which every address counted as written, is gone: pending writes are now indexed by 4 KB page (8,192 per submission). +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (it queues more than 256 writes per submission). + +### Storage buffer addresses in the game's own uniform buffer + +- **Symptom:** a storage buffer reported with `cb_slot` 3, which had no NVN slot. +- **Cause:** the shader reads a storage buffer's address from the game's own uniform buffer (c3 + 0x38), which Ryujinx reports with `cb_slot` 3. +- **Fix:** the build marks it `sbuf_ubo` (uniform slot `cb_slot - 3`, byte offset), and the replay reads the 64-bit address there at bind time, as it reads bindless handles. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (about 30 compute shaders). + +### Bindless handles through a loop phi + +- **Symptom:** `no resolved slot number`, the shader does not load; the translation has a 32-entry texture-pool array (`fp_t2d_cb0_40000000`, handle type Direct). +- **Cause:** the handle is `c2[0x1A0] | 0x10500000` (separate texture slot 7, constant sampler 0x105), kept in a loop variable. Ryujinx's bindless elimination compared only the phi's direct inputs, gave up, and turned the access into an index into a 32-entry texture-pool array, which the replay cannot bind. +- **Fix:** the vendored translator now follows the phis that feed a phi, and treats two `constant buffer | constant` handles as the same value ([third_party/ryujinx/README.md](../../dm/third_party/ryujinx/README.md)). Those accesses become ordinary type-3 handles. +- **Found on:** [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (218 pixel shaders). + +### Texture slots picked at run time + +- **Symptom:** the shaders' bindless arrays have no slot and used not to load; the translator falls back to a 32-entry array such as `array<texturecube<float>, 32> fp_tcube_cb0_40000000`. +- **Cause:** the shader reads one of several c2 texture-table words, chosen by branches. With separate textures (Octopath Traveler 0) it ORs the word with a sampler-table word; with combined ones (Paper Mario) the handle is read from the combined texture table (c2 `0x20 + 8 × slot`, constant offsets only). The translator cannot trace that to one slot and falls back to Ryujinx's bindless arrays, `array<texture_buffer<float>, 32> vp_tb_cb0_40000000` and `array<sampler, 32> vp_s_cb0_40000000`, indexed by the handle's texture ID (low 20 bits) and sampler ID (high 12), capped at 31; the build leaves them without a slot. +- **Fix:** with the profile key `tex_dyn_slots` the reflection reads them as `NVN_RB_TEXD` / `NVN_RB_SAMPD` arrays (`nvn_refl.m`), the replay writes slot indices into that shader's c2 tables (texture slot r → r, sampler slot r → r << 20; in the combined table `r | r << 20` in slot r) and binds slot k at element k, so the shader's own arithmetic lands on the binding: the handle names element r, which holds the texture bound in slot r. The array's other elements hold whatever the other slots do, mostly 2D textures where the array is of cubes; an array element bound to a 2D texture gets the zero cube without a type mismatch report (`tex_fit_elem`), since the shader does not pick it there. The host logs each such shader's first encode with the slots that held textures. Host only: the chain version does not change. +- **Check:** check first that the handles come only from c2's tables: in Octopath Traveler 0 every integer read of c2 in those shaders is a table slot (texture slots 0-10, samplers 0-4). +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): 61 Unreal vertex shaders, particles reading their data from buffer textures (separate textures and samplers). [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md): 2,386 pixel shaders (5,476 cube samples, 487 2D), slots 0, 7, 8, 13 and 28 (combined table). + +### A texture chosen at run time between two uniform-buffer handles + +- **Symptom:** Ryujinx's fallback arrays (`fp_t2d_cb0_40000000`) in pixel shaders whose handle comes from the game's own uniform buffers; the shaders do not load and their draws are skipped. +- **Cause:** the handle is one of two words of the game's own uniform buffers, chosen by a flag, packed as texture ID | sampler ID << 20. `tex_dyn_slots` covers only handles from the driver's c2 tables ([Texture slots picked at run time](#texture-slots-picked-at-run-time)). +- **Fix:** not done. Resolving them needs the translator to report such a choice of two uniform-buffer handles, a chain change for every profile. +- **Found on:** [Big Brain Academy: Brain vs. Brain](../../dm/games/0100620012D6E000/README.md): ten pixel shaders (six in `03GamePerception.bnbshpk`, two PBR, two in `main`); in `e10f24835bbaa5f6` the handle is c3 +0x130 or c12 +0x1d0, chosen by a flag in c12. None of the activities checked uses them; a long run of activities met one once. + +### An array of handles in a uniform buffer + +- **Symptom:** the host stops at the shader's first load: `unexpected argument buffer layout: encodedLength=640 maxAid=48`. +- **Cause:** a loop reads one handle per light from the game's uniform buffer at a computed offset; Ryujinx's texture array from a buffer indexes it by the handle's byte offset ÷ 8 (an NVN handle is 64-bit, the shader computes `(word >> 2) << 1 | (word & 3) >> 1`). The build reads the name as one bindless texture (`tex_ubo`, offset 0), so the argument buffer was 32 entries longer than the reflection said. +- **Fix:** the host's reflection reader takes the array length from the MSL type, and the replay fills element k from the handle at byte 8k of that uniform slot (`NVN_RB_TEXU` / `SAMPU` with `n`); elements whose bytes are not a registered texture or sampler ID read the dummy, silently. No chain change: the build already wrote the array type. +- **Found on:** [Paper Mario: The Thousand-Year Door](../../dm/games/0100ECD018EBE000/README.md): the reflection shaders (`reflection.bfsha`: `array<texture2d<float>, 32> fp_t2d_cb14_0` and its samplers), c14 (uniform slot 11). The first run showed elements 0–3, 6 and 8–12 holding textures, among them the shader's fixed handles at +8, +16 and +24 and the lights' from +64. + +### An array of handles from the driver table c2 + +- **Symptom:** `shaders with a binding whose slot was not decoded 588`; shadow cascades read through `array<texture2d_array<float>, 32> fp_t2das_cb2_0`. +- **Cause:** Ryujinx's texture array from a buffer, as Paper Mario's uniform-buffer arrays ([An array of handles in a uniform buffer](#an-array-of-handles-in-a-uniform-buffer)), but over the driver table c2, indexed by (0x80 + a cascade offset) ÷ 8. +- **Fix:** `dmsh_build.py` `c2_array` names it `tex_c2a` / `samp_c2a` with the c2 byte offset of element 0; the replay fills element k from the slot whose c2 word is at that offset + 8k (combined, else separate), and the dummy elsewhere (`NVN_RB_TEXA` / `NVN_RB_SAMPA`, `c2_handle`). +- **Found on:** [Luigi's Mansion 3](../../dm/games/0100DCA0064A6000/README.md) (586 pixel shaders, the shadow cascades). + +### A texture slot that shares its c2 word with an image slot + +- **Symptom:** characters and props glowing white, about 10⁴ too bright, in DRAGON QUEST VII Reimagined's opening cutscene only, while walls and gameplay look right. +- **Cause:** a size query (`GetDimensions` on a `RWTexture3D`) is a TXQ on the image's handle, c2 `0x120` (image slot 0): the build numbers it combined texture slot 32, the slot that shares that word, and with no type in the facts table the translator read it as 2D. The host bound the empty texture slot 32, and the 1×1 dummy's size passed only thread (0, 0, 0). +- **Fix:** two general fixes: a texture binding of slot 32–39 reads the image bound in slot 0–7 when there is one, as c2 holds it (`nvn_mtl.m` `tex_slot_h`; Hades II's texture slot 34 without an image still reads the texture), and a query on an image handle the facts do not type is translated again with the type of that image from the first translation (`dmsh_xlate` `ImageTypesOf`). +- **Found on:** [DRAGON QUEST VII Reimagined](../../dm/games/0100A9D01C446000/README.md): the per-pixel volumetric lightmap read brick (0, 0, 0) of the brick atlas for almost every cell. Unreal merges each streamed level's lightmap into one atlas on the GPU: brick copies (`0a7ff4171bd1a82a`), then a patch of the shared 16³ `RGBA8_UINT` indirection texture (`7b958a7b21e72801`) whose bounds come from `GetDimensions` on the very `RWTexture3D` it writes. Only cell (0, 0, 0) was written, every other cell kept size 0, and the shader fell back to the atlas origin: the first daytime level's bricks, baked about 2¹⁵ brighter (the same bricks in gameplay hold 0.5–1.4). [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) has four such size queries (`cp_t_tcb_48` / `_4A`). +- **Check:** how it was found: the raw scene colour (`texraw`) put the characters at 1,000–2,000 and the walls at 0.05; a debug build of one character shader that wrote its terms in alternating 8-pixel columns isolated the volumetric lightmap ambient term; dumps of the atlas (`texlist` / `texraw`), of the uploaded level bricks (the 3D upload dump) and of the indirection texture, with the cell computed from the primitive's origin and View `VolumetricLightmapWorldToUVScale / Add` (View 220 / 221), gave the rest. + +### Image atomics + +- **Symptom:** the replay put a texture resource ID where the shader wanted an address; with the old 64 KB dummy buffer, pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced and the game or the replay crashed, or `kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers (GitHub issue #101). +- **Cause:** SPIRV-Cross's MSL 2.3 emulation declares the image twice: the texture (`texture_buffer<uint> cp_i_cb2_4C_r32ui`) and a `device atomic_uint* cp_i_cb2_4C_r32ui_atomic` pointer to the same memory, which the build read as a second image of the same slot. +- **Fix:** the pointer is now its own kind (`NVN_RB_IMGA`), bound to the storage of the image in that slot: a buffer texture is a view of its memory pool, so the atomics and the texture reads see the same words. A block-linear image is a deswizzled Metal copy whose pool memory has another layout, so its atomics go to a scratch buffer (logged red) until a game needs them followed; a 3D image does not compile at all ([Unreal Engine 5 shader code archives (Oodle)](#unreal-engine-5-shader-code-archives-oodle)). The scratch buffer is at least as large as the image (`atom_scratch`): SPIRV-Cross indexes the atomics by the image bound in the same slot (`spvImage2DAtomicCoord`: x + y × its width). +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): 2,084 such shaders (Unreal's GPU particles); three ran in the first 100 seconds. [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): in the 64 KB dummy buffer used before, its atomic max into a 480x270 r32ui 2D array (fragment shader `2bd5037003980716`, the scene after the story opening's first cutscene) ran about 1.4 MB past its end, writing into zero-copy pools over the game's heap and into unmapped addresses (GitHub issue #101). +- **Note:** Metal shader validation (`MTL_SHADER_VALIDATION=1`) stops out-of-bounds accesses, so under it the scene played. + +## Shaders compiled at run time + +### Built-in shaders compiled with the game's own GLSLC + +- **Symptom:** shaders missing from the map that no game file holds. +- **Cause:** a game whose shaders come from its own GLSLC at boot has no file holding their output. KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (`glslcCompile` from `main+0x9c89f8`; the sources are strings in `main`). +- **Fix:** they are captured once on the Mac (`DM_GLSLC_CAPTURE`): `DM_GLSLC_CAPTURE=<dir>` saves every compile's output (`hle_glslc.c`; the compile object's first word points at {status, output}), and `shader_split.py` reads `<version dir>/glslc_capture` (the NSP script makes it for any game whose `main` holds GLSL sources and imports `glslcCompile`). The outputs are identical on every run. `shader_split.py` asks for captured GLSLC outputs only when a module of the profile's defines `glslcCompile`, as `profile_guess.py` does. +- **Found on:** + - [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md) (SDK 6.5.0): the 2D primitive, text and loading-feather shaders. + - [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md) (SDK 8.2.1): its two built-in programs are captured on the Mac the same way; they compile only once the host reports the GLSLC's GPU code version ([next entry](#glslc-gpu-code-version-against-the-drivers-range)). + - [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md): `subsdk0` is NVN's runtime GLSL compiler and `main` imports `glslcCompile*`, but up to the house, the beach and the menus every shader the game loaded was in the offline tables. + - [Captain Toad: Treasure Tracker](../../dm/games/01009BF0072D4000/README.md) (the rule that a module must define `glslcCompile`): `main` imports `glslcCompile` and holds agl's GLSL sources, but no module defines it, so nothing is compiled at run time. + +### GLSLC GPU code version against the driver's range + +- **Symptom:** a null read at `main+0x3316dc` in `nnMain`, right after the window setup (the program object's interface pointers at +0xd0 / +0xd8 both null). +- **Cause:** the game compiles two built-in programs at boot with its own GLSLC (`subsdk0`, the same build as Link's Awakening's), and first checks that GLSLC's GPU code version against `nvnDeviceGetInteger` items 0x2E–0x31 (the driver's supported range, `main+0x330dcc`). The host reported Super Mario Odyssey's SDK 7 driver range, 1.5–1.14; this GLSLC writes 1.15, so nothing was compiled and the first uniform lookup read the empty program. +- **Fix:** every SDK's driver runs the code of the GLSLC in the same SDK, so the host now reports at least the version the game's own `glslcGetVersion` returns (`hle_glslc.c` `dm_glslc_gpu_code`, `hle_nvn.c`). The newest version the host reports also follows the SDK since Bayonetta 3 ([Newest GPU code version reported by the driver](nvn-objects.md#newest-gpu-code-version-reported-by-the-driver)). +- **Found on:** [Catherine: Full Body](../../dm/games/0100BF00112C0000/README.md) (SDK 8.2.1, Atlus's GFD engine, CRI middleware; one boot). + +## Pipelines and caches + +### Shader cache key covering the pairs + +- **Symptom:** a vertex shader whose pairs changed was restored unpadded from `cas/`. +- **Cause:** `dmsh_build.py`'s shader cache was keyed without the pairs. +- **Fix:** the per-shader key covers the pairs, and a paired vertex shader's key also covers its pixel shaders' keys (`shader_key`, `pair_keyed`), so a change to a profile's pairs rebuilds what it affects ([Pixel shader inputs the vertex shader does not write](#pixel-shader-inputs-the-vertex-shader-does-not-write)). +- **Found on:** [13 Sentinels: Aegis Rim](../../dm/games/01003FC01670C000/README.md). + +### Function constants specialized at load + +- **Symptom:** `fragmentFunction main0 cannot be used to build a pipeline state` (a Metal validation assertion). +- **Cause:** SPIRV-Cross declares an optional function constant, `spvLinearTextureAlignmentOverride`, in shaders that read a texel buffer, and Metal refuses any function with function constants until it is specialized. +- **Fix:** `shader_load` specializes such functions with an empty set of values. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (one imgui pixel shader). + +### Layer-writing vertex shaders get the triangle class + +- **Symptom:** a pipeline for a vertex shader that writes `render_target_array_index` cannot be built. +- **Cause:** Metal builds such a pipeline only with an input primitive class, and the translator writes a point size in every vertex shader, which needs the point class: no class fits both. +- **Fix:** `vs_layer` drops the point size from vertex shaders that write the layer, and the host gives their pipelines the triangle class once Metal names the reason (`layer_out` in `pso_get_` and in the background prewarm). Their passes are layered ([Layered passes for vertex shaders that write the layer](rendering.md#layered-passes-for-vertex-shaders-that-write-the-layer)). +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): two full-screen vertex shaders (`57246f9a04af8529`, `b29f08619f379a83`: one triangle per instance, layer = instance + offset, 20 pixel shaders read the layer). + +### A cold Metal cache looks like a long load + +- **Symptom:** the first boot sat on the loading clock for about a minute, the CPU idle, with every file read done after 12 s. +- **Cause:** that run compiled its Metal libraries and pipelines on first use (cold cache). With a warm cache the title is up after about 15 s; what looked like a 40-second wait in the read log (`DM_FSLOG`) was the title screen itself, which reads nothing. +- **Check:** screenshots every few seconds (`ctl.sh snap` from a script) told the two apart. +- **Found on:** [Bayonetta 3](../../dm/games/01004A4010FEA000/README.md) (SDK 13.3.2). diff --git a/docs/reference/threads.md b/docs/reference/threads.md new file mode 100644 index 0000000..fd43aaf --- /dev/null +++ b/docs/reference/threads.md @@ -0,0 +1,118 @@ +# Threads + +Guest threads, their synchronization and how the host schedules them. Each entry: symptom, cause, fix, the games it was found on. The [symptom index](../agents/symptoms.md) lists every symptom. + +## Thread creation and cores + +### Core mask changes move the ideal core into the new mask + +- **Symptom:** per-core job queues never drained. +- **Cause:** a `SetThreadCoreMask` that pins a thread to other cores (ideal core −3 = keep) left the ideal core outside the new mask, so `GetCurrentProcessorNumber` reported a core the thread cannot run on. +- **Fix:** the host moves the ideal core into the new mask, as Horizon does, so `GetCurrentProcessorNumber` reports a core the thread can run on. + +### Ideal core −2 resolves to the process default core + +- **Symptom:** crash indexing a per-core table with a huge offset (index −2). +- **Cause:** threads created with ideal core −2 (*use process default*) were recorded as core −2, and `GetCurrentProcessorNumber` returned −2. +- **Fix:** −2 resolves to the process default core (0), and a negative core is never returned (`hle_thread.c`). +- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md), in its font engine (`main+0x107dbc0`). + +### Free thread count (`svcGetInfo` type 24) + +- **Symptom:** the main thread waits forever on an event right after `nnMain` starts, with only three or four threads created; `DM_THR=1` shows no `CreateThread` for the thread it waits on. +- **Cause:** Unity's thread start (`main+0x3e1a50`) creates a thread only when `nn::os::GetFreeThreadCount()` is at least 1. That is `svcGetInfo` type 24 (FreeThreadCount, Horizon 11 and later), which the host answered 0 like every type it did not know, so the thread was never asked for. The kernel answers the thread resource limit less the threads alive (Atmosphère `kern_svc_info.cpp`); `pm` gives an application 96 threads (`pm_spec.cpp`, 256 with a boost). +- **Fix:** the host answers 96 less the game's threads that have not exited (`thr_limit_used`; guest threads the host spawns for itself are left out). +- **Found on:** [Monster Train 2](../../dm/games/010051701FB46000/README.md) (Unity 2022.3.54f1 on SDK 18.3.5: `main+0x3e7288` starts a thread and waits for it to signal `main+0x539cf80`). With the fix the game creates 40 threads and reaches its title. +- **Note:** of the 20 games extracted on this machine, only this `main` imports the function. + +### Pausing threads and reading their context for garbage collectors + +- **Symptom:** in garbage-collected titles (Unity / IL2CPP), live objects freed, with their finalizers releasing native resources. +- **Cause:** the Boehm collector pauses each thread, reads its registers for roots and scans its stack. `SetThreadActivity` (0x32) and `GetThreadContext3` (0x33) were placeholders that only reported success: threads kept running and the context was empty, so roots held only in registers were missed. +- **Fix:** both really pause and report registers (`hle_thread.c`). A thread inside an SVC counts as paused at once, with the gate frame as its context (`thr_svc_enter` / `thr_svc_leave` around every gate SVC; it waits at the SVC exit while paused). A thread in guest code gets SIGUSR2, and the handler saves the ucontext and waits there. A thread in host code is retried every 100 µs. The context is Horizon's 0x320-byte `ThreadContext`. +- **Found on:** Unity / IL2CPP titles in general. + +### Approximating same-core preemption (`core_preempt`) + +- **Symptom:** crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`. +- **Cause:** the game relies on Horizon's per-core preemption. Its main thread (priority 44, core 0) posts "unlink these holders" to a dispatcher thread and leaves messages for a worker thread (both priority 30, only core 0), then frees the object and builds the next one in the same memory without waiting. On hardware the two threads preempt the main thread as soon as it signals them, so their work is done first; on the host all threads run in parallel. +- **Fix:** two fixes, both needed (with only the second, 3 of 6 runs still crashed; with both, 12 of 12 were clean): + - an observation gate after each request the game posts (OBS 19 / 20 in its `patches.py`, this build only) waits until the dispatcher's request ring is empty (`movie_ring_wait` in `hle_movie.c`). Gates may now sit right after a `bl` as well as at function entries; + - an approximation of same-core preemption (`svc_thread` / `dm_yield_to_core_peers` in `hle_thread.c`): after a thread SVC, the caller waits (at most 5 ms) while a higher-priority thread pinned to its core is running guest code. "Running" means the Mach thread state is running, the thread is not in a guest wait or timed sleep, and it is not doing host NVN submit / present work (`dm_host_work`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md). +- **Note:** the preemption approximation is a profile key, `core_preempt`, on only for Dread: in [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md) the loader threads would wait for busy decompression threads, and its first frame moved from 0.6 s to 2.9 s. + +### Simultaneous crashes of several threads + +- **Symptom:** several threads crash at once and the dumps interleave, then the process hangs instead of exiting. +- **Cause:** Unity's job threads tend to crash together. +- **Fix:** `crash.c` lets only the first crashing thread print (`crash_gate`); the others stop in `pause()`, and a 15 s alarm ends the process if the dump itself stalls. +- **Found on:** Unity titles in general. + +## Synchronization and address arbitration + +### `SignalToAddress` type 2 with count ≤ 0 + +- **Symptom:** none seen; found while investigating [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md)'s demo crash, which it did not change (that was the address-space shortage, [Size-aligned regions exhaust the alias address space](memory.md#size-aligned-regions-exhaust-the-alias-address-space)). +- **Cause:** for `SignalToAddress` type 2 (`SignalAndModifyByWaitingCountIfEqual`) with count ≤ 0 and waiters present, the new value is `value − 2`, not `value − 1` (mesosphere `KAddressArbiter`; Ryujinx and yuzu agree). The host had −1. +- **Fix:** the host writes `value − 2`, the specified behaviour. + +### `WaitForAddress` type 3: 64-bit wait if equal + +- **Symptom:** a 64-bit wait could return at once or block when it should not (not met in a game). +- **Cause:** Horizon 18 added a 64-bit "wait if equal" arbitration type, which SDK 19 exports as `nn::os::WaitForAddress(const long*, long)`; the host used to fall into its 32-bit "wait if less than" test. +- **Fix:** type 3 compares the 64-bit value (`hle_thread.c`). +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md) (SDK 19.3.11): Z-A does not call it, but the SDK has it. + +### Late PhysX task destructor after the unlock + +- **Symptom:** a PhysX worker crashes in a task's `removeReference` with a null task manager, right as the game switches to 30 fps (`pc=main+0x10ccf28`, `lr=main+0x13c236c`); on the iPhone and in 2 of 5 Mac runs, in every Mac run with 12 busy loops loading the CPU. +- **Cause:** a destructor that runs a moment too late. The game rebuilds each physics world's completion task in place before every `simulate` (`main+0x15730`, main thread). A worker runs the task (`fetchResults`, sets the world's done flag, notifies, unlocks) and then calls its `release`, which calls the task's own destructor (vtable back to the `PxLightCpuTask` base, `mTm = 0`). At 30 fps the game steps twice per frame, so the unlock wakes a main thread that steps again at once. On the Switch the `release` follows the unlock within nanoseconds; on Apple devices the worker can be descheduled between them, the main thread rebuilds the task, and the late destructor wipes the new one. The next simulation's `NpSceneCompletion::release` then dereferences `mTm`. +- **Fix:** a per-build cut makes that one `release` a `ret` (its `patches.py`); the destructor frees nothing and the step rewrites every field. +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md). How it was found is in [05](../agents/05-debugging.md#racy-crashes-load-trace-gates). + +## Yields and spin waits + +The yield back-off (`DM_YIELD_BACKOFF`) and the rewriter's spin waits (`patch_nso.py` "Spin waits", `dm_spin_wait` in `hle_thread.c`), with their measurements, are in [development.md](../development.md#power) ("Yield loops", "Spin loops"). + +### Timer slack in guest sleeps (`sleep_precise`) + +- **Symptom:** a 30 Hz game thread runs at 27 Hz, and whatever it paces comes out slow (cutscenes at 36 ms per frame instead of 33.3; gameplay was fine at 60). +- **Cause:** macOS timer slack: `nanosleep` and `mach_wait_until` overshoot by about 10% at any QoS (measured: 33 ms → 36.4 ms, 5 ms → 6.1 ms, 1 ms → 1.26 ms). Dread's sound thread (`main+0xf95b0`) reads a millisecond clock, updates, then sleeps "33 ms minus this round's work", so on the host its period was 36.9 ms. The cutscene loop renders a frame whenever the audio play position, which that thread refreshes, moves (`main+0x119a3d0`; the position is played samples ÷ sample rate of the cutscene's stream), so cutscenes ran at about 27 fps. +- **Fix:** `SleepThread` sleeps three quarters of the remaining time per step until within 20 µs (`sleep_precise` in `hle_thread.c`). +- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md): cutscenes at 33.3 ms, the sound thread at 33.2 ms. [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s frame-time p95 also went from 19.7 to 18.4 ms. +- **Note:** timed waits (`WaitSynchronization`, condition variables) still use the host's own timeouts. + +### Sleeps under 20 µs take the yield path + +- **Symptom:** a guest thread keeps a core at 100% with billions of `SleepThread` calls. +- **Cause:** `sleep_precise` returns at once for anything under 20 µs (its nanosleep would overshoot), so a loop of such sleeps never sleeps. +- **Fix:** sleeps under 20 µs take the yield path and its back-off (plain yields for the first 2 ms of a run, then short sleeps). +- **Found on:** [Super Mario Galaxy](../../dm/games/010099C022B96000/README.md): Hagi polls with `SleepThread(100 ns)` (`main+0x517c`), 1.4 billion calls in 200 s, and the render thread nearly as busy. The two threads went from 199 s and 171 s of CPU per 200 s to 5 s and 19 s per 120 s, at the same frame rate. + +### Yields that make progress return at once (`thr_yield`) + +- **Symptom:** a loading screen sits for a minute with the CPU nearly idle; one thread yields thousands of times a second, every yield a back-off sleep (the power line's top yielder, with as many back-off sleeps). +- **Cause:** the thread was working, not waiting: its loop calls `nn::os::YieldThread` once per item, and the back-off made each yield a 250 µs sleep. +- **Fix:** a yield whose caller's x19–x28 changed since the last one is progress (`thr_yield` in `hle_thread.c`; the SDK reaches the SVC through tail branches, so those registers are the loop's own): it returns at once with no host call, and the run starts again. Polls keep those registers (Three Houses' limiter, Galaxy 2's Hagi sleep poll at `main+0x11ec`); counters move them. +- **Found on:** [Animal Crossing: New Horizons](../../dm/games/01006F8002326000/README.md): a black screen with the plane icon for 77 s after the opening movie, `SaveProc` at about 2,600 yields a second, its grid walks over the new island (`main+0x157d020`) yielding once per cell; also about 40 s of the same at boot. The island now shows about 10 s after the movie, as with `DM_YIELD_BACKOFF=0`. Polls that keep their registers: [Fire Emblem: Three Houses](../../dm/games/010055D009F78000/README.md), [Super Mario Galaxy 2](../../dm/games/0100FD8022DAA000/README.md). +- **Check:** check such a change on [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) outdoors: its `PlacementMgr` and `RadarMgr` turned out to yield per item too, and a `sched_yield` on each of their 200,000 yields a second cost a third of a core more than the back-off had; with no host call, CPU energy fell instead (numbers in [development.md](../development.md) "Yield loops"). +- **Note:** a temporary log of which of x19–x28 move at each yield, per thread, is what told the two kinds apart. + +## Events and notifications + +### One auto-clear event shared by a pool's workers + +- **Symptom:** the main thread waits forever on a job's done event at boot while every worker sleeps, two of them on one event (`main+0x1426c` waiting on `Job::m_doneEvent`); 4 of 6 Mac runs. +- **Cause:** the engine's job pools wake one worker through the pool's single auto-clear event. A job goes into worker k's own queue (`main+0x154c0`), and if k's state reads asleep the caller signals the event (`main+0x12554`); the kernel wakes the waiter that slept first, which is k only while the workers sleep in index order. A worker woken for another's job runs only its own queue and sleeps again, so with two workers in one pool the signal is lost: at boot when worker 1 went to sleep before worker 0, and later as a job left in worker 0's queue for 12 s. The thread dump's `wait #` (the order condition waiters arrived in, which `SignalProcessWideKey` wakes them in at equal priority) showed the order in every hung run. +- **Fix:** the game's own wait wrapper (`main+0x12500`, `{event, pending}`) signals the event once more when it takes a nonzero pending word after waking, and nothing set it: a build cut makes the signal set it first (its `patches.py`), so a worker woken in another's place passes the wake on; 6 of 6 runs boot. +- **Found on:** [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md). +- **Note:** a host change that made `StartThread` return only once the new thread runs (as a free Switch core takes it within microseconds) brought the hangs to 1 in 6 and was dropped. + +### The oe notification event and its queue + +- **Symptom:** abort in `os::WaitSystemEvent` under `oe::PopNotificationMessage` (The Legend of Zelda: Link's Awakening); one guest thread at about 1000 ms/s on the iPhone only, the phone hot, CPU energy 5–7 W (Monster Hunter Rise's `MessageSuperviseThread`, priority 32, in the `[nvn] threads` lines). +- **Cause:** the host keeps the notifications in a queue of its own. The blocking pop waits on the system event `oe::Initialize` would have created, and the host's `Initialize` is a no-op. Monster Hunter Rise's thread waits on `oe::GetNotificationMessageEvent()` with `os::WaitAny`, which does not clear an event, then pops notifications until the queue is empty. The host only ever signalled its notification event, so after the first focus change (iOS changes focus at launch; the Mac does not) it stayed set and the loop never blocked again. +- **Fix:** the host now implements the blocking pop on its notification queue (a condition variable, then the `TryPop` path), and the event follows the queue as the am service's does: set while a notification is queued, cleared when the last one is popped (`hle_oe.c` `mq_ev_sync`, `hle_clear_system_event`). +- **Found on:** [The Legend of Zelda: Link's Awakening](../../dm/games/01006BB00C6F0000/README.md) (SDK 8.2.0, the blocking pop); [Monster Hunter Rise](../../dm/games/0100B04011742000/README.md) (the event). +- **Check:** Mac repro `DM_OE_SCRIPT=focus=2@5,focus=1@6` (the script also starts at the game's first `GetNotificationMessageEvent`, since such a game pops only once the event fires): before 995 ms/s, CPU 109%, 6.97 W; after, the thread is idle and CPU energy is 0.05 W.