Pokémon Sword (0100ABF008968000, update 1.3.2, Game Freak's engine on SDK 7.3.2): game profile plus general changes to the shader split, the shader build, the NVN layer and HID / offline services. Plays on the iPhone (IPA 12.4 GB, user). On the Mac from a fresh save through the first boot's choices, the opening, Postwick, Hop's house, the walk to Wedgehurst and Leon's arrival, the starter choice (Scorbunny) and the first battle against Hop (won), and from a save in the bedroom through the title and out of the front door: 30 fps as on the Switch (frame time p50 33.3 / p95 about 35 ms, GPU 4-9 ms a frame on an M6, CPU about half a core), sound, 1.8 GB resident after the title and 2.4 GB after 17 minutes of new areas, 1.9 GB of Metal allocations, no red line on that route. The sdk and movie module (subsdk0) builds are Super Mario Odyssey 1.4.1's; subsdk1 is GLSLC, which main imports but never calls; no virtual address memory (a 3.13 GB heap in the 4 GB layout, mem_used 76 MB). shader_split.py: everything is in 7,699 GFPAK archives (GFLXPACK: file count at +0x10, 0x18-byte file entries of compression, length, stored length and offset; every entry LZ4, zlib and stored read too, Oodle a red line), whose BNSH entries the split now reads (gfpak_members): 3,126 shaders before, 38,361 after with the loose BNSH files and two in main. The 38 shaders still in no map at the title were Silicon Studio's YEBIS post effects: a .tfxo file (3,155 shaders, control sections each followed by its code area of control length + 0x704) in a 39.5 MB zlib stream in main (+0x1c98ed9), which tfxo_streams finds by its zlib header and the inflated file's magic (the header check inflates 4,096 bytes; 64 did not get past the first block header) and tfxo_shaders reads, single-stage programs: 41,516 shaders in all. Of the other games' mains the stream reader finds nothing, and main's loose control section reader (raw_shaders) finds nothing in this one, so every other profile's tables stay as they were. dmsh_build.py / nvn_mtl.m: the G-buffer pixel shaders write RGBA16Uint at colour 1 between two float targets, and the all-integer entry point (fs_uint, main0_u) made Metal refuse the pipeline for the float ones; a pixel shader whose outputs are partly integer bits now also has main0_m, which returns only those as integers (uint_mask_m in the reflection, 336 shaders here), and pso_desc takes it when the bound targets' integer formats are exactly those outputs (pso_mixed_uint). Chain version dec5ee141a9f. nvn_mtl.m tex_by_id: a cube view whose layer count is not a multiple of 6 (a view of one layer, as the player leaves the house) aborted in Metal; it now counts its layers in cubes when they fit, and is read as a 2D array with a red line otherwise. Packaged textures: the game writes beside its textures in the 16 KB host pages they share with other data, so each such write re-uploaded the texture, 73k re-uploads over a few minutes and frames from 33 to about 75 ms; the bytes of a texture's storage that lie in its shared first and last pages are kept after each upload (pkg_edges_take), and a dirty texture whose interior pages were not written and whose edge bytes are unchanged is re-protected without a re-upload (pkg_edges_clean; the packaged-textures log line counts them): 0 re-uploads on the route since. New t_nvn case 6c1 (writes beside a texture, then a real write), 110 checks. hle_nvn.c: nvnDeviceGetWindowOriginMode and nvnDeviceGetDepthMode, which the game reads without setting either. hle_hid.c: the Poké Ball Plus (Palma): boost mode a no-op, connection enable / disable succeed, GetPalmaConnectionHandle and every handle call fail with an invalid handle (the first boot crashed at SetPalmaBoostMode). hle_offline.c: pdm::QueryApplicationPlayStatistics answers each asked title as never played (the game asks about the Let's Go games; it crashed there next), err::ShowNetworkServiceError is logged. Profile: pad_layout full (the camera is on the right stick), shader groups bin / systemresource, a prewarm list recorded on the Mac over the route above (3,273 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); after the rebase onto 6d42bc9 the host unit tests, the 41 tool tests, t_nvn, t_dirty_slots, t_dirty_race, t_integer_blend, t_upgrade_alias, t_clear_ds and t_stencil_view (under validation) pass, and the route out of the house runs as before. README (games table, a Mac picture of Postwick), AGENTS.md (53 -> 54), docs/development.md and docs/agents 01 / 03 / 04 say so

This commit is contained in:
seedds committed 2026-10-24 18:55:11 +08:00
1 parent 6d42bc9815
commit 0045c1dfb6
23 files changed
+61487 -24

No files matched your search

+1 -1
View File
@@ -1,6 +1,6 @@
# Notes for agents
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Fifty-three games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`), Monster Train 2 (update 1.1.1, SDK 18.3.5, `010051701FB46000`), Raging Loop (base game 1.0.0, SDK 7.3.2, `010020400E1C2000`), DARK SOULS: REMASTERED (update 1.0.3, SDK 4.5.0, `01004AB00A260000`), Paper Mario: The Thousand-Year Door (update 1.0.1, SDK 15.3.2, `0100ECD018EBE000`), Hollow Knight: Silksong (update 1.0.30000, SDK 20.5.8, `010013C00E930000`), Catherine: Full Body (update 1.0.1a, SDK 8.2.1, `0100BF00112C0000`), Digimon Story Cyber Sleuth: Complete Edition (update 1.0.3, SDK 6.4.0, `010014E00DB56000`), Animal Crossing: New Horizons (update 3.0.3, SDK 20.5.4, `01006F8002326000`), DRAGON QUEST VII Reimagined (update 1.2.0, SDK 19.3.1, `0100A9D01C446000`), FINAL FANTASY RESONANCE (update 1.0.1, SDK 20.5.17, `0100606026F12000`), Luigi's Mansion 3 (update 1.4.0, SDK 7.3.2, `0100DCA0064A6000`), AI: THE SOMNIUM FILES (update 1.0.1, SDK 6.4.2, `010089B00D09C000`), ANIMAL WELL (update 1.0.10, SDK 20.5.7, `010020D01AD24000`), AI: THE SOMNIUM FILES - nirvanA Initiative (update 1.0.2, SDK 13.3.5, `0100713016CCC000`), Mario Kart 8 Deluxe (update 4.0.0, SDK 4.4.0, `0100152000022000`), Balatro (update 1.1.3, SDK 18.3.2, `0100CD801CE5E000`), ROMANCE OF THE THREE KINGDOMS 8 REMAKE (update 1.1.3, SDK 17.5.3, `0100C2A01A730000`), Taiko Risshiden V DX (update 1.1.1, SDK 12.3.3, `0100346017304000`), Super Smash Bros. Ultimate (update 13.0.5, SDK 8.2.1, `01006A800016E000`), Bayonetta 3 (update 1.2.0, SDK 13.3.2, `01004A4010FEA000`) and Advance Wars 1+2: Re-Boot Camp (base game 1.0.0, SDK 10.4.1, `0100300012F2A000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`FRAME_STEP` in their profiles' `patches.py`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout, and its `sdk` build (Breath of the Wild's) has the SDK's address-region node area cut (`sdk_patches.py`): without it the resource heaps ran out of alias region at the first descent from the Great Sky Island and the game played its memory-shortage Blood Moon, which put Link back at the start (Line truncated
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Fifty-four games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`), Monster Train 2 (update 1.1.1, SDK 18.3.5, `010051701FB46000`), Raging Loop (base game 1.0.0, SDK 7.3.2, `010020400E1C2000`), DARK SOULS: REMASTERED (update 1.0.3, SDK 4.5.0, `01004AB00A260000`), Paper Mario: The Thousand-Year Door (update 1.0.1, SDK 15.3.2, `0100ECD018EBE000`), Hollow Knight: Silksong (update 1.0.30000, SDK 20.5.8, `010013C00E930000`), Catherine: Full Body (update 1.0.1a, SDK 8.2.1, `0100BF00112C0000`), Digimon Story Cyber Sleuth: Complete Edition (update 1.0.3, SDK 6.4.0, `010014E00DB56000`), Animal Crossing: New Horizons (update 3.0.3, SDK 20.5.4, `01006F8002326000`), DRAGON QUEST VII Reimagined (update 1.2.0, SDK 19.3.1, `0100A9D01C446000`), FINAL FANTASY RESONANCE (update 1.0.1, SDK 20.5.17, `0100606026F12000`), Luigi's Mansion 3 (update 1.4.0, SDK 7.3.2, `0100DCA0064A6000`), AI: THE SOMNIUM FILES (update 1.0.1, SDK 6.4.2, `010089B00D09C000`), ANIMAL WELL (update 1.0.10, SDK 20.5.7, `010020D01AD24000`), AI: THE SOMNIUM FILES - nirvanA Initiative (update 1.0.2, SDK 13.3.5, `0100713016CCC000`), Mario Kart 8 Deluxe (update 4.0.0, SDK 4.4.0, `0100152000022000`), Balatro (update 1.1.3, SDK 18.3.2, `0100CD801CE5E000`), ROMANCE OF THE THREE KINGDOMS 8 REMAKE (update 1.1.3, SDK 17.5.3, `0100C2A01A730000`), Taiko Risshiden V DX (update 1.1.1, SDK 12.3.3, `0100346017304000`), Super Smash Bros. Ultimate (update 13.0.5, SDK 8.2.1, `01006A800016E000`), Bayonetta 3 (update 1.2.0, SDK 13.3.2, `01004A4010FEA000`), Advance Wars 1+2: Re-Boot Camp (base game 1.0.0, SDK 10.4.1, `0100300012F2A000`) and Pokémon Sword (update 1.3.2, SDK 7.3.2, `0100ABF008968000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`FRAME_STEP` in their profiles' `patches.py`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout, and its `sdk` build (Breath of the Wild's) has the SDK's address-region node area cut (`sdk_patches.py`): without it the resource heaps ran out of alias region at the first descent from the Great Sky Island and the game played its mLine truncated
- Each game's constants and shader tables form a game profile in `dm/games/<id>/`, where `<id>` is the base game's title ID: `game.tsv` plus the tables (and `patches.py` when the rewriter has patches for its builds). Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)).
- Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it.
- Per-build knowledge is keyed by the NSO build ID: rewriter cuts, observation gates and frame-step stubs in the profile's `patches.py` (a game's own modules) or `dm/tools/sdk_patches.py` (Nintendo's `sdk` modules, which several games ship), which `dm/tools/patch_nso.py` (the general rewriter) loads, the profile's `build` rows (the builds the game was brought up on, and their display versions; the NSP script packs any other version from a derived profile, without those builds' fixes), and host behaviour that uses fixed module addresses checks `module_t.build_id` (from `fixups.tsv`; `dm_main_build()`). When a new game build appears, carry addresses over with `dm/tools/nso_fnmap.py` and check them by disassembly; never let fixed addresses run on a build they were not found in.
+1 -1
View File
@@ -52,7 +52,7 @@ Needs iOS / iPadOS 26 or later on an iPhone 15 Pro or newer (A17 Pro chip or lat
| ![Mario Kart 8 Deluxe](docs/images/mario-kart-8-deluxe.jpg)<br>Mario Kart 8 Deluxe<br><sub>IPA ~11.9 GB · update 4.0.0 · Mii heads not drawn (no firmware)</sub> | ![Balatro](docs/images/balatro.jpg)<br>Balatro<br><sub>IPA ~82 MB · update 1.1.3 · played by touch</sub> |
| ![ROMANCE OF THE THREE KINGDOMS 8 REMAKE](docs/images/romance-of-the-three-kingdoms-8-remake.jpg)<br>ROMANCE OF THE THREE KINGDOMS 8 REMAKE<br><sub>IPA ~16.8 GB · update 1.1.3 · controller only</sub> | ![Taiko Risshiden V DX](docs/images/taiko-risshiden-v-dx.jpg)<br>Taiko Risshiden V DX<br><sub>IPA ~3.3 GB · update 1.1.1 · Japanese · played by touch</sub> |
| ![Super Smash Bros. Ultimate](docs/images/super-smash-bros-ultimate.jpg)<br>Super Smash Bros. Ultimate<br><sub>IPA ~17.5 GB · update 13.0.5 · offline</sub> | ![Bayonetta 3](docs/images/bayonetta-3.jpg)<br>Bayonetta 3<br><sub>IPA ~14.2 GB · update 1.2.0</sub> |
| ![Advance Wars 1+2: Re-Boot Camp](docs/images/advance-wars-1-2-re-boot-camp.jpg)<br>Advance Wars 1+2: Re-Boot Camp<br><sub>IPA ~7.1 GB</sub> | |
| ![Advance Wars 1+2: Re-Boot Camp](docs/images/advance-wars-1-2-re-boot-camp.jpg)<br>Advance Wars 1+2: Re-Boot Camp<br><sub>IPA ~7.1 GB</sub> | ![Pokémon Sword](docs/images/pokemon-sword.jpg)<br>Pokémon Sword<br><sub>IPA ~12.4 GB · update 1.3.2</sub> |
## Pack an IPA
+1
View File
@@ -0,0 +1 @@
# generated
1 # generated
+42
View File
@@ -0,0 +1,42 @@
# Game profile of Pokémon Sword (Switch, base game v0 + update 1.3.2 = v458752). Plays on the iPhone
# Format: see dm/games/README.md
# Game Freak's engine (the one before Trinity) on SDK 7.3.2, NVN, Wwise, Pia / NEX networking, with Silicon Studio's YEBIS post effects. The same sdk and movie module (subsdk0) builds as
# Super Mario Odyssey 1.4.1; subsdk1 is GLSLC, which main imports but never calls. Started from the provisional profile scripts/build_ipa_from_nsp.py wrote from its survey (dm/games/profile_guess.py)
# ---- Identity ----
title_id 0100ABF008968000
program_id 0100ABF008968000
# ---- App ----
name Pokémon Sword
ipa PokMonSword.ipa
app_category public.app-category.games
# the camera is on the right stick
pad_layout full
# ---- Versions ----
build v458752 1.3.2 main=a3b75bcd3311385aeed67fbeeb79cbb7 subsdk0=93ff8252d975adf07230cb86c83301c5 subsdk1=a36dac0e0a5ed3b41482b6bb85625c73 sdk=fcf93a41b08c5193a1410d262602dc62
package v458752
shader_versions v458752
# ---- Files ----
# every module main imports from
modules main subsdk0 subsdk1 sdk
romfs_marker bin/sound/NX64/wwise/bank/Stream.pck
zip_store .arc:.bffnt:.bfotf:.gfbmdl:.gfpak:.mp4:.pck:.metallib
# ---- Memory ----
# No virtual address memory (the NPDM declares no system resource): nninitStartup sets a 3.13 GB heap in the 4 GB layout
mem_reserve 0x100000000
mem_alias 0x10000000
mem_stack 0x10000000
app_pool 0xCD500000
sys_resource 0x0
# the rewritten modules are larger than the default 64 MB of used memory (main 25 MB of text, GLSLC 4.8 MB)
mem_used 0x4c00000
# ---- Shader split ----
# 41,516 shaders: the BNSH entries of the 7,699 GFPAK archives under bin/archive (LZ4 entries, shader_split.py gfpak_members), the loose BNSH files of bin/battle, bin/graphics and
# system_resource/shader, two BNSH in main, and YEBIS's post effects, 3,155 shaders of a .tfxo file in a zlib stream in main (tfxo_streams)
shader_groups /bin/=bin /system_resource/=systemresource
shader_note Pokémon Sword shaders
1 # Game profile of Pokémon Sword (Switch, base game v0 + update 1.3.2 = v458752). Plays on the iPhone
2 # Format: see dm/games/README.md
3 # Game Freak's engine (the one before Trinity) on SDK 7.3.2, NVN, Wwise, Pia / NEX networking, with Silicon Studio's YEBIS post effects. The same sdk and movie module (subsdk0) builds as
4 # Super Mario Odyssey 1.4.1; subsdk1 is GLSLC, which main imports but never calls. Started from the provisional profile scripts/build_ipa_from_nsp.py wrote from its survey (dm/games/profile_guess.py)
5 # ---- Identity ----
6 title_id 0100ABF008968000
7 program_id 0100ABF008968000
8 # ---- App ----
9 name Pokémon Sword
10 ipa PokMonSword.ipa
11 app_category public.app-category.games
12 # the camera is on the right stick
13 pad_layout full
14 # ---- Versions ----
15 build v458752 1.3.2 main=a3b75bcd3311385aeed67fbeeb79cbb7 subsdk0=93ff8252d975adf07230cb86c83301c5 subsdk1=a36dac0e0a5ed3b41482b6bb85625c73 sdk=fcf93a41b08c5193a1410d262602dc62
16 package v458752
17 shader_versions v458752
18 # ---- Files ----
19 # every module main imports from
20 modules main subsdk0 subsdk1 sdk
21 romfs_marker bin/sound/NX64/wwise/bank/Stream.pck
22 zip_store .arc:.bffnt:.bfotf:.gfbmdl:.gfpak:.mp4:.pck:.metallib
23 # ---- Memory ----
24 # No virtual address memory (the NPDM declares no system resource): nninitStartup sets a 3.13 GB heap in the 4 GB layout
25 mem_reserve 0x100000000
26 mem_alias 0x10000000
27 mem_stack 0x10000000
28 app_pool 0xCD500000
29 sys_resource 0x0
30 # the rewritten modules are larger than the default 64 MB of used memory (main 25 MB of text, GLSLC 4.8 MB)
31 mem_used 0x4c00000
32 # ---- Shader split ----
33 # 41,516 shaders: the BNSH entries of the 7,699 GFPAK archives under bin/archive (LZ4 entries, shader_split.py gfpak_members), the loose BNSH files of bin/battle, bin/graphics and
34 # system_resource/shader, two BNSH in main, and YEBIS's post effects, 3,155 shaders of a .tfxo file in a zlib stream in main (tfxo_streams)
35 shader_groups /bin/=bin /system_resource/=systemresource
36 shader_note Pokémon Sword shaders
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,73 @@
# Answer table for translation queries (generated by dm/gfx/tools/shader_split.py, do not edit): key <TAB> fact JSON. This game has no PC version to pair with: only the thread group / shared memory of compute shaders (taken from the control section); other queries get the translator defaults
011a43cb50e4dc06 {"local":[32,1,1],"shared":0}
05a44e20c76869a7 {"local":[32,1,1],"shared":0}
0786007b7dc21e9c {"local":[32,1,1],"shared":0}
0cfd5e85029f5692 {"local":[32,1,1],"shared":0}
0f91981d6098cad5 {"local":[32,1,1],"shared":0}
13e01e6fe9783119 {"local":[32,1,1],"shared":0}
146e01bdaeb1dd05 {"local":[32,1,1],"shared":0}
15e220b7564a1cbb {"local":[32,1,1],"shared":0}
18a2d7f790ebb0da {"local":[32,1,1],"shared":0}
1c56b3be6a5ceb81 {"local":[32,1,1],"shared":0}
1f362018763dc959 {"local":[32,1,1],"shared":0}
211aaab6e20e09cd {"local":[32,1,1],"shared":0}
218084506e8e5cbd {"local":[32,1,1],"shared":0}
2605e3a2a8017f38 {"local":[32,1,1],"shared":0}
29da2c64af02a29b {"local":[32,1,1],"shared":0}
29e340f4c5c9745a {"local":[32,1,1],"shared":0}
3031336d651c6ed5 {"local":[32,1,1],"shared":0}
31c1a32672559d5e {"local":[32,1,1],"shared":0}
31e2aefd95320077 {"local":[32,1,1],"shared":0}
322c4964f952a261 {"local":[32,1,1],"shared":0}
3646d30686eed87a {"local":[32,1,1],"shared":0}
396fb1e9a40340b5 {"local":[32,1,1],"shared":0}
3b14df4deffd54f9 {"local":[32,1,1],"shared":0}
3b3197167c0b35cc {"local":[32,1,1],"shared":0}
3d17d3bbbd2aebbd {"local":[32,1,1],"shared":0}
4055a9a90f4ce304 {"local":[32,1,1],"shared":0}
415315107eadf382 {"local":[32,1,1],"shared":0}
432a0c854e52de87 {"local":[32,1,1],"shared":0}
43993b16e70fdb2b {"local":[32,1,1],"shared":0}
43a0f790a9a1a048 {"local":[32,1,1],"shared":0}
4683759d98fb3664 {"local":[32,1,1],"shared":0}
491f79886db70c6a {"local":[32,1,1],"shared":0}
5a08671db22b3d4f {"local":[32,1,1],"shared":0}
5c397b84bff24140 {"local":[32,1,1],"shared":0}
63f47d8e281ea206 {"local":[32,1,1],"shared":0}
66c7cbf4b0da5fd8 {"local":[32,1,1],"shared":0}
67d02efd4c8e1f28 {"local":[32,1,1],"shared":0}
6d7b5fd47027180e {"local":[32,1,1],"shared":0}
72d88868f5ccbb68 {"local":[32,1,1],"shared":0}
7de45a7a9409e16a {"local":[32,1,1],"shared":0}
7e1328b9dcbcb123 {"local":[32,1,1],"shared":0}
7e746d09b6fed628 {"local":[32,1,1],"shared":0}
7f825420c3c95da8 {"local":[32,1,1],"shared":0}
84243c740d0116ba {"local":[32,1,1],"shared":0}
84fda212a190523d {"local":[32,1,1],"shared":0}
853f2bcbcf152dce {"local":[32,1,1],"shared":0}
9b5c281a406e3838 {"local":[32,1,1],"shared":0}
9c3ea94cebe8d5f6 {"local":[32,1,1],"shared":0}
9e6fe66bd7244ea0 {"local":[32,1,1],"shared":0}
9e999c0c2e40ecb7 {"local":[32,1,1],"shared":0}
a8488da1dae449dd {"local":[32,1,1],"shared":0}
a8884b655c500f65 {"local":[32,1,1],"shared":0}
a99ab40a8dd73a06 {"local":[32,1,1],"shared":0}
bb9dc03f45e3573a {"local":[32,1,1],"shared":0}
bdbfaa0c5878121d {"local":[32,1,1],"shared":0}
c4336d300890b3f9 {"local":[32,1,1],"shared":0}
c5821fc4c338363e {"local":[32,1,1],"shared":0}
c6a22eed58ca12b9 {"local":[32,1,1],"shared":0}
c9b77378377a4b73 {"local":[32,1,1],"shared":0}
ce7199264c2d5af5 {"local":[32,1,1],"shared":0}
d475e4ce6dc00c17 {"local":[32,1,1],"shared":0}
da01ca6d6502bdaf {"local":[32,1,1],"shared":0}
dd2a1863c06115ed {"local":[32,1,1],"shared":0}
dfec361ca619bb10 {"local":[32,1,1],"shared":0}
e1b28ffe052340aa {"local":[32,1,1],"shared":0}
e307d5dbbd476b73 {"local":[32,1,1],"shared":0}
e63a891f3c9fd504 {"local":[32,1,1],"shared":0}
e747c9e8cc109716 {"local":[32,1,1],"shared":0}
e8a09777306a22ad {"local":[32,1,1],"shared":0}
f23323b965180c2a {"local":[32,1,1],"shared":0}
f8af20d6d90dc349 {"local":[32,1,1],"shared":0}
fbaf67ae7f77b71e {"local":[32,1,1],"shared":0}
Can't render this file because it contains an unexpected character in line 2 and column 19.
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,2 @@
# kind: evidence control-section key conflict rulings: empty
dm_ctl_sha16 chosen_pc class candidates records
1 # kind: evidence control-section key conflict rulings: empty
2 dm_ctl_sha16 chosen_pc class candidates records
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,3 +1,3 @@
// Generated file (dm/gfx/tools/dmsh_chain.py --header): the B shader library translation chain version this source tree expects (default options), one for every game;
// the host loads only <library dir>/b-<V>/. Run dmsh_chain.py --header after changing the chain's tools and commit this file with the change
#define DMSH_CHAIN "657c42552d82"
#define DMSH_CHAIN "dec5ee141a9f"
+5 -2
View File
@@ -187,6 +187,9 @@ NVN(nvnDeviceInstallDebugCallback, void, NVNdevice *d, const PFNNVNDEBUGCALLBACK
}
NVN(nvnDeviceSetWindowOriginMode, void, NVNdevice *d, NVNwindowOriginMode m) { ((dev_t_ *)d)->origin = m; mtl_set_origin_lower_left(m == NVN_WINDOW_ORIGIN_MODE_LOWER_LEFT); LOG("[nvn] window origin %s", m ? "upper left (as D3D / Metal)" : "lower left (GL)"); }
NVN(nvnDeviceSetDepthMode, void, NVNdevice *d, NVNdepthMode m) { ((dev_t_ *)d)->depth_mode = m; mtl_set_depth_zero(m != 0); LOG("[nvn] depth mode %s", m ? "near=0 (as D3D / Metal)" : "near=-w (GL)"); }
// Pokémon Sword sets neither mode and reads both, keeping NVN's defaults (lower-left origin, near = -w); the two were red stubs
NVN(nvnDeviceGetWindowOriginMode, NVNwindowOriginMode, const NVNdevice *d) { return ((const dev_t_ *)d)->origin; }
NVN(nvnDeviceGetDepthMode, NVNdepthMode, const NVNdevice *d) { return ((const dev_t_ *)d)->depth_mode; }
NVN(nvnDeviceFinalizeCommandHandle, void, NVNdevice *d, NVNcommandHandle h) { (void)d; (void)h; }
// ---------- Memory pools ----------
@@ -1417,8 +1420,8 @@ static void tsummary(uint64_t frame) {
LOG("[nvn] thread policy: main thread real-time computation %.2f ms / period %.2f ms, real-time policy %s, current priority %d (base %d, policy %d); %u worker threads QoS %#x; present QoS %#x", g_rt_comp_ms, g_rt_period_ms, rt_live ? "in effect" : "not in effect", okx ? xi.pth_curpri : -1, okx ? xi.pth_priority : -1, okx ? xi.pth_policy : -1, wn, wq, mtl_present_qos()); }
py = cy; pk = ck; pw = cw; }
pi = ci; pc = cc; pd = dr_; pf = frame; }
uint64_t up, re, fa, pr; mtl_pkg_stats(&up, &re); nvn_dirty_stats(&fa, &pr);
LOG("[nvn] packaged textures: first uploads %llu, re-uploads on pool content change %llu; write-protection faults %llu, pages unprotected before host writes %llu; missed %llu (cumulative)", up, re, fa, pr, mtl_pkg_miss());
uint64_t up, re, es = mtl_pkg_edge_skips(), fa, pr; mtl_pkg_stats(&up, &re); nvn_dirty_stats(&fa, &pr);
LOG("[nvn] packaged textures: first uploads %llu, re-uploads on pool content change %llu, writes beside a texture only (not re-uploaded) %llu; write-protection faults %llu, pages unprotected before host writes %llu; missed %llu (cumulative)", up, re, es, fa, pr, mtl_pkg_miss());
memset(g_tn, 0, sizeof g_tn);
}
// DM_SLOWFRAME=1 (test switch, read over the log port on a real device): per-frame timings are first stored in a ring of the last 16 frames; dropped frames are judged 8 frames late (GPU / on-screen present times are back by then),
+65 -6
View File
@@ -1293,7 +1293,7 @@ void mtl_drain(void) { present_drop_for_suspend(); mtl_present_flush(); mtl_init
// ================= Render replay =================
#include "../../host/front.h"
static int g_red_once[110];
static int g_red_once[111];
#define RED_ONCE(k, ...) do { if (!g_red_once[k]++) MLOG("[RED] " __VA_ARGS__); } while (0)
// NVN vertex streams 0–15 → Metal vertex buffer slots: 0–13 go to 16–29 (30 is the constant fallback buffer), 14 and 15 to 14 and 15. Metal has only the 31 slots 0–30; translated shaders use only buffer(0) (the argument buffer),
// and the host's own vertex functions read no vertex buffers. 13 Sentinels uses stream 15, and the old 16 + stream went out of range (Metal assertion: buffer index (31) must be < 31)
@@ -1430,16 +1430,45 @@ static void tex_put(uint64_t k, id o) { // serials from 2^62 up are external t
// Host record of a packaged texture: the current MTLTexture, the upload stamp, and the global write generation at the last check
// spare: the texture the last re-upload replaced, kept resident and written by the next re-upload once the GPU has passed spare_fence (a texture the CPU rewrites every frame, a movie frame,
// alternates between two textures instead of allocating one per frame: Link's Awakening's title movie made and freed an 8 MB texture a frame)
@interface NvPkg : NSObject { @public id<MTLTexture> tex, spare, view; uint64_t stamp, seen, hash, spare_fence, vgen, vseen; const void *span; int sparse, vbase; } @end // hash: full hash of the uploaded content under DM_NVN_DIRTYCHECK; span: its tracked range (nvn_dirty_span, NULL = untracked);
@interface NvPkg : NSObject { @public id<MTLTexture> tex, spare, view; uint64_t stamp, seen, hash, spare_fence, vgen, vseen; const void *span; int sparse, vbase; uint8_t *edge[2]; uint32_t edge_n[2]; } @end // hash: full hash of the uploaded content under DM_NVN_DIRTYCHECK; span: its tracked range (nvn_dirty_span, NULL = untracked);
// sparse: in a sparse virtual pool, read through its page table (hle_nvn.c nvn_sparse_*), vgen the page table's generation at the upload and vseen at the last look; view: what it is bound as (sparse_view)
// edge: the uploaded bytes of the storage's first and last host pages where other data shares them (pkg_edges_take)
@implementation NvPkg
- (void)dealloc { res_remove(tex); if (spare) res_remove(spare); }
- (void)dealloc { res_remove(tex); if (spare) res_remove(spare); free(edge[0]); free(edge[1]); }
@end
static uint64_t g_pkg_uploads, g_pkg_reuploads, g_pkg_miss;
static uint64_t g_pkg_uploads, g_pkg_reuploads, g_pkg_miss, g_pkg_edge_skips;
static uint64_t g_pfv_upgrades;
// DM_NVN_DIRTYCHECK=1 (diagnostic, compared against ground truth): fully hash the pool content on every use; changed but not detected by dirty-page tracking = a missed detection
static int dirtycheck(void) { static int v = -1; if (v < 0) v = getenv("DM_NVN_DIRTYCHECK") != NULL; return v; }
static uint64_t full_hash(const uint8_t *m, uint64_t n) { uint64_t h = 1469598103934665603ull; for (uint64_t i = 0; i < n; i++) h = (h ^ m[i]) * 1099511628211ull; return h; }
// Writes beside a texture. The tracker marks whole host pages (16 KB) dirty, and guest memory pools are only 4 KB aligned: the first and last pages of a texture's storage are shared with
// whatever lies beside it. Pokémon Sword keeps each texture of its GFPAK models in a pool of its own, with the pool's object 4 KB before the storage, in the same host page, and writes there
// every frame: 73,232 re-uploads in 70 s of play from a save, every one checked of unchanged content, 12 GB of upload staging in 8 minutes. So the bytes of the storage in those edge pages
// are kept at the upload (pkg_edges_take), and a texture whose only written pages are its edges, with those bytes unchanged, is protected again instead of re-uploaded (pkg_edges_clean).
// The edges are the whole storage when it lies within two pages (no page wholly inside)
static void pkg_edge_ranges(const tex_t *t, uintptr_t r[2][2], uintptr_t *ia, uintptr_t *ie) {
uintptr_t pg = (uintptr_t)getpagesize(), a = (uintptr_t)t->mem, e = a + t->L.size;
*ia = (a + pg - 1) & ~(pg - 1); *ie = e & ~(pg - 1);
if (*ie <= *ia) { r[0][0] = a; r[0][1] = e; r[1][0] = r[1][1] = 0; *ia = *ie = 0; return; }
r[0][0] = a; r[0][1] = *ia; r[1][0] = *ie; r[1][1] = e;
}
static void pkg_edges_take(NvPkg *pk, const tex_t *t) { // right after an upload, with its stamp in pk->stamp
for (int i = 0; i < 2; i++) { free(pk->edge[i]); pk->edge[i] = NULL; pk->edge_n[i] = 0; }
if (pk->sparse || !pk->span || !t->mem) return;
uintptr_t r[2][2], ia, ie; pkg_edge_ranges(t, r, &ia, &ie);
for (int i = 0; i < 2; i++) if (r[i][1] > r[i][0]) { pk->edge_n[i] = (uint32_t)(r[i][1] - r[i][0]); pk->edge[i] = malloc(pk->edge_n[i]); memcpy(pk->edge[i], (const void *)r[i][0], pk->edge_n[i]); }
// A write after the upload's stamp may have reached the copy but not the upload: then there is no copy to trust, and the next write re-uploads as before
if (nvn_dirty_since_span(pk->span, t->mem, t->L.size, pk->stamp)) for (int i = 0; i < 2; i++) { free(pk->edge[i]); pk->edge[i] = NULL; pk->edge_n[i] = 0; }
}
static int pkg_edges_clean(NvPkg *pk, const tex_t *t) { // the covered pages were written since the upload: 1 = only edge pages, and the texture's bytes there are as uploaded
if (!pk->edge_n[0] && !pk->edge_n[1]) return 0;
uintptr_t r[2][2], ia, ie; pkg_edge_ranges(t, r, &ia, &ie);
nvn_dirty_protect(t->mem, t->L.size); uint64_t st = nvn_dirty_now(); // protect, take the stamp, then read (as pkg_upload): a later write is seen at the next check
if (ie > ia && nvn_dirty_since_span(pk->span, (const void *)ia, ie - ia, pk->stamp)) return 0;
for (int i = 0; i < 2; i++) if (pk->edge_n[i] && memcmp(pk->edge[i], (const void *)r[i][0], pk->edge_n[i])) return 0;
pk->stamp = st; g_pkg_edge_skips++;
return 1;
}
static MTLTextureDescriptor *tex_desc(const tex_t *t, int packaged) {
uint32_t pf = 0; uint8_t swz[4];
if (!nvn_fmt_to_mtl(t->d.format, &pf, swz) || !pf) { RED_ONCE(0, "texture format %#x has no Metal equivalent (first case)", t->d.format); return nil; }
@@ -2001,6 +2030,7 @@ static id<MTLTexture> tex_mtl_(const tex_t *ct) {
MLOG("[RED] dirty-page miss: texture ser=%llu pool content changed but was not detected", t->serial);
return pk->view ? pk->view : pk->tex;
}
if (!pk->sparse && pkg_edges_clean(pk, t)) return pk->view ? pk->view : pk->tex; // only the pages it shares with other data were written
// Pool content changed (streamed assets written into the same storage, a movie frame): switch to another texture, as in-flight draws still read the old one. That is the spare
// when the GPU is done with it (see NvPkg), else a new one; the old one becomes the spare (pkg_upload_ made nt resident)
id<MTLTexture> reuse = nil;
@@ -2010,7 +2040,7 @@ static id<MTLTexture> tex_mtl_(const tex_t *ct) {
if (pk->spare && pk->spare != nt) defer_obj_release(pk->spare); // still in use, or of another shape: retired as before
if (pk->sparse) sparse_view(pk, t);
zb_report(t, pk->sparse ? (unsigned)pk->vbase : 0, "re-upload");
pk->spare = old; pk->spare_fence = g_fence + 1; g_ser_gen++; pk->stamp = st; pk->vgen = pk->vseen = vg; pk->span = nvn_dirty_span(t->mem, t->L.size); pk->seen = nvn_dirty_now(); g_pkg_reuploads++; if (dirtycheck() && !pk->sparse) pk->hash = full_hash(t->mem, t->L.size); }
pk->spare = old; pk->spare_fence = g_fence + 1; g_ser_gen++; pk->stamp = st; pk->vgen = pk->vseen = vg; pk->span = nvn_dirty_span(t->mem, t->L.size); pk->seen = nvn_dirty_now(); g_pkg_reuploads++; if (dirtycheck() && !pk->sparse) pk->hash = full_hash(t->mem, t->L.size); pkg_edges_take(pk, t); }
return pk->view ? pk->view : pk->tex;
}
pk = [NvPkg new];
@@ -2019,6 +2049,7 @@ static id<MTLTexture> tex_mtl_(const tex_t *ct) {
if (!pk->tex) return nil;
pk->span = nvn_dirty_span(t->mem, t->L.size);
pk->seen = nvn_dirty_now(); g_pkg_uploads++; if (dirtycheck() && !pk->sparse) pk->hash = full_hash(t->mem, t->L.size);
pkg_edges_take(pk, t);
if (pk->sparse) sparse_view(pk, t);
zb_report(t, pk->sparse ? (unsigned)pk->vbase : 0, "upload");
tex_put(t->serial, pk);
@@ -2110,6 +2141,7 @@ void mtl_mem_stat(unsigned long long frame, unsigned long long rss_mb) {
MLOG("GPU allocation growth by op: op%d %llu MB, op%d %llu MB, op%d %llu MB, op%d %llu MB", top[0], g_ap_bytes[top[0]] >> 20, top[1], g_ap_bytes[top[1]] >> 20, top[2], g_ap_bytes[top[2]] >> 20, top[3], g_ap_bytes[top[3]] >> 20); }
}
void mtl_pkg_stats(uint64_t *uploads, uint64_t *reuploads) { *uploads = g_pkg_uploads; *reuploads = g_pkg_reuploads; }
uint64_t mtl_pkg_edge_skips(void) { return g_pkg_edge_skips; }
uint64_t mtl_pkg_miss(void) { return g_pkg_miss; }
void mtl_vc_drop(uint64_t ser);
void mtl_tex_release(tex_t *t) {
@@ -2852,7 +2884,7 @@ static void shmap_load(void) {
warm_start();
}
@interface NvShader : NSObject { @public id<MTLFunction> fn, tf_fn, fn_u, geo_fn; // fn_u: the integer-output entry point (dmsh_build.py fs_uint), for pipelines whose colour attachments are all unsigned integer (pso_desc)
@interface NvShader : NSObject { @public id<MTLFunction> fn, tf_fn, fn_u, fn_m, geo_fn; uint8_t mask_m; // fn_u: the integer-output entry point (dmsh_build.py fs_uint), for pipelines whose colour attachments are all unsigned integer (pso_desc); fn_m: the mixed one, its outputs in mask_m unsigned integer
uint16_t tf_stride[4]; uint32_t cb_ext[16], sb_ro; nvn_refl rf; id<MTLArgumentEncoder> ae; NSUInteger arg_len; int stage, range_ro, afr, layer_out; const char *builtin; uint64_t logged; char key[24]; id<MTLBuffer> c1buf; uint64_t c1_gpu; size_t c1off;
uint32_t geo_vertices, geo_invocations, geo_output, geo_stride, geo_index_stride;
id<MTLComputePipelineState> geo_pso;
@@ -3038,10 +3070,13 @@ static NvShader *shader_load(NSString *pc, int stage, int layer) {
if (!o->tf_fn) MLOG("[RED] shader %s: no transform feedback entry point %s in its library", pc.UTF8String, [j[@"entry_tf"] UTF8String]);
}
if (stage == 1 && j[@"entry_u"]) { o->fn_u = [lib newFunctionWithName:j[@"entry_u"]]; if (!o->fn_u) MLOG("[RED] shader %s: no integer-output entry point %s in its library", pc.UTF8String, [j[@"entry_u"] UTF8String]); }
if (stage == 1 && j[@"entry_m"]) { o->fn_m = [lib newFunctionWithName:j[@"entry_m"]]; o->mask_m = (uint8_t)[j[@"uint_mask_m"] unsignedIntValue];
if (!o->fn_m) MLOG("[RED] shader %s: no mixed integer-output entry point %s in its library", pc.UTF8String, [j[@"entry_m"] UTF8String]); }
if (o->fn.functionConstantsDictionary.count) {
NSError *fe = nil; id<MTLFunction> f = [lib newFunctionWithName:entry constantValues:[MTLFunctionConstantValues new] error:&fe];
if (f) o->fn = f; else MLOG("[RED] shader %s: specializing its function constants failed: %s", pc.UTF8String, fe.localizedDescription.UTF8String);
if (o->fn_u) { f = [lib newFunctionWithName:j[@"entry_u"] constantValues:[MTLFunctionConstantValues new] error:&fe]; o->fn_u = f; }
if (o->fn_m) { f = [lib newFunctionWithName:j[@"entry_m"] constantValues:[MTLFunctionConstantValues new] error:&fe]; o->fn_m = f; }
}
if (o->rf.nb) {
o->ae = [o->fn newArgumentEncoderWithBufferIndex:o->rf.abi];
@@ -3415,6 +3450,15 @@ static id<MTLTexture> tex_by_id(int tid) { // Caller holds g_rmu
NSUInteger ns = base.textureType == MTLTextureType3D ? 1 : (r.v.set_layers && r.v.nlayers ? r.v.nlayers : base.arrayLength * (base.textureType == MTLTextureTypeCube || base.textureType == MTLTextureTypeCubeArray ? 6 : 1) - s0);
MTLTextureType tt = r.v.set_target ? ttype(r.v.target, base.textureType) : base.textureType;
if (tt == MTLTextureType2D && ns > 1) ns = 1;
// A cube view whose layer range is not whole cubes of faces counts its layers in cubes, as the hardware's texture header does for cube maps (Pokémon Sword's environment maps:
// 6-layer 2D array textures sampled through views with target CUBEMAP and layers 0 + 1, as the outdoors loads). Metal counts faces, and its validation aborts on a cube view of 1 slice
if ((tt == MTLTextureTypeCube || tt == MTLTextureTypeCubeArray) && ns % 6) {
NSUInteger total = base.textureType == MTLTextureType3D ? 1 : base.arrayLength * (base.textureType == MTLTextureTypeCube || base.textureType == MTLTextureTypeCubeArray ? 6 : 1);
int fits = (s0 + ns) * 6 <= total;
static int noted; if (fits && !noted++) MLOG("[nvn-mtl] cube texture view of %u + %u layers (texture ID %d, %lu faces): counted in cubes, as the hardware does", (unsigned)s0, (unsigned)ns, tid, (unsigned long)total);
if (!fits) RED_ONCE(110, "cube texture view of %u + %u layers past the %lu faces of texture ID %d: read as a 2D array", (unsigned)s0, (unsigned)ns, (unsigned long)total, tid);
if (fits) { s0 *= 6; ns *= 6; } else tt = MTLTextureType2DArray;
}
vc_key k; memset(&k, 0, sizeof k); k.pf = pf; k.tt = (uint8_t)tt; k.l0 = (uint32_t)l0; k.nl = (uint32_t)nl; k.s0 = (uint32_t)s0; k.ns = (uint32_t)ns; memcpy(k.sw, sw, 4);
if (!(v = vc_get(rp->t.serial, &k, base))) {
v = need_sw ? [base newTextureViewWithPixelFormat:(MTLPixelFormat)pf textureType:tt levels:NSMakeRange(l0, nl) slices:NSMakeRange(s0, ns) swizzle:MTLTextureSwizzleChannelsMake(sw[0], sw[1], sw[2], sw[3])]
@@ -4100,6 +4144,20 @@ static int pso_all_uint(const pso_key *k, NvShader *fs, int merged) {
return n > 0;
}
static int pf_int_kind(MTLPixelFormat f);
// Some written colour attachments are unsigned integer formats and some are not, and the integer ones are exactly the outputs the mixed entry point returns as integers (fn_m, mask_m:
// Pokémon Sword's G-buffer, colour 1 RGBA16Uint between two float targets)
static int pso_mixed_uint(const pso_key *k, NvShader *fs, int merged) {
int ints = 0, floats = 0;
for (int i = 0; i < 8; i++) {
if (!k->cfmt[i]) continue;
int l = merged ? k->lmap[i] : i;
if (l == 0xff || (fs->rf.color_known && !(fs->rf.color_mask >> l & 1))) continue; // an output the shader does not write
int u = mtl_fmt_uint((MTLPixelFormat)k->cfmt[i]);
if (u != (l < 8 && (fs->mask_m >> l & 1))) return 0;
if (u) ints++; else floats++;
}
return ints && floats;
}
static MTLRenderPipelineDescriptor *pso_desc(const pso_key *k, NvShader *vs, NvShader *fs, id<MTLFunction> fsfn, int merged) {
MTLRenderPipelineDescriptor *d = [MTLRenderPipelineDescriptor new];
d.vertexFunction = vs->fn; d.fragmentFunction = fsfn ? fsfn : fs ? fs->fn : nil;
@@ -4118,6 +4176,7 @@ static MTLRenderPipelineDescriptor *pso_desc(const pso_key *k, NvShader *vs, NvS
if (hl) d.fragmentFunction = [hl newFunctionWithName:[NSString stringWithFormat:@"hl%d", i]];
break; } }
if (fs && fs->fn_u && !fsfn && d.fragmentFunction == fs->fn && pso_all_uint(k, fs, merged)) d.fragmentFunction = fs->fn_u; // integer colour targets (dmsh_build.py fs_uint)
else if (fs && fs->fn_m && !fsfn && d.fragmentFunction == fs->fn && pso_mixed_uint(k, fs, merged)) d.fragmentFunction = fs->fn_m; // integer and float colour targets
if (vs->layer_out && k->fsfn != GEO_MARK) d.inputPrimitiveTopology = vs->geo_output == 1 ? MTLPrimitiveTopologyClassPoint : vs->geo_output == 6 ? MTLPrimitiveTopologyClassLine : MTLPrimitiveTopologyClassTriangle;
if (k->fsfn == TF_MARK || k->fsfn == GEO_MARK) { d.vertexFunction = k->fsfn == GEO_MARK ? vs->geo_fn : vs->tf_fn; d.fragmentFunction = nil; d.rasterizationEnabled = NO; } // transform feedback capture (tf_draw)
d.rasterSampleCount = k->samples ? k->samples : 1;
+1
View File
@@ -198,6 +198,7 @@ double mtl_presented_time(uint64_t n);
void mtl_tex_init(const tex_t *t); // TextureInitialize: the serial belongs to a live texture object until mtl_tex_release
void mtl_tex_release(tex_t *t);
void mtl_pkg_stats(uint64_t *uploads, uint64_t *reuploads);
uint64_t mtl_pkg_edge_skips(void); // packaged texture checks that found only writes beside the texture (pkg_edges_clean)
uint64_t mtl_pkg_miss(void);
uint64_t mtl_upload_ns_take(void); // Take and zero: time spent uploading packaged textures on the submit thread since last time // DM_NVN_DIRTYCHECK diagnostic: number of missed detections // Packaged textures: number of first uploads, number of re-uploads due to pool content changes
uint64_t mtl_upload_gpu_ns_take(void); // Take and zero: GPU execution time of upload command buffers (sum of GPUEndTime − GPUStartTime)
+28
View File
@@ -417,6 +417,34 @@ int main(void) {
mtl_pkg_stats(&up, &re); EXPECT("host write triggers a re-upload, 2 in total", 2, re - re0);
P(nvnMemoryPoolFinalize)(&wp);
}
// 6c1. Writes beside a texture (pkg_edges_clean): the texture starts 4 KB into a host page whose first 4 KB hold other data (Pokémon Sword: the pool's object). A write there dirties the
// page, but the texture's bytes are as uploaded: protected again, no re-upload; a write into the texture's own bytes in that page still re-uploads
{
static uint8_t eb_[256 << 10] __attribute__((aligned(16384)));
NVNmemoryPoolBuilder epb; NVNmemoryPool ep; P(nvnMemoryPoolBuilderSetDefaults)(&epb); P(nvnMemoryPoolBuilderSetStorage)(&epb, eb_, sizeof eb_);
P(nvnMemoryPoolInitialize)(&ep, &epb);
NVNtextureBuilder b; NVNtexture te; NVNpackagedTextureLayout lay = {{1, 0, 0, 0, 7, 0, 1, 0}};
P(nvnTextureBuilderSetDevice)(&b, &dev); P(nvnTextureBuilderSetDefaults)(&b); P(nvnTextureBuilderSetWidth)(&b, 16); P(nvnTextureBuilderSetHeight)(&b, 8);
P(nvnTextureBuilderSetFormat)(&b, NVN_FORMAT_RGBA8); P(nvnTextureBuilderSetPackagedTextureLayout)(&b, &lay);
P(nvnTextureBuilderSetStorage)(&b, &ep, 0x11000); P(nvnTextureInitialize)(&te, &b);
nvn_texdesc dd = ((tex_t *)&te)->d;
uint8_t lin[16 * 8 * 4], *store = eb_ + 0x11000, *outb = eb_ + 0x30000;
FILLPAT(5); nvn_swizzle(&dd, 0, 0, lin, 64, store);
NVNcopyRegion all = {0, 0, 0, 16, 8, 1};
#define GPU_READ_EQ_E(k, name) do { P(nvnCommandBufferBeginRecording)(&cb); P(nvnCommandBufferSetCopyRowStride)(&cb, 0); \
P(nvnCommandBufferCopyTextureToBuffer)(&cb, &te, NULL, &all, (NVNbufferAddress)outb, 0); NVNcommandHandle hh = P(nvnCommandBufferEndRecording)(&cb); \
P(nvnQueueSubmitCommands)(&q, 1, &hh); P(nvnQueueFinish)(&q); FILLPAT(k); EXPECT(name, 0, memcmp(outb, lin, sizeof lin)); } while (0)
uint64_t up0, re0, es0, up, re; mtl_pkg_stats(&up0, &re0); es0 = mtl_pkg_edge_skips();
GPU_READ_EQ_E(5, "texture sharing its host page: first upload = version 5");
for (int i = 0; i < 3; i++) { eb_[0x10800 + i] ^= 0x5a; GPU_READ_EQ_E(5, "a write beside the texture: still version 5"); }
mtl_pkg_stats(&up, &re);
EXPECT("writes beside the texture: no re-upload", 0, re - re0);
EXPECT("writes beside the texture: found clean three times", 3, mtl_pkg_edge_skips() - es0);
FILLPAT(6); nvn_swizzle(&dd, 0, 0, lin, 64, store);
GPU_READ_EQ_E(6, "a write into the texture's own bytes in that page: version 6");
mtl_pkg_stats(&up, &re); EXPECT("a write into the texture: re-upload +1", 1, re - re0);
P(nvnMemoryPoolFinalize)(&ep);
}
// 6c2. WriteTexels region writes: the result must be byte-identical to "decode the whole level → patch the region → re-encode" (Metroid Dread's minimap writes one texel at a time, which used to cost a whole-level round trip each).
// RGBA8 100×37 (width not a whole number of GOBs), BC1 64×64 (4×4 blocks, 8 bytes per block), RGBA16F 33×17 two-layer array; several regions each, including 1×1, a whole row, GOB-crossing and the whole level
+18 -10
View File
@@ -671,21 +671,29 @@ def fs_uint(src):
refuses a pipeline whose float output meets an integer attachment. Luigi's Mansion 3 draws its whole scene into an R32Uint + RGBA8Uint G-buffer (1,377 of its 4,307 pixel shaders write
an output from integer bits): a hand-written row per shader (shader_facts_manual.tsv) does not scale to that. A pixel shader that writes an output from integer bits gets a second entry
point, main0_u, the same body returning every colour output as the unsigned integer vector of the same bits (as_type, nothing converted); the host takes it for a pipeline whose colour
attachments are all unsigned integer formats (nvn_mtl.m). → the source with main0_u added, or None"""
attachments are all unsigned integer formats (nvn_mtl.m).
A G-buffer may mix the two: Pokémon Sword's scene shaders write colour 0 and 2 as floats and colour 1 from integer bits into an RGBA16Uint target. A shader whose outputs include both
gets a third entry point, main0_m, that returns only the outputs written from integer bits as unsigned integers (their colour indices: the mask); the host takes it for a pipeline whose
integer attachments are exactly those. → (the source with main0_u (and main0_m) added, the main0_m mask or 0), or (None, 0)"""
m = re.search(r'^.*\bfragment\s+main0_out\s+main0\s*\(', src, re.M); st = re.search(r'struct main0_out\s*\{(.*?)\};', src, re.S)
if not m or not st: return None
if not m or not st: return None, 0
mem = re.findall(r'^\s*(\w+)\s+(\w+)\s*(\[\[[^\]]*\]\])?;', st.group(1), re.M)
cols = {nm: ty for ty, nm, at in mem if 'color(' in at and ty.startswith('float')}
# The hardware writes an output's bits to an integer target whatever the shader computed, so main0_u is right for any pixel shader; only those that look like integer writers get one
# (the library stays small): an output from integer bits, or nothing but constants (a clear drawn into an integer target, 35e1865e313f0553 writes 0 into Luigi's Mansion 3's R32Uint target)
ints = any(re.search(r'\bout\.%s(?:\.\w+)? = as_type<float\d?>\(' % nm, src) for nm in cols)
from_int = {nm for nm in cols if re.search(r'\bout\.%s(?:\.\w+)? = as_type<float\d?>\(' % nm, src)}
writes = [m.group(1) for nm in cols for m in re.finditer(r'\bout\.%s(?:\.\w+)? = ([^;]+);' % nm, src)]
if not ints and not (writes and all(re.fullmatch(r'-?[0-9.]+f?|float\d\([-0-9., ]*\)', w.strip()) for w in writes)): return None
if not from_int and not (writes and all(re.fullmatch(r'-?[0-9.]+f?|float\d\([-0-9., ]*\)', w.strip()) for w in writes)): return None, 0
uty = lambda ty: 'u' + ty.replace('float', 'int') # float4 → uint4, float → uint
struct = 'struct main0_out_u\n{\n' + ''.join(f' {uty(cols[nm]) if nm in cols else ty} {nm} {at};\n' for ty, nm, at in mem) + '};\n\n'
ret = 'return main0_out_u{' + ', '.join(f'as_type<{uty(cols[nm])}>(out.{nm})' if nm in cols else f'out.{nm}' for _, nm, _ in mem) + '};'
body = re.sub(r'\bfragment\s+main0_out\s+main0\s*\(', 'fragment main0_out_u main0_u(', src[m.start():], count=1).replace('return out;', ret)
return src + '\n' + struct + body
def entry(sfx, as_uint): # main0 again as main0_<sfx>, returning the outputs in as_uint as the unsigned integer vectors of their bits (dmsh_fetch.py cuts the source at main0_out_u)
struct = f'struct main0_out_{sfx}\n{{\n' + ''.join(f' {uty(cols[nm]) if nm in as_uint else ty} {nm} {at};\n' for ty, nm, at in mem) + '};\n\n'
ret = f'return main0_out_{sfx}{{' + ', '.join(f'as_type<{uty(cols[nm])}>(out.{nm})' if nm in as_uint else f'out.{nm}' for _, nm, _ in mem) + '};'
return '\n' + struct + re.sub(r'\bfragment\s+main0_out\s+main0\s*\(', f'fragment main0_out_{sfx} main0_{sfx}(', src[m.start():], count=1).replace('return out;', ret)
out, mask = src + entry('u', set(cols)), 0
if from_int and set(cols) - from_int: # some outputs from integer bits, some not: the mixed entry point
out += entry('m', from_int)
mask = sum(1 << int(re.search(r'color\((\d+)\)', at).group(1)) for _, nm, at in mem if nm in from_int)
return out, mask
SG_OPS = {333: 'true', 334: 'pred', 335: 'pred', 336: 'true', 337: 'value', 338: 'value', 339: 'ballot', 345: 'value', 346: 'value', 347: 'value', 348: 'value'} # OpGroupNonUniform* → its one-lane value
SG_BUILTINS = {36: 'one', 41: 'zero', 4416: 'one4', 4417: 'one4', 4418: 'zero4', 4419: 'one4', 4420: 'zero4'} # SubgroupSize, SubgroupLocalInvocationId, Eq / Ge / Gt / Le / Lt masks
@@ -831,7 +839,7 @@ def finish(key, w, j, c1):
plain = full
full, lay = vs_layer_entry(full) # layer entry points for a slice-writing geometry shader (see vs_layer_entry)
open(msl, 'w').write(full)
fu = fs_uint(src) if j['stage'] == 'ps' else None # the integer-output entry point (see fs_uint); the analysis below uses the source with only main0
fu, fmask = fs_uint(src) if j['stage'] == 'ps' else (None, 0) # the integer-output entry points (see fs_uint); the analysis below uses the source with only main0
if fu: plain = src; open(msl, 'w').write(fu)
res = json.load(open(os.path.join(w, key + '.res.json')))
try: MTL.compile('mac', msl, lib) # MSL identical to one compiled before takes that library (dmsh_mtlcache.py)
@@ -849,7 +857,7 @@ def finish(key, w, j, c1):
refl = {'key': key + '_b', 'route': 'B', 'chain': CHAIN, 'dm_ctl': key, 'source': f'{name}#{k}', 'pc_paired': pc_of[key], 'stage': j['stage'],
'entry': entry.group(2) if entry else 'main0', 'msl_entry_kind': entry.group(1) if entry else None, **({'entry_z': 'main0_z', 'entry_geometry_capture': 'main0_gs'} if j['stage'] == 'vs' else {}),
**({'entry_tf': 'main0_tf', 'tf_strides': [e[2] if e else 0 for e in j['tf']]} if j.get('tf') else {}),
**({'entry_layer': 'main0_l', 'entry_layer_z': 'main0_lz', 'layer_from': lay} if lay else {}), **({'entry_u': 'main0_u'} if fu else {}),
**({'entry_layer': 'main0_l', 'entry_layer_z': 'main0_lz', 'layer_from': lay} if lay else {}), **({'entry_u': 'main0_u'} if fu else {}), **({'entry_m': 'main0_m', 'uint_mask_m': fmask} if fu and fmask else {}),
'threadgroup_size': j.get('local'), 'c1': c1.hex(), 'notes': res['notes'] + (['not in the answer table, queries use defaults'] if j.get('no_facts') else []), 'drv_reads': dr, 'drv_unfillable': unfillable(dr),
'bindings': reflect(key, j['stage'], src, res),
'vertex_inputs': [{'location': int(m.group(2)), 'msl_type': m.group(1)} for m in re.finditer(r'(\w+)\s+\w+\s*\[\[attribute\((\d+)\)\]\]', src)],
+61 -3
View File
@@ -28,6 +28,8 @@
# · data.arc (Super Smash Bros. Ultimate, smash_arc.py): GLSLC outputs in its SSBH shader databases and BNSH in its effects and UI layouts, each of its current files (zstd) scanned, see smash_arc_jobs
# · PKZ (PlatinumGames, Bayonetta 3): shader objects (VSO / PSO / CSO / HSO / DSO, a control section and a code area each) in the DAT archives of the PKZ entries (Oodle Kraken, decoded by the
# game's own decoder in the Mac host), see pkz_entries, pgdat_members and pg_shader
# · GFPAK (Game Freak, Pokémon Sword): BNSH entries of the archives under bin/archive (LZ4 entries), see gfpak_members
# · .tfxo (Silicon Studio's YEBIS post effects, Pokémon Sword): control sections each followed by its code area, in a zlib stream in main, see tfxo_streams and tfxo_shaders
# · control sections loose in main, in no GLSLC output or BNSH file (Advance Wars 1+2: Re-Boot Camp's Bink 2 shaders): paired with their code areas by data records or adjacency, see raw_shaders
# Code area: 0x30 header + (graphics) 0x50 SPH + instructions; header magic 0x12345678 for graphics, 0x12345679 for compute. The control section starts with 0x98761234, +0x704 = code area length,
# +0x700 = start of the c1 constant segment within the code area; for compute +0x748 thread group xyz (u32 ×3), +0x754 shared memory bytes (checked one by one against PC reflection on D2R's 22 compute shaders).
@@ -489,6 +491,57 @@ def bea_members(d, label):
if len(data) != usz: red(f'{label}/{path}: {len(data):#x} bytes, the entry says {usz:#x}')
yield path, data
TFXO = b'.tfxo file\0'
def tfxo_shaders(d, label):
"""YEBIS (Silicon Studio's post-effect library: bloom, glare, depth of field, SSAO …) keeps its compiled effects in one '.tfxo file' (Pokémon Sword 1.3.2: a 39.5 MB zlib stream in main,
3,155 shaders, see tfxo_streams): a text header (the file length, the shader count, 'hlsl', 'TexUtil_NVN_NX', the technique list 'tech_…,vs_…,ps_…'), then the shaders, each a control section
(its own length at +0x20: 0x850 on SDK 7) directly followed by its code area (control +0x704 bytes). The effect sets each stage as a program of its own (separable_pairs)"""
out = []
for m in re.finditer(re.escape(CTL_MAGIC), d):
a = m.start(); n = struct.unpack_from('<I', d, a + 0x20)[0]
if not 0x760 <= n <= 0x880: continue
ctl = d[a:a + n]; ncode = struct.unpack_from('<I', ctl, 0x704)[0]; code = d[a + n:a + n + ncode]
st = 'cs' if code[:4] == CS_MAGIC else sph_stage(code) if code[:4] == GFX_MAGIC else None
if not st: red(f'{label}: the control section at {a:#x} is not followed by a code area')
why = check(label, st, ctl, code)
if why: red(f'{label}: shader at {a:#x}: {why}')
out.append((f'{label}#TFXO@{a:#x}', st, ctl, code))
return out
def tfxo_streams(d, label):
"""YEBIS effect files linked into a module as zlib streams (Pokémon Sword: one, at main+0x1c98ed9 in 1.3.2, read through the engine's own inflate): a stream is taken when its first bytes
inflate to the '.tfxo file' header"""
import zlib
out = []
for m in ZLIB_HEAD.finditer(d):
try: head = zlib.decompressobj().decompress(d[m.start():m.start() + 4096], len(TFXO)) # a dynamic block's code tables come first
except zlib.error: continue
if head != TFXO: continue
z = zlib.decompressobj(); x = z.decompress(d[m.start():])
out += tfxo_shaders(x, f'{label}@{m.start():#x}')
return out
GFPAK = b'GFLXPACK'
def gfpak_members(d, label):
"""Game Freak's GFPAK archive (Pokémon Sword: 7,699 '.gfpak' files under bin/archive, 7.9 GB): 'GFLXPACK', u32 version 0x1000, u32 0, u32 file count, u32 folder count,
u64 offset of the file table, u64 offset of the file hashes (FNV-1a 64 of each path; no names are stored), u64 offset of the folder table pointers. The file table holds
0x18 bytes per file {u16 level, u16 compression, u32 length, u32 stored length, u32 0, u64 data offset}: compression 0 stored, 1 zlib, 2 LZ4 (one block), 3 Oodle Kraken (pkNX).
Every entry of the game's is LZ4. Inside: BNTX textures, BNSH shader archives (the models' and effects'), the models and animations. → (#index, data) of each entry"""
if d[8:12] != b'\0\x10\0\0': red(f'{label}: GFPAK version {d[8:12].hex()}, not 0x1000')
n, _, files = struct.unpack_from('<IIQ', d, 0x10)
for i in range(n):
ct, usz, csz, off = struct.unpack_from('<2xHII4xQ', d, files + 0x18 * i)
raw = d[off:off + csz]
if ct == 2:
import lz4.block
data = lz4.block.decompress(raw, uncompressed_size=usz)
elif ct == 1: data = __import__('zlib').decompress(raw)
elif ct == 0: data = raw
else: red(f'{label}: entry {i} has compression {ct} (Oodle), which this reader does not decode')
if len(data) != usz: red(f'{label}#{i}: {len(data):#x} bytes, the entry says {usz:#x}')
yield f'#{i}', data
NLG_DICT = bytes.fromhex('5824f3a9')
def nlg_dict(d):
"""Next Level Games' archive index (Luigi's Mansion 3: '<name>.dict' beside '<name>.data', every RomFS file but the Wwise banks): u32 magic 0xa9f32458, u8 4, u8 1, u8 compressed,
@@ -1094,6 +1147,7 @@ def scan_blob(d, label, out):
for nm, m in sarc_members(d): scan_blob(m, f'{label}/{nm}', out)
return
if d[:4] == b'BAHS': out.extend(bahs_shaders(d, label)); return
if d[:len(TFXO)] == TFXO: out.extend(tfxo_shaders(d, label)); return
if d[:4] == b'MSHD': out.extend(mshd_shaders(d, label)); return
if d[:4] == PGDAT and len(d) >= 0x1c and (mem := pgdat_members(d, label)) is not None: # PlatinumGames
for nm, m in mem: scan_blob(m, f'{label}/{nm}', out)
@@ -1115,10 +1169,10 @@ def ps_inputs(code):
"""Generic attributes a pixel shader reads: SPH words 6–13 are the input map, 8 bits for each of the 32 attributes (Ryujinx ShaderHeader.ImapTypes), non-zero = read; the translated MSL inputs are exactly user(locn<attribute>)"""
return {i for i in range(32) if struct.unpack_from('<I', code, 0x30 + 4 * (6 + (i >> 2)))[0] >> ((i & 3) * 8) & 0xff}
SINGLE_STAGE = ('#GLSLC@', '#UE', '#MTSPK@', '#MGFX@', '#NLG', '#PGSO', '#RAW@') # sources whose every shader is a program of one stage: GLSLC outputs, Unreal's shader records, MT Framework's shader package (BNSH variations, Unity programs, MSHD and SDF pair theirs);
SINGLE_STAGE = ('#GLSLC@', '#UE', '#MTSPK@', '#MGFX@', '#NLG', '#PGSO', '#RAW@', '#TFXO@') # sources whose every shader is a program of one stage: GLSLC outputs, Unreal's shader records, MT Framework's shader package (BNSH variations, Unity programs, MSHD and SDF pair theirs);
# MonoGame's effects, whose passes may name no vertex shader and keep the one bound (Celeste: 10 of its 13 effects draw through SpriteBatch's vertex shader, which is in main); Next Level Games'
# archives (Luigi's Mansion 3 makes 5,317 programs of one stage each at boot); PlatinumGames' shader objects (Bayonetta 3: one stage each, pg_shader); and control sections loose in main
# (Advance Wars 1+2: Re-Boot Camp's Bink shaders, raw_shaders), which no file pairs
# archives (Luigi's Mansion 3 makes 5,317 programs of one stage each at boot); PlatinumGames' shader objects (Bayonetta 3: one stage each, pg_shader); control sections loose in main
# (Advance Wars 1+2: Re-Boot Camp's Bink shaders, raw_shaders), which no file pairs; and YEBIS's effect file (tfxo_shaders)
def separable_pairs(keys):
"""Single-stage programs (13 Sentinels: each GLSLC output becomes a program with one stage, and BindProgram's stage bits combine vertex and pixel programs per draw; Unreal combines its shaders per draw
from the pipeline state) have their real pairs only at draw time. vs_pad uses only the union of the locations the paired pixel shaders read, so every single-stage vertex shader is paired with
@@ -1211,6 +1265,9 @@ def scan_file(args):
if head[:4] == SCNE: # a Bezel Engine archive (bea_members): each entry scanned as a file of its own (the shaders are BNSH in its .bfsha entries)
for nm, m in bea_members(open(path, 'rb').read(), '/' + rel): scan_blob(m, f'/{rel}/{nm}', found)
return rel, uniq_keys(found), var_pairs(found), trees
if head[:8] == GFPAK: # a Game Freak archive (gfpak_members): each entry scanned as a file of its own (the shaders are BNSH entries)
for nm, m in gfpak_members(open(path, 'rb').read(), '/' + rel): scan_blob(m, f'/{rel}{nm}', found)
return rel, uniq_keys(found), var_pairs(found), trees
d = unzs(path) if head[:4] == ZSTD_MAGIC else open(path, 'rb').read() # by the magic: Paper Mario: The Thousand-Year Door names its zstd files .zst (8,726, its shaders among them)
if d[:4] == b'Yaz0': d = yaz0(d)
if d[:8] == b'UnityFS\0' or (d[8:11] == b'\0\0\0' and __import__('unityfs').is_serialized(d)):
@@ -1261,6 +1318,7 @@ def main():
except ValueError: common = None # not a Unity game
found = []; scan_blob(img, 'main' + ('@' + tag if tag else ''), found)
found += glslc_shaders(img, 'main' + ('@' + tag if tag else ''), need=False)
found += tfxo_streams(img, 'main' + ('@' + tag if tag else '')) # YEBIS's effect file, a zlib stream in main (Pokémon Sword)
found += raw_shaders(img, 'main' + ('@' + tag if tag else ''), {c for _, _, c, _ in found})
# GLSLC outputs the game compiles at run time, captured on the Mac (hle_glslc.c): <version dir>/glslc_capture, or DM_GLSLC_CAPTURE. A game needs them when main holds GLSL sources and
# calls glslcCompile in a module it loads (Fire Emblem: Three Houses; profile_guess.py shader_sources). Captain Toad: Treasure Tracker imports it with agl's GLSL sources, but no module defines it
+32
View File
@@ -499,3 +499,35 @@ HLE_FORCE("_ZN2nn3hid23GetActualVibrationValueEPNS0_14VibrationValueERKNS0_21Vib
uint32_t targets = h ? vib_targets(*h) : 0;
*v = targets ? dm_haptics_actual(__builtin_ctz(targets), *h >> 16 & 1) : (dm_vibration){0, 160, 0, 320};
}
// Poké Ball Plus (Palma, Pokémon Sword / Shield, SDK 7.3.2): the host has none. The SDK's own calls reach the hid service (SetPalmaBoostMode, the first, at start-up: a null read in
// hid::detail::SetPalmaBoostMode, sdk+0x238eb0). No npad reports the Palma style, so the game never gets a connection handle; every call on a handle fails as for a handle that is not
// connected (hid 202-3302, ResultPalmaInvalidHandle in yuzu's hid errors), and the operation complete event is never bound
#define RES_PALMA_INVALID_HANDLE (202u | 3302u << 9)
HLE_FORCE("_ZN2nn3hid17SetPalmaBoostModeEb", "Poké Ball Plus boost mode; the host has no Poké Ball Plus", void, h_hid_PalmaBoost, unsigned on) { (void)on; }
HLE_FORCE("_ZN2nn3hid24EnableAnyPalmaConnectionEv", "Poké Ball Plus pairing; the host has none", Result, h_hid_PalmaAnyOn, void) { return RES_OK; }
HLE_FORCE("_ZN2nn3hid25DisableAnyPalmaConnectionEv", "Poké Ball Plus pairing; the host has none", Result, h_hid_PalmaAnyOff, void) { return RES_OK; }
HLE_FORCE("_ZN2nn3hid24GetPalmaConnectionHandleEPNS0_21PalmaConnectionHandleERKj", "Poké Ball Plus connection handle: none connected", Result, h_hid_PalmaHandle, uint64_t *h, const uint32_t *id) {
(void)id; if (h) *h = 0; return RES_PALMA_INVALID_HANDLE;
}
#define PALMA_FAIL(sym, what, fn, ...) HLE_FORCE(sym, "Poké Ball Plus " what ": fails, none connected", Result, fn, __VA_ARGS__) { return RES_PALMA_INVALID_HANDLE; }
PALMA_FAIL("_ZN2nn3hid15InitializePalmaERKNS0_21PalmaConnectionHandleE", "initialization", h_hid_PalmaInit, const void *h)
PALMA_FAIL("_ZN2nn3hid31BindPalmaOperationCompleteEventERKNS0_21PalmaConnectionHandleEPNS_2os15SystemEventTypeENS4_14EventClearModeE", "operation complete event", h_hid_PalmaBind, const void *h, void *ev, int mode)
PALMA_FAIL("_ZN2nn3hid21GetPalmaOperationInfoEPNS0_18PalmaOperationInfoERKNS0_21PalmaConnectionHandleE", "operation result", h_hid_PalmaOpInfo, void *info, const void *h)
PALMA_FAIL("_ZN2nn3hid17PlayPalmaActivityERKNS0_21PalmaConnectionHandleERKt", "activity", h_hid_PalmaPlay, const void *h, const uint16_t *i)
PALMA_FAIL("_ZN2nn3hid18SetPalmaFrModeTypeERKNS0_21PalmaConnectionHandleERKNS0_15PalmaFrModeTypeE", "FR mode", h_hid_PalmaFr, const void *h, const void *t)
PALMA_FAIL("_ZN2nn3hid13ReadPalmaStepERKNS0_21PalmaConnectionHandleE", "step count", h_hid_PalmaStepRead, const void *h)
PALMA_FAIL("_ZN2nn3hid15EnablePalmaStepERKNS0_21PalmaConnectionHandleEb", "step counting", h_hid_PalmaStepOn, const void *h, unsigned on)
PALMA_FAIL("_ZN2nn3hid14ResetPalmaStepERKNS0_21PalmaConnectionHandleE", "step count reset", h_hid_PalmaStepReset, const void *h)
PALMA_FAIL("_ZN2nn3hid27ReadPalmaApplicationSectionERKNS0_21PalmaConnectionHandleEim", "application section read", h_hid_PalmaAppRead, const void *h, int off, uint64_t n)
PALMA_FAIL("_ZN2nn3hid28WritePalmaApplicationSectionERKNS0_21PalmaConnectionHandleEimRKNS0_35PalmaApplicationSectionAccessBufferE", "application section write", h_hid_PalmaAppWrite, const void *h, int off, uint64_t n, const void *buf)
PALMA_FAIL("_ZN2nn3hid19ReadPalmaUniqueCodeERKNS0_21PalmaConnectionHandleE", "unique code", h_hid_PalmaCode, const void *h)
PALMA_FAIL("_ZN2nn3hid25SetPalmaUniqueCodeInvalidERKNS0_21PalmaConnectionHandleE", "unique code reset", h_hid_PalmaCodeOff, const void *h)
PALMA_FAIL("_ZN2nn3hid23WritePalmaActivityEntryERKNS0_21PalmaConnectionHandleERKtPKNS0_18PalmaActivityEntryE", "activity entry write", h_hid_PalmaActWrite, const void *h, const uint16_t *i, const void *e)
PALMA_FAIL("_ZN2nn3hid28WritePalmaRgbLedPatternEntryERKNS0_21PalmaConnectionHandleERKtPKvm", "LED pattern write", h_hid_PalmaLed, const void *h, const uint16_t *i, const void *p, uint64_t n)
PALMA_FAIL("_ZN2nn3hid19WritePalmaWaveEntryERKNS0_21PalmaConnectionHandleERKNS0_12PalmaWaveSetERKtPKvmm", "sound write", h_hid_PalmaWave, const void *h, const void *set, const uint16_t *i, const void *p, uint64_t n, uint64_t m)
PALMA_FAIL("_ZN2nn3hid37GetPalmaDataBaseIdentificationVersionERKNS0_21PalmaConnectionHandleE", "database version", h_hid_PalmaDbGet, const void *h)
PALMA_FAIL("_ZN2nn3hid37SetPalmaDataBaseIdentificationVersionERKNS0_21PalmaConnectionHandleEi", "database version", h_hid_PalmaDbSet, const void *h, int v)
PALMA_FAIL("_ZN2nn3hid19SuspendPalmaFeatureERKNS0_21PalmaConnectionHandleERKNS_4util10BitFlagSetILi32ENS0_12PalmaFeatureEEE", "feature suspension", h_hid_PalmaSuspend, const void *h, const void *f)
PALMA_FAIL("_ZN2nn3hid16ReadPalmaPlayLogERKNS0_21PalmaConnectionHandleERKt", "play log read", h_hid_PalmaLogRead, const void *h, const uint16_t *i)
PALMA_FAIL("_ZN2nn3hid17ResetPalmaPlayLogERKNS0_21PalmaConnectionHandleERKt", "play log reset", h_hid_PalmaLogReset, const void *h, const uint16_t *i)
+11
View File
@@ -144,6 +144,17 @@ HLE_FORCE("_ZN2nn3err40ShowUnacceptableAddOnContentVersionErrorEv", "add-on cont
HLE_FORCE("_ZN2nn3err39ShowUnacceptableApplicationVersionErrorEv", "game version mismatch error applet; the host only logs it", void, h_err_AppVer, void) { LOG("game asks to show an error: game version mismatch"); }
// Pokémon Legends: Z-A (SDK 19): the application's own error screen (an error code number and two messages in the argument) goes through the error applet as well
HLE_FORCE("_ZN2nn3err20ShowApplicationErrorERKNS0_19ApplicationErrorArgE", "application error applet; the host only logs it", void, h_err_ShowApp, const void *arg) { (void)arg; LOG("game asks to show an application error (ApplicationErrorArg)"); }
// Pokémon Sword (SDK 7.3.2): the error applet's network service form, after a failed online call
HLE_FORCE("_ZN2nn3err23ShowNetworkServiceErrorENS0_9ErrorCodeE", "network service error applet; the host only logs it", void, h_err_ShowNet, uint64_t code) { LOG("game asks to show a network service error: %04u-%04u", (unsigned)(code & 0xffffffff), (unsigned)(code >> 32)); }
// ---------- pdm: the system's play log ----------
// Pokémon Sword asks at boot how often this user has launched two other games (main+0xc88074, one ID each call, 0x18 bytes {application ID, play time, launch count} per entry; the result is
// not read, a launch count of at least 1 is): the host keeps no play log, so every one was never played. The SDK's own call opens the pdm service (null read at sdk+0x336ccc)
HLE_FORCE("_ZN2nn3pdm30QueryApplicationPlayStatisticsEPNS0_25ApplicationPlayStatisticsERKNS_7account3UidEPKNS_13ApplicationIdEi", "play statistics of other games: never played (the host keeps no play log)",
Result, h_pdm_QueryStats, uint64_t *out, const void *uid, const uint64_t *ids, int n) {
(void)uid; for (int i = 0; out && ids && i < n; i++) { out[3 * i] = ids[i]; out[3 * i + 1] = 0; out[3 * i + 2] = 0; }
return RES_OK;
}
// ---------- web: web page applet (online lobby). Unavailable ----------
HLE_FORCE("_ZN2nn3web13ShowLobbyPageEPNS0_20LobbyPageReturnValueERKNS0_16ShowLobbyPageArgE", "online lobby web applet (cut)", Result, h_web_Lobby, void *rv, const void *arg) { (void)rv; (void)arg; LOG("web::ShowLobbyPage (cut)"); return RES_FAIL; }
+1
View File
@@ -65,6 +65,7 @@ The work folder `$W` sits in the game's NSP folder, beside the NSP script's `<ti
- Unreal Engine 4.18 (DRAGON QUEST XI S, Square Enix) writes legacy paks, version 4: `JackGame/Content/Paks/pakchunk0-Switch.pak` (6.6 GB, 264,844 files) and `pakchunk1-Switch.pak` (7.2 GB, 147,824 files), and the update adds `*_0_P.pak` patch paks, which the engine mounts over the others (a path in a `_P` pak wins). The footer is the last 0x2d bytes (encrypted index flag, magic, version, index offset and size, SHA-1) and the index is not encrypted: the mount point, the file count, then each file's name and its FPakEntry (offset, stored and raw size, compression flags, SHA-1, the blocks' start and end, which are file offsets before version 5, the encrypted flag and the block size, 64 KB). The compression flags are no standard value: 3, which is LZ5 2.0 (LZ5 v2, later renamed Lizard; `main` exports `LZ5_*`, `LZ5_versionNumber` = 20001, and `FCompression::UncompressMemory` calls `LZ5_decompress_safe` for flags 3, `main+0x16f5c70`). Every block starts with its level, 0x2d = 45 (LIZv1 with Huffman). The repository has no LZ5 decoder: the game's own runs in the Mac host's tool mode (`hle_oodle.c`), which `uepak.py` (`Pak.read` with a decode function) and the shader split use. It takes 512 KB of work memory from the game's `malloc` on every call, so the host serves it only after `nninitStartup`. Scanning both paks (13.8 GB) with 8 workers takes 28 s.
- Capcom's MT Framework (The Great Ace Attorney Chronicles) keeps everything under `nativeNXx64/`: 447 ARC archives (`*.arc`, 17,910 files) beside loose `.tex` textures, Opus / ADPCM sound and H.264 `.mp4` movies. An ARC is `ARC\0`, u16 version 7, u16 file count, then one 0x90-byte entry per file: the path (128 bytes, `\`-separated, no extension), u32 type hash, u32 stored length, u32 length in the low 29 bits (flags above), u32 offset. Each file is one zlib stream (stored when the two lengths are equal), and its first bytes name the format (`TEX\0`, `MOD\0`, `MRL\0`, `GMD\0`, `SDL\0`, `SPK\0` the shader package, …). `shader_split.py` reads them (`mtarc_members`).
- NintendoWare Bezel Engine (Big Brain Academy: Brain vs. Brain) keeps nearly everything in Bezel Engine archives: 603 `.bea` files (`Archive/<Permanent|Temporary>~<name>.nx[.<language>].bea`, plus the engine's own `System.nx.bea` at the RomFS root), 7,665 files in all, 2.6 GB unpacked from 0.38 GB. A `.bea` is a NintendoWare binary file with magic `SCNE` (version 0x10100, BOM `FF FE`): +0x20 u64 entry count, +0x28 u64 offset of an array of u64 entry offsets, +0x30 the name dictionary (`_DIC`), +0x40 the archive name. Each entry is a 0x30-byte `ASST` record: +0x10 u16 kind (2; 258 for the one sound project), u16 alignment shift, +0x14 u32 stored length, +0x18 u32 length, +0x20 u64 data offset, +0x28 u64 name offset (u16 length, then the path). Every entry is one zstd frame, even the two whose stored length equals their length. Inside: layouts (`.lyt`, SARCs with their own `__ArchiveShader.bnsh`), BNTX textures, `.bfsha` shader archives, models (`.fmdb`), MSBT text and fonts; the sound is loose `stream/*.bfstm` and the movies loose `movie/Movie/*.mp4` (H.264). `shader_split.py` reads the archives itself (`bea_members`).
- Game Freak's engine before Trinity (Pokémon Sword 1.3.2, SDK 7.3.2) keeps its models, textures, effects and maps in 7,699 GFPAK archives under `bin/archive` (7.9 GB of the 12.1 GB RomFS), beside loose message tables, battle data, Wwise banks and BNSH files. A GFPAK is `GFLXPACK`, u32 version 0x1000, u32 0, u32 file count, u32 folder count, u64 offset of the file table, u64 offset of the file hashes (FNV-1a 64 of each path; no names are stored), u64 offset of the folder table pointers; the file table holds 0x18 bytes per file {u16 level, u16 compression, u32 length, u32 stored length, u32 0, u64 data offset}, compression 0 stored, 1 zlib, 2 LZ4 (one block), 3 Oodle Kraken (pkNX). All 201,635 entries of Sword's are LZ4, so the split needs no Oodle (`shader_split.py` `gfpak_members`). Inside: BNTX textures, BNSH shader archives, models (`.gfbmdl`) and animations.
- Game Freak's Trinity engine (Pokémon Legends: Z-A) keeps almost everything in one archive, `arc/data.trpfs` (4.4 GB, 13,181 packs, 238,546 files, 17.7 GB unpacked), indexed by `arc/data.trpfd`; both are FlatBuffers. The descriptor's root table holds the file hashes (u64, sorted), the pack names (`arc/<the first file's path without its slashes>.trpak`), one info table per file (its pack index) and one per pack (size, file count). The file system starts with `ONEPACK\0` and the u64 offset of its own root table: the pack name hashes and the pack offsets. The name hash is not a plain FNV-1a 64 of the descriptor's name, so a pack is named through its first file's hash. A pack is a table of file hashes and file tables `{1 compression i8, 2 level u8, 3 length u64, 4 data}`; compression 3 is Oodle (every file but 15,528 small stored ones), and every Oodle stream is Mermaid (header `8C 0A`, `CC 0A` for an uncompressed chunk). Oodle is linked into `main` and there is no Mermaid decoder with a licence fit for this repository, so the shader split runs the game's own `OodleLZ_Decompress` in the Mac host's tool mode (`dm/host/hle_oodle.c`, `DM_OODLE_SERVE=1`: requests on stdin, answers on stdout). The host finds it in `main` by the one function that references its name string (as for Unreal's archives), and calls it as the game does (`main+0x1a9b7bc` on 2.0.2: fuzz safe, no CRC check, thread phase 3). The shader split's full scan (every file decoded, 17.7 GB) takes 5 minutes with 12 workers on a 12-core Apple M6.
- The older KTGL of Fire Emblem: Three Houses uses a LINKDATA archive instead: `DATA0.bin` is an index of 32-byte entries `{u64 offset, u64 length, u64 stored length, u64 compressed}` into `DATA1.bin` (6.9 GB), and `patch1`–`patch4` hold loose overrides listed by `patchN/INFO0.bin` (0x120-byte entries ending in a `rom:/patchN/…` path). Its compressed entries and `*.gz` files are another chunked zlib: u32 chunk size (0x10000), u32 chunk count, u32 length, the stored length of each chunk, then the chunks at 0x80 alignment, each a u32 length + one zlib stream, or the plain bytes when the stored length equals the chunk's length (`kt_gz` in `shader_split.py`).
- **One archive of hashed names** (Super Smash Bros. Ultimate 13.0.5): the RomFS is `data.arc` (17.6 GB) and 14 NRRs. Its header (magic `0xABCDEF9876543210`) gives the offsets of the music, file and shared-file sections and of the file system table (one zstd frame of 50 MB); paths are Hash40s (CRC-32 of the path, its length in the top byte). A path leads to its current file through its info index, file info, info-to-data entry and file data, at file section + directory offset + 4 × its offset, zstd or stored (`dm/gfx/tools/smash_arc.py` documents the layout). The archive also keeps every earlier version of updated files, which no path leads to any more: 183,882 data blobs, 169,939 of them current (the 98 fighter NROs, 764 with their old versions, which the NRRs of older releases still list). A file's first bytes name its format: SSBH `HBSS` resources (`RDHS` shader databases, `XFUN` effect libraries, `LDOM` / `HSEM` / `LTAM` / `MINA` / `LEKS` models, meshes, materials, animations, skeletons), `EFFN` effects, `SARC` UI layouts, `BNTX`, `NUS3` sound and the NROs (`NRO0` at +0x10).
+1
View File
@@ -120,6 +120,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
- **Crash in `nn::ro::InitializeWithPortName`** → `nn::ro` connects to `ldr:ro`; take over `Initialize` and answer `LookupSymbol` from the loaded modules (`hle_ro.c`). Check the mangled names against the new title's imports: SDK 19's `LoadModule` takes one more bool (`_ZN2nn2ro10LoadModuleEPNS0_6ModuleEPKvPvmib`, Hades II), so the hook for the older form did not catch it and the SDK's own code would have run.
- **A jump to pc=0 right after the NVN bootstrap loader** (Super Smash Bros. Ultimate 13.0.5: `lr=main+0x37fe930`, `[RED] NVN address lookup of an unknown name: nvnSetExternalDebuggerEnabledNVX`) → the game fetches an NVX extension of the SDK 8 driver by name and calls it without checking the pointer. Unknown names return null, as the real driver does for a name it lacks; this one the driver has, so it joins `dm/gfx/nvn/names.txt` (thunks regenerated with `gen_protos.py`; no prototype in `nvn.h`, so `NVN_NOPROTO`) as a no-op.
- **The frame loop runs, but the game waits for code it loads itself** (Super Smash Bros. Ultimate 13.0.5: `ro::RegisterModuleInfo` / `ro::GetBufferSize` red lines; `main` imports 1,045 `lua2cpp::create_agent_*` functions, global, that no ExeFS module defines, bound to stubs at start) → the fighters' code is 98 NROs inside `data.arc`, registered by `.nrr/cross2_*.nrr`. The SDK's own `LoadModule` (SDK 8.2.1 `LoadModuleInternal`, `sdk+0x347460`) uses the service only to map the image, then zeroes bss, relocates through rtld's module lists, calls the module's init and binds the variables of the modules loaded before; the host replaces rtld, so it takes over the whole call. The NROs are collected at pack time by the SHA-256s the NRRs list (`dm/tools/nro_collect.py`: loose files, the current files of an indexed archive, or a scan for stored and zstd-framed NROs; Smash's NRRs of older releases also register 666 superseded copies, which `data.arc` still holds and which are left out), rewritten like any module (`patch_nso.py` reads NROs: the flat image, segments at +0x20, build ID at +0x40) and listed in `<patched dir>/nro.tsv`. The host maps them all at start and keeps them out of symbol lookups; `LoadModule` finds one by the image's SHA-256, restores its data and bss (after the first load), relocates it against the modules loaded then in load order, binds the earlier modules' symbol relocations it defines (main's `lua2cpp` stubs), fills `nn::ro::Module` as the SDK does (+0 module, +8 state 1, +0x10 image, +0x18 buffer) and calls its `DT_INIT`; `UnloadModule` calls `DT_FINI` and binds every relocation that pointed into it again (`loader.c` "NRO modules", `hle_ro.c`). `GetBufferSize` returns the bss size from header +0x38 as the SDK does. The startup init pass must skip NROs (their init ran unrelocated and crashed). Smash loads six NROs in its first 30 s and calls none of the stubs; only data is written at a load, so it works with code embedded in the iOS executable: `gen_embed.py` packs the NROs into three sections (`__dm_nro_x0`, `__DM_NRO_W`, `__dm_nro_x1`) and a table of their pieces (`__dm_nro_tab`), since a Mach-O file has at most 255 sections and three per NRO would pass it, and `map_image` maps an NRO from there (`loader.c` `nro_pieces`). Played on an iPhone.
- **SDK 7 calls of Pokémon Sword** (one boot each; the import list against the hooks and every other game's imports named all of them beforehand: 33 `nn::` imports no host hook names and no other game imports): the Poké Ball Plus (`hid::*Palma*`, 26 functions; the first boot crashed in `hid::detail::SetPalmaBoostMode`, a null read at `sdk+0x238eb0`, the hid service never opened) answers that none is connected: no npad has the Palma style, so the game never gets a connection handle, and every call on a handle fails (`hle_hid.c`); `pdm::QueryApplicationPlayStatistics` (the second boot: a null read at `sdk+0x336ccc`) asks how often this user launched two other games and reads only the launch count, so each is never played (`hle_offline.c`); `err::ShowNetworkServiceError` is logged. `nvnDeviceGetWindowOriginMode` and `nvnDeviceGetDepthMode` were red stubs: the game sets neither mode and reads both.
- **SDK 12 names** → older SDKs export other names for the same calls (`ListAudioDeviceName` / `GetActiveAudioDeviceName`, singular `hid::GetNpadState`, `OpenDefaultAudioOut`); hook both generations.
- **SDK 11 names** (Metroid Dread) → `fs::CommitSaveData(mount)` is the older name of `fs::Commit`; `fs::DeleteDirectory` (an empty directory) was missing too.
- **`fs::Commit(const char* const*, int)`, the mount-list overload** (Animal Crossing: New Horizons, PR #25) → the SDK's own code opened fsp-srv and aborted, leaving the save writes uncommitted. The host commits the list as one transaction when every name is the same host save; cache and temporary storage in the list are skipped (their writes are already on disk, as for the single-name `Commit`), and a list of two separate saves is refused without committing either.
+7
View File
@@ -449,3 +449,10 @@ Next Level Games' deferred renderer on SDK 7.3.2 (NVN 53): the scene draws into
- **Shader containers**: PlatinumGames' shader objects, 8,024 of them in `root_000.pkz` (`shader.dat`, `shader2.dat`, `shadereff*.dat`, `autoshadereff.dat`): `VSO\0` / `PSO\0` / `CSO\0` / `HSO\0` / `DSO\0` (the stage), u32 compiler version, u32 control length, u32 control offset (0x18), u32 code length, u32 code offset, and nothing else; 6,424 distinct keys (4,288 pixel, 2,047 vertex, 78 compute, 6 domain, 5 hull). The engine makes a program of each and binds them per stage at draw time, so they are single-stage programs, paired with a cover pixel shader (`shader_split.py` `pg_shader`, [01](01-extract.md) for the archives). It finds its built-in ones by name (`hwfillcolor.vso`, …) and checks each object's GPU code version against the driver's ([03](03-bringup.md), SDK 13.3.2).
- **Two domain shaders do not build** (`DS_Vanish05_*.dso`: `SPIRV-Cross threw an exception: Subgroup builtins are not available in this type of function`); the one-lane rewrite (`spv_single_lane`) covers vertex shaders only, and tessellated draws are skipped anyway.
- From the title to the prologue's first play the picture is right as far as compared (title scene, menus, the VP9 opening movie, the first real-time scene, play with the HUD). Two red lines, once each: a pixel shader declares a cube texture where the game binds a 2D one (the zero cube is bound), and a draw whose stage 0 reads cb1 with none bound.
### Seen in Pokémon Sword
- **Shader containers**: 41,516 shaders. Nearly all are BNSH entries of the 7,699 GFPAK archives under `bin/archive` (the models' and effects' shader archives, LZ4 entries, [01](01-extract.md); `shader_split.py` `gfpak_members`), beside loose BNSH files (`bin/battle/waza/model/**/*.bnsh_vsh` / `_fsh`, `bin/graphics`, `system_resource/shader`) and two in `main`. The provisional split, with no GFPAK reader, found 3,126.
- **Shaders in no file: YEBIS** (Silicon Studio's post-effect library: bloom, glare, depth of field, vignette; `C:/workspace/YEBIS_draft/src/GPU/…` in `main`). After the GFPAK reader, 38 shaders the opening drew were still missing from the map (`DM_NVN_SHDUMP`): no RomFS file and no module held their code, stored or LZ4, and none was a patched copy of a split shader (the nearest ones of the same length differ in hundreds of bytes). The game does not compile them at run time either (`main` imports `glslcCompile` from `subsdk1` but never calls it; it holds no GLSL source). A temporary trace in `nvnProgramSetShaders` (the key's code and control addresses, a backtrace and the heap around them) showed control sections packed back to back with technique names such as `ps_BrightPassT1_THRESHOLD_LUMINANCE_S1` beside them. The names are in `main`, and so is the data: the only zlib stream in `main` that inflates (`main+0x1c98ed9`, 39.5 MB) is YEBIS's `.tfxo file`, a text header (the shader count, `hlsl`, `TexUtil_NVN_NX`, the technique list) and 3,155 shaders, each a control section (its own length 0x850 on SDK 7) directly followed by its code area (`tfxo_streams`, `tfxo_shaders`; 3,015 distinct, 3,112 pixel and 43 vertex, single-stage programs). They covered all 38.
- **A G-buffer of float and integer targets** (`failed to create render pipeline … output of type float4 is not compatible with a MTLPixelFormatRGBA16Uint color attachment`, 10 pipelines as the opening's stadium loads): the scene shaders write colour 0 and 2 as floats and colour 1 from integer bits (`as_type<float>`) into an RGBA16Uint target. Luigi's Mansion 3's integer entry point (`main0_u`, every output an integer) does not fit a pipeline with a float target as well. `fs_uint` now also gives a shader whose outputs are of both kinds a third entry point, `main0_m`, that returns only the outputs written from integer bits as unsigned integers, and records their colour indices (`uint_mask_m` in its reflection, colour 1 in all 336 of Sword's); the host takes it for a pipeline whose integer attachments are exactly those (`pso_mixed_uint`).
- **Cube views of one layer** (an abort in Metal's validation as the player first walks out of the house: `newSliceRange.length(1) must be a multiple of (6)`) → the environment maps are 6-layer textures (target 2D array, depth 6) sampled through views with target CUBEMAP and layers 0 + 1. As in the hardware's texture header, a cube view counts its layers in cubes; Metal counts faces. A cube view whose layer count is not a multiple of 6 now takes its layers as cubes when the texture has that many faces (`tex_by_id`; else it is read as a 2D array with a red line).
- **Writes beside a texture** (73,232 re-uploads in 70 s from a save, 12 GB of upload staging in 8 minutes, frame times rising to 75 ms): the game keeps each texture of its models in a memory pool of its own, the pool's storage 4 KB after its own object, in the same 16 KB host page, and writes there every frame. The dirty-page tracker marks whole host pages, so every such texture counted as changed at each use; hashing showed every re-uploaded content unchanged. A packaged texture now keeps the bytes of its storage that lie in its first and last host pages when they are shared (`pkg_edges_take`); when only those pages were written and the bytes are as uploaded, it is protected again instead of re-uploaded (`pkg_edges_clean`, counted in the `packaged textures` line as `writes beside a texture only`). The same 70 s: 0 re-uploads and 72,584 such checks; `t_nvn` 6c1 covers it.
+9
View File
@@ -695,6 +695,15 @@ The game's own work and its frame rate stay as they are; the host avoids work of
- **Input:** the 3D pad layout (the map cursor moves with the D-pad).
- **Mac input:** the work folder's `run/step.sh <shot> <wait> <presses…>` drives a run started with `DM_CTL`. About 22 s into a boot the opening movie plays: A skips it, `L+R~0.5` opens the main menu (the first time Nell's introduction, A about 25 times, then the file select); A (Campaign), A, A (Advance Wars, the file), A, A, A, A (the mission, Field Training, Troop Orders) start the first battle.
**Pokémon Sword (`0100ABF008968000`)**
- **Status:** plays on the iPhone (IPA 12.4 GB) from update 1.3.2 (`v458752`). On the Mac: the first boot's language, photo and name choices, the opening (the Champion's broadcast, the stadium), Postwick (the house, the Bag, Hop's house), the walk to Wedgehurst and Leon's arrival at the station, the evening in Hop's yard, the starter choice (Scorbunny) and the first battle against Hop (Wooloo, then Grookey), the title screen and a save and load, at 30 fps as on the Switch (frame time p50 33.3 ms, p95 35–36 ms; GPU 4–9 ms a frame on an M6; CPU 42% of a core, 0.15 W), with sound (Wwise through a 6-channel `AudioOut`); resident memory 1.8 GB after the title, 2.4 GB after 17 minutes of new areas, 1.9 GB of Metal allocations. No red line on that route. The prewarm list covers it (3,273 lines).
- **Engine:** Game Freak's engine before Trinity, on NVN, SDK 7.3.2: the same `sdk` and movie module (`subsdk0`) builds as Super Mario Odyssey 1.4.1; `subsdk1` is GLSLC, which `main` imports but never calls. No virtual address memory: `nninitStartup` sets a 3.13 GB heap in the 4 GB layout, `mem_used` 76 MB for the rewritten modules.
- **Files:** 7,699 GFPAK archives (7.9 GB, every entry LZ4) under `bin/archive` beside loose message, battle and Wwise files ([01](agents/01-extract.md)).
- **Shaders:** 41,516: the BNSH entries of the GFPAK archives (`shader_split.py` `gfpak_members`), loose BNSH files, two in `main`, and Silicon Studio's YEBIS post effects, 3,155 shaders of a `.tfxo` file in a zlib stream in `main` (`tfxo_streams`, [04](agents/04-graphics.md)).
- **Host changes, all general:** the Poké Ball Plus (`hid::*Palma*`: none connected), `pdm::QueryApplicationPlayStatistics` (never played), `err::ShowNetworkServiceError`, `nvnDeviceGetWindowOriginMode` and `nvnDeviceGetDepthMode` ([03](agents/03-bringup.md)); a pixel shader entry point for G-buffers of float and integer targets, cube views counted in cubes, and textures whose shared edge pages are written but whose own bytes are not are no longer re-uploaded ([04](agents/04-graphics.md)).
- **Input:** the 3D pad layout (the camera is on the right stick).
- **Mac input:** the work folder's `run/ctl.sh` drives a run started with `DM_CTL`, `run/step.sh <shot> [wait] <presses…>` presses and takes a screenshot (headless: a window that loses focus is throttled and its shots never come), and `run/route_out.sh <run>` boots a copy of `run/save_ref` (in the bedroom with the Bag) through the title and out of the front door. From a fresh save, A at the language (about frame 600), the photo and the name (`DM_KBD`) reach the opening; `run/save_hop` is a save in Hop's kitchen, before the walk to Wedgehurst: from it, down 2.5 s out of the house, left 3.5 s, up 0.5 s, left 5 s, right 0.6 s and up 8 s reach the hill above Wedgehurst, and A presses with the stick up through the station scene lead back to Hop's yard and the starters (the camera turns with each move: walk in short steps onto the court and press A facing the Pokémon).
## Tools (`dm/tools`)
| Tool | What it does |
Binary file not shown.

After

Width:  |  Height:  |  Size: 216 KiB