Monster Train 2 plays on the iPhone: profile 010051701FB46000 (update 1.1.1, SDK 18.3.5)
Unity 2022.3.54f1 IL2CPP on Super Mario Galaxy's sdk build, with Ace Attorney Investigations Collection's memory layout (1 GB pool, 5 GB block). Three general changes made it play: - svcGetInfo type 24 (FreeThreadCount): the host answered 0, and Unity creates a thread only when nn::os::GetFreeThreadCount is at least 1, so its main thread waited at boot for a thread it never asked for. The host answers 96 (pm's application limit) less the game's live threads, as the kernel does. - Movie audio decoders keep 64 outputs (32 since DRAGON QUEST XI S): Unity's movie player starts only once it holds about half a second of audio, measured at 27 outputs with a peak of 30. Trace lines (DM_MOVIE_TRACE) name the decoder. - shader_split.py reads Unity ComputeShader objects (unity_compute), with Unity 2022.3.54f1's layout in unityfs.py: the post-processing stack's 85 kernels, the colour grading LUT baker among them, so the battle scene is no longer black. Files whose ComputeShader has no known tree are reported and skipped: Cuphead's and Engage's tables split byte-identical. Also nvnSyncSetDebugLabel (a no-op). The game reads no touch, so it stays docked with the 3D pad layout. README games table, AGENTS.md game list, development notes and the bring-up / graphics catalogues.
This commit is contained in:
seedds committed
1 parent
69d99b85e9
commit
708a1d73d5
22 files changed
+39485
-10
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Notes for agents
|
||||
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Thirty-two games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`) and DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`patch_nso.py` `FRAME_STEP`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout. Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK archives (`shader_split.py` `kpka_jobs`, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (`hle_nvn.c` `vpool_*`, a 4 KB page table the replay reads textures through); it runs at 60 fps (30 on the Switch). Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into `main`), renders with Vulkan, and generates its shaders at run time, which the NSP script captures on the Mac (no SPIR-V file in its RomFS); its pointer is the touch screen in handheld mode (profile key `op_mode`). Ace Attorney Investigations Collection is Unity 2022 IL2CPP: its shaders keep their parameters as blob entries of their own (`shader_split.py` `unity_code_blobs`), and its C# file API goes through the SDK's C library, whose path checks the host answers (`hle_fs.c` `IsPathMounted` and the length limits). Hades II, unlike Hades, renders with NVN; its shaders bind textures at fixed slots up to 34 (slot 34's handle lies in the driver's image table, `dmsh_build.py`), and the two long lines through Melinoë while the staff's Omega attack charges are the game's own aim lines, not a rendering bug. Unicorn Overlord is Vanillaware's engine like 13 Sentinels (shaders compiled into `main`); both need the profile key `alloc_big_align`, which aligns large allocations to 64 KB so the ring heaps the engine cuts from a memory pool fit. Rhythm Heaven Groove is Nintendo's own 2D engine on SDK 21, the newest SDK so far (its DSP command list has no magic, `hle_audren.c`); it plays handheld with Boost (profile keys `op_mode handheld`, `perf_mode boost`), which skips its TV latency calibration and keeps its 1920x1080 draw size. Octopath Traveler 0 is Unreal Engine 5: everything in one pak with an AES-encrypted index whose key is found Line truncated
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Thirty-three games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`) and Monster Train 2 (update 1.1.1, SDK 18.3.5, `010051701FB46000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`patch_nso.py` `FRAME_STEP`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout. Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK archives (`shader_split.py` `kpka_jobs`, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (`hle_nvn.c` `vpool_*`, a 4 KB page table the replay reads textures through); it runs at 60 fps (30 on the Switch). Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into `main`), renders with Vulkan, and generates its shaders at run time, which the NSP script captures on the Mac (no SPIR-V file in its RomFS); its pointer is the touch screen in handheld mode (profile key `op_mode`). Ace Attorney Investigations Collection is Unity 2022 IL2CPP: its shaders keep their parameters as blob entries of their own (`shader_split.py` `unity_code_blobs`), and its C# file API goes through the SDK's C library, whose path checks the host answers (`hle_fs.c` `IsPathMounted` and the length limits). Hades II, unlike Hades, renders with NVN; its shaders bind textures at fixed slots up to 34 (slot 34's handle lies in the driver's image table, `dmsh_build.py`), and the two long lines through Melinoë while the staff's Omega attack charges are the game's own aim lines, not a rendering bug. Unicorn Overlord is Vanillaware's engine like 13 Sentinels (shaders compiled into `main`); both need the profile key `alloc_big_align`, which aligns large allocations to 64 KB so the ring heaps the engine cuts from a memory pool fit. Rhythm Heaven Groove is Nintendo's own 2D engine on SDK 21, the newest SDK so far (its DSP command list has no magic, `hle_audren.c`); it plays handheld with Boost (profile keys `op_mode handheld`, `perf_mode boost`), which skips its TV latency calibration and keeps its 1920x1080 draw size. Octopath Traveler 0 is Unreal Engine 5: eveLine truncated
|
||||
- Each game's constants and shader tables form a game profile in `dm/games/<id>/`, where `<id>` is the base game's title ID: `game.tsv` plus the tables and `dmsh_chain.h`. Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)).
|
||||
- Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it.
|
||||
- Per-build knowledge is keyed by the NSO build ID: rewriter cuts in `dm/tools/patch_nso.py`, the profile's `build` rows (the builds the NSP script accepts, and their display versions), and host behaviour that uses fixed module addresses checks `module_t.build_id` (from `fixups.tsv`; `dm_main_build()`). When a new game build appears, carry addresses over with `dm/tools/nso_fnmap.py` and check them by disassembly; never let fixed addresses run on a build they were not found in.
|
||||
|
||||
@@ -42,6 +42,7 @@ Needs iOS / iPadOS 26 or later on an iPhone 15 Pro or newer (A17 Pro chip or lat
|
||||
| <br>The Great Ace Attorney Chronicles<br><sub>IPA ~4 GB</sub> | <br>STEINS;GATE RE:BOOT<br><sub>IPA ~6 GB</sub> |
|
||||
| <br>Big Brain Academy: Brain vs. Brain<br><sub>IPA ~0.8 GB</sub> | <br>Patrick's Parabox<br><sub>IPA ~0.2 GB</sub> |
|
||||
| <br>Celeste<br><sub>IPA ~0.9 GB</sub> | <br>DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition<br><sub>IPA ~15 GB · 60 fps (30 on Switch) with update 1.0.4</sub> |
|
||||
| <br>Monster Train 2<br><sub>IPA ~1.6 GB</sub> | |
|
||||
|
||||
|
||||
## Pack an IPA
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
# generated
|
||||
|
@@ -0,0 +1,5 @@
|
||||
# chain b-b4aac6f6275a 2026-10-22T10:07:06+08:00
|
||||
0652e521bacb65a2
|
||||
53c800e241e297d7
|
||||
a96f3e01b8c90fdb
|
||||
dde472609fc34051
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010051701FB46000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "b4aac6f6275a"
|
||||
@@ -0,0 +1,47 @@
|
||||
# Game profile of Monster Train 2 (Switch, update 1.1.1 = v131072). Plays on the Mac and on the iPhone
|
||||
# Format: see dm/games/README.md
|
||||
# Unity 2022.3.54f1 (IL2CPP) with FMOD, SDK 18.3.5 (its sdk is Super Mario Galaxy's build); subsdk0 is the movie module, which plays the intro, an MP4 of H.264 and AAC.
|
||||
# The shaders are Unity's Shader objects and the post-processing stack's ComputeShader objects in the player's serialized files, which carry no type trees; no bundle lends one, so
|
||||
# unityfs.py has Unity 2022.3's layouts written out (KNOWN_TREES)
|
||||
# Controller only (the game reads no touch, docked or handheld): it stays docked (1920x1080) with the 3D pad layout, whose right stick scrolls the route map as the game's prompt shows
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 010051701FB46000
|
||||
program_id 010051701FB46000
|
||||
|
||||
# ---- App ----
|
||||
# Game name: only a label for profile listings and the Mac window title; the iOS app name is the game name in the base NSP, and the bundle ID comes from the base game title ID (build_ipa.sh / NSP script)
|
||||
name Monster Train 2
|
||||
ipa MonsterTrain2.ipa
|
||||
app_category public.app-category.games
|
||||
|
||||
# ---- Versions ----
|
||||
build v131072 1.1.1 main=b337cdea2be300e638a0dfb74e9115a9 subsdk0=8a72a4dcbfd375d2952288284259da94 sdk=6075151627016eef437c1312f2e3ed56
|
||||
package v131072
|
||||
shader_versions v131072
|
||||
|
||||
# ---- Files ----
|
||||
# load order as on Horizon: main, subsdk*, sdk (subsdk0 = NintendoSDK_movie)
|
||||
modules main subsdk0 sdk
|
||||
romfs_marker Data/globalgamemanagers
|
||||
zip_store .assets:.bundle:.dat:.mp4:.resource:.ress:.metallib
|
||||
|
||||
# ---- Memory ----
|
||||
# The NPDM declares 16 MB of system resource, so IsVirtualAddressMemoryEnabled is true: nninitStartup hands application pool − used memory to StandardAllocator as virtual address memory
|
||||
# and never calls SetMemoryHeapSize. The node area cut for this sdk build (patch_nso.py, Super Mario Galaxy's) shrinks the AddressRegionManager's node area to 62 MB. With the hardware pool
|
||||
# (3.2 GB) StandardAllocator's region took the alias region to +3.9 GB in nninitStartup; the layout of Ace Attorney Investigations Collection: a 1 GB pool and the 5 GB block
|
||||
# (256 MB in front + 4.375 GB alias region + 256 MB stack region + 128 MB heap region). Measured on the Mac: the alias region mapped up to +3.25 GB within 90 s and flat for 12 minutes
|
||||
# (+3.5 GB in a run to the first shop), at most 1.75 GB mapped (the host does not hold the game to the pool, as for Ace Attorney)
|
||||
sys_resource 0x1000000
|
||||
mem_reserve 0x140000000
|
||||
mem_alias 0x118000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0x40000000
|
||||
|
||||
# ---- Input ----
|
||||
pad_layout full
|
||||
|
||||
# ---- Shader split (shader_split.py; Unity Shader and ComputeShader objects, read by unityfs.py) ----
|
||||
# everything is under Data/: the player's serialized files (the post-processing compute shaders in sharedassets2.assets) and the StreamingAssets bundles
|
||||
shader_groups /Data/=data
|
||||
shader_note Monster Train 2 shaders
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# Prewarm seed list (shipped in the app, next to shader_map.tsv). Rows: S<TAB>shader key<TAB>stage / R<TAB>render pipeline key (pso_skey hex) / C<TAB>compute shader key.
|
||||
# Provisional profile: empty until recorded on the Mac (DM_PSO_MANIFEST=off DM_PSO_RECORD).
|
||||
|
@@ -0,0 +1,87 @@
|
||||
# Answer table for translation queries (generated by dm/gfx/tools/shader_split.py, do not edit): key <TAB> fact JSON. This game has no PC version to pair with: only the thread group / shared memory of compute shaders (taken from the control section); other queries get the translator defaults
|
||||
0af3d084776a540b {"local":[8,8,1],"shared":1024}
|
||||
0f1d89c7a5faf4ed {"local":[32,32,1],"shared":0}
|
||||
0f3c5af0648a1ec7 {"local":[8,8,1],"shared":3072}
|
||||
1099d1baa5140160 {"local":[64,1,1],"shared":0}
|
||||
117bc099bcfda8d9 {"local":[32,32,1],"shared":0}
|
||||
14188ec78f42af26 {"local":[8,8,1],"shared":2048}
|
||||
15a57f83b9b92cba {"local":[4,4,4],"shared":0}
|
||||
16102ee21175782b {"local":[8,8,1],"shared":6144}
|
||||
1782c77964bc1ddb {"local":[8,8,1],"shared":3072}
|
||||
17b7b55ec095bf56 {"local":[64,1,1],"shared":0}
|
||||
1fc740810704c4a2 {"local":[32,32,1],"shared":0}
|
||||
20bcdeb2b537ea5a {"local":[64,1,1],"shared":0}
|
||||
2c95b66e27898f0d {"local":[4,4,4],"shared":0}
|
||||
2d00e1de52d3d62b {"local":[64,1,1],"shared":0}
|
||||
2e009e15c922bd25 {"local":[8,8,1],"shared":6144}
|
||||
3144c485dc3901bc {"local":[16,16,1],"shared":512}
|
||||
383e7771e93b6ac0 {"local":[64,1,1],"shared":0}
|
||||
3953b97d3dbe7338 {"local":[8,8,1],"shared":6144}
|
||||
3b9120b938b1d053 {"local":[16,16,1],"shared":0}
|
||||
3f7294e75a1bbd8c {"local":[64,1,1],"shared":0}
|
||||
49292df2841834d8 {"local":[8,8,1],"shared":3072}
|
||||
4ac0afb681dd0683 {"local":[8,8,1],"shared":3072}
|
||||
502e5475f5bde5dd {"local":[32,32,1],"shared":0}
|
||||
526c94d038c68531 {"local":[64,1,1],"shared":0}
|
||||
54e195a283381092 {"local":[32,32,1],"shared":0}
|
||||
552bbc8ffc2459e4 {"local":[4,4,4],"shared":0}
|
||||
586aeee231dbf739 {"local":[64,1,1],"shared":0}
|
||||
59497be38446830e {"local":[4,4,4],"shared":0}
|
||||
5cddd732f0ddecba {"local":[8,8,1],"shared":2048}
|
||||
60c45e31031a63b8 {"local":[8,8,1],"shared":2048}
|
||||
637ad928c7e23998 {"local":[64,1,1],"shared":0}
|
||||
6394b9a5bb9907b7 {"local":[32,32,1],"shared":0}
|
||||
685c1fa48fb2c590 {"local":[64,1,1],"shared":0}
|
||||
69628acd803ec8fa {"local":[16,8,1],"shared":512}
|
||||
6a9296f00c8d9a5c {"local":[8,8,1],"shared":0}
|
||||
6b70726d84a0095d {"local":[32,32,1],"shared":0}
|
||||
6cd245de21a205fd {"local":[32,32,1],"shared":0}
|
||||
6f72b5296833e077 {"local":[32,32,1],"shared":0}
|
||||
7314af6521af36e6 {"local":[8,8,1],"shared":1024}
|
||||
732454f3c0e95f6c {"local":[8,8,1],"shared":6144}
|
||||
775816ce2de58109 {"local":[8,8,1],"shared":0}
|
||||
7f4c568e2fbf4f3e {"local":[64,1,1],"shared":0}
|
||||
8039afbb0f1a592a {"local":[64,1,1],"shared":0}
|
||||
818009cfaeed1bd7 {"local":[4,4,4],"shared":0}
|
||||
854a8afc9d07f88d {"local":[4,4,4],"shared":0}
|
||||
8edfd5cd45de2766 {"local":[32,32,1],"shared":0}
|
||||
93f43b8bbab5866c {"local":[64,1,1],"shared":0}
|
||||
95b062f94ae020b4 {"local":[8,8,1],"shared":1024}
|
||||
9aeb3a1e8afec016 {"local":[8,8,1],"shared":6144}
|
||||
9cfa9625da68beab {"local":[16,1,1],"shared":0}
|
||||
9f0bc000ee8099fc {"local":[32,32,1],"shared":0}
|
||||
a6bcba3114c96fad {"local":[8,8,1],"shared":6144}
|
||||
a82b935b2caf3d69 {"local":[8,8,1],"shared":3072}
|
||||
a88747b15c335c3c {"local":[32,32,1],"shared":0}
|
||||
a97e534bbbe67be0 {"local":[16,16,1],"shared":8192}
|
||||
ac384270a2c329c0 {"local":[16,16,1],"shared":4096}
|
||||
ad53d877f02d0c9c {"local":[8,8,1],"shared":6144}
|
||||
b1d50b78f8e2c3e3 {"local":[16,16,1],"shared":0}
|
||||
b546afa0706ff48e {"local":[8,8,1],"shared":3072}
|
||||
b5a608eb874ad03b {"local":[4,4,4],"shared":0}
|
||||
bcda22a7cc7bb6ee {"local":[16,16,1],"shared":4096}
|
||||
bf0656de44715530 {"local":[64,1,1],"shared":0}
|
||||
c1400acb517e0bbf {"local":[8,8,1],"shared":3072}
|
||||
c1892e2f03e7bfa5 {"local":[8,8,1],"shared":3072}
|
||||
c208f16a5b3ab215 {"local":[64,1,1],"shared":0}
|
||||
c480704779340b3b {"local":[16,16,1],"shared":0}
|
||||
c73088133a436901 {"local":[256,1,1],"shared":0}
|
||||
c9e26ac3d6600fb1 {"local":[8,8,1],"shared":3072}
|
||||
d07db1ce7835a387 {"local":[1,256,1],"shared":0}
|
||||
d43ec310e91b3943 {"local":[64,1,1],"shared":0}
|
||||
d47cf2fb8860b9bd {"local":[32,32,1],"shared":0}
|
||||
d67fee2d51a5f254 {"local":[64,1,1],"shared":0}
|
||||
d76b66c887f5e515 {"local":[8,8,1],"shared":3072}
|
||||
db4e0c84f5499e8b {"local":[8,8,1],"shared":6144}
|
||||
dde472609fc34051 {"local":[16,16,1],"shared":1024}
|
||||
e08566d0f6afa9b3 {"local":[64,1,1],"shared":0}
|
||||
e0d9327b4955a880 {"local":[64,1,1],"shared":0}
|
||||
e19158b9bf5d018a {"local":[32,32,1],"shared":0}
|
||||
e8099874095d1632 {"local":[8,8,1],"shared":2048}
|
||||
e95c570c7fedfa73 {"local":[8,8,1],"shared":3072}
|
||||
ed93ab310cd4eb96 {"local":[8,8,1],"shared":3072}
|
||||
f23eb06b0ff087a1 {"local":[16,8,1],"shared":512}
|
||||
f5ed027ca034aee2 {"local":[32,32,1],"shared":0}
|
||||
f738da3a793713ba {"local":[32,32,1],"shared":0}
|
||||
f86819e5fcb0fa81 {"local":[16,16,1],"shared":8192}
|
||||
f97e8990530fce32 {"local":[16,1,1],"shared":0}
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 19.
|
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
# kind: evidence control-section key conflict rulings: empty
|
||||
dm_ctl_sha16 chosen_pc class candidates records
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -559,6 +559,7 @@ NVN(nvnProgramSetShaders, NVNboolean, NVNprogram *p, int n, const NVNshaderData
|
||||
// ---------- Sync / events ----------
|
||||
NVN(nvnSyncInitialize, NVNboolean, NVNsync *s, NVNdevice *d) { (void)d; *(sync_t *)s = (sync_t){MAGIC('S', 'Y', 'N', 'C'), 0, 0}; return 1; }
|
||||
NVN(nvnSyncFinalize, void, NVNsync *s) { ((sync_t *)s)->magic = 0; }
|
||||
NVN(nvnSyncSetDebugLabel, void, NVNsync *s, const char *l) { (void)s; (void)l; }
|
||||
// Fence = "all commands previously submitted to the queue have completed": record the fence value of the most recent submit
|
||||
NVN(nvnQueueFenceSync, void, NVNqueue *q, NVNsync *s, NVNsyncCondition c, int f) { (void)q; (void)c; (void)f; ((sync_t *)s)->fenced = 1; ((sync_t *)s)->fence = mtl_last_fence(); }
|
||||
// A sync object that was never fenced waits until timeout, as on a real device; a fenced one is released only after the GPU has actually completed up to that fence value
|
||||
|
||||
@@ -671,8 +671,27 @@ def unity_code_blobs(o):
|
||||
if not any('m_ParameterBlobIndices' in p for p in progs): return None
|
||||
return {sp['m_BlobIndex'] for p in progs for tier in p.get('m_PlayerSubPrograms', []) for sp in tier}
|
||||
|
||||
def unity_compute(x, base, trees):
|
||||
"""ComputeShader objects (class 72) of a serialized file → [(where, 'cs', control, code)]: each kernel variant's code is a program code with one stage, compute (Monster Train 2,
|
||||
Unity 2022.3: the post-processing stack's 12 compute shaders, its colour grading LUT baker among them, in Data/sharedassets2.assets). Raises NoTree in the first pass, so the file is scanned
|
||||
again with the borrowed trees and unityfs.py's layouts; in the second pass a class still without a tree is reported and its kernels left out, as before compute shaders were read"""
|
||||
import unityfs
|
||||
out = []
|
||||
try:
|
||||
for cid, pid, o in unityfs.serialized_objects(x, COMMON, {unityfs.CLASS_COMPUTE_SHADER}, trees):
|
||||
for v in o['variants']:
|
||||
for k in v['kernels']:
|
||||
for i, u in enumerate(k['uniqueVariants']):
|
||||
where = f'{base}#{o["m_Name"]}#{k["name"]}.{i}'
|
||||
out.extend((where, st, ctl, cd) for st, ctl, cd in unity_program(u['code'], where, 'cs'))
|
||||
except unityfs.NoTree as e:
|
||||
if not TREES: raise
|
||||
print(f'[RED] {base}: ComputeShader objects that no bundle lends a type tree to and unityfs.py has no layout for, their kernels are left out ({e})', flush=True)
|
||||
return []
|
||||
return out
|
||||
|
||||
def unity_shaders(d, label, out, trees):
|
||||
"""Shaders in a UnityFS bundle or serialized file → out; raises NoTree when no type tree is available"""
|
||||
"""Shaders and compute shaders in a UnityFS bundle or serialized file → out; raises NoTree when no type tree is available"""
|
||||
import unityfs
|
||||
if COMMON is None: red(f'{label}: a Unity asset, but main has no Unity common string table')
|
||||
sers = unityfs.bundle_files(d) if d[:8] == b'UnityFS\0' else [('', d)]
|
||||
@@ -685,6 +704,7 @@ def unity_shaders(d, label, out, trees):
|
||||
where = f'{label}{":" + node if node else ""}#{name}#var{plat}.{k}'
|
||||
code = unityfs.subprogram(sp)[4]
|
||||
out.extend((where, st, ctl, cd) for st, ctl, cd in unity_program(code, where, stages.get(k)))
|
||||
out.extend(unity_compute(x, f'{label}{":" + node if node else ""}', trees))
|
||||
|
||||
def scan_blob(d, label, out):
|
||||
if d[:4] == b'Yaz0': d = yaz0(d)
|
||||
|
||||
+14
-1
@@ -162,8 +162,21 @@ PPtr<Shader>: int m_FileID; SInt64 m_PathID
|
||||
PPtr<Texture>: int m_FileID; SInt64 m_PathID
|
||||
Shader: string m_Name; SerializedShader m_ParsedForm; vector<data:unsigned int> platforms; vector<data:vector<data:unsigned int>> offsets; vector<data:vector<data:unsigned int>> compressedLengths; vector<data:vector<data:unsigned int>> decompressedLengths; vector<data:UInt8> compressedBlob; vector<data:PPtr<Shader>> m_Dependencies; map<data:pair(string,PPtr<Texture>)>~ m_NonModifiableTextures; bool! m_ShaderIsBaked
|
||||
'''
|
||||
# The Unity 2022.3 ComputeShader (Monster Train 2, Unity 2022.3.54f1: 12 in Data/sharedassets2.assets, which carries no type trees, and no bundle has one), from Unity 2022.3.54f1's type tree
|
||||
# as AssetRipper's TypeTreeDumps records it (StructsDump/release/2022.3.54f1.dump). Each kernel variant's code is a program code of one stage, compute (the layout of a Shader's subprogram code)
|
||||
COMPUTE_2022_3 = '''ComputeShaderResource: string name; string generatedName; int bindPoint; int samplerBindPoint; int texDimension
|
||||
ComputeShaderBuiltinSampler: unsigned int sampler; int bindPoint
|
||||
ComputeShaderKernel: vector<data:unsigned int> cbVariantIndices; vector<data:ComputeShaderResource> cbs; vector<data:ComputeShaderResource> textures; vector<data:ComputeShaderBuiltinSampler> builtinSamplers; vector<data:ComputeShaderResource> inBuffers; vector<data:ComputeShaderResource> outBuffers; vector<data:UInt8> code; vector<data:unsigned int>~ threadGroupSize; SInt64 requirements
|
||||
ComputeShaderKernelParent: string name; vector<data:ComputeShaderKernel> uniqueVariants; vector<data:pair(string,unsigned int)> variantIndices; vector<data:string> globalKeywords; vector<data:string> localKeywords; vector<data:string> dynamicKeywords
|
||||
ComputeShaderParam: string name; int type; unsigned int offset; unsigned int arraySize; unsigned int rowCount; unsigned int colCount
|
||||
ComputeShaderCB: string name; int byteSize; vector<data:ComputeShaderParam> params
|
||||
ComputeShaderPlatformVariant: int targetRenderer; int targetLevel; vector<data:ComputeShaderKernelParent> kernels; vector<data:ComputeShaderCB> constantBuffers; bool! resourcesResolved
|
||||
ComputeShader: string m_Name; vector<data:ComputeShaderPlatformVariant> variants
|
||||
'''
|
||||
CLASS_COMPUTE_SHADER = 72
|
||||
KNOWN_TREES = {(CLASS_SHADER, bytes.fromhex('ea424a5fc1e48db62144df956f9e1005')): ('Shader', SHADER_2022_3),
|
||||
(CLASS_SHADER, bytes.fromhex('a30e3dcf6c6d25153d6f64e5e41caf4e')): ('Shader', SHADER_2020_3)}
|
||||
(CLASS_SHADER, bytes.fromhex('a30e3dcf6c6d25153d6f64e5e41caf4e')): ('Shader', SHADER_2020_3),
|
||||
(CLASS_COMPUTE_SHADER, bytes.fromhex('bfdf63ab0ce84d5a2c765ee560d90fe7')): ('ComputeShader', COMPUTE_2022_3)}
|
||||
|
||||
def _split_top(s, sep): # s split at sep outside <> and ()
|
||||
out, depth, cur = [], 0, ''
|
||||
|
||||
+8
-5
@@ -44,7 +44,7 @@ HLE_FORCE("_ZN5movie9Extractor13SetDataSourceEPKc", "movie demuxer: the host doe
|
||||
|
||||
#define MD_NIN 4 // input buffers
|
||||
#define MD_INCAP (4u << 20) // 4 MB per input buffer (a 1080p VP9 packet is far smaller)
|
||||
#define MD_NOUT 32 // output slots, at most (an audio decoder's count)
|
||||
#define MD_NOUT 64 // output slots, at most (an audio decoder's count)
|
||||
#define MD_NOUT_VIDEO 8 // a video decoder's output slots
|
||||
#define MD_NO_INPUT 0x1011 // SDK: no free input
|
||||
#define MD_NO_OUTPUT 0x100e // SDK: no output ready
|
||||
@@ -185,9 +185,9 @@ static void movie_ring_wait(const uint64_t *regs) {
|
||||
uint64_t dt = now_ns() - t0; static _Atomic int nlong; if (dt > 50000000ull && nlong++ < 8) LOG("[movie] waited %.1f ms on the player request ring", dt / 1e6);
|
||||
}
|
||||
__attribute__((constructor)) static void movie_ring_gates(void) { gate_obs_set(OBS_MOVIE_RING_LINK, movie_ring_wait); gate_obs_set(OBS_MOVIE_RING_UNLINK, movie_ring_wait); }
|
||||
// log the first 12 calls one by one, then every 2^k-th (DM_MOVIE_TRACE=1: every call, with the calling thread)
|
||||
// log the first 12 calls one by one, then every 2^k-th (DM_MOVIE_TRACE=1: every call, with the calling thread and the decoder: Unity drives a movie's audio and video decoders from one thread)
|
||||
static int md_trace_all(void) { static int on = -1; if (on < 0) on = getenv("DM_MOVIE_TRACE") != NULL; return on; }
|
||||
#define MD_TRACE(d, slot, fmt, ...) do { unsigned long n_ = ++(d)->ncall[slot]; if (md_trace_all()) LOG("[movie] [%p] " fmt, (void *)pthread_self(), ##__VA_ARGS__); \
|
||||
#define MD_TRACE(d, slot, fmt, ...) do { unsigned long n_ = ++(d)->ncall[slot]; if (md_trace_all()) LOG("[movie] [%p] [%p] " fmt, (void *)pthread_self(), (void *)(d), ##__VA_ARGS__); \
|
||||
else if (n_ <= 12 || !(n_ & (n_ - 1))) LOG("[movie] " fmt, ##__VA_ARGS__); } while (0)
|
||||
|
||||
// VP9 key frame uncompressed header → width and height (VP9 spec 6.2: frame_marker, profile, show_existing_frame, frame_type, show_frame, error_resilient_mode, sync code, color config, frame_size)
|
||||
@@ -474,10 +474,13 @@ HLE_FORCE("_ZN5movie7DecoderC1ENS_11DecoderTypeENS_19DecoderOutputFormatENS_11De
|
||||
mdec_t *d = calloc(1, sizeof *d);
|
||||
pthread_mutex_init(&d->mu, NULL); pthread_mutex_init(&d->dmu, NULL); pthread_mutex_init(&d->wmu, NULL); pthread_cond_init(&d->wcv, NULL);
|
||||
d->type = type; d->fmt = fmt; d->mode = mode;
|
||||
// An audio decoder keeps 32 outputs (about 0.7 s of Vorbis), a video decoder 8. The host decodes far faster than the Switch, and a player that feeds its decoders in file order
|
||||
// An audio decoder keeps 64 outputs (about 1.4 s of AAC or Vorbis), a video decoder 8. The host decodes far faster than the Switch, and a player that feeds its decoders in file order
|
||||
// stops at the first track with no free input. DRAGON QUEST XI S's player fills every track before it plays (IsPreDecodingCompleted: one output queued on each, polled once a frame),
|
||||
// while its render thread's first fetch takes every decoded picture and shows the newest. With 8 audio outputs, both audio tracks were full before 240 ms of video was decoded: the
|
||||
// extractor waited for an audio input, no picture followed the ones that fetch took, and the opening movie never started (in about half the boots on the Mac: whichever of the poll and the fetch came first)
|
||||
// extractor waited for an audio input, no picture followed the ones that fetch took, and the opening movie never started (in about half the boots on the Mac: whichever of the poll and the fetch came first).
|
||||
// Unity 2022.3's movie player (Monster Train 2's intro, an MP4 of H.264 and AAC interleaved packet by packet) takes every output it is offered and starts playing once it holds about
|
||||
// half a second of audio: measured with 64 audio and 8 video slots, it starts holding 27 audio and 8 video outputs and peaks at 30 audio, close to 32 (8 audio slots hang it on a black
|
||||
// screen whatever the video count: 32 video slots did not help). An audio slot is a PCM buffer allocated when first used (MD_PCM_CAP, 128 KB), so 64 cost at most 8 MB
|
||||
d->nout = audio ? MD_NOUT : MD_NOUT_VIDEO;
|
||||
for (int i = 0; i < MD_NIN; i++) { d->in[i] = malloc(MD_INCAP); d->infree[i] = i; }
|
||||
d->ninfree = MD_NIN;
|
||||
|
||||
+5
-1
@@ -63,7 +63,7 @@ void svc_guest_range(uint8_t **base, size_t *size) { *base = g_as; *size = RESER
|
||||
enum { I_CoreMask=0, I_PriorityMask=1, I_AliasAddr=2, I_AliasSize=3, I_HeapAddr=4, I_HeapSize=5,
|
||||
I_TotalMem=6, I_UsedMem=7, I_DebuggerAttached=8, I_ResourceLimit=9, I_IdleTick=10, I_RandomEntropy=11,
|
||||
I_AslrAddr=12, I_AslrSize=13, I_StackAddr=14, I_StackSize=15, I_SysResTotal=16, I_SysResUsed=17,
|
||||
I_ProgramId=18, I_InitProcIdRange=19, I_UserExcAddr=20, I_TotalNonSys=21, I_UsedNonSys=22, I_IsApplication=23 };
|
||||
I_ProgramId=18, I_InitProcIdRange=19, I_UserExcAddr=20, I_TotalNonSys=21, I_UsedNonSys=22, I_IsApplication=23, I_FreeThreadCount=24 };
|
||||
|
||||
static uint8_t g_seen[256];
|
||||
|
||||
@@ -295,6 +295,10 @@ uint64_t svc_dispatch(int num, uint64_t *regs, uint64_t orig_addr) {
|
||||
case I_RandomEntropy: { uint64_t dm_entropy64(void); v = dm_entropy64(); break; } // a deterministic stream when DM_DETCLOCK is on (hle_os.c)
|
||||
case I_ProgramId: { static uint64_t pid; if (!pid) { const char *e = getenv("DM_PROGRAM_ID"); pid = e ? strtoull(e, NULL, 16) : DM_GAME_PROGRAM_ID; } v = pid; break; } // default from the game profile's program_id; DM_PROGRAM_ID=hex overrides
|
||||
case I_IsApplication: v = 1; break;
|
||||
// nn::os::GetFreeThreadCount (SDK 11+): the thread resource limit less the threads alive, as the kernel answers it (Atmosphère kern_svc_info.cpp). pm gives an application a limit of
|
||||
// 96 threads (pm_spec.cpp BaseApplicationThreads; a boost raises it to 256). Monster Train 2's Unity 2022.3 player creates a thread only when this is at least 1: the 0 it got here
|
||||
// left its main thread waiting at boot for a thread it never created (main+0x3e1ac4)
|
||||
case I_FreeThreadCount: { int thr_limit_used(void); int u = thr_limit_used(); v = u < 96 ? 96 - u : 0; break; }
|
||||
case I_UserExcAddr: { static _Alignas(0x1000) uint8_t ctx[0x1000]; v = (uint64_t)ctx; break; } // user exception context area: newer SDKs' SetDyingMessageRegion writes to +0x1c0 (0 would write through a null pointer)
|
||||
default: v = 0; break;
|
||||
}
|
||||
|
||||
@@ -417,6 +417,15 @@ void thr_counts(int *handles, int *live, int *objs) {
|
||||
pthread_mutex_unlock(&G);
|
||||
*handles = nh; *live = nl; *objs = no;
|
||||
}
|
||||
// The game's threads that hold a slot of the process's thread resource limit, as Horizon counts them: created (started or not) and not yet exited (GetInfo FreeThreadCount, hle_svc.c).
|
||||
// Guest threads the host spawns for itself are not the game's
|
||||
int thr_limit_used(void) {
|
||||
int n = 0;
|
||||
pthread_mutex_lock(&G);
|
||||
for (thr_t *t = g_all; t; t = t->all) n += !t->exited && !t->host_owned;
|
||||
pthread_mutex_unlock(&G);
|
||||
return n;
|
||||
}
|
||||
// Get the guest thread name by pthread (used by per-thread CPU sampling DM_THRCPU, memstat.m): returns 0 if not a guest thread
|
||||
int thr_name_of(pthread_t pt, char *out, size_t n) {
|
||||
int ok = 0; char nb[64];
|
||||
|
||||
@@ -77,6 +77,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
### Threads
|
||||
|
||||
- **Per-core job queues never drained** → a `SetThreadCoreMask` that pins a thread to other cores (ideal core −3 = keep) must move the ideal core into the new mask, as Horizon does, so `GetCurrentProcessorNumber` reports a core the thread can run on.
|
||||
- **The main thread waits forever on an event right after `nnMain` starts, with only three or four threads created** (Monster Train 2, Unity 2022.3.54f1 on SDK 18.3.5: `main+0x3e7288` starts a thread and waits for it to signal `main+0x539cf80`) → Unity's thread start (`main+0x3e1a50`) creates a thread only when `nn::os::GetFreeThreadCount()` is at least 1, and that is `svcGetInfo` type 24 (FreeThreadCount, Horizon 11 and later), which the host answered 0 like every type it did not know: the thread was never asked for (`DM_THR=1` showed no `CreateThread` for it). The kernel answers the thread resource limit less the threads alive (Atmosphère `kern_svc_info.cpp`); `pm` gives an application 96 threads (`pm_spec.cpp`, 256 with a boost). The host now answers 96 less the game's threads that have not exited (`thr_limit_used`, guest threads the host spawns for itself left out). Of the 20 games extracted on this machine, only this `main` imports the function. With it the game creates 40 threads and reaches its title.
|
||||
- **Garbage-collected titles (Unity / IL2CPP): `SetThreadActivity` (0x32) and `GetThreadContext3` (0x33) must really pause and report registers** → the Boehm collector pauses each thread, reads its registers for roots and scans its stack. As placeholders that only reported success, threads kept running and the context was empty: roots held only in registers were missed and live objects were freed, with their finalizers releasing native resources. In `hle_thread.c` a thread inside an SVC counts as paused at once, with the gate frame as its context (`thr_svc_enter` / `thr_svc_leave` around every gate SVC; it waits at the SVC exit while paused). A thread in guest code gets SIGUSR2, and the handler saves the ucontext and waits there. A thread in host code is retried every 100 µs. The context is Horizon's 0x320-byte `ThreadContext`.
|
||||
- **`SignalToAddress` type 2 (`SignalAndModifyByWaitingCountIfEqual`) with count ≤ 0 and waiters present** → the new value is `value − 2`, not `value − 1` (mesosphere `KAddressArbiter`; Ryujinx and yuzu agree). The host had −1. Fixing it did not change Engage's demo crash (that was the address-space shortage above), but it is the specified behaviour.
|
||||
- **Several threads crash at once and the dumps interleave, then the process hangs instead of exiting** → Unity's job threads tend to crash together. `crash.c` lets only the first crashing thread print (`crash_gate`); the others stop in `pause()`, and a 15 s alarm ends the process if the dump itself stalls.
|
||||
@@ -132,7 +133,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- Also as with MediaCodec, an input marked end-of-stream is the last until `Flush` / `Stop`: no input index is handed out or announced after it (`in_eos` in `hle_movie.c`). DRAGON QUEST XI S's movie player sends an end-of-stream packet (length 0, time 0) for every input it gets once its extractor is done, until the game stops the movie. When the prologue movie was skipped, each one became an end-of-stream output that the player turned into an audio sample: 2.25 GB were mapped within seconds, past the application pool, and the game crashed in its allocator.
|
||||
- The "input available" after `Start` is signalled about 2 ms later from a guest thread, as the SDK's codec thread would. Pending signals are dropped at `Stop`, `Flush` and destruction.
|
||||
- **Null read in `sdk` under `nn::applet::CreateLibraryApplet` (Engage: title menu → DLC → `nn::ec::ShowShopAddOnContentList`)** → every `Show…` call that opens a system applet (eShop, friend lists, web pages, error screens) goes through the applet proxy that `oe::Initialize` would have set up, and the host has none. Hook each `Show…` overload the game imports: list `main`'s undefined symbols containing `Show` / `Applet` and grep the host sources for each mangled name; one overload missing is enough to crash. The shop calls return `void` and act as if the player closed the shop (`hle_misc.c`); the friends and web ones return failure (`hle_friends.c`, `hle_offline.c`).
|
||||
- **The opening movie never starts, in about half the boots (DRAGON QUEST XI S: a black screen and no sound from boot on)** → Square Enix's `UMediaPlayer` has two natives of its own, `StartPreDecoding` and `IsPreDecodingCompleted`. The opening's Blueprint polls the second once a frame and only then plays its Matinee, whose event key calls `Play`. Its Switch player (vtable `main+0x77921b8`, `IsPreDecodingCompleted` at `main+0x42d4d20`) is complete when every selected track has at least one decoded output queued. The player starts its clock and its decoders on the first `TickInput` after `StartPreDecoding`. The render thread's first fetch (`main+0x39f07d0`, the `+0x5c7` flag) takes every decoded picture and shows the newest. The host decoded all eight video outputs (to 236 ms) within 36 ms. Each Vorbis decoder's 8 outputs were full by then, because nothing consumes audio before `Play`, so the extractor waited for an audio input and no picture followed the ones the render thread took. The movie started only when the game thread's poll came between the first decoded picture and that fetch. Audio decoders now keep 32 outputs (`MD_NOUT`; video keeps 8), so video keeps flowing after the first fetch: 8 boots out of 8 start. To find such a gate: probe the player's `SetRate` (`DM_PROBE=main:<off>`) to see whether the game ever asks to play, then follow the Blueprint native that calls it (the `FNameNativePtrPair` tables give the names).
|
||||
- **The opening movie never starts, in about half the boots (DRAGON QUEST XI S: a black screen and no sound from boot on)** → Square Enix's `UMediaPlayer` has two natives of its own, `StartPreDecoding` and `IsPreDecodingCompleted`. The opening's Blueprint polls the second once a frame and only then plays its Matinee, whose event key calls `Play`. Its Switch player (vtable `main+0x77921b8`, `IsPreDecodingCompleted` at `main+0x42d4d20`) is complete when every selected track has at least one decoded output queued. The player starts its clock and its decoders on the first `TickInput` after `StartPreDecoding`. The render thread's first fetch (`main+0x39f07d0`, the `+0x5c7` flag) takes every decoded picture and shows the newest. The host decoded all eight video outputs (to 236 ms) within 36 ms. Each Vorbis decoder's 8 outputs were full by then, because nothing consumes audio before `Play`, so the extractor waited for an audio input and no picture followed the ones the render thread took. The movie started only when the game thread's poll came between the first decoded picture and that fetch. Audio decoders now keep 32 outputs (`MD_NOUT`, 64 since Monster Train 2 below; video keeps 8), so video keeps flowing after the first fetch: 8 boots out of 8 start. To find such a gate: probe the player's `SetRate` (`DM_PROBE=main:<off>`) to see whether the game ever asks to play, then follow the Blueprint native that calls it (the `FNameNativePtrPair` tables give the names).
|
||||
- **A long black screen at boot with nothing in the log (Engage: about 17 s on the Mac, 32 s on an iPhone)** → sample the process (`sample <pid> 3`). If the top of stack is `getxattr` under `path_component`, the game is walking RomFS, and every path component scanned its whole parent directory. That scan maps the game's case-sensitive names onto APFS, so it calls `getxattr` on each entry, and Engage has directories of 38,000 files. Read-only mounts now use a per-directory name index built at the first lookup (`dir_index_get` in `hle_fs.c`), which also speeds up every file Wwise opens during play. The first frame then came at 5.7 s on the Mac. The rest was the listing itself: `ReadDirectory` called `stat`, `getxattr` and a failing `fopen("<file>.holes")` for every entry. Now only `.dm-case-*` names (the only ones that carry the xattr) read it, directories take their type from `d_type`, and files use `fstatat` / `openat` relative to the directory. The first frame comes at 1.8 s on the Mac (9.9 s → expected about 3 s on an iPhone), and the opening movie starts at 7 s instead of 49 s.
|
||||
- **DLC the player owns is not unlocked** → the host reports add-on content from `DM_AOC_DIR`, one numbered folder per DLC index (title ID − base − 0x1000) holding that DLC's RomFS. `build_ipa_from_nsp.py` extracts the DLC NSPs in the folder with `nx_extract --aoc`, `build_ipa.sh` copies them to `<App>/aoc/`, and the app sets `DM_AOC_DIR`. `aoc::CountAddOnContent` / `ListAddOnContent` report those indices (`hle_misc.c`), and `fs::MountAddOnContent` mounts the folder read-only (`hle_fs.c`). Engage's two DLC packages each hold one empty marker file (`e0`, `h0`), and the content is already in update 2.0.0. `DLCManager.RefreshHasList` marks a content owned when its mount succeeded (probe `main+0x29f3c88`). D2R keeps its fixed `{1}` without `DM_AOC_DIR`; `DM_AOC_OWNED=0` reports none.
|
||||
- **Abort in `nn::sm::Initialize` under `nn::pctl::IsFreeCommunicationAvailable` (Engage, entering its first battle map)** → the SDK connects to the parental-control service through `sm`. The host answers it like the other free-communication calls: not available (`hle_misc.c`).
|
||||
@@ -175,6 +176,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **Movie sound** → the decoder is asked for type 5 (Vorbis) with output format 1 (PCM16); see the movie entries above and `hle_movie.c`.
|
||||
- **Monster Hunter Rise's movies** → H.264 (type 1) into the client's NVN texture (output 3 = RGBA, mode 2), with AAC 5.1 sound (type 4, PCM16). VideoToolbox decodes to BGRA for that mode (`mavc_set_rgba`), and AAC goes through AudioToolbox's converter with an MPEG-4 ES descriptor built from the AudioSpecificConfig in `csd-0` (as FFmpeg's AudioToolbox decoder does). The game ends a stream with time −1 and the whole 4 MB input buffer as its length: that packet carries no data (decoding it gave VideoToolbox −12909 and AudioToolbox −50). It also deletes its decoders (`D0`), so the host frees the object with the game's `operator delete` after its own state.
|
||||
- **A native-texture movie shows as a strip of small grey copies at the top of the screen** (Absolum's H.264 logos, FNA3D's `nx_video_player.cpp`: type 1, output 3, mode 2) → the texture held NV12 bytes. In RGBA8, 405 rows of 1920 pixels are exactly a 1920×1080 NV12 frame, rows 0–270 the luma at video black (16) and rows 270–405 the chroma (128). This game's MP4 demuxer hands the parameter sets to `Configure` (`csd-0` / `csd-1`), which made the H.264 decoder there without switching it to RGBA, and the first packet then found it made. Both paths now go through `md_avc_open` (`hle_movie.c`). Monster Hunter Rise's movies never brought parameter sets to `Configure`.
|
||||
- **A movie stays black after its first picture, the sound never starts, and the game keeps presenting frames** (Monster Train 2's 90 s intro, `StreamingAssets/intro_fmv.mp4`: H.264 High with 2 reorder frames and AAC interleaved packet by packet, played by Unity 2022.3's movie player: `movie::Extractor` in `subsdk0` feeds both decoders from one thread in file order) → the player takes every output a decoder offers and starts playing only once it holds about half a second of audio. The host then gave each decoder 8 output slots (before DRAGON QUEST XI S's 32 for audio, above) and takes input only while one is free (as MediaCodec does). `DM_MOVIE_TRACE=1` (each line now names the decoder): Unity held all 8 audio outputs, the audio decoder kept the next 4 packets and handed out no input index, the video packets behind them in the file never came, and the video decoder had given out one picture (the host releases H.264 pictures once more than 4 are queued). Slot counts per decoder kind, measured: 8 audio / 8 video and 8 audio / 32 video hang; 64 audio / 8 video plays, starting with 27 audio and 8 video outputs held (peak 30 audio). 32 audio slots hold that peak with two to spare, so audio decoders now have 64 (`MD_NOUT`; a PCM slot is 128 KB, allocated when first used), and video keeps 8 (`MD_NOUT_VIDEO`: a 1080p picture is 3 MB, 8 MB as RGBA). The game's A, B and + do not skip this intro.
|
||||
- **One guest thread at about 1000 ms/s on the iPhone only, the phone hot, CPU energy 5–7 W** (Monster Hunter Rise's `MessageSuperviseThread`, priority 32, in the `[nvn] threads` lines) → the thread waits on `oe::GetNotificationMessageEvent()` with `os::WaitAny`, which does not clear an event, then pops notifications until the queue is empty. The host only ever signalled its notification event, so after the first focus change (iOS changes focus at launch; the Mac does not) it stayed set and the loop never blocked again. The event now follows the queue as the am service's does: set while a notification is queued, cleared when the last one is popped (`hle_oe.c` `mq_ev_sync`, `hle_clear_system_event`). Mac repro: `DM_OE_SCRIPT=focus=2@5,focus=1@6` (the script also starts at the game's first `GetNotificationMessageEvent`, since such a game pops only once the event fires): before 995 ms/s, CPU 109%, 6.97 W; after, the thread is idle and CPU energy is 0.05 W.
|
||||
- **The game freezes after switching apps and back, picture still, sound on** (Super Mario Galaxy 2, two causes; Mac repro `DM_OE_SCRIPT=bg=1@20,bg=0@25`):
|
||||
- **The game's own pause deadlocks** (the main loop waits in its pause handshake, `main+0x5d10d4` / `main+0x5cea68`) → the game set focus handling mode 3. The SDK (`oe::SetFocusHandlingMode`, SDK 18 `sdk+0x3c5770`) turns modes 0 and 3 into "no focus notifications, the system suspends the game out of focus"; only 1 and 2 notify. The host sent message 15 anyway, and Galaxy 2's Wii layer ran a pause the console never triggers. `hle_oe.c` now queues 15 only under modes 1 and 2 (or when no mode was set). Modes seen: 1 Monster Hunter Rise; 2 Wonder, Breath of the Wild, Tears of the Kingdom, Link's Awakening, Cuphead; 3 Galaxy, Galaxy 2, 3D World, Odyssey.
|
||||
|
||||
@@ -12,6 +12,7 @@ Do not stop at the obvious archive. In Mario the main `bfsha` held 4,888 of the
|
||||
- **Unity Shader objects** (Engage): Unity 2020.3 keeps each shader's compiled programs in its `Shader` object (class 48), in UnityFS bundles (`StreamingAssets/aa`) and in the player's own serialized files (`data.unity3d`, `Resources/unity default resources`). `dm/gfx/tools/unityfs.py` reads them: UnityFS blocks (LZ4 / LZMA), the serialized file, the object through its type tree, then `compressedBlob` per platform (LZ4 segments; each segment starts with a u32 count and 12-byte `{offset, length, segment}` entries). The player's files have no type trees; they borrow the bundles' `Shader` tree with the same type hash (same Unity build, same layout). Type-tree strings with the high bit set index Unity's common string table, which `unityfs.common_strings` reads from the game's own `main` (`AABB\0AnimationClip\0…`). On Switch every subprogram is one whole NVN program: its code is a 0x60-byte header of four `i32 ×6` arrays indexed by NVN stage (0 vertex, 1 fragment, 2 geometry, 3 tess control, 4 tess evaluation, 5 compute): an unused / scratch size, the block offset from +0x60 (−1 = no stage), the control size and the code size. Each block is the control section followed by the code area. The vertex and fragment stages of one program form a pair. Engage: 209 `Shader` objects in 188 bundles plus the engine files, 9,269 stage shaders (4,634 vertex / pixel pairs and one geometry shader), no compute shaders.
|
||||
- **Unity 2022 Shader objects** (Ace Attorney Investigations Collection, Unity 2022.3.17f1): the program code keeps the 0x60-byte stage header, but two things changed. From Unity 2022 the blob also holds each program's parameter tables as entries of their own (the parsed form's `m_ParameterBlobIndices`; their bytes start `PGlobals`), and the player's programs are listed in `m_PlayerSubPrograms[tier][].m_BlobIndex` (`m_SubPrograms` is empty); `unity_code_blobs` keeps only those. UnityPy's type trees for 2020.3 (Engage) and 2021.3 have neither field, so those read every entry as before. Subprogram version 202012090 (from Unity 2021.2) also has no local keyword list, so `unityfs.subprogram` reads no second keyword array (with one it read the code length as a keyword count). The player's serialized files have no type trees and the Addressables bundles hold no `Shader`, except the `unitybuiltinshaders` bundle, whose tree has the same hash and is borrowed in the second pass. 4,951 stage shaders: 2,475 vertex / pixel pairs (4,294 in the player's files, 517 built-ins, 140 in bundles) and one geometry shader, which Metal cannot express (its build fails). armyja's issue #17 proposed UnityPy and a reader of its own; neither was needed.
|
||||
- **A Unity player with no bundle to borrow a tree from** (Brain Memory Training Academy, Unity 2022.3.5f1: only `Data/*.assets`, `level*` and the two engine files in `Data/Resources`, none with type trees) → the split stopped with "no bundle has a Shader type tree of the same hash to borrow". The `Shader` type hash there (`ea424a5f…`) is the one in Ace Attorney Investigations Collection's bundles (2022.3.17f1): the same layout. `unityfs.py` now carries that layout written out (`KNOWN_TREES`: about 30 structs, `tree_from_layout` builds the tree a file with type trees would have) and the split's second pass gives it to the stripped files when no bundle lent a tree of that hash (only there: taken in the first pass, Ace Attorney's player files came before its `unitybuiltinshaders` bundle and the built-ins' source column and package indices moved, while with the second pass its tables come out byte-identical). Checked against Ace Attorney's own tree: the same 2,324 nodes with the same alignment flags, and its 40 bundle `Shader` objects decode identically with either. 496 stage shaders: 439 engine built-ins, 46 of the game's, 11 of the player's in `main`, and the built-in VR blit's geometry shader, which does not build. A Unity version whose `Shader` hash is not in the table still stops with the same message; read its tree from any bundle of that Unity version and add it. Patrick's Parabox (Unity 2020.3.30f1, built-in shaders only, no bundles) stopped there with hash `a30e3dcf…`, which one 18 KB shader bundle of Fire Emblem Engage (2020.3.18f1) carries; its tree is now in the table too (`SHADER_2020_3`: per subprogram global and local keyword indices, every subprogram in `m_SubPrograms`, no parameter blobs, keyword names or stage counts), equal to Engage's node for node (2,179 nodes, the alignment flags included), and each of the game's 48 `Shader` objects reads to its exact end. The structs both layouts share are written once (`_SHADER_COMMON`), and the 2022.3 layout comes out byte-identical. To take one file out of an NSP without extracting the RomFS, filter `nx_extract`'s `Copy` by path.
|
||||
- **Unity ComputeShader objects** (Monster Train 2, Unity 2022.3.54f1) → the 3D battle scene was black behind a correct UI, with `[RED] shader 854a8afc9d07f88d is not in the shader map` and `the dispatched program has no compute shader`. `DM_NVN_SHDUMP` saved the program; its code bytes were in `Data/sharedassets2.assets`, uncompressed, inside a `ComputeShader` (class 72): the post-processing stack's colour grading LUT baker (`Lut3DBaker#KGenLut3D_AcesTonemap`), so the LUT the scene is graded through stayed black. The split read only `Shader` objects. A `ComputeShader` holds per platform variant a list of kernels, each with its unique variants; a variant's `code` is the program code of a `Shader` subprogram (the 0x60-byte stage header, only the compute block present) and its `threadGroupSize` matches control +0x748. `unity_compute` in `shader_split.py` reads them beside the `Shader` objects of every serialized file; the game has no bundle with a class 72 tree, so `unityfs.py` carries Unity 2022.3.54f1's layout (`COMPUTE_2022_3`, written from AssetRipper's TypeTreeDumps for that version; all 12 objects decode, `resourcesResolved` true at each end). 85 compute shaders came in (86 with the player's own). A file whose `ComputeShader` has no tree still goes to the second pass; if it has none there either, the split prints a `[RED]` line and leaves its kernels out instead of stopping, so profiles whose compute shaders were never read keep their tables: Cuphead (2) and Fire Emblem Engage (4) have `ComputeShader` objects only in files without type trees, of hashes no layout covers.
|
||||
- **MSHD** (Metroid Dread, MercurySteam's engine: `system/shd/*.bshdat`, the only place its shaders live; nothing in `main` or the `.pkg` packs): each stage is a control section, whose length is at control +0x20, sometimes followed by a small compiler-option table (`fastmath true`), then the code area at the next code header. Every file holds vertex / pixel stages in alternating order (1, 2 or 4 programs) and is paired two by two. The control section carries NV assembly text after its first 0x880 bytes, and the SASS disassembly follows each code area, so the binding layout can be read straight from the files (`TEX` / `TEXS` handle immediates). 478 files, 1,342 stages, 1,010 unique keys (359 vertex, 651 pixel), 659 pairs.
|
||||
- **GLSLC outputs in `main`** (13 Sentinels, Vanillaware's engine: no shader file in RomFS, whose six CRI CPK archives hold textures, scripts, CRI audio and USM movies): 838 GLSLC compiler outputs sit in `main`'s read-only segment, each referenced from a `{pointer, size, 0}` table in `.data`. An output starts with the magic `0x19866891`, its total size is at +0x44, and it holds exactly one control section with the code area right after it (control +0x20 = control length); a debug section with the NV assembly and the GLSL source follows. Engage's `main` also has 11 of them (Unity's own), and `shader_split.py` reads them in every game's `main`, accepting only outputs that pass the control / code / length check. The stage comes from SPH word 0, as for BAHS. Loose RomFS files no other reader takes are searched the same way: FINAL FANTASY TACTICS - The Ivalice Chronicles keeps 257 whole outputs in its technique files (`*.tec`, below), Absolum 163 in `shaders.nx.bin`; in the other profiles' files only Metroid Dread's MSHD files hold any, which `mshd_shaders` reads.
|
||||
- **G2S in RDB archives** (Hyrule Warriors: Age of Calamity, Koei Tecmo's KTGL): each material's shaders are a G2S container (`_S2G3000`, type KTID `0x7bcd279f`; 23,108 entries in `asset/system.rdb.bin*`, 2.3 GB inflated) of GLSLC outputs, as in 13 Sentinels' `main`, inside zlib-chunked IDRK entries ([01](01-extract.md)); one `_DOK0000` container (type `0x20a6a0bb`, the engine's system resources) holds 151 more bare GLSLC outputs, 21 of them drawn in the first battle. `shader_split.py` maps the RDB files and inflates only entries that start with those two magics (`idrk_shaders`), and the 117 outputs in `main` as in every game. Many outputs carry 0x850-byte control sections and the compute ones have no code header, so `glslc_shaders` accepts controls from 0x760 bytes and takes the stage of headerless code from the thread group size at control +0x748. 40,726 shaders (37,043 pixel, 3,440 vertex, 241 compute, 2 geometry), Mario's texture handle layout (`0x20 + 8 × slot`); pairs come from the single-stage cover (`separable_pairs`). The older KTGL of Three Houses packs the same G2S containers differently (next item).
|
||||
|
||||
@@ -440,6 +440,15 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- **Frame rate:** 60 fps (30 on the Switch). The profile's `fps_target` writes 60 into this build's `t.MaxFPS` and 1 into `nvn.SyncInterval` (`hle_fpslock.c`, both threads' copies; the game's config sets 30 and 2); `DM_FPS_TARGET=30` (`Documents/env.txt` on the iPhone) keeps the game's 30 and saves power. On the Mac (M4 Pro) the field holds 60 fps (frame time p95 17 ms, GPU 6.6 ms p50 / 9.7 ms p95) at normal speed, since Unreal steps the game by the time that really passed, with sound intact. The game's dynamic resolution budgets are for 30 fps (`r.DynamicResolution.*TimeBudget` 28–35 ms in `SwitchEngine.ini`), so it does not lower the resolution to reach 60: where a frame needs more than 16.7 ms, frames are dropped instead.
|
||||
- **Mac input:** with the save of an earlier run (the name entered), `DM_KEYS="20:A*150/3,250:Plus*4/10,480:B,483:LUp~4"` skips the prologue movie at about 270 s and walks on Cobblestone Tor at about 480 s.
|
||||
|
||||
**Monster Train 2 (`010051701FB46000`)**
|
||||
- **Status:** brought up on the Mac: the title, the 90 s intro movie with its sound, the tutorial battle (deployment, three waves, enemies on two floors), the merchant and the route map, at 30 fps (as the game asks) with music and saves (the intro plays once per save); no red line, about 2.17 GB resident and 1.21 GB of Metal allocations, flat over twelve minutes. Playable on an iPhone.
|
||||
- **Engine:** Unity 2022.3.54f1 IL2CPP with FMOD on SDK 18.3.5, update 1.1.1 (v131072); its `sdk` is Super Mario Galaxy's build, and `subsdk0` is the movie module, whose `movie::Extractor` demuxes the intro (`StreamingAssets/intro_fmv.mp4`, H.264 High and AAC).
|
||||
- **Host:** three general fixes ([03](agents/03-bringup.md)): `svcGetInfo` type 24 (FreeThreadCount), which the host answered 0, so Unity never created the thread its main thread waits for at boot; 64 output slots for a movie's audio decoder (8 when the game was brought up, 32 since DRAGON QUEST XI S): with 8, Unity's movie player held every slot and the intro hung on a black screen, and it peaks at 30; `nvnSyncSetDebugLabel` (a no-op).
|
||||
- **Shaders:** 15,937: 15,841 vertex, pixel and geometry shaders in the player's serialized files and bundles, the player's 11 GLSLC outputs in `main` (one of them compute), and 85 kernels of the post-processing stack's `ComputeShader` objects in `Data/sharedassets2.assets`, which the split now reads (`unity_compute`, [04](agents/04-graphics.md)); without them the colour grading LUT was never baked and the 3D battle scene stayed black behind the UI. No file carries a type tree for them, so `unityfs.py` has Unity 2022.3.54f1's `ComputeShader` layout (`KNOWN_TREES`); the `Shader` objects use the 2022.3 layout already there. The built-in VR blit's geometry shader does not build.
|
||||
- **Memory:** the NPDM declares 16 MB of system resource (`sys_resource`), so virtual address memory with no heap, as in Ace Attorney Investigations Collection, Brain Memory Training Academy and Patrick\'s Parabox, on the node-area cut this `sdk` build already had (`patch_nso.py`); their 1 GB pool and 5 GB block. With the hardware pool StandardAllocator's region alone reached +3.9 GB of the 4.38 GB alias region in `nninitStartup`. The alias region is mapped up to +3.25 GB within 90 s and stays there (+3.5 GB in a run to the first shop), at most 1.75 GB of it in use.
|
||||
- **Input:** controller only: the game reads no touch, docked or handheld (taps on the title and on route map nodes in handheld mode did nothing). It stays docked at 1920x1080 with the 3D pad layout (`pad_layout full`), whose right stick scrolls the route map as the game\'s prompt shows; A selects and plays cards, B goes back, Y ends the turn.
|
||||
- **Mac input:** from a fresh save, `DM_KEYS="14:A"` starts the game; the intro movie plays to its end (A, B and + do not skip it) and the tutorial battle loads at about frame 3,400; `DM_KEYS="14:A,110:A*90/2,111:Y*20/9"` plays through it to the merchant by about frame 8,000. A save past the intro starts at the title, and A there continues the run.
|
||||
|
||||
## Tools (`dm/tools`)
|
||||
|
||||
| Tool | What it does |
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 115 KiB |
Reference in new issue
Block a user