One NVN driver layout for every game; shader sources read from the game files
The NVN driver's constant segments have the same layout in every game, so the nvn_* profile keys are gone. Combined textures are at c2 0x20 + 8 r (a slot past 31 shares the image table; Hades II binds 34), images at 0x120, separate textures at 0x168 (now always the host's slots 128 + n, as Monster Hunter Rise had them), samplers at 0x568, and the storage buffer tables at c0 0x110 / 0x310 / 0x510. dmsh_build.py decodes with that layout, the replay fills both texture tables, and the translator numbers slots the same way (rebuilt; its SPIR-V and reflection are byte-identical on 1,491 sample shaders, lock updated). The shader_* switches and graphics are gone too; the build reads them from the files (profile_guess.py shader_sources): - Vulkan when main imports Vulkan and not NVN; its capture when the RomFS has no SPIR-V. - A GLSLC capture when main holds GLSL sources and imports glslcCompile. - An Oodle decode for Unreal paks, and the tool-mode decoder for Trinity packs. - GLSLC outputs from every main. - A cover pairing for each single-stage vertex shader (GLSLC, Unreal). build_ipa.sh takes GFXAPI from the NSP script. Also removed: the last D2R pieces (the embedded-shader path in dmsh_build.py, dmsh_facts.py --freeze / --check, maxwell_refs.py, ref/hle_hid_d2r.c). Chain: every profile's dmsh_chain.h is regenerated. The c1_full lists are carried over: the change touches no c1 read, and the rebuilt Octopath Traveler 0, Hades II, Unicorn Overlord and Rhythm Heaven Groove gave the same lists, identical tables and the same reflection apart from separate texture slots + 128. Still to do: - Run the chain procedure for the other NVN profiles. - Engage's and Three Houses' shader tables need a new split: Engage gains Unity's 11 GLSLC shaders in main, and Three Houses' 17 BNSH vertex shaders lose their cover pairs. - Ace Attorney's new tables are in, but its libraries are not rebuilt.
This commit is contained in:
1 parent
d34e0446aa
commit
56b806155f
91 files changed
+348
-1056
No files matched your search
@@ -1,13 +1,13 @@
|
||||
# Notes for agents
|
||||
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Nineteen games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`) and Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`, profile key `graphics vulkan`). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (profile key `shader_glslc_capture`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`patch_nso.py` `FRAME_STEP`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout. Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK archives (`shader_split.py` `kpka_jobs`, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (`hle_nvn.c` `vpool_*`, a 4 KB page table the replay reads textures through); it runs at 60 fps (30 on the Switch). Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into `main`), renders with Vulkan, and generates its shaders at run time, which the NSP script captures on the Mac (profile key `shader_vk_capture`); its pointer is the touch screen in handheld mode (profile key `op_mode`). Ace Attorney Investigations Collection is Unity 2022 IL2CPP: its shaders keep their parameters as blob entries of their own (`shader_split.py` `unity_code_blobs`), and its C# file API goes through the SDK's C library, whose path checks the host answers (`hle_fs.c` `IsPathMounted` and the length limits). Hades II, unlike Hades, renders with NVN; its shaders bind textures at fixed slots up to 34 (profile key `nvn_tex_slots`), and the two long lines through Melinoë while the staff's Omega attack charges are the game's own aim lines, not a rendering bug. Unicorn Overlord is Vanillaware's engine like 13 Sentinels (shaders compiled into `main`); both need the profile key `alloc_big_align`, which aligns large allocations to 64 KB so the ring heaps the engine cuts from a memory pool fit. Rhythm Heaven Groove is Nintendo's own 2D engine on SDK 21, the newest SDK so far (its DSP command list has no magic, `hle_audren.c`); it plays handheld with Boost (profile keys `op_mode handheld`, `perf_mode boost`), which skips its TV latency calibration and keeps its 1920x1080 draw size. Octopath Traveler 0 is Unreal Engine 5: everything in one pak with an AES-encrypted index whose key is found in `main` (`dm/gfx/tools/uepak.py`), its shaders Oodle-compressed in the pak's shader code archives and decoded once on the Mac by the game's own Oodle (profile key `shader_ue_decode`, `DM_OODLE_JOB`), its allocator replaced by the host's because Unreal's reserves 37 GB of address space (`patch_nso.py` cut, `hle_uemalloc.c`), its address region capped by the game's own argument (profile key `game_args`), and the particle shaders that pick a texture slot at run time bound by slot (profile key `tex_dyn_slots`); it runs at 60 fps (30 on the Switch) because the host writes Unreal's `t.MaxFPS` and `rhi.SyncInterval` in game memory (`hle_fpslock.c`, profile key `fps_target`). [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and per-game notes.
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Nineteen games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`) and Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`patch_nso.py` `FRAME_STEP`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout. Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK archives (`shader_split.py` `kpka_jobs`, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (`hle_nvn.c` `vpool_*`, a 4 KB page table the replay reads textures through); it runs at 60 fps (30 on the Switch). Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into `main`), renders with Vulkan, and generates its shaders at run time, which the NSP script captures on the Mac (no SPIR-V file in its RomFS); its pointer is the touch screen in handheld mode (profile key `op_mode`). Ace Attorney Investigations Collection is Unity 2022 IL2CPP: its shaders keep their parameters as blob entries of their own (`shader_split.py` `unity_code_blobs`), and its C# file API goes through the SDK's C library, whose path checks the host answers (`hle_fs.c` `IsPathMounted` and the length limits). Hades II, unlike Hades, renders with NVN; its shaders bind textures at fixed slots up to 34 (slot 34's handle lies in the driver's image table, `dmsh_build.py`), and the two long lines through Melinoë while the staff's Omega attack charges are the game's own aim lines, not a rendering bug. Unicorn Overlord is Vanillaware's engine like 13 Sentinels (shaders compiled into `main`); both need the profile key `alloc_big_align`, which aligns large allocations to 64 KB so the ring heaps the engine cuts from a memory pool fit. Rhythm Heaven Groove is Nintendo's own 2D engine on SDK 21, the newest SDK so far (its DSP command list has no magic, `hle_audren.c`); it plays handheld with Boost (profile keys `op_mode handheld`, `perf_mode boost`), which skips its TV latency calibration and keeps its 1920x1080 draw size. Octopath Traveler 0 is Unreal Engine 5: everything in one pak with an AES-encrypted index whose key is found in `main` (`dm/gfx/tools/uepak.py`), its shaders Oodle-compressed in the pak's shader code archives and decoded once on the Mac by the game's own Oodle (any game with Unreal paks, `DM_OODLE_JOB`), its allocator replaced by the host's because Unreal's reserves 37 GB of address space (`patch_nso.py` cut, `hle_uemalloc.c`), its address region capped by the game's own argument (profile key `game_args`), and the particle shaders that pick a texture slot at run time bound by slot (profile key `tex_dyn_slots`); it runs at 60 fps (30 on the Switch) because the host writes Unreal's `t.MaxFPS` and `rhi.SyncInterval` in game memory (`hle_fpslock.c`, profile key `fps_target`). [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and per-game notes.
|
||||
- Each game's constants and shader tables form a game profile in `dm/games/<id>/`, where `<id>` is the base game's title ID: `game.tsv` plus the tables and `dmsh_chain.h`. Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)). Prefer a rule that holds for every game (Horizon / NVN semantics, or what the binary, its SDK version, its imports or its files show) to a per-game key, cut or build-ID gate, and use those only when no general rule can do it.
|
||||
- Per-build knowledge is keyed by the NSO build ID: rewriter cuts in `dm/tools/patch_nso.py`, the profile's `build` rows (the builds the NSP script accepts, and their display versions), and host behaviour that uses fixed module addresses checks `module_t.build_id` (from `fixups.tsv`; `dm_main_build()`). When a new game build appears, carry addresses over with `dm/tools/nso_fnmap.py` and check them by disassembly; never let fixed addresses run on a build they were not found in.
|
||||
- To analyse or port **another** Switch title, first run `scripts/build_ipa_from_nsp.py` on its NSP folder. With no matching profile, it writes a survey to `<NSP folder>/<title ID>_survey/survey.md`, then a provisional profile guessed from the game files (`dm/games/profile_guess.py`; `<NSP folder>/<title ID>_build/profile/<title ID>/`, found by the tools through `DM_GAMES_DIR`), with a short heap check on the Mac when the game has virtual address memory, and a best-effort IPA that may not boot. Then follow [docs/agents](docs/agents/README.md) (extraction, binary survey, host bring-up failure catalogue, graphics specifics, debugging recipes) and add a profile.
|
||||
- Scripts the user runs live in `scripts/` at the repository root (now `build_ipa_from_nsp.py`). Each one runs both in Spyder (F5, configured by the settings block at its top) and from a terminal (command-line options override those settings; non-zero exit on error). Tools that only agents or other scripts call stay under `dm/`.
|
||||
- Game files, keys and anything extracted from them live in the game's NSP folder, never in this repository and never in the home folder (it is nearly full): the work folder `<NSP folder>/<id>-work` for extraction, runs and analysis, beside the NSP script's `<id>_build/`. Tool venvs and other large scratch data go on that drive too. Do not print key values.
|
||||
- Language: everything is in English: `README.md`, docs, code comments, log messages, script output and other files. Red log lines (something unimplemented or wrong) start with `[RED] `. Text that must match a game's own Chinese data is written as escapes (`"\xE8\xB5\xB7"`), never as characters.
|
||||
- After changing anything on the shader translation chain, run the chain procedure for each affected profile. The chain covers `dm/gfx/tools/dmsh_*`, `Program.cs`, the profile's `shader_map.tsv` / `shader_facts.tsv` / `shader_pairs.tsv`, and its `nvn_*` / `shader_*` keys. The procedure:
|
||||
- After changing anything on the shader translation chain, run the chain procedure for each affected profile. The chain covers `dm/gfx/tools/dmsh_*`, `Program.cs`, the profile's `shader_map.tsv` / `shader_facts.tsv` / `shader_pairs.tsv`, and its `shader_*` keys. The procedure:
|
||||
1. Run `dmsh_chain.py --header`, which writes the profile's `dmsh_chain.h`.
|
||||
2. Rebuild the shaders (Mac and, with `IOSOUT`, iOS).
|
||||
3. Regenerate the profile's `c1_full.tsv`.
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
# chain b-166a14482eae 2026-10-20T06:37:59+08:00 (list carried over from b-f53065c05507: that chain change did not touch any c1 read)
|
||||
# chain b-16260256abdc 2026-10-21T07:21:04+08:00 (list carried over from b-166a14482eae: that chain change did not touch any c1 read)
|
||||
6405f3e55facc7c9
|
||||
d9097b8aba3a7080
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100000000010000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "166a14482eae"
|
||||
#define DMSH_CHAIN "16260256abdc"
|
||||
@@ -27,14 +27,6 @@ mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (Mario's values, the translator's built-in layout) ----
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Layout captures ----
|
||||
# The HUD's counters (coin, regional coin) and other layout panes sample single-channel render targets mapped (R, 0, 0, 1) and use their alpha as the mask, as in Breath of the Wild:
|
||||
# read literally the panes came out as solid white cards. All 26 pixel-shader variants of the layout library (main's BNSH at 0x132c000)
|
||||
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 68.
|
@@ -1,4 +1,4 @@
|
||||
# chain b-c792d8a4c610 2026-10-20T18:17:18+08:00 (list carried over from b-7aa26e75458e: that chain change did not touch any c1 read)
|
||||
# chain b-6c7fd55d331c 2026-10-21T07:21:04+08:00 (list carried over from b-c792d8a4c610: that chain change did not touch any c1 read)
|
||||
0ffd2c92087e4b54
|
||||
69d3868c8d828fe6
|
||||
8a06e3dda01235c1
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010005501E68C000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "c792d8a4c610"
|
||||
#define DMSH_CHAIN "6c7fd55d331c"
|
||||
@@ -40,15 +40,6 @@ mem_alias 0x118000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0x40000000
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# Engage's layout (Unity binds separate textures and samplers): c2 texture handle table 0x168 + 8 r (128 slots), separate sampler table 0x568 + 8 r
|
||||
nvn_tex_base 0x168
|
||||
nvn_tex_slots 128
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py; Unity Shader objects, read by unityfs.py) ----
|
||||
# engine built-ins (the unitybuiltinshaders bundle, Resources/) go to engine, the player's serialized files to data, the other Addressables bundles to assets
|
||||
shader_groups _unitybuiltinshaders_=engine /Resources/=engine /StreamingAssets/=assets /Data/=data
|
||||
|
||||
|
@@ -1 +1,2 @@
|
||||
# Answer table for translation queries (generated by dm/gfx/tools/shader_split.py, do not edit): key <TAB> fact JSON. This game has no PC version to pair with: only the thread group / shared memory of compute shaders (taken from the control section); other queries get the translator defaults
|
||||
552bbc8ffc2459e4 {"local":[4,4,4],"shared":0}
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 19.
|
@@ -1,4 +1,4 @@
|
||||
# kind: evidence Ace Attorney Investigations Collection shaders (Unity Shader in RomFS + main) → control section + code area keys (dm/gfx/tools/shader_split.py)
|
||||
# kind: evidence Ace Attorney Investigations Collection shaders (Unity Shader / GLSLC in RomFS + main) → control section + code area keys (dm/gfx/tools/shader_split.py)
|
||||
# column 5 records where each key first appears; this game has no PC version to pair with
|
||||
package index stage dm_ctl_sha16 source
|
||||
ps_assets.bin 0 pixel 0b06b1dd56a1d46b /Data/StreamingAssets/aa/Switch/gk1_3d_assets_all_1b905485efebcefe1935aa56174b9d06.bundle:CAB-d29090a312ddbfe6a168c5d439c559bc#Unlit/Transparent Colored#var19.1
|
||||
@@ -4952,3 +4952,14 @@ ps_engine.bin 256 pixel 32643b6d855a41a6 /Data/StreamingAssets/aa/Switch/979ebaa
|
||||
vs_engine.bin 256 vertex 9bb9f25b7d580d92 /Data/StreamingAssets/aa/Switch/979ebaa5a17ed2834a47fa1373510580_unitybuiltinshaders_ee931e49447544578e64857d426f80ec.bundle:CAB-4ea63932e2e0a85490d7f9eebee4670d#Unlit/Texture#var19.1
|
||||
ps_engine.bin 257 pixel 0830d959f0d373da /Data/StreamingAssets/aa/Switch/979ebaa5a17ed2834a47fa1373510580_unitybuiltinshaders_ee931e49447544578e64857d426f80ec.bundle:CAB-4ea63932e2e0a85490d7f9eebee4670d#Unlit/Transparent#var19.1
|
||||
vs_engine.bin 257 vertex f8bbf4fbd13d3781 /Data/StreamingAssets/aa/Switch/979ebaa5a17ed2834a47fa1373510580_unitybuiltinshaders_ee931e49447544578e64857d426f80ec.bundle:CAB-4ea63932e2e0a85490d7f9eebee4670d#Unlit/Transparent#var19.1
|
||||
vs_main.bin 0 vertex 24e4cd391d993137 main#GLSLC@0x4621478
|
||||
vs_main.bin 1 vertex d19b47f9ea5c3cfe main#GLSLC@0x4622038
|
||||
ps_main.bin 0 pixel 17b082dfef30eba9 main#GLSLC@0x4622cf8
|
||||
ps_main.bin 1 pixel 889465df59838dc0 main#GLSLC@0x46239b8
|
||||
ps_main.bin 2 pixel e46f75eec323df1d main#GLSLC@0x4624778
|
||||
ps_main.bin 3 pixel a23dda46c07f800d main#GLSLC@0x4625638
|
||||
ps_main.bin 4 pixel bf76784f461275c7 main#GLSLC@0x46262f8
|
||||
ps_main.bin 5 pixel abcdb66b44f1313d main#GLSLC@0x46270b8
|
||||
vs_main.bin 2 vertex f3a2af4f812f7366 main#GLSLC@0x4627f78
|
||||
ps_main.bin 6 pixel 987c069874bbe974 main#GLSLC@0x4628b38
|
||||
cs_main.bin 0 compute 552bbc8ffc2459e4 main#GLSLC@0x46296f8
|
||||
|
Can't render this file because it is too large.
|
@@ -354,6 +354,8 @@
|
||||
24785ad990b0116a 94d430cff8f06751
|
||||
24a859de74551f1c 8276f75f3175d9cc
|
||||
24d8cc05625e02db e2d0af31cd84fe31
|
||||
24e4cd391d993137 0040513ac1ec9d42
|
||||
24e4cd391d993137 0098d97aa0bab969
|
||||
24eb2f3502bbaecc b8d5b14113a57adb
|
||||
24f015810658974d 02039651445a40e7
|
||||
2504c1a6d3ab9cfc e5f0d6c775ffedbb
|
||||
@@ -2021,6 +2023,8 @@ d160cde97b126776 0abd7b1e18cda3e4
|
||||
d1646788791273cd 401011bbed6e4bc7
|
||||
d193b141b41c1a3d 1ed9a4af3a4c5dff
|
||||
d193f1566ab34232 519471ac7a5539aa
|
||||
d19b47f9ea5c3cfe 0040513ac1ec9d42
|
||||
d19b47f9ea5c3cfe 0098d97aa0bab969
|
||||
d1e569006f11ab3d c52605af5e521abf
|
||||
d1fb6387666c9859 666e6fa8c329b595
|
||||
d1fd328daef29040 08e252c88586b54c
|
||||
@@ -2367,6 +2371,8 @@ f2da2b5f2867d8b0 256261d3edaf0b0e
|
||||
f2ec48902836a6aa e17b1942f7afbd65
|
||||
f31de7a0f0671d1f dc6da78e11c89cec
|
||||
f3399e4f3d1f3ad4 3e452330babbce48
|
||||
f3a2af4f812f7366 0040513ac1ec9d42
|
||||
f3a2af4f812f7366 0098d97aa0bab969
|
||||
f3eeee413e8d4287 53f98aaa0e8e5e5f
|
||||
f4271ab1b2ad901e 9dad1806dfcfb789
|
||||
f439e9b801318b22 52a4b739fb4bee8b
|
||||
|
||||
|
@@ -1,4 +1,4 @@
|
||||
# chain b-dde5a04eafc6 2026-10-20T06:18:38+08:00 (list carried over from b-48e79badc676: that chain change did not touch any c1 read)
|
||||
# chain b-5ca6f7dcb1a9 2026-10-21T07:21:04+08:00 (list carried over from b-dde5a04eafc6: that chain change did not touch any c1 read)
|
||||
35cbc686dd8461b8
|
||||
4872bec9a5567f8b
|
||||
54a618792b2d5f0f
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010015100B514000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "dde5a04eafc6"
|
||||
#define DMSH_CHAIN "5ca6f7dcb1a9"
|
||||
@@ -36,16 +36,6 @@ mem_alias 0xB8000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time; measured over all shaders) ----
|
||||
# c2: texture handle table 0x20 + 8 r (32 slots), separate sampler table 0x568, image handle table 0x120 + 8 r (8 slots); the translator has the same values built in, so a game that differs needs a configurable translator first
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
# c0: storage buffer table bases (16 slots × 0x10 per stage)
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py) ----
|
||||
# source groups: a path containing the first part goes to the second group (first match wins; shaders embedded in main always go to main, no match goes to other)
|
||||
shader_groups /Shader/SecredUber=SecredUber .sharcb=agl /Effect/=effect /Layout/=layout /Shader/=SecredUber
|
||||
|
||||
|
@@ -1,3 +1,3 @@
|
||||
# chain b-821df0347986 2026-10-20T06:23:06+08:00 (list carried over from b-aa62503b91c1: that chain change did not touch any c1 read)
|
||||
# chain b-7b0126e8d949 2026-10-21T07:21:04+08:00 (list carried over from b-821df0347986: that chain change did not touch any c1 read)
|
||||
25c5829f891f9b2c
|
||||
976839d3ba92c8f3
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010028600EBDA000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "821df0347986"
|
||||
#define DMSH_CHAIN "7b0126e8d949"
|
||||
@@ -34,15 +34,6 @@ mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# Mario's values until measured over all shaders
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py) ----
|
||||
shader_groups /ShaderData/=shader /EffectData/=effect /LayoutData/=layout /SystemData/=agl
|
||||
shader_note Super Mario 3D World + Bowser's Fury shaders
|
||||
|
@@ -1,4 +1,4 @@
|
||||
# chain b-2ec808c15267 2026-10-20T07:05:13+08:00 (list carried over from b-cc188a5e9879: that chain change did not touch any c1 read)
|
||||
# chain b-a9687d01ebfa 2026-10-21T07:21:04+08:00 (list carried over from b-2ec808c15267: that chain change did not touch any c1 read)
|
||||
0002d193f6c52811
|
||||
0010e810186cdc91
|
||||
001299ca51e856c2
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01002B00111A2000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "2ec808c15267"
|
||||
#define DMSH_CHAIN "a9687d01ebfa"
|
||||
@@ -49,16 +49,6 @@ tex_flush_upload on
|
||||
# over the bound ranges that was millions of vertices a frame in battle (26 fps at any target on the Mac); only over the vertices each draw reads
|
||||
vconv_range on
|
||||
|
||||
# ---- NVN driver constant segments (Mario's values, the translator's built-in layout) ----
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_groups /asset/system.rdb=system /asset/MaterialEditor.rdb=material /asset/FieldEditor4.rdb=field /asset/CharacterEditor.rdb=character /asset/EventMaker3.rdb=event /asset/ScreenLayout.rdb=layout /asset/RRPreview.rdb=preview /asset/KIDSSystemResource.rdb=kids
|
||||
shader_note Hyrule Warriors: Age of Calamity shaders
|
||||
shader_main_glslc on
|
||||
shader_separable on
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 68.
|
@@ -1 +1 @@
|
||||
# chain b-b0883f45f985 2026-10-20T06:20:26+08:00 (list carried over from b-2c262d3399d1: that chain change did not touch any c1 read)
|
||||
# chain b-601174f81d6a 2026-10-21T07:21:04+08:00 (list carried over from b-b0883f45f985: that chain change did not touch any c1 read)
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01003FC01670C000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "b0883f45f985"
|
||||
#define DMSH_CHAIN "601174f81d6a"
|
||||
@@ -1,7 +1,7 @@
|
||||
# Game profile of 13 Sentinels: Aegis Rim (Switch, base game v0 + update 1.10 = v65536). In bring-up: on the Mac a new game gets through the prologue into the first battle
|
||||
# Format: see dm/games/README.md
|
||||
# Vanillaware's own engine, SDK 11.4.3 (sdk and subsdk0 are byte-identical to Metroid Dread's), NVN; all data is in 6 CRI CPK archives (ROBO_*.CPK), audio is CRI ADX2 (acb / awb), movies are CRI USM.
|
||||
# The shaders are not in RomFS: 838 GLSLC compiler outputs (magic 0x19866891) are compiled into main's read-only segment and referenced from a table in .data (read by shader_split.py, see shader_main_glslc)
|
||||
# The shaders are not in RomFS: 838 GLSLC compiler outputs (magic 0x19866891) are compiled into main's read-only segment and referenced from a table in .data (read by shader_split.py glslc_shaders)
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 01003FC01670C000
|
||||
@@ -35,18 +35,6 @@ app_pool 0xCD500000
|
||||
# The engine cuts ring heaps out of a memory pool at 64 KB GPU alignment, and the pool's storage comes from malloc: allocations of 64 KB or more are 64 KB aligned (hle_os.c), or the third ring heap keeps base 0
|
||||
alloc_big_align on
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# Mario's values. All 751 shaders translate with these bases and every binding resolves to a slot (dmsh_build: 0 shaders with unresolved bindings); textures go through bindless handles
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py; GLSLC outputs in main) ----
|
||||
shader_main_glslc on
|
||||
# Each GLSLC output becomes a single-stage program (838 ProgramSetShaders at startup), and BindProgram's stage bits combine vertex and pixel per draw: the pair table is written as a cover (shader_split.py separable_pairs)
|
||||
shader_separable on
|
||||
# ---- Shader split (shader_split.py: GLSLC outputs in main, each a single-stage program, 838 ProgramSetShaders at startup; BindProgram's stage bits combine vertex and pixel per draw) ----
|
||||
shader_groups ROBO=cpk
|
||||
shader_note 13 Sentinels: Aegis Rim shaders
|
||||
|
@@ -1,4 +1,4 @@
|
||||
# chain b-57e1140395a0 2026-10-20T21:58:38+08:00 (list carried over from b-ba76ba138e78: that chain change did not touch any c1 read)
|
||||
# chain b-2eb33bdda057 2026-10-21T07:21:04+08:00 (list carried over from b-57e1140395a0: that chain change did not touch any c1 read)
|
||||
006a737523e70720
|
||||
00da789b4ca18c03
|
||||
00e5f2d3b5851fab
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01005270232F2000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "57e1140395a0"
|
||||
#define DMSH_CHAIN "2eb33bdda057"
|
||||
@@ -37,22 +37,13 @@ mem_alias 0xC8000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments ----
|
||||
# Measured over all translated shaders: separate textures at c2 0x168 + 8 x slot and separate samplers at 0x568 + 8 x slot (Ryujinx handles fp_t_cb196610_1056805A ...: words 0x5A / 0x15A), as Engage
|
||||
nvn_tex_base 0x168
|
||||
nvn_tex_slots 128
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
# 61 vertex shaders pick one of several c2 table slots at run time (texture slots 0-10, samplers 0-4), which the translator cannot trace to one slot: slot indices in c2, the bound slots in its arrays
|
||||
# ---- NVN ----
|
||||
# 61 vertex shaders pick one of several c2 table slots at run time (separate texture slots 0-10, samplers 0-4), which the translator cannot trace to one slot: slot indices in c2, the bound slots in its arrays
|
||||
tex_dyn_slots on
|
||||
|
||||
# ---- Shader split ----
|
||||
# The decoded shader code archives: the engine's global shaders and the game's material shaders
|
||||
shader_groups ShaderArchive-Global-=global ShaderArchive-=game
|
||||
shader_note OCTOPATH TRAVELER 0 shaders
|
||||
# The shaders are in the shader code archives of the pak (ShaderArchive-*.ushaderbytecode), each one Oodle-compressed; the NSP script decodes them on the Mac with the game's own Oodle (uepak.py, DM_OODLE_JOB)
|
||||
shader_ue_decode on
|
||||
# Unreal combines vertex and pixel shaders per draw from its pipeline state: every vertex shader is made to link with any pixel shader
|
||||
shader_separable on
|
||||
# The shaders are in the shader code archives of the pak (ShaderArchive-*.ushaderbytecode), each one Oodle-compressed; the NSP script decodes them on the Mac with the game's own Oodle (uepak.py, DM_OODLE_JOB),
|
||||
# as for any game with Unreal paks. Unreal combines vertex and pixel shaders per draw from its pipeline state, so every vertex shader is made to link with any pixel shader (shader_split.py separable_pairs)
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100535012974000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "101896cbf0ed"
|
||||
#define DMSH_CHAIN "b6dd6fe0cd3c"
|
||||
@@ -21,8 +21,7 @@ package v589824
|
||||
shader_versions v589824
|
||||
|
||||
# ---- Graphics ----
|
||||
# Vulkan through the host's layer (dm/gfx/vk/hle_vk.m); dm/gfx/tools/vk_shaders.py translates the SPIR-V shaders into <shader dir>/vk/
|
||||
graphics vulkan
|
||||
# main imports Vulkan, not NVN: the host's Vulkan layer (dm/gfx/vk/hle_vk.m); dm/gfx/tools/vk_shaders.py translates the SPIR-V shaders into <shader dir>/vk/
|
||||
|
||||
# ---- Files ----
|
||||
# subsdk0 / subsdk1 (Vulkan loader, NVIDIA driver) are replaced by the host's Vulkan layer and not loaded
|
||||
@@ -37,13 +36,5 @@ mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (Mario's values; Hades has no NVN shaders) ----
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_note Hades has no NVN shaders (its SPIR-V shaders go through the Vulkan layer)
|
||||
|
Can't render this file because it contains an unexpected character in line 5 and column 17.
|
@@ -1,4 +1,4 @@
|
||||
# chain b-6f195117b2ca 2026-10-20T07:05:43+08:00 (list carried over from b-b6dcbdf4b4ba: that chain change did not touch any c1 read)
|
||||
# chain b-1866d9bc0803 2026-10-21T07:21:04+08:00 (list carried over from b-6f195117b2ca: that chain change did not touch any c1 read)
|
||||
1a2b65cd05ff557c
|
||||
21c7ff0d9591a637
|
||||
33e4340e5bcbc5c6
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010055D009F78000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "6f195117b2ca"
|
||||
#define DMSH_CHAIN "1866d9bc0803"
|
||||
@@ -38,19 +38,8 @@ mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# Mario's values until measured over all shaders
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py) ----
|
||||
shader_groups /DATA1.bin=data /patch4/=patch4 /patch3/=patch3 /patch2/=patch2 /patch1/=patch1 /nx/=nx /glslc_capture/=runtime
|
||||
shader_note Fire Emblem: Three Houses shaders
|
||||
shader_main_glslc on
|
||||
shader_separable on
|
||||
# KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (subsdk0; sources in main). Capture them once on the Mac with DM_GLSLC_CAPTURE=<version dir>/glslc_capture (hle_glslc.c)
|
||||
shader_glslc_capture on
|
||||
# KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (subsdk0; sources in main): the NSP script captures them once on the Mac with DM_GLSLC_CAPTURE=<version dir>/glslc_capture
|
||||
# (hle_glslc.c), as for any game whose main holds GLSL sources and imports glslcCompile
|
||||
|
@@ -1 +1 @@
|
||||
# chain b-a74254ba6a54 2026-10-20T21:23:08+08:00 (list carried over from b-239c170b7616: that chain change did not touch any c1 read)
|
||||
# chain b-5f5ac45ebe51 2026-10-21T07:21:04+08:00 (list carried over from b-a74254ba6a54: that chain change did not touch any c1 read)
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010069401ADB8000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "a74254ba6a54"
|
||||
#define DMSH_CHAIN "5f5ac45ebe51"
|
||||
@@ -2,7 +2,7 @@
|
||||
# Format: see dm/games/README.md
|
||||
# Vanillaware's own engine (as 13 Sentinels: Aegis Rim), SDK 16.2.4, NVN; all data is in 7 CRI CPK archives (Unicorn.CPK, UnicornPatch.CPK and one per language), audio is CRI ADX2 with Opus streams
|
||||
# (nn::codec's hardware multi-stream decoder, replaced by the SDK's software one in hle_codec.c).
|
||||
# The shaders are not in RomFS: 2188 GLSLC compiler outputs (magic 0x19866891) are compiled into main's read-only segment (1725 distinct shaders, read by shader_split.py, see shader_main_glslc)
|
||||
# The shaders are not in RomFS: 2188 GLSLC compiler outputs (magic 0x19866891) are compiled into main's read-only segment (1725 distinct shaders, read by shader_split.py glslc_shaders)
|
||||
# The two DLC (indices 1 and 2) are licences only: their RomFS holds just the SDK's placeholder file
|
||||
|
||||
# ---- Identity ----
|
||||
@@ -40,18 +40,6 @@ mem_used 0x5c00000
|
||||
# without it a thread's ring heap keeps base 0 and the game writes to 0x268 right after the title screen
|
||||
alloc_big_align on
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# Mario's values. All 1725 shaders translate with these bases and every binding resolves to a slot (dmsh_build: 0 shaders with unresolved bindings); textures go through bindless handles
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py; GLSLC outputs in main) ----
|
||||
shader_main_glslc on
|
||||
# Each GLSLC output becomes a single-stage program, and the program bound per draw combines a vertex and a pixel shader: the pair table is written as a cover (shader_split.py separable_pairs)
|
||||
shader_separable on
|
||||
# ---- Shader split (shader_split.py: GLSLC outputs in main, each a single-stage program; the program bound per draw combines a vertex and a pixel shader) ----
|
||||
shader_groups /=romfs
|
||||
shader_note Unicorn Overlord shaders
|
||||
|
@@ -1,4 +1,4 @@
|
||||
# chain b-35936311a3c9 2026-10-20T06:31:05+08:00 (list carried over from b-75a11052925e: that chain change did not touch any c1 read)
|
||||
# chain b-c3f1a81d79ec 2026-10-21T07:21:04+08:00 (list carried over from b-35936311a3c9: that chain change did not touch any c1 read)
|
||||
26c5fd7adbc12cc0
|
||||
67f23f4ad5925e06
|
||||
96a3e288731b2433
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01006BB00C6F0000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "35936311a3c9"
|
||||
#define DMSH_CHAIN "c3f1a81d79ec"
|
||||
@@ -32,15 +32,6 @@ mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# Mario's values until measured over all shaders
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py) ----
|
||||
shader_groups /region_common/shader/=shader /region_common/effect/=effect /ui/=layout
|
||||
shader_note Link's Awakening shaders
|
||||
|
@@ -1,3 +1,3 @@
|
||||
# chain b-10befbce38df 2026-10-20T14:40:41+08:00 (list carried over from b-d3a07e79011d: that chain change did not touch any c1 read)
|
||||
# chain b-40143d368395 2026-10-21T07:21:04+08:00 (list carried over from b-10befbce38df: that chain change did not touch any c1 read)
|
||||
21fe759b8114941c
|
||||
60e2041ed375cae1
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01007EF00011E000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "10befbce38df"
|
||||
#define DMSH_CHAIN "40143d368395"
|
||||
@@ -41,15 +41,6 @@ mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# Mario's values until measured over all shaders
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py) ----
|
||||
shader_groups /Shader/=shader /Effect/=effect /Layout/=layout /System/=gsys /Terrain/=terrain
|
||||
shader_note Breath of the Wild shaders
|
||||
|
@@ -1,3 +1,3 @@
|
||||
# chain b-9c3f90e7f282 2026-10-20T06:20:18+08:00 (list carried over from b-eb6b1782c78e: that chain change did not touch any c1 read)
|
||||
# chain b-ef65926816d8 2026-10-21T07:21:04+08:00 (list carried over from b-9c3f90e7f282: that chain change did not touch any c1 read)
|
||||
57eb9f0bae677756
|
||||
f826f2b59cfa39a8
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010093801237C000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "9c3f90e7f282"
|
||||
#define DMSH_CHAIN "ef65926816d8"
|
||||
@@ -38,15 +38,6 @@ app_pool 0xCD500000
|
||||
# on hardware they preempt the main thread at once and finish first. Without the approximation about half of the runs crashed at the intro / title movie changes (SignalEvent on a null MultiWait, main+0x225338 reading a freed object); with it 12 runs in a row were clean
|
||||
core_preempt on
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time) ----
|
||||
# The shader files keep the SASS disassembly: the handle immediates of every TEX / TEXS are words 0x8–0x22 (step 2), i.e. c2 bytes 0x20 + 8 r, as in Mario (the set built into the translator). The rest stay as Mario's until measured over all translated shaders
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py; MSHD containers) ----
|
||||
shader_groups /system/shd/=shd
|
||||
shader_note Metroid Dread shaders
|
||||
|
Can't render this file because it contains an unexpected character in line 37 and column 113.
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010099C022B96000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "78abd7b76d0d"
|
||||
#define DMSH_CHAIN "59873f9b6a5a"
|
||||
@@ -23,10 +23,8 @@ package v327680
|
||||
shader_versions v327680
|
||||
|
||||
# ---- Graphics ----
|
||||
# Vulkan through the host's layer (dm/gfx/vk/hle_vk.m). Hagi generates its GX shaders as SPIR-V at run time from data/database/pip/pipelineDB.pip during "Initializing system...":
|
||||
# 3,089 modules with the 56 built into main. The NSP script captures them by booting the game on the Mac (vk_capture.tsv lists the keys); vk_shaders.py translates the capture
|
||||
graphics vulkan
|
||||
shader_vk_capture on
|
||||
# main imports Vulkan, not NVN: the host's Vulkan layer (dm/gfx/vk/hle_vk.m). Hagi generates its GX shaders as SPIR-V at run time from data/database/pip/pipelineDB.pip during "Initializing system...":
|
||||
# 3,089 modules with the 56 built into main, and the RomFS has no SPIR-V file, so the NSP script captures them by booting the game on the Mac (vk_capture.tsv lists the keys); vk_shaders.py translates the capture
|
||||
|
||||
# ---- Files ----
|
||||
# load order as on Horizon: main, subsdk*, sdk; subsdk1–4 (GLSLC, nn::pcm, NVIDIA's OpenGL and Vulkan drivers) are replaced by the host's Vulkan layer and not loaded
|
||||
@@ -43,13 +41,5 @@ mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
# ---- NVN driver constant segments (Mario's values; the game has no NVN shaders) ----
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_note Super Mario Galaxy has no NVN shaders (its SPIR-V shaders go through the Vulkan layer)
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 10.
|
@@ -1 +1 @@
|
||||
# chain b-119d78df4ead 2026-10-20T19:40:15+08:00 (list carried over from b-3f6065a844fa: that chain change did not touch any c1 read)
|
||||
# chain b-22e4d6ecbc13 2026-10-21T07:21:04+08:00 (list carried over from b-119d78df4ead: that chain change did not touch any c1 read)
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100A00019DE0000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "119d78df4ead"
|
||||
#define DMSH_CHAIN "22e4d6ecbc13"
|
||||
@@ -36,16 +36,6 @@ mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
mem_used 0x6c00000
|
||||
|
||||
# ---- NVN driver constant segments (Mario's values, the translator's built-in layout, except the texture slot count) ----
|
||||
# The game's shaders bind textures at fixed, sparse slots: c2 handles at 0x20 + 8 r for r = 0, 1, 3, 6, 8, 11, 15, 21, 25, 26, 30, 31 and 34 over all 88 shaders.
|
||||
# Slot 34 (SceneFade.frag, 0x130) lies past Mario's 32 slots, where his image table starts; this game has no image bindings and no shader reads c2 at run time
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 35
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_groups /Shaders/=shaders
|
||||
shader_note Hades II shaders (Shaders/NVN/bin)
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 68.
|
@@ -1,4 +1,4 @@
|
||||
# chain b-d98265054359 2026-10-20T06:20:07+08:00 (list carried over from b-6a37b6ab3705: that chain change did not touch any c1 read)
|
||||
# chain b-eaf7da2d5f8f 2026-10-21T07:21:04+08:00 (list carried over from b-d98265054359: that chain change did not touch any c1 read)
|
||||
001275671d2c9985
|
||||
0021d75a567e2122
|
||||
00a98da1158c6f14
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100A6301214E000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "d98265054359"
|
||||
#define DMSH_CHAIN "eaf7da2d5f8f"
|
||||
@@ -41,15 +41,6 @@ mem_alias 0x118000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xA0000000
|
||||
|
||||
# ---- NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time; measured over all shaders) ----
|
||||
# Older driver layout (as D2R): c2 texture handle table 0x168 + 8 r (128 slots), separate sampler table 0x568 + 8 r; Unity uses only separate textures / samplers (BindSeparateTexture / Sampler). The image table and the c0 storage buffer table are not measured yet and stay as they were
|
||||
nvn_tex_base 0x168
|
||||
nvn_tex_slots 128
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py; Unity Shader objects, read by unityfs.py) ----
|
||||
# engine built-ins (data.unity3d, Resources/unity default resources) go to engine, the Addressables bundles to assets
|
||||
shader_groups /data.unity3d=engine /Resources/=engine /StreamingAssets/=assets
|
||||
|
||||
|
Can't render this file because it contains an unexpected character in line 38 and column 15.
|
@@ -1,4 +1,4 @@
|
||||
# chain b-6631a1590f6d 2026-10-20T05:18:53+08:00 (list carried over from b-94b2f6efaab8: that chain change did not touch any c1 read)
|
||||
# chain b-0f1c080d3eca 2026-10-21T07:21:04+08:00 (list carried over from b-6631a1590f6d: that chain change did not touch any c1 read)
|
||||
00096443b71e7fef
|
||||
001540067ae4cdf5
|
||||
002e5538fc05267e
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100B04011742000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "6631a1590f6d"
|
||||
#define DMSH_CHAIN "0f1c080d3eca"
|
||||
@@ -48,18 +48,6 @@ app_pool 0xCD500000
|
||||
# used memory must cover them, as on hardware. With 328 MB the heap is 2.88 GB
|
||||
mem_used 0x14800000
|
||||
|
||||
# ---- NVN driver constant segments ----
|
||||
# Two texture handle tables, both in use (a census of the translated shaders' texture names): bound / combined handles at c2 0x20 + 8 × slot (nvnCommandBufferBindTexture; fp_t_tcb_8 … _1E),
|
||||
# separate texture binds at 0x168 + 8 × slot (fp_t_tcb_5A, fp_t_cb196610_1017815A), separate samplers at 0x568; many shaders take a constant sampler ID instead (handle type 3, fp_t_cb65538_3041005C)
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_septex_base 0x168
|
||||
nvn_septex_slots 128
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_groups re_chunk=sdf
|
||||
shader_note Monster Hunter Rise shaders
|
||||
|
@@ -1 +1 @@
|
||||
# chain b-22c0a0e0ae72 2026-10-20T22:47:54+08:00 (list carried over from b-e42dda1cbc03: that chain change did not touch any c1 read)
|
||||
# chain b-befa9a8c84ac 2026-10-21T07:21:04+08:00 (list carried over from b-22c0a0e0ae72: that chain change did not touch any c1 read)
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100D9F01D474000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "22c0a0e0ae72"
|
||||
#define DMSH_CHAIN "befa9a8c84ac"
|
||||
@@ -38,14 +38,6 @@ app_pool 0xCD500000
|
||||
|
||||
# ---- Graphics ----
|
||||
|
||||
# ---- NVN driver constant segments (Mario's values, the translator's built-in layout; every shader binding decodes with them) ----
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py) ----
|
||||
shader_groups /mesg/=mesg /graph/=graph /sound/=sound /wavmark/=wavmark /voicetext/=voicetext /shader/=shader
|
||||
shader_note Rhythm Heaven Groove shaders
|
||||
|
@@ -1,4 +1,4 @@
|
||||
# chain b-330a5957853d 2026-10-20T13:29:14+08:00 (list carried over from b-743f7a102a72: that chain change did not touch any c1 read)
|
||||
# chain b-f5c63fb9b17f 2026-10-21T07:21:04+08:00 (list carried over from b-330a5957853d: that chain change did not touch any c1 read)
|
||||
35cbc686dd8461b8
|
||||
4872bec9a5567f8b
|
||||
759c14c6d029f531
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100F2C0115B6000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "330a5957853d"
|
||||
#define DMSH_CHAIN "f5c63fb9b17f"
|
||||
@@ -45,14 +45,6 @@ tex_dirty_upload on
|
||||
# (the first frames) the stub steps by the measured frame time
|
||||
fps_target 60
|
||||
|
||||
# ---- NVN driver constant segments (Breath of the Wild's values, the same NVN build) ----
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split (shader_split.py) ----
|
||||
shader_groups /Shader/ApplicationPackage=layout /Shader/=shader /Effect/=effect /UI/=layout
|
||||
shader_note Tears of the Kingdom shaders
|
||||
|
Can't render this file because it contains an unexpected character in line 5 and column 70.
|
@@ -1,4 +1,4 @@
|
||||
# chain b-21d0d9d3a966 2026-10-21T01:08:40+08:00
|
||||
# chain b-12de0799766c 2026-10-21T07:21:04+08:00 (list carried over from b-21d0d9d3a966: that chain change did not touch any c1 read)
|
||||
000566ec19dc74f7
|
||||
000b591830b6fa73
|
||||
0027e56f73d8e10e
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100F43008C44000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "21d0d9d3a966"
|
||||
#define DMSH_CHAIN "12de0799766c"
|
||||
@@ -24,9 +24,6 @@ modules main sdk
|
||||
romfs_marker arc/data.trpfs
|
||||
zip_store .bnk:.pck:.trpfd:.trpfs:.metallib
|
||||
|
||||
# ---- Graphics ----
|
||||
graphics nvn
|
||||
|
||||
# ---- Memory ----
|
||||
# nninitStartup (main+0x1b2e650) initializes its StandardAllocator with virtual address memory over 12 GB of address space (StandardAllocator::Initialize(nullptr, 0x300000000), main+0x1b3e3ec),
|
||||
# which the Switch's 64 GB alias region holds. Two cuts for 2.0.2 (dm/tools/patch_nso.py) make it fit the 5 GB block proven on a phone: the allocator gets 4 GB and the SDK's node area 62 MB instead
|
||||
@@ -38,18 +35,9 @@ app_pool 0xCD500000
|
||||
# nninitStartup allocates and frees all it can (pool − 64 MB) and the SDK keeps those pages mapped until a map fails: the pool is enforced as on Horizon, so it gives them back (hle_svc.c)
|
||||
mem_map_limit on
|
||||
|
||||
# ---- NVN driver constant segments (Mario's values, the translator's built-in layout, unless the engine says otherwise) ----
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split ----
|
||||
# Game Freak's Trinity engine: every shader is a BNSH inside the Oodle-compressed packs of arc/data.trpfs (shader_split.py trpfs_jobs), named by the pack's first file:
|
||||
# the material shaders in system_resource/gfx2/shader, particle effects in ik_effectdata, UI layouts (__ArchiveShader.bnsh in .truiv) in uidata
|
||||
shader_groups #arc/system_resource=system #arc/ik_effect=effect #arc/ik_pokemon=pokemon #arc/ui=layout /arc/=arc
|
||||
shader_note Pokémon Legends: Z-A shaders
|
||||
# The packs are Oodle Mermaid streams; the shader split decodes them with the game's own OodleLZ_Decompress in the Mac host (dm/host/hle_oodle.c), which the NSP script builds for it
|
||||
shader_oodle on
|
||||
# The packs are Oodle Mermaid streams; the shader split decodes them with the game's own OodleLZ_Decompress in the Mac host (dm/host/hle_oodle.c), which the NSP script builds for any game with Trinity packs
|
||||
|
+18
-18
@@ -7,7 +7,7 @@ Each game the port knows has a folder `dm/games/<id>/`, named after the base gam
|
||||
- **`shader_facts_manual.tsv`** (optional): hand-written translation facts in `shader_facts.tsv`'s row format, for what the machine code cannot tell (Link's Awakening: a vertex input the integer-input guess gets wrong; Breath of the Wild and Tears of the Kingdom: integer fragment outputs, which a game without a PC version has no other source for). `shader_split.py` merges it over the generated rows; since the table is on the chain, changing it changes only that profile's chain version.
|
||||
- **`c1_full.tsv`**: regenerated by `c1_guard.py` after each shader build.
|
||||
- **`pso_manifest.tsv`**: the prewarm seed list.
|
||||
- **`vk_capture.tsv`** (Vulkan games with `shader_vk_capture`): the keys of the SPIR-V modules the game generates at run time.
|
||||
- **`vk_capture.tsv`** (a Vulkan game that generates its shaders at run time, Super Mario Galaxy): the keys of the SPIR-V modules it generates.
|
||||
- **`bf_want.tsv`**.
|
||||
- **`dmsh_chain.h`**: the chain version, written by `dmsh_chain.py --header`.
|
||||
|
||||
@@ -15,7 +15,7 @@ Everything game-specific belongs here. Code reads the profile; it does not hard-
|
||||
|
||||
| Who | Picks the profile with | Reads |
|
||||
|---|---|---|
|
||||
| Shader tools (`shader_split.py`, `dmsh_chain.py`, `dmsh_build.py`, `dmsh_facts.py`) | `DM_GAME=<id>` | tables, `nvn_*`, `shader_*` keys |
|
||||
| Shader tools (`shader_split.py`, `dmsh_chain.py`, `dmsh_build.py`, `dmsh_facts.py`) | `DM_GAME=<id>` | tables, `shader_*` keys |
|
||||
| Host (`make -C dm/host`, `dm/gfx/tests`) | `GAME=<id>` | `gen/<id>/game_profile.h`, generated from `game.tsv` by `game.py header` |
|
||||
| `dm/ios/tools/build_ipa.sh` | `GAME=<id>` | app name, bundle ID, IPA name, modules, RomFS marker, tables |
|
||||
| `scripts/build_ipa_from_nsp.py` | the NSPs' title ID | everything above. With no matching profile it writes a survey, then a provisional profile outside the repository and a best-effort IPA (`--survey-only` stops after the survey) |
|
||||
@@ -37,7 +37,6 @@ None of them has a default game: without one they stop and list the profiles. To
|
||||
| `modules` | Code modules rewritten and loaded, in load order. `rtld` is replaced by the host |
|
||||
| `romfs_marker` | A path that must exist in the RomFS |
|
||||
| `zip_store` | Extensions the IPA stores without compressing |
|
||||
| `graphics` | `nvn` (the default) or `vulkan`: the game renders with Vulkan and the host implements its `vk*` imports on Metal (`dm/gfx/vk/hle_vk.m`, Hades). Its shaders are the game's SPIR-V files, translated by `dm/gfx/tools/vk_shaders.py <RomFS> <shader dir> [--ios <dir>]` into `<dir>/vk/`; `build_ipa.sh` then ships `$MSL_IOS/vk` instead of the NVN chain libraries, and the NVN shader tables stay empty |
|
||||
| `mem_reserve`, `mem_alias`, `mem_stack`, `app_pool` | Guest address-space partition and application pool (`hle_svc.c`) |
|
||||
| `mem_used` | Memory `GetInfo(UsedMemorySize)` reports beside the heap (default `0x4000000`, 64 MB). `SetHeapSize` checks a request against the real size of the loaded modules, so a game that sizes its heap as total − used and has more than 64 MB of modules needs at least their size here. Hyrule Warriors: Age of Calamity: 66.6 MB of modules, so the default made `SetHeapSize(pool − 64 MB)` fail and `nnMain` return at once; it sets 72 MB |
|
||||
| `game_args` | Arguments the game gets when the command line gives none (the iOS app, the NSP script's Mac runs), split at spaces; `dmhost <dir> -- …` overrides them. The SDK hands them to the game before `nninitStartup`, as on hardware (`main.c`). Octopath Traveler 0 caps the address region Unreal's `nninitStartup` reserves (62 GB by default) with `-SwitchVirtualAddressSpaceSize=2048` (MB) |
|
||||
@@ -47,7 +46,7 @@ None of them has a default game: without one they stop and list the profiles. To
|
||||
| `core_preempt` | `on` / `off` (the default): approximate Horizon's per-core preemption. When an SVC wakes a higher-priority thread pinned to the caller's core, the caller waits (at most 5 ms) until such threads block again (`hle_thread.c`). Metroid Dread's movie player relies on it; Mario's loading gets slower with it (first frame 0.6 → 2.9 s), so it is off unless a game needs it. `DM_PREEMPT=0` / `1` overrides |
|
||||
| `alloc_big_align` | `on` / `off` (the default): `nn::mem::StandardAllocator::Allocate` aligns blocks of 64 KB or more to 64 KB (`hle_os.c`). Vanillaware's engine (13 Sentinels: Aegis Rim, Unicorn Overlord) takes a memory pool's storage from malloc and cuts ring heaps out of it at 64 KB alignment relative to the pool's GPU address. On the Switch that address is big-page aligned; the host's is the CPU address, only 4 KB aligned, so the last ring heap does not fit and keeps base 0, and the game writes through a null pointer (13 Sentinels entering its first battle, Unicorn Overlord past the title screen) |
|
||||
| `shared_font` | Optional, decided by the SDK version without it (the host reads it from the `sdk` module, `loader.c` `dm_sdk_major`): what the shared-font address points at. `bfttf` (SDK 11 and later) = the 8-byte BFTTF header (with the "already decoded" marker) followed by the TTF, which Mario passes to `ScalableFontEngineHelper::Decode`; `ttf` = the TTF itself, which SDK 10 and older games hand straight to `ScalableFontEngine::LoadFont` (Super Mario 3D World + Bowser's Fury: with the header the engine fails with error 403 and the game crashes looking up a glyph; Odyssey, Link's Awakening, Age of Calamity and Three Houses alike). No profile sets it |
|
||||
| `device_api` | Optional, decided by the SDK version without it: the NVN API version `major.minor` that `nvnDeviceGetInteger` reports, 55.15 for SDK 11 and later (what Mario requires) and 53.313 for SDK 10 and older (`hle_nvn.c` `nvn_api`). Games check it and skip creating their device on a mismatch: Super Mario 3D World + Bowser's Fury (SDK 10) requires major 53 and minor ≥ 313. No profile sets it. `DM_NVN_API` overrides. Not part of the shader chain version (only `nvn_*` / `shader_*` keys are) |
|
||||
| `device_api` | Optional, decided by the SDK version without it: the NVN API version `major.minor` that `nvnDeviceGetInteger` reports, 55.15 for SDK 11 and later (what Mario requires) and 53.313 for SDK 10 and older (`hle_nvn.c` `nvn_api`). Games check it and skip creating their device on a mismatch: Super Mario 3D World + Bowser's Fury (SDK 10) requires major 53 and minor ≥ 313. No profile sets it. `DM_NVN_API` overrides. Not part of the shader chain version (only `shader_*` keys are) |
|
||||
| `op_mode` | The operation mode the game sees at start: `console` (the default, docked) or `handheld`. Super Mario Galaxy uses `handheld`: docked, its pointer is the Joy-Con gyro, which the host does not have; handheld, it is the touch screen, which the iPhone front end feeds with the touches no gamepad control takes (`DmGamepad.m`). Handheld also renders at 720p. `DM_OPMODE=0` / `1` overrides, also from `Documents/env.txt` |
|
||||
| `perf_mode` | `follow` (the default): the performance mode follows the operation mode (Boost docked, Normal handheld); `boost`: Boost whatever the operation mode (`hle_oe.c`), a combination a Switch 1 never reports, so a profile choice. Rhythm Heaven Groove plays handheld with Boost: docked means a TV to the game, which then asks for a latency calibration, and only Boost gives its 1920x1080 draw size (handheld Normal draws 1280x720) |
|
||||
| `pad_layout` | The iPhone virtual gamepad's layout on first launch: `simple` (the default) = 2D (left stick, A/B/X/Y, shoulders, − / +), `full` = 3D (adds a D-pad, a right stick and L3 / R3). The layout button at the top centre switches between them in game and the app remembers the player's choice (`DmGamepad.m`) |
|
||||
@@ -56,29 +55,30 @@ None of them has a default game: without one they stop and list the profiles. To
|
||||
| `tex_flush_upload` | `on` / `off` (the default): a texture the game fills from the CPU in place (no packaged layout, no `WriteTexels`) and publishes with `nvnMemoryPoolFlushMappedRange` is uploaded from guest memory like a packaged one, and re-uploaded when its pages change (`ft_add` in `hle_nvn.c`). Tears of the Kingdom streams all its textures this way; without it the 3D scene is black. Off by default because a flushed range in another game could cover an image a shader writes, whose GPU content the upload would replace |
|
||||
| `render_cond` | `on` / `off` (the default): `SetRenderEnableConditional` / `SetRenderEnable` are recorded and evaluated at replay, and a timestamp report's counter field is 0 as on the GPU (`OP_RENDER_COND` in `nvn_mtl.m`). Tears of the Kingdom wraps its clears in conditions that skip clearing a target nothing has drawn into since its last clear; without the key every clear runs and its 3D scene is black. Off by default: other games draw everything, as before |
|
||||
| `tex_dirty_upload` | `on` / `off` (the default), needs `tex_flush_upload`: a registered texture whose pages the CPU writes after `nvnTextureInitialize` without any NVN flush is uploaded as well (its pages are write-protected at registration, checked on first use; `nvn_ft_written` in `hle_nvn.c`). Tears of the Kingdom fills its ASTC terrain material arrays this way; without it the cave is magenta |
|
||||
| `tex_dyn_slots` | `on` / `off` (the default): a shader whose texture handle the translator could not trace to one slot gets Ryujinx's fallback arrays (`<stage>_<t|tb|s>_cb0_40000000`: 32 textures and samplers indexed by the handle's texture / sampler ID). With the key the replay puts slot indices in that shader's c2 texture and sampler tables (texture slot r → r, sampler slot r → r << 20) and binds slot k at element k, so the shader's own arithmetic picks the binding (`NVN_RB_TEXD` / `NVN_RB_SAMPD` in `nvn_mtl.m`). Right only where the handle comes from c2's tables: Octopath Traveler 0's 61 such vertex shaders read texture slots 0-10 and samplers 0-4 there and nothing else. Off, such shaders do not load. Not part of the shader chain version |
|
||||
| `tex_dyn_slots` | `on` / `off` (the default): a shader whose texture handle the translator could not trace to one slot gets Ryujinx's fallback arrays (`<stage>_<t|tb|s>_cb0_40000000`: 32 textures and samplers indexed by the handle's texture / sampler ID). With the key the replay puts slot indices in that shader's c2 texture and sampler tables (texture slot r → r, sampler slot r → r << 20) and binds slot k at element k, so the shader's own arithmetic picks the binding (`NVN_RB_TEXD` / `NVN_RB_SAMPD` in `nvn_mtl.m`). Right only where the handle comes from c2's tables: Octopath Traveler 0's 61 such vertex shaders read separate texture slots 0-10 and samplers 0-4 there and nothing else. Off, such shaders do not load. Not part of the shader chain version |
|
||||
| `gpu_timestamps` | What a timestamp report (`nvnCommandBufferReportCounter`) records: `submission` (the default) = the time the whole submission completed, the same for every report in it; `segment` = the GPU start (`TIMESTAMP_TOP`) or end (the other types) of the segment the report was encoded in (the host commits a submission in segments of 64 draws, `sub_cut` in `nvn_mtl.m`), so two reports of one submission bracket the GPU time between them, as on hardware. Breath of the Wild needs `segment`: its frame-interval controller and its dynamic resolution read the GPU time between the frame's first and last report, and with one time for both they read 0 (the game kept 60 fps in slow motion when the GPU could not keep up). `DM_NVN_GPU_TS=segment` / `submission` overrides it, also from `Documents/env.txt` |
|
||||
| `vconv_range` | `on` / `off` (the default): vertex attributes the replay converts on the CPU (integer shader inputs fed float-type formats and the like, `vconv_bind` in `nvn_mtl.m`) are converted only over the vertices each draw reads (first / count, or the smallest and largest index of a direct indexed draw, kept while the index pages stay unwritten) instead of the whole bound range. Age of Calamity binds whole mesh pools: in a battle that was millions of vertices a frame. Indirect draws, per-instance streams and indices with a GPU write queued on them keep the whole range. `DM_NVN_VCONV_CHECK=1` scans the indices again at every reuse and reports differences; `DM_NVN_VCONV_RANGE=0` / `1` overrides it |
|
||||
| `nvn_tex_base`, `nvn_tex_slots`, `nvn_samp_base`, `nvn_img_base`, `nvn_img_slots`, `nvn_ssbo_bases` | NVN driver constant segments, measured over all shaders |
|
||||
| `nvn_septex_base`, `nvn_septex_slots` | Optional: a separate texture handle table of its own, for a game that binds both kinds of handle. On hardware `nvnCommandBufferBindTexture` and `nvnCommandBufferBindSeparateTexture(s)` fill two tables of the driver segment; without these keys the host keeps both kinds in one (Mario binds only combined handles, Engage only separate ones). Monster Hunter Rise binds both, slot 0 of each in the same frames: combined handles at `nvn_tex_base` 0x20, separate textures at 0x168. With the keys, separate binds go to slots 128 + n of the replay's table (`SEPTEX_SLOT0` in `hle_nvn.c`), `dmsh_build.py` names that table's handles 128 + n, and the replay fills both tables of c2 |
|
||||
| `shader_groups`, `shader_note` | `shader_split.py` source groups and the map header note (Engage: `engine` for `data.unity3d` and `Resources/`, `assets` for the Addressables bundles) |
|
||||
| `shader_main_glslc` | `on` / `off` (the default): `shader_split.py` also reads GLSLC compiler outputs (magic `0x19866891`) in `main`. 13 Sentinels keeps all its shaders there; Engage's `main` has 11 that it does not use, so the key stays off for other games |
|
||||
| `shader_separable` | `on` / `off` (the default): the game builds single-stage programs and combines them per draw, so the real vertex / pixel pairs exist only at run time. `shader_split.py` then pairs every vertex shader with a cover of pixel shaders that together read every input any pixel shader reads, and `dmsh_build.py`'s `vs_pad` makes every vertex shader link with any pixel shader (13 Sentinels) |
|
||||
| `shader_vk_capture` | `on` / `off` (the default), with `graphics vulkan`: the game generates its SPIR-V shaders at run time (Super Mario Galaxy: Hagi's GX shaders, 3,089 modules created from its pipeline database while it shows "Initializing system..."), so no game file holds them. They are captured on the Mac with `DM_VK_CAPTURE=<dir>` (`dm/gfx/vk/hle_vk.m`; the NSP script does it itself, booting the game in the Mac host until every key the profile's `vk_capture.tsv` lists is there), and `vk_shaders.py` translates the capture folder with the RomFS |
|
||||
| `shader_oodle` | `on` / `off` (the default): the game's archives are Oodle-compressed (Pokémon Legends: Z-A's Trinity packs, `arc/data.trpfs`), so `shader_split.py` decodes them with the game's own `OodleLZ_Decompress`, run by the Mac host in its tool mode (`dm/host/hle_oodle.c`, `DM_OODLE_SERVE`, the decoder's address per `main` build). The split needs `DM_OODLE_HOST` (the Mac host) and `DM_OODLE_MODULES` (the rewritten modules); the NSP script builds the host and sets both when the split runs. `profile_guess.py` sets the key when the RomFS has `ONEPACK` files |
|
||||
| `shader_glslc_capture` | `on` / `off` (the default): the game compiles some shaders at run time with its own GLSLC (Fire Emblem: Three Houses: 9 built-in KTGL shaders at boot), so no game file holds them. They are captured on the Mac with `DM_GLSLC_CAPTURE=<dir>` (`dm/host/hle_glslc.c`; the NSP script does this itself, booting the game in the Mac host until every capture file the profile's `shader_map.tsv` names is there), and `shader_split.py` reads `<version dir>/glslc_capture/*.glslc` (beside that version's `RomFS`) or `$DM_GLSLC_CAPTURE`; without the folder those shaders get no translation and the tables differ from the profile's |
|
||||
| `shader_ue_decode` | `on` / `off` (the default): an Unreal Engine game keeps its shaders in the shader code archives of its paks (`*.ushaderbytecode`), each shader Oodle-compressed, which only the game's own Oodle decodes. The NSP script (`decode_ue_shaders`) takes the archives out of the paks with `dm/gfx/tools/uepak.py` (the pak index is AES-encrypted with a key it finds in `main`), boots the game in the Mac host with `DM_OODLE_JOB` (`dm/host/hle_oodle.c`: the game's `OodleLZ_Decompress`, found as the one function that references the library's own name string, decodes every shader right before `nnMain`, then the host exits) and writes the decoded archives to `<version dir>/ue_shaders`, where `shader_split.py` reads them (`DM_UE_SHADERS` overrides). Octopath Traveler 0: 10,596 shaders in 2 s. Its vertex and pixel shaders combine per draw, so the profile also sets `shader_separable` |
|
||||
|
||||
The translator (`dmsh_xlate/Program.cs`) still has Mario's texture / sampler bases and the depth convention built in. It uses the bases only to look up per-slot texture / image types in `shader_facts.tsv`; `dmsh_build.py` decodes the bindings from the profile. So a profile with other bases (Engage: `0x168` / `0x568`) builds as long as its facts table has no per-slot texture / image entries; otherwise `dmsh_build.py` stops, and the translator must take the bases from the profile first.
|
||||
### What the game files decide at every build
|
||||
|
||||
These are not profile keys: the tools read them from the game files each time, by rules checked against every profile (`dm/games/profile_guess.py` `shader_sources`).
|
||||
|
||||
- **Graphics API.** A game whose `main` imports Vulkan functions and not `nvnBootstrapLoader` renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; Hades, Super Mario Galaxy). Its shaders are its SPIR-V files, translated by `dm/gfx/tools/vk_shaders.py <RomFS> <shader dir> [--ios <dir>]` into `<dir>/vk/`, and `build_ipa.sh` ships `$MSL_IOS/vk` instead of the NVN chain libraries (`GFXAPI`, which the NSP script passes; by hand the shader libraries there tell). The NVN shader tables stay empty.
|
||||
- **SPIR-V generated at run time.** A Vulkan game without SPIR-V files in its RomFS generates its shaders at run time (Super Mario Galaxy: Hagi's GX shaders, 3,089 modules created from its pipeline database while it shows "Initializing system..."). The NSP script captures them on the Mac with `DM_VK_CAPTURE=<dir>` (`dm/gfx/vk/hle_vk.m`), booting the game in the Mac host until every key the profile's `vk_capture.tsv` lists is there, and `vk_shaders.py` translates the capture folder with the RomFS.
|
||||
- **Shaders compiled at run time.** A game whose `main` holds GLSL sources and calls `glslcCompile` in a module it loads compiles some shaders at run time with its own GLSLC (Fire Emblem: Three Houses: 9 built-in KTGL shaders at boot), so no game file holds them. The NSP script captures them on the Mac with `DM_GLSLC_CAPTURE=<dir>` (`dm/host/hle_glslc.c`), booting the game in the Mac host until every capture file the profile's `shader_map.tsv` names is there, and `shader_split.py` reads `<version dir>/glslc_capture/*.glslc` (beside that version's `RomFS`) or `$DM_GLSLC_CAPTURE`; without the folder it prints a red line, those shaders get no translation and the tables differ from the profile's.
|
||||
- **Unreal paks.** An Unreal Engine game keeps its shaders in the shader code archives of its paks (`*.ushaderbytecode`; a pak is Unreal's when its footer has Unreal's magic, `uepak.py` `unreal_paks`), each shader Oodle-compressed, which only the game's own Oodle decodes. The NSP script (`decode_ue_shaders`) takes the archives out of the paks with `dm/gfx/tools/uepak.py` (the pak index is AES-encrypted with a key it finds in `main`), boots the game in the Mac host with `DM_OODLE_JOB` (`dm/host/hle_oodle.c`: the game's `OodleLZ_Decompress`, found as the one function that references the library's own name string, decodes every shader right before `nnMain`, then the host exits) and writes the decoded archives to `<version dir>/ue_shaders`, where `shader_split.py` reads them (`DM_UE_SHADERS` overrides; without them it stops). Octopath Traveler 0: 10,596 shaders in 2 s.
|
||||
- **Trinity packs.** Game Freak's Trinity engine keeps nearly every file in Oodle-compressed packs (`arc/data.trpfs`, files starting `ONEPACK`; Pokémon Legends: Z-A). `shader_split.py` decodes them with the game's own `OodleLZ_Decompress`, run by the Mac host in its tool mode (`dm/host/hle_oodle.c`, `DM_OODLE_SERVE`). The split needs `DM_OODLE_HOST` (the Mac host) and `DM_OODLE_MODULES` (the rewritten modules); the NSP script builds the host and sets both when the split runs.
|
||||
- **GLSLC outputs in `main`.** `shader_split.py` reads GLSLC compiler outputs (magic `0x19866891`, each one checked whole) in every game's `main`: 13 Sentinels and Unicorn Overlord keep all their shaders there, the KTGL games some; Unity's player has 11 of its own beside the game's shaders in its bundles (Engage, Ace Attorney Investigations Collection).
|
||||
- **Single-stage programs.** A GLSLC output or an Unreal shader record is a program of one stage, combined with others per draw, so its real vertex / pixel pairs exist only at run time. `shader_split.py` pairs each such vertex shader with a cover of pixel shaders that together read every input any pixel shader reads, and `dmsh_build.py`'s `vs_pad` makes it link with any pixel shader (13 Sentinels, Unicorn Overlord, the KTGL games, Octopath Traveler 0). BNSH variations, Unity programs and the MSHD and SDF files pair their own shaders.
|
||||
- **NVN driver tables.** The driver's constant segments have one layout in every game (`dmsh_build.py`, filled by `nvn_mtl.m`): c2 holds the handles bound with `nvnCommandBufferBindTexture` at `0x20 + 8 × slot`, images at `0x120 + 8 × slot`, separate textures at `0x168 + 8 × slot` and separate samplers at `0x568 + 8 × slot`; c0 the storage buffer tables (vertex 0x110, compute 0x310, pixel 0x510). Separate texture binds are texture slots 128 + n of the host's table (`NVN_SEPTEX_SLOT0`), so a game may bind both kinds in one frame (Monster Hunter Rise: combined slot 0 beside separate slots 0-4), and a combined slot past 31 (Hades II binds slot 34) lies in the image table, where a bound texture wins over an unbound image. The translator (`dmsh_xlate/Program.cs`) numbers slots the same way when it looks up per-slot texture / image types in `shader_facts.tsv`.
|
||||
|
||||
## Adding a game
|
||||
|
||||
1. **Survey.** Run `scripts/build_ipa_from_nsp.py` on the game's NSP folder. Without a profile it extracts only the code modules and a RomFS list, then writes `<NSP folder>/<title ID>_survey/survey.md`. The survey covers modules and build IDs, SDK, engine and graphics API markers, RomFS containers, the rewrite-site census, and imports compared with the host hooks.
|
||||
Unless told `--survey-only`, the script then writes a **provisional profile** to `<NSP folder>/<title ID>_build/profile/<title ID>/` (outside the repository) and carries on to a best-effort IPA. `dm/games/profile_guess.py` fills its `game.tsv` from the game files, each key under the reason for it (the survey lists them too):
|
||||
- identity and name, the build row it found, the largest RomFS file as `romfs_marker`, big file types in `zip_store`, `shader_groups` by RomFS top-level folder, Mario's NVN values (Engage's texture base under Unity IL2CPP);
|
||||
- `graphics vulkan` when `main` imports Vulkan functions and not `nvnBootstrapLoader`, then `shader_vk_capture on` when the RomFS has no SPIR-V file;
|
||||
- identity and name, the build row it found, the largest RomFS file as `romfs_marker`, big file types in `zip_store`, `shader_groups` by RomFS top-level folder;
|
||||
- `modules`: a subsdk is loaded only when `main` imports something from it that the host does not supply itself. Under the Vulkan layer NVIDIA's OpenGL / Vulkan drivers, the Vulkan loader and GLSLC are dropped; movie modules and a GLSLC that `main` calls stay;
|
||||
- `shader_main_glslc` and `shader_separable` when `main` holds GLSLC outputs and the RomFS has no BNSH / BAHS / UnityFS; `shader_glslc_capture` when `main` also holds GLSL sources and calls `glslcCompile` in a module the game ships;
|
||||
- the memory partition. Without a system resource in the NPDM: the 4 GB layout, its heap region the pool − 64 MB − the bss of giant modules (Metroid Dread: 0x90000000). When `main` never sets a heap: a 1 GB block (Super Mario Galaxy). With a system resource and a heap: Engage's 5 GB block with the hardware pool, which the probe corrects;
|
||||
- Engine presets: Hagi (`hagi.config`) gets `op_mode handheld` and `pad_layout full`.
|
||||
|
||||
@@ -89,7 +89,7 @@ The translator (`dmsh_xlate/Program.cs`) still has Mario's texture / sampler bas
|
||||
No short run decides `render_cond`, `tex_flush_upload`, `tex_dirty_upload`, `core_preempt`, `gpu_timestamps`, `feedback_snap`, `alpha_from_red`, `vconv_range` or `present_interval`, nor `op_mode` / `pad_layout` outside the presets: they keep their defaults for a developer. A minute's boot without input counted Wonder's clears under render conditions (65,536 or more; it needs no `render_cond`) as readily as Tears of the Kingdom's, saw Breath of the Wild flush 32 CPU-filled textures with data (it needs no `tex_flush_upload`) and Age of Calamity none (its movie frames come later), and Metroid Dread's movie crash shows in about half the runs. On the iPhone, `Documents/env.txt` can still set `DM_OPMODE`, `DM_NVN_GPU_TS` and `DM_NVN_FB_SNAP`, and the pad layout switches in game.
|
||||
|
||||
The shader tables, `dmsh_chain.h` and `c1_full.tsv` are written into the same folder. A second run reuses that `game.tsv` (delete it to survey again), so it can be edited by hand. `--provisional` builds from a provisional profile even for a game that has one in `dm/games`, to test the guesses: `DM_GAMES_FIRST=1` makes every tool search `DM_GAMES_DIR` first, and the IPA gets `-provisional` in its name. Tools find profiles outside `dm/games` through `DM_GAMES_DIR` (folders separated by `:`, searched after `dm/games`); `make -C dm/host` reaches such a profile through the symlink `dm/host/gen/<id>/profile`. To keep a game, move the folder into `dm/games/` and bring it up as below.
|
||||
2. **Profile.** Copy `010015100B514000/game.tsv` (or the provisional profile) to `dm/games/<title ID>/game.tsv` (the base game's title ID, as the folder name) and fill in the identity, `build` rows and `modules`. Leave the memory and NVN values as Mario's until the bring-up measures them ([docs/agents](../../docs/agents/README.md)).
|
||||
2. **Profile.** Copy `010015100B514000/game.tsv` (or the provisional profile) to `dm/games/<title ID>/game.tsv` (the base game's title ID, as the folder name) and fill in the identity, `build` rows and `modules`. Leave the memory values as Mario's until the bring-up measures them ([docs/agents](../../docs/agents/README.md)).
|
||||
3. **Shader tables.** Run `DM_GAME=<id> shader_split.py … dm/games/<id>/shader_map.tsv dm/games/<id>/shader_facts.tsv dm/games/<id>/shader_pairs.tsv`. Add an empty `shader_map.tsv.conflicts.tsv`, `bf_want.tsv` and `pso_manifest.tsv` (copy Mario's headers).
|
||||
4. **Chain.**
|
||||
1. `DM_GAME=<id> dmsh_chain.py --header`
|
||||
|
||||
@@ -10,7 +10,6 @@ import os, re, sys
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
STAGES = {'vs': 0, 'ps': 1, 'gs': 2, 'hs': 3, 'ds': 4, 'cs': 5} # NVN shader stage numbers
|
||||
|
||||
|
||||
class Game:
|
||||
@@ -89,8 +88,6 @@ def header(g):
|
||||
"""the host's game_profile.h: the profile constants the host uses, as macros"""
|
||||
q = lambda s: '"' + s.replace('\\', '\\\\').replace('"', '\\"') + '"'
|
||||
mods = g.words('modules')
|
||||
ssbo = dict(g.pairs('nvn_ssbo_bases'))
|
||||
sb = ' : '.join(f'(stage) == {STAGES[k]} ? {v}' for k, v in ssbo.items()) + ' : -1'
|
||||
vers = ', '.join('{' + q(b['main']) + ', ' + q(b['display']) + '}' for b in g.builds().values() if 'main' in b)
|
||||
pi = g.get('present_interval', 'game') # present interval: game = as the game sets it (as on the Switch), display = one frame per display tick
|
||||
if pi not in ('game', 'display'): sys.exit(f'{g.id}: present_interval must be game or display: {pi}')
|
||||
@@ -144,13 +141,6 @@ def header(g):
|
||||
f'#define DM_GAME_MEM_USED {int(g.get("mem_used", "0x4000000"), 0):#x}ull', # used memory reported beside the heap (hle_svc.c GetInfo UsedMemorySize)
|
||||
f'#define DM_GAME_ARGS {q(g.get("game_args", ""))}', # game arguments when the command line gives none (main.c; Octopath Traveler 0: Unreal's -SwitchVirtualAddressSpaceSize=)
|
||||
f'#define DM_GAME_MEM_MAP_LIMIT {int(mml == "on")}',
|
||||
f'#define DM_GAME_NVN_SSBO_BASE(stage) ({sb})',
|
||||
f'#define DM_GAME_NVN_TEX_BASE {g.int("nvn_tex_base"):#x}',
|
||||
f'#define DM_GAME_NVN_TEX_SLOTS {g.int("nvn_tex_slots")}',
|
||||
f'#define DM_GAME_NVN_SAMP_BASE {g.int("nvn_samp_base"):#x}',
|
||||
f'#define DM_GAME_NVN_IMG_BASE {g.int("nvn_img_base"):#x}',
|
||||
f'#define DM_GAME_NVN_IMG_SLOTS {g.int("nvn_img_slots")}',
|
||||
*([f'#define DM_GAME_NVN_SEPTEX_BASE {g.int("nvn_septex_base"):#x}', f'#define DM_GAME_NVN_SEPTEX_SLOTS {g.int("nvn_septex_slots")}'] if g.get('nvn_septex_base', '') else []), # a separate texture table of its own (Monster Hunter Rise)
|
||||
f'#define DM_GAME_DISPLAY_VERSIONS {vers}',
|
||||
f'#define DM_GAME_PRESENT_DISPLAY {int(pi == "display")}',
|
||||
f'#define DM_GAME_CORE_PREEMPT {int(cp == "on")}',
|
||||
|
||||
+36
-25
@@ -22,7 +22,6 @@ PACKED = {'UnityFS', 'UnityWeb', 'Yaz0', 'zstd', 'BWAV', 'CRI CPK', 'CRI USM', '
|
||||
# IL2CPP by the metadata file every IL2CPP player opens: Unity 2022's main (Ace Attorney Investigations Collection) has no il2cpp_ API names, Engage's 2020.3 has both
|
||||
MARKERS = [(b'nvnBootstrapLoader', 'NVN'), (b'vkCreateInstance', 'Vulkan'), (b'eglGetProcAddress', 'EGL / OpenGL'), (b'global-metadata.dat', 'Unity IL2CPP'), (b'UnityEngine', 'Unity'),
|
||||
(b'sead::', 'Nintendo sead'), (b'agl::', 'Nintendo agl'), (b'gsys::', 'Nintendo gsys'), (b'Unreal', 'Unreal'), (b'UE4', 'Unreal 4'), (b'Havok', 'Havok'), (b'CRIWARE', 'CRIWARE'), (b'Wwise', 'Wwise')]
|
||||
SHADER_CONTAINERS = ('BNSH', 'BAHS', 'UnityFS', 'Trinity pack') # RomFS files that hold a game's shaders: with them, GLSLC outputs in main are not its shaders (Engage's main has 11 it does not use)
|
||||
GLSLC_MAGIC = b'\x91\x68\x86\x19' # GLSLC compiler outputs (shader_split.py glslc_shaders)
|
||||
APP_POOL = 0xCD500000 # the application memory pool of a 4 GB Switch (3285 MiB), what every profile but Engage's reports
|
||||
MiB = 1 << 20
|
||||
@@ -133,22 +132,6 @@ def guess(F, hooks=None):
|
||||
else: drop.append(f'{name} ({mods[name]["kind"]})')
|
||||
G['modules'] = (' '.join(keep), f'not loaded, main takes nothing from them the host does not supply: {", ".join(drop)}' if drop else 'every module main imports from')
|
||||
|
||||
vk = sorted(s for s in imp if re.match(r'vk[A-Z]', s))
|
||||
vulkan = bool(vk) and 'nvnBootstrapLoader' not in imp
|
||||
if vulkan:
|
||||
G['graphics'] = ('vulkan', f'main imports {len(vk)} Vulkan functions and not nvnBootstrapLoader: the host\'s Vulkan layer (dm/gfx/vk/hle_vk.m)')
|
||||
if not R['mags']['SPIR-V']:
|
||||
G['shader_vk_capture'] = ('on', 'no SPIR-V file in the RomFS: the game generates its shaders at run time (Super Mario Galaxy), captured by booting it on the Mac')
|
||||
|
||||
if F['glslc'] and not any(R['mags'][m] for m in SHADER_CONTAINERS):
|
||||
why = f'main holds {F["glslc"]} GLSLC outputs and the RomFS has no {" / ".join(SHADER_CONTAINERS)}: the shaders are compiled into main (13 Sentinels, the KTGL games)'
|
||||
G['shader_main_glslc'] = ('on', why)
|
||||
G['shader_separable'] = ('on', 'with shader_main_glslc: single-stage programs combined per draw (every GLSLC output in main seen so far is one stage)')
|
||||
if R['mags']['Trinity pack']:
|
||||
G['shader_oodle'] = ('on', 'the RomFS has Trinity packs (ONEPACK, arc/data.trpfs: Pokémon Legends: Z-A), Oodle-compressed: the shader split decodes them with the game\'s own decoder in the Mac host (dm/host/hle_oodle.c, its address per build)')
|
||||
glslc_mod = [m for m in keep if 'glslcCompile' in mods[m]['exports']]
|
||||
if F['glsl_src'] and 'glslcCompile' in imp and glslc_mod:
|
||||
G['shader_glslc_capture'] = ('on', f'main holds {F["glsl_src"]} GLSL sources and calls glslcCompile in {glslc_mod[0]}: shaders compiled at run time (Fire Emblem: Three Houses), captured by booting it on the Mac')
|
||||
|
||||
# Memory: the guest block is 256 MB in front + alias region + stack region + heap region (hle_svc.c mem_init)
|
||||
giant = sum(m['bss'] for n, m in mods.items() if n in keep and m['far'])
|
||||
@@ -163,13 +146,8 @@ def guess(F, hooks=None):
|
||||
if heap_api else 'main never sets a heap (its memory is its own bss): a 256 MB heap region')
|
||||
G['mem_reserve'] = (f'{mem[0]:#x}', mem[3]); G['mem_alias'] = (f'{mem[1]:#x}', ''); G['mem_stack'] = ('0x10000000', ''); G['app_pool'] = (f'{mem[2]:#X}'.replace('0X', '0x'), '')
|
||||
|
||||
if 'Unity IL2CPP' in F.get('main_markers', ()):
|
||||
G['nvn_tex_base'] = ('0x168', 'Unity: separate texture bindings (Engage)'); G['nvn_tex_slots'] = ('128', '')
|
||||
if F.get('main_markers', set()) & {'Unreal', 'Unreal 4'} and R['exts'].get('.pak') and 'nvnBootstrapLoader' in imp:
|
||||
G['shader_ue_decode'] = ('on', 'Unreal Engine with paks: the shaders are Oodle-compressed in the paks\' shader code archives (Octopath Traveler 0), decoded by booting the game on the Mac')
|
||||
G['shader_separable'] = ('on', 'Unreal combines vertex and pixel shaders per draw from its pipeline state')
|
||||
G['shader_groups'] = ('ShaderArchive-Global-=global ShaderArchive-=game', 'the decoded archives: the engine\'s global shaders and the game\'s material shaders')
|
||||
G['nvn_tex_base'] = ('0x168', 'Unreal: separate texture bindings (Octopath Traveler 0, measured over all its shaders)'); G['nvn_tex_slots'] = ('128', '')
|
||||
G['shader_groups'] = ('ShaderArchive-Global-=global ShaderArchive-=game', 'the decoded shader code archives: the engine\'s global shaders and the game\'s material shaders')
|
||||
G['mem_reserve'] = ('0x1028000000', 'Unreal: nninitStartup reserves a 62 GB address region and its FMallocBinned3 takes 37 GB of it, so a 64 GB alias region for the bring-up on the Mac. '
|
||||
'An iPhone needs the allocator cut for the build and a capped region (game_args), as Octopath Traveler 0\'s profile (docs/agents/03)')
|
||||
G['mem_alias'] = ('0x1008000000', '')
|
||||
@@ -180,6 +158,40 @@ def guess(F, hooks=None):
|
||||
HEAP_API = '_ZN2nn2os17SetMemoryHeapSizeEm'
|
||||
|
||||
|
||||
# ---- Shader sources: decided from the game files at every build, not kept in the profile (scripts/build_ipa_from_nsp.py; shader_split.py reads what those steps write) ----
|
||||
def shader_sources(imports, glsl_src, glslc_mods, spirv, unreal_paks, trinity_packs=0):
|
||||
"""{name: (value, reason)}: graphics (vulkan when main imports Vulkan functions and not nvnBootstrapLoader, else nvn), vk_capture (a Vulkan game without SPIR-V files generates its shaders at
|
||||
run time), glslc_capture (main holds GLSL sources and calls glslcCompile in a module the game loads: it compiles some shaders at run time), ue_decode (Unreal paks keep the shaders Oodle-compressed
|
||||
in their shader code archives, which only the game's own Oodle decodes), oodle_packs (Trinity packs, Oodle-compressed file by file, which the shader split decodes with the game's own decoder in
|
||||
the Mac host's tool mode). Checked against every profile: Hades and Super Mario Galaxy Vulkan, Galaxy alone with no SPIR-V file, Three Houses alone compiling shaders (13 Sentinels holds 106 GLSL
|
||||
sources but does not import glslcCompile), Octopath Traveler 0 alone with Unreal paks, Pokémon Legends: Z-A alone with Trinity packs"""
|
||||
vk = sum(1 for s in imports if re.match(r'vk[A-Z]', s)); vulkan = bool(vk) and 'nvnBootstrapLoader' not in imports
|
||||
return {'graphics': ('vulkan', f'main imports {vk} Vulkan functions and not nvnBootstrapLoader: the host\'s Vulkan layer (dm/gfx/vk/hle_vk.m)') if vulkan else ('nvn', 'main imports nvnBootstrapLoader'),
|
||||
'vk_capture': (vulkan and not spirv, 'no SPIR-V file in the RomFS: the game generates its shaders at run time (Super Mario Galaxy), captured by booting it on the Mac'),
|
||||
'glslc_capture': (not vulkan and bool(glsl_src) and 'glslcCompile' in imports and bool(glslc_mods),
|
||||
f'main holds {glsl_src} GLSL sources and calls glslcCompile in {", ".join(glslc_mods) or "no module it loads"}: shaders compiled at run time (Fire Emblem: Three Houses), captured by booting it on the Mac'),
|
||||
'ue_decode': (not vulkan and bool(unreal_paks), f'{unreal_paks} Unreal pak(s): the shaders are Oodle-compressed in their shader code archives (Octopath Traveler 0), decoded by booting the game on the Mac'),
|
||||
'oodle_packs': (not vulkan and bool(trinity_packs), f'{trinity_packs} Trinity pack file(s) (ONEPACK, arc/data.trpfs: Pokémon Legends: Z-A), Oodle-compressed: the shader split decodes them with the game\'s own decoder in the Mac host (dm/host/hle_oodle.c DM_OODLE_SERVE)')}
|
||||
|
||||
|
||||
def build_sources(exefs, romfs, modules):
|
||||
"""shader_sources from an extraction (its ExeFS and RomFS folders) and the modules the profile loads"""
|
||||
exefs = Path(exefs); main = nso(exefs / 'main')[0]
|
||||
imports = {s for s, defined, _ in nso_syms.dynsyms(main) if not defined}; glsl_src = main.count(b'#version ')
|
||||
glslc_mods = [m for m in modules if m != 'main' and glsl_src and 'glslcCompile' in imports and (exefs / m).exists()
|
||||
and any(s == 'glslcCompile' and defined for s, defined, _ in nso_syms.dynsyms(nso(exefs / m)[0]))]
|
||||
spirv = 0
|
||||
if any(re.match(r'vk[A-Z]', s) for s in imports): # only a Vulkan game's RomFS is read
|
||||
spirv = sum(1 for _, _, head in romfs_files(romfs) if head.startswith(bytes.fromhex('03022307')))
|
||||
sys.path.insert(0, str(HERE.parent / 'gfx/tools')); import uepak
|
||||
trinity = 0
|
||||
for d, _, names in os.walk(romfs):
|
||||
for n in names:
|
||||
if n.endswith('.trpfs'):
|
||||
with open(os.path.join(d, n), 'rb') as f: trinity += f.read(8) == b'ONEPACK\0'
|
||||
return shader_sources(imports, glsl_src, glslc_mods, spirv, len(uepak.unreal_paks(str(romfs))), trinity)
|
||||
|
||||
|
||||
def va_heap(sysres, imports): # virtual address memory (an NPDM system resource) and a heap: the alias / heap split is a starting point the heap check corrects (Wonder, Tears of the Kingdom; Engage's pool is its own measurement)
|
||||
return bool(sysres) and HEAP_API in imports
|
||||
|
||||
@@ -228,8 +240,7 @@ def heap_signals(log, g):
|
||||
|
||||
|
||||
# ---- Checking the rules against the hand-made profiles ----
|
||||
DEFAULTS = {'graphics': 'nvn', 'shader_vk_capture': 'off', 'shader_main_glslc': 'off', 'shader_separable': 'off', 'shader_glslc_capture': 'off', 'shader_ue_decode': 'off', 'shader_oodle': 'off',
|
||||
'op_mode': 'console', 'pad_layout': 'simple', 'nvn_tex_base': '0x20', 'nvn_tex_slots': '32'}
|
||||
DEFAULTS = {'op_mode': 'console', 'pad_layout': 'simple'}
|
||||
NO_SIGNAL = {'pad_layout', 'op_mode'} # no static signal outside the engine presets; the player switches the pad layout in game, DM_OPMODE in env.txt sets the mode
|
||||
|
||||
|
||||
|
||||
+4
-5
@@ -54,10 +54,10 @@ The whole chain is driven by `tools/dmsh_build.py`, and `setup.sh` runs it for y
|
||||
- **`dm/games/<id>/pso_manifest.tsv`**: the warm-up seed manifest bundled with the app: shaders (`S`), render pipelines (`R`, the hex of `pso_skey` in `nvn/nvn_pso_skey.h`) and compute pipelines (`C`) to pre-build at startup. The row order is the pre-build order (first use); do not sort it. Its companion is `pso_user.tsv`, which the host appends next to the save directory whenever it has to build something on the spot, and which is pre-built after the seed at the next launch. Both name shaders by key and describe pipeline settings only, so a list recorded on one device works on any other; lines a build cannot use are skipped. Recorded lines can be appended to the seed below its header (currently empty).
|
||||
- **Shaders embedded in the program itself**: the main program additionally contains 4 compute shaders that are not in the shader package; they were hand-written as MSL from the disassembly and are in `nvn/nvn_builtin.h`.
|
||||
|
||||
Binding conventions followed during translation (file header of `tools/maxwell_refs.py`):
|
||||
Binding conventions followed during translation (the NVN driver's layout, the same in every game: `tools/dmsh_build.py`, filled at run time by `nvn/nvn_mtl.m`):
|
||||
|
||||
- uniform buffer r is constant section `c[3 + r]` in the machine code;
|
||||
- the handle of separate texture r is at `c[2][0x168 + 8r]`, and the handle of separate sampler r is at `c[2][0x568 + 8r]`;
|
||||
- the handle of combined texture r (`nvnCommandBufferBindTexture`) is at `c[2][0x20 + 8r]`, of image r at `c[2][0x120 + 8r]`, of separate texture r at `c[2][0x168 + 8r]` (the host's texture slot 128 + r) and of separate sampler r at `c[2][0x568 + 8r]`;
|
||||
- the address of storage buffer r is at `c[0][base + 16r]`, with base: vertex 0x110, compute 0x310, fragment 0x510;
|
||||
- the window origin is at the top left and the depth range is 0 to 1, the same as Metal, so nothing is flipped during translation.
|
||||
|
||||
@@ -89,11 +89,10 @@ The scripts in `tools/`; usage is written in each file's header:
|
||||
| `dmsh_xlate/` | The shader translation tool (C#), which references the translation library in `dm/third_party/ryujinx` |
|
||||
| `dmsh_build.py` | Drives the whole translation chain to produce the shader libraries; when the environment variable `IOSOUT=directory` is set, it also produces the iOS version |
|
||||
| `dmsh_chain.py`, `dmsh_chain.lock` | Translation chain version and tool fingerprints |
|
||||
| `dmsh_facts.py` | Generation and checking of the answer table `shader_facts.tsv` |
|
||||
| `dmsh_facts.py` | Reads the answer table `shader_facts.tsv` (which `shader_split.py` writes) |
|
||||
| `dmsh_fetch.py` | Generates the "readable tile" versions of fragment shaders |
|
||||
| `c1_guard.py` | Generates `c1_full.tsv` |
|
||||
| `dmsh_dryrun.py` | Dry-run conversion of the whole library, measuring only "can it be converted" |
|
||||
| `maxwell_refs.py` | Reads resource references from the machine code |
|
||||
| `gen_protos.py` | Generates `nvn_protos.h` / the thunks |
|
||||
| `tbdr_budget.py`, `tbdr_load_dead.py`, `tbdr_store_dead.py`, `tbdr_pass_merge.py` | Read the `DM_NVN_PASSDUMP` log and analyze the load / store volume of each render pass's attachments per frame |
|
||||
| `passgpu_top.py` | Reads the `DM_NVN_PASSGPU` output and lists GPU time by pass |
|
||||
@@ -109,7 +108,7 @@ The shader libraries are generated by `setup.sh`:
|
||||
|
||||
After changing a file on the translation chain, the order for rebuilding manually is the same as in `setup.sh`: `dmsh_chain.py --header` → `dmsh_build.py` → `c1_guard.py` → recompile the host.
|
||||
|
||||
The per-shader cache key in `dmsh_build.py` (`shader_key`) covers the profile's `nvn_*` / `shader_*` rows and the shader's pairs, and a paired vertex shader's key also takes its pixel shaders' keys (`pair_keyed`), since `vs_pad` reads their MSL. Before, these were only in the chain version, so a build after changing only them hit the old per-shader entries and filed them under the new version. `$DM_ROOT` defaults to `~/dm-work`.
|
||||
The per-shader cache key in `dmsh_build.py` (`shader_key`) covers the profile's `shader_*` rows and the shader's pairs, and a paired vertex shader's key also takes its pixel shaders' keys (`pair_keyed`), since `vs_pad` reads their MSL. Before, these were only in the chain version, so a build after changing only them hit the old per-shader entries and filed them under the new version. `$DM_ROOT` defaults to `~/dm-work`.
|
||||
|
||||
## Tests
|
||||
|
||||
|
||||
@@ -846,17 +846,11 @@ static p_bind_handles *bind_handles(NVNcommandBuffer *c, int op, int st, int fir
|
||||
NVN(nvnCommandBufferBindImages, void, NVNcommandBuffer *c, NVNshaderStage st, int first, int n, const NVNimageHandle *h) {
|
||||
bind_handles(c, OP_BIND_IMG, st, first, n, (const uint64_t *)h);
|
||||
}
|
||||
// Separate texture binds: on hardware they fill their own handle table (c2 0x168 + 8 slot), apart from BindTexture's. A game that uses both (Monster Hunter Rise: BindTexture slot 0 beside separate slots 0-4
|
||||
// in the same frames) has nvn_septex_base in its profile, and its separate binds go to slots 128 + n of the replay's table, which its shaders' reflection names; other games keep slots 0-127 for both
|
||||
#ifdef DM_GAME_NVN_SEPTEX_BASE
|
||||
#define SEPTEX_SLOT0 128
|
||||
#else
|
||||
#define SEPTEX_SLOT0 0
|
||||
#endif
|
||||
// Separate texture binds: on hardware they fill their own handle table (c2 0x168 + 8 slot), apart from BindTexture's, so they go to slots 128 + n (NVN_SEPTEX_SLOT0)
|
||||
NVN(nvnCommandBufferBindSeparateTextures, void, NVNcommandBuffer *c, NVNshaderStage st, int first, int n, const NVNseparateTextureHandle *h) {
|
||||
bind_handles(c, OP_BIND_TEX, st, SEPTEX_SLOT0 + first, n, (const uint64_t *)h);
|
||||
bind_handles(c, OP_BIND_TEX, st, NVN_SEPTEX_SLOT0 + first, n, (const uint64_t *)h);
|
||||
}
|
||||
NVN(nvnCommandBufferBindSeparateTexture, void, NVNcommandBuffer *c, NVNshaderStage st, int i, NVNseparateTextureHandle h) { uint64_t v; memcpy(&v, &h, 8); bind_handles(c, OP_BIND_TEX, st, SEPTEX_SLOT0 + i, 1, &v); } // single (Engage): as the plural version
|
||||
NVN(nvnCommandBufferBindSeparateTexture, void, NVNcommandBuffer *c, NVNshaderStage st, int i, NVNseparateTextureHandle h) { uint64_t v; memcpy(&v, &h, 8); bind_handles(c, OP_BIND_TEX, st, NVN_SEPTEX_SLOT0 + i, 1, &v); } // single (Engage): as the plural version
|
||||
NVN(nvnCommandBufferBindSeparateSamplers, void, NVNcommandBuffer *c, NVNshaderStage st, int first, int n, const NVNseparateSamplerHandle *h) {
|
||||
bind_handles(c, OP_BIND_SAMP, st, first, n, (const uint64_t *)h);
|
||||
}
|
||||
|
||||
+23
-26
@@ -24,14 +24,15 @@
|
||||
#include "nvn_refl.h"
|
||||
#include "nvn_cover.h"
|
||||
#include "dmsh_chain.h" // DMSH_CHAIN: the translation chain version of the B shader library expected by this source tree (in the game profile folder, -I from the Makefile)
|
||||
#include "game_profile.h" // game profile constants (NVN driver constant segment bases, profile id)
|
||||
// Texture slot table per stage: 0-127 the slots of the bound / combined handle table (nvnCommandBufferBindTexture; for a profile without nvn_septex_base also the separate texture binds),
|
||||
// 128-255 the separate texture binds of a profile with nvn_septex_base (Monster Hunter Rise binds both kinds, the hardware keeps them in two tables: c2 0x20 and c2 0x168)
|
||||
#include "game_profile.h" // game profile constants (profile id, the profile's switches)
|
||||
// Texture slot table per stage: 0-127 the combined handles (nvnCommandBufferBindTexture), 128-255 the separate texture binds (NVN_SEPTEX_SLOT0, nvn_obj.h)
|
||||
#define NTEX 256
|
||||
_Static_assert(DM_GAME_NVN_TEX_SLOTS <= 128 && DM_GAME_NVN_IMG_SLOTS <= 32 && DM_GAME_NVN_TEX_BASE + 8 * DM_GAME_NVN_TEX_SLOTS <= 0xA00 && DM_GAME_NVN_IMG_BASE + 8 * DM_GAME_NVN_IMG_SLOTS <= 0xA00 && DM_GAME_NVN_SAMP_BASE + 8 * 128 <= 0xA00, "the profile's NVN handle tables exceed the driver segment buffer (0xA00) or the binding tables");
|
||||
#ifdef DM_GAME_NVN_SEPTEX_BASE
|
||||
_Static_assert(DM_GAME_NVN_SEPTEX_SLOTS <= 128 && DM_GAME_NVN_SEPTEX_BASE + 8 * DM_GAME_NVN_SEPTEX_SLOTS <= 0xA00, "the profile's separate texture table exceeds the driver segment buffer (0xA00) or the slot table");
|
||||
#endif
|
||||
// The NVN driver's constant segments, one layout in every game (dm/gfx/tools/dmsh_build.py), byte offsets: c2 holds the combined handles at 0x20 + 8 r (41 slots up to the separate table:
|
||||
// past slot 31 they lie in the image table, 0x120 + 8 r, 8 slots), the separate textures at 0x168 + 8 r and the separate samplers at 0x568 + 8 r (128 each);
|
||||
// c0 the storage buffer table at base + 0x10 r (address 64 bits, size 32 bits), base vertex 0x110, compute 0x310, pixel 0x510. The replay fills one DRV_SIZE buffer per segment a shader reads
|
||||
enum { C2_TEX = 0x20, C2_TEX_SLOTS = 41, C2_IMG = 0x120, C2_IMG_SLOTS = 8, C2_SEPTEX = 0x168, C2_SAMP = 0x568, DRV_SIZE = 0xA00 };
|
||||
_Static_assert(C2_TEX + 8 * C2_TEX_SLOTS == C2_SEPTEX && C2_SEPTEX + 8 * 128 == C2_SAMP && C2_SAMP + 8 * 128 <= DRV_SIZE && NVN_SEPTEX_SLOT0 + 128 == NTEX, "NVN driver handle tables");
|
||||
static int c0_ssbo_base(int stage) { return stage == 0 ? 0x110 : stage == 1 ? 0x510 : stage == 5 ? 0x310 : -1; } // NVN stages 0 vertex, 1 pixel, 5 compute; geometry / hull / domain never seen
|
||||
#include "nvn_hmap.h"
|
||||
#include "../../host/dm.h"
|
||||
#import <CommonCrypto/CommonDigest.h>
|
||||
@@ -3768,10 +3769,10 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
if (!t) { t = dummy_tbuf(b->dpf); trid = g_dummy_tbuf_rid[b->dpf & 3]; }
|
||||
slots[aid] = trid;
|
||||
USE(t, MTLResourceUsageRead);
|
||||
} else if (b->kind == NVN_RB_TEXD) { // dynamic handle array (profile key tex_dyn_slots): element k = the texture in slot k, since c2 holds slot indices for this shader (see DRV2 below)
|
||||
} else if (b->kind == NVN_RB_TEXD) { // dynamic handle array (profile key tex_dyn_slots): element k = the texture in slot k (combined, else separate), since c2 holds slot indices for this shader (see DRV2 below)
|
||||
uint64_t real = 0; // the elements that got a texture of the declared kind (logged on the shader's first encode: logged bit 0)
|
||||
for (int k = 0; k < b->n; k++) {
|
||||
int tid = k < NTEX ? (int)(S.tex[nvn_stage][k] & 0xfffff) : 0;
|
||||
uint64_t hk = k < 128 ? (S.tex[nvn_stage][k] ? S.tex[nvn_stage][k] : S.tex[nvn_stage][NVN_SEPTEX_SLOT0 + k]) : 0; int tid = (int)(hk & 0xfffff);
|
||||
void *tv = NULL; uint64_t trid = tid ? tex_fast_rid(tid, &tv) : 0; __unsafe_unretained id<MTLTexture> t = (__bridge id<MTLTexture>)tv;
|
||||
if (b->dtt == (uint8_t)MTLTextureTypeTextureBuffer) {
|
||||
if (t && t.textureType != MTLTextureTypeTextureBuffer) t = nil; // a slot the shader does not pick here may hold another kind of texture
|
||||
@@ -3816,13 +3817,13 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
USE(support_buf(), MTLResourceUsageRead);
|
||||
} else if (b->kind == NVN_RB_DRV0 || b->kind == NVN_RB_DRV2) {
|
||||
// NVN driver constant segments: those proven unread by static analysis at packaging time are supplied as 0; those read are filled on the spot from the current bindings - c0 = storage buffer table (base + 0x10 r: address 64 bits, size 32 bits),
|
||||
// c2 = texture handle table (0x168 + 8 r) and sampler handle table (0x568 + 8 r), the values being the NVN handles the game gave at bind time; positions read outside the tables are registered at packaging time and reported a failure at load time
|
||||
// c2 = the handle tables (C2_*), the values being the NVN handles the game gave at bind time; positions read outside the tables are registered at packaging time and reported a failure at load time
|
||||
int bank = b->kind == NVN_RB_DRV2 ? 2 : 0;
|
||||
if (!sh->rf.drv_read[bank]) { slots[aid] = zero_bank_gpu(); USE(zero_bank(), MTLResourceUsageRead); }
|
||||
else {
|
||||
id<MTLBuffer> db; size_t doff = arena_alloc(0xA00, &db); uint8_t *d = (uint8_t *)db.contents + doff; memset(d, 0, 0xA00);
|
||||
id<MTLBuffer> db; size_t doff = arena_alloc(DRV_SIZE, &db); uint8_t *d = (uint8_t *)db.contents + doff; memset(d, 0, DRV_SIZE);
|
||||
if (bank == 0) {
|
||||
int base = DM_GAME_NVN_SSBO_BASE(nvn_stage); // game profile nvn_ssbo_bases (Mario: vertex 0x110, pixel 0x510, compute 0x310)
|
||||
int base = c0_ssbo_base(nvn_stage);
|
||||
if (base < 0) RED_ONCE(45, "stage %d shader reads driver segment c0; the storage buffer table base was never measured", nvn_stage);
|
||||
else for (int r = 0; r < 16; r++) { uint64_t a = S.ssbo[nvn_stage][r].addr; uint32_t n = (uint32_t)S.ssbo[nvn_stage][r].size; memcpy(d + base + 16 * r, &a, 8); memcpy(d + base + 16 * r + 8, &n, 4); }
|
||||
// Compute also has the dispatch size at 0x200 (x, y, z) and the uniform buffer table at 0x210 + 16 r (address 64 bits, size 32 bits), ending where the storage buffer table
|
||||
@@ -3843,20 +3844,16 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
for (int r = 0; r < 16; r++) { uint64_t a = S.ubo[5][r].addr; uint32_t n = (uint32_t)S.ubo[5][r].size; memcpy(d + 0x210 + 16 * r, &a, 8); memcpy(d + 0x210 + 16 * r + 8, &n, 4); }
|
||||
}
|
||||
} else {
|
||||
for (int r = 0; r < DM_GAME_NVN_TEX_SLOTS; r++) { uint32_t th = (uint32_t)S.tex[nvn_stage][r]; memcpy(d + DM_GAME_NVN_TEX_BASE + 8 * r, &th, 4); }
|
||||
#ifdef DM_GAME_NVN_SEPTEX_BASE // the separate texture table (Monster Hunter Rise: c2 0x168 + 8 r) from the slots after the bound ones
|
||||
for (int r = 0; r < DM_GAME_NVN_SEPTEX_SLOTS; r++) { uint32_t th = (uint32_t)S.tex[nvn_stage][128 + r]; memcpy(d + DM_GAME_NVN_SEPTEX_BASE + 8 * r, &th, 4); }
|
||||
#endif
|
||||
for (int r = 0; r < DM_GAME_NVN_IMG_SLOTS; r++) { uint32_t ih = (uint32_t)S.img[nvn_stage][r]; memcpy(d + DM_GAME_NVN_IMG_BASE + 8 * r, &ih, 4); } // This game (NVN 55): texture handle table at 0x20 + 8 r (dm/gfx/tools/dmsh_build.py TEX_BASE)
|
||||
// separate sampler handle table (older driver layout such as Engage's: textures 0x168 + 8 r, samplers 0x568 + 8 r; Mario uses no separate samplers, so this is all zero)
|
||||
for (int r = 0; r < 128; r++) { uint32_t sh_ = (uint32_t)S.samp[nvn_stage][r]; memcpy(d + DM_GAME_NVN_SAMP_BASE + 8 * r, &sh_, 4); }
|
||||
// A shader with dynamic handle arrays (TEXD / SAMPD, profile key tex_dyn_slots) picks a table slot at run time and ORs a texture word with a sampler word into a handle,
|
||||
// whose texture / sampler IDs index the arrays. Here the tables hold slot indices instead (texture slot r → r, and r << 20 when its handle carries a sampler;
|
||||
// sampler slot r → r << 20), and the arrays hold the bound slots, so the shader's own arithmetic picks the binding (Octopath Traveler 0: texture slots 0-10, samplers 0-4)
|
||||
if (sh->rf.dyn) {
|
||||
for (int r = 0; r < DM_GAME_NVN_TEX_SLOTS; r++) { uint64_t h = S.tex[nvn_stage][r]; uint32_t v = !h ? 0 : (uint32_t)r | (h >> 20 ? (uint32_t)r << 20 : 0); memcpy(d + DM_GAME_NVN_TEX_BASE + 8 * r, &v, 4); }
|
||||
for (int r = 0; r < 128; r++) { uint32_t v = S.samp[nvn_stage][r] ? (uint32_t)r << 20 : 0; memcpy(d + DM_GAME_NVN_SAMP_BASE + 8 * r, &v, 4); }
|
||||
}
|
||||
// whose texture / sampler IDs index the arrays. For it the tables hold slot indices instead of handles (texture slot r → r, and r << 20 when a combined handle carries
|
||||
// a sampler; sampler slot r → r << 20), and the arrays hold the bound slots, so the shader's own arithmetic picks the binding (Octopath Traveler 0: separate texture slots 0-10, samplers 0-4)
|
||||
const uint64_t *tx = S.tex[nvn_stage], *im = S.img[nvn_stage]; int dyn = sh->rf.dyn;
|
||||
for (int r = 0; r < C2_IMG_SLOTS; r++) { uint32_t ih = (uint32_t)im[r]; memcpy(d + C2_IMG + 8 * r, &ih, 4); }
|
||||
for (int r = 0; r < C2_TEX_SLOTS; r++) { // a combined slot past 31 is written when it is bound and the image slot it shares is not (Hades II binds texture slot 34, no image)
|
||||
if (r >= 32 && (!tx[r] || (r - 32 < C2_IMG_SLOTS && im[r - 32]))) continue;
|
||||
uint32_t v = !dyn ? (uint32_t)tx[r] : !tx[r] ? 0 : (uint32_t)r | (tx[r] >> 20 ? (uint32_t)r << 20 : 0); memcpy(d + C2_TEX + 8 * r, &v, 4); }
|
||||
for (int r = 0; r < 128; r++) { uint64_t h = tx[NVN_SEPTEX_SLOT0 + r]; uint32_t v = !dyn ? (uint32_t)h : h ? (uint32_t)r : 0; memcpy(d + C2_SEPTEX + 8 * r, &v, 4); }
|
||||
for (int r = 0; r < 128; r++) { uint64_t h = S.samp[nvn_stage][r]; uint32_t v = !dyn ? (uint32_t)h : h ? (uint32_t)r << 20 : 0; memcpy(d + C2_SAMP + 8 * r, &v, 4); }
|
||||
}
|
||||
slots[aid] = g_cur_arena->gpu + doff;
|
||||
USE(db, MTLResourceUsageRead);
|
||||
@@ -4956,7 +4953,7 @@ planned:
|
||||
int touches = 0;
|
||||
for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr2->sh[st]; if (!sh || sh->stage >= 6) continue;
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *bb = &sh->rf.b[i]; uint64_t h;
|
||||
if (bb->kind == NVN_RB_IMG) h = bb->reg < 32 ? i2[sh->stage][bb->reg] : 0; else if (bb->kind == NVN_RB_TEX || bb->kind == NVN_RB_TBUF) h = bb->reg < 128 ? t2[sh->stage][bb->reg] : 0; else continue;
|
||||
if (bb->kind == NVN_RB_IMG) h = bb->reg < 32 ? i2[sh->stage][bb->reg] : 0; else if (bb->kind == NVN_RB_TEX || bb->kind == NVN_RB_TBUF) h = bb->reg < NTEX ? t2[sh->stage][bb->reg] : 0; else continue;
|
||||
uint64_t ser = h ? h_ser(h) : 0; if (!ser) continue;
|
||||
int j = -1; for (int m = 0; m < R.rt.n; m++) if (R.rt.color[m] && R.rt.color[m]->serial == ser) { j = m; break; }
|
||||
if (j < 0) continue;
|
||||
|
||||
@@ -119,6 +119,9 @@ typedef struct { int32_t first, n; NVNbufferRange r[16]; } p_bind_vbuf;
|
||||
typedef struct { int32_t stage, index; uint64_t addr, size; } p_bind_buf; // UBO / SSBO
|
||||
// defer / src: the handles of BindTexturesDeferred are read from src at execution time (h at recording time is only a fallback); the replay layer refreshes them with bh_fresh before reading the record (nvn_mtl.m)
|
||||
typedef struct { int32_t stage, first, n, defer; uint64_t src; uint64_t h[32]; } p_bind_handles; // Separate texture / sampler handles
|
||||
// Texture slots of a stage: 0-127 nvnCommandBufferBindTexture's (combined handles), 128-255 the separate texture binds. The driver keeps the two in tables of their own (c2 0x20 + 8 r and
|
||||
// 0x168 + 8 r, dm/gfx/tools/dmsh_build.py), which the shaders' reflection names as these slots, so a game may bind both kinds in the same frame (Monster Hunter Rise: BindTexture slot 0 beside separate slots 0-4)
|
||||
#define NVN_SEPTEX_SLOT0 128
|
||||
typedef struct { int32_t face, value_mask, mask, ref, which; } p_stencil; // which: 0 value mask 1 write mask 2 reference value
|
||||
typedef struct { float factor, units, clamp; } p_poly_offset;
|
||||
typedef struct { int32_t prim, first, count, instances, base_instance; uint64_t ind; uint64_t mdi_cnt; int32_t mdi_i, pad; } p_draw; // instances 0 counts as 1; ind != 0: DrawArraysIndirect, the arguments are read by the GPU from that address; mdi_cnt != 0: draw mdi_i of an indirect-count multi-draw (drawn only while mdi_i < the u32 at mdi_cnt)
|
||||
|
||||
@@ -41,7 +41,7 @@ typedef struct {
|
||||
int8_t pos_attr; uint8_t pos_w1, may_discard, writes_smask, color_mask, color_known; // color_known: the color list was recorded at packaging (color_mask can be trusted)
|
||||
// Merge B step 2 (readable-tile version <key>_bf, dmsh_fetch.py): fragment - nfc redirectable samples (the i-th: texture slot fc_reg, interpolated output fc_locn),
|
||||
// bitmap fc_mixed of texture slots with mixed use (these slots must not be redirected to read the tile), nfo outputs (each with color number fo_color); vertex - .xy of interpolated output locn is taken as-is from attribute uv_attr[locn] (-1 = no)
|
||||
uint8_t nfc, fc_reg[32], fc_locn[32], nfo, fo_color[4]; uint64_t fc_mixed[4]; int8_t uv_attr[32]; // fc_mixed: a bit per texture slot 0-255 (slots 128+ are separate texture binds, nvn_septex_base)
|
||||
uint8_t nfc, fc_reg[32], fc_locn[32], nfo, fo_color[4]; uint64_t fc_mixed[4]; int8_t uv_attr[32]; // fc_mixed: a bit per texture slot 0-255 (slots 128+ are separate texture binds, NVN_SEPTEX_SLOT0)
|
||||
// Vertex - .x / .y of interpolated output locn = fma(.x / .y of attribute uv_aff_attr[locn], scale, offset), where scale / offset reference a uniform: (slot << 8) | (vector << 2) | component
|
||||
int8_t uv_aff_attr[32]; uint16_t uv_aff[32][4]; // [sx, ox, sy, oy]
|
||||
uint8_t *c1; // c1 constant segment as-is (B): c1_len bytes, malloc'd by nvn_refl_from_json (NULL without one). It was a 4 KB array; Hyrule Warriors: Age of Calamity has 6 KB segments (its soldiers' vertex shaders)
|
||||
|
||||
+24
-51
@@ -1,9 +1,8 @@
|
||||
#!/usr/bin/env python3
|
||||
# Route B offline build: original Switch shaders → SPIR-V (dmsh_xlate, the Ryujinx translator) → spirv-cross MSL → metallib + replay reflection JSON.
|
||||
# No translation at runtime (iOS has no JIT): build output goes into the shader cache directory, file name = Switch control segment key (dm_ctl_sha16, same as column 4 of shader_map.tsv) + "_b".
|
||||
# Source of query answers: the answer table in the game profile, dm/games/<id>/shader_facts.tsv (dmsh_facts.py) - texture types, vertex attribute / fragment output component types, compute thread group size and shared memory; the build needs no PC version files.
|
||||
# Usage: dmsh_build.py <Switch package dir> <output cache dir> [key...] (no keys = all); build output is actually written to <output cache dir>/b-<chain version>/ (same for IOSOUT), for the chain version see dmsh_chain.py Environment: DM_GAME=game profile (default smbw, dm/games/), XLATE=path to the translation tool (default the in-repo dmsh_xlate_bin), DMSH_CAS=content-addressed cache dir (default <output cache dir>/cas), DMSH_PREFILL=Mac library root[=iOS library root] (separated by :, their b-*/ go into the MSL-keyed compile cache first; sibling b-*/ always do), J=concurrency
|
||||
# Graphics shaders embedded in the executable itself (mapping table rows main.rodata@offset) are extracted from MAIN (Switch main NSO, default 3.3.94012 Update/ExeFS/main); extracting the NSO needs lz4, run with dm's venv python
|
||||
# Source of query answers: the answer table in the game profile, dm/games/<id>/shader_facts.tsv (dmsh_facts.py) - texture types, vertex attribute / fragment output component types, compute thread group size and shared memory.
|
||||
# Usage: dmsh_build.py <Switch package dir> <output cache dir> [key...] (no keys = all); build output is actually written to <output cache dir>/b-<chain version>/ (same for IOSOUT), for the chain version see dmsh_chain.py Environment: DM_GAME=game profile (required, dm/games/<id>), XLATE=path to the translation tool (default the in-repo dmsh_xlate_bin), DMSH_CAS=content-addressed cache dir (default <output cache dir>/cas), DMSH_PREFILL=Mac library root[=iOS library root] (separated by :, their b-*/ go into the MSL-keyed compile cache first; sibling b-*/ always do), J=concurrency
|
||||
# Build output: <key>_b.metallib, <key>_b.json (replay reflection, including drv_reads, the runtime read offsets into driver segments c0 / c2), <key>_b.metal (MSL)
|
||||
# Environment IOSOUT=dir: the same MSL also goes through dm/ios/tools/ios_metal.compile to produce an iphoneos metallib + reflection (chain_metal records the iOS compile convention)
|
||||
# Final summary: success / failure counts and the distribution of reasons; distribution of driver segment read offsets (those outside the range the replay layer can fill are listed separately)
|
||||
@@ -15,7 +14,7 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))); import dmsh_chai
|
||||
_bad = dmsh_chain.check_tools()
|
||||
if _bad: sys.exit('\n'.join('[RED] translation chain tools do not match the lock file: ' + b for b in _bad) + '\n(if the tools changed, run dmsh_chain.py --update-lock and commit the lock file; if the translator just was not rebuilt, rebuild it)')
|
||||
CHAIN = dmsh_chain.version(); out = os.path.join(out, 'b-' + CHAIN)
|
||||
GAME = dmsh_chain.GAME # the current game profile (DM_GAME, default smbw): the tables and NVN constants come from here
|
||||
GAME = dmsh_chain.GAME # the current game profile (DM_GAME): the tables come from here
|
||||
if f'"{CHAIN}"' not in open(dmsh_chain.HEADER).read(): sys.exit(f'[RED] {dmsh_chain.HEADER} does not hold the current translation chain version {CHAIN}: run dmsh_chain.py --header first (otherwise the host cannot find these libraries)')
|
||||
XLATE = os.environ.get('XLATE', os.path.join(dmsh_chain.XLATE_DIR, 'dmsh_xlate'))
|
||||
FACTS = dmsh_facts.load()
|
||||
@@ -24,7 +23,6 @@ IOSOUT = os.environ.get('IOSOUT') # if set, also build the iOS version (iphone
|
||||
if IOSOUT:
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '../../ios/tools')); import ios_metal
|
||||
IOSOUT = os.path.join(IOSOUT, 'b-' + CHAIN); os.makedirs(IOSOUT, exist_ok=True)
|
||||
MAIN = os.environ.get('MAIN', f'{DM_ROOT}/data/switch/3.3.94012/Update/ExeFS/main')
|
||||
MAP = str(GAME.table('shader_map.tsv'))
|
||||
BF_WANT_FILE = str(GAME.table('bf_want.tsv'))
|
||||
BF_WANT = {line.strip() for line in open(BF_WANT_FILE) if line.strip() and not line.lstrip().startswith('#')} if os.path.exists(BF_WANT_FILE) else set()
|
||||
@@ -39,18 +37,15 @@ MAC_RECORD = ' '.join(MAC_METAL) + '; xcrun -sdk macosx metallib'
|
||||
def mac_compile(msl, air, lib): # same interface as ios_metal.compile
|
||||
subprocess.run([*MAC_METAL, msl, '-o', air], check=True)
|
||||
subprocess.run(['xcrun', '-sdk', 'macosx', 'metallib', air, '-o', lib], check=True)
|
||||
# Texture handle table (the driver segment holding texture instructions' immediate handles): Mario (NVN 55) uses 0x20 + 8 r, 32 slots - the immediate handles of all 4888 shaders are exactly 0x20..0xF0 in steps of 8 (D2R's older driver used 0x168).
|
||||
# This game uses only combined handles (nvnCommandBufferBindTexture), with no separate samplers; the image handle table is still at 0x120 + 8 r (8 slots, right after the texture table; measured 0x120 / 0x128 / 0x130 on agl's 10 compute shaders, the same address as D2R).
|
||||
TEX_BASE, TEX_SLOTS, SAMP_BASE, IMG_BASE, IMG_SLOTS = (GAME.int(k) for k in ('nvn_tex_base', 'nvn_tex_slots', 'nvn_samp_base', 'nvn_img_base', 'nvn_img_slots'))
|
||||
SSBO = {k: int(v, 0) for k, v in GAME.pairs('nvn_ssbo_bases')} # c0 storage buffer table bases (per stage)
|
||||
# A separate texture table of its own (Monster Hunter Rise: bound / combined handles at 0x20 + 8 r, separate texture binds at 0x168 + 8 r, both in use): its slots are reported as 128 + r, where
|
||||
# the replay keeps the separate binds of such a profile (hle_nvn.c SEPTEX_SLOT0). Profiles without the key have one texture table (nvn_tex_base) for both kinds of bind
|
||||
SEPTEX_BASE, SEPTEX_SLOTS = (GAME.int('nvn_septex_base'), GAME.int('nvn_septex_slots')) if GAME.get('nvn_septex_base', '') else (None, 0)
|
||||
# The translator (dmsh_xlate/Program.cs) has the texture / sampler handle table bases built in, used only to turn a handle into a slot and look up that slot's texture / image type in the answer table (TexSlot); the bindings' slots are decoded by this script from the profile (decode_tex).
|
||||
# So the profile's bases must equal the translator's only when the answer table has per-slot texture / image facts (otherwise the wrong slot is looked up); Engage (0x168 / 0x568) has no such facts, so the translator needs no change
|
||||
_xb = re.search(r'const int TexBase = (0x[0-9a-fA-F]+), SampBase = (0x[0-9a-fA-F]+);', open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'dmsh_xlate/Program.cs')).read())
|
||||
if (not _xb or (int(_xb.group(1), 16), int(_xb.group(2), 16)) != (TEX_BASE, SAMP_BASE)) and any(f.get('tex') or f.get('img') for f in FACTS.values()):
|
||||
sys.exit(f'[RED] game profile {GAME.id}: nvn_tex_base / nvn_samp_base ({TEX_BASE:#x} / {SAMP_BASE:#x}) differ from the values built into the translator (dmsh_xlate/Program.cs TexBase / SampBase), and the answer table has per-slot texture / image facts: make the translator take the values from the profile first')
|
||||
# The NVN driver's constant segments have one layout in every game (NVN 53 and 55 drivers, SDK 6 to 21: the handles of every shader of every profile fit it); nvn_mtl.m fills them the same way at run time.
|
||||
# c2 0x20 + 8 r: handles bound with nvnCommandBufferBindTexture (combined texture and sampler), slot r. The driver has 32 such slots, but a game may bind higher ones, whose words lie in the
|
||||
# image table (Hades II binds slot 34, at 0x130, and no image), so a texture instruction's handle anywhere below the separate texture table names a combined slot
|
||||
# c2 0x120 + 8 r: images (nvnCommandBufferBindImage), slot r < 8
|
||||
# c2 0x168 + 8 r: separate textures (nvnCommandBufferBindSeparateTexture), reported as slot 128 + r, where the host binds them (hle_nvn.c, NVN_SEPTEX_SLOT0); Unity, Unreal and RE Engine bind these
|
||||
# c2 0x568 + 8 r: separate samplers (nvnCommandBufferBindSeparateSampler), slot r < 128
|
||||
# c0 base + 0x10 r: storage buffer r (64-bit address, 32-bit size), base vertex 0x110, compute 0x310, pixel 0x510 (the hull / domain stages' bases were never seen)
|
||||
TEX_BASE, IMG_BASE, IMG_SLOTS, SEPTEX_BASE, SEPTEX_SLOT0, SAMP_BASE = 0x20, 0x120, 8, 0x168, 128, 0x568
|
||||
SSBO = {'vs': 0x110, 'cs': 0x310, 'ps': 0x510}
|
||||
|
||||
def records(d):
|
||||
r = [m.start() for m in re.finditer(b'BTCDBTCD', d)]
|
||||
@@ -92,8 +87,7 @@ def shader_key(key, name, rec):
|
||||
MSL. Before, these were only in the chain version, so a build after changing only them hit the old entries and filed them under the new version (with an empty DMSH_CAS as the workaround)"""
|
||||
pc = dmsh_facts.row(key, FACTS[key]).encode() if key in FACTS else 'missing'.encode()
|
||||
map_row = MAP_ROWS.get(key, 'missing'.encode()) + MAP_CONFLICT_ROWS.get(key, b'')
|
||||
embedded = rec if name.startswith('main.rodata@') else b''
|
||||
payload = (b'record\0' + rec + b'\0map\0' + map_row + b'\0pc\0' + pc + b'\0embedded\0' + embedded +
|
||||
payload = (b'record\0' + rec + b'\0map\0' + map_row + b'\0pc\0' + pc +
|
||||
b'\0tools\0' + TOOL_FP + b'\0code\0' + CODE_FP.encode() +
|
||||
b'\0rule\0' + shader_rule_source(name) + b'\0profile\0' + PROFILE_ROWS + b'\0pairs\0' + ','.join(sorted(PAIRS.get(key, ()))).encode())
|
||||
return hashlib.sha256(payload).hexdigest()
|
||||
@@ -229,7 +223,7 @@ def fetch_finish(key, src, refl):
|
||||
write_refl(key, refl)
|
||||
return refl
|
||||
|
||||
pc_of, loc, stage_of = {}, {}, {}
|
||||
pc_of, loc = {}, {}
|
||||
MAP_ROWS, MAP_CONFLICT_ROWS = {}, {}
|
||||
for raw in open(MAP, 'rb'):
|
||||
c = raw.rstrip(b'\n').split(b'\t')
|
||||
@@ -240,21 +234,12 @@ for raw in open(MAP + '.conflicts.tsv', 'rb'):
|
||||
for ln in open(MAP):
|
||||
c = ln.rstrip('\n').split('\t')
|
||||
if len(c) > 4 and c[0].endswith('.bin'): pc_of.setdefault(c[3], c[4]); loc.setdefault(c[3], (c[0], int(c[1])))
|
||||
if len(c) > 4 and c[0].startswith('main.rodata@'): # embedded: location = main image offset (start of the control segment), stage taken from column 3 of the mapping table
|
||||
pc_of.setdefault(c[3], c[4]); loc.setdefault(c[3], (c[0], int(c[0].split('@')[1], 16))); stage_of[c[3]] = {'vertex': 'vs', 'pixel': 'ps'}[c[2]]
|
||||
for ln in open(MAP + '.conflicts.tsv'):
|
||||
c = ln.rstrip('\n').split('\t')
|
||||
if len(c) > 1 and not ln.startswith('#') and not ln.startswith('dm_') and c[0] in pc_of: pc_of[c[0]] = c[1]
|
||||
|
||||
def main_rec(mi, o):
|
||||
"""Embedded shader → bytes in the same layout as a package record: in a package record the code area starts at +0x38 (0x30 header + SPH + instructions, marker 78563412), immediately followed by the control segment (34127698);
|
||||
the embedded code area sits immediately before the control segment (checked 5: code area length 0x100–0x300, c1 empty in all, consistent with the layout of the same-named package records)"""
|
||||
g = mi.rfind(bytes.fromhex('78563412'), o - 0x1000, o)
|
||||
if g < 0 or mi[o:o + 4] != bytes.fromhex('34127698'): sys.exit(f'[RED] main image @{o:#x} is not the control section of an embedded shader (MAIN={MAIN})')
|
||||
return bytes(0x38) + mi[g:o + 0x880]
|
||||
|
||||
def job_for(key, name, rec, w):
|
||||
ci = rec.find(bytes.fromhex('34127698')); code = rec[0x38:ci]; stage = stage_of.get(key, name[:2])
|
||||
ci = rec.find(bytes.fromhex('34127698')); code = rec[0x38:ci]; stage = name[:2]
|
||||
c1o, c1t = struct.unpack_from('<2I', rec, ci + 0x700) # control segment +0x700: c1 relative to the start of the instruction area, +0x704: total length of the instruction area (c1 = what lies between the two; lengths across the whole library are 0 / 256 / 512 / 768)
|
||||
c1 = rec[0x38 + c1o:0x38 + c1t]
|
||||
if stage != 'cs': code = code[0x30:] # strip the 0x30 header (the 0x50 SPH after it is read by the translator itself)
|
||||
@@ -267,7 +252,7 @@ def job_for(key, name, rec, w):
|
||||
tf = tf_layout(key, rec[ci:]) if stage == 'vs' else None
|
||||
j = {'key': key, 'stage': stage, 'code': cp, 'tex': {}, 'img': {}, 'attr': {}, 'fout': {}, 'c1': c1.hex(), **({'tf': tf} if tf else {}),
|
||||
'debug': os.environ.get('XLATE_DEBUG', '1') != '0', 'reduced': os.environ.get('XLATE_REDUCED', '1') == '1'} # translation options (see dmsh_xlate/Program.cs); relaxed precision is on by default (GPU float last-bit error does not enter game state, whole frame −26% on the second Mac) # c1: used by the translator to read indirect jump tables
|
||||
if key not in FACTS: j['pc_missing'] = True; return j, c1 # not in the answer table: queries use defaults (listed separately in the summary)
|
||||
if key not in FACTS: j['no_facts'] = True; return j, c1 # not in the answer table: queries use defaults (listed separately in the summary)
|
||||
j.update(FACTS[key])
|
||||
return j, c1
|
||||
|
||||
@@ -359,9 +344,8 @@ def decode_tex(name):
|
||||
if re.match(r'[a-z]+_i', name): # image
|
||||
return next((slot(x, IMG_BASE, tb) for x in words if IMG_BASE <= x * 4 < IMG_BASE + 8 * IMG_SLOTS and slot(x, IMG_BASE, tb) is not None), None), None, ty
|
||||
tw = [o0] if ty == 3 else words
|
||||
t = next((slot(x, TEX_BASE, tb) for x in tw if slot(x, TEX_BASE, tb) is not None and x * 4 < TEX_BASE + 8 * TEX_SLOTS), None)
|
||||
if t is None and SEPTEX_BASE is not None:
|
||||
t = next((128 + slot(x, SEPTEX_BASE, tb) for x in tw if slot(x, SEPTEX_BASE, tb) is not None and x * 4 < SEPTEX_BASE + 8 * SEPTEX_SLOTS), None)
|
||||
t = next((slot(x, TEX_BASE, tb) for x in tw if slot(x, TEX_BASE, tb) is not None and x * 4 < SEPTEX_BASE), None)
|
||||
if t is None: t = next((SEPTEX_SLOT0 + slot(x, SEPTEX_BASE, tb) for x in tw if slot(x, SEPTEX_BASE, tb) is not None and x * 4 < SAMP_BASE), None)
|
||||
s = next((slot(x, SAMP_BASE, sb) for x in words if x * 4 >= SAMP_BASE and slot(x, SAMP_BASE, sb) is not None), None) if ty != 3 else None
|
||||
return t, s, ty
|
||||
|
||||
@@ -426,8 +410,7 @@ def reflect(key, stage, msl, res):
|
||||
return binds
|
||||
|
||||
DRV_FILL = {0: [(b, b + 0x100) for b in sorted(SSBO.values())], # c0: storage buffer address tables (vertex / compute / pixel, 16 slots × 0x10 each)
|
||||
2: [(TEX_BASE, TEX_BASE + 8 * TEX_SLOTS), (IMG_BASE, IMG_BASE + 8 * IMG_SLOTS), (SAMP_BASE, SAMP_BASE + 8 * 128)] + ([(SEPTEX_BASE, SEPTEX_BASE + 8 * SEPTEX_SLOTS)] if SEPTEX_BASE is not None else [])}
|
||||
# c2: texture handle table (Mario 0x20 + 8 r, 32 slots; Engage 0x168 + 8 r, 128 slots), image handle table (0x120 + 8 r, 8 slots), separate sampler handle table (0x568 + 8 r, 128 slots; the replay layer fills them all)
|
||||
2: [(TEX_BASE, SEPTEX_BASE), (IMG_BASE, IMG_BASE + 8 * IMG_SLOTS), (SEPTEX_BASE, SAMP_BASE), (SAMP_BASE, SAMP_BASE + 8 * 128)]} # c2: the handle tables above, all filled by the replay layer
|
||||
|
||||
def drv_reads(src):
|
||||
"""Reads of driver segments c0 / c2 in the MSL: constant index → byte range; non-constant indices are recorded as dynamic"""
|
||||
@@ -459,7 +442,7 @@ _CAST = ('as_type<int>(', 'as_type<uint>(')
|
||||
def int_inputs(src):
|
||||
"""Inputs that a vertex shader only uses as integers (under a float declaration every use is an as_type<int/uint> bit cast, or a plain copy) → {location: 'uint'}.
|
||||
Maxwell does no type conversion when reading vertex attribute registers: the bit pattern of integer formats (bone indices R8UI / RG8UI / RGBA8UI etc.) goes into the register as is and the shader uses it as an integer.
|
||||
This game has no PC version to pair reflection with, so the translator's attribute type queries all defaulted to float: Metal converts the integer format to a float value, and after the bit cast it is garbage - every bone index is wrong and skinned models (the characters) are stretched into streaks or thrown off screen."""
|
||||
The answer table has no attribute types (the machine code does not say them), so the translator's attribute type queries all defaulted to float: Metal converts the integer format to a float value, and after the bit cast it is garbage - every bone index is wrong and skinned models (the characters) are stretched into streaks or thrown off screen."""
|
||||
ins = {m.group(2): int(m.group(3)) for m in re.finditer(r'(\w+)\s+(\w+)\s*\[\[attribute\((\d+)\)\]\]', src)}
|
||||
i = src.find('vertex main0_out main0(')
|
||||
if i < 0: return {}
|
||||
@@ -626,7 +609,7 @@ def finish(key, w, j, c1):
|
||||
refl = {'key': key + '_b', 'route': 'B', 'chain': CHAIN, 'dm_ctl': key, 'source': f'{name}#{k}', 'pc_paired': pc_of[key], 'stage': j['stage'],
|
||||
'entry': entry.group(2) if entry else 'main0', 'msl_entry_kind': entry.group(1) if entry else None, **({'entry_z': 'main0_z'} if j['stage'] == 'vs' else {}),
|
||||
**({'entry_tf': 'main0_tf', 'tf_strides': [e[2] if e else 0 for e in j['tf']]} if j.get('tf') else {}),
|
||||
'threadgroup_size': j.get('local'), 'c1': c1.hex(), 'notes': res['notes'] + (['no paired PC reflection, queries use defaults'] if j.get('pc_missing') else []), 'drv_reads': dr, 'drv_unfillable': unfillable(dr),
|
||||
'threadgroup_size': j.get('local'), 'c1': c1.hex(), 'notes': res['notes'] + (['not in the answer table, queries use defaults'] if j.get('no_facts') else []), 'drv_reads': dr, 'drv_unfillable': unfillable(dr),
|
||||
'bindings': reflect(key, j['stage'], src, res),
|
||||
'vertex_inputs': [{'location': int(m.group(2)), 'msl_type': m.group(1)} for m in re.finditer(r'(\w+)\s+\w+\s*\[\[attribute\((\d+)\)\]\]', src)],
|
||||
'raster': raster_facts(j['stage'], src), 'vs_pad': pad}
|
||||
@@ -649,18 +632,8 @@ packs, jobs, c1s = {}, {}, {}
|
||||
digests, rule_fps, cache_reasons, cache_hits = {}, {}, {}, set()
|
||||
def rec_of(key): # → (package name, the shader's record)
|
||||
name, k = loc[key]
|
||||
if name.startswith('main.rodata@'):
|
||||
if 'main' not in packs:
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '../../verify'))
|
||||
try: import nsoimg
|
||||
except ImportError as e: sys.exit(f'[RED] extracting the main NSO needs lz4 ({e}): run with $DM_ROOT/build/cache/dm/venv/bin/python')
|
||||
packs['main'] = nsoimg.load(MAIN)
|
||||
rec = main_rec(packs['main'], k)
|
||||
if hashlib.sha256(rec[rec.find(bytes.fromhex('34127698')):][:0x880]).hexdigest()[:16] != key: sys.exit(f'[RED] {name}: control section hash does not match shader map key {key} (MAIN={MAIN})')
|
||||
else:
|
||||
if name not in packs: packs[name] = records(open(os.path.join(dm_dir, name), 'rb').read())
|
||||
rec = packs[name][k]
|
||||
return name, rec
|
||||
if name not in packs: packs[name] = records(open(os.path.join(dm_dir, name), 'rb').read())
|
||||
return name, packs[name][k]
|
||||
recs = {}
|
||||
for key in keys: recs[key] = rec_of(key); digests[key] = shader_key(key, *recs[key])
|
||||
def plain_digest(k): # a paired pixel shader's key, also when it is not built this time
|
||||
@@ -716,5 +689,5 @@ unk = [x for x in ok if any(b['kind'] == '?' for b in x['bindings'])]
|
||||
print(f'shaders with a binding whose slot was not decoded {len(noreg)}, with an unrecognized binding kind {len(unk)} (loading them at run time prints [RED] lines)')
|
||||
for e, n in collections.Counter(f"{b['kind']} {b['msl_name'][:28]}" for x in noreg + unk for b in x['bindings'] if b['kind'] == '?' or (b['kind'] in SLOTK and b.get('reg') is None)).most_common(8): print(f' {n:5d} {e}')
|
||||
nt = collections.Counter(nn.split(' (')[0] for x in ok for nn in x['notes'])
|
||||
for e, n in nt.most_common(8): print(f' queries without a PC answer {n:5d} {e}')
|
||||
for e, n in nt.most_common(8): print(f' queries without an answer {n:5d} {e}')
|
||||
json.dump([{'key': r[0], 'st': r[1], 'err': r[2]} for r in res], open(os.path.join(out, '_b_build.json'), 'w'), ensure_ascii=False)
|
||||
@@ -1,8 +1,8 @@
|
||||
# Translation chain tool fingerprints (written by dmsh_chain.py --update-lock; regenerate after changing tools and commit it)
|
||||
dmsh_xlate.dll a7f696bb0dbf1ac0
|
||||
Ryujinx.Graphics.Shader.dll a4b556413e6a98f5
|
||||
Ryujinx.Common.dll 5222da62ffb43db1
|
||||
xlate_source 38a1c0b4e3ebd5fa
|
||||
dmsh_xlate.dll 986fad61f44dc9a4
|
||||
Ryujinx.Graphics.Shader.dll 6f995909ae6fecb5
|
||||
Ryujinx.Common.dll 917c02ac432765af
|
||||
xlate_source 3cc20c29e9528168
|
||||
spirv-cross d9dc197f061b42a5
|
||||
metal_macosx Apple metal version 32023.921 (metalfe-32023.921.6)
|
||||
metal_iphoneos Apple metal version 32023.921 (metalfe-32023.921.6)
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env python3
|
||||
# Translation chain version of the B shader library (the build result must be determined by source alone).
|
||||
# Chain version V = first 12 digits of the sha256 of the contents of the following in-repo files + the translation options: this script, dmsh_build.py, dmsh_fetch.py (readable-tile version), the dmsh_xlate sources (Program.cs, csproj),
|
||||
# from the current game profile (dm/games/<id>/, chosen by DM_GAME, default smbw) the mapping table (including the conflict table), the vertex / pixel pair table (shader_pairs.tsv, used by vs_pad), the answer table for translation queries (dmsh_facts.py and shader_facts.tsv),
|
||||
# from the current game profile (dm/games/<id>/, chosen by DM_GAME) the mapping table (including the conflict table), the vertex / pixel pair table (shader_pairs.tsv, used by vs_pad), the answer table for translation queries (dmsh_facts.py and shader_facts.tsv),
|
||||
# the keys of the profile's game.tsv that affect the build (nvn_*, shader_*), the iOS compile convention ios_metal.py, and the tool lock dmsh_chain.lock (measured fingerprints of the Ryujinx shader dll / spirv-cross / metal compiler). The tables differ per game, so V does too.
|
||||
# The build (dmsh_build.py) writes into <cache dir>/b-<V>/ and the reflection json records chain = V; at compile time the host takes V from the game profile's dmsh_chain.h, loads only from b-<V>/ and verifies each json.
|
||||
# Usage: dmsh_chain.py print V (default options)
|
||||
|
||||
@@ -1,21 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
# Answer table for translation queries (dm/games/<id>/shader_facts.tsv in the game profile folder): one row per Switch shader key, recording the few facts the translator asks for but that cannot be read from the Switch machine code -
|
||||
# dimension type of textures / images, image format, component types of vertex attributes and fragment outputs, thread group size and shared memory bytes of compute shaders.
|
||||
# These facts come from the reflection of the PC shader paired one-to-one with it (NVN slot number = D3D register number, see dm/gfx/README.md). The table ships with the repo; building the shader library reads only this table
|
||||
# and needs no PC game files; for keys not in the table, translation queries are answered with defaults (listed separately in the build summary).
|
||||
# Usage: dmsh_facts.py --freeze regenerate the whole table from PC reflection (for maintainers; needs the PCCACHE reflection dir, the PC DXBC dir and vkd3d-compiler)
|
||||
# dmsh_facts.py --check recompute and compare key by key against the in-repo table; on mismatch print [RED] lines and exit 1 (as above, needs the PC-side inputs)
|
||||
# dmsh_facts.py print the table's row count and the coverage count of each fact
|
||||
# shader_split.py writes the table from what the game files show (merging the profile's shader_facts_manual.tsv over it); for keys not in the table, translation queries are answered with defaults
|
||||
# (listed separately in the build summary).
|
||||
# Usage: dmsh_facts.py print the table's row count and the coverage count of each fact
|
||||
# Row format: key <TAB> JSON (keys in lexicographic order, compact form; the same input gives a byte-identical table)
|
||||
import json, os, re, subprocess, sys
|
||||
DM_ROOT = os.environ.get('DM_ROOT') or os.path.expanduser('~/dm-work') # Workspace root (game files, build outputs and run artifacts are all under it)
|
||||
HERE = os.path.dirname(os.path.abspath(__file__)); HOME = os.path.expanduser('~')
|
||||
import json, os, sys
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, os.path.join(HERE, '../../games')); import game as _game # the tables are in the current game profile folder (DM_GAME)
|
||||
TABLE = str(_game.load().table('shader_facts.tsv')); MAP = str(_game.load().table('shader_map.tsv'))
|
||||
TT = {'texture1d': '1d', 'texture1darray': '1darray', 'texture2d': '2d', 'texture2darray': '2darray', 'texture2dms': '2dms',
|
||||
'texture2dmsarray': '2dmsarray', 'texture3d': '3d', 'texturecube': 'cube', 'texturecubearray': 'cubearray', 'buffer_typed': 'buffer'}
|
||||
CT = {'float': 'float', 'int': 'sint', 'uint': 'uint'}
|
||||
|
||||
TABLE = str(_game.load().table('shader_facts.tsv'))
|
||||
def load():
|
||||
"""key → fact dict; returns an empty dict if the table does not exist"""
|
||||
if not os.path.exists(TABLE): return {}
|
||||
@@ -23,63 +16,8 @@ def load():
|
||||
|
||||
def row(key, facts): return key + '\t' + json.dumps(facts, sort_keys=True, separators=(',', ':'), ensure_ascii=False) + '\n'
|
||||
|
||||
def from_pc(pj, stage, dxbc_asm):
|
||||
"""PC reflection json (pj) + DXBC disassembly text of the compute shader (None if absent) → fact dict"""
|
||||
f = {'tex': {}, 'img': {}, 'attr': {}, 'fout': {}}
|
||||
for b in pj['bindings']:
|
||||
if b['kind'] == 't' and b['type'] in TT: f['tex'][str(b['reg'])] = TT[b['type']]
|
||||
if b['kind'] == 'u' and b['type'] in TT:
|
||||
f['img'][str(b['reg'])] = {'type': TT[b['type']], 'fmt': 'R32Uint' if 'uint' in (b.get('msl_type') or '') else 'R16G16B16A16Float'}
|
||||
for v in pj.get('vertex_inputs') or []: f['attr'][str(v.get('location', v.get('index')))] = CT.get(v.get('component_type'), 'float')
|
||||
for o in pj.get('fragment_outputs') or []: f['fout'][str(o.get('index', 0))] = CT.get(o.get('component_type'), 'float')
|
||||
if pj.get('threadgroup_size'): f['local'] = pj['threadgroup_size']
|
||||
if stage == 'cs': # shared memory: sum of the groupshared declarations in the PC DXBC (the translator default is 48 KB, over Metal's 32 KB limit)
|
||||
asm = dxbc_asm or ''
|
||||
n = sum(int(m.group(1)) for m in re.finditer(r'dcl_tgsm_raw\s+g\d+,\s*(\d+)', asm)) + \
|
||||
sum(int(m.group(1)) * int(m.group(2)) for m in re.finditer(r'dcl_tgsm_structured\s+g\d+,\s*(\d+),\s*(\d+)', asm))
|
||||
f['shared'] = (n + 15) // 16 * 16
|
||||
if dxbc_asm is None: f['pc_missing'] = True
|
||||
return f
|
||||
|
||||
def pairs():
|
||||
"""Mapping table (with the conflict table's overrides) → {key: (paired PC name, stage)}"""
|
||||
pc_of, stage_of = {}, {}
|
||||
for ln in open(MAP):
|
||||
c = ln.rstrip('\n').split('\t')
|
||||
if len(c) > 4 and c[0].endswith('.bin'): pc_of.setdefault(c[3], c[4]); stage_of.setdefault(c[3], c[0][:2])
|
||||
if len(c) > 4 and c[0].startswith('main.rodata@'): pc_of.setdefault(c[3], c[4]); stage_of[c[3]] = {'vertex': 'vs', 'pixel': 'ps'}[c[2]]
|
||||
for ln in open(MAP + '.conflicts.tsv'):
|
||||
c = ln.rstrip('\n').split('\t')
|
||||
if len(c) > 1 and not ln.startswith('#') and not ln.startswith('dm_') and c[0] in pc_of: pc_of[c[0]] = c[1]
|
||||
return {k: (pc_of[k], stage_of[k]) for k in pc_of}
|
||||
|
||||
def recompute():
|
||||
pcc = os.environ.get('PCCACHE', f'{DM_ROOT}/build/cache/metal_shader_cache')
|
||||
dxdirs = os.environ.get('PCDXBC', f'{DM_ROOT}/build/ios/dxbc_all_0930:{DM_ROOT}/data/d3d12_study/shaders').split(':')
|
||||
vk = os.environ.get('VKD3D', f'{DM_ROOT}/build/vkd3d/inst/bin/vkd3d-compiler')
|
||||
if not os.path.isdir(pcc): sys.exit(f'[RED] no PC reflection dir {pcc} (PCCACHE=)')
|
||||
out, missing = {}, 0
|
||||
for key, (pc, stage) in sorted(pairs().items()):
|
||||
pp = os.path.join(pcc, pc + '.json')
|
||||
if not os.path.exists(pp): missing += 1; continue # no paired PC reflection: not entered in the table, queries use defaults
|
||||
asm = None
|
||||
if stage == 'cs':
|
||||
dx = next((p for d in dxdirs for p in (os.path.join(d, pc + '.dxbc'), os.path.join(d, pc[:12] + '.dxbc')) if os.path.exists(p)), None)
|
||||
if dx: asm = subprocess.run([vk, '-x', 'dxbc-tpf', '-b', 'd3d-asm', dx], capture_output=True, text=True).stdout
|
||||
out[key] = from_pc(json.load(open(pp)), stage, asm)
|
||||
return out, missing
|
||||
|
||||
if __name__ == '__main__':
|
||||
a = sys.argv[1:]
|
||||
if a == ['--freeze']:
|
||||
t, missing = recompute()
|
||||
open(TABLE, 'w').write('# Answer table for translation queries (generated by dmsh_facts.py --freeze, do not edit): key <TAB> fact JSON\n' + ''.join(row(k, t[k]) for k in sorted(t)))
|
||||
print(f'wrote {TABLE}: {len(t)} keys; without a paired PC reflection, left out of the table: {missing}')
|
||||
elif a == ['--check']:
|
||||
t, missing = recompute(); have = load()
|
||||
bad = sorted(k for k in set(t) | set(have) if t.get(k) != have.get(k))
|
||||
for k in bad[:10]: print(f'[RED] {k}: table {have.get(k)} / recomputed {t.get(k)}')
|
||||
print(f'compared key by key {len(set(t) | set(have))}: mismatches {len(bad)}; without a paired PC reflection {missing}'); sys.exit(1 if bad else 0)
|
||||
elif not a:
|
||||
t = load(); print(f'{len(t)} keys: ' + ', '.join(f'{n} {sum(bool(f.get(n)) for f in t.values())}' for n in ('tex', 'img', 'attr', 'fout', 'local', 'shared', 'pc_missing')))
|
||||
if not a:
|
||||
t = load(); print(f'{len(t)} keys: ' + ', '.join(f'{n} {sum(bool(f.get(n)) for f in t.values())}' for n in ('tex', 'img', 'attr', 'fout', 'local', 'shared')))
|
||||
else: sys.exit('usage: see the file header')
|
||||
@@ -1,6 +1,6 @@
|
||||
// Original Switch shaders (Maxwell machine code) → SPIR-V (the Ryujinx.Graphics.Shader translator), for route B's offline metallib build.
|
||||
// Difference from Ryujinx.ShaderTools: the translator's "queries" are answered truthfully rather than always with defaults -
|
||||
// texture type / image format / vertex attribute type / fragment output type / compute thread group size come from the job file (given by the PC reflection paired via the mapping table; same slot number = same resource);
|
||||
// texture type / image format / vertex attribute type / fragment output type / compute thread group size come from the job file (the game profile's answer table shader_facts.tsv, by the slot numbers of the reflection);
|
||||
// window origin upper-left; depth is not rewritten (z goes out unchanged, NVN's near=0 convention). A game's depth convention is runtime state (nvnDeviceSetDepthMode: Mario / Metroid etc. keep the default near=-w, the GL convention;
|
||||
// Link's Awakening / 13 Sentinels / Engage set near=0), so dmsh_build.py adds the GL rewrite z' = z/2 + w/2 in the MSL as two entry points, main0 (rewritten, GL) and main0_z (unchanged, near=0), and the host picks by the game's convention.
|
||||
// This used to rewrite for GL always: a near=0 game's depth was squeezed into 0.5..1, and effects that turn depth into distance (Link's Awakening's outdoor depth of field blurring the whole screen, occlusion) were all wrong.
|
||||
@@ -21,7 +21,9 @@ using System.Text.Json.Nodes;
|
||||
|
||||
static class SwshXlate
|
||||
{
|
||||
const int TexBase = 0x20, SampBase = 0x568; // byte base of NVN texture handles in c[2] (8 bytes per slot): 0x20 for this game (NVN 55), measured over the whole library, see TEX_BASE in dm/gfx/tools/dmsh_build.py; separate samplers keep D2R's 0x568 (unused by this game)
|
||||
// NVN's handle tables in c[2], byte offsets, 8 bytes per slot (one layout in every game, see dm/gfx/tools/dmsh_build.py): combined textures 0x20 + 8 r, images 0x120 + 8 r (8 slots),
|
||||
// separate textures 0x168 + 8 r (slot 128 + r), separate samplers 0x568 + 8 r
|
||||
const int TexBase = 0x20, ImgBase = 0x120, ImgSlots = 8, SepTexBase = 0x168, SepTexSlot0 = 128, SampBase = 0x568;
|
||||
|
||||
sealed class Accessor : IGpuAccessor
|
||||
{
|
||||
@@ -41,17 +43,27 @@ static class SwshXlate
|
||||
public int QueryTextureArrayLengthFromPool() => 32;
|
||||
|
||||
string Get(string table, int slot) => _job[table] is JsonObject t && t[slot.ToString()] is JsonNode n ? (n is JsonObject o ? (string)o["type"] : (string)n) : null;
|
||||
int TexSlot(int handle, int cbufSlot)
|
||||
// A handle's texture or image slot, numbered as dmsh_build.py decode_tex numbers the reflection's (so an answer table row names the slot the reflection shows): the handle ORs two
|
||||
// c[2] words in no fixed order, so both are tried (only the first for a combined handle or a constant sampler); -1 when neither is in the table
|
||||
static int Slot(int handle, int cbufSlot, bool image)
|
||||
{
|
||||
(int off0, _, _) = TextureHandle.UnpackOffsets(handle);
|
||||
int bytes = off0 * 4;
|
||||
return (cbufSlot < 0 || (cbufSlot & 0xffff) == 2) && bytes >= TexBase && bytes < SampBase && (bytes - TexBase) % 8 == 0 ? (bytes - TexBase) / 8 : -1;
|
||||
if (cbufSlot >= 0 && (cbufSlot & 0xffff) != 2) return -1;
|
||||
(int w0, int w1, TextureHandleType ty) = TextureHandle.UnpackOffsets(handle);
|
||||
foreach (int w in ty == TextureHandleType.CombinedSampler || ty == TextureHandleType.SeparateConstantSamplerHandle ? new[] { w0 } : new[] { w0, w1 })
|
||||
{
|
||||
int b = w * 4;
|
||||
if (b % 8 != 0) continue;
|
||||
if (image && b >= ImgBase && b < ImgBase + 8 * ImgSlots) return (b - ImgBase) / 8;
|
||||
if (!image && b >= TexBase && b < SepTexBase) return (b - TexBase) / 8;
|
||||
if (!image && b >= SepTexBase && b < SampBase) return SepTexSlot0 + (b - SepTexBase) / 8;
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
public SamplerType QuerySamplerType(int handle, int cbufSlot = -1)
|
||||
{
|
||||
int slot = TexSlot(handle, cbufSlot);
|
||||
string t = slot >= 0 ? Get("tex", slot) ?? Get("img", slot) : null;
|
||||
if (t == null) { Notes.Add($"texture handle {handle:X} (constant buffer {cbufSlot}, slot {slot}) has no PC type, using 2D"); return SamplerType.Texture2D; }
|
||||
int slot = Slot(handle, cbufSlot, false), islot = Slot(handle, cbufSlot, true);
|
||||
string t = (slot >= 0 ? Get("tex", slot) : null) ?? (islot >= 0 ? Get("img", islot) : null);
|
||||
if (t == null) { Notes.Add($"texture handle {handle:X} (constant buffer {cbufSlot}, slot {slot}) has no type in the answer table, using 2D"); return SamplerType.Texture2D; }
|
||||
return t switch
|
||||
{
|
||||
"1d" => SamplerType.Texture1D, "1darray" => SamplerType.Texture1D | SamplerType.Array,
|
||||
@@ -63,16 +75,16 @@ static class SwshXlate
|
||||
}
|
||||
public TextureFormat QueryTextureFormat(int handle, int cbufSlot = -1)
|
||||
{
|
||||
int slot = TexSlot(handle, cbufSlot);
|
||||
int slot = Slot(handle, cbufSlot, true);
|
||||
string f = _job["img"] is JsonObject t && slot >= 0 && t[slot.ToString()] is JsonObject o ? (string)o["fmt"] : null;
|
||||
if (f == null) { Notes.Add($"image handle {handle:X} slot {slot} has no PC format, using R8G8B8A8Unorm"); return TextureFormat.R8G8B8A8Unorm; }
|
||||
if (f == null) { Notes.Add($"image handle {handle:X} slot {slot} has no format in the answer table, using R8G8B8A8Unorm"); return TextureFormat.R8G8B8A8Unorm; }
|
||||
return Enum.TryParse(f, true, out TextureFormat tf) ? tf : TextureFormat.R8G8B8A8Unorm;
|
||||
}
|
||||
int Local(int i) => _job["local"] is JsonArray a && a.Count > i ? (int)a[i] : 1;
|
||||
public int QueryComputeLocalSizeX() => Local(0);
|
||||
public int QueryComputeLocalSizeY() => Local(1);
|
||||
public int QueryComputeLocalSizeZ() => Local(2);
|
||||
public int QueryComputeSharedMemorySize() => _job["shared"] is JsonNode n ? (int)n : 0; // sum of the paired PC shader's groupshared declarations
|
||||
public int QueryComputeSharedMemorySize() => _job["shared"] is JsonNode n ? (int)n : 0; // the shared memory size in the compute shader's control section (shader_split.py)
|
||||
static AttributeType Attr(string s) => s switch { "sint" => AttributeType.Sint, "uint" => AttributeType.Uint, _ => AttributeType.Float };
|
||||
public GpuGraphicsState QueryGraphicsState()
|
||||
{
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,60 +0,0 @@
|
||||
# Resource references in Maxwell (Tegra X1) shader machine code: only decodes the few kinds needed for the binding mapping; not a full disassembler.
|
||||
# Instruction bit patterns are taken from the yuzu shader decode table; code segment layout: graphics stage = 0x30 NVN header (0x12345678) + 0x50 SPH + instructions,
|
||||
# compute stage is instructions directly; the 1st of every 4 64-bit words is the scheduling control word.
|
||||
# A full measurement (10847 variants, tools/shader_map.py) gives NVN binding ↔ D3D register:
|
||||
# uniform buffer binding r ↔ cb_r, which is c[3 + r] in machine code (14 per stage, c[3]..c[16]; the higher c[22]/c[27] are driver-internal)
|
||||
# separate texture slot r ↔ t_r, handle at c[2][0x168 + 8r] (two ways to read it: a c[2] operand, or the texture instruction's immediate texture index ×4)
|
||||
# separate sampler slot r ↔ s_r, handle at c[2][0x568 + 8r] (1 exception: cs_terrain_composite additionally uses s2, absent on PC)
|
||||
# storage buffer slot r ↔ u_r or read-only structured/raw buffer t_r (same numbering space), address/size at c[0][base + 0x10 r]
|
||||
# base: vertex 0x110, compute 0x310, pixel 0x510; all 10847 match exactly in both directions
|
||||
import struct
|
||||
|
||||
TEX_BASE, SAMP_BASE, UBO_FIRST, UBO_LAST = 0x168, 0x568, 3, 16
|
||||
SSBO_BASE = {'vs': 0x110, 'cs': 0x310, 'ps': 0x510}
|
||||
|
||||
def instrs(code, graphics):
|
||||
off = 0x80 if graphics and code[:4] == bytes.fromhex('78563412') else 0
|
||||
out = []
|
||||
for k in range(off, len(code) - 7, 8):
|
||||
if ((k - off) // 8) % 4 == 0: continue
|
||||
out.append(struct.unpack_from('<Q', code, k)[0])
|
||||
return out
|
||||
|
||||
def match(w, pat):
|
||||
for i, ch in enumerate(pat):
|
||||
if ch != '-' and int(ch) != (w >> (63 - i)) & 1: return False
|
||||
return True
|
||||
|
||||
def cbuf_alu(ins):
|
||||
"""ALU instructions whose operand class is constant buffer (top 4 bits = 0100): {(index, byte offset)}"""
|
||||
return {((w >> 34) & 0x1f, ((w >> 20) & 0x3fff) * 4) for w in ins if (w >> 60) == 0x4 and w}
|
||||
|
||||
def ldc(ins):
|
||||
"""LDC (read constant buffer by index): {(index, byte offset)}"""
|
||||
return {((w >> 36) & 0x1f, (w >> 20) & 0xffff) for w in ins if match(w, '1110111110010---')}
|
||||
|
||||
def bindings(code, stage):
|
||||
ins = instrs(code, stage != 'cs')
|
||||
refs = cbuf_alu(ins) | ldc(ins)
|
||||
ubo = {i - UBO_FIRST for i, _ in refs if UBO_FIRST <= i <= UBO_LAST}
|
||||
c2 = {o for i, o in refs if i == 2}
|
||||
# Immediate texture index (bits 36..48 of TEX/TEXS/TLD/TLDS/TLD4/TLD4S) = word offset of the handle within c[2]; ×4 gives the byte offset (verified with the embedded shader's TLDS 0x6e→t10)
|
||||
for w in ins:
|
||||
for pat in ('110000----111---', '110010----111---', '11011100--11----', '1101111100------', '1101-00---------', '1101-01---------'):
|
||||
if match(w, pat): c2.add(((w >> 36) & 0x1fff) * 4); break
|
||||
tex = {(o - TEX_BASE) // 8 for o in c2 if TEX_BASE <= o < SAMP_BASE and (o - TEX_BASE) % 8 == 0}
|
||||
samp = {(o - SAMP_BASE) // 8 for o in c2 if o >= SAMP_BASE and (o - SAMP_BASE) % 8 == 0}
|
||||
base = SSBO_BASE.get(stage)
|
||||
ssbo = {(o - base) // 16 for i, o in refs if i == 0 and base and o >= base and (o - base) % 16 == 0 and (o - base) // 16 < 16} if base else set()
|
||||
drv = sorted(o for o in c2 if o < TEX_BASE)
|
||||
return {'ubo': ubo, 'tex': tex, 'samp': samp, 'ssbo': ssbo, 'c2drv': drv}
|
||||
|
||||
def vs_inputs(code):
|
||||
"""Generic attribute slots read by the vertex shader: attribute address bits 20..29 of ALD (1110111111011---); generic attribute n is at 0x80 + 16n"""
|
||||
ins = instrs(code, True)
|
||||
return {((w >> 20) & 0x3ff) - 0x80 >> 4 for w in ins if match(w, '1110111111011---') and 0x80 <= ((w >> 20) & 0x3ff) < 0x280}
|
||||
|
||||
def sph_stage(code):
|
||||
"""Stage field of the graphics shader SPH header (word 0, bits 10..13): 1 vertex, 5 pixel"""
|
||||
if code[:4] != bytes.fromhex('78563412'): return None
|
||||
return {1: 'vs', 5: 'ps'}.get((struct.unpack_from('<I', code, 0x30)[0] >> 10) & 0xf)
|
||||
@@ -32,12 +32,11 @@
|
||||
# Pair table: (vertex key, pixel key) of every BNSH variation (every Unity program); dmsh_build uses it to give vertex shaders the outputs their pixel shaders read but they do not write (see dmsh_build.py vs_pad)
|
||||
import collections, concurrent.futures as cf, hashlib, json, os, re, struct, subprocess, sys
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '../../games')); import game as _game
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))); import uepak
|
||||
GAME = _game.load() # the current game profile (DM_GAME, required): source group rules and the table header note
|
||||
GROUPS, NOTE = GAME.pairs('shader_groups'), GAME.get('shader_note')
|
||||
GLSLC = GAME.get('shader_main_glslc', 'off') == 'on' # GLSLC compiler outputs in main (glslc_shaders)
|
||||
SEPARABLE = GAME.get('shader_separable', 'off') == 'on' # single-stage programs combined per stage at draw time (separable_pairs)
|
||||
CAPTURE = GAME.get('shader_glslc_capture', 'off') == 'on' # GLSLC outputs the game compiles at run time, captured on the Mac (DM_GLSLC_CAPTURE, hle_glslc.c)
|
||||
UE = GAME.get('shader_ue_decode', 'off') == 'on' # Unreal Engine shader code archives, decoded on the Mac with the game's own Oodle (uepak.py, DM_OODLE_JOB, hle_oodle.c)
|
||||
# The sources are what the game files hold, with no profile switch: GLSLC outputs in main (glslc_shaders), the outputs the game compiles at run time when a capture is there (DM_GLSLC_CAPTURE,
|
||||
# hle_glslc.c), the shader code archives of Unreal paks once decoded on the Mac (uepak.py, DM_OODLE_JOB, hle_oodle.c); the NSP script decides from the files which of the last two to make
|
||||
|
||||
CTL_MAGIC, GFX_MAGIC, CS_MAGIC = bytes.fromhex('34127698'), bytes.fromhex('78563412'), bytes.fromhex('79563412')
|
||||
STAGES = ['vs', 'hs', 'ds', 'gs', 'ps', 'cs']
|
||||
@@ -124,8 +123,9 @@ def mshd_shaders(d, label):
|
||||
|
||||
def glslc_shaders(d, label, need=True):
|
||||
"""GLSLC compiler outputs (838 in 13 Sentinels' main read-only segment, one stage each): from the magic 0x19866891, +0x44 = the whole output's length; each output holds exactly one control section with the code area right after it (control +0x20 = control length).
|
||||
The magic is only 4 bytes and other games' mains have it too (Fire Emblem Engage has 11), so this runs only when the profile's shader_main_glslc is on, and keeps only outputs that pass the triple check.
|
||||
need: stop when there is none (main with shader_main_glslc on); a Koei Tecmo G2S container (idrk_shaders) may hold none.
|
||||
The magic is only 4 bytes, so only outputs that pass the triple check are kept. Every game's main is searched: Unity's player has 11 of its own (Fire Emblem Engage, Ace Attorney Investigations
|
||||
Collection) beside the game's shaders in its bundles, KTGL and Vanillaware's engine compile theirs into main (13 Sentinels: 838), the others have none.
|
||||
need: stop when there is none (a capture file); a Koei Tecmo G2S container (idrk_shaders) or a main may hold none.
|
||||
SDK 6's GLSLC (output version 0x1d.2, Fire Emblem: Three Houses) writes 0x848- and 0x850-byte control sections (ctl_len's lower range) and compute code with no 0x12345679 header: the instructions
|
||||
start at code +0 (the first word is a scheduling word), and the stage then comes from the control's thread group (+0x748), which only compute sections fill. Its few version-0x1d.0 outputs keep
|
||||
something else at control +0x20 (odd values such as 0x136b), so a length that is not a multiple of 4 is not a control length"""
|
||||
@@ -141,7 +141,7 @@ def glslc_shaders(d, label, need=True):
|
||||
st = 'cs' if code[:4] == CS_MAGIC or headerless_cs else SPH_STAGE.get((struct.unpack_from('<I', code, 0x30)[0] >> 10) & 0xf) if code[:4] == GFX_MAGIC else None
|
||||
if not st or check(label, st, ctl, code): continue
|
||||
out.append((f'{label}#GLSLC@{h:#x}', st, ctl, code))
|
||||
if not out and need: red(f'{label}: the profile has shader_main_glslc on, but there is no GLSLC output')
|
||||
if not out and need: red(f'{label}: no GLSLC output')
|
||||
return out
|
||||
|
||||
IDRK, ZCHUNKED = b'IDRK0000', 0x100000
|
||||
@@ -348,7 +348,7 @@ def trpfs_jobs(files, parts=64):
|
||||
_OODLE = None
|
||||
def oodle(comp, n):
|
||||
"""The game's own OodleLZ_Decompress (Trinity's packs are Oodle Mermaid, and Oodle is linked into main): one Mac host per worker process in its tool mode (dm/host/hle_oodle.c, DM_OODLE_SERVE),
|
||||
fed over a pipe. DM_OODLE_HOST = the host, DM_OODLE_MODULES = the rewritten modules it loads (the NSP script sets both for profiles with shader_oodle on)"""
|
||||
fed over a pipe. DM_OODLE_HOST = the host, DM_OODLE_MODULES = the rewritten modules it loads (the NSP script sets both for a game with Trinity packs, profile_guess.py shader_sources)"""
|
||||
global _OODLE
|
||||
if _OODLE is None:
|
||||
import tempfile
|
||||
@@ -520,22 +520,29 @@ def ps_inputs(code):
|
||||
"""Generic attributes a pixel shader reads: SPH words 6–13 are the input map, 8 bits for each of the 32 attributes (Ryujinx ShaderHeader.ImapTypes), non-zero = read; the translated MSL inputs are exactly user(locn<attribute>)"""
|
||||
return {i for i in range(32) if struct.unpack_from('<I', code, 0x30 + 4 * (6 + (i >> 2)))[0] >> ((i & 3) * 8) & 0xff}
|
||||
|
||||
SINGLE_STAGE = ('#GLSLC@', '#UE') # sources whose every shader is a program of one stage: GLSLC outputs, Unreal's shader records (BNSH variations, Unity programs, MSHD and SDF pair theirs)
|
||||
def separable_pairs(keys):
|
||||
"""Single-stage programs (13 Sentinels: each GLSLC output becomes a program with one stage, and BindProgram's stage bits combine vertex and pixel programs per draw) have their real pairs only at draw time.
|
||||
vs_pad uses only the union of the locations the paired pixel shaders read, so every vertex shader is paired with a "cover": the fewest pixel shaders whose read locations together equal those of all pixel shaders (greedy, ties go to the smaller key);
|
||||
padded that way, a vertex shader links with any pixel shader. All pairs would be 376 × 375 rows; the cover needs a few rows × 376"""
|
||||
"""Single-stage programs (13 Sentinels: each GLSLC output becomes a program with one stage, and BindProgram's stage bits combine vertex and pixel programs per draw; Unreal combines its shaders per draw
|
||||
from the pipeline state) have their real pairs only at draw time. vs_pad uses only the union of the locations the paired pixel shaders read, so every single-stage vertex shader is paired with
|
||||
a "cover": the fewest pixel shaders whose read locations together equal those of all pixel shaders (greedy, ties go to the smaller key); padded that way, a vertex shader links with any pixel shader.
|
||||
All pairs would be 376 × 375 rows; the cover needs a few rows × 376. No single-stage vertex shader, no cover"""
|
||||
single = [v for v in keys if keys[v][1] == 'vs' and any(s in keys[v][0] for s in SINGLE_STAGE)]
|
||||
if not single: return set(), []
|
||||
ins = {k: ps_inputs(v[3]) for k, v in keys.items() if v[1] == 'ps'}
|
||||
left, cover = set().union(*ins.values()) if ins else set(), []
|
||||
while left:
|
||||
k = min(ins, key=lambda k: (-len(ins[k] & left), k)); cover.append(k); left -= ins[k]
|
||||
return {(v, p) for v in keys if keys[v][1] == 'vs' for p in cover}, cover
|
||||
return {(v, p) for v in single for p in cover}, cover
|
||||
|
||||
UE_TAG = {'SV': 'vs', 'SH': 'hs', 'SD': 'ds', 'SG': 'gs', 'SP': 'ps', 'SC': 'cs'}
|
||||
def main_imports(img): # the names main imports (its dynamic symbols)
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), '../../tools')); import nso_syms
|
||||
return {s for s, defined, _ in nso_syms.dynsyms(img) if not defined}
|
||||
|
||||
def ue_shaders(d, label):
|
||||
"""An Unreal Engine shader code archive with every shader decoded (Octopath Traveler 0: the two .ushaderbytecode files of its pak, whose shaders are Oodle-compressed; the NSP script decodes them
|
||||
on the Mac with the game's own Oodle, uepak.py): each shader is an LSLG record holding one stage's code area and control section (uepak.lslg); the stage is the record's tag (SV vertex,
|
||||
SP pixel, SC compute), checked against the archive entry's frequency. The engine combines vertex and pixel shaders per draw, so the profile pairs them with shader_separable"""
|
||||
import uepak
|
||||
SP pixel, SC compute), checked against the archive entry's frequency. The engine combines vertex and pixel shaders per draw, so they are paired as single-stage programs (separable_pairs)"""
|
||||
a, out = uepak.archive(d), []
|
||||
for i, (off, size, raw, fr) in enumerate(a['shaders']):
|
||||
if size != raw: red(f'{label}: shader {i} is still compressed (an archive the NSP script has not decoded)')
|
||||
@@ -620,17 +627,19 @@ def main():
|
||||
try: common = unityfs.common_strings(img)
|
||||
except ValueError: common = None # not a Unity game
|
||||
found = []; scan_blob(img, 'main' + ('@' + tag if tag else ''), found)
|
||||
if GLSLC: found += glslc_shaders(img, 'main' + ('@' + tag if tag else ''))
|
||||
if CAPTURE: # GLSLC outputs the game compiles at run time, captured on the Mac (hle_glslc.c): <version dir>/glslc_capture, or DM_GLSLC_CAPTURE
|
||||
cap = os.environ.get('DM_GLSLC_CAPTURE') or os.path.join(os.path.dirname(os.path.abspath(romfs)), 'glslc_capture')
|
||||
names = sorted(f for f in os.listdir(cap) if f.endswith('.glslc')) if os.path.isdir(cap) else []
|
||||
if not names: print(f'[RED] the profile has shader_glslc_capture on, but there are no captured GLSLC outputs in {cap}: the shaders the game compiles at run time will have no translation', flush=True)
|
||||
for nm in names: found += glslc_shaders(open(os.path.join(cap, nm), 'rb').read(), f'/glslc_capture/{nm}')
|
||||
if UE: # Unreal Engine shader code archives decoded on the Mac (uepak.py): <version dir>/ue_shaders, or DM_UE_SHADERS
|
||||
ued = os.environ.get('DM_UE_SHADERS') or os.path.join(os.path.dirname(os.path.abspath(romfs)), 'ue_shaders')
|
||||
names = sorted(f for f in os.listdir(ued) if f.endswith('.ushaderbytecode')) if os.path.isdir(ued) else []
|
||||
if not names: red(f'the profile has shader_ue_decode on, but there are no decoded shader code archives in {ued} (the NSP script writes them)')
|
||||
for nm in names: found += ue_shaders(open(os.path.join(ued, nm), 'rb').read(), f'/ue_shaders/{nm}')
|
||||
found += glslc_shaders(img, 'main' + ('@' + tag if tag else ''), need=False)
|
||||
# GLSLC outputs the game compiles at run time, captured on the Mac (hle_glslc.c): <version dir>/glslc_capture, or DM_GLSLC_CAPTURE. A game needs them when main holds GLSL sources and
|
||||
# imports glslcCompile (Fire Emblem: Three Houses; profile_guess.py shader_sources)
|
||||
cap = os.environ.get('DM_GLSLC_CAPTURE') or os.path.join(os.path.dirname(os.path.abspath(romfs)), 'glslc_capture')
|
||||
names = sorted(f for f in os.listdir(cap) if f.endswith('.glslc')) if os.path.isdir(cap) else []
|
||||
if not names and img.count(b'#version ') and 'glslcCompile' in main_imports(img):
|
||||
print(f'[RED] main holds GLSL sources and imports glslcCompile, but there are no captured GLSLC outputs in {cap}: the shaders the game compiles at run time will have no translation', flush=True)
|
||||
for nm in names: found += glslc_shaders(open(os.path.join(cap, nm), 'rb').read(), f'/glslc_capture/{nm}')
|
||||
# Unreal Engine shader code archives decoded on the Mac (uepak.py): <version dir>/ue_shaders, or DM_UE_SHADERS; needed when the RomFS has Unreal paks
|
||||
ued = os.environ.get('DM_UE_SHADERS') or os.path.join(os.path.dirname(os.path.abspath(romfs)), 'ue_shaders')
|
||||
names = sorted(f for f in os.listdir(ued) if f.endswith('.ushaderbytecode')) if os.path.isdir(ued) else []
|
||||
if not names and uepak.unreal_paks(romfs): red(f'the RomFS has Unreal paks, but there are no decoded shader code archives in {ued} (the NSP script writes them)')
|
||||
for nm in names: found += ue_shaders(open(os.path.join(ued, nm), 'rb').read(), f'/ue_shaders/{nm}')
|
||||
results.append(('main', {skey(c, k): (w, s, c, k) for w, s, c, k in found}, var_pairs(found), {}))
|
||||
zdicts = {k: v for r in romfs_v for k, v in zs_dicts(r).items()} # Tears of the Kingdom's zstd dictionaries (zs_dicts)
|
||||
keys = {}
|
||||
@@ -661,9 +670,8 @@ def main():
|
||||
if not same_shader(keys[key][2:4], v[2:4]): red(f'key {key} is the same but the control section or code area differs ({v[0]} / {keys[key][0]})')
|
||||
continue
|
||||
keys[key] = v; per_src[group(v[0])] += 1
|
||||
if SEPARABLE:
|
||||
sp, cover = separable_pairs(keys); pairs |= sp
|
||||
print(f'single-stage programs: every vertex shader is paired with a cover of {len(cover)} pixel shader(s) ({" ".join(cover)})')
|
||||
sp, cover = separable_pairs(keys); pairs |= sp
|
||||
if cover: print(f'single-stage programs: {len({v for v, _ in sp}):,} vertex shaders are paired with a cover of {len(cover)} pixel shader(s) ({" ".join(cover)})')
|
||||
os.makedirs(pkg, exist_ok=True)
|
||||
for f in os.listdir(pkg):
|
||||
if f.endswith('.bin'): os.unlink(os.path.join(pkg, f))
|
||||
|
||||
+11
-1
@@ -2,7 +2,7 @@
|
||||
# Unreal Engine game files for the shader tools: pak files (version 11, UE 4.26 and later; Octopath Traveler 0 is UE5), shader code archives (.ushaderbytecode version 2) and the job files the Mac host decodes Oodle with
|
||||
# (DM_OODLE_JOB, dm/host/hle_oodle.c). Octopath Traveler 0 keeps everything in one pak, Octopath_Traveler0/Content/Paks/pakchunk0-Switch.pak (1.5 GB, 159,757 files): its index is AES-256 encrypted,
|
||||
# its files are stored or Oodle-compressed in blocks. Its two shader code archives are stored, but every shader inside is Oodle-compressed (Mermaid, block header 8C 0A), which no decoder in this
|
||||
# repository can undo, so the game's own OodleLZ_Decompress decodes them in the Mac host (profile key shader_ue_decode, the NSP script's decode_ue_shaders).
|
||||
# repository can undo, so the game's own OodleLZ_Decompress decodes them in the Mac host (the NSP script's decode_ue_shaders, for any game with Unreal paks).
|
||||
# The AES key is in main (Unreal embeds it): the 32-byte window of main's read-only or data segment that decrypts the index start to its mount point ("../../../"; Octopath Traveler 0: one window of
|
||||
# 116,287 tried). It is only kept in memory, never printed or written.
|
||||
# Pak footer (version 11, the last 0xdd bytes): encryption key GUID (16), encrypted index flag (u8), magic 0x5A6F12E1, version (u32), index offset (u64), index size (u64), index SHA-1 (20),
|
||||
@@ -142,6 +142,16 @@ def paks(romfs):
|
||||
"""Every pak under the RomFS"""
|
||||
return sorted(os.path.join(dp, f) for dp, _, fs in os.walk(romfs) for f in fs if f.endswith('.pak'))
|
||||
|
||||
def unreal_paks(romfs):
|
||||
"""The paks under the RomFS that Unreal wrote (its magic in the footer, any version): a game whose shaders the NSP script decodes (decode_ue_shaders) and shader_split.py reads"""
|
||||
out = []
|
||||
for p in paks(romfs):
|
||||
try:
|
||||
with open(p, 'rb') as f: f.seek(-0xdd, 2); ft = f.read(0xdd)
|
||||
except OSError: continue # shorter than a footer
|
||||
if ft.find(struct.pack('<I', PAK_MAGIC)) >= 1: out.append(p)
|
||||
return out
|
||||
|
||||
def job(romfs, main_nso, work):
|
||||
"""The shader code archives of every pak under the RomFS into <work>/archives, and one Oodle job (<work>/oodle.job) of every compressed shader in them; <work>/job.json lists them"""
|
||||
os.makedirs(os.path.join(work, 'archives'), exist_ok=True)
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
// GLSLC at run time. Fire Emblem: Three Houses (KTGL, SDK 6.5.0) compiles 9 built-in GLSL shaders at boot with the game's own runtime compiler (subsdk0, glslcCompile called through a function
|
||||
// table from main+0x9c89f8; the sources are plain strings in main, e.g. "in vec3 ktglPosition0; …"). Their outputs are in no game file, and shaders are translated offline only, so the
|
||||
// outputs are captured once on the Mac: DM_GLSLC_CAPTURE=<dir> writes every successful compile's GLSLC output (magic 0x19866891, the whole output's length at +0x44) to
|
||||
// <dir>/<FNV-1a of the output>.glslc. shader_split.py reads that folder like any other source of GLSLC outputs (profile key shader_glslc_capture).
|
||||
// <dir>/<FNV-1a of the output>.glslc. shader_split.py reads that folder like any other source of GLSLC outputs (the NSP script makes it for a game whose main holds GLSL sources and imports glslcCompile).
|
||||
// The compile object's first word points at its last results {GLSLCcompilationStatus *, GLSLCoutput *}; the status has "success" at +0xc and "allocation error" at +0xd, which is what the
|
||||
// game itself reads after the call (main+0x9c8a04)
|
||||
#include "hle.h"
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
# Xcode builds only the app; tools/pack_ipa.py then packs it with the shader libraries, the add-on content and the RomFS straight from where they are (no Payload/ copy, nothing through the home disk). At install time a signing tool (AltStore / Sideloadly etc.) swaps in your own certificate and bundle ID.
|
||||
# Prerequisites: the rewriter (patched directory) and the iOS shader build (IOSOUT) have been run as in the README; Xcode and xcodegen are installed.
|
||||
# Usage: dm/ios/tools/build_ipa.sh [output.ipa]; environment: GAME=game profile (dm/games/<base game title ID>, or one in DM_GAMES_DIR; required), W=the game's work folder (required: <NSP folder>/<title ID>-work, never the home folder), PATCHED / ROMFS / MSL_IOS can each be overridden
|
||||
# AOC=add-on content directory (default $W/data/aoc; when it exists it goes to DmIOS.app/aoc/<index>/; an empty string = none)
|
||||
# AOC=add-on content directory (default $W/data/aoc; when it exists it goes to DmIOS.app/aoc/<index>/; an empty string = none), GFXAPI=nvn / vulkan (default: by the shader libraries in MSL_IOS)
|
||||
# The app name is the game name in the base NSP's control data ($W/data/v0/control.nacp, else the one next to the RomFS), the app version is the display version in the packed game's control data (the one next to the RomFS:
|
||||
# nx_extract saves the update's there when an update is packed, else the base game's), the bundle ID is org.nxnative.<base game title ID in lowercase> (DM_DISPLAY_NAME / DM_APP_VERSION / DM_BUNDLE_ID override them);
|
||||
# the IPA name, code modules, RomFS marker and shader tables come from the game profile
|
||||
@@ -28,8 +28,9 @@ export LC_ALL=C DM_BUNDLE_ID=${DM_BUNDLE_ID:-org.nxnative.$(echo "$GAME" | tr 'A
|
||||
for f in $MODS; do [ -f "$PATCHED/$f.patched.bin" ] && [ -f "$PATCHED/$f.fixups.tsv" ] || die "no rewritten module $PATCHED/$f (run dm/tools/patch_nso.py first)"; done
|
||||
[ -e "$ROMFS/$MARKER" ] || die "no RomFS at $ROMFS (missing $MARKER)"
|
||||
V=$(sed -n 's/^#define DMSH_CHAIN "\(.*\)"/\1/p' "$GD/dmsh_chain.h")
|
||||
# graphics vulkan (Hades): the shaders are the game's SPIR-V, translated by dm/gfx/tools/vk_shaders.py into $MSL_IOS/vk; there are no NVN chain libraries
|
||||
GFXAPI=$(python3 "$DM/games/game.py" "$GAME" graphics 2> /dev/null) || GFXAPI=nvn
|
||||
# A Vulkan game (Hades): the shaders are the game's SPIR-V, translated by dm/gfx/tools/vk_shaders.py into $MSL_IOS/vk; there are no NVN chain libraries. GFXAPI=nvn / vulkan says which
|
||||
# (the NSP script passes what main imports); without it, the shader libraries in $MSL_IOS tell: the chain's b-$V, else vk
|
||||
if [ -z "${GFXAPI:-}" ]; then if ls "$MSL_IOS/b-$V" 2> /dev/null | grep -q '_b.metallib$'; then GFXAPI=nvn; elif [ -f "$MSL_IOS/vk/vk_shaders.metallib" ]; then GFXAPI=vulkan; else GFXAPI=nvn; fi; fi
|
||||
if [ "$GFXAPI" = vulkan ]; then [ -f "$MSL_IOS/vk/vk_shaders.metallib" ] || die "no iOS Vulkan shader library $MSL_IOS/vk (dm/gfx/tools/vk_shaders.py \"$ROMFS\" <Mac dir> --ios \"$MSL_IOS\")"
|
||||
else [ -n "$V" ] && ls "$MSL_IOS/b-$V" 2> /dev/null | grep -q '_b.metallib$' || die "no iOS shader libraries $MSL_IOS/b-$V (IOSOUT=$MSL_IOS dmsh_build.py ...)"; fi
|
||||
|
||||
|
||||
@@ -145,7 +145,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **SDK 6 (Fire Emblem: Three Houses, 6.5.0)**:
|
||||
- **The first loading screen never ends; `DM_FS=1` shows `save:/system` "not found" and no `fs mount save`** → in SDK 6 `account::OpenPreselectedUser` returns a Result, and the game skips its save mount when it is not 0 (`main+0x53b7e0`). The host's hook returned void and left the handle pointer in w0. It returns 0 now; SDK 7 and later callers ignore it.
|
||||
- **No sound; red `audren: command 1 in the command list is malformed (magic 0 size 0)`** → SDK 6's DSP command header is +0x04 size (s16), +0x06 command ID (s16), +0x10 enabled, where SDK 8–20 have +0x04 enabled, +0x05 ID, +0x06 size (`CommandListProcessor::Process`, `sdk+0xaaa00`); its DeviceSink is ID 0x10. `capture` in `hle_audren.c` takes the layout whose walk finds the 0xcafebabe magic on every command of the first full list.
|
||||
- **Shaders missing from the map that no game file holds** → KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (`glslcCompile` from `main+0x9c89f8`; the sources are strings in `main`). `DM_GLSLC_CAPTURE=<dir>` saves every compile's output (`hle_glslc.c`; the compile object's first word points at {status, output}), and with the profile key `shader_glslc_capture on` `shader_split.py` reads `<version dir>/glslc_capture`. The outputs are identical on every run.
|
||||
- **Shaders missing from the map that no game file holds** → KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (`glslcCompile` from `main+0x9c89f8`; the sources are strings in `main`). `DM_GLSLC_CAPTURE=<dir>` saves every compile's output (`hle_glslc.c`; the compile object's first word points at {status, output}), and `shader_split.py` reads `<version dir>/glslc_capture` (the NSP script makes it for any game whose `main` holds GLSL sources and imports `glslcCompile`). The outputs are identical on every run.
|
||||
- **SDK 8 (Link's Awakening, 8.2.0)**:
|
||||
- **Abort in `os::WaitSystemEvent` under `oe::PopNotificationMessage`** → the blocking pop waits on the system event `oe::Initialize` would have created, and the host's `Initialize` is a no-op. The host now implements it on its notification queue (a condition variable, then the `TryPop` path).
|
||||
- **SIGSEGV in `vi::SetLayerCrop`** → the SDK's version dereferences vi service objects the host never made; `SetLayerCrop`, `DestroyLayer`, `CloseDisplay` and `vi::Finalize` are host no-ops.
|
||||
|
||||
+19
-18
@@ -12,12 +12,12 @@ Do not stop at the obvious archive. In Mario the main `bfsha` held 4,888 of the
|
||||
- **Unity Shader objects** (Engage): Unity 2020.3 keeps each shader's compiled programs in its `Shader` object (class 48), in UnityFS bundles (`StreamingAssets/aa`) and in the player's own serialized files (`data.unity3d`, `Resources/unity default resources`). `dm/gfx/tools/unityfs.py` reads them: UnityFS blocks (LZ4 / LZMA), the serialized file, the object through its type tree, then `compressedBlob` per platform (LZ4 segments; each segment starts with a u32 count and 12-byte `{offset, length, segment}` entries). The player's files have no type trees; they borrow the bundles' `Shader` tree with the same type hash (same Unity build, same layout). Type-tree strings with the high bit set index Unity's common string table, which `unityfs.common_strings` reads from the game's own `main` (`AABB\0AnimationClip\0…`). On Switch every subprogram is one whole NVN program: its code is a 0x60-byte header of four `i32 ×6` arrays indexed by NVN stage (0 vertex, 1 fragment, 2 geometry, 3 tess control, 4 tess evaluation, 5 compute): an unused / scratch size, the block offset from +0x60 (−1 = no stage), the control size and the code size. Each block is the control section followed by the code area. The vertex and fragment stages of one program form a pair. Engage: 209 `Shader` objects in 188 bundles plus the engine files, 9,269 stage shaders (4,634 vertex / pixel pairs and one geometry shader), no compute shaders.
|
||||
- **Unity 2022 Shader objects** (Ace Attorney Investigations Collection, Unity 2022.3.17f1): the program code keeps the 0x60-byte stage header, but two things changed. From Unity 2022 the blob also holds each program's parameter tables as entries of their own (the parsed form's `m_ParameterBlobIndices`; their bytes start `PGlobals`), and the player's programs are listed in `m_PlayerSubPrograms[tier][].m_BlobIndex` (`m_SubPrograms` is empty); `unity_code_blobs` keeps only those. UnityPy's type trees for 2020.3 (Engage) and 2021.3 have neither field, so those read every entry as before. Subprogram version 202012090 (from Unity 2021.2) also has no local keyword list, so `unityfs.subprogram` reads no second keyword array (with one it read the code length as a keyword count). The player's serialized files have no type trees and the Addressables bundles hold no `Shader`, except the `unitybuiltinshaders` bundle, whose tree has the same hash and is borrowed in the second pass. 4,951 stage shaders: 2,475 vertex / pixel pairs (4,294 in the player's files, 517 built-ins, 140 in bundles) and one geometry shader, which Metal cannot express (its build fails). armyja's issue #17 proposed UnityPy and a reader of its own; neither was needed.
|
||||
- **MSHD** (Metroid Dread, MercurySteam's engine: `system/shd/*.bshdat`, the only place its shaders live; nothing in `main` or the `.pkg` packs): each stage is a control section, whose length is at control +0x20, sometimes followed by a small compiler-option table (`fastmath true`), then the code area at the next code header. Every file holds vertex / pixel stages in alternating order (1, 2 or 4 programs) and is paired two by two. The control section carries NV assembly text after its first 0x880 bytes, and the SASS disassembly follows each code area, so the binding layout can be read straight from the files (`TEX` / `TEXS` handle immediates). 478 files, 1,342 stages, 1,010 unique keys (359 vertex, 651 pixel), 659 pairs.
|
||||
- **GLSLC outputs in `main`** (13 Sentinels, Vanillaware's engine: no shader file in RomFS, whose six CRI CPK archives hold textures, scripts, CRI audio and USM movies): 838 GLSLC compiler outputs sit in `main`'s read-only segment, each referenced from a `{pointer, size, 0}` table in `.data`. An output starts with the magic `0x19866891`, its total size is at +0x44, and it holds exactly one control section with the code area right after it (control +0x20 = control length); a debug section with the NV assembly and the GLSL source follows. Engage's `main` also has 11 of them, so `shader_split.py` reads them only when the profile sets `shader_main_glslc on` (and accepts only outputs that pass the control / code / length check). The stage comes from SPH word 0, as for BAHS.
|
||||
- **G2S in RDB archives** (Hyrule Warriors: Age of Calamity, Koei Tecmo's KTGL): each material's shaders are a G2S container (`_S2G3000`, type KTID `0x7bcd279f`; 23,108 entries in `asset/system.rdb.bin*`, 2.3 GB inflated) of GLSLC outputs, as in 13 Sentinels' `main`, inside zlib-chunked IDRK entries ([01](01-extract.md)); one `_DOK0000` container (type `0x20a6a0bb`, the engine's system resources) holds 151 more bare GLSLC outputs, 21 of them drawn in the first battle. `shader_split.py` maps the RDB files and inflates only entries that start with those two magics (`idrk_shaders`), with `shader_main_glslc on` for the 117 outputs in `main`. Many outputs carry 0x850-byte control sections and the compute ones have no code header, so `glslc_shaders` accepts controls from 0x760 bytes and takes the stage of headerless code from the thread group size at control +0x748. 40,726 shaders (37,043 pixel, 3,440 vertex, 241 compute, 2 geometry), Mario's texture handle layout (`0x20 + 8 × slot`); pairs come from `shader_separable on`. The older KTGL of Three Houses packs the same G2S containers differently (next item).
|
||||
- **GLSLC outputs in `main`** (13 Sentinels, Vanillaware's engine: no shader file in RomFS, whose six CRI CPK archives hold textures, scripts, CRI audio and USM movies): 838 GLSLC compiler outputs sit in `main`'s read-only segment, each referenced from a `{pointer, size, 0}` table in `.data`. An output starts with the magic `0x19866891`, its total size is at +0x44, and it holds exactly one control section with the code area right after it (control +0x20 = control length); a debug section with the NV assembly and the GLSL source follows. Engage's `main` also has 11 of them (Unity's own), and `shader_split.py` reads them in every game's `main`, accepting only outputs that pass the control / code / length check. The stage comes from SPH word 0, as for BAHS.
|
||||
- **G2S in RDB archives** (Hyrule Warriors: Age of Calamity, Koei Tecmo's KTGL): each material's shaders are a G2S container (`_S2G3000`, type KTID `0x7bcd279f`; 23,108 entries in `asset/system.rdb.bin*`, 2.3 GB inflated) of GLSLC outputs, as in 13 Sentinels' `main`, inside zlib-chunked IDRK entries ([01](01-extract.md)); one `_DOK0000` container (type `0x20a6a0bb`, the engine's system resources) holds 151 more bare GLSLC outputs, 21 of them drawn in the first battle. `shader_split.py` maps the RDB files and inflates only entries that start with those two magics (`idrk_shaders`), and the 117 outputs in `main` as in every game. Many outputs carry 0x850-byte control sections and the compute ones have no code header, so `glslc_shaders` accepts controls from 0x760 bytes and takes the stage of headerless code from the thread group size at control +0x748. 40,726 shaders (37,043 pixel, 3,440 vertex, 241 compute, 2 geometry), Mario's texture handle layout (`0x20 + 8 × slot`); pairs come from the single-stage cover (`separable_pairs`). The older KTGL of Three Houses packs the same G2S containers differently (next item).
|
||||
- **SPKG in LINKDATA** (Fire Emblem: Three Houses, the older KTGL): shader packages (`SPKG`: `patch*/nx/shader/*.spkg` and 132 entries of `DATA1.bin`) of G2S containers as above, one `_SPK4100` package without G2S (the post effects, `DATA1.bin` entry 549), and map and model archives carrying packages of their own. `kt_shaders` inflates every LINKDATA entry and loose `*.gz` / `*.spkg` file ([01](01-extract.md)), takes every GLSLC output that passes the triple check, and scans its SARCs for BNSH; `DATA1.bin` is split into 64 scan jobs by entry range (`linkdata_jobs`). Its GLSLC (output version 0x1d.2) writes the 0x848 / 0x850-byte controls and headerless compute code described above, which `glslc_shaders` used to drop: 24 shaders before, 1,355 after. A few outputs of version 0x1d.0 keep something else at control +0x20 (odd values such as 0x136b), so a control length must be a multiple of 4. Nine more shaders exist only at run time (Seen in Fire Emblem: Three Houses, below).
|
||||
- **SDF in RE Engine PAKs** (Monster Hunter Rise): shader definitions (`SDF\0`, 2,685 files in the PAK view, [01](01-extract.md)): u16 version 1, u16 entry count, u64, then 0x108-byte entries from +0x10. Entry +0x10 holds six u64 offsets (vertex, hull, domain, geometry, pixel, compute; 0 = none), each to `{u32 control length, u32 code length}` followed by the control section (0x880 bytes) and the code area; compute code has no header. An entry's stages are one program, so its vertex / pixel stages are a pair. Empty definitions are 16 bytes of version 0. `kpka_jobs` maps the PAKs, takes the newest copy of each path, peeks at 4 decompressed bytes and inflates only SDF entries (`sdf_shaders`): 63,111 shaders (30,233 vertex, 21,475 pixel, 11,271 compute, 53 geometry, 33 hull, 46 domain) and 102,345 pairs, in 5 s with 12 jobs.
|
||||
- **Unreal Engine shader code archives** (Octopath Traveler 0: `ShaderArchive-Global-SF_GLSL_SWITCH_MOBILE-SP_GLSL_SWITCH_MOBILE.ushaderbytecode`, 1,219 shaders, and `ShaderArchive-Octopath_Traveler0_Chunk0-…`, 9,377, both stored in the pak, [01](01-extract.md)): version 2 archives of shader map hashes, shader hashes, shader map entries, shader entries `{u64 offset, u32 size, u32 raw size, u8 frequency}` (17 bytes packed; frequency 0 vertex, 3 pixel, 5 compute), preload entries and shader indices, then the code. Every shader is Oodle-compressed (Mermaid, block header `8C 0A`); there is no Oodle decoder in this repository, so the game's own `OodleLZ_Decompress` (statically linked into `main`; the host finds it as the one function that takes the address of the library's own name string "OodleLZ_Decompress", its profiler scope: Octopath Traveler 0 `main+0x4f77e10`, Pokemon Legends: Z-A `main+0x1a9c9e0`) decodes them in the Mac host: profile key `shader_ue_decode`, `DM_OODLE_JOB` (`dm/host/hle_oodle.c`), the NSP script's `decode_ue_shaders`. A decoded shader is an `LSLG` record: `LSLG`, a stage tag (`SV`, `SP`, `SC`), a header with the source name (`/Engine/Private/…usf:MainPS`), then `u32` code length, the code area, `u32` control length (0x880), the control section and a 24- or 29-byte tail, all at unaligned offsets (`uepak.lslg`). The host keys came out exactly: 37 of 37 shaders a first run reported missing were in the two archives. 10,324 unique shaders (1,457 vertex, 6,346 pixel, 2,521 compute); vertex and pixel shaders combine per draw, so `shader_separable on`. Bindings: separate textures and samplers, Engage's layout (`0x168` / `0x568`); 61 vertex shaders pick a texture slot at run time (profile key `tex_dyn_slots`, below), 2,084 shaders do atomics on buffer images (below). 16 compute shaders do atomics on a 3D `r32ui` image, which SPIRV-Cross's MSL 2.3 image-atomic emulation indexes with an `int3` (`array subscript is not an integer`): all are Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run), so they stay unbuilt.
|
||||
- **BNSH in Trinity packs** (Pokémon Legends: Z-A, Game Freak's engine): every shader is a BNSH inside the Oodle-compressed files of `arc/data.trpfs` ([01](01-extract.md)). With the profile key `shader_oodle on`, `trpfs_jobs` / `trinity_shaders` in `shader_split.py` name the packs from `data.trpfd`, decode every file with the game's own Oodle in the Mac host (`DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script) and scan it like a loose file. 28,603 shaders (11,633 vertex, 16,845 pixel, 125 compute) and 26,051 variation pairs: the material shaders in `system_resource/gfx2/shader/NX64/env_ikkaku` (`material/standard.bnsh`, `material/transparent.bnsh`, `system/light.bnsh`, and the `special/imgui.trsha` pack with the engine's system shaders), particle effects (`.ptcl` / `.trtml` in `ik_effectdata`, 19,842 of the shaders), UI layouts (`__ArchiveShader.bnsh` in the `.truiv` packs of `uidata`) and the 40 BNSH in `main`. The layout archives' control sections are 104 KB long, and their stage descriptors run 456 bytes past the control's own length (control +0x20), with bytes that differ between copies of one shader; `same_shader` compares copies only up to the control's own length.
|
||||
- **Unreal Engine shader code archives** (Octopath Traveler 0: `ShaderArchive-Global-SF_GLSL_SWITCH_MOBILE-SP_GLSL_SWITCH_MOBILE.ushaderbytecode`, 1,219 shaders, and `ShaderArchive-Octopath_Traveler0_Chunk0-…`, 9,377, both stored in the pak, [01](01-extract.md)): version 2 archives of shader map hashes, shader hashes, shader map entries, shader entries `{u64 offset, u32 size, u32 raw size, u8 frequency}` (17 bytes packed; frequency 0 vertex, 3 pixel, 5 compute), preload entries and shader indices, then the code. Every shader is Oodle-compressed (Mermaid, block header `8C 0A`); there is no Oodle decoder in this repository, so the game's own `OodleLZ_Decompress` (statically linked into `main`; the host finds it as the one function that takes the address of the library's own name string "OodleLZ_Decompress", its profiler scope: Octopath Traveler 0 `main+0x4f77e10`, Pokemon Legends: Z-A `main+0x1a9c9e0`) decodes them in the Mac host: `DM_OODLE_JOB` (`dm/host/hle_oodle.c`), the NSP script's `decode_ue_shaders` for any game with Unreal paks. A decoded shader is an `LSLG` record: `LSLG`, a stage tag (`SV`, `SP`, `SC`), a header with the source name (`/Engine/Private/…usf:MainPS`), then `u32` code length, the code area, `u32` control length (0x880), the control section and a 24- or 29-byte tail, all at unaligned offsets (`uepak.lslg`). The host keys came out exactly: 37 of 37 shaders a first run reported missing were in the two archives. 10,324 unique shaders (1,457 vertex, 6,346 pixel, 2,521 compute); vertex and pixel shaders combine per draw, so every vertex shader is paired with the single-stage cover. Bindings: separate textures and samplers, as Engage's (`0x168` / `0x568`); 61 vertex shaders pick a texture slot at run time (profile key `tex_dyn_slots`, below), 2,084 shaders do atomics on buffer images (below). 16 compute shaders do atomics on a 3D `r32ui` image, which SPIRV-Cross's MSL 2.3 image-atomic emulation indexes with an `int3` (`array subscript is not an integer`): all are Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run), so they stay unbuilt.
|
||||
- **BNSH in Trinity packs** (Pokémon Legends: Z-A, Game Freak's engine): every shader is a BNSH inside the Oodle-compressed files of `arc/data.trpfs` ([01](01-extract.md)). For any game with Trinity packs (`ONEPACK` files), `trpfs_jobs` / `trinity_shaders` in `shader_split.py` name the packs from `data.trpfd`, decode every file with the game's own Oodle in the Mac host (`DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script) and scan it like a loose file. 28,603 shaders (11,633 vertex, 16,845 pixel, 125 compute) and 26,051 variation pairs: the material shaders in `system_resource/gfx2/shader/NX64/env_ikkaku` (`material/standard.bnsh`, `material/transparent.bnsh`, `system/light.bnsh`, and the `special/imgui.trsha` pack with the engine's system shaders), particle effects (`.ptcl` / `.trtml` in `ik_effectdata`, 19,842 of the shaders), UI layouts (`__ArchiveShader.bnsh` in the `.truiv` packs of `uidata`) and the 40 BNSH in `main`. The layout archives' control sections are 104 KB long, and their stage descriptors run 456 bytes past the control's own length (control +0x20), with bytes that differ between copies of one shader; `same_shader` compares copies only up to the control's own length.
|
||||
- **BAHS** (agl `sharcb`, engine utility shaders): code region at header +0x1c (size +0x18). Binary entries in the table area are 16 bytes `{code offset, 0, control size (0x880/0x88c; 0x848 in BAHS version 9, Super Mario 3D World), signed offset from this entry to the control section}`. Control sections are shared between programs, so pair by entry, never by searching for code near a control.
|
||||
- **Shorter control sections (SDK 10, Super Mario 3D World + Bowser's Fury)**: the control section stores its own length at +0x20. Mario's, Engage's, Dread's and 13 Sentinels' are all at least 0x880 bytes; 3D World has 2,888 of 0x850 bytes (its ShaderData bfsha and six layouts) and 0x848 in BAHS version 9, each followed directly by other data. The key therefore hashes the first `min(0x880, control +0x20)` bytes (`ctl_len` in `shader_split.py`, `keys_hash` in `hle_nvn.c`); the other games' tables came out byte-identical.
|
||||
- **Compute code without a header (SDK 10)**: 3D World's particle compute shaders start their instructions at code +0 (like D2R), not after a `0x12345679` header. `shader_split.py` accepts both; `dmsh_build.py` strips only a header that is there.
|
||||
@@ -32,26 +32,27 @@ The host key is `sha256(control[0:0x880] + code)[:16]` (`prog_get` in `nvn_mtl.m
|
||||
|
||||
## 3. Find the binding layout by census
|
||||
|
||||
The NVN driver puts binding tables in driver constant banks, and the layout changed between NVN versions. Don't guess: translate everything once (the translator alone takes seconds) and tabulate what the shaders reference.
|
||||
The NVN driver puts its binding tables in driver constant banks. Every game so far, NVN 53 and 55 drivers from SDK 6 to 21, has the one layout below, which `dmsh_build.py` decodes and `nvn_mtl.m` fills at run time. Check a new game against it with a census of what its translated shaders reference (the translator alone takes seconds) rather than assuming it.
|
||||
|
||||
| Binding | D2R (NVN 53) | Mario (NVN 55) | Engage (Unity, SDK 12) | How it was measured |
|
||||
|---|---|---|---|---|
|
||||
| Bound texture handles (texture buffer) | `0x168 + 8 × slot`, 128 slots | **`0x20 + 8 × slot`**, 32 slots | `0x168 + 8 × slot`, 128 slots | Mario: immediate handles of all TEX instructions, 0x20…0xF0 in steps of 8. Engage: the translator's texture names (`fp_t_cb<slots>_<handle>`, Ryujinx `TextureHandle` packing) give word 0x5A = 0x168 for the texture and 0x15A = 0x568 for the sampler |
|
||||
| Image handles | `0x120 + 8 × slot` | `0x120 + 8 × slot` | not measured (no compute shaders) | agl compute shaders: 0x120 / 0x128 / 0x130 |
|
||||
| Separate samplers | `0x568 + 8 × slot` | not used | `0x568 + 8 × slot` (Unity binds only separate textures and samplers) | — |
|
||||
| Uniform buffers | `c[3 + slot]` | same | same | |
|
||||
| Storage buffers | `c0[base + 16 × slot]`, base vs 0x110 / cs 0x310 / ps 0x510 | same | not measured | |
|
||||
| Largest point size | not measured | not measured | not measured | Super Mario Odyssey (SDK 7): GLSLC clamps a shader-written `gl_PointSize` to `c0 + 0x610` (`alPointSpriteShader`: `min(size, c0[0x610])`). It is not the `SetPointSize` value, which the game leaves at 1. The host writes 2047 there for the graphics stages |
|
||||
| Binding | Where | How it was measured |
|
||||
|---|---|---|
|
||||
| Combined texture handles (`nvnCommandBufferBindTexture`) | **`c2[0x20 + 8 × slot]`**, 32 slots; a higher slot's word lies in the image table | Mario: immediate handles of all TEX instructions, 0x20…0xF0 in steps of 8. Hades II binds slot 34 (0x130) and no image |
|
||||
| Image handles | `c2[0x120 + 8 × slot]`, 8 slots | agl compute shaders: 0x120 / 0x128 / 0x130 |
|
||||
| Separate textures (`BindSeparateTexture(s)`) | `c2[0x168 + 8 × slot]`, 128 slots: the host's texture slots 128 + n (`NVN_SEPTEX_SLOT0`) | Engage (Unity binds only separate textures and samplers): the translator's texture names (`fp_t_cb<slots>_<handle>`, Ryujinx `TextureHandle` packing) give word 0x5A = 0x168 for the texture and 0x15A = 0x568 for the sampler. D2R (NVN 53) had the same |
|
||||
| Separate samplers | `c2[0x568 + 8 × slot]`, 128 slots | as above |
|
||||
| Uniform buffers | `c[3 + slot]` | |
|
||||
| Storage buffers | `c0[base + 16 × slot]`, base vs 0x110 / cs 0x310 / ps 0x510 | |
|
||||
| Largest point size | `c0 + 0x610` | Super Mario Odyssey (SDK 7): GLSLC clamps a shader-written `gl_PointSize` to it (`alPointSpriteShader`: `min(size, c0[0x610])`). It is not the `SetPointSize` value, which the game leaves at 1. The host writes 2047 there for the graphics stages |
|
||||
|
||||
Ace Attorney Investigations Collection (Unity 2022.3, SDK 17.5.3) uses Engage's layout: built with `nvn_tex_base 0x168` and `nvn_samp_base 0x568`, all 4,950 translatable shaders decode every binding (`shaders with a binding whose slot was not decoded 0`), and 190 read `c2` rows as data.
|
||||
Ace Attorney Investigations Collection (Unity 2022.3, SDK 17.5.3) binds separate textures and samplers like Engage: all 4,950 translatable shaders decode every binding (`shaders with a binding whose slot was not decoded 0`), and 190 read `c2` rows as data.
|
||||
|
||||
Metroid Dread (SDK 11.4.3) uses Mario's texture handle layout, `0x20 + 8 × slot`: the `TEX` / `TEXS` handle immediates in its shader files' own SASS text are words 0x8–0x22 in steps of 2. Its full build leaves no unresolved bindings and no unfillable driver-bank reads; the other rows are not measured separately.
|
||||
|
||||
Hades II (SDK 19.3.9) has Mario's layout too, but its 88 shaders bind textures at fixed, sparse slots up to 34 (`fp_t_tcb_4C`: c2 word 0x4C = 0x20 + 8 × 34, in `SceneFade.frag`). With the profile's 32 slots, `dmsh_build.py` reported that binding as not decoded ("shaders with a binding whose slot was not decoded 1", which prints a red line when the shader loads). Its `nvn_tex_slots` is 35. The census of `fp_t_*` names in the translated libraries gives the slots in use; the host's tables hold up to 128.
|
||||
Hades II (SDK 19.3.9) binds combined textures at fixed, sparse slots up to 34 (`fp_t_tcb_4C`: c2 word 0x4C = 0x20 + 8 × 34, in `SceneFade.frag`). With a 32-slot table, `dmsh_build.py` reported that binding as not decoded ("shaders with a binding whose slot was not decoded 1", which prints a red line when the shader loads). The combined table now runs up to the separate textures at 0x168 (41 slots); past slot 31 it shares its words with the image table, and the replay writes a bound texture there when the image slot is unbound. The census of `fp_t_*` names in the translated libraries gives the slots in use.
|
||||
|
||||
Monster Hunter Rise (RE Engine, SDK 11.4.3) uses both tables at once: combined handles from `nvnCommandBufferBindTexture` at `0x20 + 8 × slot` and separate textures from `BindSeparateTexture(s)` at `0x168 + 8 × slot`, with separate samplers at 0x568, and binds slot 0 of both in the same frames. A census of 23,478 translated shaders' texture names: 33,813 combined (`fp_t_tcb_1E`), 29,342 separate textures with a constant sampler ID (handle type 3, `fp_t_cb65538_3041005C`: texture at c2 word 0x5C, sampler 0x104 ORed in), 8,265 separate texture + separate sampler (type 1, `fp_t_cb196610_1017815A`), 5,227 separate textures whose own handle carries the sampler (`fp_t_tcb_5A`), about 1,500 uniform-buffer handles with a constant sampler (`fp_t_cb65541_30410028`), and 121 fully dynamic handles (type 4, `fp_t2d_cb0_40000000`, not supported). The profile keys `nvn_septex_base` / `nvn_septex_slots` give the separate table: separate binds go to slots 128 + n of the replay's table and their handles decode to 128 + n; a constant sampler is the binding kind `samp_id` (`NVN_RB_SAMPI`, reg = the sampler ID).
|
||||
Monster Hunter Rise (RE Engine, SDK 11.4.3) uses both tables at once: combined handles from `nvnCommandBufferBindTexture` at `0x20 + 8 × slot` and separate textures from `BindSeparateTexture(s)` at `0x168 + 8 × slot`, with separate samplers at 0x568, and binds slot 0 of both in the same frames. A census of 23,478 translated shaders' texture names: 33,813 combined (`fp_t_tcb_1E`), 29,342 separate textures with a constant sampler ID (handle type 3, `fp_t_cb65538_3041005C`: texture at c2 word 0x5C, sampler 0x104 ORed in), 8,265 separate texture + separate sampler (type 1, `fp_t_cb196610_1017815A`), 5,227 separate textures whose own handle carries the sampler (`fp_t_tcb_5A`), about 1,500 uniform-buffer handles with a constant sampler (`fp_t_cb65541_30410028`), and 121 fully dynamic handles (type 4, `fp_t2d_cb0_40000000`, not supported). Separate binds go to slots 128 + n of the host's table and their handles decode to 128 + n, as for every game; a constant sampler is the binding kind `samp_id` (`NVN_RB_SAMPI`, reg = the sampler ID).
|
||||
|
||||
The profile's `nvn_*` keys hold these constants; `dmsh_build.py` decodes the bindings from them (`decode_tex`). `Program.cs` still has Mario's `TexBase` built in, but uses it only to look up per-slot texture / image types in the facts table, so a profile with other bases builds as long as its facts table has no such entries (Engage's has none; `dmsh_build.py` stops otherwise); the replay layer fills driver banks with the same layout: the texture, image and separate-sampler tables of `c2` (`NVN_RB_DRV2` in `encode_args`). Some shaders read `c2` rows as data at run time (Engage: 243, the texture and sampler handles of slots 0–11); the build lists reads outside those tables as `cannot fill`. After a full build, the summary must say `shaders with a binding whose slot was not decoded 0`.
|
||||
`dmsh_build.py` decodes the bindings with this layout (`decode_tex`), `Program.cs` numbers slots the same way when it looks up per-slot texture / image types in the facts table, and the replay layer fills driver banks with it: the combined, image, separate texture and separate sampler tables of `c2` (`NVN_RB_DRV2` in `encode_args`). Some shaders read `c2` rows as data at run time (Engage: 243, the texture and sampler handles of slots 0–11); the build lists reads outside those tables as `cannot fill`. After a full build, the summary must say `shaders with a binding whose slot was not decoded 0`.
|
||||
|
||||
## 4. Bindless textures
|
||||
|
||||
@@ -118,7 +119,7 @@ So the translator leaves z alone (`QueryGraphicsState` depth mode false in `Prog
|
||||
|
||||
### Seen in 13 Sentinels
|
||||
|
||||
- **Single-stage programs**: the game builds one NVN program per GLSLC output (838 `ProgramSetShaders` at startup) and combines vertex and pixel programs per draw with `BindProgram` stage bits, so the composite-program path from Metroid Dread does all the pairing. The real pairs exist only at draw time. Every pixel shader input is a plain `float4` at locations 1–9 (the SPH input map, words 6–13, matches the translated MSL inputs for all 375 pixel shaders), so with `shader_separable on` `shader_split.py` pairs every vertex shader with a *cover*: the fewest pixel shaders whose inputs together are every input any pixel shader reads (here one shader). `vs_pad` then gives each vertex shader all of them, and any vertex / pixel combination links.
|
||||
- **Single-stage programs**: the game builds one NVN program per GLSLC output (838 `ProgramSetShaders` at startup) and combines vertex and pixel programs per draw with `BindProgram` stage bits, so the composite-program path from Metroid Dread does all the pairing. The real pairs exist only at draw time. Every pixel shader input is a plain `float4` at locations 1–9 (the SPH input map, words 6–13, matches the translated MSL inputs for all 375 pixel shaders), so `shader_split.py` pairs every vertex shader with a *cover*: the fewest pixel shaders whose inputs together are every input any pixel shader reads (here one shader). `vs_pad` then gives each vertex shader all of them, and any vertex / pixel combination links.
|
||||
- **`dmsh_build.py`'s shader cache is keyed without the pairs**: a vertex shader whose pairs changed is restored unpadded from `cas/`. After changing a profile's pairs, delete that profile's cache (`build/cache/metal_shader_cache/cas` and `b-*`) and rebuild. The NSP script always starts from its own cache, which has the same problem only when a profile's pairs change between two of its runs.
|
||||
- **Triangle fans and indexed quads**: Metal has neither. Non-indexed fans draw through a shared `(0, i, i+1)` index buffer with `first` as the base vertex; indexed fans and quads are expanded on the CPU into a 32-bit triangle list in the frame arena at replay (`nvn_mtl.m`, as for 8-bit indices).
|
||||
- **Vertex streams 14 and 15**: the host bound NVN stream *n* to Metal vertex buffer 16 + *n*, but Metal has slots 0–30 only (30 is the host's constant fallback), and stream 15 hit the assertion `buffer index (31) must be < 31` a minute and a half into the prologue. Translated shaders use only `buffer(0)`, so streams 14 and 15 now go to slots 14 and 15 (`VB_SLOT`).
|
||||
|
||||
+12
-12
@@ -47,8 +47,8 @@ What the script does:
|
||||
1. It finds the game profile from the title ID (the `[title ID]` in the file names, or the NSP's ticket; the update by its `[vN]` tag).
|
||||
2. It extracts the NSPs.
|
||||
3. It rewrites the code modules, checking each build ID against the profile. A provisional profile (a game without one in `dm/games`) then gets `mem_used` from the rewritten modules, and, when the game has virtual address memory and sets a heap, a heap check: the Mac host is built into `<id>_build/tools/dmhost` and the game boots in it until its first heap request (a few seconds, log `<id>_build/logs/heap_check.log`), which sizes the heap region (see [dm/games/README.md](../dm/games/README.md), Adding a game).
|
||||
4. For a profile with `shader_glslc_capture on` (Three Houses), it captures the shaders the game compiles at run time, which no game file holds. It builds the Mac host into `<id>_build/tools/dmhost` and boots the game in it, headless and silent, with `DM_GLSLC_CAPTURE` pointing at `<id>_build/extract/<version>/glslc_capture`. The save, temporary storage and log all stay in `<id>_build/`. It stops the game as soon as every capture file named in the profile's shader map is there (Three Houses: 9, about a second after launch), and gives up after 30 s. A folder that already holds them all is reused.
|
||||
A Vulkan profile with `shader_vk_capture on` (Super Mario Galaxy) gets the same kind of run: the game boots in the Mac host with `DM_VK_CAPTURE` pointing at `<id>_build/extract/<version>/vk_capture` until every SPIR-V key in the profile's `vk_capture.tsv` is there (Galaxy: 3,089, about 7 s after launch), and gives up after 120 s.
|
||||
4. For a game whose `main` holds GLSL sources and calls `glslcCompile` (Three Houses), it captures the shaders the game compiles at run time, which no game file holds. It builds the Mac host into `<id>_build/tools/dmhost` and boots the game in it, headless and silent, with `DM_GLSLC_CAPTURE` pointing at `<id>_build/extract/<version>/glslc_capture`. The save, temporary storage and log all stay in `<id>_build/`. It stops the game as soon as every capture file named in the profile's shader map is there (Three Houses: 9, about a second after launch), and gives up after 30 s. A folder that already holds them all is reused.
|
||||
A Vulkan game without SPIR-V files in its RomFS (Super Mario Galaxy) gets the same kind of run: the game boots in the Mac host with `DM_VK_CAPTURE` pointing at `<id>_build/extract/<version>/vk_capture` until every SPIR-V key in the profile's `vk_capture.tsv` is there (Galaxy: 3,089, about 7 s after launch), and gives up after 120 s.
|
||||
5. It splits out the shaders and checks them against the profile's tables (a Vulkan profile: it translates the RomFS's SPIR-V and the capture with `vk_shaders.py` instead).
|
||||
6. It builds the iOS shader libraries.
|
||||
7. It calls `build_ipa.sh`. Xcode builds the app, and `dm/ios/tools/pack_ipa.py` packs it with the shader libraries, the add-on content and the RomFS straight from where they are. It reads and deflates files on all cores and writes at about the T7's write speed (300 MB/s): Breath of the Wild's 16 GB IPA takes 57 s, Info-ZIP's copy and zip took 2 min 50 s.
|
||||
@@ -71,7 +71,7 @@ On the device, environment switches go in `Documents/env.txt` (one `KEY=VALUE` p
|
||||
## Port another game
|
||||
|
||||
Each game has a profile in `dm/games/<id>/`:
|
||||
- `game.tsv` holds the title ID, known builds, code modules, RomFS marker, memory split, NVN driver tables, app name and bundle ID;
|
||||
- `game.tsv` holds the title ID, known builds, code modules, RomFS marker, memory split, app name and bundle ID;
|
||||
- the shader tables: `shader_map.tsv`, `shader_facts.tsv`, `shader_pairs.tsv`, `c1_full.tsv`, `pso_manifest.tsv`;
|
||||
- `dmsh_chain.h` holds the chain version.
|
||||
|
||||
@@ -196,17 +196,17 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- **Fog:** the Cap Kingdom ground fog parts around Mario through a small fluid simulation fed by point-sprite splats. Their vertex shader clamps the point size to the driver segment's `c0 + 0x610`, which the host left at 0, so no splat was ever drawn and the fog never moved; the host now writes the largest point size there ([04](agents/04-graphics.md)).
|
||||
|
||||
**Hades (`0100535012974000`)**
|
||||
- **Engine:** Supergiant's engine on The Forge renderer, SDK 9.3.0, update 1.0.38233. It renders with **Vulkan**: `main` imports 100 `vk*` functions from subsdk0 (the Vulkan loader, over NVIDIA's driver in subsdk1). The host implements them on Metal itself (`dm/gfx/vk/hle_vk.m`), so only main and sdk are loaded; the profile says `graphics vulkan`.
|
||||
- **Engine:** Supergiant's engine on The Forge renderer, SDK 9.3.0, update 1.0.38233. It renders with **Vulkan**: `main` imports 100 `vk*` functions from subsdk0 (the Vulkan loader, over NVIDIA's driver in subsdk1). The host implements them on Metal itself (`dm/gfx/vk/hle_vk.m`), so only main and sdk are loaded.
|
||||
- **Vulkan layer:** one Tegra X1-like device (Vulkan 1.1); device memory is a shared `MTLBuffer` made on first use, images are their own textures; command buffers are recorded and replayed into one Metal command buffer per submit on the layer's own queue; swapchain images are shown through the NVN layer's present path once their frame's GPU work is done. Names the game asks for through `vkGet*ProcAddr` are logged once each.
|
||||
- **Shaders:** 74 SPIR-V files in `Content/EffectsForgeSwitch`. `dm/gfx/tools/vk_shaders.py <RomFS> <shader dir> [--ios <dir>]` translates them with SPIRV-Cross into `<dir>/vk/`: one library, and a table of the Metal slot each (set, binding) moved to. The game hands the same words to `vkCreateShaderModule`, so the host finds each shader by the hash of its SPIR-V.
|
||||
- **Notification event:** the game polls `oe::GetNotificationMessageEvent` every frame; the SDK's own event was never created (the host's `oe::Initialize` is a no-op), so the host now hands out an event of its own, signalled while a notification is queued.
|
||||
- **Online:** its `EOS_*` imports (Epic Online Services, an NRO it would load through `nn::ro`) stay offline.
|
||||
- **Mac input:** `DM_KEYS="f800:A,f1000:A,f1200:A,f1500:A,f1900:A"` goes through the title and the intro into Tartarus.
|
||||
- **Packing:** for a `graphics vulkan` profile the NSP script runs `vk_shaders.py` (Mac and iOS libraries in `<id>_build/shader/vk` and `shader/ios/vk`) in place of the NVN shader split and chain build.
|
||||
- **Packing:** for a Vulkan game the NSP script runs `vk_shaders.py` (Mac and iOS libraries in `<id>_build/shader/vk` and `shader/ios/vk`) in place of the NVN shader split and chain build.
|
||||
|
||||
**Hades II (`0100A00019DE0000`)**
|
||||
- **Engine:** Supergiant Games' engine, SDK 19.3.9, base game v0 (the NACP's display version is 0.0.126424). Unlike Hades it renders with **NVN**, and its ExeFS holds only main and sdk. The NSP script's provisional profile booted it on the Mac with no host change: title, a new save, the opening and Erebus at 60 fps. It plays on an iPhone.
|
||||
- **Shaders:** 88 BNSH files in `Shaders/NVN/bin` (`.vert` / `.frag`; the `.shader` files list each program's pair). They bind textures at fixed, sparse slots: 0, 1, 3, 6, 8, 11, 15, 21, 25, 26, 30, 31 and 34. Slot 34 (`SceneFade.frag`, c2 0x130) lies past the 32 slots of Mario's layout, so its binding was not decoded until the profile's `nvn_tex_slots` became 35. The game has no image bindings, so the overlap with the image table at 0x120 does not matter.
|
||||
- **Shaders:** 88 BNSH files in `Shaders/NVN/bin` (`.vert` / `.frag`; the `.shader` files list each program's pair). They bind textures at fixed, sparse slots: 0, 1, 3, 6, 8, 11, 15, 21, 25, 26, 30, 31 and 34. Slot 34 (`SceneFade.frag`, c2 0x130) lies past the driver's 32 combined slots, inside the image table at 0x120; the game has no image bindings, and a bound texture there wins over an unbound image (`nvn_mtl.m`).
|
||||
- **Online:** its `EOS_*` imports (Epic Online Services, `nro/EOSSDK-Switch-Shipping.nro`) stay offline, as in Hades: `ro::RegisterModuleInfo` fails (red line), so the game never loads the NRO. SDK 19's `nn::ro::LoadModule` takes one more bool (`…mib`); the host hooks that form and `ro::Finalize` too, so neither reaches the SDK's `ldr:ro` client.
|
||||
- **Memory:** no system resource; the game asks for a 3.10 GB heap, which the 4 GB layout's heap region holds. main has an 82 MB bss, so the profile's `mem_used` is 108 MB. Resident memory levels off at about 3.4 GB in Erebus on the Mac.
|
||||
- **Not a bug: two long lines through Melinoë.** While the staff's Omega attack charges, two thin white lines run from her to the screen edges, one forward and one back. They are that weapon's aim lines (`WeaponStaffSwing5`: `AimLineCountOverride = 2`, `AimLineAngleOffsetOverride = 180`, animation `AuraAimLineNoMovementStaff`). The sprite is a long arrow with white edges, a dark fill drawn emissive and its tip off-screen, and the game hides it while she moves. Traced with `DM_NVN_CBDUMP` vertex dumps and `DM_PROBE` on the aim-line update (`main+0x39ade0`); a scripted run that taps Y shows them often.
|
||||
@@ -227,7 +227,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
**Fire Emblem: Three Houses (`010055D009F78000`)**
|
||||
- **Engine:** Koei Tecmo's older KTGL (the game is co-developed with Intelligent Systems), SDK 6.5.0 (the oldest so far), update 1.2.0. Modules main + subsdk0 (GLSLC, NVN's runtime shader compiler) + subsdk1 (movie) + sdk. RomFS: the LINKDATA archive (`DATA0.bin` is the index, `DATA1.bin` holds 6.9 GB), loose overrides in `patch1`–`patch4`, WebM movies and KTSL sound banks; compressed entries and `*.gz` files are Koei Tecmo's chunked zlib ([01](agents/01-extract.md)).
|
||||
- **Shaders:** 1,364. `shader_split.py` reads the SPKG packages (G2S containers, as in Age of Calamity) in `DATA1.bin` and `patch*/nx/shader`, a `_SPK4100` post-effect package, and the BNSH and GLSLC outputs in `main`. SDK 6's GLSLC writes 0x848 / 0x850-byte control sections and compute code with no header, which the reader used to drop ([04](agents/04-graphics.md)).
|
||||
- **Shaders compiled at run time:** KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (2D primitives, text, the loading quill). No game file holds their output, so they are captured on the Mac. The game runs with `DM_GLSLC_CAPTURE=<version dir>/glslc_capture` (`hle_glslc.c`), and the profile key `shader_glslc_capture on` makes `shader_split.py` read that folder. The outputs are identical on every run. The NSP script does this by itself in its `GLSLC capture` stage, so the NSPs and prod.keys are enough. The stage builds the Mac host into `<id>_build/tools/` and boots the game until the 9 files are in `<id>_build/extract/v327680/glslc_capture`, which takes about 10 s in all. Without the folder the split tables would differ from the profile, and the script would stop at its table check. By hand, run the game for a few seconds with that variable (step 5 of the build by hand).
|
||||
- **Shaders compiled at run time:** KTGL compiles 9 built-in GLSL shaders at boot with the game's own GLSLC (2D primitives, text, the loading quill). No game file holds their output, so they are captured on the Mac. The game runs with `DM_GLSLC_CAPTURE=<version dir>/glslc_capture` (`hle_glslc.c`), and `shader_split.py` reads that folder. The outputs are identical on every run. The NSP script does this by itself in its `GLSLC capture` stage, so the NSPs and prod.keys are enough. The stage builds the Mac host into `<id>_build/tools/` and boots the game until the 9 files are in `<id>_build/extract/v327680/glslc_capture`, which takes about 10 s in all. Without the folder the split tables would differ from the profile, and the script would stop at its table check. By hand, run the game for a few seconds with that variable (step 5 of the build by hand).
|
||||
- **Host changes it needed** ([03](agents/03-bringup.md), [04](agents/04-graphics.md)):
|
||||
- it submits on two NVN queues from two threads (submissions are serialized now);
|
||||
- its command buffer records into one 16 MB ring chunk and frees the ring by `GetCommandMemoryUsed`, which now counts from the last `AddCommandMemory` as NVN does (frames stopped after about 15 s);
|
||||
@@ -247,7 +247,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- **Engine:** the Wii game (codename Comet) on NERD's Hagi layer, SDK 18.3.5, update 1.3.1 (v327680). The PowerPC code is statically recompiled into `main` (`hagi.config`: `Cpu` `Jit false`, `Statifier true`), so there is no JIT; Hagi stands in for the Wii hardware: GX on Vulkan, the DSP with its ROMs in `data/dsp`, the disc from the RomFS, HD texture packs in `data/texture_replace` and `data/ondemand_texture_replace`. Modules main + subsdk0 (movie) + sdk; subsdk1 (GLSLC), subsdk2 (`nn::pcm`) and subsdk3 / subsdk4 (NVIDIA's OpenGL and Vulkan drivers) are not loaded.
|
||||
- **Memory:** `main`'s bss is 2.9 GB (the emulated memory); the game never calls `SetHeapSize`, so the guest block is 1 GB. Footprint on the Mac 1.9 GB, 1.1 GB of it Metal.
|
||||
- **Vulkan:** 99 `vk*` imports through Hades' layer (`hle_vk.m`), which Galaxy extended: push constants, descriptor arrays, dual-source blending from an output at location 1, copies inside a render pass, linear images over their memory, timestamp and occlusion queries, pipeline caches, `vkCmdDraw`, image copies and read-backs, the GX vertex formats (USCALED / SSCALED, three components), viewport and scissor kept across render passes, the swapchain's INHERIT transform and the vi layer crop ([04](agents/04-graphics.md)).
|
||||
- **Shaders:** Hagi generates its GX shaders as SPIR-V at run time. While it shows "Initializing system..." it creates 3,089 modules (the 56 built into `main` among them) and 4,535 pipelines from `data/database/pip/pipelineDB.pip`, the same on every boot (about 15 s on the Mac). Profile key `shader_vk_capture on` and the key list `vk_capture.tsv`: the NSP script's `Vulkan shader capture` stage boots the game in the Mac host with `DM_VK_CAPTURE=<version dir>/vk_capture` until every listed key is there (under a minute), then `vk_shaders.py` translates the RomFS and the capture. Its two geometry shaders (GX points and lines, expanded to quads) are skipped: those draws come out as one-pixel points and lines.
|
||||
- **Shaders:** Hagi generates its GX shaders as SPIR-V at run time. While it shows "Initializing system..." it creates 3,089 modules (the 56 built into `main` among them) and 4,535 pipelines from `data/database/pip/pipelineDB.pip`, the same on every boot (about 15 s on the Mac). The RomFS has no SPIR-V file, so the NSP script's `Vulkan shader capture` stage, with the profile's key list `vk_capture.tsv`, boots the game in the Mac host with `DM_VK_CAPTURE=<version dir>/vk_capture` until every listed key is there (under a minute), then `vk_shaders.py` translates the RomFS and the capture. Its two geometry shaders (GX points and lines, expanded to quads) are skipped: those draws come out as one-pixel points and lines.
|
||||
- **Pointer:** docked, Galaxy's pointer is the Joy-Con gyro, which the host does not have; handheld, it is the touch screen. The profile's `op_mode handheld` starts it in handheld mode (720p). On the iPhone, touches no gamepad control takes go to the game's touch screen (all of them while a controller is connected); on the Mac the mouse is a finger, and `DM_KEYS` taps with `T<x>x<y>` (normalized picture coordinates).
|
||||
- **CPU:** Hagi polls with `SleepThread(100 ns)`, which the host used to return from at once: one thread kept a core at 100%. Sleeps under 20 µs now count as yields and get the yield back-off ([03](agents/03-bringup.md)).
|
||||
- **Not yet:** GX points and lines (the geometry shaders), sampler LOD bias (GX uses it; Metal samplers have none, red line). The base NSP in hand was damaged (147 MB zeroed near its end: the two WebM prologue movies, which the storybook prologue does not use); it was packed with `--allow-damaged`.
|
||||
@@ -260,7 +260,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
|
||||
**Monster Hunter Rise (`0100B04011742000`)**
|
||||
- **Engine:** Capcom's RE Engine, SDK 11.4.3, update 16.0.2 (v2228224, Sunbreak). The base game is an XCI (cartridge dump; nx_extract reads its secure partition), the update an NSP, and its 252 DLC NSPs are licences only. `main` has 125 MB of text: the rewriter puts its patch area right before the base (`BIG_TEXT`, a second pass with `DM_PATCH_FRONT`).
|
||||
- **Files and shaders:** everything sits in KPKA PAK archives, later patch PAKs overriding earlier ones by path hash. The 63,111 shaders are in SDF files (`shader_split.py` `kpka_jobs`), with combined and separate texture tables at once (profile key `nvn_septex_base`).
|
||||
- **Files and shaders:** everything sits in KPKA PAK archives, later patch PAKs overriding earlier ones by path hash. The 63,111 shaders are in SDF files (`shader_split.py` `kpka_jobs`), with combined and separate texture tables at once (separate binds are texture slots 128 + n).
|
||||
- **GPU heap:** one 64 GB sparse virtual pool built from 4 MB physical pools. The host aliases mapped pages (`vm_remap`) and keeps a 4 KB page table the replay reads textures through (streamed textures 32 GB in, mip tails at 4 KB offsets). The scene is GPU-driven (`MultiDrawElementsIndirectCount`).
|
||||
- **Keyboard:** the hunter, Palamute and Palico are named through the inline keyboard (`DM_KBD` on the Mac, the input box on the iPhone); the name is masked through `ngc::ProfanityFilter`'s vtable, which the host fills.
|
||||
- **Frame rate:** 60 fps (30 on the Switch). The frame loop (main+0x6b106c8) sleeps until `[obj+0x430] / [obj+0x434]` sixtieths of a second have passed and steps the game by the time that really passed; the game sets the limit at `[main+0x1295b3f0]+0x434` to 30 after boot and asks for present interval 2. The host writes the profile's `fps_target` there (`hle_fpslock.c`, FPSLocker's address for this build) and the profile's `present_interval display` ignores the interval; both are needed. `DM_FPS_TARGET=30` gives back 30, and the Mac `DM_KEYS` routes below count frames at 30, so run them with it.
|
||||
@@ -279,7 +279,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
**Unicorn Overlord (`010069401ADB8000`)**
|
||||
- **Status:** brought up on the Mac (logos, title, a new game and the opening story scene; no red line) and playable on an iPhone.
|
||||
- **Engine:** Vanillaware's engine, as in 13 Sentinels, SDK 16.2.4, update 1.05 (v262144). Its ExeFS holds only main and sdk. All data is in 7 CRI CPK archives; audio is CRI ADX2 with Opus streams. Both DLC NSPs are licences only.
|
||||
- **Shaders:** 2,188 GLSLC outputs compiled into main (1,725 distinct: 614 vertex, 1,111 pixel), single-stage programs combined per draw (`shader_main_glslc`, `shader_separable`). Binding layout as Mario's; every binding decodes.
|
||||
- **Shaders:** 2,188 GLSLC outputs compiled into main (1,725 distinct: 614 vertex, 1,111 pixel), single-stage programs combined per draw. Every binding decodes.
|
||||
- **Service calls:** three host changes, each found by one boot ([03](agents/03-bringup.md)):
|
||||
- the hardware multi-stream Opus decoder's C interface, forwarded to the SDK's software decoder;
|
||||
- `nn::crypto::GenerateCryptographicallyRandomBytes`, which the sdk's own `getentropy` calls under `std::random_device`;
|
||||
@@ -290,7 +290,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
|
||||
**Octopath Traveler 0 (`01005270232F2000`)**
|
||||
- **Engine:** Unreal Engine 5 (the paged global distance field shaders and the LowLevelTasks task system) on its mobile renderer, SDK 19.3.13, update 1.0.5 (v327680); the base game is an XCI. Everything is in one pak, `pakchunk0-Switch.pak` (AES-encrypted index; the key is found in `main` and never printed), with CRIWARE sound banks and a USM movie beside it.
|
||||
- **Shaders:** 10,324, in the pak's two shader code archives, each one Oodle-compressed. The NSP script decodes them on the Mac with the game's own `OodleLZ_Decompress` (`shader_ue_decode`, `dm/gfx/tools/uepak.py`, `DM_OODLE_JOB`), about 10 s for all of them. 61 vertex shaders (particles) pick a texture slot at run time; the profile key `tex_dyn_slots` binds them ([04](agents/04-graphics.md)). 2,084 shaders do atomics on buffer images, now bound to the image's memory. 16 compute shaders with 3D image atomics (Unreal's paged global distance field, which the mobile renderer does not dispatch) do not build.
|
||||
- **Shaders:** 10,324, in the pak's two shader code archives, each one Oodle-compressed. The NSP script decodes them on the Mac with the game's own `OodleLZ_Decompress` (as for any game with Unreal paks: `dm/gfx/tools/uepak.py`, `DM_OODLE_JOB`), about 10 s for all of them. 61 vertex shaders (particles) pick a texture slot at run time; the profile key `tex_dyn_slots` binds them ([04](agents/04-graphics.md)). 2,084 shaders do atomics on buffer images, now bound to the image's memory. 16 compute shaders with 3D image atomics (Unreal's paged global distance field, which the mobile renderer does not dispatch) do not build.
|
||||
- **Memory:** Unreal reserves one address region (62 GB on the Switch) and its allocator, FMallocBinned3, 37 GB of it. A cut skips the allocator's constructor and the host runs Unreal's heap on a malloc zone (`hle_uemalloc.c`); the region is capped at 2 GB (`game_args`) and the SDK's node area cut to 62 MB, so the guest block is 3.75 GB. 15 minutes into the story on the Mac: region mapped up to 1.26 GB of the alias region, Unreal heap 0.75 GB, 2.8 GB resident, 1.84 GB of GPU allocations.
|
||||
- **Frame rate:** 60 fps (30 on the Switch). The game's config caps Unreal at 30: `t.MaxFPS=30` in `[ConsoleVariables]` of `DefaultEngine.ini` (and `FrameRateLimit=30` in `SwitchGameUserSettings.ini`, which sets it again), and `rhi.syncinterval=2` in `SwitchDeviceProfiles.ini`, the present interval 2 the game asks for. The shipping build reads no console variables from the command line (no `-ExecCmds`, `-DPCVars` or `-ini:`), so the host writes 60 and 1 into the two console variables (`hle_fpslock.c`, profile key `fps_target 60`), whenever the game sets them back; the game then asks for present interval 1 itself. Unreal steps the game by the time that really passed: on the Mac, `DM_NVN_REGLUM` traces of the same input script from a fresh save at 30 and 60 fps (frame time p50 33.4 → 16.7 ms) put the 1.5 s fade to white 70 s in at the same real time with the same shape, and both runs reach the same screens at the same times. `DM_FPS_TARGET=30` keeps the game's own 30.
|
||||
- **Keyboard:** the character's name and favourite dish come from the system keyboard (`DM_KBD` on the Mac, the input box on the iPhone). The game keeps running behind it, so the guest clock does too (it used to freeze: the sound stuttered and the box lagged on the iPhone).
|
||||
@@ -301,7 +301,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
**Pokémon Legends: Z-A (`0100F43008C44000`)**
|
||||
- **Status:** brought up on the Mac: language select, character creation, the opening and the first minutes of play in Lumiose City at a steady 30 fps (a 5-minute walk: GPU about 6 ms a frame, no crash; the red lines are the render condition the host does not evaluate, the placeholder cube and an occasional memory-pool retry). On an iPhone the first build (before the base-instance fix in [04](agents/04-graphics.md)) ran at 30 fps with a memory footprint of about 2 GB.
|
||||
- **Engine:** Game Freak's Trinity engine on SDK 19.3.11 (NVN 55) with Havok 2022.2, update 2.0.2 (v393216). The SDK 19 calls the host had to answer are in [03](agents/03-bringup.md).
|
||||
- **Files and shaders:** every file sits in the Trinity packs of `arc/data.trpfs` (FlatBuffers, Oodle Mermaid). The shader split decodes them with the game's own `OodleLZ_Decompress`, run in the Mac host's tool mode (`hle_oodle.c`, profile key `shader_oodle`), and finds 28,603 shaders ([01](agents/01-extract.md)).
|
||||
- **Files and shaders:** every file sits in the Trinity packs of `arc/data.trpfs` (FlatBuffers, Oodle Mermaid). The shader split decodes them with the game's own `OodleLZ_Decompress`, run in the Mac host's tool mode (`hle_oodle.c`; the NSP script sets it up for any game with Trinity packs), and finds 28,603 shaders ([01](agents/01-extract.md)).
|
||||
- **Memory:** the 5 GB guest block with two cuts for 2.0.2: the StandardAllocator's address space 12 GB → 4 GB and the SDK's node area 1 GB → 62 MB. With `mem_map_limit` the host refuses a map past the application pool, as Horizon does, so the SDK gives back the 3.15 GB it keeps mapped after `nninitStartup`.
|
||||
- **Graphics:** characters are skinned with transform feedback, the colour grading LUT is drawn as a layered pass, the deferred lighting reads stencil through a stencil-mode texture view, and the occlusion queries that decide how characters are animated need the draw's base instance in the driver segment ([04](agents/04-graphics.md)).
|
||||
- **Mac input:** a fresh save reaches play in Lumiose City with `DM_KBD=Taylor DM_KEYS="f450:A,f1200:LRight~1,f1600:A,f1900:Down,f2000:Down,f2100:A,f2400:A*80/90"`; a save made there continues with `DM_KEYS="f300:A*3/150"` (play from about frame 600), and `LUp~12` then walks along the platform.
|
||||
|
||||
@@ -1,414 +0,0 @@
|
||||
// nn::hid (controller / touchscreen): on the real device shared memory is read through the hid service; the host takes it from the current dm_input state, written by the front end (Mac keyboard / iOS touchscreen).
|
||||
// The connected controller follows the operation mode: DM_OPMODE=1 (console, default) → 1P full-key controller; 0 (handheld) → handheld controller. Vibration is not supported (device count 0).
|
||||
#include "hle.h"
|
||||
#include "dm_uiscreen.h"
|
||||
#include "dm_actions.h"
|
||||
#include "dm_cursor_item.h"
|
||||
#include <unistd.h>
|
||||
#include "dm_input.h"
|
||||
#include "front.h"
|
||||
#include <stdatomic.h>
|
||||
#include "dm_cursor.h"
|
||||
#include <math.h>
|
||||
_Atomic int g_dm_l_presses;
|
||||
|
||||
enum { NPAD_HANDHELD = 0x20 };
|
||||
enum { STYLE_FULLKEY = 1u << 0, STYLE_HANDHELD = 1u << 1 };
|
||||
enum { ATTR_CONNECTED = 1u << 0 };
|
||||
|
||||
// nn::hid::NpadFullKeyState / NpadHandheldState have the same layout
|
||||
typedef struct {
|
||||
int64_t sampling;
|
||||
uint64_t buttons;
|
||||
int32_t lx, ly, rx, ry;
|
||||
uint32_t attributes;
|
||||
uint32_t _pad;
|
||||
} NpadState;
|
||||
_Static_assert(sizeof(NpadState) == 0x28, "NpadState");
|
||||
|
||||
static struct { _Atomic uint64_t buttons; _Atomic int32_t lx, ly, rx, ry; } g_in;
|
||||
static _Atomic int64_t g_sampling;
|
||||
static _Atomic uint64_t g_sampled_right;
|
||||
void dm_input_sampled_right(int32_t *rx,int32_t *ry) {
|
||||
uint64_t v=atomic_load_explicit(&g_sampled_right,memory_order_acquire);
|
||||
*rx=(int32_t)(uint32_t)v;*ry=(int32_t)(uint32_t)(v>>32);
|
||||
}
|
||||
|
||||
// Test switch DM_TEST_HID_STALL_MS=<milliseconds>: each time a press edge of the A published by the front end appears, the game's next controller read first sleeps this long before reading -- simulates "the game stalls at the moment of the tap".
|
||||
// After opening the function page the game did not read input for about 0.4 s, and a tap pulse of 0.12 s wall clock was lost entirely (tapped, no response); the front end now ends the pulse by dm_input_samples
|
||||
static int g_stall_ms = -1; static _Atomic int g_stall_due;
|
||||
// Press-edge latch: when a key published by the front end goes from "not pressed" to "pressed" it is recorded in g_press_latch, delivered along with the game's next controller read and cleared once delivered.
|
||||
// When press and release fell between two controller reads of the game (two steps sent back to back when the script lags, or the game stalling input reads while loading / opening a panel) the whole key press used to be lost -- real device rf1, restarts 2 and 3:
|
||||
// the bottom bar menu key had only 0.09 ms from press to release, the game did not see it and the function page did not open. The latch only gives one extra sample, afterwards the front end's current value applies (released is released)
|
||||
static _Atomic uint64_t g_press_latch;
|
||||
// Controller (dm_input.h dm_pad_set): buttons and the four axes are packed into two atomics, and press edges are likewise latched until the game's next
|
||||
// read (short controller presses are not lost)
|
||||
static _Atomic uint64_t g_pad_btn, g_pad_axes, g_pad_latch;
|
||||
void dm_pad_set(uint64_t buttons, int32_t lx, int32_t ly, int32_t rx, int32_t ry) {
|
||||
uint64_t old = atomic_exchange(&g_pad_btn, buttons); atomic_fetch_or(&g_pad_latch, buttons & ~old);
|
||||
atomic_store(&g_pad_axes, (uint64_t)(uint16_t)lx | (uint64_t)(uint16_t)ly << 16 | (uint64_t)(uint16_t)rx << 32 | (uint64_t)(uint16_t)ry << 48);
|
||||
}
|
||||
void dm_input_set(uint64_t buttons, int32_t lx, int32_t ly, int32_t rx, int32_t ry) {
|
||||
if (g_stall_ms < 0) { const char *e = getenv("DM_TEST_HID_STALL_MS"); g_stall_ms = e ? atoi(e) : 0; if (g_stall_ms) LOG("DM_TEST_HID_STALL_MS: after each A press edge the game's controller read first sleeps %d ms", g_stall_ms); }
|
||||
if (g_stall_ms > 0 && (buttons & DM_BTN_A) && !(g_in.buttons & DM_BTN_A)) atomic_store(&g_stall_due, 1);
|
||||
atomic_fetch_or(&g_press_latch, buttons & ~g_in.buttons); // record newly pressed keys, guaranteeing the game reads them at least once (see fill)
|
||||
g_in.buttons = buttons; g_in.lx = lx; g_in.ly = ly; g_in.rx = rx; g_in.ry = ry;
|
||||
}
|
||||
// Test switch DM_UI_TAPS="frame:x,y[:action];..." (x / y = per-mille of the picture). Default action = one tap (via dm_ui_touch: press, lift 6 frames later); the others go through dm_ui_ptr (for probing / UI cases):
|
||||
// k tap a tab / dialog menu item (the host presses keys to switch to it); t<frames> finger held this many frames then lifted (via dm_ui_touch); h<frames> hover; p<frames> left button held this many frames then released; r right button; l left button (pointer path); d<x2>,<y2> hold and drag to another point (done in 20 frames); g<x2>,<y2> continuous gesture (dm_ui_drag_*: typed as scroll / slider by the start-to-end direction, done in 60 frames); W tap the game world (dm_world_tap); N<value> set the quantity box value (dm_num_set); A<number> buy/sell / stat point entries (dm_actions.c); w<amount> wheel (120 per notch, negative = down); m<0|1> map back to minimap / to full screen (dm_ui_map_full); prefix s / c = holding Shift / Ctrl (VK 0x10 / 0x11)
|
||||
static void ui_taps_step(void) {
|
||||
static struct { uint64_t fr; float x, y, x2, y2; int st, kind, arg, shift; } tp[128]; static int n = -1;
|
||||
if (n < 0) { n = 0; const char *e = getenv("DM_UI_TAPS"); char buf[4096]; if (e) { strlcpy(buf, e, sizeof buf); for (char *t = strtok(buf, ";"); t && n < 128; t = strtok(NULL, ";")) {
|
||||
unsigned long long f; int x, y, c = 0; if (sscanf(t, "%llu:%d,%d%n", &f, &x, &y, &c) < 3) continue;
|
||||
tp[n].fr = f; tp[n].x = x / 1000.0f; tp[n].y = y / 1000.0f; tp[n].kind = 0; const char *a = t + c;
|
||||
if (*a == ':') { a++; if (*a == 's' || *a == 'c') { tp[n].shift = *a == 's' ? 0x10 : 0x11; a++; } tp[n].kind = *a; int p = 0, q = 0;
|
||||
if ((*a == 'd' || *a == 'g') && sscanf(a + 1, "%d,%d", &p, &q) == 2) { tp[n].x2 = p / 1000.0f; tp[n].y2 = q / 1000.0f; } else tp[n].arg = atoi(a + 1); }
|
||||
n++; } LOG("DM_UI_TAPS: %d entries", n); } }
|
||||
uint64_t fr = dm_guest_frame();
|
||||
static int rel = -1; if (rel < 0) rel = getenv("DM_UI_TAPS_AT") != NULL;
|
||||
if (rel) { uint64_t b = dm_ui_taps_base(); if (!b || fr < b) return; fr -= b; }
|
||||
for (int i = 0; i < n; i++) { if (fr < tp[i].fr || tp[i].st < 0) continue; int d = (int)(fr - tp[i].fr), k = tp[i].kind; float x = tp[i].x, y = k == 'W' ? tp[i].y : dm_ui_ny(tp[i].y); // script coordinates are per-mille of the real picture; world taps use them as is, UI taps are converted into the UI rectangle
|
||||
if (!k || k == 't') { int hold = k ? tp[i].arg : 6; if (tp[i].st == 0) { tp[i].st = 1; dm_ui_touch(0, x, y); } else if (tp[i].st == 1 && d >= hold) { tp[i].st = -1; dm_ui_touch(2, x, y); } continue; }
|
||||
if (k == 'N') { void dm_num_set(int); dm_num_set(tp[i].arg); tp[i].st = -1; continue; } // set the quantity box value (dm_textinput.c; -1 all, -2 half)
|
||||
if (k == 'A') { void dm_shop_buy(void), dm_shop_sell_cursor(void), dm_stat_add(int, int), dm_skill_add_selected(void); int v = tp[i].arg; // action entries (dm_actions.c): A1 buy, A2 sell what is on the cursor, A3 skill +1, A4 pick up the first sellable item in the inventory (for testing), A5 put the first potion in the inventory into the belt (for testing), A6 directly sell the first sellable item carried (for testing), A21-A24 immediately use the first potion / scroll / tome / cube in the inventory (for testing, dm_item_use), A1000+id put that item into the belt, A1xy add y (1 / 5 / 9 = all) to stat x (0–3)
|
||||
if (v == 1) dm_shop_buy(); else if (v == 2) dm_shop_sell_cursor(); else if (v == 3) dm_skill_add_selected(); else if (v == 4) { void dm_test_hold_first_sellable(void); dm_test_hold_first_sellable(); } else if (v == 6) { void dm_test_sell_first(void); dm_test_sell_first(); } else if (v == 5) { void dm_test_belt_first_potion(void); dm_test_belt_first_potion(); } else if (v == 35 || v == 38 || v == 39) { void dm_inv_sort(int); dm_inv_sort(v == 35 ? 0 : v == 38 ? 3 : 4); } else if (v == 36) { void dm_name_random(void); dm_name_random(); } else if (v >= 44 && v <= 46) { void dm_test_identify(int); dm_test_identify(v - 43); } else if (v == 74) { void dm_held_stow(void); dm_held_stow(); } else if (v == 75) { void dm_test_swap_held(void); dm_test_swap_held(); } else if (v >= 70 && v <= 73) { dm_uiscreen_enable(v - 70); } /* A70-A73: UI picture off / top-aligned / centered / bottom-aligned (through the same entry the front end uses) */ else if (v == 47 || v == 48) { dm_stash_take(v == 47 ? 1 : 3); } else if (v == 43) { dm_cursor_item_info ci; int hw = 0, hh = 0, c = dm_cursor_item(&ci), hd = dm_held_size(&hw, &hh); LOG("test: cursor item %s container %d cell %d,%d size %dx%d stack %d; held %d (%dx%d); awaiting target %d", c ? ci.code : "none", c ? ci.container : 0, c ? ci.gx : 0, c ? ci.gy : 0, c ? ci.gw : 0, c ? ci.gh : 0, c ? ci.stack : 0, hd, hw, hh, dm_item_targeting()); } else if (v >= 50 && v <= 69) { void dm_test_stash_take(int); dm_test_stash_take(v - 50 ? v - 50 : 1); } else if (v >= 40 && v <= 42) { void dm_test_equip(int); dm_test_equip(v - 39); } else if (v == 33 || v == 34) { void dm_belt_expand(int); dm_belt_expand(v == 33); } else if (v == 31 || v == 32) { void dm_item_compare(int); dm_item_compare(v == 31); } else if (v == 30) { void dm_test_hover_at(float, float); dm_test_hover_at(x, y); } else if (v >= 21 && v <= 26) { void dm_test_use_first(int); dm_test_use_first(v - 20); } else if (v >= 1000) { void dm_belt_put(uint32_t); dm_belt_put((uint32_t)(v - 1000)); } else if (v >= 100) dm_stat_add(v / 10 % 10, v % 10 == 9 ? -1 : v % 10); tp[i].st = -1; continue; }
|
||||
if (k == 'W') { unsigned dm_world_tap(float, float); dm_world_tap(x, y); tp[i].st = -1; continue; } // tap the game world (dm_world.c: tapping a target = interact)
|
||||
if (k == 'g') { float t = d / 60.0f; if (t > 1) t = 1;
|
||||
if (tp[i].st == 0) { tp[i].st = 1; int hz = fabsf(tp[i].x2 - x) * 16 > fabsf(tp[i].y2 - y) * 9; dm_ui_drag_begin(hz ? 2 : 1, x, y, tp[i].arg ? tp[i].arg / 1000.0f : 0.05f); }
|
||||
dm_ui_drag_move(x + (tp[i].x2 - x) * t, y + (tp[i].y2 - y) * t); if (d >= 62) { dm_ui_drag_end(); tp[i].st = -1; } continue; }
|
||||
if (k == 'k') { static unsigned ks[128]; if (tp[i].st == 0) { tp[i].st = 1; ks[i] = dm_ui_item_query(x, y); } else if (d >= 3) { tp[i].st = -1; if (dm_ui_item_result(ks[i]) == 2) dm_ui_tab_go(); } continue; } // k = tap once the "tab / dialog menu" way (query the hit → if yes, let the host press the key)
|
||||
if (tp[i].st == 0) { tp[i].st = 1; dm_ui_item_query(x, y); dm_ui_ptr(DM_UP_HOVER, x, y, 0); if (tp[i].shift) dm_ui_ptr(DM_UP_KEYDOWN, x, y, tp[i].shift); continue; }
|
||||
int done = 0;
|
||||
if (k == 'h') done = d >= tp[i].arg;
|
||||
else if (k == 'r' || k == 'l') { if (tp[i].st == 1 && d >= 4) { tp[i].st = 2; dm_ui_ptr(k == 'r' ? DM_UP_RDOWN : DM_UP_LDOWN, x, y, 0); dm_ui_ptr(k == 'r' ? DM_UP_RUP : DM_UP_LUP, x, y, 0); } done = d >= 10; }
|
||||
else if (k == 'p') { if (tp[i].st == 1 && d >= 4) { tp[i].st = 2; dm_ui_ptr(DM_UP_LDOWN, x, y, 0); } else if (tp[i].st == 2 && d >= 4 + tp[i].arg) { tp[i].st = 3; dm_ui_ptr(DM_UP_LUP, x, y, 0); } done = d >= 10 + tp[i].arg; }
|
||||
else if (k == 'm') { if (tp[i].st == 1) { tp[i].st = 2; dm_ui_map_full(tp[i].arg); } done = d >= 4; }
|
||||
else if (k == 'w') { if (tp[i].st == 1 && d >= 4) { tp[i].st = 2; dm_ui_ptr(DM_UP_SCROLL, x, y, tp[i].arg); } done = d >= 8; }
|
||||
else if (k == 'd') { if (tp[i].st == 1 && d >= 4) { tp[i].st = 2; dm_ui_ptr(DM_UP_LDOWN, x, y, 0); }
|
||||
else if (tp[i].st == 2 && d >= 6) { float t = (d - 6) / 20.0f; if (t > 1) t = 1; dm_ui_ptr(DM_UP_MOVE, x + (tp[i].x2 - x) * t, y + (tp[i].y2 - y) * t, 0); if (d >= 30) { tp[i].st = 3; dm_ui_ptr(DM_UP_LUP, 0, 0, 0); } }
|
||||
done = d >= 36; }
|
||||
else done = 1;
|
||||
if (done) { tp[i].st = -1; if (tp[i].shift) dm_ui_ptr(DM_UP_KEYUP, x, y, tp[i].shift); dm_ui_ptr(DM_UP_END, x, y, 0); } }
|
||||
}
|
||||
static _Atomic uint64_t g_stick_ns;
|
||||
uint64_t dm_stick_last_ns(void) { return atomic_load(&g_stick_ns); } // criterion for "click-to-move packets may only come from the stick" (diag.c c2s_deliver)
|
||||
uint64_t dm_input_samples(void) { return (uint64_t)atomic_load(&g_sampling); } // cumulative number of times the game has read the controller (the front end uses it to judge whether a key press has been seen by the game)
|
||||
|
||||
static uint32_t connected_id(void) { return dm_opmode() == 0 ? NPAD_HANDHELD : 0; } // follows the operation mode (hle_oe.c is the single source)
|
||||
static uint32_t style_of(uint32_t id) {
|
||||
if (id != connected_id()) return 0;
|
||||
return id == NPAD_HANDHELD ? STYLE_HANDHELD : STYLE_FULLKEY;
|
||||
}
|
||||
// Scripted input (can inject without opening a window), two kinds of comma-separated items:
|
||||
// By time: key@seconds[+hold seconds] DM_KEYS="A@8,A@12+0.5" = press A at second 8 (held 0.2 s by default), press A for 0.5 s at second 12; time counts from host initialization
|
||||
// By UI message: key@pre[%delay seconds][+hold seconds][>confirm] DM_KEYS="A@OpenPanel:MainMenuButtonRibbon>OpenPanel:CharacterDifficultyModal,A@OpenPanel:CharacterDifficultyModal>FrontEndNavigation:ToGame"
|
||||
// Same rules as dm/verify/ryu/drive.py: message string = "category:action:panel name", pre / confirm match by substring; steps run in order.
|
||||
// The pre has appeared after the previous step was pressed (first step: since start) and the previous key press is idle (see below) → press for 3 frames (50 ms;
|
||||
// "+hold seconds" changes the hold duration, e.g. LSL@UnloadPanel:LoadScreenPanel+0.9 = push the left stick for 0.9 s as soon as the loading screen ends, so a key pressed by host time does not land before the run is entered);
|
||||
// only re-press if no confirm arrives within 150 server ticks, after 3 times without one report a failure and abort; the 60 s wall clock is only a fallback and reports a failure; a step with no confirm item is done once pressed.
|
||||
// An empty pre (e.g. LSU@%0.2+0.35) = press after a delay right after the previous step completes (confirm arrived / released if no confirm item).
|
||||
// Idle = the previous key press has been released, and the last InputMessage:ControllerActionBegin during the hold has got its ControllerActionEnd (Begin is only sent when a panel consumes
|
||||
// input: in the wild1 ground truth, pressing A in a run to open the waypoint has no Begin / End before or after, they only appear in pairs once a panel takes over; no Begin during the hold ⇒ idle on release.
|
||||
// Stick pseudo-keys follow the same rule). The former "must see End" never arrived after in-run key presses.
|
||||
// Key name = DM_BTN_* from dm_input.h without the prefix, plus the left stick pseudo-keys LSL / LSR / LSU / LSD (pushed to full while held). ORed bitwise with the keys written by the front end.
|
||||
// Touch item T<x>:<y> (only for steps driven by UI messages): while held, one finger (id 0, via dm_touch_set) is put down at normalized rendered-picture coordinates (x, y), lifted on release / confirm.
|
||||
#define K(n) {DM_BTN_##n, #n} // key names are generated directly from DM_BTN_* in dm_input.h, the bit order is defined in one place only
|
||||
static const struct { uint32_t mask; const char *name; } k_names[] = {
|
||||
K(A), K(B), K(X), K(Y), K(LS), K(RS), K(L), K(R), K(ZL), K(ZR), K(PLUS), K(MINUS), K(LEFT), K(UP), K(RIGHT), K(DOWN)};
|
||||
#undef K
|
||||
// Left stick pseudo-keys (used only in scripts): while held the stick is pushed fully in that direction; the bits lie outside DM_BTN_* and do not enter the button bits
|
||||
enum { PK_LSL = 1u << 24, PK_LSR = 1u << 25, PK_LSU = 1u << 26, PK_LSD = 1u << 27, PK_MASK = 0xFu << 24, PK_TOUCH = 1u << 28, PK_MSG = 1u << 29, PK_CLICK = 1u << 30 }; // PK_CLICK: UI click step (C<x>:<y>, via dm_ui_touch = the same injected mouse message path as a real finger on the iOS front end) // PK_TOUCH: touch point step (T<x>:<y>, via dm_touch_set)
|
||||
static const struct { uint32_t mask; const char *name; } k_stick[] = {{PK_LSL, "LSL"}, {PK_LSR, "LSR"}, {PK_LSU, "LSU"}, {PK_LSD, "LSD"}};
|
||||
#define KMAX 256
|
||||
static struct { uint32_t mask; uint64_t t0, t1; } g_keys[KMAX];
|
||||
static int g_nkeys;
|
||||
static uint64_t g_kbase;
|
||||
// Steps driven by UI messages (state is driven from two sides, the UI message callback and the controller read, protected by g_klock)
|
||||
enum { KS_PRE, KS_HOLD, KS_OK, KS_DONE };
|
||||
static struct { uint32_t mask; char pre[96], ok[96]; int seen_pre, seen_server, hold_override; double delay_seconds, hold_seconds; uint64_t seen_clock, seen_at; float tx, ty; char msg[96]; } g_steps[KMAX]; // in a run, by server ticks; before entering the game, by presented frames
|
||||
static int g_nsteps, g_kcur, g_kstate, g_ktries, g_krel = 1, g_kbeg, g_kend; // beg/end are 0/1 markers of the action lifecycle, not message counts; idle = released and the last Begin has got its End
|
||||
static uint64_t g_kpress, g_kpress_clock; static int g_kpress_server;
|
||||
static pthread_mutex_t g_klock = PTHREAD_MUTEX_INITIALIZER;
|
||||
static struct { char msg[160]; int32_t x, y; } g_cursor_tests[KMAX];
|
||||
static int g_ncursor_tests, g_cursor_cur;
|
||||
#define SERVER_TICKS_PER_SECOND 25ull
|
||||
#define PRODUCT_FRAMES_PER_SECOND 60ull
|
||||
#define ACTION_HOLD_TICKS 2ull // action keys 2 ticks (about 80 ms, spanning at least one full tick): held too long, minus is taken as a repeated press and the automap switches to the full-screen mode (measured: automap red at 15 ticks)
|
||||
#define DIRECTION_HOLD_TICKS 3ull // direction keys at least 3 ticks: three consecutive samples are enough to catch it reliably
|
||||
#define OK_WAIT_TICKS 150ull
|
||||
#define WALL_FALLBACK_NS 60000000000ull
|
||||
|
||||
typedef struct { uint64_t value; int server; } script_clock;
|
||||
static script_clock script_clock_now(void) {
|
||||
uint64_t tick = dm_server_tick(); { extern _Atomic int g_dm_frozen; if (atomic_load(&g_dm_frozen)) tick = UINT64_MAX; } // while the world is frozen (panel freeze, diag.c) the tick number does not advance, the script times by presented frames instead, otherwise delays and hold durations would never be reached
|
||||
return tick == UINT64_MAX ? (script_clock){dm_guest_frame(), 0} : (script_clock){tick, 1};
|
||||
}
|
||||
static uint64_t script_clock_elapsed(script_clock now, uint64_t start, int start_server) {
|
||||
return now.server == start_server && now.value >= start ? now.value - start : 0;
|
||||
}
|
||||
static uint64_t script_seconds_to_units(double seconds, int server) {
|
||||
return (uint64_t)(seconds * (server ? SERVER_TICKS_PER_SECOND : PRODUCT_FRAMES_PER_SECOND));
|
||||
}
|
||||
static uint64_t step_hold_units(const typeof(g_steps[0]) *st, int server) {
|
||||
if (st->hold_override) return script_seconds_to_units(st->hold_seconds, server);
|
||||
return (st->mask & (DM_BTN_UP | DM_BTN_DOWN | DM_BTN_LEFT | DM_BTN_RIGHT | PK_MASK)) ? DIRECTION_HOLD_TICKS : ACTION_HOLD_TICKS;
|
||||
}
|
||||
|
||||
static void cursor_tests_parse(void) {
|
||||
g_ncursor_tests = g_cursor_cur = 0;
|
||||
const char *e = getenv("DM_TEST_CURSOR"); if (!e || !*e) return;
|
||||
char buf[4096]; strncpy(buf, e, sizeof buf - 1); buf[sizeof buf - 1] = 0;
|
||||
for (char *save, *tok = strtok_r(buf, ";", &save); tok; tok = strtok_r(NULL, ";", &save)) {
|
||||
char *colon = strrchr(tok, ':'); int x, y; char extra;
|
||||
if (!colon || sscanf(colon + 1, "%d,%d%c", &x, &y, &extra) != 2 || g_ncursor_tests == KMAX)
|
||||
die("DM_TEST_CURSOR entry '%s' format: UI-message-trigger:x,y", tok);
|
||||
*colon = 0;
|
||||
if (!*tok) die("DM_TEST_CURSOR entry has an empty trigger string");
|
||||
strncpy(g_cursor_tests[g_ncursor_tests].msg, tok, sizeof g_cursor_tests[0].msg - 1);
|
||||
g_cursor_tests[g_ncursor_tests].msg[sizeof g_cursor_tests[0].msg - 1] = 0;
|
||||
g_cursor_tests[g_ncursor_tests].x = (int32_t)x;
|
||||
g_cursor_tests[g_ncursor_tests].y = (int32_t)y;
|
||||
g_ncursor_tests++;
|
||||
}
|
||||
}
|
||||
|
||||
static void step_chain(void) { // holds g_klock: the current step's pre is left empty ⇒ counts as seen as soon as the previous step completes
|
||||
if (g_kcur < g_nsteps && !g_steps[g_kcur].pre[0] && !g_steps[g_kcur].seen_pre) { script_clock c = script_clock_now(); g_steps[g_kcur].seen_pre = 1; g_steps[g_kcur].seen_at = now_ns(); g_steps[g_kcur].seen_clock = c.value; g_steps[g_kcur].seen_server = c.server; }
|
||||
}
|
||||
static void keys_on_uimsg(const char *cat, const char *act, const char *arg) {
|
||||
char m[256]; snprintf(m, sizeof m, "%s:%s:%s", cat, act, arg);
|
||||
pthread_mutex_lock(&g_klock);
|
||||
if (g_cursor_cur < g_ncursor_tests && strstr(m, g_cursor_tests[g_cursor_cur].msg)) {
|
||||
int i = g_cursor_cur++;
|
||||
dm_cursor_set(g_cursor_tests[i].x, g_cursor_tests[i].y);
|
||||
}
|
||||
// Repeat triggers of a long press/stick send a Begin each time, release sends only one End (observed on the iOS host: 73 Begin/1 End).
|
||||
// A new Begin marks the action as still busy and cancels the old End; repeated Begins do not increase the wait count, only End marks it finished.
|
||||
if (strstr(m, "InputMessage:ControllerActionBegin") && !g_krel) { g_kbeg = 1; g_kend = 0; }
|
||||
if (strstr(m, "InputMessage:ControllerActionEnd")) g_kend = 1;
|
||||
if (g_kcur < g_nsteps) {
|
||||
if (g_kstate != KS_PRE && g_steps[g_kcur].ok[0] && strstr(m, g_steps[g_kcur].ok)) {
|
||||
LOG("hid script step %d confirmed '%s' (press %d, waited %llu clock units, wall clock %llu ms)", g_kcur + 1, m, g_ktries,
|
||||
(unsigned long long)(script_clock_now().value - g_kpress_clock), (unsigned long long)((now_ns() - g_kpress) / 1000000ull));
|
||||
if (g_kstate == KS_HOLD && (g_steps[g_kcur].mask & PK_TOUCH)) dm_touch_set(NULL, 0);
|
||||
if (g_kstate == KS_HOLD && (g_steps[g_kcur].mask & PK_CLICK)) dm_ui_touch(2, g_steps[g_kcur].tx, g_steps[g_kcur].ty); // touch step confirmed while held: lift the finger
|
||||
g_krel = 1; // confirmed while still held (e.g. opening a panel): the state leaves KS_HOLD and no more key is output ⇒ released at this moment (wild1 step 3 used to get stuck here)
|
||||
g_kcur++; g_kstate = KS_PRE; g_ktries = 0; step_chain();
|
||||
}
|
||||
// Pre: checked while the current step is waiting for its pre; when the current step is already pressed, the next step's is checked (only messages appearing after the previous step was pressed count)
|
||||
int j = g_kstate == KS_PRE ? g_kcur : g_kcur + 1;
|
||||
if (j < g_nsteps && !g_steps[j].seen_pre && g_steps[j].pre[0] && strstr(m, g_steps[j].pre)) { script_clock c = script_clock_now(); g_steps[j].seen_pre = 1; g_steps[j].seen_at = now_ns(); g_steps[j].seen_clock = c.value; g_steps[j].seen_server = c.server; }
|
||||
}
|
||||
pthread_mutex_unlock(&g_klock);
|
||||
}
|
||||
static uint32_t steps_now(void) {
|
||||
if (!g_nsteps) return 0;
|
||||
uint64_t t = now_ns(); uint32_t m = 0;
|
||||
pthread_mutex_lock(&g_klock);
|
||||
if (g_kcur < g_nsteps) {
|
||||
typeof(g_steps[0]) *st = &g_steps[g_kcur];
|
||||
script_clock c = script_clock_now();
|
||||
// Clock source switch (main menu frames ↔ in-run ticks ↔ frozen-period frames): the start point is recounted from the new source, otherwise the elapsed time across sources is always 0 and the wait never ends
|
||||
if (st->seen_pre && c.server != st->seen_server) { st->seen_clock = c.value; st->seen_server = c.server; }
|
||||
if (g_kstate != KS_PRE && c.server != g_kpress_server) { g_kpress_clock = c.value; g_kpress_server = c.server; }
|
||||
uint64_t elapsed = script_clock_elapsed(c, st->seen_clock, st->seen_server);
|
||||
int dm_loadscreen_up(void); void dm_hold_for_loadscreen(void);
|
||||
if (g_kstate == KS_PRE && st->seen_pre && elapsed >= script_seconds_to_units(st->delay_seconds, c.server) && dm_loadscreen_up()) dm_hold_for_loadscreen(); // due (pre seen and delay elapsed) but the loading screen is still up: freeze the tick, do not press yet (diag.c). Not frozen while the delay has not elapsed: after a waypoint act change a step due to press 62 seconds later was queued, and as soon as the loading screen opened the clock was frozen before the server handled the teleport, loading waited for the server, the server waited for the clock, deadlock
|
||||
else if (g_kstate == KS_PRE && st->seen_pre && g_krel && (g_kend >= g_kbeg || (st->mask & (PK_MSG | PK_CLICK))) && elapsed >= script_seconds_to_units(st->delay_seconds, c.server)) { // a message step presses no key, so it need not wait for the previous key's controller action to wrap up (a Begin re-triggered while the previous step is held sometimes never gets its End, 1 of 3 runs of the Mac iOS app got stuck here)
|
||||
g_kstate = KS_HOLD; g_kpress = t; g_kpress_clock = c.value; g_kpress_server = c.server; g_krel = 0; g_kbeg = g_kend = 0; g_ktries = 1;
|
||||
if (st->mask & PK_TOUCH) { dm_touch p = {0, st->tx, st->ty}; dm_touch_set(&p, 1); }
|
||||
if (st->mask & PK_MSG) dm_ui_send_queue(st->msg);
|
||||
if (st->mask & PK_CLICK) dm_ui_touch(0, st->tx, st->ty);
|
||||
LOG("hid script step %d pressed mask=%#x (pre '%s')", g_kcur + 1, st->mask, st->pre);
|
||||
} else if (g_kstate == KS_OK && t - g_kpress >= WALL_FALLBACK_NS) {
|
||||
die("[RED] hid script step %d: wall-clock fallback timeout (waited %llu server ticks, wall clock %llu ms, confirm '%s')", g_kcur + 1,
|
||||
(unsigned long long)script_clock_elapsed(c, g_kpress_clock, g_kpress_server), (unsigned long long)((t - g_kpress) / 1000000ull), st->ok);
|
||||
} else if (g_kstate == KS_OK && script_clock_elapsed(c, g_kpress_clock, g_kpress_server) >= OK_WAIT_TICKS) {
|
||||
if (g_ktries >= 3) die("[RED] hid script step %d: no confirm '%s' after 3 presses (waited %llu server ticks, wall clock %llu ms)", g_kcur + 1, st->ok,
|
||||
(unsigned long long)script_clock_elapsed(c, g_kpress_clock, g_kpress_server), (unsigned long long)((t - g_kpress) / 1000000ull));
|
||||
LOG("hid script step %d waiting for confirm: %llu server ticks, wall clock %llu ms; pressing again (press %d)", g_kcur + 1,
|
||||
(unsigned long long)script_clock_elapsed(c, g_kpress_clock, g_kpress_server), (unsigned long long)((t - g_kpress) / 1000000ull), g_ktries + 1);
|
||||
g_kstate = KS_HOLD; g_kpress = t; g_kpress_clock = c.value; g_kpress_server = c.server; g_krel = 0; g_kbeg = g_kend = 0; g_ktries++;
|
||||
if (st->mask & PK_TOUCH) { dm_touch p = {0, st->tx, st->ty}; dm_touch_set(&p, 1); }
|
||||
if (st->mask & PK_MSG) dm_ui_send_queue(st->msg);
|
||||
if (st->mask & PK_CLICK) dm_ui_touch(0, st->tx, st->ty);
|
||||
LOG("hid script step %d pressed again (press %d)", g_kcur + 1, g_ktries);
|
||||
}
|
||||
if (g_kstate == KS_HOLD) {
|
||||
uint64_t held = script_clock_elapsed(c, g_kpress_clock, g_kpress_server);
|
||||
if (held < step_hold_units(st, c.server)) m = st->mask & ~(PK_TOUCH | PK_MSG | PK_CLICK);
|
||||
else if (st->ok[0]) { LOG("hid script step %d released (server tick %llu, held %llu ticks)", g_kcur + 1, (unsigned long long)c.value, (unsigned long long)held); g_krel = 1; g_kstate = KS_OK; if (st->mask & PK_TOUCH) dm_touch_set(NULL, 0); if (st->mask & PK_CLICK) dm_ui_touch(2, st->tx, st->ty); }
|
||||
else { // no confirm item: done on release
|
||||
LOG("hid script step %d released (clock %llu, held %llu ticks)", g_kcur + 1, (unsigned long long)c.value, (unsigned long long)held); g_krel = 1; if (st->mask & PK_TOUCH) dm_touch_set(NULL, 0); if (st->mask & PK_CLICK) dm_ui_touch(2, st->tx, st->ty);
|
||||
g_kcur++; g_kstate = KS_PRE; g_ktries = 0; step_chain();
|
||||
}
|
||||
}
|
||||
}
|
||||
pthread_mutex_unlock(&g_klock);
|
||||
return m;
|
||||
}
|
||||
static void keys_parse(void) {
|
||||
g_nkeys = 0; g_kbase = now_ns();
|
||||
cursor_tests_parse();
|
||||
const char *e = getenv("DM_KEYS"); if (!e) e = "";
|
||||
char buf[4096]; strncpy(buf, e, sizeof buf - 1); buf[sizeof buf - 1] = 0;
|
||||
for (char *save, *tok = strtok_r(buf, ",", &save); tok; tok = strtok_r(NULL, ",", &save)) {
|
||||
char *at = strchr(tok, '@'); if (!at) die("DM_KEYS entry '%s' lacks @time / @message", tok);
|
||||
*at = 0;
|
||||
uint32_t mask = 0; float tx = 0, ty = 0;
|
||||
if (tok[0] == 'T' && ((tok[1] >= '0' && tok[1] <= '9') || tok[1] == '.')) { // touch point: T<x>:<y> (normalized coordinates of the rendered picture [0,1), origin at top left)
|
||||
char *c = strchr(tok, ':'); if (!c) die("DM_KEYS touch entry '%s' format: T<x>:<y>@pre…", tok);
|
||||
tx = (float)atof(tok + 1); ty = (float)atof(c + 1); mask = PK_TOUCH;
|
||||
}
|
||||
if (tok[0] == 'C' && ((tok[1] >= '0' && tok[1] <= '9') || tok[1] == '.')) { // UI click: C<x>:<y> (normalized coordinates of the rendered picture), dm_ui_touch is sent once each for press / release
|
||||
char *c = strchr(tok, ':'); if (!c) die("DM_KEYS click entry '%s' format: C<x>:<y>@pre…", tok);
|
||||
tx = (float)atof(tok + 1); ty = (float)atof(c + 1); mask = PK_CLICK;
|
||||
}
|
||||
const char *msg = NULL; // step that sends a UI message: MSG:<category>:<action>[:<argument>]@pre[>confirm] -- presses no key, at the due time sends this message to the UI root widget (after the character select page was switched to the keyboard-and-mouse UI the controller confirm key can no longer enter a run, so the last step of test run entry sends MainMenuHD:Play instead)
|
||||
if (!strncmp(tok, "MSG:", 4)) { if (!strchr(tok + 4, ':') || strlen(tok + 4) >= sizeof g_steps[0].msg) die("DM_KEYS message entry '%s' format: MSG:<category>:<action>[:<argument>]@pre[>confirm]", tok); msg = tok + 4; mask = PK_MSG; }
|
||||
for (int i = 0; i < (int)(sizeof k_names / sizeof *k_names); i++) if (!strcmp(tok, k_names[i].name)) mask = k_names[i].mask;
|
||||
for (int i = 0; i < 4; i++) if (!strcmp(tok, k_stick[i].name)) mask = k_stick[i].mask;
|
||||
if (!mask) die("DM_KEYS unknown key name '%s'", tok);
|
||||
const char *v = at + 1;
|
||||
if ((*v >= '0' && *v <= '9') || *v == '.') {
|
||||
if (mask & (PK_TOUCH | PK_MSG | PK_CLICK)) die("DM_KEYS touch / message entries only support UI-message steps (…@pre…), not timed ones");
|
||||
char *plus = strchr(v, '+');
|
||||
double t = atof(v), d = plus ? atof(plus + 1) : 0.2;
|
||||
if (g_nkeys == KMAX) die("DM_KEYS has more than %d entries", KMAX);
|
||||
g_keys[g_nkeys++] = (typeof(g_keys[0])){mask, (uint64_t)(t * 1e9), (uint64_t)((t + d) * 1e9)};
|
||||
} else {
|
||||
if (g_nsteps == KMAX) die("DM_KEYS has more than %d entries", KMAX);
|
||||
typeof(g_steps[0]) *st = &g_steps[g_nsteps++]; st->mask = mask; st->tx = tx; st->ty = ty; st->msg[0] = 0; if (msg) strcpy(st->msg, msg);
|
||||
const char *gt = strchr(v, '>');
|
||||
size_t np = gt ? (size_t)(gt - v) : strlen(v);
|
||||
if (np >= sizeof st->pre || (gt && (!gt[1] || strlen(gt + 1) >= sizeof st->ok))) die("DM_KEYS message step '%s' format: key@pre[%%delay seconds][+hold seconds][>confirm]", v);
|
||||
memcpy(st->pre, v, np); st->pre[np] = 0;
|
||||
char *pp = strchr(st->pre, '+'); if (pp) { st->hold_override = 1; st->hold_seconds = atof(pp + 1); *pp = 0; } // pre+hold seconds → 25 ticks/s in a run, 60 presented frames/s in the main menu; less than 1 unit cannot be sampled
|
||||
char *pc = strchr(st->pre, '%'); if (pc) { st->delay_seconds = atof(pc + 1); *pc = 0; } // pre%seconds → 25 ticks/s in a run, 60 presented frames/s in the main menu
|
||||
if (gt) strcpy(st->ok, gt + 1);
|
||||
}
|
||||
}
|
||||
if (g_nsteps || g_ncursor_tests) { g_steps[0].seen_pre = 0; if (g_nsteps && !g_steps[0].pre[0]) { script_clock c = script_clock_now(); g_steps[0].seen_pre = 1; g_steps[0].seen_at = now_ns(); g_steps[0].seen_clock = c.value; g_steps[0].seen_server = c.server; } void uimsg_subscribe(void (*)(const char *, const char *, const char *)); uimsg_subscribe(keys_on_uimsg); }
|
||||
LOG("hid script input: %d timed entries, %d UI-message steps", g_nkeys, g_nsteps);
|
||||
}
|
||||
static pthread_once_t g_kone = PTHREAD_ONCE_INIT;
|
||||
void hid_keys_init(void) { pthread_once(&g_kone, keys_parse); }
|
||||
static uint64_t keys_now(void) {
|
||||
pthread_once(&g_kone, keys_parse);
|
||||
uint64_t t = now_ns() - g_kbase, m = steps_now();
|
||||
for (int i = 0; i < g_nkeys; i++) if (t >= g_keys[i].t0 && t < g_keys[i].t1) m |= g_keys[i].mask;
|
||||
static _Atomic uint64_t seen; // print one line the first time each key is really read by the game (proves the injection arrived)
|
||||
uint64_t newbits = m & ~atomic_fetch_or(&seen, m);
|
||||
if (newbits) LOG("hid script keys read for the first time: mask=%#llx @%.2fs", newbits, t / 1e9);
|
||||
return m;
|
||||
}
|
||||
|
||||
// Optional sampling of actual GetNpadState returns; state changes + an activity witness at most once per second, a change in sampling alone does not flood the log.
|
||||
// After 1024 entries limit_reached is stated explicitly, a truncated log does not count as complete gesture evidence; without the environment variable no file is written.
|
||||
static FILE *g_receipt_file;
|
||||
static pthread_once_t g_receipt_once = PTHREAD_ONCE_INIT;
|
||||
static pthread_mutex_t g_receipt_lock = PTHREAD_MUTEX_INITIALIZER;
|
||||
static void receipt_init(void) {
|
||||
const char *path = getenv("DM_HID_RECEIPT_LOG");
|
||||
if (path && *path) {
|
||||
g_receipt_file = fopen(path, "w");
|
||||
if (!g_receipt_file) LOG("[RED] HID receipt log file failed to open: %s", path);
|
||||
}
|
||||
}
|
||||
static void receipt(const NpadState *s, uint32_t id, uint32_t style, uint64_t km, uint64_t read_start_ns) {
|
||||
if (!g_receipt_file) return;
|
||||
static NpadState last; static uint32_t last_id, last_style;
|
||||
static uint64_t last_km, last_ns; static unsigned count; static int have, truncated;
|
||||
NpadState state = *s; state.sampling = 0; uint64_t t = now_ns();
|
||||
pthread_mutex_lock(&g_receipt_lock);
|
||||
if (truncated || (have && id == last_id && style == last_style && km == last_km && !memcmp(&state, &last, sizeof state) && t - last_ns < 1000000000ull)) {
|
||||
pthread_mutex_unlock(&g_receipt_lock); return;
|
||||
}
|
||||
if (count == 1024) {
|
||||
fprintf(g_receipt_file, "{\"event\":\"limit_reached\",\"limit\":1024}\n"); truncated = 1;
|
||||
} else {
|
||||
fprintf(g_receipt_file, "{\"event\":\"npad_return\",\"read_start_ns\":%llu,\"time_ns\":%llu,\"sampling\":%lld,\"id\":%u,\"style\":%u,\"buttons\":%llu,\"lx\":%d,\"ly\":%d,\"rx\":%d,\"ry\":%d,\"attributes\":%u,\"script_mask\":%llu}\n",
|
||||
(unsigned long long)read_start_ns, (unsigned long long)t, (long long)s->sampling, id, style, (unsigned long long)s->buttons,
|
||||
s->lx, s->ly, s->rx, s->ry, s->attributes, (unsigned long long)km);
|
||||
last = state; last_id = id; last_style = style; last_km = km; last_ns = t; have = 1; count++;
|
||||
}
|
||||
if (fflush(g_receipt_file)) { LOG("[RED] HID receipt log write failed"); truncated = 1; }
|
||||
pthread_mutex_unlock(&g_receipt_lock);
|
||||
}
|
||||
|
||||
void (*dm_pad_hook)(uint64_t, uint64_t *, int32_t *, int32_t *, int32_t *, int32_t *);
|
||||
static void fill(NpadState *s, uint32_t id, uint32_t style) {
|
||||
pthread_once(&g_receipt_once, receipt_init);
|
||||
ui_taps_step();
|
||||
dm_ui_frontend_pump();
|
||||
{ void dm_textinput_pump(void); dm_textinput_pump(); } // the extra "bring up keyboard" press for text boxes, setting the value of quantity boxes (dm_textinput.c)
|
||||
{ void dm_update_pump(void); dm_update_pump(); } // build number and new-version notice on the character select page (dm_update.c)
|
||||
{ void dm_actions_pump(void); dm_actions_pump(); } // vendor buy/sell / stat point entry and readout (dm_actions.c)
|
||||
{ void dm_world_pump(void); dm_world_pump(); } // tapping a world target = interact (dm_world.c)
|
||||
dm_ui_mouse_pump(); // UI mouse messages (native clicks) are sent at the controller read: main thread, one per frame
|
||||
{ void dm_aim_cone_pump(void); dm_aim_cone_pump(); } // target selection by finger direction: restored when the right stick returns to zero (hle_private_aim.c)
|
||||
if (atomic_load(&g_stall_due) && atomic_exchange(&g_stall_due, 0)) usleep((useconds_t)g_stall_ms * 1000); // DM_TEST_HID_STALL_MS
|
||||
uint64_t read_start_ns = g_receipt_file ? now_ns() : 0; // covers the whole read interval of the multi-atomic fields
|
||||
memset(s, 0, sizeof *s);
|
||||
s->sampling = ++g_sampling;
|
||||
if (!(style_of(id) & style)) { receipt(s, id, style, 0, read_start_ns); return; } // this style is not connected: an all-zero return is sampled as well
|
||||
uint64_t km = keys_now();
|
||||
// Merge the buttons of all channels first (touch / key script / host-pressed + controller; all 0 when no controller is connected), then arbitrate the
|
||||
// "exclusive press": the press made on the player's behalf must look at what the game will actually receive this time,
|
||||
// not at one single channel (previously only the touch channel was checked, so a controller player pressing + to open the inventory in the first 1.5
|
||||
// seconds after joining would still turn the exclusive L into a tab switch)
|
||||
{ uint64_t dm_ui_alone_take(uint64_t);
|
||||
uint64_t tb = ((g_in.buttons | atomic_load(&g_press_latch)) & ~dm_ui_auto_release()) | (km & ~(uint64_t)PK_MASK) | dm_ui_auto_buttons(), pb = atomic_load(&g_pad_btn) | atomic_load(&g_pad_latch);
|
||||
static int t_old = -1; if (t_old < 0) t_old = getenv("DM_TEST_ALONE_NOPAD") != NULL; // Test control: reproduces the pre-change behavior (the arbitration ignores controller buttons, and controller buttons are still merged in during an exclusive press)
|
||||
uint64_t alone = dm_ui_alone_take(t_old ? tb : tb | pb);
|
||||
if (alone) s->buttons = t_old ? alone | atomic_load(&g_pad_btn) | atomic_exchange(&g_pad_latch, 0) : alone; // The exclusive press (dm_cursor.c): other buttons are withheld for these few frames, neither press-edge latch is cleared, and they are delivered as usual afterwards
|
||||
else s->buttons = ((g_in.buttons | atomic_exchange(&g_press_latch, 0)) & ~dm_ui_auto_release()) | (km & ~(uint64_t)PK_MASK) | dm_ui_auto_buttons() | atomic_load(&g_pad_btn) | atomic_exchange(&g_pad_latch, 0); }
|
||||
s->lx = g_in.lx; s->ly = g_in.ly; s->rx = g_in.rx; s->ry = g_in.ry;
|
||||
// Sticks: if the touch stick has a value use it, otherwise use the controller's. The controller side only gives a non-zero value in controller mode and when the
|
||||
// stick has actually been pushed out (dm_gamepad.m), so a slightly drifting controller left lying aside never reaches here
|
||||
{ uint64_t ax = atomic_load(&g_pad_axes); int16_t plx = (int16_t)ax, ply = (int16_t)(ax >> 16), prx = (int16_t)(ax >> 32), pry = (int16_t)(ax >> 48);
|
||||
if (!(s->lx || s->ly) && (plx || ply)) { s->lx = plx; s->ly = ply; } if (!(s->rx || s->ry) && (prx || pry)) { s->rx = prx; s->ry = pry; } }
|
||||
{ int dm_padmode_input_gate(uint64_t *); if (dm_padmode_input_gate(&s->buttons)) s->lx = s->ly = s->rx = s->ry = 0; } // Controller / touch switch transaction in progress: neither keys nor sticks are handed to the game; keys held when the switch was triggered are not handed over either until released (dm_padmode.c)
|
||||
if (dm_pad_hook) dm_pad_hook(guest_ns(), &s->buttons, &s->lx, &s->ly, &s->rx, &s->ry);
|
||||
{ uint64_t dm_runeword_pad_filter(uint64_t); s->buttons = dm_runeword_pad_filter(s->buttons); } // While the runeword list is open in controller mode, the D-pad / A / B / L / R belong to the list (dm_runeword.c)
|
||||
{ uint64_t dm_padmode_page_filter(uint64_t); s->buttons = dm_padmode_page_filter(s->buttons); } // While one of our own pages (the Custom page etc.) is open in controller mode, the buttons the page declares belong to the page (dm_padmode.c)
|
||||
{ uint64_t dm_swapskill_filter(uint64_t); s->buttons = dm_swapskill_filter(s->buttons); } // off-hand set skill: hold that key back until the weapon swap is done (dm_actions.c)
|
||||
{ static int trx, tryy, tdelay, held, tset = -1; if (tset < 0) { const char *e = getenv("DM_TEST_RSTICK"); tset = e && sscanf(e, "%d,%d,%d", &trx, &tryy, &tdelay) >= 2; } // test switch DM_TEST_RSTICK=rx,ry[,delay count]: while the key script holds A / B / X / Y, set the right stick to this value (simulates "hold a skill button and drag", the game itself also sees the right stick); delay = on which controller read after the press it starts being given (a real finger presses first, then drags out a direction)
|
||||
if (tset > 0) { held = (s->buttons & 0xf) ? held + 1 : 0; if (held > tdelay) { s->rx = trx; s->ry = tryy; } } }
|
||||
if (s->lx || s->ly || (km & PK_MASK & ~(uint64_t)PK_TOUCH)) atomic_store(&g_stick_ns, now_ns()); // the last time the player's stick (front end / script pseudo-keys) was non-zero; a stick rewritten by the cursor does not count
|
||||
dm_cursor_apply(&s->buttons, &s->lx, &s->ly); // pointer target: within this sample, rewrite the left stick / A according to the original game's cursor position (see dm_cursor.h)
|
||||
{ static uint64_t prev; if ((s->buttons & DM_BTN_L) && !(prev & DM_BTN_L)) { int n = atomic_fetch_add(&g_dm_l_presses, 1) + 1; if (n <= 3) LOG("controller read: the 'Show Items' key (L) was delivered to the game, time %d in this process", n); } prev = s->buttons; } // number of presses of the "Show Items" key (L) delivered to the game (used by the boss drop readout, dm_bossrush.c)
|
||||
if (km & PK_LSL) s->lx = -32767; if (km & PK_LSR) s->lx = 32767; // script stick pseudo-keys: push to full (Horizon stick ±32767, up is positive)
|
||||
if (km & PK_LSU) s->ly = 32767; if (km & PK_LSD) s->ly = -32767;
|
||||
s->attributes = ATTR_CONNECTED;
|
||||
atomic_store_explicit(&g_sampled_right,(uint32_t)s->rx|((uint64_t)(uint32_t)s->ry<<32),memory_order_release);
|
||||
receipt(s, id, style, km, read_start_ns);
|
||||
}
|
||||
|
||||
HLE_FORCE("_ZN2nn3hid14InitializeNpadEv", "connect to the hid service; the host takes input from dm_input", void, h_hid_InitNpad, void) {}
|
||||
HLE_FORCE("_ZN2nn3hid21InitializeTouchScreenEv", "connect to the hid service; touch input is wired separately by the front end", void, h_hid_InitTouch, void) {}
|
||||
HLE_FORCE("_ZN2nn3hid22SetSupportedNpadIdTypeEPKjm", "supported controller ids; fixed by the host", void, h_hid_SetIds, const uint32_t *ids, uint64_t n) { (void)ids; (void)n; }
|
||||
// BitFlagSet<32> is passed by value in a w register
|
||||
HLE_FORCE("_ZN2nn3hid24SetSupportedNpadStyleSetENS_4util10BitFlagSetILi32ENS0_12NpadStyleTagEEE", "supported controller styles; fixed by the host", void, h_hid_SetStyles, uint32_t styles) { (void)styles; }
|
||||
HLE_FORCE("_ZN2nn3hid29SetNpadHandheldActivationModeENS0_26NpadHandheldActivationModeE", "handheld controller activation mode; fixed by the host", void, h_hid_SetHhMode, int m) { (void)m; }
|
||||
HLE_FORCE("_ZN2nn3hid15GetNpadStyleSetERKj", "controller style query; follows DM_OPMODE", uint32_t, h_hid_GetStyle, const uint32_t *id) { return style_of(*id); }
|
||||
HLE_FORCE("_ZN2nn3hid12GetNpadStateEPNS0_16NpadFullKeyStateERKj", "full-key controller state; from dm_input", void, h_hid_GetFullKey, NpadState *s, const uint32_t *id) { fill(s, *id, STYLE_FULLKEY); }
|
||||
HLE_FORCE("_ZN2nn3hid12GetNpadStateEPNS0_17NpadHandheldStateERKj", "handheld controller state; from dm_input", void, h_hid_GetHandheld, NpadState *s, const uint32_t *id) { fill(s, *id, STYLE_HANDHELD); }
|
||||
|
||||
// Vibration: device count 0, the game will not initialize / send any further
|
||||
HLE_FORCE("_ZN2nn3hid20IsVibrationPermittedEv", "vibration permission; not supported by the host (returns bool)", unsigned, h_hid_VibPermit, void) { return 0; }
|
||||
HLE_FORCE("_ZN2nn3hid25GetVibrationDeviceHandlesEPNS0_21VibrationDeviceHandleEiRKjNS_4util10BitFlagSetILi32ENS0_12NpadStyleTagEEE",
|
||||
"vibration devices; 0 on the host", int, h_hid_GetVibHandles, void *out, int max, const uint32_t *id, uint32_t style) { (void)out; (void)max; (void)id; (void)style; return 0; }
|
||||
HLE_FORCE("_ZN2nn3hid25InitializeVibrationDeviceERKNS0_21VibrationDeviceHandleE", "vibration; not supported by the host", void, h_hid_InitVib, const void *h) { (void)h; }
|
||||
HLE_FORCE("_ZN2nn3hid18SendVibrationValueERKNS0_21VibrationDeviceHandleERKNS0_14VibrationValueE", "vibration; not supported by the host", void, h_hid_SendVib, const void *h, const void *v) { (void)h; (void)v; }
|
||||
|
||||
// Touchscreen: nn::hid::TouchScreenState<16> = { int64 sampling; int32 count; int32 _r; TouchState touches[16] (0x28 each) }
|
||||
// Contents are filled by hle_touch.c from the touch points pushed by the front end (see that file for the layout and the short-press latch); the game only looks at entry 0, returns the number of states written, 1
|
||||
typedef struct { int64_t sampling; int32_t count, _r; uint8_t touches[16][0x28]; } TouchState16;
|
||||
_Static_assert(sizeof(TouchState16) == 0x290, "TouchScreenState<16>");
|
||||
int touch_fill(void *out, uint64_t now_ns);
|
||||
HLE_FORCE("_ZN2nn3hid20GetTouchScreenStatesILm16EEEiPNS0_16TouchScreenStateIXT_EEEi", "touch screen states: front-end touch points (hle_touch.c)", int, h_hid_GetTouch, TouchState16 *out, int n) {
|
||||
if (n < 1) return 0;
|
||||
touch_fill(out, guest_ns());
|
||||
return 1;
|
||||
}
|
||||
@@ -1,9 +1,10 @@
|
||||
#!/usr/bin/env python3
|
||||
# Unsigned IPA straight from the NSPs: identify the game (title ID → game profile dm/games/<id>/) → extract (nx_extract) → rewrite the code modules (patch_nso.py)
|
||||
# → (profiles with shader_glslc_capture: capture the shaders the game compiles at run time, by booting it in the Mac host for a few seconds) → split out the shaders and check them against the profile's tables (shader_split.py)
|
||||
# (a Vulkan game with shader_vk_capture, Super Mario Galaxy, boots the same way to capture the SPIR-V shaders it generates)
|
||||
# (a game with shader_oodle, Pokémon Legends: Z-A, has its Oodle-compressed archives decoded for the split by its own decoder, run in the Mac host's tool mode)
|
||||
# → iOS shader libraries (dmsh_build.py) → build_ipa.sh. A Vulkan game (profile key graphics vulkan: Hades) skips the NVN shader steps: vk_shaders.py translates its SPIR-V shaders instead. A game with no matching profile is surveyed (<NSP folder>/<title ID>_survey/survey.md), then gets a provisional profile guessed from
|
||||
# → (a game that compiles shaders at run time: capture them, by booting it in the Mac host for a few seconds; an Unreal game: decode its shaders with its own Oodle, the same way)
|
||||
# → split out the shaders and check them against the profile's tables (shader_split.py) (a Vulkan game without SPIR-V files, Super Mario Galaxy, boots the same way to capture the SPIR-V shaders it generates;
|
||||
# a game with Trinity packs, Pokémon Legends: Z-A, has them decoded for the split by its own Oodle decoder, run in the Mac host's tool mode)
|
||||
# → iOS shader libraries (dmsh_build.py) → build_ipa.sh. A Vulkan game (main imports Vulkan, not NVN: Hades) skips the NVN shader steps: vk_shaders.py translates its SPIR-V shaders instead.
|
||||
# Which of these steps a game needs follows from its files (dm/games/profile_guess.py shader_sources), not from its profile. A game with no matching profile is surveyed (<NSP folder>/<title ID>_survey/survey.md), then gets a provisional profile guessed from
|
||||
# the survey (<NSP folder>/<title ID>_build/profile/<title ID>/, outside the repository, found through DM_GAMES_DIR) and a best-effort IPA that may not boot; SURVEY_ONLY / --survey-only stops after the survey.
|
||||
# Two ways to run it: open this file in Spyder and press F5 (it uses the values under "Settings" below; an empty NSP folder or prod.keys is asked for at run time);
|
||||
# or in a terminal: python3 scripts/build_ipa_from_nsp.py <NSP folder> -k <prod.keys> [options] (values given on the command line override "Settings", -h lists every option;
|
||||
@@ -415,15 +416,19 @@ def survey(nsp_dir, app, fam, keys, repo, py, env):
|
||||
|
||||
# What it means for a port
|
||||
G = pg.guess(F) # the provisional profile's keys (write_profile writes the same)
|
||||
vulkan = G.get('graphics', ('nvn',))[0] == 'vulkan'
|
||||
keep = G['modules'][0].split() # how the build will get the shaders (decided again from the extraction at build time; the survey has only a RomFS manifest, so Unreal paks are guessed from main's markers)
|
||||
S = pg.shader_sources(mods['main']['imports'], F['glsl_src'], [m for m in keep if 'glslcCompile' in mods[m]['exports']], rom['mags']['SPIR-V'],
|
||||
rom['exts'].get('.pak', 0) if F.get('main_markers', set()) & {'Unreal', 'Unreal 4'} else 0, rom['mags']['Trinity pack'])
|
||||
vulkan = S['graphics'][0] == 'vulkan'
|
||||
notes = []
|
||||
if not vulkan and 'nvnBootstrapLoader' not in mods['main']['imports']: notes.append('main imports neither nvnBootstrapLoader nor Vulkan functions: the host implements only NVN and Vulkan')
|
||||
if vulkan: notes.append(f'{G["graphics"][1]}; the NVN shader steps are skipped' + (', and its shaders are captured by booting it on the Mac (no SPIR-V file in the RomFS)' if 'shader_vk_capture' in G else ''))
|
||||
if vulkan: notes.append(f'{S["graphics"][1]}; the NVN shader steps are skipped' + (', and its shaders are captured by booting it on the Mac (no SPIR-V file in the RomFS)' if S['vk_capture'][0] else ''))
|
||||
notes += [S[k][1] for k in ('glslc_capture', 'ue_decode') if S[k][0]]
|
||||
if extra: notes.append(f'modules besides main / sdk: {", ".join(f"{n} ({mods[n]["kind"]})" for n in extra)}; the provisional profile loads {G["modules"][0]} (load order main, subsdk*, sdk)')
|
||||
if rom['mags']['UnityFS'] or 'Unity IL2CPP' in F['markers']: notes.append('Unity: the shaders are Shader objects in UnityFS bundles and data.unity3d, which shader_split.py reads through unityfs.py (written for Unity 2020.3\'s Switch format, also read for 2017.4 and 2022.3; check other Unity versions first)')
|
||||
if F['glslc']: notes.append(f'main has {F["glslc"]} GLSLC compiler output magics (0x19866891): ' + ('the shaders are compiled into main (13 Sentinels, the KTGL games): shader_main_glslc on' if 'shader_main_glslc' in G else 'not its shaders, the RomFS has shader containers (Engage)'))
|
||||
if rom['mags']['Trinity pack']: notes.append('Trinity packs (Game Freak, arc/data.trpfs with its index data.trpfd): the shaders are BNSH inside Oodle Mermaid streams; shader_split.py decodes them with the game\'s own OodleLZ_Decompress in the Mac host, whose address dm/host/hle_oodle.c lists per main build (find it by its error strings, as for Pokémon Legends: Z-A)')
|
||||
elif not (rom['mags']['BNSH'] or rom['mags']['BAHS']) and not rom['mags']['UnityFS'] and not vulkan and 'shader_main_glslc' not in G:
|
||||
if F['glslc']: notes.append(f'main has {F["glslc"]} GLSLC compiler output magics (0x19866891): shader_split.py takes those that are whole outputs (13 Sentinels and the KTGL games compile their shaders into main; Unity\'s player has 11 of its own)')
|
||||
if rom['mags']['Trinity pack']: notes.append('Trinity packs (Game Freak, arc/data.trpfs with its index data.trpfd): the shaders are BNSH inside Oodle Mermaid streams; shader_split.py decodes them with the game\'s own OodleLZ_Decompress in the Mac host, which dm/host/hle_oodle.c finds by the library\'s name string')
|
||||
elif not (rom['mags']['BNSH'] or rom['mags']['BAHS']) and not rom['mags']['UnityFS'] and not vulkan and not F['glslc']:
|
||||
notes.append('no BNSH / BAHS at the top level: the shaders are inside compressed or other containers or in main, and must be located first')
|
||||
major = {v.split('_')[0] for v in F['sdkver']}
|
||||
if not major: notes.append('no SDK version string found')
|
||||
@@ -462,8 +467,9 @@ EMPTY_TABLES = { # the shader tables before (or, for a Vulkan game, instead of
|
||||
|
||||
def write_profile(pdir, app, f):
|
||||
"""A provisional game.tsv (+ the empty tables) from the survey, in pdir (outside the repository). An existing game.tsv is kept, so it can be edited by hand; delete it to regenerate.
|
||||
The keys the game files decide come from dm/games/profile_guess.py (each written under its reason): the graphics API, the modules to load, the shader sources, the memory partition,
|
||||
the SDK 8 / 10 font and NVN API, engine presets; the rest keeps game.py's defaults until the heap check (below) or a developer sets it"""
|
||||
The keys the game files decide come from dm/games/profile_guess.py (each written under its reason): the modules to load, the memory partition, engine presets; the rest keeps game.py's
|
||||
defaults until the heap check (below) or a developer sets it. What the files show at every build is not written: the graphics API and the shader sources (shader_sources), the shared-font
|
||||
layout and the NVN API version (the host reads the SDK version)"""
|
||||
pdir.mkdir(parents=True, exist_ok=True)
|
||||
if (pdir / 'game.tsv').exists():
|
||||
say(f'reusing the provisional profile: {pdir / "game.tsv"} (delete it to regenerate from the survey)')
|
||||
@@ -489,16 +495,12 @@ def write_profile(pdir, app, f):
|
||||
'# ---- App ----', f'name{T}{name}', f'ipa{T}{ipa_name(name, app)}', f'app_category{T}public.app-category.games', *key('pad_layout'), *key('op_mode'), '',
|
||||
'# ---- Versions ----', T.join(['build', f['version'], f['display']] + [f'{m}={F["mods"][m]["bid"]}' for m in mods]), f'package{T}{f["version"]}', f'shader_versions{T}{f["version"]}', '',
|
||||
'# ---- Files ----', *key('modules'), f'romfs_marker{T}{R["biggest"][1]}', f'zip_store{T}{":".join(store + [".metallib"])}', '',
|
||||
'# ---- Graphics ----', *key('graphics'), *key('shader_vk_capture'), '',
|
||||
'# ---- Memory ----', *key('mem_reserve'), *key('mem_alias'), *key('mem_stack'), *key('app_pool'), '',
|
||||
'# ---- NVN driver constant segments (Mario\'s values, the translator\'s built-in layout, unless the engine says otherwise) ----',
|
||||
*key('nvn_tex_base', '0x20'), *key('nvn_tex_slots', '32'), f'nvn_samp_base{T}0x568', f'nvn_img_base{T}0x120', f'nvn_img_slots{T}8', f'nvn_ssbo_bases{T}vs=0x110 ps=0x510 cs=0x310', '',
|
||||
'# ---- Shader split ----', *key('shader_groups', " ".join(groups) or "/=romfs"), f'shader_note{T}{name} shaders (provisional profile)',
|
||||
*key('shader_main_glslc'), *key('shader_separable'), *key('shader_glslc_capture'), *key('shader_ue_decode'), *key('shader_oodle')]
|
||||
'# ---- Shader split ----', *key('shader_groups', " ".join(groups) or "/=romfs"), f'shader_note{T}{name} shaders (provisional profile)']
|
||||
(pdir / 'game.tsv').write_text('\n'.join(L) + '\n', encoding='utf-8')
|
||||
for n, text in EMPTY_TABLES.items(): (pdir / n).write_text(text)
|
||||
say(f'wrote a provisional game profile: {pdir / "game.tsv"}')
|
||||
for k in ('graphics', 'modules', 'shader_vk_capture', 'shader_main_glslc', 'shader_glslc_capture', 'shader_ue_decode', 'shader_oodle', 'mem_reserve', 'op_mode'):
|
||||
for k in ('modules', 'mem_reserve', 'op_mode'):
|
||||
if k in G and G[k][1]: say(f' {k} {G[k][0]}: {G[k][1]}')
|
||||
|
||||
|
||||
@@ -544,7 +546,7 @@ def reused(keyfile, key): # the note the stage's last successful run wrote bes
|
||||
|
||||
|
||||
def tool_files(repo): # what the shader stages run: dm/gfx/tools' scripts and the lock of the translator's fingerprints, the profile reader, the iOS compile convention, the NSO reader
|
||||
return sorted((repo / 'dm/gfx/tools').glob('*.py')) + [repo / p for p in ('dm/gfx/tools/dmsh_chain.lock', 'dm/games/game.py', 'dm/ios/tools/ios_metal.py', 'dm/verify/nsoimg.py')]
|
||||
return sorted((repo / 'dm/gfx/tools').glob('*.py')) + [repo / p for p in ('dm/gfx/tools/dmsh_chain.lock', 'dm/games/game.py', 'dm/ios/tools/ios_metal.py')]
|
||||
|
||||
|
||||
def profile_files(g): # every file of the game profile (game.tsv and its tables)
|
||||
@@ -711,7 +713,7 @@ def host_stopped(log, rc, secs): # a capture run whose host stopped by itself
|
||||
|
||||
|
||||
def capture_glslc(repo, dirs, pkg_ver, patched, work, env, logs, g):
|
||||
"""Profiles with shader_glslc_capture on (Fire Emblem: Three Houses): the game compiles some shaders at run time with its own GLSLC, so no game file holds them. The game boots in the Mac host
|
||||
"""A game that compiles some shaders at run time with its own GLSLC (Fire Emblem: Three Houses; profile_guess.py shader_sources), so no game file holds them. The game boots in the Mac host
|
||||
(headless and silent) with DM_GLSLC_CAPTURE (dm/host/hle_glslc.c), which writes each output to <version dir>/glslc_capture, where shader_split.py reads them; the outputs are the same on every run.
|
||||
dirs: {shader version: its extraction}; a folder that already holds every file the profile's shader map names is reused"""
|
||||
want = glslc_wanted(g)
|
||||
@@ -755,7 +757,7 @@ def ue_stamp(d): # what an extraction's decoded shader archives were made from
|
||||
|
||||
|
||||
def decode_ue_shaders(repo, dirs, pkg_ver, patched, work, env, logs, g):
|
||||
"""Profiles with shader_ue_decode on (Octopath Traveler 0): Unreal Engine keeps its shaders in the shader code archives of its paks, each shader Oodle-compressed, and only the game's own Oodle
|
||||
"""A game with Unreal paks (Octopath Traveler 0): Unreal Engine keeps its shaders in the shader code archives of its paks, each shader Oodle-compressed, and only the game's own Oodle
|
||||
decodes them. dm/gfx/tools/uepak.py takes the archives out of the paks (the pak index is AES-encrypted with a key from main) and writes one Oodle job; the game's code decodes it in the Mac host
|
||||
(DM_OODLE_JOB, dm/host/hle_oodle.c: right before nnMain, then the host exits), and the decoded archives go to <version dir>/ue_shaders, where shader_split.py reads them.
|
||||
The job runs the packed version's code (Oodle's format does not change between builds). A folder whose stamp matches its paks is reused"""
|
||||
@@ -774,7 +776,7 @@ def decode_ue_shaders(repo, dirs, pkg_ver, patched, work, env, logs, g):
|
||||
for s in ('save', 'temp'): (rd / s).mkdir(parents=True)
|
||||
PROG.set(step=f'{v}: shader archives out of the paks')
|
||||
listing, n = uepak.job(str(d / 'RomFS'), str(d / 'ExeFS/main'), str(jd))
|
||||
if not listing: raise BuildError(f'the profile has shader_ue_decode on, but no pak under {d / "RomFS"} holds a shader code archive')
|
||||
if not listing: raise BuildError(f'the RomFS has Unreal paks, but none under {d / "RomFS"} holds a shader code archive: where the shaders are has to be found first')
|
||||
say(f'{v}: ' + '; '.join(f'{x["name"]} {x["shaders"]:,} shaders' for x in listing) + f'; decoding {n:,} with the game\'s Oodle in the Mac host')
|
||||
PROG.set(step=f'{v}: decoding {n:,} shaders in the Mac host')
|
||||
henv = dict(env, DM_ROMFS=str(dirs[pkg_ver] / 'RomFS'), DM_SAVE=str(rd / 'save'), DM_TEMP=str(rd / 'temp'), DM_MSL_DIR=str(rd / 'msl'), DM_OODLE_JOB=str(jd / 'oodle.job'))
|
||||
@@ -805,7 +807,7 @@ def vk_done(d): # keys of the whole SPIR-V files in a capture folder (the host
|
||||
|
||||
|
||||
def capture_vk(repo, d, patched, work, env, logs, g):
|
||||
"""Profiles with shader_vk_capture on (Super Mario Galaxy): the game generates its SPIR-V shaders at run time (Hagi's GX shaders, from its pipeline database), so no game file holds them.
|
||||
"""A Vulkan game without SPIR-V files (Super Mario Galaxy): it generates its SPIR-V shaders at run time (Hagi's GX shaders, from its pipeline database), so no game file holds them.
|
||||
The game boots in the Mac host (headless and silent, no shader library) with DM_VK_CAPTURE (dm/gfx/vk/hle_vk.m), which writes each module to <version dir>/vk_capture, where vk_shaders.py
|
||||
reads them with the RomFS; the modules are the same on every run. The run stops once every key the profile's vk_capture.tsv lists is there; without the table, once none has been added for 20 s.
|
||||
A folder that already holds every listed key is reused"""
|
||||
@@ -919,7 +921,7 @@ def data_rows(b): # a table's data rows (without # comment lines: the header c
|
||||
def split_shaders(py, repo, roms, mains, pkg, env, logs, g, provisional=False, oodle=None):
|
||||
"""The shader split into pkg, checked against the profile's tables. Returns its key (None for a provisional profile or damaged files, which never reuse it): the package is reused
|
||||
when the tools, the profile and each version's main, RomFS (paths and sizes) and run-time capture are as at the last successful split.
|
||||
oodle: for profiles with shader_oodle on (Pokémon Legends: Z-A's Trinity packs), a function that builds the Mac host and returns the split's DM_OODLE_HOST / DM_OODLE_MODULES; called only when the split runs"""
|
||||
oodle: for a game with Trinity packs (Pokémon Legends: Z-A; profile_guess.py shader_sources), a function that builds the Mac host and returns the split's DM_OODLE_HOST / DM_OODLE_MODULES; called only when the split runs"""
|
||||
names = ('shader_map.tsv', 'shader_facts.tsv', 'shader_pairs.tsv')
|
||||
PROG.stage('shader split', step='checking the inputs') # then a spinner: shader_split.py writes the package and the tables only at the end
|
||||
kf = pkg.with_name(pkg.name + '.key')
|
||||
@@ -954,9 +956,9 @@ class ShaderWatch:
|
||||
and writes <key>_b.json again for every miss and every pixel shader: the furthest key written so far tells how far it got"""
|
||||
def __init__(self, g, ios, tmp):
|
||||
keys = set()
|
||||
for ln in open(g.table('shader_map.tsv'), errors='replace'): # the keys dmsh_build.py builds: column 4 of the rows from a shader package or from main
|
||||
for ln in open(g.table('shader_map.tsv'), errors='replace'): # the keys dmsh_build.py builds: column 4 of the rows from a shader package
|
||||
c = ln.rstrip('\n').split('\t')
|
||||
if len(c) > 4 and (c[0].endswith('.bin') or c[0].startswith('main.rodata@')): keys.add(c[3])
|
||||
if len(c) > 4 and c[0].endswith('.bin'): keys.add(c[3])
|
||||
pairs = {ln.split()[0] for ln in open(g.table('shader_pairs.tsv'), errors='replace') if ln.strip() and not ln.startswith('#')}
|
||||
order = sorted(keys - pairs) + sorted(keys & pairs)
|
||||
self.pos, self.n = {k: i for i, k in enumerate(order)}, len(order)
|
||||
@@ -1008,7 +1010,7 @@ def build_shaders(py, repo, sh, xlate, env, logs, g, provisional=False, split_ke
|
||||
if split_key:
|
||||
PROG.set(step='checking the inputs')
|
||||
key = fingerprint(split_key, ('files', tool_files(repo)), ('stat', xlate.parent), ('files', profile_files(g)), *toolchain(), str(xlate),
|
||||
{k: env.get(k) for k in ('MAIN', 'XLATE_DEBUG', 'XLATE_REDUCED')}, env=env)
|
||||
{k: env.get(k) for k in ('XLATE_DEBUG', 'XLATE_REDUCED')}, env=env)
|
||||
if next(libs.glob('*_b.metallib'), None) and (note := reused(kf, key)) is not None:
|
||||
PROG.end(f'{note}, reused (inputs unchanged)'); return ios
|
||||
else: key = None
|
||||
@@ -1175,9 +1177,6 @@ def main(args=None):
|
||||
g, provisional = gamemod.load(f'{app:016X}'), True
|
||||
say('[RED] provisional game profile: values guessed from the survey, the game may not boot; bring it up per docs/agents after packing')
|
||||
say(f'game profile: {g.id}{" (provisional: " + str(g.dir) + ")" if provisional else ""}')
|
||||
gfx = g.get('graphics', 'nvn') # nvn, or vulkan (Hades: the host's Vulkan layer, SPIR-V shaders)
|
||||
if gfx not in ('nvn', 'vulkan'): raise BuildError(f'game profile {g.id}: graphics must be nvn or vulkan: {gfx}')
|
||||
if gfx == 'vulkan': say('Vulkan game: vk_shaders.py translates the shaders instead of the NVN shader chain')
|
||||
builds, need = g.builds(), g.words('shader_versions')
|
||||
pkg_ver = args.package or PACKAGE or g.get('package')
|
||||
if pkg_ver not in builds: raise BuildError(f"the version to pack (-p / PACKAGE) must be one the game profile registers: {', '.join(builds)}")
|
||||
@@ -1198,11 +1197,8 @@ def main(args=None):
|
||||
ex = work / 'extract'; ex.mkdir(parents=True, exist_ok=True); (work / 'tmp').mkdir(exist_ok=True)
|
||||
gb = sum({p.stat().st_size for b, u in picks.values() for p in (b, u) if p}) / 2**30 + 6 # extraction + shader cache + slack
|
||||
if free_gb(work) < gb and not all((ex / v).exists() for v in versions): raise BuildError(f'less than {gb:.0f} GB free on the volume of {work}')
|
||||
glslc = gfx == 'nvn' and g.get('shader_glslc_capture', 'off') == 'on' # shaders the game compiles at run time, captured in the Mac host (capture_glslc)
|
||||
vkcap = gfx == 'vulkan' and g.get('shader_vk_capture', 'off') == 'on' # SPIR-V shaders the game generates at run time, captured in the Mac host (capture_vk)
|
||||
uedec = gfx == 'nvn' and g.get('shader_ue_decode', 'off') == 'on' # Unreal Engine shaders, Oodle-compressed in the paks' shader code archives, decoded in the Mac host (decode_ue_shaders)
|
||||
heapck = provisional and HEAP_CHECK and not args.no_heap_check and not heap_checked(g) # the heap check, when the game needs one (once per provisional profile: it stamps the profile)
|
||||
PROG.plan(1 + len(versions) + bool(fam['dlc']) + glslc + vkcap + uedec + (5 if gfx == 'nvn' else 4)) # nx_extract, each version's extraction, the DLC, rewrite, the GLSLC / Vulkan capture or the UE shader decode, shader split + iOS shaders (or the Vulkan shaders), app build, IPA pack
|
||||
PROG.plan(1 + len(versions) + bool(fam['dlc']) + 4) # nx_extract, each version's extraction, the DLC, rewrite, the Vulkan shaders (or the NVN shader split; iOS shaders and the run-time steps added below), app build, IPA pack
|
||||
PROG.stage('build nx_extract')
|
||||
nx = build_nx_extract(repo, work, logs, env)
|
||||
PROG.end()
|
||||
@@ -1212,6 +1208,16 @@ def main(args=None):
|
||||
dirs[ver] = extract(nx, keys, ex, ver, b, u, clone, logs, g)
|
||||
if u is None and clone is None and can_clone(ex): clone = dirs[ver] / 'RomFS'
|
||||
game = dirs[pkg_ver]
|
||||
# how the shaders are got follows from the packed version's files (profile_guess.py shader_sources): the graphics API, and the shaders made only at run time
|
||||
src = pg.build_sources(game / 'ExeFS', game / 'RomFS', g.words('modules'))
|
||||
gfx = src['graphics'][0] # nvn, or vulkan (Hades: the host's Vulkan layer, SPIR-V shaders)
|
||||
glslc = gfx == 'nvn' and src['glslc_capture'][0] # shaders the game compiles at run time, captured in the Mac host (capture_glslc)
|
||||
vkcap = gfx == 'vulkan' and src['vk_capture'][0] # SPIR-V shaders the game generates at run time, captured in the Mac host (capture_vk)
|
||||
uedec = gfx == 'nvn' and src['ue_decode'][0] # Unreal Engine shaders, Oodle-compressed in the paks' shader code archives, decoded in the Mac host (decode_ue_shaders)
|
||||
if gfx == 'vulkan': say(f'Vulkan game ({src["graphics"][1]}): vk_shaders.py translates the shaders instead of the NVN shader chain')
|
||||
for on, k in ((glslc, 'glslc_capture'), (vkcap, 'vk_capture'), (uedec, 'ue_decode'), (gfx == 'nvn' and src['oodle_packs'][0], 'oodle_packs')):
|
||||
if on: say(src[k][1])
|
||||
PROG.add(glslc + vkcap + uedec + (gfx == 'nvn'))
|
||||
aoc = extract_aoc(nx, keys, work, app, fam, logs)
|
||||
base = next((dirs[v] for v in versions if picks[v][1] is None), dirs[versions[0]]) # the folder the base game was extracted to alone
|
||||
# the app name and bundle ID both come from the base NSP: the game name in its control data, and a bundle ID from its title ID; a setting / the command line overrides them
|
||||
@@ -1236,14 +1242,14 @@ def main(args=None):
|
||||
cap = capture_vk(repo, game, patched, work, env, logs, g) if vkcap else None
|
||||
ios = build_vk_shaders(py, repo, [game / 'RomFS'] + ([cap] if cap else []), sh, env, logs, g, reuse=not (provisional or ALLOW_DAMAGED))
|
||||
else:
|
||||
# Trinity packs (shader_oodle): the split decodes them with the packed version's own Oodle decoder, in the Mac host built for the profile
|
||||
oodle = (lambda: {'DM_OODLE_HOST': str(build_mac_host(repo, work / 'tools/dmhost', env, logs, g)), 'DM_OODLE_MODULES': str(patched)}) if g.get('shader_oodle', 'off') == 'on' else None
|
||||
# Trinity packs: the split decodes them with the packed version's own Oodle decoder, in the Mac host built for the profile
|
||||
oodle = (lambda: {'DM_OODLE_HOST': str(build_mac_host(repo, work / 'tools/dmhost', env, logs, g)), 'DM_OODLE_MODULES': str(patched)}) if src['oodle_packs'][0] else None
|
||||
skey = split_shaders(py, repo, [dirs[v] / 'RomFS' for v in need], [dirs[v] / 'ExeFS/main' for v in need], sh / 'pkg', env, logs, g, provisional, oodle)
|
||||
ios = build_shaders(py, repo, sh, xlate, env, logs, g, provisional, skey)
|
||||
|
||||
if free_gb(out_ipa.parent) < 5: raise BuildError(f'less than 5 GB free on the volume of {out_ipa.parent}')
|
||||
say(f'building the IPA → {out_ipa}')
|
||||
ipa_env = dict(env, GAME=g.id, W=str(work / 'ipa'), PATCHED=str(patched), ROMFS=str(game / 'RomFS'), MSL_IOS=str(ios), FONT_PY=py, DM_DISPLAY_NAME=app_name, DM_BUNDLE_ID=bundle_id, AOC=str(aoc or ''))
|
||||
ipa_env = dict(env, GAME=g.id, W=str(work / 'ipa'), PATCHED=str(patched), ROMFS=str(game / 'RomFS'), MSL_IOS=str(ios), GFXAPI=gfx, FONT_PY=py, DM_DISPLAY_NAME=app_name, DM_BUNDLE_ID=bundle_id, AOC=str(aoc or ''))
|
||||
if app_version: ipa_env['DM_APP_VERSION'] = app_version
|
||||
# app icon: the game icon nx_extract saved from the control data (English, icon.jpg); the packed version's first, then the base game's
|
||||
icon = next((d / 'icon.jpg' for d in (game, base) if (d / 'icon.jpg').exists()), None)
|
||||
|
||||
Reference in new issue
Block a user