Monster Hunter Rise: plays on the Mac into the opening; translator phi fix, all chains rebuilt
- Profile 0100B04011742000 (update 16.0.2, RE Engine): KPKA PAK shaders (shader_split.py kpka_jobs), XCI base game in nx_extract and the NSP script, BIG_TEXT patch area in patch_nso.py. - Sparse virtual pool: a 4 KB page table (vpt_*) that the replay reads textures through, dirty tracking per mapped page, texture views from the first mapped level. - MultiDraw*IndirectCount with a CPU wait for the count; pending GPU writes indexed by page; bindless handles read from queued inline updates. - sbuf_ubo binding kind, integer vertex formats into float inputs, clears on textures that are not render targets. - Inline keyboard text, state and finished callback; ngc::ProfanityFilter host vtable. - Ryujinx BindlessElimination follows phi webs (218 MHR pixel shaders); dmsh_xlate_bin and dmsh_chain.lock updated; dmsh_chain.h and c1_full.tsv rebuilt for every Maxwell profile. - README: Monster Hunter Rise under being ported, gallery image, XCI base games, Ryujinx local fixes.
This commit is contained in:
1 parent
6563bb1990
commit
8d7f957fa2
65 files changed
+182756
-192
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Notes for agents
|
||||
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Twelve games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`) and Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`, profile key `graphics vulkan`). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (profile key `shader_glslc_capture`). The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) is being ported: it runs on an iPhone into the opening cave (5 GB guest layout); its frame rate is still being worked on. Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into `main`), renders with Vulkan, and generates its shaders at run time, which the NSP script captures on the Mac (profile key `shader_vk_capture`); its pointer is the touch screen in handheld mode (profile key `op_mode`). [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and per-game notes.
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Twelve games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`) and Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`, profile key `graphics vulkan`). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (profile key `shader_glslc_capture`). The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) is being ported: it runs on an iPhone into the opening cave (5 GB guest layout); its frame rate is still being worked on. Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`) is being ported too: RE Engine, files in KPKA PAK archives (`shader_split.py` `kpka_jobs`, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (`hle_nvn.c` `vpool_*`, a 4 KB page table the replay reads textures through); on the Mac it plays from character creation into the opening in Kamura village. Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into `main`), renders with Vulkan, and generates its shaders at run time, which the NSP script captures on the Mac (profile key `shader_vk_capture`); its pointer is the touch screen in handheld mode (profile key `op_mode`). [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and per-game notes.
|
||||
- Each game's constants and shader tables form a game profile in `dm/games/<id>/`, where `<id>` is the base game's title ID: `game.tsv` plus the tables and `dmsh_chain.h`. Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)).
|
||||
- Per-build knowledge is keyed by the NSO build ID: rewriter cuts in `dm/tools/patch_nso.py`, the profile's `build` rows (the builds the NSP script accepts, and their display versions), and host behaviour that uses fixed module addresses checks `module_t.build_id` (from `fixups.tsv`; `dm_main_build()`, `DM_D2R_BUILD` for code inherited from D2R). When a new game build appears, carry addresses over with `dm/tools/nso_fnmap.py` and check them by disassembly; never let fixed addresses run on a build they were not found in.
|
||||
- To analyse or port **another** Switch title, first run `scripts/build_ipa_from_nsp.py` on its NSP folder. With no matching profile, it writes a survey to `<NSP folder>/<title ID>_survey/survey.md`, then a provisional profile guessed from the game files (`dm/games/profile_guess.py`; `<NSP folder>/<title ID>_build/profile/<title ID>/`, found by the tools through `DM_GAMES_DIR`), with a short heap check on the Mac when the game has virtual address memory, and a best-effort IPA that may not boot. Then follow [docs/agents](docs/agents/README.md) (extraction, binary survey, host bring-up failure catalogue, graphics specifics, debugging recipes) and add a profile.
|
||||
|
||||
@@ -9,7 +9,7 @@ It is not an emulator: the game's own code runs directly on the Apple chip, with
|
||||
This project is built on other people's work:
|
||||
|
||||
- **[d2r-mobile](https://github.com/jhaizhou-ops/d2r-mobile)** (MIT) developed this native-execution approach, for Diablo II: Resurrected. nxnative (first called smbw-mobile) started as a copy of its generic parts: the NSO rewriter, the host that stands in for Horizon, the NVN-on-Metal layer and the iOS app shell. Fixes go both ways.
|
||||
- **Ryujinx** (MIT): its shader translator (`Ryujinx.Graphics.Shader`, `Ryujinx.Common` and `Spv.Generator` 1.3.2, unmodified in `dm/third_party/ryujinx`) turns the games' Maxwell shaders into SPIR-V. Its emulation of the kernel and system services is also a reference when the host's behaviour is in doubt.
|
||||
- **Ryujinx** (MIT): its shader translator (`Ryujinx.Graphics.Shader`, `Ryujinx.Common` and `Spv.Generator` 1.3.2, in `dm/third_party/ryujinx` with a few fixes listed in [its README](dm/third_party/ryujinx/README.md)) turns the games' Maxwell shaders into SPIR-V. Its emulation of the kernel and system services is also a reference when the host's behaviour is in doubt.
|
||||
- **[Atmosphère](https://github.com/Atmosphere-NX/Atmosphere)**: its kernel, mesosphère, is the reference for Horizon's synchronization semantics (address arbiter, condition variables).
|
||||
- **yuzu**: Maxwell instruction encodings for the shader reference tool, and a cross-check for kernel behaviour.
|
||||
- **[LibHac](https://github.com/Thealexbarney/LibHac)**: reads NSPs and layers updates over the base game in `dm/tools/nx_extract`. **hactool** did the first extractions.
|
||||
@@ -40,6 +40,7 @@ All of these are tested on an iPhone and playable:
|
||||
Being ported:
|
||||
|
||||
- The Legend of Zelda: Tears of the Kingdom (base game + update 1.4.3): runs on an iPhone into the opening cave; the frame rate is still being worked on.
|
||||
- Monster Hunter Rise (base game + update 16.0.2): plays on the Mac from character creation into the opening in Kamura village; not yet tested on an iPhone.
|
||||
|
||||
| | |
|
||||
|:---:|:---:|
|
||||
@@ -49,7 +50,7 @@ Being ported:
|
||||
| <br>The Legend of Zelda: Link's Awakening | <br>Super Mario Odyssey |
|
||||
| <br>Hades | <br>Hyrule Warriors: Age of Calamity |
|
||||
| <br>Fire Emblem: Three Houses | <br>Super Mario Galaxy |
|
||||
| <br>The Legend of Zelda: Tears of the Kingdom | |
|
||||
| <br>The Legend of Zelda: Tears of the Kingdom | <br>Monster Hunter Rise |
|
||||
|
||||
Needs iOS / iPadOS 26 or later on an iPhone 15 Pro or newer (A17 Pro chip or later) or an iPad with an M1 chip or later: the games' textures are BC-compressed, and older iPhones can't read that format. Other games aren't supported yet. For them the script writes a survey and still tries its best to pack an IPA, guessing the game's settings; such an IPA may not start.
|
||||
|
||||
@@ -64,7 +65,7 @@ brew install --cask dotnet-sdk
|
||||
python3 -m pip install lz4 capstone fonttools
|
||||
```
|
||||
|
||||
**2. Put one game's NSPs in a folder**, base game and update (and any DLC):
|
||||
**2. Put one game's NSPs in a folder**, base game (an NSP, or an XCI cartridge dump) and update (and any DLC):
|
||||
|
||||
```
|
||||
~/Games/MarioWonder/
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
# chain b-9ca75fb4a525 2026-10-19T15:53:20+08:00
|
||||
# chain b-f53065c05507 2026-10-20T06:37:59+08:00
|
||||
6405f3e55facc7c9
|
||||
d9097b8aba3a7080
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100000000010000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "9ca75fb4a525"
|
||||
#define DMSH_CHAIN "f53065c05507"
|
||||
@@ -1,4 +1,4 @@
|
||||
# chain b-d740cde14039 2026-10-19T15:33:24+08:00
|
||||
# chain b-48e79badc676 2026-10-20T06:18:38+08:00
|
||||
35cbc686dd8461b8
|
||||
4872bec9a5567f8b
|
||||
54a618792b2d5f0f
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010015100B514000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "d740cde14039"
|
||||
#define DMSH_CHAIN "48e79badc676"
|
||||
@@ -1,3 +1,3 @@
|
||||
# chain b-5b81a2e21261 2026-10-19T15:40:49+08:00
|
||||
# chain b-aa62503b91c1 2026-10-20T06:23:06+08:00
|
||||
25c5829f891f9b2c
|
||||
976839d3ba92c8f3
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010028600EBDA000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "5b81a2e21261"
|
||||
#define DMSH_CHAIN "aa62503b91c1"
|
||||
@@ -1,4 +1,4 @@
|
||||
# chain b-613639403635 2026-10-19T16:30:28+08:00
|
||||
# chain b-cc188a5e9879 2026-10-20T07:05:13+08:00
|
||||
0002d193f6c52811
|
||||
0010e810186cdc91
|
||||
001299ca51e856c2
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01002B00111A2000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "613639403635"
|
||||
#define DMSH_CHAIN "cc188a5e9879"
|
||||
@@ -1 +1 @@
|
||||
# chain b-286d88dfacfa 2026-10-19T15:29:08+08:00
|
||||
# chain b-2c262d3399d1 2026-10-20T06:20:26+08:00
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01003FC01670C000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "286d88dfacfa"
|
||||
#define DMSH_CHAIN "2c262d3399d1"
|
||||
@@ -1,4 +1,4 @@
|
||||
# chain b-b215c66de31d 2026-10-19T15:41:17+08:00
|
||||
# chain b-b6dcbdf4b4ba 2026-10-20T07:05:43+08:00
|
||||
1a2b65cd05ff557c
|
||||
21c7ff0d9591a637
|
||||
33e4340e5bcbc5c6
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010055D009F78000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "b215c66de31d"
|
||||
#define DMSH_CHAIN "b6dcbdf4b4ba"
|
||||
@@ -1,4 +1,4 @@
|
||||
# chain b-e58ec56c1d38 2026-10-19T15:36:36+08:00
|
||||
# chain b-75a11052925e 2026-10-20T06:31:05+08:00
|
||||
26c5fd7adbc12cc0
|
||||
67f23f4ad5925e06
|
||||
96a3e288731b2433
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01006BB00C6F0000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "e58ec56c1d38"
|
||||
#define DMSH_CHAIN "75a11052925e"
|
||||
@@ -1,3 +1,3 @@
|
||||
# chain b-72b3f2ce8dd0 2026-10-19T15:46:26+08:00
|
||||
# chain b-859b358c84a7 2026-10-20T06:28:11+08:00
|
||||
21fe759b8114941c
|
||||
60e2041ed375cae1
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 01007EF00011E000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "72b3f2ce8dd0"
|
||||
#define DMSH_CHAIN "859b358c84a7"
|
||||
@@ -1,3 +1,3 @@
|
||||
# chain b-36f66cb1ca43 2026-10-19T15:28:59+08:00
|
||||
# chain b-eb6b1782c78e 2026-10-20T06:20:18+08:00
|
||||
57eb9f0bae677756
|
||||
f826f2b59cfa39a8
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 010093801237C000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "36f66cb1ca43"
|
||||
#define DMSH_CHAIN "eb6b1782c78e"
|
||||
@@ -1,4 +1,4 @@
|
||||
# chain b-af88a8c9e8e6 2026-10-19T15:38:08+08:00
|
||||
# chain b-6a37b6ab3705 2026-10-20T06:20:07+08:00
|
||||
001275671d2c9985
|
||||
0021d75a567e2122
|
||||
00a98da1158c6f14
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100A6301214E000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "af88a8c9e8e6"
|
||||
#define DMSH_CHAIN "6a37b6ab3705"
|
||||
@@ -0,0 +1 @@
|
||||
# generated
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100B04011742000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "94b2f6efaab8"
|
||||
@@ -0,0 +1,62 @@
|
||||
# Game profile of Monster Hunter Rise (Switch, base game cartridge dump v0 + update 16.0.2 = v2228224, Sunbreak included)
|
||||
# Format: see dm/games/README.md
|
||||
# Capcom's RE Engine on SDK 11.4.3 (the same sdk / subsdk0 builds as Metroid Dread and 13 Sentinels), NVN. Everything lives in RE Engine PAK archives (re_chunk_000.pak + 16 patches, 19.6 GB, KPKA v4,
|
||||
# zstd / deflate entries); the shaders are its SDF definitions inside them (shader_split.py kpka_jobs / sdf_shaders: 63,111 shaders, 102,345 vertex / pixel pairs). main is 314 MB (125 MB of text),
|
||||
# so the rewriter puts its patch area right before the base (patch_nso.py BIG_TEXT). The GPU heap is one 64 GB sparse virtual memory pool (hle_nvn.c vpool_*)
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 0100B04011742000
|
||||
program_id 0100B04011742000
|
||||
|
||||
# ---- App ----
|
||||
# Game name: only a label for profile listings and the Mac window title; the iOS app name is the game name in the base game's control data, and the bundle ID comes from the base game title ID
|
||||
name Monster Hunter Rise
|
||||
ipa MonsterHunterRise.ipa
|
||||
app_category public.app-category.action-games
|
||||
# a 3D action game: camera on the right stick, item bar on the D-pad
|
||||
pad_layout full
|
||||
|
||||
# ---- Versions ----
|
||||
# only the merged 16.0.2 copy is extracted
|
||||
build v2228224 16.0.2 main=c9a3dd7702075ecd90101f4555389582 subsdk0=507f19d1240d09d00992248e8a0fd77e sdk=2d369d9999e3e10e665106f840ffff9b
|
||||
package v2228224
|
||||
shader_versions v2228224
|
||||
|
||||
# ---- Files ----
|
||||
# load order as on Horizon: main, subsdk*, sdk (subsdk0 = NintendoSDK_movie 11.4.3: the opening movies are H.264 into an NVN texture with AAC 5.1 sound)
|
||||
modules main subsdk0 sdk
|
||||
romfs_marker re_chunk_000.pak.patch_001.pak
|
||||
zip_store .pak:.metallib
|
||||
|
||||
# ---- Graphics ----
|
||||
graphics nvn
|
||||
# RE Engine fills small textures from the CPU in place (the menu cursor's 8 × 8 BC1 texture: 32 bytes written into a CPU-visible pool, no packaged layout, no WriteTexels), so they must be uploaded
|
||||
# from guest memory (without these keys the cursor highlight composited black on black and the menus looked frozen)
|
||||
tex_flush_upload on
|
||||
tex_dirty_upload on
|
||||
|
||||
# ---- Memory ----
|
||||
# The NPDM declares no system resource, so there is no virtual address memory: the game sizes its heap as pool − used memory with SetHeapSize. 4 GB block: 256 MB alias, 256 MB stacks, 3.25 GB heap region
|
||||
mem_reserve 0x100000000
|
||||
mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
# SetHeapSize asked for the pool − 64 MB − bss (0xc9000000) with the default, but the loaded modules take 0x13fa4000 (main alone 314 MB of text + ro + data), so the request failed at boot:
|
||||
# used memory must cover them, as on hardware. With 328 MB the heap is 2.88 GB
|
||||
mem_used 0x14800000
|
||||
|
||||
# ---- NVN driver constant segments ----
|
||||
# Two texture handle tables, both in use (a census of the translated shaders' texture names): bound / combined handles at c2 0x20 + 8 × slot (nvnCommandBufferBindTexture; fp_t_tcb_8 … _1E),
|
||||
# separate texture binds at 0x168 + 8 × slot (fp_t_tcb_5A, fp_t_cb196610_1017815A), separate samplers at 0x568; many shaders take a constant sampler ID instead (handle type 3, fp_t_cb65538_3041005C)
|
||||
nvn_tex_base 0x20
|
||||
nvn_tex_slots 32
|
||||
nvn_septex_base 0x168
|
||||
nvn_septex_slots 128
|
||||
nvn_samp_base 0x568
|
||||
nvn_img_base 0x120
|
||||
nvn_img_slots 8
|
||||
nvn_ssbo_bases vs=0x110 ps=0x510 cs=0x310
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_groups re_chunk=sdf
|
||||
shader_note Monster Hunter Rise shaders
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# Prewarm seed list (shipped in the app, next to shader_map.tsv). Rows: S<TAB>shader key<TAB>stage / R<TAB>render pipeline key (pso_skey hex) / C<TAB>compute shader key.
|
||||
# Not recorded yet for this game (DM_PSO_RECORD): an empty list = no prewarm. Row order = first-use order; do not sort.
|
||||
|
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
# kind: evidence control-section key conflict rulings: empty
|
||||
dm_ctl_sha16 chosen_pc class candidates records
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -1,4 +1,4 @@
|
||||
# chain b-d99146495c62 2026-10-19T16:03:32+08:00
|
||||
# chain b-e7245308050d 2026-10-20T07:16:18+08:00
|
||||
35cbc686dd8461b8
|
||||
4872bec9a5567f8b
|
||||
759c14c6d029f531
|
||||
|
||||
|
@@ -1,2 +1,2 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header, game 0100F2C0115B6000): the B shader library translation chain version this source tree expects (default options); the host loads only <library dir>/b-<V>/
|
||||
#define DMSH_CHAIN "d99146495c62"
|
||||
#define DMSH_CHAIN "e7245308050d"
|
||||
@@ -53,6 +53,7 @@ None of them has a default game: without one they stop and list the profiles. To
|
||||
| `tex_dirty_upload` | `on` / `off` (the default), needs `tex_flush_upload`: a registered texture whose pages the CPU writes after `nvnTextureInitialize` without any NVN flush is uploaded as well (its pages are write-protected at registration, checked on first use; `nvn_ft_written` in `hle_nvn.c`). Tears of the Kingdom fills its ASTC terrain material arrays this way; without it the cave is magenta |
|
||||
| `gpu_timestamps` | What a timestamp report (`nvnCommandBufferReportCounter`) records: `submission` (the default) = the time the whole submission completed, the same for every report in it; `segment` = the GPU start (`TIMESTAMP_TOP`) or end (the other types) of the segment the report was encoded in (the host commits a submission in segments of 64 draws, `sub_cut` in `nvn_mtl.m`), so two reports of one submission bracket the GPU time between them, as on hardware. Breath of the Wild needs `segment`: its frame-interval controller and its dynamic resolution read the GPU time between the frame's first and last report, and with one time for both they read 0 (the game kept 60 fps in slow motion when the GPU could not keep up). `DM_NVN_GPU_TS=segment` / `submission` overrides it, also from `Documents/env.txt` |
|
||||
| `nvn_tex_base`, `nvn_tex_slots`, `nvn_samp_base`, `nvn_img_base`, `nvn_img_slots`, `nvn_ssbo_bases` | NVN driver constant segments, measured over all shaders |
|
||||
| `nvn_septex_base`, `nvn_septex_slots` | Optional: a separate texture handle table of its own, for a game that binds both kinds of handle. On hardware `nvnCommandBufferBindTexture` and `nvnCommandBufferBindSeparateTexture(s)` fill two tables of the driver segment; without these keys the host keeps both kinds in one (Mario binds only combined handles, Engage only separate ones). Monster Hunter Rise binds both, slot 0 of each in the same frames: combined handles at `nvn_tex_base` 0x20, separate textures at 0x168. With the keys, separate binds go to slots 128 + n of the replay's table (`SEPTEX_SLOT0` in `hle_nvn.c`), `dmsh_build.py` names that table's handles 128 + n, and the replay fills both tables of c2 |
|
||||
| `shader_groups`, `shader_note` | `shader_split.py` source groups and the map header note (Engage: `engine` for `data.unity3d` and `Resources/`, `assets` for the Addressables bundles) |
|
||||
| `shader_main_glslc` | `on` / `off` (the default): `shader_split.py` also reads GLSLC compiler outputs (magic `0x19866891`) in `main`. 13 Sentinels keeps all its shaders there; Engage's `main` has 11 that it does not use, so the key stays off for other games |
|
||||
| `shader_separable` | `on` / `off` (the default): the game builds single-stage programs and combines them per draw, so the real vertex / pixel pairs exist only at run time. `shader_split.py` then pairs every vertex shader with a cover of pixel shaders that together read every input any pixel shader reads, and `dmsh_build.py`'s `vs_pad` makes every vertex shader link with any pixel shader (13 Sentinels) |
|
||||
|
||||
@@ -136,6 +136,7 @@ def header(g):
|
||||
f'#define DM_GAME_NVN_SAMP_BASE {g.int("nvn_samp_base"):#x}',
|
||||
f'#define DM_GAME_NVN_IMG_BASE {g.int("nvn_img_base"):#x}',
|
||||
f'#define DM_GAME_NVN_IMG_SLOTS {g.int("nvn_img_slots")}',
|
||||
*([f'#define DM_GAME_NVN_SEPTEX_BASE {g.int("nvn_septex_base"):#x}', f'#define DM_GAME_NVN_SEPTEX_SLOTS {g.int("nvn_septex_slots")}'] if g.get('nvn_septex_base', '') else []), # a separate texture table of its own (Monster Hunter Rise)
|
||||
f'#define DM_GAME_DISPLAY_VERSIONS {vers}',
|
||||
f'#define DM_GAME_PRESENT_DISPLAY {int(pi == "display")}',
|
||||
f'#define DM_GAME_CORE_PREEMPT {int(cp == "on")}',
|
||||
|
||||
+1
-1
@@ -112,7 +112,7 @@ The shader libraries are generated by `setup.sh`:
|
||||
|
||||
After changing a file on the translation chain, the order for rebuilding manually is the same as in `setup.sh`: `dmsh_chain.py --header` → `dmsh_build.py` → `c1_guard.py` → recompile the host.
|
||||
|
||||
Known gap, to fix with the next chain change (the fix edits `dmsh_build.py`, which changes every game's chain version): the per-shader cache key in `dmsh_build.py` (`shader_key`) leaves out `shader_pairs.tsv`, the profile's `nvn_*` / `shader_*` rows and the inputs of a vertex shader's paired pixel shaders (`vs_pad`). The chain version does include them, so after changing only those the build hits the old per-shader cache entries and files them under the new version. Until the fix, rebuild after such a change with `DMSH_CAS` set to an empty folder (the MSL-keyed metallib cache in it is refilled from the earlier `b-*/` folders beside the output, so compiles are still reused). `$DM_ROOT` defaults to `~/dm-work`.
|
||||
The per-shader cache key in `dmsh_build.py` (`shader_key`) covers the profile's `nvn_*` / `shader_*` rows and the shader's pairs, and a paired vertex shader's key also takes its pixel shaders' keys (`pair_keyed`), since `vs_pad` reads their MSL. Before, these were only in the chain version, so a build after changing only them hit the old per-shader entries and filed them under the new version. `$DM_ROOT` defaults to `~/dm-work`.
|
||||
|
||||
## Tests
|
||||
|
||||
|
||||
+192
-6
@@ -22,6 +22,8 @@
|
||||
#include <mach/mach.h>
|
||||
#include <TargetConditionals.h>
|
||||
#include <unistd.h>
|
||||
#include <sys/mman.h>
|
||||
#include <errno.h>
|
||||
#if TARGET_OS_OSX
|
||||
#define DM_MEM_CAP_DEFAULT_GB 24
|
||||
#else
|
||||
@@ -164,16 +166,19 @@ NVN(nvnMemoryPoolBuilderSetDefaults, void, NVNmemoryPoolBuilder *b) { memset(b,
|
||||
NVN(nvnMemoryPoolBuilderSetDevice, void, NVNmemoryPoolBuilder *b, NVNdevice *d) { ((pool_builder_t *)b)->dev = d; }
|
||||
NVN(nvnMemoryPoolBuilderSetStorage, void, NVNmemoryPoolBuilder *b, void *m, size_t n) { ((pool_builder_t *)b)->mem = m; ((pool_builder_t *)b)->size = n; }
|
||||
NVN(nvnMemoryPoolBuilderSetFlags, void, NVNmemoryPoolBuilder *b, int f) { ((pool_builder_t *)b)->flags = f; }
|
||||
static uint8_t *vpool_reserve(const pool_t *o); static void vpool_release(const pool_t *o); // sparse virtual pools, see MapVirtual
|
||||
NVN(nvnMemoryPoolInitialize, NVNboolean, NVNmemoryPool *p, const NVNmemoryPoolBuilder *b) {
|
||||
const pool_builder_t *s = (const pool_builder_t *)b; pool_t *o = (pool_t *)p;
|
||||
o->magic = MAGIC('P', 'O', 'O', 'L'); o->flags = s->flags; o->mem = s->mem; o->size = s->size;
|
||||
if (o->mem) { mtl_pool_add(o->mem, o->size); nvn_dirty_add_range(o->mem, o->size); }
|
||||
if (!o->mem && (o->flags & 0x200)) { uint8_t *vr = vpool_reserve(o); if (vr) o->mem = vr; } // a sparse virtual pool's own address range (see MapVirtual)
|
||||
else if (o->mem) { mtl_pool_add(o->mem, o->size); nvn_dirty_add_range(o->mem, o->size); }
|
||||
if (!o->mem && !(o->flags & 0x200)) LOG("[RED] MemoryPoolInitialize: non-virtual pool has no storage flags=%#x size=%#llx", o->flags, o->size);
|
||||
if (getenv("DM_NVN_VLOG")) { LOG("[nvn] MemoryPool %p mem=%p size=%#llx flags=%#x", (void *)p, (void *)o->mem, (unsigned long long)o->size, o->flags); void dm_backtrace(uint64_t fp); dm_backtrace((uint64_t)__builtin_frame_address(0)); }
|
||||
OLOG("MemoryPool %p mem=%p size=%#llx flags=%#x", (void *)p, (void *)o->mem, o->size, o->flags);
|
||||
return 1;
|
||||
}
|
||||
NVN(nvnMemoryPoolFinalize, void, NVNmemoryPool *p) { if (((pool_t *)p)->mem && !(((pool_t *)p)->flags & 0x200)) { // A virtual pool's storage is borrowed from a physical pool; do not unregister it
|
||||
NVN(nvnMemoryPoolFinalize, void, NVNmemoryPool *p) { if (((pool_t *)p)->flags & 0x200) vpool_release((pool_t *)p);
|
||||
if (((pool_t *)p)->mem && !(((pool_t *)p)->flags & 0x200)) { // A virtual pool's storage is borrowed from a physical pool; do not unregister it
|
||||
nvn_dirty_remove_range(((pool_t *)p)->mem); mtl_pool_del(((pool_t *)p)->mem); } mtl_alias_drop_pool(p); ((pool_t *)p)->magic = 0; }
|
||||
static void ft_flush(const uint8_t *a, uint64_t n);
|
||||
NVN(nvnMemoryPoolFlushMappedRange, void, const NVNmemoryPool *p, ptrdiff_t off, size_t n) { const pool_t *o = (const pool_t *)p; if (o->mem) ft_flush(o->mem + off, n); } // see ft_add
|
||||
@@ -369,6 +374,7 @@ NVN(nvnTextureGetWidth, int, const NVNtexture *t) { return (int)CTX(t)->d.width;
|
||||
NVN(nvnTextureGetHeight, int, const NVNtexture *t) { return (int)CTX(t)->d.height; }
|
||||
NVN(nvnTextureGetDepth, int, const NVNtexture *t) { return (int)CTX(t)->d.depth; }
|
||||
NVN(nvnTextureGetStorageSize, int, const NVNtexture *t) { return (int)CTX(t)->L.size; }
|
||||
NVN(nvnTextureGetStride, ptrdiff_t, const NVNtexture *t) { return (ptrdiff_t)CTX(t)->d.stride; } // the row stride of a linear texture as built (0 for block-linear ones)
|
||||
// The queries below are used by Engage (Unity) at startup: it calls MemoryPoolMap on "the pool the texture / buffer lives in", and with a stub returning 0 it dereferenced a null pool and crashed
|
||||
NVN(nvnTextureGetLevels, int, const NVNtexture *t) { return (int)CTX(t)->d.levels; }
|
||||
NVN(nvnTextureGetSamples, int, const NVNtexture *t) { return CTX(t)->d.samples; }
|
||||
@@ -818,10 +824,17 @@ static p_bind_handles *bind_handles(NVNcommandBuffer *c, int op, int st, int fir
|
||||
NVN(nvnCommandBufferBindImages, void, NVNcommandBuffer *c, NVNshaderStage st, int first, int n, const NVNimageHandle *h) {
|
||||
bind_handles(c, OP_BIND_IMG, st, first, n, (const uint64_t *)h);
|
||||
}
|
||||
// Separate texture binds: on hardware they fill their own handle table (c2 0x168 + 8 slot), apart from BindTexture's. A game that uses both (Monster Hunter Rise: BindTexture slot 0 beside separate slots 0-4
|
||||
// in the same frames) has nvn_septex_base in its profile, and its separate binds go to slots 128 + n of the replay's table, which its shaders' reflection names; other games keep slots 0-127 for both
|
||||
#ifdef DM_GAME_NVN_SEPTEX_BASE
|
||||
#define SEPTEX_SLOT0 128
|
||||
#else
|
||||
#define SEPTEX_SLOT0 0
|
||||
#endif
|
||||
NVN(nvnCommandBufferBindSeparateTextures, void, NVNcommandBuffer *c, NVNshaderStage st, int first, int n, const NVNseparateTextureHandle *h) {
|
||||
bind_handles(c, OP_BIND_TEX, st, first, n, (const uint64_t *)h);
|
||||
bind_handles(c, OP_BIND_TEX, st, SEPTEX_SLOT0 + first, n, (const uint64_t *)h);
|
||||
}
|
||||
NVN(nvnCommandBufferBindSeparateTexture, void, NVNcommandBuffer *c, NVNshaderStage st, int i, NVNseparateTextureHandle h) { uint64_t v; memcpy(&v, &h, 8); bind_handles(c, OP_BIND_TEX, st, i, 1, &v); } // single (Engage): as the plural version
|
||||
NVN(nvnCommandBufferBindSeparateTexture, void, NVNcommandBuffer *c, NVNshaderStage st, int i, NVNseparateTextureHandle h) { uint64_t v; memcpy(&v, &h, 8); bind_handles(c, OP_BIND_TEX, st, SEPTEX_SLOT0 + i, 1, &v); } // single (Engage): as the plural version
|
||||
NVN(nvnCommandBufferBindSeparateSamplers, void, NVNcommandBuffer *c, NVNshaderStage st, int first, int n, const NVNseparateSamplerHandle *h) {
|
||||
bind_handles(c, OP_BIND_SAMP, st, first, n, (const uint64_t *)h);
|
||||
}
|
||||
@@ -935,6 +948,11 @@ NVN(nvnVertexAttribStateSetStreamIndex, void, NVNvertexAttribState *s, int i) {
|
||||
NVN(nvnVertexStreamStateSetDefaults, void, NVNvertexStreamState *s) { *ST(vs_st, s) = (vs_st){0, 0}; }
|
||||
NVN(nvnVertexStreamStateSetStride, void, NVNvertexStreamState *s, ptrdiff_t st) { ST(vs_st, s)->stride = (uint32_t)st; }
|
||||
NVN(nvnVertexStreamStateSetDivisor, void, NVNvertexStreamState *s, int d) { ST(vs_st, s)->divisor = d; }
|
||||
// Getters: Monster Hunter Rise reads its stream states back when it builds a draw (about 24,000 calls in its character creation)
|
||||
NVN(nvnVertexStreamStateGetStride, ptrdiff_t, const NVNvertexStreamState *s) { return (ptrdiff_t)((const vs_st *)s)->stride; }
|
||||
NVN(nvnVertexStreamStateGetDivisor, int, const NVNvertexStreamState *s) { return ((const vs_st *)s)->divisor; }
|
||||
NVN(nvnVertexAttribStateGetFormat, void, const NVNvertexAttribState *s, NVNformat *f, ptrdiff_t *off) { const va_st *o = (const va_st *)s; if (f) *f = (NVNformat)o->fmt; if (off) *off = (ptrdiff_t)o->offset; }
|
||||
NVN(nvnVertexAttribStateGetStreamIndex, int, const NVNvertexAttribState *s) { return ((const va_st *)s)->stream; }
|
||||
|
||||
static _Atomic uint64_t g_sub_wall_ns, g_sub_cpu_ns, g_sub_instr; // Wall clock / this thread's CPU time / this thread's retired instruction count for submit (excluding present) (taken away per frame)
|
||||
static _Atomic uint64_t g_cum_sub_instr, g_cum_sub_cpu_ns; // Cumulative versions of the above (the "translation" summary row every 300 frames takes the difference; on a real device read through the log port: instructions per draw / submit CPU per frame)
|
||||
@@ -1000,8 +1018,6 @@ NVN(nvnQueueSubmitCommands, void, NVNqueue *q, int n, const NVNcommandHandle *h)
|
||||
static void nvn_nop(void) {}
|
||||
static const char *const g_nops[] = {
|
||||
"nvnCommandBufferSetPatchSize",
|
||||
"nvnCommandBufferMultiDrawArraysIndirectCount",
|
||||
"nvnCommandBufferMultiDrawElementsIndirectCount",
|
||||
"nvnCommandBufferSetDepthBounds",
|
||||
"nvnCommandBufferWaitSync",
|
||||
"nvnCommandBufferSetShaderScratchMemory",
|
||||
@@ -1513,8 +1529,109 @@ NVN(nvnProgramSetDebugLabel, void, NVNprogram *p, const char *s) { (void)p; (voi
|
||||
// Inconsistent mappings (true sparse) are not implemented: reported red, returns failure. It used to always fail: the game treated the pool as unusable, a buffer later carved from it got a null pool, and MemoryPoolGetFlags read NULL+4 and crashed
|
||||
typedef struct { NVNmemoryPool *phys; int64_t poff, voff; uint64_t size; int32_t storage_class, _pad; } map_req;
|
||||
_Static_assert(sizeof(map_req) == 0x28, "NVNmappingRequest");
|
||||
// True sparse virtual pools (Monster Hunter Rise: RE Engine's GPU heap is one 64 GB virtual pool, built from separately allocated 4 MB physical pools mapped at offsets of their own; boot maps 45 of them
|
||||
// over the first 180 MB). A virtual pool larger than the whole application pool cannot be one alias of a physical pool, so at Initialize it gets an address range of its own (reserved, inaccessible),
|
||||
// and every mapping request vm_remaps the physical pool's pages over its part of that range: the virtual addresses then read and write the same memory as the physical pool, as the GPU page table does
|
||||
// on hardware, and buffer / texture addresses stay CPU addresses. Each run of mapped pages is registered with the replay as a pool of its own (mtl_pool_add), rebuilt when a request joins or splits runs.
|
||||
// Unmapping puts inaccessible pages back. The reservation is the pool's size, at most 64 GB on the Mac and 1 GB on iOS.
|
||||
// NVN maps at 4 KB, the host aliases at its 16 KB page: a request that is not 16 KB aligned (Monster Hunter Rise's ASTC 6x6 textures, whose mip tail starts 0x81000 into the texture),
|
||||
// or one beyond the reservation, cannot be aliased. So every request is also kept in a page table at 4 KB (the physical memory behind each virtual page, and the mapping generation
|
||||
// that set it), and the host reads textures of a sparse pool through it (nvn_sparse_copy): such a texture is uploaded from what the table points at, with unmapped pages read as 0,
|
||||
// and uploaded again only when a mapping in its range changes or the CPU writes a physical page behind it (nvn_sparse_since; the physical pools are dirty-tracked). The game fills
|
||||
// its sparse textures (flag 0x20, 32 GB into the pool) that way: it writes the tiles on the CPU into 4 MB physical pools, then maps them; no GPU copy writes them
|
||||
enum { NVPOOL = 4, NVRUN = 1024, VPT_LEAF = 512 }; // a page-table leaf covers 2 MB
|
||||
typedef struct { const uint8_t *phys; uint64_t gen; } vpte;
|
||||
static struct vpool { const pool_t *pool; uint8_t *base; uint64_t len, size, top; int nrun; struct { uint64_t lo, hi; } run[NVRUN]; vpte **pt; } g_vp[NVPOOL]; // len: reserved; size: the pool's; top: highest mapped end so far (logged per 128 MB)
|
||||
static pthread_mutex_t g_vp_mu = PTHREAD_MUTEX_INITIALIZER;
|
||||
static _Atomic uint64_t g_vp_gen; // bumped per mapping request
|
||||
static struct vpool *vpool_of(const pool_t *o) { for (int i = 0; i < NVPOOL; i++) if (g_vp[i].pool == o && g_vp[i].base) return &g_vp[i]; return NULL; }
|
||||
static uint8_t *vpool_reserve(const pool_t *o) { // → the reserved range, or NULL for a pool that keeps the alias
|
||||
if (!(o->flags & 0x200) || o->size <= DM_GAME_APP_POOL) return NULL;
|
||||
#if TARGET_OS_IPHONE // 1 GB: Monster Hunter Rise's buffers stay under 0x20100000 through its opening, its textures (32 GB in) are read through the page table, and 4 GB guest + 1 GB stays near the 5 GB proven on a phone
|
||||
uint64_t len = o->size < (1ull << 30) ? o->size : 1ull << 30;
|
||||
#else
|
||||
uint64_t len = o->size < (64ull << 30) ? o->size : 64ull << 30;
|
||||
#endif
|
||||
void *a = mmap(NULL, len, PROT_NONE, MAP_ANON | MAP_PRIVATE, -1, 0);
|
||||
if (a == MAP_FAILED) { LOG("[RED] virtual pool %p: cannot reserve %#llx bytes of address space (errno %d)", (const void *)o, (unsigned long long)len, errno); return NULL; }
|
||||
pthread_mutex_lock(&g_vp_mu);
|
||||
int i = 0; while (i < NVPOOL && g_vp[i].base) i++;
|
||||
if (i == NVPOOL) { pthread_mutex_unlock(&g_vp_mu); munmap(a, len); LOG("[RED] more than %d sparse virtual pools", NVPOOL); return NULL; }
|
||||
g_vp[i].pool = o; g_vp[i].base = a; g_vp[i].len = len; g_vp[i].size = o->size; g_vp[i].nrun = 0;
|
||||
if (!(g_vp[i].pt = calloc((o->size + (2u << 20) - 1) >> 21, sizeof *g_vp[i].pt))) die("sparse pool page table allocation failed");
|
||||
pthread_mutex_unlock(&g_vp_mu);
|
||||
LOG("[nvn] virtual pool %p (%#llx bytes, flags %#x): sparse, own address range %p+%#llx, physical pages are remapped into it", (const void *)o, (unsigned long long)o->size, o->flags, a, (unsigned long long)len);
|
||||
return a;
|
||||
}
|
||||
static void vpool_release(const pool_t *o) {
|
||||
pthread_mutex_lock(&g_vp_mu); struct vpool *v = vpool_of(o);
|
||||
if (v) { for (int k = 0; k < v->nrun; k++) mtl_pool_del(v->base + v->run[k].lo); munmap(v->base, v->len);
|
||||
for (uint64_t l = 0; l < (v->size + (2u << 20) - 1) >> 21; l++) free(v->pt[l]);
|
||||
free(v->pt); memset(v, 0, sizeof *v); }
|
||||
pthread_mutex_unlock(&g_vp_mu);
|
||||
}
|
||||
// Mapped runs of a sparse pool: [lo, hi) sorted, disjoint, adjacent runs merged. Caller holds g_vp_mu
|
||||
static void vrun_set(struct vpool *v, uint64_t lo, uint64_t hi, int mapped) {
|
||||
uint64_t nl[NVRUN][2]; int nn = 0; uint64_t mlo = lo, mhi = hi;
|
||||
for (int k = 0; k < v->nrun; k++) {
|
||||
uint64_t a = v->run[k].lo, b = v->run[k].hi;
|
||||
if (b < lo || a > hi || (!mapped && (b == lo || a == hi))) { if (nn < NVRUN) { nl[nn][0] = a; nl[nn][1] = b; nn++; } continue; } // untouched (an unmap leaves adjacent runs alone)
|
||||
mtl_pool_del(v->base + a);
|
||||
if (mapped) { if (a < mlo) mlo = a; if (b > mhi) mhi = b; continue; } // joins the new run
|
||||
if (a < lo && nn < NVRUN) { nl[nn][0] = a; nl[nn][1] = lo; nn++; mtl_pool_add(v->base + a, lo - a); } // what the unmap leaves on either side
|
||||
if (b > hi && nn < NVRUN) { nl[nn][0] = hi; nl[nn][1] = b; nn++; mtl_pool_add(v->base + hi, b - hi); }
|
||||
}
|
||||
if (mapped && nn < NVRUN) { nl[nn][0] = mlo; nl[nn][1] = mhi; nn++; mtl_pool_add(v->base + mlo, mhi - mlo); }
|
||||
if (nn == NVRUN) LOG("[RED] virtual pool %p: more than %d separate mapped runs", (const void *)v->pool, NVRUN);
|
||||
for (int a = 1; a < nn; a++) for (int b = a; b > 0 && nl[b][0] < nl[b - 1][0]; b--) { uint64_t t0 = nl[b][0], t1 = nl[b][1]; nl[b][0] = nl[b - 1][0]; nl[b][1] = nl[b - 1][1]; nl[b - 1][0] = t0; nl[b - 1][1] = t1; }
|
||||
v->nrun = nn; for (int k = 0; k < nn; k++) { v->run[k].lo = nl[k][0]; v->run[k].hi = nl[k][1]; }
|
||||
}
|
||||
// Page table of a sparse pool (see above). Caller holds g_vp_mu
|
||||
static const vpte *vpt_get(const struct vpool *v, uint64_t o) { const vpte *l = v->pt[o >> 21]; return l ? &l[(o >> 12) & (VPT_LEAF - 1)] : NULL; }
|
||||
static void vpt_set(struct vpool *v, uint64_t voff, uint64_t size, const uint8_t *phys) {
|
||||
uint64_t gen = atomic_fetch_add(&g_vp_gen, 1) + 1;
|
||||
for (uint64_t o = 0; o < size; o += 0x1000) {
|
||||
vpte **l = &v->pt[(voff + o) >> 21];
|
||||
if (!*l && !phys) continue; // never mapped: stays unmapped
|
||||
if (!*l && !(*l = calloc(VPT_LEAF, sizeof **l))) die("sparse pool page table allocation failed");
|
||||
(*l)[((voff + o) >> 12) & (VPT_LEAF - 1)] = (vpte){phys ? phys + o : NULL, gen};
|
||||
}
|
||||
}
|
||||
static int vpool_map(struct vpool *v, const map_req *r) { // caller holds g_vp_mu; → 0 on failure
|
||||
uint64_t pg = (uint64_t)getpagesize();
|
||||
if (r->voff < 0 || (uint64_t)r->voff + r->size > v->size) { LOG("[RED] virtual pool %p: request at %#llx+%#llx is beyond the pool's %#llx", (const void *)v->pool, (unsigned long long)r->voff, (unsigned long long)r->size, (unsigned long long)v->size); return 0; }
|
||||
if (((uint64_t)r->voff | r->size | (uint64_t)r->poff) & 0xFFF) { LOG("[RED] virtual pool %p: request %#llx+%#llx (physical offset %#llx) is not 4 KB aligned", (const void *)v->pool, (unsigned long long)r->voff, (unsigned long long)r->size, (unsigned long long)r->poff); return 0; }
|
||||
const pool_t *ph = (const pool_t *)r->phys;
|
||||
if (ph && (!ph->mem || r->poff < 0 || (uint64_t)r->poff + r->size > ph->size)) { LOG("[RED] virtual pool %p: physical pool %p (%#llx bytes, storage %p) cannot supply %#llx+%#llx", (const void *)v->pool, (const void *)ph, (unsigned long long)ph->size, (void *)ph->mem, (unsigned long long)r->poff, (unsigned long long)r->size); return 0; }
|
||||
vpt_set(v, (uint64_t)r->voff, r->size, ph ? ph->mem + r->poff : NULL);
|
||||
vm_address_t to = (vm_address_t)(v->base + r->voff);
|
||||
// Not aliased (see above): only textures read through the page table see it; the GPU cannot address it directly
|
||||
if ((uint64_t)r->voff + r->size > v->len || ((uintptr_t)to | r->size | (uint64_t)r->voff) & (pg - 1) || (ph && (((uintptr_t)ph->mem + (uintptr_t)r->poff) & (pg - 1)))) {
|
||||
static _Atomic uint64_t n; if (!atomic_fetch_add(&n, 1)) LOG("[nvn] virtual pool %p: a request %s (%s %#llx+%#llx) is kept in the page table only: textures read it, the GPU cannot address it (later cases are not logged)",
|
||||
(const void *)v->pool, (uint64_t)r->voff + r->size > v->len ? "beyond the reservation" : "not aligned to the host page", r->phys ? "map" : "unmap", (unsigned long long)r->voff, (unsigned long long)r->size);
|
||||
return 1;
|
||||
}
|
||||
if (!r->phys) { // unmap: inaccessible pages again
|
||||
if (mmap((void *)to, r->size, PROT_NONE, MAP_ANON | MAP_PRIVATE | MAP_FIXED, -1, 0) == MAP_FAILED) { LOG("[RED] virtual pool unmap %#lx+%#llx failed (errno %d)", (unsigned long)to, (unsigned long long)r->size, errno); return 0; }
|
||||
vrun_set(v, (uint64_t)r->voff, (uint64_t)r->voff + r->size, 0); return 1;
|
||||
}
|
||||
uintptr_t from = (uintptr_t)ph->mem + (uintptr_t)r->poff;
|
||||
vm_prot_t cur, max;
|
||||
kern_return_t kr = vm_remap(mach_task_self(), &to, r->size, 0, VM_FLAGS_FIXED | VM_FLAGS_OVERWRITE, mach_task_self(), (vm_address_t)from, FALSE, &cur, &max, VM_INHERIT_NONE);
|
||||
if (kr != KERN_SUCCESS || to != (vm_address_t)(v->base + r->voff)) { LOG("[RED] virtual pool remap %#lx → %p (%#llx bytes) failed: %d", (unsigned long)from, (void *)(v->base + r->voff), (unsigned long long)r->size, kr); return 0; }
|
||||
mprotect((void *)to, r->size, PROT_READ | PROT_WRITE); // the source may be write-protected by the dirty tracker at this moment; the alias itself is plain memory
|
||||
vrun_set(v, (uint64_t)r->voff, (uint64_t)r->voff + r->size, 1);
|
||||
uint64_t end = (uint64_t)r->voff + r->size;
|
||||
if (end >> 27 > v->top >> 27) LOG("[nvn] virtual pool %p: mapped up to %#llx (%d runs)", (const void *)v->pool, (unsigned long long)end, v->nrun);
|
||||
if (end > v->top) v->top = end;
|
||||
return 1;
|
||||
}
|
||||
NVN(nvnMemoryPoolMapVirtual, NVNboolean, NVNmemoryPool *p, int n, const NVNmappingRequest *rq) {
|
||||
pool_t *o = (pool_t *)p; const map_req *r = (const map_req *)rq;
|
||||
pthread_mutex_lock(&g_vp_mu); struct vpool *v = vpool_of(o); int ok = 1;
|
||||
for (int i = 0; v && i < n && ok; i++) ok = vpool_map(v, &r[i]);
|
||||
pthread_mutex_unlock(&g_vp_mu);
|
||||
if (v) return (NVNboolean)ok;
|
||||
for (int i = 0; i < n; i++) {
|
||||
const pool_t *ph = (const pool_t *)r[i].phys;
|
||||
if (!ph) continue; // Unmapping (null physical pool): the host keeps the storage as is
|
||||
@@ -1582,6 +1699,8 @@ NVN(nvnColorStateSetLogicOp, void, NVNcolorState *s, NVNlogicOp op) { ST(color_s
|
||||
|
||||
// Device: combined texture handle = texture id | sampler id << 20 (same encoding as the separate handles)
|
||||
NVN(nvnDeviceGetTextureHandle, NVNtextureHandle, const NVNdevice *d, int tex, int samp) { (void)d; return (NVNtextureHandle)((uint64_t)(uint32_t)tex | (uint64_t)(uint32_t)samp << 20); }
|
||||
// Handle for texel fetches only (Monster Hunter Rise): a fetch reads the texture descriptor and no sampler, so the handle is the texture ID with sampler ID 0
|
||||
NVN(nvnDeviceGetTexelFetchHandle, NVNtextureHandle, const NVNdevice *d, int tex) { (void)d; return (NVNtextureHandle)(uint64_t)(uint32_t)tex; }
|
||||
// Counter timestamps are 19.2 MHz GPU ticks (nvn_mtl.m gpu_ticks: ns × 12 / 625); convert back to ns. They used to be returned as is, about 52 times smaller than GetCurrentTimestampInNanoseconds, so subtracting one from the other gave garbage
|
||||
NVN(nvnDeviceGetTimestampInNanoseconds, uint64_t, const NVNdevice *d, const NVNcounterData *c) { (void)d; return c->timestamp * 625 / 12; }
|
||||
// Current GPU time (ns): the same clock as the counter timestamps -- the guest clock (guest_ns, nvn_mtl.m gpu_ticks)
|
||||
@@ -1639,6 +1758,73 @@ NVN(nvnCommandBufferDrawArraysInstanced, void, NVNcommandBuffer *c, NVNdrawPrimi
|
||||
*REC(c, OP_DRAW, p_draw) = (p_draw){pr, first, n, inst, bi};
|
||||
}
|
||||
// Indirect, as DrawArraysIndirect: the record at ind is read by the GPU (nvn_mtl.m draw)
|
||||
// Indirect-count multi-draws (Monster Hunter Rise's GPU-driven scene: compute culling writes the draw records and a u32 count, then up to max draws are made from them; 1–49 per call,
|
||||
// stride 20, the records and counts in its sparse virtual pool). Metal has no draw count read by the GPU, so each multi-draw becomes max ordinary indirect draws that carry the count's address
|
||||
// and their index; the replay reads the count once the GPU has written it and skips draws past it (nvn_mtl.m mdi_live). Both used to be empty, and the whole scene stayed undrawn
|
||||
enum { MDI_MAX = 1024 };
|
||||
NVN(nvnCommandBufferMultiDrawElementsIndirectCount, void, NVNcommandBuffer *c, NVNdrawPrimitive pr, NVNindexType it, NVNbufferAddress ib, NVNbufferAddress ind, NVNbufferAddress cnt, int max, ptrdiff_t stride) {
|
||||
if (max > MDI_MAX) { ONCE_LOG("[RED] indirect-count multi-draw of up to %d draws: only the first %d are made", max, MDI_MAX); max = MDI_MAX; }
|
||||
for (int i = 0; i < max; i++) *REC(c, OP_DRAW_IDX, p_draw_idx) = (p_draw_idx){pr, it, 0, 0, 0, 1, ib, ind + (uint64_t)i * (uint64_t)stride, cnt, i};
|
||||
}
|
||||
NVN(nvnCommandBufferMultiDrawArraysIndirectCount, void, NVNcommandBuffer *c, NVNdrawPrimitive pr, NVNbufferAddress ind, NVNbufferAddress cnt, int max, ptrdiff_t stride) {
|
||||
if (max > MDI_MAX) { ONCE_LOG("[RED] indirect-count multi-draw of up to %d draws: only the first %d are made", max, MDI_MAX); max = MDI_MAX; }
|
||||
for (int i = 0; i < max; i++) *REC(c, OP_DRAW, p_draw) = (p_draw){pr, 0, 0, 1, 0, ind + (uint64_t)i * (uint64_t)stride, cnt, i};
|
||||
}
|
||||
// Whether the host can read n bytes at a guest address: outside a sparse virtual pool always; inside one only where pages are mapped (nvn_mtl.m reads multi-draw counts there)
|
||||
int nvn_host_readable(uint64_t a, uint64_t n) {
|
||||
int ok = 1; pthread_mutex_lock(&g_vp_mu);
|
||||
for (int k = 0; k < NVPOOL; k++) { struct vpool *v = &g_vp[k]; if (!v->base || a < (uint64_t)(uintptr_t)v->base || a >= (uint64_t)(uintptr_t)v->base + v->len) continue;
|
||||
uint64_t o = a - (uint64_t)(uintptr_t)v->base; ok = 0;
|
||||
for (int r = 0; r < v->nrun; r++) if (o >= v->run[r].lo && o + n <= v->run[r].hi) { ok = 1; break; }
|
||||
break; }
|
||||
pthread_mutex_unlock(&g_vp_mu); return ok;
|
||||
}
|
||||
// Textures of a sparse pool, read through its page table (see the sparse pools above): whether a pool is one, a copy of a texture's bytes (unmapped pages read as 0; → bytes not mapped),
|
||||
// whether its content may have changed since an upload (a mapping in its range newer than gen, or a physical page behind it written after the dirty-page stamp), and the write
|
||||
// protection of the physical pages behind it (taken before an upload, as for any packaged texture)
|
||||
static struct vpool *vpool_at(const void *pool, uint64_t a) {
|
||||
for (int k = 0; k < NVPOOL; k++) { struct vpool *v = &g_vp[k]; if (v->base && v->pool == pool && a >= (uint64_t)(uintptr_t)v->base && a < (uint64_t)(uintptr_t)v->base + v->size) return v; }
|
||||
return NULL;
|
||||
}
|
||||
int nvn_sparse_pool(const void *pool) { int r = 0; for (int k = 0; k < NVPOOL; k++) r |= g_vp[k].base && g_vp[k].pool == pool; return r; } // no lock: a slot is set once at Initialize, cleared at Finalize; called per texture lookup
|
||||
uint64_t nvn_sparse_gen(void) { return atomic_load(&g_vp_gen); }
|
||||
uint64_t nvn_sparse_copy(const void *pool, void *dst, uint64_t a, uint64_t n) {
|
||||
pthread_mutex_lock(&g_vp_mu); struct vpool *v = vpool_at(pool, a); uint64_t miss = 0;
|
||||
if (!v) { pthread_mutex_unlock(&g_vp_mu); memset(dst, 0, n); return n; }
|
||||
for (uint64_t p = 0, o = a - (uint64_t)(uintptr_t)v->base; p < n;) {
|
||||
uint64_t at = o + p, k = 0x1000 - (at & 0xFFF); if (k > n - p) k = n - p;
|
||||
const vpte *e = at < v->size ? vpt_get(v, at) : NULL;
|
||||
if (e && e->phys) memcpy((uint8_t *)dst + p, e->phys + (at & 0xFFF), k); else { memset((uint8_t *)dst + p, 0, k); miss += k; }
|
||||
p += k;
|
||||
}
|
||||
pthread_mutex_unlock(&g_vp_mu); return miss;
|
||||
}
|
||||
// Runs of consecutive physical pages behind [a, a+n): fn on each (nonzero stops the walk and is returned), or 1 at the first page mapped after gen (gen ~0 = do not look)
|
||||
static int vpt_walk(const void *pool, uint64_t a, uint64_t n, uint64_t gen, int (*fn)(const uint8_t *, uint64_t, uint64_t), uint64_t arg) {
|
||||
pthread_mutex_lock(&g_vp_mu); struct vpool *v = vpool_at(pool, a); int r = 0;
|
||||
if (!v) { pthread_mutex_unlock(&g_vp_mu); return 1; }
|
||||
const uint8_t *rs = NULL, *re = NULL; uint64_t o = a - (uint64_t)(uintptr_t)v->base;
|
||||
for (uint64_t at = o & ~0xFFFull; at < o + n && at < v->size && !r; at += 0x1000) {
|
||||
const vpte *e = vpt_get(v, at);
|
||||
if (!e) { at |= (2u << 20) - 1; at -= 0xFFF; continue; } // a leaf never mapped: skip its 2 MB
|
||||
if (gen != ~0ull && e->gen > gen) { r = 1; break; }
|
||||
if (!e->phys) continue;
|
||||
if (rs && e->phys == re) { re += 0x1000; continue; }
|
||||
if (rs) r = fn(rs, (uint64_t)(re - rs), arg);
|
||||
rs = e->phys; re = rs + 0x1000;
|
||||
}
|
||||
if (!r && rs) r = fn(rs, (uint64_t)(re - rs), arg);
|
||||
pthread_mutex_unlock(&g_vp_mu); return r;
|
||||
}
|
||||
static int run_written(const uint8_t *p, uint64_t n, uint64_t stamp) { return nvn_dirty_since(p, n, stamp); }
|
||||
static int run_protect(const uint8_t *p, uint64_t n, uint64_t unused) { (void)unused; nvn_dirty_protect(p, n); return 0; }
|
||||
int nvn_sparse_since(const void *pool, uint64_t a, uint64_t n, uint64_t gen, uint64_t stamp) { return vpt_walk(pool, a, n, gen, run_written, stamp); }
|
||||
int nvn_sparse_mapped(const void *pool, uint64_t a, uint64_t n) { // every page of [a, a+n) has memory behind it
|
||||
pthread_mutex_lock(&g_vp_mu); struct vpool *v = vpool_at(pool, a); int ok = v != NULL;
|
||||
for (uint64_t o = v ? (a - (uint64_t)(uintptr_t)v->base) & ~0xFFFull : 0, end = v ? a - (uint64_t)(uintptr_t)v->base + n : 0; ok && o < end; o += 0x1000) { const vpte *e = o < v->size ? vpt_get(v, o) : NULL; ok = e && e->phys; }
|
||||
pthread_mutex_unlock(&g_vp_mu); return ok;
|
||||
}
|
||||
void nvn_sparse_protect(const void *pool, uint64_t a, uint64_t n) { vpt_walk(pool, a, n, ~0ull, run_protect, 0); }
|
||||
NVN(nvnCommandBufferDrawElementsIndirect, void, NVNcommandBuffer *c, NVNdrawPrimitive pr, NVNindexType it, NVNbufferAddress ib, NVNbufferAddress ind) {
|
||||
*REC(c, OP_DRAW_IDX, p_draw_idx) = (p_draw_idx){pr, it, 0, 0, 0, 1, ib, ind};
|
||||
}
|
||||
|
||||
+214
-79
@@ -24,7 +24,13 @@
|
||||
#include "nvn_cover.h"
|
||||
#include "dmsh_chain.h" // DMSH_CHAIN: the translation chain version of the B shader library expected by this source tree (in the game profile folder, -I from the Makefile)
|
||||
#include "game_profile.h" // game profile constants (NVN driver constant segment bases, profile id)
|
||||
// Texture slot table per stage: 0-127 the slots of the bound / combined handle table (nvnCommandBufferBindTexture; for a profile without nvn_septex_base also the separate texture binds),
|
||||
// 128-255 the separate texture binds of a profile with nvn_septex_base (Monster Hunter Rise binds both kinds, the hardware keeps them in two tables: c2 0x20 and c2 0x168)
|
||||
#define NTEX 256
|
||||
_Static_assert(DM_GAME_NVN_TEX_SLOTS <= 128 && DM_GAME_NVN_IMG_SLOTS <= 32 && DM_GAME_NVN_TEX_BASE + 8 * DM_GAME_NVN_TEX_SLOTS <= 0xA00 && DM_GAME_NVN_IMG_BASE + 8 * DM_GAME_NVN_IMG_SLOTS <= 0xA00 && DM_GAME_NVN_SAMP_BASE + 8 * 128 <= 0xA00, "the profile's NVN handle tables exceed the driver segment buffer (0xA00) or the binding tables");
|
||||
#ifdef DM_GAME_NVN_SEPTEX_BASE
|
||||
_Static_assert(DM_GAME_NVN_SEPTEX_SLOTS <= 128 && DM_GAME_NVN_SEPTEX_BASE + 8 * DM_GAME_NVN_SEPTEX_SLOTS <= 0xA00, "the profile's separate texture table exceeds the driver segment buffer (0xA00) or the slot table");
|
||||
#endif
|
||||
#include "nvn_hmap.h"
|
||||
#include "../../host/dm.h"
|
||||
#import <CommonCrypto/CommonDigest.h>
|
||||
@@ -496,9 +502,45 @@ static vis_list *vis_snapshot(void) {
|
||||
return l;
|
||||
}
|
||||
// Target ranges of buffer writes (copies / inline uniform updates) already encoded in this submission but not yet executed on the GPU: reading these addresses on the CPU side at encode time yields old values (uv_proof avoids them when reading uniforms)
|
||||
#define PEND_MAX 256
|
||||
static uint64_t g_pend_lo[PEND_MAX], g_pend_hi[PEND_MAX]; static int g_npend, g_pend_over;
|
||||
static int pend_hits(uint64_t a, uint64_t n) { if (g_pend_over) return 1; for (int i = 0; i < g_npend; i++) if (a < g_pend_hi[i] && a + n > g_pend_lo[i]) return 1; return 0; }
|
||||
// Inline updates keep where their data sits in the update ring (g_pend_upd: index + 1 into g_upd), so a bindless handle they wrote can be read as the GPU will see it (pend_read).
|
||||
// Monster Hunter Rise queues more than 256 writes in a submission (the old limit, after which every address counted as written). The writes are indexed by 4 KB page (a chain per hash
|
||||
// bucket, newest first); a write over more than PEND_SPAN pages goes to a short list that every lookup scans
|
||||
enum { PEND_MAX = 8192, PEND_HB = 2048, PEND_NODES = 16384, PEND_SPAN = 8, PEND_WIDE = 256 };
|
||||
static uint64_t g_pend_lo[PEND_MAX], g_pend_hi[PEND_MAX]; static int g_npend, g_pend_over; static uint8_t g_pend_kind[PEND_MAX]; static uint16_t g_pend_upd[PEND_MAX]; // kind: 1 inline update, 2 copy, 3 fill
|
||||
static int32_t g_pend_head[PEND_HB]; static uint32_t g_pend_hgen[PEND_HB], g_pend_gen = 1; // a bucket's chain is valid when its generation is this submission's
|
||||
static struct { int32_t e, next; } g_pend_node[PEND_NODES]; static int g_npnode;
|
||||
static int32_t g_pend_wide[PEND_WIDE]; static int g_nwide;
|
||||
typedef struct { uint64_t lo; size_t off; __unsafe_unretained id<MTLBuffer> buf; } upd_ent; // the ring block lives until this submission completes (mtl_sub_update)
|
||||
static upd_ent g_upd[PEND_MAX]; static int g_nupd;
|
||||
static inline unsigned pend_bucket(uint64_t page) { return (unsigned)((page * 0x9E3779B97F4A7C15ull) >> 53) & (PEND_HB - 1); }
|
||||
static void pend_reset(void) { g_npend = g_pend_over = g_nupd = g_npnode = g_nwide = 0; if (!++g_pend_gen) { memset(g_pend_hgen, 0, sizeof g_pend_hgen); g_pend_gen = 1; } }
|
||||
static int pend_add(uint64_t dst, uint64_t n, int kind) {
|
||||
if (g_pend_over || g_npend == PEND_MAX) { g_pend_over = 1; return -1; }
|
||||
int e = g_npend; uint64_t p0 = dst >> 12, p1 = (dst + (n ? n : 1) - 1) >> 12;
|
||||
if (p1 - p0 < PEND_SPAN && g_npnode + (int)(p1 - p0 + 1) <= PEND_NODES)
|
||||
for (uint64_t pg = p0; pg <= p1; pg++) { unsigned b = pend_bucket(pg); int k = g_npnode++;
|
||||
g_pend_node[k].e = e; g_pend_node[k].next = g_pend_hgen[b] == g_pend_gen ? g_pend_head[b] : -1; g_pend_head[b] = k; g_pend_hgen[b] = g_pend_gen; }
|
||||
else if (g_nwide < PEND_WIDE) g_pend_wide[g_nwide++] = e;
|
||||
else { g_pend_over = 1; return -1; }
|
||||
g_pend_lo[e] = dst; g_pend_hi[e] = dst + n; g_pend_kind[e] = (uint8_t)kind; g_pend_upd[e] = 0; return g_npend++;
|
||||
}
|
||||
// The newest queued write overlapping [a, a+n): its index, -1 for none, -2 when it cannot be told (overflow)
|
||||
static int pend_last(uint64_t a, uint64_t n) {
|
||||
if (g_pend_over) return -2;
|
||||
int best = -1; uint64_t p0 = a >> 12, p1 = (a + (n ? n : 1) - 1) >> 12;
|
||||
for (int i = 0; i < g_nwide; i++) { int e = g_pend_wide[i]; if (e > best && a < g_pend_hi[e] && a + n > g_pend_lo[e]) best = e; }
|
||||
if (p1 - p0 >= PEND_SPAN) { for (int e = g_npend - 1; e > best; e--) if (a < g_pend_hi[e] && a + n > g_pend_lo[e]) return e; return best; }
|
||||
for (uint64_t pg = p0; pg <= p1; pg++) { unsigned b = pend_bucket(pg); if (g_pend_hgen[b] != g_pend_gen) continue;
|
||||
for (int k = g_pend_head[b]; k >= 0 && g_pend_node[k].e > best; k = g_pend_node[k].next) { int e = g_pend_node[k].e; if (a < g_pend_hi[e] && a + n > g_pend_lo[e]) { best = e; break; } } }
|
||||
return best;
|
||||
}
|
||||
static int pend_hits(uint64_t a, uint64_t n) { return pend_last(a, n) != -1; }
|
||||
// The n bytes at a as the last write queued in this submission leaves them, when that write is an inline update covering them all → 1; otherwise → 0 and *kind = what wrote them (0 = too many writes to tell)
|
||||
static int pend_read(uint64_t a, uint64_t n, void *out, int *kind) {
|
||||
int e = pend_last(a, n); *kind = e >= 0 ? g_pend_kind[e] : 0;
|
||||
if (e < 0 || !g_pend_upd[e] || a < g_pend_lo[e] || a + n > g_pend_hi[e]) return 0;
|
||||
const upd_ent *u = &g_upd[g_pend_upd[e] - 1]; memcpy(out, (const uint8_t *)u->buf.contents + u->off + (a - u->lo), n); return 1;
|
||||
}
|
||||
typedef struct { uint64_t ser; uint16_t level, slice; } zskip_zero;
|
||||
static zskip_zero g_zskip_zero[4096]; static int g_zskip_n, g_zskip_count;
|
||||
static zskip_zero g_zskip_cand[8]; static uint8_t g_zskip_cand_ok[8];
|
||||
@@ -561,7 +603,7 @@ void mtl_sub_begin(void) {
|
||||
g_commit_blocks = [NSMutableArray array];
|
||||
// The overflow buffer created when the previous submission ran out of slots (see g_vis_n >= VIS_SLOTS): retire it. It used to be just cleared here and stayed in the residency set for good (from d2r-mobile 2eb575877)
|
||||
if (g_vis && !vis_in_ring(g_vis)) defer_obj_release(g_vis);
|
||||
g_npost = 0; g_vis = nil; g_vis_n = 0; g_vis_cur = -1; g_npend = 0; g_pend_over = 0;
|
||||
g_npost = 0; g_vis = nil; g_vis_n = 0; g_vis_cur = -1; pend_reset();
|
||||
zskip_clear(); g_zskip_count = 0;
|
||||
int pick = -1;
|
||||
for (int i = 0; i < g_vis_ring_n; i++) if (g_vis_ring[i].fence <= g_done) { pick = i; break; }
|
||||
@@ -680,7 +722,7 @@ static id<MTL4ComputeCommandEncoder> blit(void) {
|
||||
}
|
||||
|
||||
void mtl_sub_copy(uint64_t src, uint64_t dst, uint64_t n) {
|
||||
if (g_npend < PEND_MAX) { g_pend_lo[g_npend] = dst; g_pend_hi[g_npend++] = dst + n; } else g_pend_over = 1;
|
||||
pend_add(dst, n, 2);
|
||||
void *sb, *db; uint64_t so, dof;
|
||||
if (!mtl_addr(src, n, &sb, &so) || !mtl_addr(dst, n, &db, &dof))
|
||||
MDIE("buffer copy address is in no memory pool src=%#llx dst=%#llx n=%#llx", (unsigned long long)src, (unsigned long long)dst, (unsigned long long)n);
|
||||
@@ -735,7 +777,7 @@ static void ubd_read(uint64_t lo, size_t n, void *out) {
|
||||
}
|
||||
void mtl_sub_update(const void *data, uint64_t dst, uint64_t n) {
|
||||
if (!n) return;
|
||||
if (g_npend < PEND_MAX) { g_pend_lo[g_npend] = dst; g_pend_hi[g_npend++] = dst + n; } else g_pend_over = 1;
|
||||
int pi = pend_add(dst, n, 1);
|
||||
void *db; uint64_t dof;
|
||||
if (!mtl_addr(dst, n, &db, &dof)) MDIE("uniform update target is in no memory pool dst=%#llx n=%#llx", (unsigned long long)dst, (unsigned long long)n);
|
||||
int defer = enc_open() && g_nubd < UBD_MAX && ubd_on();
|
||||
@@ -748,6 +790,7 @@ void mtl_sub_update(const void *data, uint64_t dst, uint64_t n) {
|
||||
res_add(g_ubs_buf); g_ubs_off = 0;
|
||||
}
|
||||
memcpy((uint8_t *)g_ubs_buf.contents + g_ubs_off, data, n);
|
||||
if (pi >= 0) { g_upd[g_nupd] = (upd_ent){dst, g_ubs_off, g_ubs_buf}; g_pend_upd[pi] = (uint16_t)++g_nupd; }
|
||||
if (defer) { g_ubd[g_nubd++] = (ubd_ent){dst, dst + n, g_ubs_buf.gpuAddress + g_ubs_off, g_ubs_off, g_ubs_buf}; g_ubd_deferred++; args_dirty_all(); } // see ubd_shadow
|
||||
else enc_b_copy_buffer(blit_buf(0, 0, dst, dst + n), g_ubs_buf, g_ubs_off, (__bridge id<MTLBuffer>)db, dof, n);
|
||||
g_ubs_off += need;
|
||||
@@ -756,7 +799,7 @@ void mtl_sub_update(const void *data, uint64_t dst, uint64_t n) {
|
||||
// In-order buffer fill within the submission: all four bytes equal → fillBuffer; otherwise copied in chunks from a 64 KiB pattern buffer
|
||||
void mtl_sub_fill(uint64_t dst, uint64_t n, uint32_t value) {
|
||||
if (!n) return;
|
||||
if (g_npend < PEND_MAX) { g_pend_lo[g_npend] = dst; g_pend_hi[g_npend++] = dst + n; } else g_pend_over = 1;
|
||||
pend_add(dst, n, 3);
|
||||
void *db; uint64_t dof;
|
||||
if (!mtl_addr(dst, n, &db, &dof)) MDIE("buffer fill address is in no memory pool dst=%#llx n=%#llx", (unsigned long long)dst, (unsigned long long)n);
|
||||
uint8_t b0 = (uint8_t)value;
|
||||
@@ -1071,7 +1114,8 @@ static void tex_put(uint64_t k, id o) { // serials from 2^62 up are external t
|
||||
// Host record of a packaged texture: the current MTLTexture, the upload stamp, and the global write generation at the last check
|
||||
// spare: the texture the last re-upload replaced, kept resident and written by the next re-upload once the GPU has passed spare_fence (a texture the CPU rewrites every frame, a movie frame,
|
||||
// alternates between two textures instead of allocating one per frame: Link's Awakening's title movie made and freed an 8 MB texture a frame)
|
||||
@interface NvPkg : NSObject { @public id<MTLTexture> tex, spare; uint64_t stamp, seen, hash, spare_fence; const void *span; } @end // hash: full hash of the uploaded content under DM_NVN_DIRTYCHECK; span: its tracked range (nvn_dirty_span, NULL = untracked)
|
||||
@interface NvPkg : NSObject { @public id<MTLTexture> tex, spare, view; uint64_t stamp, seen, hash, spare_fence, vgen, vseen; const void *span; int sparse, vbase; } @end // hash: full hash of the uploaded content under DM_NVN_DIRTYCHECK; span: its tracked range (nvn_dirty_span, NULL = untracked);
|
||||
// sparse: in a sparse virtual pool, read through its page table (hle_nvn.c nvn_sparse_*), vgen the page table's generation at the upload and vseen at the last look; view: what it is bound as (sparse_view)
|
||||
@implementation NvPkg
|
||||
- (void)dealloc { res_remove(tex); if (spare) res_remove(spare); }
|
||||
@end
|
||||
@@ -1112,14 +1156,15 @@ static MTLTextureDescriptor *tex_desc(const tex_t *t, int packaged) {
|
||||
}
|
||||
// In-place upgrade: when the original texture was not created with PixelFormatView (off by default, to keep Apple GPU lossless compression) but a different-format view is needed, create a new one with the original spec + PixelFormatView, copy over every slice × level, and replace the registration.
|
||||
// Count: across the 4 screens character select / camp / wilderness / combat, 17473 textures needed a different-format view 0 times, so this is a fallback and does not rely on predicting which textures need it
|
||||
static id<MTLTexture> tex_upgrade_pfv(const tex_t *t, id<MTLTexture> base, MTLPixelFormat view_pf) {
|
||||
// The same upgrade adds render-target usage for a whole-level clear (clear_tex) of a texture made without it: Monster Hunter Rise clears a 32x80 packaged one
|
||||
static id<MTLTexture> tex_upgrade(const tex_t *t, id<MTLTexture> base, MTLTextureUsage add, MTLPixelFormat view_pf) {
|
||||
MTLTextureDescriptor *d = [MTLTextureDescriptor new];
|
||||
d.textureType = base.textureType; d.pixelFormat = base.pixelFormat; d.width = base.width; d.height = base.height; d.depth = base.depth; // Keep the original format, only add the format-view usage
|
||||
d.arrayLength = base.arrayLength; d.mipmapLevelCount = base.mipmapLevelCount; d.sampleCount = base.sampleCount;
|
||||
d.storageMode = base.storageMode; d.cpuCacheMode = base.cpuCacheMode; d.hazardTrackingMode = base.hazardTrackingMode;
|
||||
d.resourceOptions = base.resourceOptions; d.usage = base.usage | MTLTextureUsagePixelFormatView;
|
||||
d.resourceOptions = base.resourceOptions; d.usage = base.usage | add;
|
||||
id<MTLTexture> upgraded = [g_dev newTextureWithDescriptor:d];
|
||||
if (!upgraded) { RED_ONCE(45, "texture upgrade failed ser=%llu format %lu → %lu", t->serial, (unsigned long)base.pixelFormat, (unsigned long)view_pf); return base; }
|
||||
if (!upgraded) { RED_ONCE(45, "texture upgrade failed ser=%llu format %lu → %lu usage %#lx", t->serial, (unsigned long)base.pixelFormat, (unsigned long)view_pf, (unsigned long)d.usage); return base; }
|
||||
res_add(upgraded);
|
||||
id<MTL4ComputeCommandEncoder> b = blit();
|
||||
NSUInteger slices = base.textureType == MTLTextureType3D ? 1 : base.arrayLength * (base.textureType == MTLTextureTypeCube || base.textureType == MTLTextureTypeCubeArray ? 6 : 1);
|
||||
@@ -1131,15 +1176,17 @@ static id<MTLTexture> tex_upgrade_pfv(const tex_t *t, id<MTLTexture> base, MTLPi
|
||||
enc_b_copy_texture(b, base, s, l, MTLOriginMake(0, 0, 0), MTLSizeMake(w, h, 1), upgraded, s, l, MTLOriginMake(0, 0, 0));
|
||||
}
|
||||
id holder = tex_get(t->serial);
|
||||
if ([holder isKindOfClass:[NvPkg class]]) { id<MTLTexture> old = ((NvPkg *)holder)->tex; ((NvPkg *)holder)->tex = upgraded; defer_obj_release(old); g_ser_gen++; }
|
||||
if ([holder isKindOfClass:[NvPkg class]]) { id<MTLTexture> old = ((NvPkg *)holder)->tex; ((NvPkg *)holder)->tex = upgraded; defer_obj_release(old); g_ser_gen++;
|
||||
if (((NvPkg *)holder)->view) { defer_obj_unref(((NvPkg *)holder)->view); ((NvPkg *)holder)->view = nil; ((NvPkg *)holder)->vbase = 0; } } // a sparse texture's view of the old one: bound whole until its next upload
|
||||
else {
|
||||
tex_put(t->serial, upgraded);
|
||||
alias_key ak; memset(&ak, 0, sizeof ak); ak.pool = (uint64_t)(uintptr_t)t->pool; ak.off = t->off; ak.d = t->d; // Switch the sharing table to the upgraded one too, so later objects on the same storage get it directly
|
||||
pthread_mutex_lock(&g_smu); if (hm_get_id(&g_alias, &ak) == base) { hm_put_id(&g_alias, &ak, upgraded); CFSetAddValue(g_alias_set, (__bridge const void *)upgraded); } pthread_mutex_unlock(&g_smu);
|
||||
}
|
||||
MLOG("texture upgraded to allow format views ser=%llu base format %lu → view format %lu (upgrades=%llu)", t->serial, (unsigned long)base.pixelFormat, (unsigned long)view_pf, ++g_pfv_upgrades);
|
||||
MLOG("texture upgraded to allow %s ser=%llu base format %lu → view format %lu (upgrades=%llu)", add & MTLTextureUsageRenderTarget ? "render-target use" : "format views", t->serial, (unsigned long)base.pixelFormat, (unsigned long)view_pf, ++g_pfv_upgrades);
|
||||
return upgraded;
|
||||
}
|
||||
static id<MTLTexture> tex_upgrade_pfv(const tex_t *t, id<MTLTexture> base, MTLPixelFormat view_pf) { return tex_upgrade(t, base, MTLTextureUsagePixelFormatView, view_pf); }
|
||||
// Packaged texture upload: first set read-only, then take the stamp, then read the pool memory and deswizzle into a staging buffer, and blit into a new private texture.
|
||||
// Batching: uploads within one replay submission share one upload command buffer (compute encoder deswizzle + a unified blit), flushed before the replay command buffer is committed;
|
||||
// the same queue executes in commit order ⇒ uploads are always finished before the first sample. Uploads outside a submission are flushed on the spot. The staging area is cut from 32 MB private blocks, held by the command buffer until completion.
|
||||
@@ -1310,6 +1357,9 @@ static void up_blit_add(id<MTLBuffer> src, size_t off, size_t row, size_t image,
|
||||
}
|
||||
|
||||
static id<MTLTexture> pkg_upload_(const tex_t *t, uint64_t *stamp, id<MTLTexture> reuse);
|
||||
int nvn_host_readable(uint64_t a, uint64_t n); // hle_nvn.c: guest memory and sparse pools
|
||||
int nvn_sparse_pool(const void *pool); uint64_t nvn_sparse_gen(void); uint64_t nvn_sparse_copy(const void *pool, void *dst, uint64_t a, uint64_t n);
|
||||
int nvn_sparse_since(const void *pool, uint64_t a, uint64_t n, uint64_t gen, uint64_t stamp); void nvn_sparse_protect(const void *pool, uint64_t a, uint64_t n); int nvn_sparse_mapped(const void *pool, uint64_t a, uint64_t n);
|
||||
static id<MTLTexture> pkg_upload(const tex_t *t, uint64_t *stamp, id<MTLTexture> reuse) { // reuse: a texture of the same description, idle on the GPU, to upload into instead of a new one
|
||||
struct timespec a, b; clock_gettime(CLOCK_MONOTONIC, &a);
|
||||
id<MTLTexture> r = pkg_upload_(t, stamp, reuse);
|
||||
@@ -1330,7 +1380,8 @@ static id<MTLTexture> pkg_upload_(const tex_t *t, uint64_t *stamp, id<MTLTexture
|
||||
if (!mt) { RED_ONCE(2, "failed to create MTLTexture %ux%u format %#x target %d", t->d.width, t->d.height, t->d.format, t->d.target); return nil; }
|
||||
if (g_allocprof > 0) { static int n; if (n++ < 30 || n % 500 == 0) MLOG("new texture (%s) #%d ser=%llu %ux%ux%u format %#x target %d levels %u storage %p", __func__, n, (unsigned long long)t->serial, t->d.width, t->d.height, t->d.depth, t->d.format, t->d.target, t->d.levels, (void *)t->mem); }
|
||||
res_add(mt); // Both first upload and re-upload register residency here (Metal 4 does not make resources resident automatically; a missed registration = GPU address page fault)
|
||||
nvn_dirty_protect(t->mem, t->L.size);
|
||||
int sparse = t->pool && nvn_sparse_pool(t->pool);
|
||||
if (sparse) nvn_sparse_protect(t->pool, (uint64_t)(uintptr_t)t->mem, t->L.size); else nvn_dirty_protect(t->mem, t->L.size);
|
||||
*stamp = nvn_dirty_now();
|
||||
int bw, bh, bpb = nvn_fmt_bpb(t->d.format, &bw, &bh);
|
||||
unsigned layers = (t->d.target == 4 || t->d.target == 9) ? t->d.depth : t->d.target == 8 ? 6 : t->d.target == 3 ? t->d.height : 1;
|
||||
@@ -1342,8 +1393,8 @@ static id<MTLTexture> pkg_upload_(const tex_t *t, uint64_t *stamp, id<MTLTexture
|
||||
}
|
||||
g_bp_up_bytes += total;
|
||||
void *mb; uint64_t mo;
|
||||
int use_gpu = g_nvn_deswz_gpu && mtl_addr((uint64_t)t->mem, t->L.size, &mb, &mo);
|
||||
if (g_nvn_deswz_gpu && !use_gpu) RED_ONCE(40, "swizzled texture storage is not in a memory pool; falling back to the CPU");
|
||||
int use_gpu = !sparse && g_nvn_deswz_gpu && mtl_addr((uint64_t)t->mem, t->L.size, &mb, &mo); // a sparse pool's texture is gathered through its page table on the CPU
|
||||
if (g_nvn_deswz_gpu && !use_gpu && !sparse) RED_ONCE(40, "swizzled texture storage is not in a memory pool; falling back to the CPU");
|
||||
if (use_gpu) {
|
||||
// The GPU reads the memory pool at execution time, which is closer to a real Switch than a snapshot at CPU upload time; the dirty-page stamp is still taken before encoding, and later writes still trigger a re-upload.
|
||||
up_begin();
|
||||
@@ -1370,6 +1421,12 @@ static id<MTLTexture> pkg_upload_(const tex_t *t, uint64_t *stamp, id<MTLTexture
|
||||
RED_ONCE(41, "GPU deswizzle does not support format %#x target %d; falling back to the CPU", t->d.format, t->d.target);
|
||||
}
|
||||
|
||||
// A sparse pool's texture is read through a copy gathered from its page table, unmapped pages as 0 (a mip tail at a 4 KB offset has no host alias; reading the alias stopped the host with SIGBUS)
|
||||
const uint8_t *src = t->mem; uint8_t *part = NULL;
|
||||
if (sparse) {
|
||||
if (!(part = malloc(t->L.size ? t->L.size : 1))) MDIE("sparse texture copy of %llu bytes failed", (unsigned long long)t->L.size);
|
||||
nvn_sparse_copy(t->pool, part, (uint64_t)(uintptr_t)t->mem, t->L.size); src = part;
|
||||
}
|
||||
up_begin();
|
||||
id<MTLBuffer> stg = [g_dev newBufferWithLength:total ? total : 16 options:MTLResourceStorageModeShared];
|
||||
res_add(stg); [g_up_keep addObject:stg];
|
||||
@@ -1378,11 +1435,12 @@ static id<MTLTexture> pkg_upload_(const tex_t *t, uint64_t *stamp, id<MTLTexture
|
||||
uint32_t w = MAX(t->d.width >> l, 1u), h = MAX(t->d.height >> l, 1u), z = t->d.target == 2 ? MAX(t->d.depth >> l, 1u) : 1;
|
||||
size_t pitch = (size_t)((w + bw - 1) / bw) * bpb, rows = (h + bh - 1) / bh;
|
||||
for (unsigned y = 0; y < layers; y++) {
|
||||
if (nvn_deswizzle(&t->d, l, y, t->mem, (uint8_t *)stg.contents + off, pitch)) { RED_ONCE(9, "deswizzle failed format %#x target %d", t->d.format, t->d.target); break; }
|
||||
if (nvn_deswizzle(&t->d, l, y, src, (uint8_t *)stg.contents + off, pitch)) { RED_ONCE(9, "deswizzle failed format %#x target %d", t->d.format, t->d.target); break; }
|
||||
up_blit_add(stg, off, pitch, pitch * rows, MTLSizeMake(w, h, z), mt, y, l);
|
||||
off += pitch * rows * z;
|
||||
}
|
||||
}
|
||||
free(part);
|
||||
up_flush();
|
||||
return mt;
|
||||
}
|
||||
@@ -1485,12 +1543,31 @@ static int tex_cpu_written(uint64_t serial) {
|
||||
}
|
||||
static id<MTLTexture> tex_mtl_(const tex_t *ct);
|
||||
static id<MTLTexture> tex_mtl(const tex_t *ct) { AP_B; id<MTLTexture> r = tex_mtl_(ct); AP_E(4); return r; }
|
||||
// A sparse texture is bound from its first level whose pages, and those of every smaller level, are all mapped: a view of levels [r, n) samples as with a minimum LOD of r.
|
||||
// The translator drops TEX's LOD clamp and residency predicate, so Monster Hunter Rise's streamed textures were sampled at levels it had not mapped yet; those read 0, which ASTC
|
||||
// decodes as its error colour (magenta clothes on the title screen). Single-layer 2D textures only; the view follows each upload, as the mappings change only with one
|
||||
static void sparse_view(NvPkg *pk, const tex_t *t) {
|
||||
unsigned n = (unsigned)pk->tex.mipmapLevelCount, r = n;
|
||||
if (t->d.target == NVN_TEXTURE_TARGET_2D && n > 1)
|
||||
while (r > 0) { uint64_t lo = nvn_tex_level_offset(&t->d, r - 1), hi = r < n ? nvn_tex_level_offset(&t->d, r) : t->L.size;
|
||||
if (!nvn_sparse_mapped(t->pool, (uint64_t)(uintptr_t)t->mem + lo, hi - lo)) break;
|
||||
r--; }
|
||||
else r = 0;
|
||||
if (r == n) r = n - 1; // nothing mapped: the smallest level (reads 0)
|
||||
if (pk->view) defer_obj_unref(pk->view); // a view shares its texture's allocation: drop only the reference, the residency set keeps the texture
|
||||
pk->view = r ? [pk->tex newTextureViewWithPixelFormat:pk->tex.pixelFormat textureType:pk->tex.textureType levels:NSMakeRange(r, n - r) slices:NSMakeRange(0, 1)] : nil;
|
||||
if (r != (unsigned)pk->vbase) { static int nl; if (nl++ < 6) MLOG("sparse texture ser=%llu %ux%u: sampled from level %u of %u (finer levels not mapped)", (unsigned long long)t->serial, t->d.width, t->d.height, r, n); }
|
||||
pk->vbase = (int)r;
|
||||
}
|
||||
static uint64_t g_texinfo_ser[8]; static int g_texinfo_n;
|
||||
static id<MTLTexture> tex_mtl_(const tex_t *ct) {
|
||||
tex_t *t = (tex_t *)ct;
|
||||
if (!t || !t->serial) return nil;
|
||||
if (t->magic != MAGIC('T', 'E', 'X', 0)) { RED_ONCE(70, "texture object no longer valid (magic %#x, first case ser=%llu): no MTLTexture created", t->magic, (unsigned long long)t->serial); return nil; }
|
||||
int packaged = (t->has_layout || tex_cpu_written(t->serial)) && t->mem;
|
||||
// A sampled texture in a sparse virtual pool holds what the CPU wrote into the physical pages mapped there (Monster Hunter Rise: the hunter's innerwear, ASTC 6x6 with no packaged
|
||||
// layout, came out magenta, and the face black): it is read through the page table like a packaged one. Render targets, images, linear and buffer textures stay GPU textures (as ft_add)
|
||||
int packaged = (t->has_layout || tex_cpu_written(t->serial) || (t->pool && (t->pool->flags & NVN_MEMORY_POOL_FLAGS_VIRTUAL) && t->d.target != NVN_TEXTURE_TARGET_BUFFER
|
||||
&& !(t->d.flags & (NVN_TEXTURE_FLAGS_LINEAR | NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET | NVN_TEXTURE_FLAGS_COMPRESSIBLE | NVN_TEXTURE_FLAGS_IMAGE)) && nvn_sparse_pool(t->pool))) && t->mem;
|
||||
{ // Diagnostic DM_NVN_TEXINFO=WxH: log storage and description of textures of that size on first resolution (to tell whether two textures share storage and why they were not merged)
|
||||
static int ti = -1; static uint32_t tw, th; static uint64_t last[64]; static int nl;
|
||||
if (ti > 0 && t->d.width == tw && t->d.height == th && g_texinfo_n < 8) { int dup = 0; for (int i = 0; i < g_texinfo_n; i++) dup |= g_texinfo_ser[i] == t->serial; if (!dup && !(t->d.flags & 8)) g_texinfo_ser[g_texinfo_n++] = t->serial; } // remember the non-render-target ones (no COMPRESSIBLE) for DM_NVN_TEXRAW=info
|
||||
@@ -1498,7 +1575,10 @@ static id<MTLTexture> tex_mtl_(const tex_t *ct) {
|
||||
if (ti && t->d.width == tw && t->d.height == th) {
|
||||
int seen = 0; for (int i = 0; i < nl; i++) if (last[i] == t->serial) seen = 1;
|
||||
static uint64_t n; if (!seen && (n++ < 40 || n % 600 == 0)) { if (nl < 64) last[nl++] = t->serial; else last[n % 64] = t->serial;
|
||||
uint64_t nz = 0; if (t->mem && t->L.size <= (64u << 20)) for (uint64_t i = 0; i < t->L.size; i++) nz += t->mem[i] != 0; // non-zero bytes in guest memory: content the CPU wrote
|
||||
uint64_t nz = 0; if (t->mem && t->L.size <= (64u << 20)) { uint8_t *m = t->mem, *part = NULL; // non-zero bytes in guest memory: content the CPU wrote (a sparse pool's texture through its page table)
|
||||
if (t->pool && nvn_sparse_pool(t->pool) && (part = malloc(t->L.size))) { nvn_sparse_copy(t->pool, part, (uint64_t)(uintptr_t)t->mem, t->L.size); m = part; }
|
||||
for (uint64_t i = 0; i < t->L.size; i++) nz += m[i] != 0;
|
||||
free(part); }
|
||||
MLOG("TEXINFO ser=%llu swizzled=%d has_layout=%d mem=%p pool=%p(%#x) off=%#llx target=%d format=%#x flags=%#x samples=%d %ux%ux%u levels=%u stride=%u block_h=%d storage %llu bytes non-zero %llu",
|
||||
(unsigned long long)t->serial, packaged, t->has_layout, t->mem, t->pool, t->pool ? t->pool->flags : 0, (unsigned long long)t->off, t->d.target, t->d.format, t->d.flags, t->d.samples,
|
||||
t->d.width, t->d.height, t->d.depth, t->d.levels, t->d.stride, t->d.pkg_block_h, (unsigned long long)t->L.size, (unsigned long long)nz); }
|
||||
@@ -1512,30 +1592,34 @@ static id<MTLTexture> tex_mtl_(const tex_t *ct) {
|
||||
uint64_t now = nvn_dirty_now();
|
||||
int tracked = pk->span != NULL; // kept from the upload: a texture's pool stays registered while the texture lives
|
||||
int clean = tracked && (now == pk->seen || !nvn_dirty_since_span(pk->span, t->mem, t->L.size, pk->stamp));
|
||||
if (pk->sparse) { uint64_t g = nvn_sparse_gen(); clean = (now == pk->seen && g == pk->vseen) || !nvn_sparse_since(t->pool, (uint64_t)(uintptr_t)t->mem, t->L.size, pk->vgen, pk->stamp); pk->vseen = g; }
|
||||
pk->seen = now;
|
||||
if (clean && atomic_load_explicit(&g_up_lost_n, memory_order_relaxed) && up_lost_take(pk->tex)) { clean = 0; uint64_t k = atomic_load(&g_up_lost_redo); if (k <= 3 || !atomic_load(&g_up_lost_n)) MLOG("re-uploading a texture whose upload was dropped ser=%llu (%llu in total, %d left in the record)", (unsigned long long)t->serial, (unsigned long long)k, atomic_load(&g_up_lost_n)); }
|
||||
if (clean) {
|
||||
if (dirtycheck() && full_hash(t->mem, t->L.size) != pk->hash && !g_pkg_miss++)
|
||||
if (dirtycheck() && !pk->sparse && full_hash(t->mem, t->L.size) != pk->hash && !g_pkg_miss++)
|
||||
MLOG("[RED] dirty-page miss: texture ser=%llu pool content changed but was not detected", t->serial);
|
||||
return pk->tex;
|
||||
return pk->view ? pk->view : pk->tex;
|
||||
}
|
||||
// Pool content changed (streamed assets written into the same storage, a movie frame): switch to another texture, as in-flight draws still read the old one. That is the spare
|
||||
// when the GPU is done with it (see NvPkg), else a new one; the old one becomes the spare (pkg_upload_ made nt resident)
|
||||
id<MTLTexture> reuse = nil;
|
||||
if (pk->spare) { pthread_mutex_lock(&g_mu); uint64_t done = g_done; pthread_mutex_unlock(&g_mu); if (pk->spare_fence <= done && pk->spare.usage == pk->tex.usage) reuse = pk->spare; }
|
||||
uint64_t st; id<MTLTexture> nt = pkg_upload(t, &st, reuse);
|
||||
uint64_t st, vg = nvn_sparse_gen(); id<MTLTexture> nt = pkg_upload(t, &st, reuse);
|
||||
if (nt) { id<MTLTexture> old = pk->tex; pk->tex = nt;
|
||||
if (pk->spare && pk->spare != nt) defer_obj_release(pk->spare); // still in use, or of another shape: retired as before
|
||||
pk->spare = old; pk->spare_fence = g_fence + 1; g_ser_gen++; pk->stamp = st; pk->span = nvn_dirty_span(t->mem, t->L.size); pk->seen = nvn_dirty_now(); g_pkg_reuploads++; if (dirtycheck()) pk->hash = full_hash(t->mem, t->L.size); }
|
||||
return pk->tex;
|
||||
if (pk->sparse) sparse_view(pk, t);
|
||||
pk->spare = old; pk->spare_fence = g_fence + 1; g_ser_gen++; pk->stamp = st; pk->vgen = pk->vseen = vg; pk->span = nvn_dirty_span(t->mem, t->L.size); pk->seen = nvn_dirty_now(); g_pkg_reuploads++; if (dirtycheck() && !pk->sparse) pk->hash = full_hash(t->mem, t->L.size); }
|
||||
return pk->view ? pk->view : pk->tex;
|
||||
}
|
||||
pk = [NvPkg new];
|
||||
pk->sparse = t->pool && nvn_sparse_pool(t->pool); pk->vgen = pk->vseen = nvn_sparse_gen();
|
||||
pk->tex = pkg_upload(t, &pk->stamp, nil);
|
||||
if (!pk->tex) return nil;
|
||||
pk->span = nvn_dirty_span(t->mem, t->L.size);
|
||||
pk->seen = nvn_dirty_now(); g_pkg_uploads++; if (dirtycheck()) pk->hash = full_hash(t->mem, t->L.size);
|
||||
pk->seen = nvn_dirty_now(); g_pkg_uploads++; if (dirtycheck() && !pk->sparse) pk->hash = full_hash(t->mem, t->L.size);
|
||||
if (pk->sparse) sparse_view(pk, t);
|
||||
tex_put(t->serial, pk);
|
||||
return pk->tex;
|
||||
return pk->view ? pk->view : pk->tex;
|
||||
}
|
||||
if (have) return have;
|
||||
if (t->d.target == 10) { // Buffer texture (TARGET_BUFFER): create the texture-buffer view directly on the memory pool buffer (zero copy; the GPU reads the pool's current content at execution time, same as NVN semantics)
|
||||
@@ -2763,7 +2847,7 @@ static struct {
|
||||
va_st attr[16]; int nattr; vs_st stream[16]; int nstream;
|
||||
NVNbufferRange vbuf[16];
|
||||
p_bind_buf ubo[6][16], ssbo[6][16];
|
||||
uint64_t tex[6][128], samp[6][128], img[6][32];
|
||||
uint64_t tex[6][NTEX], samp[6][128], img[6][32];
|
||||
uint32_t st_vmask[2], st_wmask[2], st_ref[2];
|
||||
p_poly_offset po; float vp[16][4]; float dr[2]; int has_vp; int depth_clamp;
|
||||
float blend_color[4]; int64_t copy_stride, copy_img_stride;
|
||||
@@ -2845,7 +2929,7 @@ static MTLVertexFormat vfmt_for_ct(MTLVertexFormat f, int ct, int nvn_fmt) {
|
||||
for (int i = 0; i < 4; i++) if (F[i] == f) iff = i;
|
||||
if (ct == NVN_CT_UINT) { if (iu >= 0) return f; if (ii >= 0) return U[ii]; if (iff >= 0) return U[8 + iff]; }
|
||||
else if (ct == NVN_CT_INT) { if (ii >= 0) return f; if (iu >= 0) return I[iu]; if (iff >= 0) return I[8 + iff]; }
|
||||
else { if (iu < 0 && ii < 0) return f; }
|
||||
else { if (iu < 0 && ii < 0) return f; if ((iu >= 8 && iu < 12) || (ii >= 8 && ii < 12)) return F[(iu >= 8 ? iu : ii) - 8]; } // 32-bit integers into a float input: the same bits as floats
|
||||
// An integer input fed a normalized, half or packed format: Maxwell would hand the shader the bits of the converted float, which Metal cannot, and Metal refuses the whole pipeline
|
||||
// ("Vertex attribute … of type uint4 cannot be read using MTLAttributeFormatHalf2"). Read the same bytes as integers of the same layout instead, so the pipeline exists and the draw is kept.
|
||||
// Fire Emblem: Three Houses: KTGL's character shaders read attribute 5 as integer bits (skinning data) and static meshes bind a Half2 there; with the pipeline refused, the characters'
|
||||
@@ -2928,10 +3012,14 @@ static void vconv_elem(int f, const uint8_t *p, float o[4]) {
|
||||
break; }
|
||||
}
|
||||
}
|
||||
// A float input fed an 8 / 16-bit integer format takes the same 32-bit register values through a Float4 attribute, which Metal passes through bit for bit: Monster Hunter Rise's
|
||||
// skinned vertex shaders (40609cd5c1443df4, 1369a37113da0012) read RGBA8UI attribute 5 as flags (as_type<int>(x) & 2 ...) and also as a float, so the build leaves the input float
|
||||
static int vconv_wanted(const nvn_refl *rf, int v, const pso_key *k) { // does Metal input v of this pipeline take converted data
|
||||
int a = rf->v[v].nvn, ct = rf->v[v].ct, ib, is;
|
||||
if ((ct != NVN_CT_UINT && ct != NVN_CT_INT) || a >= 16) return 0;
|
||||
if (a >= 16) return 0;
|
||||
int f = k ? k->attr[a].fmt : S.attr[a].fmt;
|
||||
if (ct == NVN_CT_FLOAT) return vconv_int(f, &ib, &is) > 0;
|
||||
if (ct != NVN_CT_UINT && ct != NVN_CT_INT) return 0;
|
||||
return vconv_ncomp(f) || (vconv_int(f, &ib, &is) && is != (ct == NVN_CT_INT));
|
||||
}
|
||||
// Per submission, one conversion per (range, format, layout): KTGL draws the same mesh several times a frame (depth, shadow, G-buffer)
|
||||
@@ -2950,7 +3038,8 @@ static void vconv_bind(id<MTL4RenderCommandEncoder> en, const nvn_refl *rf) {
|
||||
for (uint64_t i = 0; i < nv; i++) vconv_elem(f, (const uint8_t *)(uintptr_t)(addr + off + i * stride), d + 4 * i);
|
||||
gpu = b.gpuAddress + o;
|
||||
int c = g_vconv_next++ % 64; g_vconv[c].addr = addr; g_vconv[c].size = size; g_vconv[c].serial = g_sub_serial; g_vconv[c].fmt = (uint32_t)f; g_vconv[c].off = off; g_vconv[c].stride = stride; g_vconv[c].gpu = gpu;
|
||||
if (vconv_ncomp(f)) RED_ONCE(89, "vertex attribute format %#x feeds an integer input: converted to float32 bit patterns on the CPU at replay and bound separately (the conversion Maxwell does on vertex fetch)", f);
|
||||
if (rf->v[v].ct == NVN_CT_FLOAT) RED_ONCE(99, "vertex attribute format %#x (8 / 16-bit integer) feeds a float input: the 32-bit register values Maxwell's fetch makes are bound as float bit patterns, converted on the CPU at replay", f);
|
||||
else if (vconv_ncomp(f)) RED_ONCE(89, "vertex attribute format %#x feeds an integer input: converted to float32 bit patterns on the CPU at replay and bound separately (the conversion Maxwell does on vertex fetch)", f);
|
||||
else RED_ONCE(96, "vertex attribute format %#x (8 / 16-bit integer) feeds an integer input of the opposite signedness: extended to 32 bits with the format's own signedness on the CPU at replay and bound separately", f);
|
||||
} else if (nv) RED_ONCE(90, "vertex attribute conversion: a range of %llu vertices exceeds the per-frame staging (format %#x)", (unsigned long long)nv, f);
|
||||
}
|
||||
@@ -3047,7 +3136,7 @@ static MTLRenderPipelineDescriptor *pso_desc(const pso_key *k, NvShader *vs, NvS
|
||||
int st = k->attr[a].stream;
|
||||
if (vconv_wanted(rf, v, k) && nconv < VCONV_MAX) { // converted on the CPU at draw time (vconv_bind): the register's 32-bit values, 4 components, its own slot
|
||||
int slot = VCONV_SLOT0 + nconv++;
|
||||
vd.attributes[loc].format = rf->v[v].ct == NVN_CT_UINT ? MTLVertexFormatUInt4 : MTLVertexFormatInt4; vd.attributes[loc].offset = 0; vd.attributes[loc].bufferIndex = slot;
|
||||
vd.attributes[loc].format = rf->v[v].ct == NVN_CT_UINT ? MTLVertexFormatUInt4 : rf->v[v].ct == NVN_CT_INT ? MTLVertexFormatInt4 : MTLVertexFormatFloat4; vd.attributes[loc].offset = 0; vd.attributes[loc].bufferIndex = slot;
|
||||
vd.layouts[slot].stride = 16;
|
||||
vd.layouts[slot].stepFunction = k->stream[st].stride == 0 ? MTLVertexStepFunctionConstant : k->stream[st].divisor ? MTLVertexStepFunctionPerInstance : MTLVertexStepFunctionPerVertex;
|
||||
vd.layouts[slot].stepRate = k->stream[st].stride == 0 ? 0 : k->stream[st].divisor ? k->stream[st].divisor : 1;
|
||||
@@ -3349,16 +3438,21 @@ static uint64_t ui_mat_patch(const p_bind_buf *u, void *mb, uint64_t mo) {
|
||||
// Bindless handle (TEXU / SAMPU): the 32-bit word at byte offset off of uniform slot reg = the low 32 bits of the NVN handle the game wrote there (low 20 bits texture ID, high 12 bits sampler ID).
|
||||
// Read on the CPU side at replay time (the memory at the same moment a real GPU would read it at execution); slot unbound / out of range / unreadable = 0 (empty slot: dummy texture / dummy sampler).
|
||||
// A location with a GPU-side write queued in this submit (pend_hits) is still read at its current value; the first case is reported red.
|
||||
static uint32_t ubo_handle(int stage, int reg, int off) {
|
||||
if (stage < 0 || stage >= 6 || reg < 0 || reg >= 16) return 0;
|
||||
// The same read serves storage buffer addresses kept in uniform memory (SBUFU: 8 bytes)
|
||||
static void ubo_read(int stage, int reg, int off, size_t n, void *out) {
|
||||
memset(out, 0, n);
|
||||
if (stage < 0 || stage >= 6 || reg < 0 || reg >= 16) return;
|
||||
const p_bind_buf *u = &S.ubo[stage][reg];
|
||||
if (!u->addr || (uint64_t)off + 4 > u->size) return 0;
|
||||
if (g_nubd && ubd_hits(u->addr + off, u->addr + off + 4) && ubd_covers(u->addr + off, u->addr + off + 4)) { uint32_t h; ubd_read(u->addr + off, 4, &h); return h; } // written by an inline update deferred in this pass
|
||||
if (pend_hits(u->addr + off, 4)) RED_ONCE(47, "the uniform buffer holding bindless handles (stage %d slot %d +%#x) has GPU-side writes in this submission: the current value is read", stage, reg, off);
|
||||
if (!u->addr || (uint64_t)off + n > u->size) return;
|
||||
if (g_nubd && ubd_hits(u->addr + off, u->addr + off + n) && ubd_covers(u->addr + off, u->addr + off + n)) { ubd_read(u->addr + off, n, out); return; } // written by an inline update deferred in this pass
|
||||
if (pend_hits(u->addr + off, n)) { int kind; if (pend_read(u->addr + off, n, out, &kind)) return; // written by an inline update queued in this submission: its data
|
||||
RED_ONCE(47, "the uniform buffer holding bindless handles (stage %d slot %d +%#x) has GPU-side writes in this submission (%s): the current value is read", stage, reg, off,
|
||||
kind == 2 ? "a buffer copy" : kind == 3 ? "a fill" : kind == 1 ? "an inline update covering part of it" : "more writes than the host tracks"); }
|
||||
void *mb; uint64_t mo; const uint8_t *base;
|
||||
if (!mtl_addr(u->addr + off, 4, &mb, &mo) || !(base = [(__bridge id<MTLBuffer>)mb contents])) return 0;
|
||||
uint32_t h; memcpy(&h, base + mo, 4); return h;
|
||||
if (!mtl_addr(u->addr + off, n, &mb, &mo) || !(base = [(__bridge id<MTLBuffer>)mb contents])) return;
|
||||
memcpy(out, base + mo, n);
|
||||
}
|
||||
static uint32_t ubo_handle(int stage, int reg, int off) { uint32_t h; ubo_read(stage, reg, off, 4, &h); return h; }
|
||||
static uint32_t g_dispatch_dim[3]; // the current dispatch's workgroup counts (driver segment c0 + 0x200 for compute)
|
||||
static __unsafe_unretained id<MTLBuffer> g_c0_dim_buf; static size_t g_c0_dim_off; // where encode_args put them for this dispatch (nil: the shader reads no c0); an indirect dispatch copies its GPU-written counts there
|
||||
static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, int metal_vertex);
|
||||
@@ -3404,9 +3498,9 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
}
|
||||
slots[aid] = gpu;
|
||||
USE((__bridge id<MTLBuffer>)mb, MTLResourceUsageRead);
|
||||
} else if (b->kind == NVN_RB_SAMP || b->kind == NVN_RB_SAMPT || b->kind == NVN_RB_SAMPU) { // SAMPT: the sampler part of a combined handle (texture ID | sampler ID << 20); SAMPU: the same, with the handle in a uniform buffer
|
||||
} else if (b->kind == NVN_RB_SAMP || b->kind == NVN_RB_SAMPT || b->kind == NVN_RB_SAMPU || b->kind == NVN_RB_SAMPI) { // SAMPT: the sampler part of a combined handle (texture ID | sampler ID << 20); SAMPU: the same, with the handle in a uniform buffer
|
||||
int sid = b->kind == NVN_RB_SAMPU ? (int)(ubo_handle(nvn_stage, reg, b->off) >> 20) :
|
||||
reg < 128 ? (int)((b->kind == NVN_RB_SAMPT ? S.tex[nvn_stage][reg] : S.samp[nvn_stage][reg]) >> 20) : 0;
|
||||
b->kind == NVN_RB_SAMPI ? (int)reg : b->kind == NVN_RB_SAMPT ? (reg < NTEX ? (int)(S.tex[nvn_stage][reg] >> 20) : 0) : reg < 128 ? (int)(S.samp[nvn_stage][reg] >> 20) : 0; // SAMPI: a sampler ID the shader names as a constant
|
||||
__unsafe_unretained id<MTLSamplerState> ss = sid ? (__bridge id<MTLSamplerState>)samp_fast(sid) : nil;
|
||||
slots[aid] = sid && sid < g_nsreg && g_srid[sid] ? g_srid[sid] : dummy_samp_rid();
|
||||
USE(ss ? ss : dummy_samp(), MTLResourceUsageRead);
|
||||
@@ -3414,7 +3508,7 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
if (ok) PD("B %llu st%d samp reg%d sid=%d mag=%d min=%d (0 nearest / 1 linear; 2–5 with mips) wrap=%d,%d,%d border=%g,%g,%g,%g lod=%g..%g bias=%g aniso=%g cmp=%d/%d %s", g_stat_pass, nvn_stage, reg, sid, (int)sp.magf, (int)sp.minf,
|
||||
(int)sp.ws, (int)sp.wt, (int)sp.wr, sp.border[0], sp.border[1], sp.border[2], sp.border[3], sp.lod_min, sp.lod_max, sp.lod_bias, sp.aniso, (int)sp.cmp_mode, (int)sp.cmp_func, ss ? "real" : "failed"); }
|
||||
} else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TEXU) {
|
||||
int tid = b->kind == NVN_RB_TEXU ? (int)(ubo_handle(nvn_stage, reg, b->off) & 0xfffff) : reg < 128 ? (int)(S.tex[nvn_stage][reg] & 0xfffff) : 0;
|
||||
int tid = b->kind == NVN_RB_TEXU ? (int)(ubo_handle(nvn_stage, reg, b->off) & 0xfffff) : reg < NTEX ? (int)(S.tex[nvn_stage][reg] & 0xfffff) : 0;
|
||||
void *tv = NULL; uint64_t trid = tid ? tex_fast_rid(tid, &tv) : 0; __unsafe_unretained id<MTLTexture> t = (__bridge id<MTLTexture>)tv;
|
||||
if (t && g_fb_n && g_fb_seq == g_draw_seq && nvn_stage == 1) for (int q = 0; q < g_fb_n; q++) if (t == g_fb_src[q]) { t = g_fb_use[q]; trid = g_fb_rid[q]; break; } // same-pass feedback sampling: read the snapshot instead (see fb_prepare)
|
||||
if (t && tid < g_ntreg) use_rec("tex", (uint64_t)g_treg[tid].t.mem, g_treg[tid].t.L.size, tid, &g_treg[tid].t);
|
||||
@@ -3437,7 +3531,7 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
slots[aid] = trid;
|
||||
USE(t, MTLResourceUsageRead | MTLResourceUsageWrite);
|
||||
} else if (b->kind == NVN_RB_TBUF) { // Typed buffer: the buffer texture (target BUFFER) in texture slot r
|
||||
int tid = reg < 128 ? (int)(S.tex[nvn_stage][reg] & 0xfffff) : 0;
|
||||
int tid = reg < NTEX ? (int)(S.tex[nvn_stage][reg] & 0xfffff) : 0;
|
||||
void *tv = NULL; uint64_t trid = tid ? tex_fast_rid(tid, &tv) : 0; __unsafe_unretained id<MTLTexture> t = (__bridge id<MTLTexture>)tv;
|
||||
if (t && t.textureType != MTLTextureTypeTextureBuffer) { RED_ONCE(40, "typed buffer slot is bound to something other than a buffer texture (texture ID %d type %lu)", tid, (unsigned long)t.textureType); t = nil; }
|
||||
if (bl) PD("B %llu st%d tbuf reg%d tid=%d %s ser=%llu", g_stat_pass, nvn_stage, reg, tid, t ? "real" : "dummy", t && tid < g_ntreg ? g_treg[tid].t.serial : 0ull);
|
||||
@@ -3450,6 +3544,12 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
if (u && u->addr && mtl_gpu_addr(u->addr, u->size ? u->size : 1, &mb, &mo, &gpu)) { use_rec("buf", u->addr, u->size, 0, NULL); slots[aid] = gpu; USE((__bridge id<MTLBuffer>)mb, b->rw ? MTLResourceUsageRead | MTLResourceUsageWrite : MTLResourceUsageRead); }
|
||||
else { slots[aid] = dummy_buf_gpu(); USE(dummy_buf(), MTLResourceUsageRead); }
|
||||
if (bl) PD("B %llu st%d sbufp reg%d %s addr=%#llx size=%llu %s", g_stat_pass, nvn_stage, reg, u && u->addr ? "real" : "dummy", u ? (unsigned long long)u->addr : 0ull, u ? (unsigned long long)u->size : 0ull, b->rw ? "rw" : "ro");
|
||||
} else if (b->kind == NVN_RB_SBUFU) { // B: a storage buffer whose GPU address the game keeps in its own uniform buffer (Monster Hunter Rise's compute shaders: c3 + 0x38); read like a bindless handle
|
||||
uint64_t addr; ubo_read(nvn_stage, reg, b->off, 8, &addr);
|
||||
void *mb; uint64_t mo, gpu;
|
||||
if (addr && mtl_gpu_addr(addr, 1, &mb, &mo, &gpu)) { use_rec("buf", addr, 1, 0, NULL); slots[aid] = gpu; USE((__bridge id<MTLBuffer>)mb, b->rw ? MTLResourceUsageRead | MTLResourceUsageWrite : MTLResourceUsageRead); }
|
||||
else { slots[aid] = dummy_buf_gpu(); USE(dummy_buf(), MTLResourceUsageRead); }
|
||||
if (bl) PD("B %llu st%d sbufu reg%d+%#x addr=%#llx %s", g_stat_pass, nvn_stage, reg, b->off, (unsigned long long)addr, b->rw ? "rw" : "ro");
|
||||
} else if (b->kind == NVN_RB_C1) { // B: the shader's own constant segment, built when the shader is fetched
|
||||
id<MTLBuffer> cb = sh->c1buf ? sh->c1buf : zero_bank(); // c1 has length 0 (600 of them in the whole library) yet the shader still declares it: reads 0
|
||||
slots[aid] = (sh->c1buf ? sh->c1_gpu : zero_bank_gpu()) + (sh->c1buf ? sh->c1off : 0);
|
||||
@@ -3482,6 +3582,9 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
}
|
||||
} else {
|
||||
for (int r = 0; r < DM_GAME_NVN_TEX_SLOTS; r++) { uint32_t th = (uint32_t)S.tex[nvn_stage][r]; memcpy(d + DM_GAME_NVN_TEX_BASE + 8 * r, &th, 4); }
|
||||
#ifdef DM_GAME_NVN_SEPTEX_BASE // the separate texture table (Monster Hunter Rise: c2 0x168 + 8 r) from the slots after the bound ones
|
||||
for (int r = 0; r < DM_GAME_NVN_SEPTEX_SLOTS; r++) { uint32_t th = (uint32_t)S.tex[nvn_stage][128 + r]; memcpy(d + DM_GAME_NVN_SEPTEX_BASE + 8 * r, &th, 4); }
|
||||
#endif
|
||||
for (int r = 0; r < DM_GAME_NVN_IMG_SLOTS; r++) { uint32_t ih = (uint32_t)S.img[nvn_stage][r]; memcpy(d + DM_GAME_NVN_IMG_BASE + 8 * r, &ih, 4); } // This game (NVN 55): texture handle table at 0x20 + 8 r (dm/gfx/tools/dmsh_build.py TEX_BASE)
|
||||
// separate sampler handle table (older driver layout such as Engage's: textures 0x168 + 8 r, samplers 0x568 + 8 r; Mario uses no separate samplers, so this is all zero)
|
||||
for (int r = 0; r < 128; r++) { uint32_t sh_ = (uint32_t)S.samp[nvn_stage][r]; memcpy(d + DM_GAME_NVN_SAMP_BASE + 8 * r, &sh_, 4); }
|
||||
@@ -3738,7 +3841,7 @@ static int state_op(uint32_t op, const void *a) {
|
||||
case OP_BIND_VBUF: { const p_bind_vbuf *b = a; memcpy(&S.vbuf[b->first], b->r, sizeof *b->r * b->n); break; }
|
||||
case OP_BIND_UBO: { const p_bind_buf *b = a; if (b->stage < 6 && b->index < 16 && memcmp(&S.ubo[b->stage][b->index], b, sizeof *b)) { S.ubo[b->stage][b->index] = *b; g_args_dirty[b->stage] = 1; } break; }
|
||||
case OP_BIND_SSBO: { const p_bind_buf *b = a; if (b->stage < 6 && b->index < 16 && memcmp(&S.ssbo[b->stage][b->index], b, sizeof *b)) { S.ssbo[b->stage][b->index] = *b; g_args_dirty[b->stage] = 1; } break; }
|
||||
case OP_BIND_TEX: { const p_bind_handles *b = a; bh_fresh(b); if (b->stage < 6 && b->first + b->n <= 128 && memcmp(&S.tex[b->stage][b->first], b->h, 8 * b->n)) { memcpy(&S.tex[b->stage][b->first], b->h, 8 * b->n); g_args_dirty[b->stage] = 1; } break; }
|
||||
case OP_BIND_TEX: { const p_bind_handles *b = a; bh_fresh(b); if (b->stage < 6 && b->first + b->n <= NTEX && memcmp(&S.tex[b->stage][b->first], b->h, 8 * b->n)) { memcpy(&S.tex[b->stage][b->first], b->h, 8 * b->n); g_args_dirty[b->stage] = 1; } break; }
|
||||
case OP_BIND_SAMP: { const p_bind_handles *b = a; bh_fresh(b); if (b->stage < 6 && b->first + b->n <= 128 && memcmp(&S.samp[b->stage][b->first], b->h, 8 * b->n)) { memcpy(&S.samp[b->stage][b->first], b->h, 8 * b->n); g_args_dirty[b->stage] = 1; } break; }
|
||||
case OP_BIND_IMG: { const p_bind_handles *b = a; bh_fresh(b); if (b->stage < 6 && b->first >= 0 && b->first + b->n <= 32) { if (memcmp(&S.img[b->stage][b->first], b->h, 8 * b->n)) { memcpy(&S.img[b->stage][b->first], b->h, 8 * b->n); g_args_dirty[b->stage] = 1; } } else RED_ONCE(31, "image slots out of range stage %d first %d count %d", b->stage, b->first, b->n); break; }
|
||||
case OP_STENCIL: {
|
||||
@@ -3927,7 +4030,7 @@ static int draw_samples_rt(NvProg *pr) { // Sampled attachments keep the origi
|
||||
if (!pr) return 1;
|
||||
for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh) continue;
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; uint64_t h;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 && sh->stage < 6 ? S.img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < 128 && sh->stage < 6 ? S.tex[sh->stage][b->reg] : 0; else continue;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 && sh->stage < 6 ? S.img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < NTEX && sh->stage < 6 ? S.tex[sh->stage][b->reg] : 0; else continue;
|
||||
int tid = (int)(h & 0xfffff); if (!tid) continue;
|
||||
pthread_mutex_lock(&g_rmu); uint64_t ser = tid < g_ntreg && g_treg[tid].ok ? g_treg[tid].t.serial : 0; pthread_mutex_unlock(&g_rmu);
|
||||
for (int j = 0; j < nr; j++) if (ser && ser == rs[j]) return 1; } }
|
||||
@@ -3977,7 +4080,7 @@ static int sdx_bucket(uint64_t ser, int add) { // Returns the bucket number; r
|
||||
}
|
||||
static const uint8_t **g_sdx_op; static uint32_t g_sdx_nop, g_sdx_opcap;
|
||||
static uint32_t g_exec_seq; // Sequence number of the current operation during replay
|
||||
static uint64_t g_pp_slot[6][160]; static const prog_t *g_pp_prog, *g_pp_cprog; static const p_set_rt *g_pp_rt; static uint32_t g_pp_rtseq;
|
||||
static uint64_t g_pp_slot[6][NTEX + 32]; static const prog_t *g_pp_prog, *g_pp_cprog; static const p_set_rt *g_pp_rt; static uint32_t g_pp_rtseq; // g_pp_slot: textures, then images
|
||||
static uint32_t *g_rtx_seq, *g_drx_seq; static uint32_t g_rtx_n, g_rtx_cap, g_drx_n, g_drx_cap; // This submit's SET_RT / draw sequence number tables (used by the pre-screen of the readable tile merge)
|
||||
static void seq_push(uint32_t **v, uint32_t *n, uint32_t *cap, uint32_t x) { if (*n == *cap) { *cap = *cap ? *cap * 2 : 1024; *v = realloc(*v, *cap * sizeof **v); } (*v)[(*n)++] = x; }
|
||||
static void sdx_ev_add(sdx_ev e) { if (sdx_bucket(e.ser, 0) < 0) return; if (g_sdx_n == g_sdx_cap) { g_sdx_cap = g_sdx_cap ? g_sdx_cap * 2 : 4096; g_sdx = realloc(g_sdx, g_sdx_cap * sizeof *g_sdx); g_sdx_tmp = realloc(g_sdx_tmp, g_sdx_cap * sizeof *g_sdx); } g_sdx[g_sdx_n++] = e; }
|
||||
@@ -4007,17 +4110,17 @@ static void sdx_walk(const uint8_t *p, int depth) {
|
||||
if (c->depth_mask) sdx_ev_add((sdx_ev){seq, g_pp_rtseq, r->depth->serial, p, lv, sl, SDX_CLEAR, 1, !r->has_dview, 0});
|
||||
if ((c->stencil_mask & 0xff) == 0xff) sdx_ev_add((sdx_ev){seq, g_pp_rtseq, r->depth->serial, p, lv, sl, SDX_CLEAR, 2, !r->has_dview, 0});
|
||||
break; }
|
||||
case OP_BIND_TEX: case OP_BIND_IMG: { const p_bind_handles *b = a; bh_fresh(b); int base = h.op == OP_BIND_IMG ? 128 : 0, lim = h.op == OP_BIND_IMG ? 32 : 128;
|
||||
case OP_BIND_TEX: case OP_BIND_IMG: { const p_bind_handles *b = a; bh_fresh(b); int base = h.op == OP_BIND_IMG ? NTEX : 0, lim = h.op == OP_BIND_IMG ? 32 : NTEX;
|
||||
if (b->stage < 6 && b->first >= 0 && b->first + b->n <= lim) for (int k = 0; k < b->n; k++) g_pp_slot[b->stage][base + b->first + k] = h_ser(b->h[k]);
|
||||
break; }
|
||||
case OP_BIND_PROGRAM: { const p_bind_program *bp_ = a; if (bp_->stages & 0x20) g_pp_cprog = bp_->prog; if (bp_->stages & 0x1f) g_pp_prog = bp_->prog; break; }
|
||||
case OP_DRAW: case OP_DRAW_IDX: case OP_DISPATCH: {
|
||||
if (h.op != OP_DISPATCH) seq_push(&g_drx_seq, &g_drx_n, &g_drx_cap, seq);
|
||||
const prog_t *pp = h.op == OP_DISPATCH ? g_pp_cprog : g_pp_prog; __unsafe_unretained NvProg *pr = pp ? prog_get(pp) : nil;
|
||||
if (!pr) { for (int st = 0; st < 6; st++) for (int k = 0; k < 160; k++) if (g_pp_slot[st][k]) sdx_ev_add((sdx_ev){seq, 0, g_pp_slot[st][k], p, 0, 0, SDX_READ, 0, 0, 1}); }
|
||||
if (!pr) { for (int st = 0; st < 6; st++) for (int k = 0; k < NTEX + 32; k++) if (g_pp_slot[st][k]) sdx_ev_add((sdx_ev){seq, 0, g_pp_slot[st][k], p, 0, 0, SDX_READ, 0, 0, 1}); }
|
||||
else for (int st = 0; st < 6; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh || (h.op == OP_DISPATCH) != (st == 5)) continue;
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; int k = b->kind == NVN_RB_IMG ? 128 + b->reg : (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) ? b->reg : -1;
|
||||
if (k < 0 || k >= 160 || sh->stage < 0 || sh->stage >= 6) continue; uint64_t ser = g_pp_slot[sh->stage][k];
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; int k = b->kind == NVN_RB_IMG ? NTEX + b->reg : (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) ? b->reg : -1;
|
||||
if (k < 0 || k >= NTEX + 32 || sh->stage < 0 || sh->stage >= 6) continue; uint64_t ser = g_pp_slot[sh->stage][k];
|
||||
if (ser) sdx_ev_add((sdx_ev){seq, 0, ser, p, 0, 0, SDX_READ, 0, 0, 1}); } }
|
||||
const p_set_rt *r = g_pp_rt;
|
||||
if (h.op != OP_DISPATCH && r) {
|
||||
@@ -4041,8 +4144,8 @@ void mtl_sub_index(const uint64_t *h, int n) { // Start of submit (hle_nvn.c Q
|
||||
for (int i = 0; i < g_xv_n; i++) sdx_bucket(g_xv[i].ser, 1); // Textures in the ledger enter the candidate set: otherwise, when one is only sampled and not used as an attachment in this submit, its read events are not indexed and the fallback cannot see them
|
||||
if (R.has_rt) { for (int k = 0; k < R.rt.n && k < 8; k++) if (R.rt.color[k]) sdx_bucket(R.rt.color[k]->serial, 1); if (R.rt.depth) sdx_bucket(R.rt.depth->serial, 1); } // The targets at the start of the submit (the first draw may reopen the render pass without going through SetRenderTargets)
|
||||
g_pp_prog = S.prog; g_pp_cprog = S.cprog; g_pp_rt = NULL; g_pp_rtseq = SDX_NONE;
|
||||
for (int st = 0; st < 6; st++) { for (int k = 0; k < 128; k++) g_pp_slot[st][k] = S.tex[st][k] ? h_ser(S.tex[st][k]) : 0;
|
||||
for (int k = 0; k < 32; k++) g_pp_slot[st][128 + k] = S.img[st][k] ? h_ser(S.img[st][k]) : 0; }
|
||||
for (int st = 0; st < 6; st++) { for (int k = 0; k < NTEX; k++) g_pp_slot[st][k] = S.tex[st][k] ? h_ser(S.tex[st][k]) : 0;
|
||||
for (int k = 0; k < 32; k++) g_pp_slot[st][NTEX + k] = S.img[st][k] ? h_ser(S.img[st][k]) : 0; }
|
||||
for (int i = 0; i < n && g_sdx_ok; i++) sdx_walk((const uint8_t *)h[i], 0);
|
||||
if (!g_sdx_ok) return;
|
||||
// Counting sort into buckets: events are appended in sequence-number order, and within the same sequence number READ precedes ATT (the same draw is judged for reads first, then attachments), so after a stable scatter each bucket is ordered
|
||||
@@ -4127,7 +4230,7 @@ static void hoist_try(void) {
|
||||
if (rt_same(&R.rt, r)) { // Back to this render pass's targets: execute if hoisting holds
|
||||
if (!ng) return;
|
||||
uint64_t bound[512]; int nb = 0; // The order of the original "check at the start": the current slots (stage x textures 128 + images 32), followed by those bound within the scan range
|
||||
for (int st = 0; st < 6; st++) { for (int k = 0; k < 128 && nb < 512; k++) if (S.tex[st][k]) bound[nb++] = h_ser(S.tex[st][k]);
|
||||
for (int st = 0; st < 6; st++) { for (int k = 0; k < NTEX && nb < 512; k++) if (S.tex[st][k]) bound[nb++] = h_ser(S.tex[st][k]);
|
||||
for (int k = 0; k < 32 && nb < 512; k++) if (S.img[st][k]) bound[nb++] = h_ser(S.img[st][k]); }
|
||||
for (int k = 0; k < nbh && nb < 512; k++) bound[nb++] = h_ser(bh[k]);
|
||||
for (int gi = 0; gi < ng; gi++) { // Disjoint and not bound
|
||||
@@ -4228,11 +4331,11 @@ static void fz_end(void) { // End of a merged render pass (called by rp_end):
|
||||
else { for (int i = 0; i < g_nfzs; i++) free(g_fzs[i].S_); g_nfzs = 0; g_fzs_bad = 0; }
|
||||
}
|
||||
static int sub_eq(const p_set_rt *a, int ka, const p_set_rt *b, int kb) { uint64_t s1, s2; int l1, l2, x1, x2; rt_sub(a, ka, &s1, &l1, &x1); rt_sub(b, kb, &s2, &l2, &x2); return s1 && s1 == s2 && l1 == l2 && x1 == x2; }
|
||||
static int prog_samples_ser(const prog_t *pg, uint64_t tex[6][128], uint64_t img[6][32], uint64_t ser) { // Whether this texture is bound in any texture / image / typed buffer slot declared by the program's reflection
|
||||
static int prog_samples_ser(const prog_t *pg, uint64_t tex[6][NTEX], uint64_t img[6][32], uint64_t ser) { // Whether this texture is bound in any texture / image / typed buffer slot declared by the program's reflection
|
||||
__unsafe_unretained NvProg *pr = pg ? prog_get(pg) : nil; if (!pr) return 1;
|
||||
for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh || sh->stage >= 6) continue;
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; uint64_t h;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 ? img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < 128 ? tex[sh->stage][b->reg] : 0; else continue;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 ? img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < NTEX ? tex[sh->stage][b->reg] : 0; else continue;
|
||||
if (h && h_ser(h) == ser) return 1; } }
|
||||
return 0;
|
||||
}
|
||||
@@ -4249,12 +4352,12 @@ static int defer_handle_sub(uint64_t h, const p_set_rt *r, int k, int cover) {
|
||||
view_t v = g_treg[tid].v; int n = v.nlevels > 0 ? v.nlevels : 1;
|
||||
return lv >= v.base_level && lv < v.base_level + n && (!v.set_layers || (sl >= v.min_layer && sl < v.min_layer + (v.nlayers > 0 ? v.nlayers : 1)));
|
||||
}
|
||||
static int defer_prog_hits(const prog_t *pg, uint64_t tex[6][128], uint64_t img[6][32], const p_set_rt *r, int color_only, int cover) {
|
||||
static int defer_prog_hits(const prog_t *pg, uint64_t tex[6][NTEX], uint64_t img[6][32], const p_set_rt *r, int color_only, int cover) {
|
||||
__unsafe_unretained NvProg *pr = pg ? prog_get(pg) : nil; if (!pr) return 1;
|
||||
for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh || sh->stage < 0 || sh->stage >= 6) continue;
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; uint64_t h = 0;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 ? img[sh->stage][b->reg] : 0;
|
||||
else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < 128 ? tex[sh->stage][b->reg] : 0;
|
||||
else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < NTEX ? tex[sh->stage][b->reg] : 0;
|
||||
else continue;
|
||||
if (!h) continue;
|
||||
for (int k = 0; k < (color_only ? r->n : 9); k++) if ((color_only ? r->color[k] : (k < 8 ? r->color[k] : r->depth)) && defer_handle_sub(h, r, k, cover)) return 1;
|
||||
@@ -4262,12 +4365,12 @@ static int defer_prog_hits(const prog_t *pg, uint64_t tex[6][128], uint64_t img[
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
static void defer_collect_reads(defer_t *d, const prog_t *pg, uint64_t tex[6][128], uint64_t img[6][32]) {
|
||||
static void defer_collect_reads(defer_t *d, const prog_t *pg, uint64_t tex[6][NTEX], uint64_t img[6][32]) {
|
||||
__unsafe_unretained NvProg *pr = pg ? prog_get(pg) : nil; if (!pr) { d->nreads = 513; return; }
|
||||
for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh || sh->stage < 0 || sh->stage >= 6) continue;
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; uint64_t h = 0;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 ? img[sh->stage][b->reg] : 0;
|
||||
else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < 128 ? tex[sh->stage][b->reg] : 0;
|
||||
else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < NTEX ? tex[sh->stage][b->reg] : 0;
|
||||
else continue;
|
||||
if (!h) continue; int seen = 0; for (int j = 0; j < d->nreads; j++) seen |= d->p_reads[j] == h;
|
||||
if (!seen && d->nreads < 512) d->p_reads[d->nreads++] = h;
|
||||
@@ -4386,7 +4489,7 @@ static void defer_try(const p_draw *dw, const p_draw_idx *di) {
|
||||
if (!defer_pre()) return;
|
||||
memset(&g_defer, 0, sizeof g_defer); g_defer.p_rt = R.rt; memcpy(g_defer.clear_c, R.clear_c, 8); memcpy(g_defer.cc, R.cc, sizeof R.cc);
|
||||
g_defer.clear_d = R.clear_d; g_defer.clear_s = R.clear_s; g_defer.cd = R.cd; g_defer.cs = R.cs;
|
||||
uint64_t tex[6][128], img[6][32]; memcpy(tex, S.tex, sizeof tex); memcpy(img, S.img, sizeof img); const prog_t *pg = S.prog; ds_st ds = S.ds;
|
||||
uint64_t tex[6][NTEX], img[6][32]; memcpy(tex, S.tex, sizeof tex); memcpy(img, S.img, sizeof img); const prog_t *pg = S.prog; ds_st ds = S.ds;
|
||||
const uint8_t *p = g_exec_next, *p_end = NULL, *qop = NULL; const p_set_rt *q = NULL; p_set_rt scan_rt = R.rt; int srt_hit = -1, ci = 0, draws = 1, p_has_draw = 1; defer_collect_reads(&g_defer, pg, tex, img); // The current draw is P's first
|
||||
for (int guard = 0; guard < 200000; guard++) {
|
||||
op_hdr h; memcpy(&h, p, sizeof h); const void *a = p + 8;
|
||||
@@ -4396,7 +4499,7 @@ static void defer_try(const p_draw *dw, const p_draw_idx *di) {
|
||||
case OP_CALL: g_defer.reason = 2; goto fail;
|
||||
case OP_JUMP: memcpy(&p, a, 8); continue;
|
||||
case OP_BIND_PROGRAM: pg = ((const p_bind_program *)a)->prog; break;
|
||||
case OP_BIND_TEX: case OP_BIND_IMG: { const p_bind_handles *b = a; bh_fresh(b); int lim = h.op == OP_BIND_IMG ? 32 : 128;
|
||||
case OP_BIND_TEX: case OP_BIND_IMG: { const p_bind_handles *b = a; bh_fresh(b); int lim = h.op == OP_BIND_IMG ? 32 : NTEX;
|
||||
if (b->stage >= 0 && b->stage < 6 && b->first >= 0 && b->first + b->n <= lim) for (int k = 0; k < b->n; k++) { if (h.op == OP_BIND_IMG) img[b->stage][b->first + k] = b->h[k]; else tex[b->stage][b->first + k] = b->h[k]; } break; }
|
||||
case OP_BIND_STATE: { const p_bind_state *b = a; if (b->kind == 5) memcpy(&ds, b->bytes, sizeof ds); break; }
|
||||
case OP_DRAW: case OP_DRAW_IDX:
|
||||
@@ -4451,7 +4554,7 @@ q_found:
|
||||
if (!q_body && h.op == OP_SET_RT && !q_clears) { const p_set_rt *r2 = a; if (!defer_q_superset(r2)) { g_defer.reason = 15; goto fail; } q = r2; qop = p; p += h.size; continue; }
|
||||
if (h.op == OP_CLEAR_COLOR || h.op == OP_CLEAR_DS) q_clears = 1;
|
||||
if (!q_body && (h.op == OP_END || h.op == OP_SET_RT || h.op == OP_UPDATE_UB || h.op == OP_COPY_B2B || h.op == OP_CLEAR_BUF || h.op == OP_DISPATCH)) { g_defer.reason = 9; g_defer.reason_op = h.op; goto fail; }
|
||||
if (h.op == OP_BIND_PROGRAM) pg = ((const p_bind_program *)a)->prog; else if (h.op == OP_BIND_TEX || h.op == OP_BIND_IMG) { const p_bind_handles *b = a; bh_fresh(b); int lim = h.op == OP_BIND_IMG ? 32 : 128; if (b->stage >= 0 && b->stage < 6 && b->first >= 0 && b->first + b->n <= lim) for (int k = 0; k < b->n; k++) { if (h.op == OP_BIND_IMG) img[b->stage][b->first + k] = b->h[k]; else tex[b->stage][b->first + k] = b->h[k]; } }
|
||||
if (h.op == OP_BIND_PROGRAM) pg = ((const p_bind_program *)a)->prog; else if (h.op == OP_BIND_TEX || h.op == OP_BIND_IMG) { const p_bind_handles *b = a; bh_fresh(b); int lim = h.op == OP_BIND_IMG ? 32 : NTEX; if (b->stage >= 0 && b->stage < 6 && b->first >= 0 && b->first + b->n <= lim) for (int k = 0; k < b->n; k++) { if (h.op == OP_BIND_IMG) img[b->stage][b->first + k] = b->h[k]; else tex[b->stage][b->first + k] = b->h[k]; } }
|
||||
else if (h.op == OP_CLEAR_COLOR) { const p_clear_color *c = a; if (c->index >= 0 && c->index < q->n && q->color[c->index]) for (int i = 0; i < g_defer.p_rt.n; i++) if (g_defer.p_rt.color[i] && sub_eq(q, c->index, &g_defer.p_rt, i)) { g_defer.reason = 10; goto fail; } }
|
||||
p += h.size;
|
||||
}
|
||||
@@ -4547,7 +4650,7 @@ static void fuse_try(void) {
|
||||
if (!R.rt.depth && r2->depth) { U.depth = r2->depth; U.dview = r2->dview; U.has_dview = r2->has_dview; depth_from_r2 = 1; }
|
||||
if ((cd_f || cs_f) && !depth_from_r2) return; // The depth is already in use in the earlier pass: the depth clear at the start of the later pass has nowhere to go
|
||||
// Four: draws of the earlier pass must not sample attachments brought in by the later pass; when the depth comes from the later pass, depth-stencil testing must not be enabled
|
||||
static uint64_t tex[6][128], img[6][32]; memcpy(tex, S.tex, sizeof tex); memcpy(img, S.img, sizeof img);
|
||||
static uint64_t tex[6][NTEX], img[6][32]; memcpy(tex, S.tex, sizeof tex); memcpy(img, S.img, sizeof img);
|
||||
const prog_t *pg = S.prog; ds_st ds = S.ds;
|
||||
cur = (fz_cursor){.p = g_exec_next};
|
||||
for (;;) { // The first loop iteration checks the draw that opens the render pass (current state), then checks the draws in the range one by one
|
||||
@@ -4560,18 +4663,18 @@ static void fuse_try(void) {
|
||||
if (q == r2op) goto planned;
|
||||
if (hq.op == OP_BIND_PROGRAM) pg = ((const p_bind_program *)aq)->prog;
|
||||
else if (hq.op == OP_BIND_STATE) { const p_bind_state *b = aq; if (b->kind == 5) memcpy(&ds, b->bytes, sizeof ds); }
|
||||
else if (hq.op == OP_BIND_TEX || hq.op == OP_BIND_IMG) { const p_bind_handles *b = aq; bh_fresh(b); int lim = hq.op == OP_BIND_IMG ? 32 : 128;
|
||||
else if (hq.op == OP_BIND_TEX || hq.op == OP_BIND_IMG) { const p_bind_handles *b = aq; bh_fresh(b); int lim = hq.op == OP_BIND_IMG ? 32 : NTEX;
|
||||
if (b->stage >= 0 && b->stage < 6 && b->first >= 0 && b->first + b->n <= lim) for (int k = 0; k < b->n; k++) { if (hq.op == OP_BIND_IMG) img[b->stage][b->first + k] = b->h[k]; else tex[b->stage][b->first + k] = b->h[k]; } }
|
||||
else if (hq.op == OP_DRAW || hq.op == OP_DRAW_IDX) break;
|
||||
}
|
||||
}
|
||||
planned:
|
||||
{ // Five: pre-screen the first draw of the later pass (static conditions), to avoid attaching the union of attachments for nothing: track the program and texture bindings up to that draw
|
||||
const prog_t *pg2 = S.prog; static uint64_t t2[6][128], i2[6][32]; memcpy(t2, S.tex, sizeof t2); memcpy(i2, S.img, sizeof i2);
|
||||
const prog_t *pg2 = S.prog; static uint64_t t2[6][NTEX], i2[6][32]; memcpy(t2, S.tex, sizeof t2); memcpy(i2, S.img, sizeof i2);
|
||||
cur = (fz_cursor){.p = g_exec_next};
|
||||
for (;;) { const uint8_t *q; op_hdr hq; const void *aq; if (!fz_next(&cur, &q, &hq, &aq)) return; if (q == p) break;
|
||||
if (hq.op == OP_BIND_PROGRAM) pg2 = ((const p_bind_program *)aq)->prog;
|
||||
else if (hq.op == OP_BIND_TEX || hq.op == OP_BIND_IMG) { const p_bind_handles *b = aq; bh_fresh(b); int lim = hq.op == OP_BIND_IMG ? 32 : 128;
|
||||
else if (hq.op == OP_BIND_TEX || hq.op == OP_BIND_IMG) { const p_bind_handles *b = aq; bh_fresh(b); int lim = hq.op == OP_BIND_IMG ? 32 : NTEX;
|
||||
if (b->stage >= 0 && b->stage < 6 && b->first >= 0 && b->first + b->n <= lim) for (int k = 0; k < b->n; k++) { if (hq.op == OP_BIND_IMG) i2[b->stage][b->first + k] = b->h[k]; else t2[b->stage][b->first + k] = b->h[k]; } }
|
||||
}
|
||||
NvProg *pr2 = pg2 ? prog_get(pg2) : nil; NvShader *vs2 = pr2 ? pr2->sh[0] : nil, *fs2 = pr2 ? pr2->sh[1] : nil;
|
||||
@@ -4682,7 +4785,7 @@ static int zskip_can_draw(NvProg *pr) {
|
||||
for (int i = 0; !why && i < fs->rf.nb; i++) { nvn_rbind *b = &fs->rf.b[i];
|
||||
if (b->kind == NVN_RB_IMG) { why = 4; break; }
|
||||
if (b->kind != NVN_RB_TEX && b->kind != NVN_RB_TBUF) continue;
|
||||
int tid = b->reg < 128 ? (int)(S.tex[1][b->reg] & 0xfffff) : 0; if (!zskip_view_zero(tid)) why = 5; }
|
||||
int tid = b->reg < NTEX ? (int)(S.tex[1][b->reg] & 0xfffff) : 0; if (!zskip_view_zero(tid)) why = 5; }
|
||||
if (!why && (!R.has_rt || !R.rt.n)) why = 6;
|
||||
for (int i = 0; !why && i < R.rt.n && i < 8; i++) { if (!R.rt.color[i]) continue;
|
||||
int pending = g_enc ? g_zskip_cand_ok[i] : zskip_zero_clear(i); // Render pass already open: look at the candidates noted when the render pass was opened (not yet drawn to); not open: look at the pending clears
|
||||
@@ -4714,7 +4817,7 @@ static int fz_p2_draw(NvProg *pr, const p_draw *dw, const p_draw_idx *di, uint8_
|
||||
int fx[32]; for (int i = 0; i < 32; i++) fx[i] = -1; uint8_t fbmask = 0;
|
||||
for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh || sh->stage >= 6) continue;
|
||||
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; uint64_t h;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 ? S.img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < 128 ? S.tex[sh->stage][b->reg] : 0; else continue;
|
||||
if (b->kind == NVN_RB_IMG) h = b->reg < 32 ? S.img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = b->reg < NTEX ? S.tex[sh->stage][b->reg] : 0; else continue;
|
||||
uint64_t ser = h ? h_ser(h) : 0; if (!ser) continue;
|
||||
int j = -1; for (int m = 0; m < g_fz.n1; m++) if (R.rt.color[m] && R.rt.color[m]->serial == ser) { j = m; break; }
|
||||
if (j < 0) continue; // What is sampled is not an attachment of the earlier pass
|
||||
@@ -4813,7 +4916,7 @@ int mtl_ui_ghost_rect(float r[4]) { uint64_t v = atomic_load(&g_ghost_found); if
|
||||
static struct { uint64_t pass; int n, nt, busy; float area, r[4]; struct { void *S_; p_draw dw; uint64_t tex; } c[4]; uint64_t tid[128]; uint16_t tn[128]; } g_gh;
|
||||
static uint64_t ghost_tex(void) { // The texture actually sampled by this draw's fragment shader (slot taken from shader reflection; the other slots in S.tex are left over from earlier draws)
|
||||
__unsafe_unretained NvProg *pr = S.prog ? prog_get(S.prog) : nil; NvShader *f = pr ? pr->sh[1] : nil; if (!f) return 0;
|
||||
for (int i = 0; i < f->rf.nb; i++) if ((f->rf.b[i].kind == NVN_RB_TEX || f->rf.b[i].kind == NVN_RB_SAMP) && f->rf.b[i].reg < 128 && S.tex[1][f->rf.b[i].reg]) return S.tex[1][f->rf.b[i].reg];
|
||||
for (int i = 0; i < f->rf.nb; i++) if ((f->rf.b[i].kind == NVN_RB_TEX || f->rf.b[i].kind == NVN_RB_SAMP) && f->rf.b[i].reg < NTEX && S.tex[1][f->rf.b[i].reg]) return S.tex[1][f->rf.b[i].reg];
|
||||
return 0; }
|
||||
// Whether the current UI draw is "untextured and exactly covering the whole UI rectangle" (the UI rectangle occupies the upper hf of the texture)
|
||||
static int ui_cover_all(double hf) {
|
||||
@@ -5265,11 +5368,41 @@ static int ubd_conflict(NvProg *pr, const p_draw *dw, const p_draw_idx *di) {
|
||||
const p_bind_buf *u = b->kind == NVN_RB_SBUFP ? &S.ssbo[st][b->reg] : &S.ubo[st][b->reg];
|
||||
if (!u->addr) continue;
|
||||
if (b->kind == NVN_RB_CBUF) { if (ubd_hits(u->addr, u->addr + u->size) && !ubd_covers(u->addr, u->addr + u->size)) return 1; }
|
||||
else if (b->kind == NVN_RB_TEXU || b->kind == NVN_RB_SAMPU) { uint64_t h = u->addr + b->off; if (ubd_hits(h, h + 4) && !ubd_covers(h, h + 4)) return 1; }
|
||||
else if (b->kind == NVN_RB_TEXU || b->kind == NVN_RB_SAMPU || b->kind == NVN_RB_SBUFU) { uint64_t h = u->addr + b->off, n = b->kind == NVN_RB_SBUFU ? 8 : 4; if (ubd_hits(h, h + n) && !ubd_covers(h, h + n)) return 1; }
|
||||
else if (b->kind == NVN_RB_SBUFP || b->kind == NVN_RB_UBOP) { if (ubd_hits(u->addr, u->addr + (u->size ? u->size : 1))) return 1; } } }
|
||||
return 0;
|
||||
}
|
||||
// Draws of an indirect-count multi-draw (hle_nvn.c nvnCommandBufferMultiDrawElementsIndirectCount): draw i is made only while i < the u32 count the game's GPU culling wrote earlier in the
|
||||
// submission. Metal cannot skip draws by a count in GPU memory, so before the first such draw after GPU writes (a dispatch, copy, update or fill since the last wait, g_wr_epoch; or a new
|
||||
// submission) the replay ends the render pass, commits what it has encoded and waits for it, then reads the count on the CPU; later draws of the group use the same value. One wait per
|
||||
// GPU-driven pass instead of a compute pass that masks the records (not done yet). The records themselves are still read by the GPU (ordinary indirect draws)
|
||||
static id<MTLSharedEvent> g_mdi_ev; static uint64_t g_mdi_v, g_mdi_waits, g_mdi_epoch = ~0ull, g_mdi_sub, g_mdi_addr; static uint32_t g_mdi_count;
|
||||
static void mdi_wait(void) {
|
||||
rp_end();
|
||||
if (g_blit) { enc_b_end(g_blit); g_blit = nil; }
|
||||
up_flush(); res_commit();
|
||||
[g_cb endCommandBuffer];
|
||||
id<MTL4CommandBuffer> buffers[] = {g_cb};
|
||||
[g_q4 commit:buffers count:1];
|
||||
if (!g_mdi_ev) g_mdi_ev = [g_dev newSharedEvent];
|
||||
[g_q4 signalEvent:g_mdi_ev value:++g_mdi_v];
|
||||
if (![g_mdi_ev waitUntilSignaledValue:g_mdi_v timeoutMS:10000]) RED_ONCE(97, "indirect-count multi-draw: the GPU did not finish within 10 s; the count is read anyway");
|
||||
g_cb = [g_dev newCommandBuffer];
|
||||
[g_cb beginCommandBufferWithAllocator:g_alloc4[g_alloc_cur]];
|
||||
if (!g_mdi_waits++) MLOG("indirect-count multi-draw: first wait for the GPU-written count at frame %llu pass %llu", (unsigned long long)atomic_load(&g_cur_frame), g_stat_pass);
|
||||
}
|
||||
static int mdi_live(uint64_t cnt, int i) {
|
||||
if (cnt != g_mdi_addr || g_mdi_epoch != g_wr_epoch || g_mdi_sub != g_sub_serial) {
|
||||
if (g_mdi_epoch != g_wr_epoch || g_mdi_sub != g_sub_serial) { mdi_wait(); g_mdi_epoch = g_wr_epoch; g_mdi_sub = g_sub_serial; }
|
||||
uint32_t v = 0;
|
||||
if (nvn_host_readable(cnt, 4)) memcpy(&v, (const void *)(uintptr_t)cnt, 4);
|
||||
else RED_ONCE(98, "indirect-count multi-draw: the count at %#llx is in an unmapped page of a sparse pool, nothing drawn", (unsigned long long)cnt);
|
||||
g_mdi_addr = cnt; g_mdi_count = v;
|
||||
}
|
||||
return (uint32_t)i < g_mdi_count;
|
||||
}
|
||||
static void draw(const p_draw *dw, const p_draw_idx *di) {
|
||||
if ((di && di->mdi_cnt && !mdi_live(di->mdi_cnt, di->mdi_i)) || (dw && dw->mdi_cnt && !mdi_live(dw->mdi_cnt, dw->mdi_i))) return; // past the multi-draw's count
|
||||
if (!S.prog) { RED_ONCE(18, "no program bound before the draw"); PD("Z %llu draw skipped: no program bound", g_stat_pass + !g_enc); return; }
|
||||
__unsafe_unretained NvProg *pr = prog_get(S.prog);
|
||||
if (g_nubd && g_enc && ubd_conflict(pr, dw, di)) { g_ubd_conflicts++; rp_end(); } // land the deferred inline updates before this draw (see ubd_conflict)
|
||||
@@ -5302,9 +5435,9 @@ static void draw(const p_draw *dw, const p_draw_idx *di) {
|
||||
{ static int nofxaa = -1; static uint64_t fx_from, fx_to; if (nofxaa < 0) { const char *e = getenv("DM_NVN_NOFXAA"); nofxaa = e && !strcmp(e, "1"); if (nofxaa) MLOG("DM_NVN_NOFXAA=1: skipping FXAA"); }
|
||||
if (nofxaa && tbdr_on() && !g_defer_replaying && pr->sh[1]) { NvShader *f = pr->sh[1];
|
||||
if (!strncmp(f->key, "ebcfcd1fa0a77dbc", 16) && !g_enc && R.rt.n == 1 && R.rt.color[0] && !R.clear_c[0]) {
|
||||
uint64_t h = 0; for (int i = 0; i < f->rf.nb && !h; i++) if (f->rf.b[i].kind == NVN_RB_TEX && f->rf.b[i].reg < 128) h = S.tex[1][f->rf.b[i].reg];
|
||||
uint64_t h = 0; for (int i = 0; i < f->rf.nb && !h; i++) if (f->rf.b[i].kind == NVN_RB_TEX && f->rf.b[i].reg < NTEX) h = S.tex[1][f->rf.b[i].reg];
|
||||
if (h) { fx_from = R.rt.color[0]->serial; fx_to = h; PD("N %llu FXAA skipped: readers of ser=%llu read ser=%llu instead", g_stat_pass, (unsigned long long)fx_from, (unsigned long long)h_ser(h)); return; } }
|
||||
else if (fx_from && !strncmp(f->key, "52aac1f2306b8fc5", 16)) { for (int r = 0; r < 128; r++) if (S.tex[1][r] && h_ser(S.tex[1][r]) == fx_from) S.tex[1][r] = fx_to; fx_from = 0; } } }
|
||||
else if (fx_from && !strncmp(f->key, "52aac1f2306b8fc5", 16)) { for (int r = 0; r < NTEX; r++) if (S.tex[1][r] && h_ser(S.tex[1][r]) == fx_from) S.tex[1][r] = fx_to; fx_from = 0; } } }
|
||||
{ static uint64_t bloomq_frame; if (bloomq_frame != atomic_load(&g_cur_frame)) { bloomq_frame = atomic_load(&g_cur_frame); bloomq_reset(); }
|
||||
if (bloomq_enabled() && tbdr_on() && !g_defer_replaying && pr->sh[1]) {
|
||||
NvShader *f = pr->sh[1]; uint64_t tgt = 0; int lv, sl;
|
||||
@@ -5313,8 +5446,8 @@ static void draw(const p_draw *dw, const p_draw_idx *di) {
|
||||
if (bloomq_target(&tgt, &lv, &sl) && R.rt.color[0]) {
|
||||
uint64_t h = 0; const tex_t *st = NULL; int cbuf = 0;
|
||||
for (int i = 0; i < f->rf.nb; i++) { nvn_rbind *b = &f->rf.b[i];
|
||||
if (b->kind == NVN_RB_TEX && b->reg < 128 && !h) { h = S.tex[1][b->reg]; int tid = (int)(h & 0xfffff); if (tid > 0 && tid < g_ntreg && g_treg[tid].ok) st = &g_treg[tid].t; }
|
||||
if (b->kind == NVN_RB_SAMPT && b->reg < 128) g_bloomq.samp_h = S.tex[1][b->reg] >> 20;
|
||||
if (b->kind == NVN_RB_TEX && b->reg < NTEX && !h) { h = S.tex[1][b->reg]; int tid = (int)(h & 0xfffff); if (tid > 0 && tid < g_ntreg && g_treg[tid].ok) st = &g_treg[tid].t; }
|
||||
if (b->kind == NVN_RB_SAMPT && b->reg < NTEX) g_bloomq.samp_h = S.tex[1][b->reg] >> 20;
|
||||
if (b->kind == NVN_RB_SAMP && b->reg < 128) g_bloomq.samp_h = S.samp[1][b->reg] >> 20;
|
||||
if (b->kind == NVN_RB_CBUF && !cbuf) cbuf = ubo_float(1, (uint16_t)(b->reg << 8), &g_bloomq.threshold); // c7[0].x
|
||||
}
|
||||
@@ -5325,20 +5458,20 @@ static void draw(const p_draw *dw, const p_draw_idx *di) {
|
||||
}
|
||||
}
|
||||
} else if (g_bloomq.state == 1 && !strncmp(f->key, "019ce2f82edbda84", 16)) {
|
||||
uint64_t src = 0; for (int i = 0; i < f->rf.nb; i++) if (f->rf.b[i].kind == NVN_RB_TEX && f->rf.b[i].reg < 128) { src = h_ser(S.tex[1][f->rf.b[i].reg]); break; }
|
||||
uint64_t src = 0; for (int i = 0; i < f->rf.nb; i++) if (f->rf.b[i].kind == NVN_RB_TEX && f->rf.b[i].reg < NTEX) { src = h_ser(S.tex[1][f->rf.b[i].reg]); break; }
|
||||
if (src == g_bloomq.b0_ser && bloomq_target(&tgt, &lv, &sl) && bloomq_capture()) { R.clear_c[0] = 0; g_bloomq.b1_ser = tgt; g_bloomq.state = 2; PD("B %llu bloom 1/4 native capture B0=%llu B1=%llu", g_stat_pass, (unsigned long long)g_bloomq.b0_ser, (unsigned long long)g_bloomq.b1_ser); return; }
|
||||
RED_ONCE(62, "bloom 1/4 start: the capture was skipped but the native capture did not draw; this frame lacks bloom"); bloomq_reset();
|
||||
} else if (g_bloomq.state == 2 && !strncmp(f->key, "019ce2f82edbda84", 16)) {
|
||||
for (int i = 0; i < f->rf.nb; i++) if (f->rf.b[i].kind == NVN_RB_TEX && f->rf.b[i].reg < 128 && h_ser(S.tex[1][f->rf.b[i].reg]) == g_bloomq.b1_ser) { g_bloomq.b1_h = S.tex[1][f->rf.b[i].reg]; break; }
|
||||
for (int i = 0; i < f->rf.nb; i++) if (f->rf.b[i].kind == NVN_RB_TEX && f->rf.b[i].reg < NTEX && h_ser(S.tex[1][f->rf.b[i].reg]) == g_bloomq.b1_ser) { g_bloomq.b1_h = S.tex[1][f->rf.b[i].reg]; break; }
|
||||
} else if (g_bloomq.state == 2 && !strncmp(f->key, "cc264583f2000c81", 16)) {
|
||||
if (bloomq_target(&tgt, &lv, &sl) && tgt == g_bloomq.b0_ser) { g_bloomq.state = 3; PD("C %llu bloom 1/4 skips the B0 upsample", g_stat_pass); return; }
|
||||
} else if (g_bloomq.state == 3 && !strncmp(f->key, "2e2d6700a2569795", 16)) {
|
||||
if (g_bloomq.b1_h) { for (int r = 0; r < 128; r++) if (S.tex[1][r] && h_ser(S.tex[1][r]) == g_bloomq.b0_ser) S.tex[1][r] = g_bloomq.b1_h; PD("D %llu bloom 1/4 composite reads B1 instead", g_stat_pass); g_bloomq_last_b0 = g_bloomq.b0_ser; g_bloomq_last_b1h = g_bloomq.b1_h; g_bloomq_last_b1s = g_bloomq.b1_ser; bloomq_reset(); }
|
||||
if (g_bloomq.b1_h) { for (int r = 0; r < NTEX; r++) if (S.tex[1][r] && h_ser(S.tex[1][r]) == g_bloomq.b0_ser) S.tex[1][r] = g_bloomq.b1_h; PD("D %llu bloom 1/4 composite reads B1 instead", g_stat_pass); g_bloomq_last_b0 = g_bloomq.b0_ser; g_bloomq_last_b1h = g_bloomq.b1_h; g_bloomq_last_b1s = g_bloomq.b1_ser; bloomq_reset(); }
|
||||
else { RED_ONCE(63, "bloom 1/4 state lacks the B1 handle"); bloomq_reset(); }
|
||||
} else if (!g_bloomq.state && g_bloomq_last_b1h && h_ser(g_bloomq_last_b1h) == g_bloomq_last_b1s && !strncmp(f->key, "2e2d6700a2569795", 16)) { // The B1 handle still points to the same texture (not recognized after a rebuild on resolution change)
|
||||
// The bloom overlay is reached without the bloom chain having run this frame (the extrapolated frame of runtime extrapolation skipped it: outside the region the input 469 is the previous frame's LDR, so it cannot run): redirect the read to B1 as last time, overlaying the previous frame's bloom.
|
||||
// Without the redirect it would read B0 -- with bloom 1/4 on, B0's upsample is always skipped and never written, so what is read is uninitialized video memory (on M4 Pro the units in the region turn entirely green; elsewhere it is mostly 0, i.e. missing bloom)
|
||||
int hit = 0; for (int r = 0; r < 128; r++) if (S.tex[1][r] && h_ser(S.tex[1][r]) == g_bloomq_last_b0) { S.tex[1][r] = g_bloomq_last_b1h; hit = 1; }
|
||||
int hit = 0; for (int r = 0; r < NTEX; r++) if (S.tex[1][r] && h_ser(S.tex[1][r]) == g_bloomq_last_b0) { S.tex[1][r] = g_bloomq_last_b1h; hit = 1; }
|
||||
if (hit) { PD("D %llu bloom chain did not run: composite reuses the previous frame's B1", g_stat_pass); static uint64_t n; if (n++ % 600 == 0) MLOG("bloom 1/4: a frame without the bloom chain; the composite reuses the previous frame's B1 (#%llu)", (unsigned long long)n); }
|
||||
} else if (g_bloomq.state) { RED_ONCE(61, "bloom 1/4 start: another draw came in the middle of the chain; this frame's bloom is incomplete"); bloomq_reset(); }
|
||||
}
|
||||
@@ -5967,6 +6100,8 @@ static void clear_tex(const p_clear_tex *c) {
|
||||
unsigned lv = c->has_view && c->view.set_levels ? c->view.base_level : 0, l0 = c->has_view && c->view.set_layers ? c->view.min_layer : 0;
|
||||
NSUInteger w = MAX(t.width >> lv, 1u), h = MAX(t.height >> lv, 1u);
|
||||
int is3d = t.textureType == MTLTextureType3D, nz = c->r.depth ? c->r.depth : 1;
|
||||
if (!(t.usage & MTLTextureUsageRenderTarget) && !mtl_fmt_block(t.pixelFormat) && t.sampleCount == 1 && !c->r.xoffset && !c->r.yoffset && (NSUInteger)c->r.width == w && (NSUInteger)c->r.height == h && (c->mask & 0xf) == 0xf)
|
||||
t = tex_upgrade(c->tex, t, MTLTextureUsageRenderTarget, t.pixelFormat); // a whole-level clear of a texture made without render-target use (packaged): give it that use
|
||||
if (c->has_view && c->view.set_format) { // View with changed format: the clear value is interpreted in the view format (all 5 of the game's calls are the ui version with an integer view), and the clear is done on a view of the same format
|
||||
uint32_t vpf = 0; uint8_t swz[4];
|
||||
if (!nvn_fmt_to_mtl(c->view.format, &vpf, swz) || !vpf) { RED_ONCE(42, "texture clear view format %#x has no equivalent", c->view.format); return; }
|
||||
@@ -5981,8 +6116,8 @@ static void clear_tex(const p_clear_tex *c) {
|
||||
int depth = pf == MTLPixelFormatDepth32Float || pf == MTLPixelFormatDepth16Unorm || pf == MTLPixelFormatDepth32Float_Stencil8, stencil = pf == MTLPixelFormatStencil8 || pf == MTLPixelFormatDepth32Float_Stencil8;
|
||||
if (c->r.xoffset || c->r.yoffset || (NSUInteger)c->r.width != w || (NSUInteger)c->r.height != h || (c->mask & 0xf) != 0xf
|
||||
|| !(t.usage & MTLTextureUsageRenderTarget)) {
|
||||
RED_ONCE(44, "texture clear case not implemented: region %d,%d %dx%d (level size %lux%lu) mask %#x render-target capable %d", c->r.xoffset, c->r.yoffset, c->r.width, c->r.height,
|
||||
(unsigned long)w, (unsigned long)h, c->mask, !!(t.usage & MTLTextureUsageRenderTarget));
|
||||
RED_ONCE(44, "texture clear case not implemented: region %d,%d %dx%d (level size %lux%lu) mask %#x render-target capable %d (format %#x flags %#x)", c->r.xoffset, c->r.yoffset, c->r.width, c->r.height,
|
||||
(unsigned long)w, (unsigned long)h, c->mask, !!(t.usage & MTLTextureUsageRenderTarget), c->tex->d.format, c->tex->d.flags);
|
||||
return;
|
||||
}
|
||||
rp_end(); if (g_blit) { enc_b_end(g_blit); g_blit = nil; }
|
||||
|
||||
@@ -121,8 +121,8 @@ typedef struct { int32_t stage, index; uint64_t addr, size; } p_bind_buf; // UB
|
||||
typedef struct { int32_t stage, first, n, defer; uint64_t src; uint64_t h[32]; } p_bind_handles; // Separate texture / sampler handles
|
||||
typedef struct { int32_t face, value_mask, mask, ref, which; } p_stencil; // which: 0 value mask 1 write mask 2 reference value
|
||||
typedef struct { float factor, units, clamp; } p_poly_offset;
|
||||
typedef struct { int32_t prim, first, count, instances, base_instance; uint64_t ind; } p_draw; // instances 0 counts as 1; ind != 0: DrawArraysIndirect, the arguments are read by the GPU from that address
|
||||
typedef struct { int32_t prim, itype, count, base_vertex, base_instance, instances; uint64_t ibuf, ind; } p_draw_idx; // ind != 0: DrawElementsIndirect, {count, instances, first index, base vertex, base instance} read by the GPU from that address
|
||||
typedef struct { int32_t prim, first, count, instances, base_instance; uint64_t ind; uint64_t mdi_cnt; int32_t mdi_i, pad; } p_draw; // instances 0 counts as 1; ind != 0: DrawArraysIndirect, the arguments are read by the GPU from that address; mdi_cnt != 0: draw mdi_i of an indirect-count multi-draw (drawn only while mdi_i < the u32 at mdi_cnt)
|
||||
typedef struct { int32_t prim, itype, count, base_vertex, base_instance, instances; uint64_t ibuf, ind; uint64_t mdi_cnt; int32_t mdi_i, pad; } p_draw_idx; // ind != 0: DrawElementsIndirect, {count, instances, first index, base vertex, base instance} read by the GPU from that address; mdi_cnt: as in p_draw
|
||||
typedef struct { int32_t x, y, z; uint64_t caller, ind; } p_dispatch; // caller: return address of the game call site (to locate the dispatcher); ind != 0: DispatchComputeIndirect, the group counts are read by the GPU from that address
|
||||
typedef struct { const dpool_t *pool; } p_set_pool;
|
||||
typedef struct { uint64_t src; const tex_t *tex; view_t view; uint8_t has_view; NVNcopyRegion r; int32_t flags; } p_copy_b2t;
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
#include <stdint.h>
|
||||
|
||||
enum { NVN_RB_CBUF = 1, NVN_RB_SAMP, NVN_RB_TEX, NVN_RB_IMG, NVN_RB_TBUF,
|
||||
NVN_RB_SBUFP, NVN_RB_SUPPORT, NVN_RB_C1, NVN_RB_DRV0, NVN_RB_DRV2, NVN_RB_SAMPT, NVN_RB_UBOP, NVN_RB_TEXU, NVN_RB_SAMPU };
|
||||
NVN_RB_SBUFP, NVN_RB_SUPPORT, NVN_RB_C1, NVN_RB_DRV0, NVN_RB_DRV2, NVN_RB_SAMPT, NVN_RB_UBOP, NVN_RB_TEXU, NVN_RB_SAMPU, NVN_RB_SAMPI, NVN_RB_SBUFU };
|
||||
// SAMPI: the sampler ID is a constant in the shader (reg = the ID; Ryujinx's SeparateConstantSamplerHandle: the handle ORs a constant sampler into a bound texture, Monster Hunter Rise)
|
||||
// UBOP: device pointer of compute-stage uniform slot r (the Maxwell compute stage reads uniforms via the c0 address table + global memory) // SAMPT: the sampler is taken from the combined handle of texture slot r (high bits = sampler ID) // Binding kind (type in the json); IMG = writable texture (D3D u_r, NVN image slot r); TBUF = typed buffer (D3D t_r Buffer<T>, buffer texture in NVN texture slot r)
|
||||
enum { NVN_DPF_FLOAT = 0, NVN_DPF_UINT, NVN_DPF_SINT, NVN_DPF_DEPTH }; // Dummy texture pixel class (determined by msl_type)
|
||||
enum { NVN_CT_FLOAT = 0, NVN_CT_UINT, NVN_CT_INT }; // Vertex input component type
|
||||
@@ -18,7 +19,7 @@ typedef struct {
|
||||
uint8_t kind, rw; // rw: storage buffer access is read_write / write / device
|
||||
uint8_t dtt, dpf; // Dummy texture when the texture slot is unbound: MTLTextureType value, pixel class
|
||||
uint16_t reg, aid; // D3D register number, index within the argument buffer
|
||||
uint16_t off; // TEXU / SAMPU: byte offset of the handle within the uniform buffer
|
||||
uint16_t off; // TEXU / SAMPU: byte offset of the handle within the uniform buffer; SBUFU: of the 64-bit storage buffer address
|
||||
} nvn_rbind;
|
||||
typedef struct {
|
||||
int nb, abi, nv; // Binding count, argument buffer slot, vertex input count
|
||||
@@ -34,7 +35,7 @@ typedef struct {
|
||||
int8_t pos_attr; uint8_t pos_w1, may_discard, writes_smask, color_mask, color_known; // color_known: the color list was recorded at packaging (color_mask can be trusted)
|
||||
// Merge B step 2 (readable-tile version <key>_bf, dmsh_fetch.py): fragment - nfc redirectable samples (the i-th: texture slot fc_reg, interpolated output fc_locn),
|
||||
// bitmap fc_mixed of texture slots with mixed use (these slots must not be redirected to read the tile), nfo outputs (each with color number fo_color); vertex - .xy of interpolated output locn is taken as-is from attribute uv_attr[locn] (-1 = no)
|
||||
uint8_t nfc, fc_reg[32], fc_locn[32], nfo, fo_color[4]; uint64_t fc_mixed[2]; int8_t uv_attr[32];
|
||||
uint8_t nfc, fc_reg[32], fc_locn[32], nfo, fo_color[4]; uint64_t fc_mixed[4]; int8_t uv_attr[32]; // fc_mixed: a bit per texture slot 0-255 (slots 128+ are separate texture binds, nvn_septex_base)
|
||||
// Vertex - .x / .y of interpolated output locn = fma(.x / .y of attribute uv_aff_attr[locn], scale, offset), where scale / offset reference a uniform: (slot << 8) | (vector << 2) | component
|
||||
int8_t uv_aff_attr[32]; uint16_t uv_aff[32][4]; // [sx, ox, sy, oy]
|
||||
uint8_t *c1; // c1 constant segment as-is (B): c1_len bytes, malloc'd by nvn_refl_from_json (NULL without one). It was a 4 KB array; Hyrule Warriors: Age of Calamity has 6 KB segments (its soldiers' vertex shaders)
|
||||
|
||||
@@ -36,15 +36,17 @@ int nvn_refl_from_json(NSDictionary *j, nvn_refl *o, char *err, int errlen) {
|
||||
else if ([k isEqualToString:@"tex"]) { r->kind = NVN_RB_TEX; dummy_kind(b[@"msl_type"], &r->dtt, &r->dpf); }
|
||||
else if ([k isEqualToString:@"tex_ubo"]) { r->kind = NVN_RB_TEXU; dummy_kind(b[@"msl_type"], &r->dtt, &r->dpf); r->off = (uint16_t)[b[@"off"] intValue]; o->bindless = 1; }
|
||||
else if ([k isEqualToString:@"samp_ubo"]) { r->kind = NVN_RB_SAMPU; r->off = (uint16_t)[b[@"off"] intValue]; o->bindless = 1; }
|
||||
else if ([k isEqualToString:@"samp_id"]) r->kind = NVN_RB_SAMPI;
|
||||
else if ([k isEqualToString:@"img"]) { r->kind = NVN_RB_IMG; dummy_kind(b[@"msl_type"], &r->dtt, &r->dpf); }
|
||||
else if ([k isEqualToString:@"tbuf"]) { r->kind = NVN_RB_TBUF; uint8_t tt; dummy_kind(b[@"msl_type"], &tt, &r->dpf); }
|
||||
else if ([k isEqualToString:@"sbuf"]) { r->kind = NVN_RB_SBUFP; r->rw = [b[@"rw"] boolValue]; }
|
||||
else if ([k isEqualToString:@"ubo_ptr"]) r->kind = NVN_RB_UBOP;
|
||||
else if ([k isEqualToString:@"sbuf_ubo"]) { r->kind = NVN_RB_SBUFU; r->rw = [b[@"rw"] boolValue]; r->off = (uint16_t)[b[@"off"] intValue]; }
|
||||
else if ([k isEqualToString:@"support"]) r->kind = NVN_RB_SUPPORT;
|
||||
else if ([k isEqualToString:@"c1"]) r->kind = NVN_RB_C1;
|
||||
else if ([k isEqualToString:@"drvcb"]) r->kind = [b[@"bank"] intValue] == 2 ? NVN_RB_DRV2 : NVN_RB_DRV0;
|
||||
else FAIL("B binding kind %s not implemented (%s)", k.UTF8String, [b[@"msl_name"] UTF8String]);
|
||||
if ((r->kind == NVN_RB_CBUF || r->kind == NVN_RB_SAMP || r->kind == NVN_RB_SAMPT || r->kind == NVN_RB_TEX || r->kind == NVN_RB_IMG || r->kind == NVN_RB_TBUF || r->kind == NVN_RB_SBUFP || r->kind == NVN_RB_UBOP || r->kind == NVN_RB_TEXU || r->kind == NVN_RB_SAMPU) && reg < 0)
|
||||
if ((r->kind == NVN_RB_CBUF || r->kind == NVN_RB_SAMP || r->kind == NVN_RB_SAMPT || r->kind == NVN_RB_TEX || r->kind == NVN_RB_IMG || r->kind == NVN_RB_TBUF || r->kind == NVN_RB_SBUFP || r->kind == NVN_RB_UBOP || r->kind == NVN_RB_SBUFU || r->kind == NVN_RB_TEXU || r->kind == NVN_RB_SAMPU || r->kind == NVN_RB_SAMPI) && reg < 0)
|
||||
FAIL("B binding %s has no resolved slot number", [b[@"msl_name"] UTF8String]);
|
||||
o->nb++;
|
||||
}
|
||||
@@ -89,9 +91,9 @@ int nvn_refl_from_json(NSDictionary *j, nvn_refl *o, char *err, int errlen) {
|
||||
NSDictionary *fe = j[@"fetch"];
|
||||
if ([fe isKindOfClass:[NSDictionary class]]) {
|
||||
for (NSDictionary *c in fe[@"calls"]) { int i = [c[@"fc"] intValue], r = [c[@"tex_reg"] intValue], l = [c[@"varying_locn"] intValue];
|
||||
if (i != o->nfc || i >= 32 || r < 0 || r >= 128 || l < 0 || l >= 32) FAIL("fetch call table not contiguous or out of range (entry %d)", i);
|
||||
if (i != o->nfc || i >= 32 || r < 0 || r >= 256 || l < 0 || l >= 32) FAIL("fetch call table not contiguous or out of range (entry %d)", i);
|
||||
o->fc_reg[o->nfc] = (uint8_t)r; o->fc_locn[o->nfc] = (uint8_t)l; o->nfc++; }
|
||||
for (NSNumber *r in fe[@"mixed_regs"]) if (r.intValue >= 0 && r.intValue < 128) o->fc_mixed[r.intValue >> 6] |= 1ull << (r.intValue & 63);
|
||||
for (NSNumber *r in fe[@"mixed_regs"]) if (r.intValue >= 0 && r.intValue < 256) o->fc_mixed[r.intValue >> 6] |= 1ull << (r.intValue & 63);
|
||||
for (NSDictionary *u in fe[@"outputs"]) { if (o->nfo >= 4) FAIL("more than 4 fetch outputs"); o->fo_color[o->nfo++] = (uint8_t)[u[@"color"] intValue]; }
|
||||
}
|
||||
NSArray *tg = j[@"threadgroup_size"];
|
||||
|
||||
+96
-13
@@ -43,6 +43,9 @@ def mac_compile(msl, air, lib): # same interface as ios_metal.compile
|
||||
# This game uses only combined handles (nvnCommandBufferBindTexture), with no separate samplers; the image handle table is still at 0x120 + 8 r (8 slots, right after the texture table; measured 0x120 / 0x128 / 0x130 on agl's 10 compute shaders, the same address as D2R).
|
||||
TEX_BASE, TEX_SLOTS, SAMP_BASE, IMG_BASE, IMG_SLOTS = (GAME.int(k) for k in ('nvn_tex_base', 'nvn_tex_slots', 'nvn_samp_base', 'nvn_img_base', 'nvn_img_slots'))
|
||||
SSBO = {k: int(v, 0) for k, v in GAME.pairs('nvn_ssbo_bases')} # c0 storage buffer table bases (per stage)
|
||||
# A separate texture table of its own (Monster Hunter Rise: bound / combined handles at 0x20 + 8 r, separate texture binds at 0x168 + 8 r, both in use): its slots are reported as 128 + r, where
|
||||
# the replay keeps the separate binds of such a profile (hle_nvn.c SEPTEX_SLOT0). Profiles without the key have one texture table (nvn_tex_base) for both kinds of bind
|
||||
SEPTEX_BASE, SEPTEX_SLOTS = (GAME.int('nvn_septex_base'), GAME.int('nvn_septex_slots')) if GAME.get('nvn_septex_base', '') else (None, 0)
|
||||
# The translator (dmsh_xlate/Program.cs) has the texture / sampler handle table bases built in, used only to turn a handle into a slot and look up that slot's texture / image type in the answer table (TexSlot); the bindings' slots are decoded by this script from the profile (decode_tex).
|
||||
# So the profile's bases must equal the translator's only when the answer table has per-slot texture / image facts (otherwise the wrong slot is looked up); Engage (0x168 / 0x568) has no such facts, so the translator needs no change
|
||||
_xb = re.search(r'const int TexBase = (0x[0-9a-fA-F]+), SampBase = (0x[0-9a-fA-F]+);', open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'dmsh_xlate/Program.cs')).read())
|
||||
@@ -82,14 +85,20 @@ def tool_fingerprint():
|
||||
'\0xlate\0' + dmsh_chain.xlate_source_sha() + '\0spirv-cross\0' + sc.stdout + sc.stderr +
|
||||
'\0xc\0' + repr(XC) + '\0mac\0' + MAC_RECORD + '\0ios\0' + (ios_metal.RECORD if IOSOUT else ''))
|
||||
|
||||
# The profile keys that change how a shader is built (nvn_*, shader_*), as dmsh_chain.version takes them: part of every shader's cache key
|
||||
PROFILE_ROWS = ''.join(f'{k}\t{chr(9).join(v)}\n' for k, v in GAME.rows if k.startswith(('nvn_', 'shader_'))).encode()
|
||||
def shader_key(key, name, rec):
|
||||
"""The per-shader cache key. It covers the profile's build keys and the shader's pairs; a vertex shader with pairs also takes its pixel shaders' keys (pair_keyed), since vs_pad reads their
|
||||
MSL. Before, these were only in the chain version, so a build after changing only them hit the old entries and filed them under the new version (with an empty DMSH_CAS as the workaround)"""
|
||||
pc = dmsh_facts.row(key, FACTS[key]).encode() if key in FACTS else 'missing'.encode()
|
||||
map_row = MAP_ROWS.get(key, 'missing'.encode()) + MAP_CONFLICT_ROWS.get(key, b'')
|
||||
embedded = rec if name.startswith('main.rodata@') else b''
|
||||
payload = (b'record\0' + rec + b'\0map\0' + map_row + b'\0pc\0' + pc + b'\0embedded\0' + embedded +
|
||||
b'\0tools\0' + TOOL_FP + b'\0code\0' + CODE_FP.encode() +
|
||||
b'\0rule\0' + shader_rule_source(name))
|
||||
b'\0rule\0' + shader_rule_source(name) + b'\0profile\0' + PROFILE_ROWS + b'\0pairs\0' + ','.join(sorted(PAIRS.get(key, ()))).encode())
|
||||
return hashlib.sha256(payload).hexdigest()
|
||||
def pair_keyed(digest, pixel_digests): # a vertex shader's key with its pixel shaders' keys (vs_pad)
|
||||
return hashlib.sha256((digest + '\0' + ','.join(pixel_digests)).encode()).hexdigest()
|
||||
|
||||
def cache_meta_path(key): return os.path.join(CAS, 'index', key + '.json')
|
||||
def cache_path(digest): return os.path.join(CAS, 'shader', digest)
|
||||
@@ -303,7 +312,8 @@ def raster_facts(stage, src):
|
||||
return {'pos_attr': attr[m0[0]], 'pos_w': w, 'uv_from_attr': uv, 'uv_affine': aff}
|
||||
|
||||
def decode_tex(name):
|
||||
"""fp_t_cb{slots}_{handle} / fp_t_tcb_{handle}: texture slot, sampler slot (Ryujinx TextureHandle encoding; c[2] handle table)"""
|
||||
"""fp_t_cb{slots}_{handle} / fp_t_tcb_{handle}: texture slot, sampler slot (Ryujinx TextureHandle encoding; c[2] handle table). Handle type 3 (SeparateConstantSamplerHandle) carries a constant
|
||||
sampler ID instead of a second offset, so only the first offset can name the texture (const_sampler gives the ID)"""
|
||||
m = re.match(r'[a-z]+_[ti]\w*?_(?:cb(\d+)|tcb)_([0-9A-F]+)', name)
|
||||
if not m: return None
|
||||
slots = int(m.group(1)) if m.group(1) else -1; h = int(m.group(2), 16)
|
||||
@@ -313,20 +323,31 @@ def decode_tex(name):
|
||||
words = [o0] + ([o1] if ty else [])
|
||||
if re.match(r'[a-z]+_i', name): # image
|
||||
return next((slot(x, IMG_BASE, tb) for x in words if IMG_BASE <= x * 4 < IMG_BASE + 8 * IMG_SLOTS and slot(x, IMG_BASE, tb) is not None), None), None, ty
|
||||
t = next((slot(x, TEX_BASE, tb) for x in words if slot(x, TEX_BASE, tb) is not None and x * 4 < TEX_BASE + 8 * TEX_SLOTS), None)
|
||||
s = next((slot(x, SAMP_BASE, sb) for x in words if x * 4 >= SAMP_BASE and slot(x, SAMP_BASE, sb) is not None), None)
|
||||
tw = [o0] if ty == 3 else words
|
||||
t = next((slot(x, TEX_BASE, tb) for x in tw if slot(x, TEX_BASE, tb) is not None and x * 4 < TEX_BASE + 8 * TEX_SLOTS), None)
|
||||
if t is None and SEPTEX_BASE is not None:
|
||||
t = next((128 + slot(x, SEPTEX_BASE, tb) for x in tw if slot(x, SEPTEX_BASE, tb) is not None and x * 4 < SEPTEX_BASE + 8 * SEPTEX_SLOTS), None)
|
||||
s = next((slot(x, SAMP_BASE, sb) for x in words if x * 4 >= SAMP_BASE and slot(x, SAMP_BASE, sb) is not None), None) if ty != 3 else None
|
||||
return t, s, ty
|
||||
|
||||
def const_sampler(name):
|
||||
"""The constant sampler ID of a type-3 handle (fp_t_cb65538_3041005C: the texture handle at c2 word 0x5C ORed with sampler 0x104), or None"""
|
||||
m = re.match(r'[a-z]+_t\w*?_(?:cb(\d+)|tcb)_([0-9A-F]+)', name)
|
||||
if not m: return None
|
||||
h = int(m.group(2), 16)
|
||||
return (h >> 14) & 0x3fff if h >> 28 == 3 else None
|
||||
|
||||
def decode_ubo_tex(name):
|
||||
"""Bindless texture: the handle is written by the game into its own uniform buffer (fp_t_cb7_2A = c[7] word 0x2A). Returns (texture: (uniform slot, byte offset), sampler: (slot, offset));
|
||||
handle type 0 = combined handle (one 32-bit word: low 20 bits texture ID, high 12 bits sampler ID), 1 = the sampler handle is in another word (its slot is the high 16 bits of slots − 1). This game's whole library uses only type 0 (1850 shaders)"""
|
||||
handle type 0 = combined handle (one 32-bit word: low 20 bits texture ID, high 12 bits sampler ID), 1 = the sampler handle is in another word (its slot is the high 16 bits of slots − 1). This game's whole library uses only type 0 (1850 shaders)
|
||||
Type 3: the sampler is a constant ID (Monster Hunter Rise: a texture handle at c[5] word 0x28 with sampler 0x104); the sampler part is then ('id', ID)"""
|
||||
m = re.match(r'[a-z]+_t\w*?_cb(\d+)_([0-9A-F]+)', name)
|
||||
if not m: return None
|
||||
slots, h = int(m.group(1)), int(m.group(2), 16)
|
||||
tb = slots & 0xffff; sb = (slots >> 16) - 1 if slots >> 16 else tb
|
||||
o0, o1, ty = h & 0x3fff, (h >> 14) & 0x3fff, h >> 28
|
||||
if not 3 <= tb <= 16 or ty not in (0, 1) or (ty and not 3 <= sb <= 16): return None
|
||||
return (tb - 3, o0 * 4), ((sb - 3, o1 * 4) if ty else (tb - 3, o0 * 4))
|
||||
if not 3 <= tb <= 16 or ty not in (0, 1, 3) or (ty == 1 and not 3 <= sb <= 16): return None
|
||||
return (tb - 3, o0 * 4), (('id', o1) if ty == 3 else (sb - 3, o1 * 4) if ty else (tb - 3, o0 * 4))
|
||||
|
||||
def reflect(key, stage, msl, res):
|
||||
"""MSL argument buffer struct member → replay reflection: kind (same as nvn_refl.h) + NVN slot number + argument buffer member index"""
|
||||
@@ -344,11 +365,11 @@ def reflect(key, stage, msl, res):
|
||||
e.update(kind='sbuf', index=int(re.fullmatch(r'[a-z]+_s(\d+)', nm).group(1)))
|
||||
elif decode_ubo_tex(nm[:-5] if nm.endswith('Smplr') else nm): # bindless: the handle is in a uniform buffer (reg = uniform slot, off = byte offset)
|
||||
ut, us = decode_ubo_tex(nm[:-5] if nm.endswith('Smplr') else nm)
|
||||
if nm.endswith('Smplr'): e.update(kind='samp_ubo', reg=us[0], off=us[1])
|
||||
if nm.endswith('Smplr'): e.update(kind='samp_id', reg=us[1]) if us[0] == 'id' else e.update(kind='samp_ubo', reg=us[0], off=us[1])
|
||||
elif 'texture_buffer' in ty: e.update(kind='?') # bindless typed buffer: not implemented in the replay layer
|
||||
else: e.update(kind='tex_ubo', reg=ut[0], off=ut[1])
|
||||
elif nm.endswith('Smplr'): # handle type 0 = texture and sampler combined in one handle: the sampler index is taken from the high bits of that texture slot's handle
|
||||
t, s, ty = decode_tex(nm[:-5]); e.update(kind='samp', reg=s) if ty else e.update(kind='samp_tex', reg=t)
|
||||
t, s, ty = decode_tex(nm[:-5]); e.update(kind='samp_id', reg=const_sampler(nm[:-5])) if ty == 3 else e.update(kind='samp', reg=s) if ty else e.update(kind='samp_tex', reg=t)
|
||||
else:
|
||||
d = decode_tex(nm)
|
||||
if not d: e.update(kind='?')
|
||||
@@ -363,12 +384,14 @@ def reflect(key, stage, msl, res):
|
||||
off = b['cb_offset'] * 4 if b and b['cb_slot'] == 0 else None
|
||||
if stage == 'cs' and off is not None and 0x210 <= off < 0x310 and (off - 0x210) % 16 == 0:
|
||||
e.update(kind='ubo_ptr', reg=(off - 0x210) // 16) # compute-stage uniform slot r: address at c0[0x210 + 0x10 r], read through global memory (checked against cs_lut_combine_c reading cb10[66])
|
||||
elif b and b['cb_slot'] >= 3: # the address is in the game's own uniform buffer c[n] = uniform slot n - 3 (Monster Hunter Rise's compute shaders: c3 + 0x38); the replay reads it there
|
||||
e.update(kind='sbuf_ubo', reg=b['cb_slot'] - 3, off=b['cb_offset'] * 4)
|
||||
else: e['reg'] = (off - base) // 16 if off is not None and base is not None and 0 <= off - base < 0x100 and (off - base) % 16 == 0 else None
|
||||
e['rw'] = bool(b and b['flags'] & 2)
|
||||
return binds
|
||||
|
||||
DRV_FILL = {0: [(b, b + 0x100) for b in sorted(SSBO.values())], # c0: storage buffer address tables (vertex / compute / pixel, 16 slots × 0x10 each)
|
||||
2: [(TEX_BASE, TEX_BASE + 8 * TEX_SLOTS), (IMG_BASE, IMG_BASE + 8 * IMG_SLOTS), (SAMP_BASE, SAMP_BASE + 8 * 128)]}
|
||||
2: [(TEX_BASE, TEX_BASE + 8 * TEX_SLOTS), (IMG_BASE, IMG_BASE + 8 * IMG_SLOTS), (SAMP_BASE, SAMP_BASE + 8 * 128)] + ([(SEPTEX_BASE, SEPTEX_BASE + 8 * SEPTEX_SLOTS)] if SEPTEX_BASE is not None else [])}
|
||||
# c2: texture handle table (Mario 0x20 + 8 r, 32 slots; Engage 0x168 + 8 r, 128 slots), image handle table (0x120 + 8 r, 8 slots), separate sampler handle table (0x568 + 8 r, 128 slots; the replay layer fills them all)
|
||||
|
||||
def drv_reads(src):
|
||||
@@ -469,9 +492,59 @@ def vs_depth(src):
|
||||
single = src[:m.start()] + gl
|
||||
return single, single + '\n' + re.sub(r'\bmain0\s*\(', 'main0_z(', body, count=1)
|
||||
|
||||
SG_OPS = {333: 'true', 334: 'pred', 335: 'pred', 336: 'true', 337: 'value', 338: 'value', 339: 'ballot', 345: 'value', 346: 'value', 347: 'value', 348: 'value'} # OpGroupNonUniform* → its one-lane value
|
||||
SG_BUILTINS = {36: 'one', 41: 'zero', 4416: 'one4', 4417: 'one4', 4418: 'zero4', 4419: 'one4', 4420: 'zero4'} # SubgroupSize, SubgroupLocalInvocationId, Eq / Ge / Gt / Le / Lt masks
|
||||
def spv_single_lane(path):
|
||||
"""Metal has no SIMD-group functions in vertex functions, and SPIRV-Cross's subgroup emulation covers only Elect. Monster Hunter Rise's vertex shaders (about 4%) scalarize uniform loads:
|
||||
ballot the active lanes, shuffle the first lane's value to all, and take a fast path when all agree. With a subgroup of one lane that is exact: the ballot is the lane's own bit, shuffles and
|
||||
broadcasts return the lane's own value, all / any are the predicate, all-equal and elect are true, the lane ID is 0, the size 1 and the masks follow. The SPIR-V is rewritten that way
|
||||
(the builtin variables, their decorations and the subgroup capabilities removed). → True when rewritten, False when there is nothing to do or an operation it does not cover"""
|
||||
d = open(path, 'rb').read(); words = list(struct.unpack(f'<{len(d) // 4}I', d)); hdr, ins, i = words[:5], [], 5
|
||||
while i < len(words): n = words[i] >> 16; ins.append(words[i:i + n]); i += n
|
||||
op = lambda x: x[0] & 0xffff
|
||||
if not any(op(x) in SG_OPS or (op(x) == 71 and len(x) > 3 and x[2] == 11 and x[3] in SG_BUILTINS) for x in ins): return False
|
||||
if any(340 <= op(x) <= 366 and op(x) not in SG_OPS for x in ins): return False # other subgroup operations (bit counts, arithmetic): not covered
|
||||
bvar = {x[1]: SG_BUILTINS[x[3]] for x in ins if op(x) == 71 and len(x) > 3 and x[2] == 11 and x[3] in SG_BUILTINS}
|
||||
uint = next((x[1] for x in ins if op(x) == 21 and x[2:4] == [32, 0]), None); boolt = next((x[1] for x in ins if op(x) == 20), None)
|
||||
uvec4 = next((x[1] for x in ins if op(x) == 23 and x[2] == uint and x[3] == 4), None)
|
||||
if uint is None or boolt is None or uvec4 is None: return False
|
||||
bound = hdr[3]; c = {}
|
||||
for nm in ('zero', 'one', 'zero4', 'one4', 'true'): c[nm] = bound; bound += 1
|
||||
consts = [[43 | 4 << 16, uint, c['zero'], 0], [43 | 4 << 16, uint, c['one'], 1], [44 | 7 << 16, uvec4, c['zero4'], c['zero'], c['zero'], c['zero'], c['zero']],
|
||||
[44 | 7 << 16, uvec4, c['one4'], c['one'], c['zero'], c['zero'], c['zero']], [41 | 3 << 16, boolt, c['true']]]
|
||||
out, placed = [], False
|
||||
for x in ins:
|
||||
o = op(x)
|
||||
if o == 17 and x[1] in (61, 62, 63, 64, 65, 66, 67, 68, 4423, 4431): continue # GroupNonUniform* / SubgroupBallotKHR / SubgroupVoteKHR capabilities
|
||||
if o == 10 and (b'ballot' in struct.pack(f'<{len(x) - 1}I', *x[1:]) or b'subgroup' in struct.pack(f'<{len(x) - 1}I', *x[1:])): continue
|
||||
if o in (71, 5) and x[1] in bvar: continue # the builtin's decoration and name
|
||||
if o == 59 and x[2] in bvar: continue # the builtin variable itself
|
||||
if o == 15: x = _ep_drop(x, bvar)
|
||||
if o == 54 and not placed: out += consts; placed = True
|
||||
if o == 61 and x[3] in bvar: out.append([83 | 4 << 16, x[1], x[2], c[bvar[x[3]]]]); continue # a load of a builtin → its constant
|
||||
if o in SG_OPS:
|
||||
kind = SG_OPS[o]
|
||||
if kind == 'ballot': out.append([169 | 6 << 16, x[1], x[2], x[4], c['one4'], c['zero4']]); continue # OpSelect(pred, own bit, 0)
|
||||
out.append([83 | 4 << 16, x[1], x[2], c['true'] if kind == 'true' else x[4]]); continue
|
||||
out.append(x)
|
||||
if not placed: return False
|
||||
if any(op(x) == 65 and len(x) > 3 and x[3] in bvar for x in out): return False # reached through an access chain: not covered
|
||||
hdr[3] = bound
|
||||
flat = hdr + [v for x in out for v in x]; open(path, 'wb').write(struct.pack(f'<{len(flat)}I', *flat))
|
||||
return True
|
||||
def _ep_drop(x, ids): # OpEntryPoint without the given interface IDs: model, function, the name (a nul-terminated string in whole words), then the interface list
|
||||
k = 3
|
||||
while k < len(x):
|
||||
k += 1
|
||||
if any((x[k - 1] >> (8 * b)) & 0xff == 0 for b in range(4)): break # the word holding the string's nul ends the name
|
||||
y = x[:k] + [v for v in x[k:] if v not in ids]
|
||||
y[0] = len(y) << 16 | (x[0] & 0xffff)
|
||||
return y
|
||||
|
||||
def finish(key, w, j, c1):
|
||||
name, k = loc[key]
|
||||
msl = os.path.join(w, key + '.metal'); lib = os.path.join(out, key + '_b.metallib')
|
||||
if j['stage'] == 'vs': spv_single_lane(os.path.join(w, key + '.spv')) # one-lane subgroups (see spv_single_lane)
|
||||
r = subprocess.run(['spirv-cross', *XC, os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True)
|
||||
if r.returncode: return 'MSL failed', r.stderr.strip()[:160], None
|
||||
src = open(msl).read()
|
||||
@@ -481,6 +554,7 @@ def finish(key, w, j, c1):
|
||||
j['attr'].update(extra); jp = os.path.join(w, f'job1_{key}.json'); json.dump([j], open(jp, 'w'))
|
||||
rx = subprocess.run([XLATE, jp, w], capture_output=True, text=True)
|
||||
if rx.returncode or not os.path.exists(os.path.join(w, key + '.spv')): return 'translation failed', ('integer-input retranslation: ' + rx.stderr.strip())[:160], None
|
||||
spv_single_lane(os.path.join(w, key + '.spv'))
|
||||
r = subprocess.run(['spirv-cross', *XC, os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True)
|
||||
if r.returncode: return 'MSL failed', r.stderr.strip()[:160], None
|
||||
src = open(msl).read()
|
||||
@@ -526,7 +600,7 @@ for _spec in [cache_root + ('=' + os.path.dirname(IOSOUT) if IOSOUT else '')] +
|
||||
for _ln in MTL.fill(os.path.join(_mac, _b), os.path.join(_ios, _b) if _ios else None): print(_ln, flush=True)
|
||||
packs, jobs, c1s = {}, {}, {}
|
||||
digests, rule_fps, cache_reasons, cache_hits = {}, {}, {}, set()
|
||||
for key in keys:
|
||||
def rec_of(key): # → (package name, the shader's record)
|
||||
name, k = loc[key]
|
||||
if name.startswith('main.rodata@'):
|
||||
if 'main' not in packs:
|
||||
@@ -539,7 +613,16 @@ for key in keys:
|
||||
else:
|
||||
if name not in packs: packs[name] = records(open(os.path.join(dm_dir, name), 'rb').read())
|
||||
rec = packs[name][k]
|
||||
digests[key] = shader_key(key, name, rec)
|
||||
return name, rec
|
||||
recs = {}
|
||||
for key in keys: recs[key] = rec_of(key); digests[key] = shader_key(key, *recs[key])
|
||||
def plain_digest(k): # a paired pixel shader's key, also when it is not built this time
|
||||
if k in digests: return digests[k]
|
||||
return shader_key(k, *rec_of(k)) if k in loc else 'missing'
|
||||
for key in keys:
|
||||
if PAIRS.get(key): digests[key] = pair_keyed(digests[key], [plain_digest(p) for p in sorted(PAIRS[key])])
|
||||
for key in keys:
|
||||
name, rec = recs[key]
|
||||
rule_fps[key] = hashlib.sha256(shader_rule_source(name) + ','.join(sorted(PAIRS.get(key, ()))).encode()).hexdigest() # the vertex shader must be rebuilt when its pairs change (vs_pad)
|
||||
hit, cache_reasons[key] = cache_lookup(key, digests[key], rule_fps[key], CODE_FP)
|
||||
if hit: cache_hits.add(key)
|
||||
@@ -580,7 +663,7 @@ ok = [r[3] for r in res if r[1] == 'ok']
|
||||
rd = [x for x in ok if x['drv_reads']]
|
||||
print(f'shaders reading driver segments at run time {len(rd)} (c0 {sum("0" in x["drv_reads"] for x in rd)}, c2 {sum("2" in x["drv_reads"] for x in rd)}); reading what the replay layer cannot fill {sum(bool(x["drv_unfillable"]) for x in ok)}')
|
||||
for e, n in collections.Counter(u for x in ok for u in x['drv_unfillable']).most_common(15): print(f' cannot fill {n:5d} {e}')
|
||||
SLOTK = {'cbuf', 'samp', 'samp_tex', 'tex', 'img', 'tbuf', 'sbuf', 'ubo_ptr', 'tex_ubo', 'samp_ubo'}
|
||||
SLOTK = {'cbuf', 'samp', 'samp_tex', 'tex', 'img', 'tbuf', 'sbuf', 'ubo_ptr', 'sbuf_ubo', 'tex_ubo', 'samp_ubo', 'samp_id'}
|
||||
noreg = [x for x in ok if any(b['kind'] in SLOTK and b.get('reg') is None for b in x['bindings'])]
|
||||
unk = [x for x in ok if any(b['kind'] == '?' for b in x['bindings'])]
|
||||
print(f'shaders with a binding whose slot was not decoded {len(noreg)}, with an unrecognized binding kind {len(unk)} (loading them at run time prints [RED] lines)')
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Translation chain tool fingerprints (written by dmsh_chain.py --update-lock; regenerate after changing tools and commit it)
|
||||
dmsh_xlate.dll a7f696bb0dbf1ac0
|
||||
Ryujinx.Graphics.Shader.dll 011c50cb1c08e44f
|
||||
Ryujinx.Graphics.Shader.dll a4b556413e6a98f5
|
||||
Ryujinx.Common.dll 5222da62ffb43db1
|
||||
xlate_source 38a1c0b4e3ebd5fa
|
||||
spirv-cross d9dc197f061b42a5
|
||||
|
||||
Binary file not shown.
@@ -16,6 +16,7 @@
|
||||
# NV assembly text follows the first 0x880 bytes of the control section, and SASS disassembly text follows the code area. All 478 files hold "vertex, pixel" in pairs (1, 2 or 4 pairs), paired two by two in order
|
||||
# · G2S (Koei Tecmo's KTGL, Hyrule Warriors: Age of Calamity): GLSLC compiler outputs inside the IDRK entries of the RDB data files (zlib in chunks), see idrk_shaders
|
||||
# · SPKG (the older KTGL of Fire Emblem: Three Houses): shader packages of G2S containers, loose and inside the LINKDATA archive DATA0.bin / DATA1.bin, see kt_shaders and linkdata_jobs
|
||||
# · SDF (Capcom's RE Engine, Monster Hunter Rise): shader definitions inside the PAK archives (KPKA, entries zstd / deflate compressed), only the newest copy of each path, see kpka_jobs and sdf_shaders
|
||||
# Code area: 0x30 header + (graphics) 0x50 SPH + instructions; header magic 0x12345678 for graphics, 0x12345679 for compute. The control section starts with 0x98761234, +0x704 = code area length,
|
||||
# +0x700 = start of the c1 constant segment within the code area; for compute +0x748 thread group xyz (u32 ×3), +0x754 shared memory bytes (checked one by one against PC reflection on D2R's 22 compute shaders).
|
||||
# Stage: BNSH uses the stage slot; BAHS uses bits 10..13 of SPH word 0 (1 vertex 2 hull 3 domain 4 geometry 5 pixel), and a compute header means compute.
|
||||
@@ -216,6 +217,71 @@ def linkdata_shaders(path, rel, rng, out):
|
||||
off, n, stored, comp = struct.unpack_from('<QQQQ', idx, i * 32)
|
||||
if n: kt_shaders(d[off:off + (stored if comp else n)], f'/{rel}#{i}', out)
|
||||
|
||||
KPKA = b'KPKA'
|
||||
SDF_STAGES = ['vs', 'hs', 'ds', 'gs', 'ps', 'cs'] # an SDF entry's stage offsets, in BNSH order
|
||||
def kpka_entries(path):
|
||||
"""Capcom's RE Engine PAK (Monster Hunter Rise: re_chunk_000.pak plus 16 .patch_NNN.pak, 19.6 GB): 'KPKA', u8 version 4, u8 0, u16 features (0: the table is not encrypted), u32 count, u32 fingerprint,
|
||||
then count 0x30-byte entries {u32 hash of the lower-case path, u32 hash of the upper-case path, u64 offset, u64 stored length, u64 length, u64 attributes, u64 checksum}.
|
||||
Attribute bits 0-3: 0 stored, 1 raw deflate, 2 zstd. → [(path hash, offset, stored length, length, attributes)]"""
|
||||
with open(path, 'rb') as f:
|
||||
h = f.read(16)
|
||||
if h[:4] != KPKA: return None
|
||||
ver, feat, n = h[4], struct.unpack_from('<H', h, 6)[0], struct.unpack_from('<I', h, 8)[0]
|
||||
if ver != 4 or feat: red(f'{path}: PAK version {ver}, features {feat:#x}: only version 4 with a plain entry table is known')
|
||||
t = f.read(0x30 * n)
|
||||
return [(lo | hi << 32, off, csz, usz, attr) for lo, hi, off, csz, usz, attr, _ in struct.iter_unpack('<IIqqqqQ', t)]
|
||||
|
||||
def kpka_data(d, e, head=None):
|
||||
"""An entry's data (head: only its first bytes)"""
|
||||
import zlib
|
||||
_, off, csz, usz, attr = e; raw = d[off:off + csz]
|
||||
if attr & 0xf == 0: return raw[:head] if head else raw
|
||||
if attr & 0xf == 1: return zlib.decompressobj(-15).decompress(raw, head or 0)
|
||||
if attr & 0xf == 2:
|
||||
from compression import zstd
|
||||
if not head: return zstd.decompress(raw)
|
||||
return zstd.ZstdDecompressor().decompress(raw, max_length=head)
|
||||
red(f'PAK entry {e[0]:016x}: compression {attr & 0xf} is not known')
|
||||
|
||||
def kpka_jobs(files, parts=64):
|
||||
"""The game reads its PAKs as one view: a path in a later patch PAK (re_chunk_000.pak.patch_NNN.pak, in name order after re_chunk_000.pak) replaces the same path hash in earlier ones.
|
||||
Only the newest copy of each path is scanned, so shaders of replaced files do not enter the tables. → ([(path, rel, entries), …] in about equal stored sizes, the PAK paths taken)"""
|
||||
paks = [(p, r) for p, r in files if p.endswith('.pak') and kpka_entries(p) is not None]
|
||||
if not paks: return [], set()
|
||||
view = {}
|
||||
for p, r in sorted(paks, key=lambda x: x[1]):
|
||||
for e in kpka_entries(p): view[e[0]] = (p, r, e)
|
||||
jobs, cur, size, step = [], collections.defaultdict(list), 0, sum(e[2] for _, _, e in view.values()) / parts
|
||||
for p, r, e in sorted(view.values(), key=lambda x: (x[1], x[2][1])):
|
||||
cur[(p, r)].append(e); size += e[2]
|
||||
if size >= step: jobs += [(p_, r_, tuple(es)) for (p_, r_), es in cur.items()]; cur, size = collections.defaultdict(list), 0
|
||||
jobs += [(p_, r_, tuple(es)) for (p_, r_), es in cur.items()]
|
||||
return jobs, {p for p, _ in paks}
|
||||
|
||||
def sdf_shaders(d, label):
|
||||
"""RE Engine shader definition ('SDF\\0', 2,685 entries in Monster Hunter Rise's PAKs): u16 version 1, u16 entry count, u64, then 0x108-byte entries from +0x10. Entry +0x10: six u64 offsets
|
||||
(vertex, hull, domain, geometry, pixel, compute; 0 = no stage), each to {u32 control length, u32 code length} followed by the control section and the code area. Compute code has no header.
|
||||
The stages of one entry form a program, so its vertex / pixel stages are a pair"""
|
||||
ver, n = struct.unpack_from('<HH', d, 4); out = []
|
||||
if not ver and not n: return out # an empty definition: 16 bytes, version 0, no entries
|
||||
if ver != 1: red(f'{label}: SDF version {ver}')
|
||||
for i in range(n):
|
||||
offs = struct.unpack_from('<6Q', d, 0x10 + 0x108 * i + 0x10)
|
||||
for st, o in zip(SDF_STAGES, offs):
|
||||
if not o: continue
|
||||
nctl, ncode = struct.unpack_from('<II', d, o); ctl, code = d[o + 8:o + 8 + nctl], d[o + 8 + nctl:o + 8 + nctl + ncode]
|
||||
why = check(label, st, ctl, code)
|
||||
if not why and st != 'cs' and SPH_STAGE.get((struct.unpack_from('<I', code, 0x30)[0] >> 10) & 0xf) != st: why = 'the SPH stage does not match the entry slot'
|
||||
if why: red(f'{label} SDF entry {i} {st}: {why}')
|
||||
out.append((f'{label}#SDF#var{i}', st, ctl, code))
|
||||
return out
|
||||
|
||||
def kpka_shaders(path, rel, ents, out):
|
||||
import mmap
|
||||
with open(path, 'rb') as f, mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as d:
|
||||
for e in ents:
|
||||
if e[3] >= 0x10 and kpka_data(d, e, 4) == b'SDF\0': out.extend(sdf_shaders(kpka_data(d, e), f'/{rel}#{e[0]:016x}'))
|
||||
|
||||
def yaz0(s):
|
||||
"""Yaz0 decompression (Super Mario 3D World + Bowser's Fury: almost all of RomFS is .szs = SARC wrapped in Yaz0). Pure Python runs at about 20–45 MB/s per core, under a minute for the whole 4.7 GB RomFS in parallel, and needs no extra dependency.
|
||||
Group header bits: 1 = one literal byte; 0 = back-reference, 2 bytes {length high nibble, 12-bit distance} (distance + 1); length 0 reads one more byte + 0x12, else + 2. Overlapping copies are expanded by their period"""
|
||||
@@ -372,6 +438,9 @@ def scan_file(args):
|
||||
if head[:8] == IDRK:
|
||||
idrk_shaders(path, '/' + rel, found)
|
||||
return rel, uniq_keys(found), var_pairs(found), trees
|
||||
if len(args) > 2 and head[:4] == KPKA: # entries of an RE Engine PAK (kpka_jobs)
|
||||
kpka_shaders(path, rel, args[2], found)
|
||||
return rel, uniq_keys(found), var_pairs(found), trees
|
||||
if len(args) > 2: # a range of LINKDATA entries (linkdata_jobs)
|
||||
linkdata_shaders(path, rel, args[2], found)
|
||||
return rel, uniq_keys(found), var_pairs(found), trees
|
||||
@@ -435,7 +504,9 @@ def main():
|
||||
# First pass: every file; Unity bundles carry type trees, and the Shader trees are collected on the way. Second pass: Unity serialized files without type trees are scanned again with the borrowed trees
|
||||
J = int(os.environ.get('J', '8'))
|
||||
big = [j for p, r in files for j in (linkdata_jobs(p, r) or [])] # Three Houses' 6.9 GB DATA1.bin: one job per range of entries, not one read of the whole file
|
||||
files = [(p, r) for p, r in files if not linkdata_jobs(p, r)]
|
||||
pak_jobs, paks = kpka_jobs(files) # Monster Hunter Rise's PAKs: the newest copy of each path, in ranges of entries
|
||||
big += pak_jobs
|
||||
files = [(p, r) for p, r in files if not linkdata_jobs(p, r) and p not in paks]
|
||||
with cf.ProcessPoolExecutor(J, initializer=_pool_init, initargs=(common, {}, zdicts)) as ex:
|
||||
results += list(ex.map(scan_file, big)) + list(ex.map(scan_file, files, chunksize=16))
|
||||
trees, defer = {}, []
|
||||
@@ -477,8 +548,8 @@ def main():
|
||||
for name, rr in recs.items():
|
||||
with open(os.path.join(pkg, name), 'wb') as f: f.write(b''.join(rr))
|
||||
with open(mapout, 'w') as f:
|
||||
kinds = ' / '.join(k for k, hit in (('BNSH / BAHS', any('#BNSH@' in r[4] or '#BAHS@' in r[4] for r in rows)), ('Unity Shader', any('#var' in r[4] and '#BNSH@' not in r[4] and '#MSHD#' not in r[4] for r in rows)),
|
||||
('MSHD', any('#MSHD#' in r[4] for r in rows)), ('GLSLC', any('#GLSLC@' in r[4] for r in rows))) if hit)
|
||||
kinds = ' / '.join(k for k, hit in (('BNSH / BAHS', any('#BNSH@' in r[4] or '#BAHS@' in r[4] for r in rows)), ('Unity Shader', any('#var' in r[4] and '#BNSH@' not in r[4] and '#MSHD#' not in r[4] and '#SDF#' not in r[4] for r in rows)),
|
||||
('MSHD', any('#MSHD#' in r[4] for r in rows)), ('GLSLC', any('#GLSLC@' in r[4] for r in rows)), ('RE Engine SDF', any('#SDF#' in r[4] for r in rows))) if hit)
|
||||
f.write(f'# kind: evidence {NOTE} ({kinds} in RomFS + main) → control section + code area keys (dm/gfx/tools/shader_split.py)\n')
|
||||
f.write('# column 5 records where each key first appears; this game has no PC version to pair with\n')
|
||||
f.write('package\tindex\tstage\tdm_ctl_sha16\tsource\n')
|
||||
|
||||
+9
-1
@@ -29,7 +29,7 @@ enum { DIRMODE_DIR = 1, DIRMODE_FILE = 2 };
|
||||
typedef struct { char name[0x301]; uint8_t pad0[3]; int8_t type; uint8_t pad1[3]; int64_t size; } DirectoryEntry;
|
||||
_Static_assert(sizeof(DirectoryEntry) == 0x310, "DirectoryEntry layout");
|
||||
|
||||
#define MAX_MOUNT 16
|
||||
#define MAX_MOUNT 512 // Monster Hunter Rise mounts every add-on content it owns (aoc0 … : 252 licences in the folder) right after the first-boot settings: 16 ended in "mount table full"
|
||||
// Save transactions (HLE audit items 1–3): on the real device a save can be rolled back as a whole before commit, and after a crash / kill a remount sees only the version from the last Commit.
|
||||
// Host: at mount, the committed directory commit (= $DM_SAVE) is clonefile'd as a whole tree into the working directory <commit>.work (APFS copy-on-write, nearly zero cost), and the mount points at the working directory,
|
||||
// all game reads and writes happen in the working directory; Commit = clonefile(work → <commit>.new), then renamex_np(RENAME_SWAP) swaps it atomically with commit, then the swapped-out old version is deleted.
|
||||
@@ -367,6 +367,14 @@ HLE_FORCE("_ZN2nn2fs17MountCacheStorageEPKci", "fsp-srv; cache storage → host
|
||||
HLE_FORCE("_ZN2nn2fs18DeleteCacheStorageEi", "fsp-srv; cache storage → deletes the host directory", Result, h_fs_DeleteCache, int index) {
|
||||
char b[1100]; rm_tree(cache_dir(index, b, sizeof b)); return RES_OK;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn2fs17MountCacheStorageEPKc", "fsp-srv; cache storage 0 → host directory (SDK 11 one-argument form, Monster Hunter Rise)", Result, h_fs_MountCache0, const char *name) { return h_fs_MountCache(name, 0); }
|
||||
// Save data size (Monster Hunter Rise, SDK 11): the save directory has no size of its own, so it reports 256 MB of data and journal, more than a game asks for; extending succeeds without doing anything
|
||||
HLE_FORCE("_ZN2nn2fs15GetSaveDataSizeEPlS1_RKNS_7account3UidE", "fsp-srv; save data size: 256 MB data + 256 MB journal", Result, h_fs_SaveSize, int64_t *size, int64_t *journal, const void *uid) {
|
||||
(void)uid; if (size) *size = 256 << 20; if (journal) *journal = 256 << 20; return RES_OK;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn2fs14ExtendSaveDataERKNS_7account3UidEll", "fsp-srv; extend save data: the host directory has no size limit", Result, h_fs_ExtendSave, const void *uid, int64_t size, int64_t journal) {
|
||||
(void)uid; LOG("fs extend save data to %lld bytes (journal %lld): nothing to do", (long long)size, (long long)journal); return RES_OK;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn2fs14EnsureSaveDataERKNS_7account3UidE", "fsp-srv; the save directory is created at mount time", Result, h_fs_EnsureSave, const void *uid) { (void)uid; return RES_OK; }
|
||||
// Settings defaults for the touch product (dropped item names shown permanently by default, can be turned off in settings): Settings/Settings.json in the save only records keys the player has changed, unrecorded ones use the game default.
|
||||
// Before mounting the save, fill in the touch product defaults for the two keys "the player has not touched yet"; existing keys are never changed (if the player changed it in the options page, the player wins):
|
||||
|
||||
@@ -169,6 +169,12 @@ HLE_FORCE("_ZN2nn3hid18InitializeKeyboardEv", "keyboard; not provided by the hos
|
||||
HLE_FORCE("_ZN2nn3hid16GetKeyboardStateEPNS0_13KeyboardStateE", "keyboard state: not connected (KeyboardState 0x30 bytes zeroed)", void, h_hid_KbdState, void *s) {
|
||||
memset(s, 0, 0x30); ((int64_t *)s)[0] = atomic_fetch_add(&g_sampling, 1) + 1;
|
||||
}
|
||||
// The history form (Monster Hunter Rise): every requested entry, newest first, with consecutive sampling numbers, as GetNpadStates fills them
|
||||
HLE_FORCE("_ZN2nn3hid17GetKeyboardStatesEPNS0_13KeyboardStateEi", "keyboard state history: not connected (each 0x30-byte entry zeroed)", int, h_hid_KbdStates, uint8_t *s, int n) {
|
||||
int64_t top = atomic_fetch_add(&g_sampling, 1) + 1;
|
||||
for (int i = 0; i < n; i++) { memset(s + 0x30 * i, 0, 0x30); *(int64_t *)(s + 0x30 * i) = top - i; }
|
||||
return n > 0 ? n : 0;
|
||||
}
|
||||
|
||||
// ---------- SDK 12 interfaces (Engage): single GetNpadState, mode queries / setters, touch screen, gestures, six-axis details, controller colors, etc. ----------
|
||||
// In the original all of these read hid shared memory or go through the hid service; the host has no shared memory and answers from the same front-end state as above. Library functions that only modify the game's own buffers (vibration file parse / generate, ControllerSupportArg SetDefault / SetExplainText) keep running in the SDK
|
||||
|
||||
+35
-1
@@ -77,7 +77,41 @@ HLE_FORCE("_ZN2nn3err9ShowErrorENS_6ResultE", "error display applet; the host pr
|
||||
}
|
||||
|
||||
// ---------- ngc profanity filter: the word list is in a system data save (mounted via IPC); single-player has no text visible to others, always judged as no profanity ----------
|
||||
HLE_FORCE("_ZN2nn3ngc15ProfanityFilterC1Ev", "profanity filter, the word list is in a system save; the host builds none", void, h_ngc_Ctor, void *self) { (void)self; }
|
||||
// The class is polymorphic, and games call it through its vtable too: Monster Hunter Rise checks the hunter's name with slot 8, MaskProfanityWordsInText(int *, char16_t *, uint32_t)
|
||||
// (main+0x6bca160). The constructor used to do nothing, the object kept what the heap held there, and that call jumped into the heap. It now gets a host vtable: each slot of the SDK's
|
||||
// (_ZTVN2nn3ngc15ProfanityFilterE) is matched by address with the class's methods and given the host's version (no word list: nothing is profanity, nothing is masked); others return 0
|
||||
static Result ngc_v_zero(void) { return RES_OK; }
|
||||
static void ngc_v_none(void *self) { (void)self; }
|
||||
static Result ngc_v_check(void *self, uint32_t *masks, uint32_t pat, const void **words, uint64_t n) { (void)self; (void)pat; (void)words; for (uint64_t i = 0; masks && i < n; i++) masks[i] = 0; return RES_OK; }
|
||||
static Result ngc_v_mask(void *self, int *masked, void *text, uint32_t region) { (void)self; (void)text; (void)region; if (masked) *masked = 0; return RES_OK; }
|
||||
static void *ngc_vtable(void) {
|
||||
enum { NV = 40 }; static void *vt[NV + 2]; static int built;
|
||||
if (built) return built > 0 ? &vt[2] : NULL;
|
||||
void **sdk = guest_sym("_ZTVN2nn3ngc15ProfanityFilterE");
|
||||
if (!sdk) { built = -1; LOG("[RED] ProfanityFilter: the SDK has no vtable symbol; objects keep none"); return NULL; }
|
||||
static const struct { const char *name; void *fn; } M[] = {
|
||||
{"_ZN2nn3ngc15ProfanityFilterD1Ev", (void *)ngc_v_none}, {"_ZN2nn3ngc15ProfanityFilterD0Ev", (void *)ngc_v_none}, {"_ZN2nn3ngc15ProfanityFilter8FinalizeEv", (void *)ngc_v_none},
|
||||
{"_ZN2nn3ngc15ProfanityFilter19CheckProfanityWordsEPjjPPKcm", (void *)ngc_v_check}, {"_ZN2nn3ngc15ProfanityFilter19CheckProfanityWordsEPjjPPKDsm", (void *)ngc_v_check},
|
||||
{"_ZN2nn3ngc15ProfanityFilter19CheckProfanityWordsEPjjPPKtm", (void *)ngc_v_check},
|
||||
{"_ZN2nn3ngc15ProfanityFilter24MaskProfanityWordsInTextEPiPcj", (void *)ngc_v_mask}, {"_ZN2nn3ngc15ProfanityFilter24MaskProfanityWordsInTextEPiPDsj", (void *)ngc_v_mask},
|
||||
{"_ZN2nn3ngc15ProfanityFilter24MaskProfanityWordsInTextEPiPtj", (void *)ngc_v_mask},
|
||||
};
|
||||
void *addr[sizeof M / sizeof *M]; for (unsigned k = 0; k < sizeof M / sizeof *M; k++) addr[k] = guest_sym(M[k].name);
|
||||
module_t *mod = module_of(sdk); int n = 0, named = 0;
|
||||
vt[0] = sdk[0]; vt[1] = sdk[1]; // offset to top, type info
|
||||
// A slot holds SDK code, or the host's version where the loader bound the slot by name to a forced take-over (Initialize, GetContentVersion, CheckProfanityWords): those stay.
|
||||
// The table ends at the first value that is neither (Monster Hunter Rise's SDK: 9 slots, then a 1)
|
||||
for (; n < NV && (uintptr_t)sdk[2 + n] >= 0x10000; n++) {
|
||||
if (module_of(sdk[2 + n]) != mod) { if (module_of(sdk[2 + n])) break; vt[2 + n] = sdk[2 + n]; continue; } // another guest module: not this class's table
|
||||
vt[2 + n] = (void *)ngc_v_zero;
|
||||
for (unsigned k = 0; k < sizeof M / sizeof *M; k++) if (addr[k] && sdk[2 + n] == addr[k]) { vt[2 + n] = M[k].fn; named++; break; }
|
||||
}
|
||||
built = 1; LOG("ngc: ProfanityFilter host vtable of %d slots (%d matched to the class's methods)", n, named);
|
||||
return &vt[2];
|
||||
}
|
||||
HLE_FORCE("_ZN2nn3ngc15ProfanityFilterC1Ev", "profanity filter, the word list is in a system save; the host builds none and gives the object a host vtable", void, h_ngc_Ctor, void *self) {
|
||||
void *vt = ngc_vtable(); if (vt && self) memcpy(self, &vt, sizeof vt);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn3ngc15ProfanityFilterD1Ev", "profanity filter", void, h_ngc_Dtor, void *self) { (void)self; }
|
||||
HLE_FORCE("_ZN2nn3ngc15ProfanityFilter10InitializeEPvm", "profanity filter init; the host has no word list", Result, h_ngc_Init, void *self, void *work, uint64_t sz) { (void)self; (void)work; (void)sz; return RES_OK; }
|
||||
HLE_FORCE("_ZN2nn3ngc15ProfanityFilter17GetContentVersionEv", "word list version; the host returns 0", uint32_t, h_ngc_Ver, void *self) { (void)self; return 0; }
|
||||
|
||||
+79
-19
@@ -8,6 +8,7 @@
|
||||
#include "vpx/vpx_decoder.h"
|
||||
#include "vpx/vp8dx.h"
|
||||
#include "hle_movie_avc.h" // H.264 on VideoToolbox
|
||||
#include <AudioToolbox/AudioToolbox.h> // AAC
|
||||
// Vorbis (movie audio tracks, Link's Awakening): stb_vorbis, built into this file (dm/third_party/stb_vorbis); only its push-data API, floats out
|
||||
#define STB_VORBIS_NO_STDIO
|
||||
#define STB_VORBIS_NO_PULLDATA_API
|
||||
@@ -50,7 +51,9 @@ HLE_FORCE("_ZN5movie9Extractor13SetDataSourceEPKc", "movie demuxer: the host doe
|
||||
#define MD_MODE_TEX 2 // native texture mode: the decoder writes frames into the client's NVN texture
|
||||
#define MD_TYPE_AVC 1
|
||||
#define MD_TYPE_VP9 3
|
||||
#define MD_TYPE_VORBIS 5 // audio: the SDK's Decoder accepts AAC (4) and Vorbis (5) in CPU mode, output format 1 only (interleaved PCM16)
|
||||
#define MD_TYPE_AAC 4 // audio: the SDK's Decoder accepts AAC (4) and Vorbis (5) in CPU mode, output format 1 only (interleaved PCM16)
|
||||
#define MD_TYPE_VORBIS 5
|
||||
#define MD_AUDIO(d) ((d)->type == MD_TYPE_VORBIS || (d)->type == MD_TYPE_AAC)
|
||||
#define MD_FMT_PCM16 1
|
||||
#define MD_PCM_CAP (8192 * 8 * 2) // bytes per audio output: a Vorbis packet gives at most 4096 samples per channel (blocksize 8192), 8 channels at most
|
||||
|
||||
@@ -70,6 +73,8 @@ typedef struct {
|
||||
vpx_codec_ctx_t vpx; int vpx_on, red_dec, red_img;
|
||||
// Vorbis: the identification and setup headers from Configure's MediaData ("csd-0", "csd-1", as with Android's MediaCodec), the decoder (opened at the first packet), channels / rate, Ogg page sequence
|
||||
uint8_t *vhdr[2]; int vhdr_n[2]; stb_vorbis *vb; int ch, rate; uint32_t ogg_seq; int64_t ogg_gran; int red_vb;
|
||||
// AAC (Monster Hunter Rise's movies): AudioToolbox's converter, opened at the first packet from the AudioSpecificConfig (Configure's csd-0, kept in vhdr[0]); the packet being fed
|
||||
AudioConverterRef aac; const uint8_t *aac_pkt; UInt32 aac_pkt_n; AudioStreamPacketDescription aac_desc;
|
||||
uint64_t dec_n, dec_ns, dec_max; // decode timing (on the host thread, per packet)
|
||||
unsigned long ncall[24];
|
||||
} mdec_t;
|
||||
@@ -293,16 +298,65 @@ static int md_vorbis(mdec_t *d, const uint8_t *p, int n, int slot) {
|
||||
d->out[slot].bytes = need;
|
||||
return 1;
|
||||
}
|
||||
// ---------- AAC ----------
|
||||
// The demuxer hands over raw AAC frames (1024 samples each) and the AudioSpecificConfig in csd-0. AudioToolbox takes the configuration as an MPEG-4 ES descriptor (the magic cookie built
|
||||
// here as FFmpeg's AudioToolbox decoder builds it); without csd-0 an AAC-LC configuration is made from the sample rate and channel count
|
||||
static uint8_t *esds_descr(uint8_t *p, int tag, unsigned n) { *p++ = (uint8_t)tag; for (int i = 3; i > 0; i--) *p++ = (uint8_t)(n >> (7 * i) | 0x80); *p++ = (uint8_t)(n & 0x7f); return p; }
|
||||
static OSStatus aac_feed(AudioConverterRef c, UInt32 *np, AudioBufferList *bl, AudioStreamPacketDescription **pd, void *ud) {
|
||||
(void)c; mdec_t *d = ud;
|
||||
if (!d->aac_pkt) { *np = 0; return 1; } // this packet is used up: a non-zero status ends the fill with what was decoded
|
||||
bl->mNumberBuffers = 1; bl->mBuffers[0].mData = (void *)d->aac_pkt; bl->mBuffers[0].mDataByteSize = d->aac_pkt_n; bl->mBuffers[0].mNumberChannels = (UInt32)d->ch;
|
||||
d->aac_desc = (AudioStreamPacketDescription){0, 0, d->aac_pkt_n}; if (pd) *pd = &d->aac_desc;
|
||||
*np = 1; d->aac_pkt = NULL; return 0;
|
||||
}
|
||||
static int md_aac_open(mdec_t *d) {
|
||||
static const int rates[] = {96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350};
|
||||
uint8_t asc[64]; int an = d->vhdr_n[0] > 0 && d->vhdr_n[0] <= (int)sizeof asc ? d->vhdr_n[0] : 0;
|
||||
if (an) memcpy(asc, d->vhdr[0], (size_t)an);
|
||||
if (an >= 2) { // sampling frequency index and channel configuration from the configuration itself
|
||||
int fi = (asc[0] & 7) << 1 | asc[1] >> 7, cc = asc[1] >> 3 & 15;
|
||||
if (fi < 13 && d->rate <= 0) d->rate = rates[fi]; if (cc && cc < 8 && d->ch <= 0) d->ch = cc == 7 ? 8 : cc;
|
||||
}
|
||||
if (d->rate <= 0) d->rate = 48000; if (d->ch <= 0) d->ch = 2;
|
||||
if (!an) { int fi = 3; for (int i = 0; i < 13; i++) if (rates[i] == d->rate) fi = i; asc[0] = (uint8_t)(2 << 3 | fi >> 1); asc[1] = (uint8_t)((fi & 1) << 7 | (d->ch & 15) << 3); an = 2; }
|
||||
AudioStreamBasicDescription in = {.mSampleRate = d->rate, .mFormatID = kAudioFormatMPEG4AAC, .mFramesPerPacket = 1024, .mChannelsPerFrame = (UInt32)d->ch};
|
||||
AudioStreamBasicDescription out = {.mSampleRate = d->rate, .mFormatID = kAudioFormatLinearPCM, .mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked,
|
||||
.mBytesPerPacket = 2u * d->ch, .mFramesPerPacket = 1, .mBytesPerFrame = 2u * d->ch, .mChannelsPerFrame = (UInt32)d->ch, .mBitsPerChannel = 16};
|
||||
OSStatus st = AudioConverterNew(&in, &out, &d->aac);
|
||||
if (st) { if (!d->red_vb++) LOG("[RED] movie: AudioToolbox cannot create an AAC decoder (%d; %d channels %d Hz), outputting silence", (int)st, d->ch, d->rate); d->aac = NULL; return 0; }
|
||||
uint8_t ck[128], *p = ck;
|
||||
p = esds_descr(p, 3, 3 + 5 + 13 + 5 + (unsigned)an); *p++ = 0; *p++ = 0; *p++ = 0; // ES descriptor: ES ID 0, no flags
|
||||
p = esds_descr(p, 4, 13 + 5 + (unsigned)an); *p++ = 0x40; *p++ = 0x15; memset(p, 0, 11); p += 11; // decoder configuration: MPEG-4 audio, audio stream; buffer size and bit rates 0
|
||||
p = esds_descr(p, 5, (unsigned)an); memcpy(p, asc, (size_t)an); p += an;
|
||||
st = AudioConverterSetProperty(d->aac, kAudioConverterDecompressionMagicCookie, (UInt32)(p - ck), ck);
|
||||
if (st && !d->red_vb++) LOG("[RED] movie: AAC configuration not accepted by AudioToolbox (%d)", (int)st);
|
||||
LOG("[movie] AAC: %d channels %d Hz, AudioSpecificConfig %d bytes (object type %d)", d->ch, d->rate, an, asc[0] >> 3);
|
||||
return 1;
|
||||
}
|
||||
// one packet → PCM16 interleaved in the slot; 1 when there are samples (the decoder's start-up delay may give none for the first packet)
|
||||
static int md_aac(mdec_t *d, const uint8_t *p, int n, int slot) {
|
||||
if (!d->aac && !md_aac_open(d)) return 0;
|
||||
if (n <= 0) return 0;
|
||||
if (d->out[slot].cap < MD_PCM_CAP) { free(d->out[slot].frm); d->out[slot].frm = malloc(MD_PCM_CAP); d->out[slot].cap = MD_PCM_CAP; }
|
||||
d->aac_pkt = p; d->aac_pkt_n = (UInt32)n;
|
||||
UInt32 frames = (UInt32)(MD_PCM_CAP / (2 * d->ch));
|
||||
AudioBufferList ob = {1, {{(UInt32)d->ch, frames * 2u * d->ch, d->out[slot].frm}}};
|
||||
OSStatus st = AudioConverterFillComplexBuffer(d->aac, aac_feed, d, &frames, &ob, NULL);
|
||||
d->aac_pkt = NULL;
|
||||
if (st && st != 1 && !d->red_dec++) LOG("[RED] movie: AAC decode error (AudioToolbox %d, first case)", (int)st);
|
||||
d->out[slot].bytes = (size_t)frames * 2 * d->ch;
|
||||
return frames > 0;
|
||||
}
|
||||
// H.264: decode the packet (if any) and hand out the next picture in presentation order (with jflush every queued one in turn); 1 = a picture is in the slot, its time in jpts_out
|
||||
static int md_avc(mdec_t *d, const uint8_t *p, int n, int slot, int take_only) {
|
||||
if (!d->avc) d->avc = mavc_new();
|
||||
if (!d->avc) { d->avc = mavc_new(); if (d->fmt == MD_FMT_ABGR) mavc_set_rgba(d->avc); }
|
||||
if (!take_only && n > 0) mavc_decode(d->avc, p, n, d->jpts);
|
||||
if (d->w <= 0 || d->h <= 0) return 0;
|
||||
size_t need = (size_t)d->w * d->h * 3 / 2;
|
||||
size_t need = (size_t)d->w * d->h * (d->fmt == MD_FMT_ABGR ? 4 : 3) / (d->fmt == MD_FMT_ABGR ? 1 : 2);
|
||||
if (d->out[slot].cap < need) { free(d->out[slot].frm); d->out[slot].frm = malloc(need); d->out[slot].cap = need; }
|
||||
return mavc_take(d->avc, d->jflush, d->out[slot].frm, d->w, d->h, &d->jpts_out);
|
||||
}
|
||||
static void md_close(mdec_t *d) { if (d->vpx_on) { vpx_codec_destroy(&d->vpx); d->vpx_on = 0; } if (d->vb) { stb_vorbis_close(d->vb); d->vb = NULL; } if (d->avc) { mavc_free(d->avc); d->avc = NULL; } }
|
||||
static void md_close(mdec_t *d) { if (d->vpx_on) { vpx_codec_destroy(&d->vpx); d->vpx_on = 0; } if (d->vb) { stb_vorbis_close(d->vb); d->vb = NULL; } if (d->aac) { AudioConverterDispose(d->aac); d->aac = NULL; } if (d->avc) { mavc_free(d->avc); d->avc = NULL; } }
|
||||
void dm_host_helper_qos(qos_class_t q); // hle_thread.c
|
||||
static void *md_worker(void *arg) {
|
||||
mdec_t *d = arg;
|
||||
@@ -315,7 +369,8 @@ static void *md_worker(void *arg) {
|
||||
pthread_mutex_unlock(&d->wmu);
|
||||
if (job == MDJ_DECODE || job == MDJ_TAKE) {
|
||||
uint64_t t0 = now_ns();
|
||||
d->jgot = d->type == MD_TYPE_AVC ? md_avc(d, d->jp, d->jn, d->jslot, job == MDJ_TAKE) : d->type == MD_TYPE_VORBIS ? md_vorbis(d, d->jp, d->jn, d->jslot) : md_vp9(d, d->jp, d->jn, d->jslot);
|
||||
d->jgot = d->type == MD_TYPE_AVC ? md_avc(d, d->jp, d->jn, d->jslot, job == MDJ_TAKE) : d->type == MD_TYPE_VORBIS ? md_vorbis(d, d->jp, d->jn, d->jslot) :
|
||||
d->type == MD_TYPE_AAC ? md_aac(d, d->jp, d->jn, d->jslot) : md_vp9(d, d->jp, d->jn, d->jslot);
|
||||
uint64_t dt = now_ns() - t0; d->dec_n++; d->dec_ns += dt; if (dt > d->dec_max) d->dec_max = dt;
|
||||
}
|
||||
else md_close(d);
|
||||
@@ -359,12 +414,13 @@ static void md_pump(mdec_t *d) {
|
||||
pthread_mutex_unlock(&d->mu);
|
||||
int got = 0, avc = d->type == MD_TYPE_AVC;
|
||||
int eos = (pk.flags & 4) != 0; // MediaCodec BUFFER_FLAG_END_OF_STREAM
|
||||
if ((pk.size > 0 && (d->type == MD_TYPE_VP9 || d->type == MD_TYPE_VORBIS || avc)) || (avc && eos)) {
|
||||
if (eos && pk.pts < 0) pk.size = 0; // Monster Hunter Rise ends a stream with time -1 and the whole input buffer (4 MB) as its length: no data (decoding it gave VideoToolbox -12909 and AudioToolbox -50)
|
||||
if ((pk.size > 0 && (d->type == MD_TYPE_VP9 || MD_AUDIO(d) || avc)) || (avc && eos)) {
|
||||
d->jp = d->in[pk.in] + pk.off; d->jn = pk.size; d->jslot = slot; d->jpts = pk.pts; d->jflush = 0; md_job(d, MDJ_DECODE); got = d->jgot; }
|
||||
int more = avc && eos && d->avc && mavc_queued(d->avc) > 0; // H.264 pictures still queued: they follow this output, the last of them carries the end-of-stream flag
|
||||
pthread_mutex_lock(&d->mu);
|
||||
d->infree[d->ninfree++] = pk.in;
|
||||
int ready = got || eos || (d->type != MD_TYPE_VP9 && d->type != MD_TYPE_VORBIS && !avc); // an end-of-stream packet without a picture still yields an output to carry the flag back; VP8 has no decoder and gives black frames
|
||||
int ready = got || eos || (d->type != MD_TYPE_VP9 && !MD_AUDIO(d) && !avc); // an end-of-stream packet without a picture still yields an output to carry the flag back; VP8 has no decoder and gives black frames
|
||||
if (ready) { d->out[slot].pic = got; d->out[slot].pts = got && avc ? d->jpts_out : pk.pts; d->out[slot].flags = more ? pk.flags & ~4u : pk.flags; d->outq[d->noutq++] = slot; }
|
||||
else d->out[slot].used = 0;
|
||||
if (more) d->avc_drain = 1;
|
||||
@@ -380,12 +436,11 @@ HLE_FORCE("_ZN5movie7DecoderC1ENS_11DecoderTypeENS_19DecoderOutputFormatENS_11De
|
||||
static void *vt; if (!vt) { uint8_t *t = guest_sym("_ZTVN5movie7DecoderE"); if (!t) die("movie: missing the vtable of movie::Decoder"); vt = t + 0x10; }
|
||||
md_syms();
|
||||
self[0] = vt; self[1] = NULL;
|
||||
// Only video (AVC / VP8 / VP9 = 1..3): NV12 output (Engage's Unity, CPU buffers), or RGBA output into the client's NVN texture (mode 2, Metroid Dread);
|
||||
// Only video (AVC / VP8 / VP9 = 1..3): NV12 output (Engage's Unity, CPU buffers), or RGBA output into the client's NVN texture (mode 2: Metroid Dread's VP9, Monster Hunter Rise's AVC);
|
||||
// anything else (audio) keeps a null impl and every method returns the SDK's failure code. VP9 is decoded with libvpx, AVC with VideoToolbox (hle_movie_avc.c); VP8 gives black frames
|
||||
// Audio: Vorbis to PCM16 (Link's Awakening's movies carry a Vorbis track; without a decoder the SDK's player plays them silent)
|
||||
int audio = type == MD_TYPE_VORBIS && fmt == MD_FMT_PCM16;
|
||||
// Audio: Vorbis to PCM16 (Link's Awakening's movies carry a Vorbis track; without a decoder the SDK's player plays them silent), AAC to PCM16 with AudioToolbox (Monster Hunter Rise)
|
||||
int audio = (type == MD_TYPE_VORBIS || type == MD_TYPE_AAC) && fmt == MD_FMT_PCM16;
|
||||
if (!audio && (type < 1 || type > 3 || (fmt != MD_FMT_NV12 && !(fmt == MD_FMT_ABGR && mode == MD_MODE_TEX)))) { LOG("[RED] movie: decoder type %d output %d mode %d not supported by the host, every method returns failure", type, fmt, mode); return; }
|
||||
if (!audio && type == MD_TYPE_AVC && fmt != MD_FMT_NV12) { LOG("[RED] movie: AVC outputs NV12 only (requested output %d mode %d), every method returns failure", fmt, mode); return; }
|
||||
if (!audio && type != MD_TYPE_VP9 && type != MD_TYPE_AVC) LOG("[RED] movie: decoder type %d (VP8) is not decoded by the host, outputting black frames", type);
|
||||
mdec_t *d = calloc(1, sizeof *d);
|
||||
pthread_mutex_init(&d->mu, NULL); pthread_mutex_init(&d->dmu, NULL); pthread_mutex_init(&d->wmu, NULL); pthread_cond_init(&d->wcv, NULL);
|
||||
@@ -414,17 +469,22 @@ static void md_free(void **self) {
|
||||
HLE_FORCE("_ZN5movie7DecoderD2Ev", "movie decoder destructor: frees the host object", void, h_movie_DecD2, void **self) { md_free(self); }
|
||||
// complete-object destructor (D1): Metroid Dread calls it directly (main+0x225a08); without the take-over the SDK destroys the host object as a DecoderImpl and crashes
|
||||
HLE_FORCE("_ZN5movie7DecoderD1Ev", "movie decoder destructor (complete object): frees the host object", void, h_movie_DecD1, void **self) { md_free(self); }
|
||||
// deleting destructor: Unity allocates the object with its own allocator, calls D2 and frees it itself; the object itself is not freed here
|
||||
HLE_FORCE("_ZN5movie7DecoderD0Ev", "movie decoder deleting destructor: frees the host object", void, h_movie_DecD0, void **self) { LOG("[RED] movie: deleting destructor (the object itself is not freed)"); md_free(self); }
|
||||
// deleting destructor: Unity allocates the object with its own allocator, calls D2 and frees it itself. Monster Hunter Rise deletes its decoders (new / delete), so after the host object the
|
||||
// object itself goes back through the game's operator delete (guest code, called on the calling guest thread)
|
||||
HLE_FORCE("_ZN5movie7DecoderD0Ev", "movie decoder deleting destructor: frees the host object, then the object with operator delete", void, h_movie_DecD0, void **self) {
|
||||
md_free(self);
|
||||
static void (*del)(void *); if (!del) del = (void (*)(void *))guest_sym("_ZdlPv");
|
||||
if (del) del(self); else LOG("[RED] movie: no operator delete in the game; the decoder object is not freed");
|
||||
}
|
||||
|
||||
HLE_FORCE("_ZN5movie7Decoder9ConfigureEPNS_9MediaDataE", "movie decoder configure", int, h_movie_Configure, void **self, void *md) {
|
||||
mdec_t *d = md_of(self); if (!d) return 0x1000;
|
||||
if (d->type == MD_TYPE_VORBIS) { // codec-specific data as Android's MediaCodec takes it: csd-0 = identification header, csd-1 = setup header
|
||||
if (MD_AUDIO(d)) { // codec-specific data as Android's MediaCodec takes it: Vorbis csd-0 = identification header, csd-1 = setup header; AAC csd-0 = AudioSpecificConfig
|
||||
for (int k = 0; k < 2 && md; k++) { void *b = NULL; char key[8]; snprintf(key, sizeof key, "csd-%d", k);
|
||||
if (G.find_buf(md, key, &b) && b && G.data(b) && G.size(b)) { size_t n = G.size(b); free(d->vhdr[k]); d->vhdr[k] = malloc(n); memcpy(d->vhdr[k], G.data(b), n); d->vhdr_n[k] = (int)n; } }
|
||||
int ch = 0, rate = 0; if (md) { G.find_i32(md, "channel-count", &ch); G.find_i32(md, "sample-rate", &rate); }
|
||||
if (ch > 0) d->ch = ch; if (rate > 0) d->rate = rate;
|
||||
MD_TRACE(d, 0, "Configure (Vorbis): %d channels %d Hz, headers %d + %d bytes", ch, rate, d->vhdr_n[0], d->vhdr_n[1]);
|
||||
MD_TRACE(d, 0, "Configure (%s): %d channels %d Hz, headers %d + %d bytes", d->type == MD_TYPE_AAC ? "AAC" : "Vorbis", ch, rate, d->vhdr_n[0], d->vhdr_n[1]);
|
||||
return 0;
|
||||
}
|
||||
if (d->type == MD_TYPE_AVC && md) { // SPS / PPS: csd-0 / csd-1 (Annex B, as Android's MediaCodec takes them) or, from this SDK's MP4 demuxer (Super Mario Odyssey), "avcc" = the avcC box
|
||||
@@ -507,7 +567,7 @@ HLE_FORCE("_ZN5movie7Decoder24SendInputBufferForDecodeEiiilj", "movie decoder: s
|
||||
pthread_mutex_lock(&d->mu);
|
||||
int fmt_now = 0;
|
||||
if (d->type == MD_TYPE_VP9 && !d->w && size > 0 && vp9_size(d->in[i] + off, size, &d->w, &d->h)) LOG("[movie] VP9 key frame: %d×%d", d->w, d->h);
|
||||
if (!d->fmt_sent) { d->fmt_sent = 1; fmt_now = 1; if (!d->w && d->type != MD_TYPE_VORBIS) { d->w = 1920; d->h = 1080; } } // the size is fixed from here on (slots are allocated by it)
|
||||
if (!d->fmt_sent) { d->fmt_sent = 1; fmt_now = 1; if (!d->w && !MD_AUDIO(d)) { d->w = 1920; d->h = 1080; } } // the size is fixed from here on (slots are allocated by it)
|
||||
d->pend[d->npend].in = i; d->pend[d->npend].off = off; d->pend[d->npend].size = size; d->pend[d->npend].pts = pts; d->pend[d->npend].flags = flags; d->npend++;
|
||||
int queued = d->npend;
|
||||
pthread_mutex_unlock(&d->mu);
|
||||
@@ -532,7 +592,7 @@ HLE_FORCE("_ZN5movie7Decoder24AcquireOutputBufferIndexEPiPlPjS1_", "movie decode
|
||||
return r;
|
||||
}
|
||||
static void md_format(mdec_t *d, void *md) {
|
||||
if (d->type == MD_TYPE_VORBIS) { G.set_i32(md, "channel-count", d->ch > 0 ? d->ch : 2); G.set_i32(md, "sample-rate", d->rate > 0 ? d->rate : 48000); return; }
|
||||
if (MD_AUDIO(d)) { G.set_i32(md, "channel-count", d->ch > 0 ? d->ch : 2); G.set_i32(md, "sample-rate", d->rate > 0 ? d->rate : 48000); return; }
|
||||
int w, h, stride, slice; md_dims(d, &w, &h, &stride, &slice);
|
||||
G.set_i32(md, "width", w); G.set_i32(md, "height", h);
|
||||
G.set_rect(md, "crop", 0, 0, w - 1, h - 1);
|
||||
@@ -554,7 +614,7 @@ HLE_FORCE("_ZN5movie7Decoder21GetOutputBufferFormatEiPNS_9MediaDataE", "movie de
|
||||
HLE_FORCE("_ZN5movie7Decoder19GetOutputBufferSizeEPiS1_", "movie decoder: output buffer size (one NV12 frame)", int, h_movie_OutSize, void **self, int *size, int *mode) {
|
||||
mdec_t *d = md_of(self); if (!d) return 0x1000;
|
||||
if (!size || !mode) return 0x100b;
|
||||
if (d->type == MD_TYPE_VORBIS) { *size = MD_PCM_CAP; *mode = d->mode; MD_TRACE(d, 11, "GetOutputBufferSize → %d (PCM)", *size); return 0; }
|
||||
if (MD_AUDIO(d)) { *size = MD_PCM_CAP; *mode = d->mode; MD_TRACE(d, 11, "GetOutputBufferSize → %d (PCM)", *size); return 0; }
|
||||
int w, h, stride, slice; md_dims(d, &w, &h, &stride, &slice);
|
||||
*size = stride * slice * 3 / 2; *mode = d->mode; // the SDK fills the second output with the decoder mode
|
||||
MD_TRACE(d, 11, "GetOutputBufferSize → %d (%d×%d)", *size, w, h);
|
||||
@@ -587,7 +647,7 @@ HLE_FORCE("_ZN5movie7Decoder15GetOutputBufferEiPNS_6BufferE", "movie decoder: ge
|
||||
mdec_t *d = md_of(self); if (!d) return 0x1000;
|
||||
if (!buf || i < 0 || i >= MD_NOUT) return 0x100b;
|
||||
if (d->mode == MD_MODE_TEX) return md_out_tex(d, i, buf);
|
||||
if (d->type == MD_TYPE_VORBIS) { // PCM16 interleaved; an output without samples (end of stream) is empty
|
||||
if (MD_AUDIO(d)) { // PCM16 interleaved; an output without samples (end of stream) is empty
|
||||
size_t n = d->out[i].pic ? d->out[i].bytes : 0, cap = G.cap(buf); uint8_t *p = G.data(buf);
|
||||
if (!p || cap < n) { LOG("[RED] movie: client audio output buffer is %zu bytes, this packet needs %zu", cap, n); G.set_range(buf, 0, 0); return 0x100b; }
|
||||
if (n) memcpy(p, d->out[i].frm, n);
|
||||
|
||||
+16
-2
@@ -5,6 +5,7 @@
|
||||
// in presentation order, so pictures wait in a queue sorted by presentation time and leave it once more than `depth` are queued (4 when the profile has B-frames, 0 for Baseline), or all at the end of the stream
|
||||
#include "hle.h"
|
||||
#include <VideoToolbox/VideoToolbox.h>
|
||||
#include <arm_neon.h>
|
||||
#include "hle_movie_avc.h"
|
||||
|
||||
#define QMAX 16
|
||||
@@ -13,7 +14,7 @@ struct mavc {
|
||||
CMVideoFormatDescriptionRef fd; VTDecompressionSessionRef vs;
|
||||
struct { CVPixelBufferRef pb; int64_t pts; } q[QMAX]; int nq, depth;
|
||||
uint8_t *au; size_t aucap; // length-prefixed copy of the access unit being decoded
|
||||
int red;
|
||||
int red, rgba; // rgba: pictures as 4-byte RGBA (native texture mode, Monster Hunter Rise) instead of NV12
|
||||
};
|
||||
|
||||
mavc *mavc_new(void) { mavc *a = calloc(1, sizeof *a); if (!a) die("movie: AVC decoder allocation failed"); return a; }
|
||||
@@ -76,7 +77,7 @@ static int open_session(mavc *a) {
|
||||
const uint8_t *ps[2] = {a->sps, a->pps}; size_t sz[2] = {(size_t)a->nsps, (size_t)a->npps};
|
||||
OSStatus st = CMVideoFormatDescriptionCreateFromH264ParameterSets(NULL, 2, ps, sz, 4, &a->fd);
|
||||
if (st) { if (!a->red++) LOG("[RED] movie: AVC parameter sets give no format description (%d), outputting black frames", (int)st); return 0; }
|
||||
int32_t pf = kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange; // NV12, video range: what the SDK's decoder hands out
|
||||
int32_t pf = a->rgba ? kCVPixelFormatType_32BGRA : kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange; // NV12, video range: what the SDK's decoder hands out; BGRA (converted by VideoToolbox with the stream's colour matrix) for RGBA output
|
||||
CFNumberRef n = CFNumberCreate(NULL, kCFNumberSInt32Type, &pf);
|
||||
const void *k[] = {kCVPixelBufferPixelFormatTypeKey}, *v[] = {n};
|
||||
CFDictionaryRef attrs = CFDictionaryCreate(NULL, k, v, 1, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks); CFRelease(n);
|
||||
@@ -110,11 +111,24 @@ int mavc_decode(mavc *a, const uint8_t *p, int n, int64_t pts_us) {
|
||||
if (st) { if (!a->red++) LOG("[RED] movie: AVC decode submission failed (%d, first case)", (int)st); return -1; }
|
||||
return 0;
|
||||
}
|
||||
void mavc_set_rgba(mavc *a) { if (!a->rgba) { a->rgba = 1; a->dirty = 1; } }
|
||||
// BGRA picture → RGBA rows of w pixels (stride w * 4); a smaller picture is padded with opaque black, a larger one cropped
|
||||
static void take_rgba(CVPixelBufferRef pb, uint8_t *dst, int w, int h) {
|
||||
int pw = (int)CVPixelBufferGetWidth(pb), ph = (int)CVPixelBufferGetHeight(pb), cw = pw < w ? pw : w, chh = ph < h ? ph : h;
|
||||
const uint8_t *src = CVPixelBufferGetBaseAddress(pb); size_t ss = CVPixelBufferGetBytesPerRow(pb);
|
||||
if (cw < w || chh < h) for (size_t i = 0; i < (size_t)w * h; i++) ((uint32_t *)dst)[i] = 0xff000000u;
|
||||
for (int r = 0; r < chh; r++) {
|
||||
const uint8_t *s = src + r * ss; uint8_t *o = dst + (size_t)r * w * 4; int x = 0;
|
||||
for (; x + 16 <= cw; x += 16) { uint8x16x4_t v = vld4q_u8(s + 4 * x); uint8x16_t t = v.val[0]; v.val[0] = v.val[2]; v.val[2] = t; vst4q_u8(o + 4 * x, v); }
|
||||
for (; x < cw; x++) { o[4 * x] = s[4 * x + 2]; o[4 * x + 1] = s[4 * x + 1]; o[4 * x + 2] = s[4 * x]; o[4 * x + 3] = 255; }
|
||||
}
|
||||
}
|
||||
int mavc_take(mavc *a, int flush, uint8_t *dst, int w, int h, int64_t *pts_us) {
|
||||
if (!a->nq || (!flush && a->nq <= a->depth)) return 0;
|
||||
CVPixelBufferRef pb = a->q[0].pb; if (pts_us) *pts_us = a->q[0].pts;
|
||||
memmove(a->q, a->q + 1, --a->nq * sizeof a->q[0]);
|
||||
CVPixelBufferLockBaseAddress(pb, kCVPixelBufferLock_ReadOnly);
|
||||
if (a->rgba) { take_rgba(pb, dst, w, h); CVPixelBufferUnlockBaseAddress(pb, kCVPixelBufferLock_ReadOnly); CVPixelBufferRelease(pb); return 1; }
|
||||
int pw = (int)CVPixelBufferGetWidthOfPlane(pb, 0), ph = (int)CVPixelBufferGetHeightOfPlane(pb, 0);
|
||||
const uint8_t *y = CVPixelBufferGetBaseAddressOfPlane(pb, 0), *uv = CVPixelBufferGetBaseAddressOfPlane(pb, 1);
|
||||
size_t ys = CVPixelBufferGetBytesPerRowOfPlane(pb, 0), uvs = CVPixelBufferGetBytesPerRowOfPlane(pb, 1);
|
||||
|
||||
@@ -13,4 +13,5 @@ int mavc_decode(mavc *a, const uint8_t *p, int n, int64_t pts_us); // one acce
|
||||
// 1 = a picture was written (its time in *pts_us), 0 = none yet; with flush every queued picture comes out (end of stream)
|
||||
int mavc_take(mavc *a, int flush, uint8_t *nv12, int w, int h, int64_t *pts_us);
|
||||
int mavc_queued(const mavc *a);
|
||||
void mavc_set_rgba(mavc *a); // pictures come out as 4-byte RGBA (stride w * 4) instead of NV12: the native texture mode
|
||||
void mavc_drop(mavc *a); // a flush / seek: queued pictures are discarded
|
||||
@@ -103,6 +103,15 @@ HLE_FORCE("_ZN2nn2oe22SetCopyrightVisibilityEb", "copyright overlay visibility;
|
||||
// OperationMode: 0 Handheld, 1 Console; PerformanceMode: 0 Normal, 1 Boost (one-to-one with the operation mode)
|
||||
HLE_FORCE("_ZN2nn2oe16GetOperationModeEv", "operation mode query; a fixed value from DM_OPMODE", int, h_oe_GetOpMode, void) { int u = g_ui_opmode; if (atomic_exchange(&g_ui_armed, 0)) atomic_fetch_add(&g_ui_reads, 1); return u >= 0 ? u : opmode(); }
|
||||
HLE_FORCE("_ZN2nn2oe18GetPerformanceModeEv", "performance mode query; same as the operation mode", int, h_oe_GetPerfMode, void) { return opmode(); }
|
||||
// SDK 11 calls of Monster Hunter Rise: the CPU overclock query reads an apm session the host never opened (null read in sdk+0x35258 at the start of nnMain, main+0x6bc8650);
|
||||
// the default display resolution is the TV's (1080p docked) or the panel's (720p handheld) and only changes with the operation mode
|
||||
HLE_FORCE("_ZN2nn2oe21IsCpuOverclockEnabledEv", "CPU overclock state (through apm): not enabled", unsigned, h_oe_CpuOc, void) { return 0; }
|
||||
HLE_FORCE("_ZN2nn2oe27GetDefaultDisplayResolutionEPiS1_", "default display resolution: 1920x1080 docked, 1280x720 handheld", void, h_oe_DefRes, int *w, int *h) {
|
||||
int con = opmode() == 1; if (w) *w = con ? 1920 : 1280; if (h) *h = con ? 1080 : 720;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn2oe38GetDefaultDisplayResolutionChangeEventEPNS_2os11SystemEventE", "default display resolution change event: bound, never signaled (the operation mode notification carries the change)",
|
||||
void, h_oe_DefResEv, void *ev) { hle_bind_system_event(ev, 0); }
|
||||
HLE_FORCE("_ZN2nn2oe40SendServerMaintenanceOverlayNotificationENS_4time9PosixTimeES2_", "server maintenance overlay (through am); the host has no system overlay", void, h_oe_MaintOverlay, int64_t a, int64_t b) { (void)a; (void)b; }
|
||||
// apm performance configurations: Normal 0x00010000 (CPU 1020 / GPU 384 / EMC 1600 MHz, the handheld default), Boost 0x00010001 (GPU 768 MHz, the docked default)
|
||||
HLE_FORCE("_ZN2nn2oe27GetPerformanceConfigurationENS0_15PerformanceModeE", "query the CPU/GPU clock config (through apm); reports the default for the mode", int, h_oe_GetPerfCfg, int mode) { return mode == 1 ? 0x00010001 : 0x00010000; }
|
||||
HLE_FORCE("_ZN2nn2oe24SetAlbumImageOrientationENS_5album16ImageOrientationE", "screenshot orientation (through am); the host has no system screenshot", void, h_oe_AlbumOrient, int o) { (void)o; }
|
||||
|
||||
@@ -123,3 +123,13 @@ HLE_FORCE("nnsocketGAIStrError", "resolver error text", const char *, h_c_sk_Gai
|
||||
HLE_FORCE("_ZN2nn4bcat10InitializeEv", "data delivery init (connects to the bcat service); the host has none, no-op", void, h_bcat_Init, void) {}
|
||||
HLE_FORCE("_ZN2nn4bcat25MountDeliveryCacheStorageEv", "mount delivery cache: no delivery data, fails", Result, h_bcat_Mount, void) { return RES_FAIL; }
|
||||
HLE_FORCE("_ZN2nn4bcat27UnmountDeliveryCacheStorageEv", "unmount delivery cache", void, h_bcat_Unmount, void) {}
|
||||
|
||||
// ---------- album: the capture album (Monster Hunter Rise saves its own photo-mode pictures there). Initialize connects to the capture service (capsrv, an abort in sm::Initialize at the start of
|
||||
// nnMain, main+0x6bc5478); the host has no album, so saving fails and showing it does nothing. GetImageWidth / GetImageHeight only compute and keep running in the SDK ----------
|
||||
HLE_FORCE("_ZN2nn5album10InitializeEv", "capture album init (capsrv); the host has no album, no-op", void, h_album_Init, void) {}
|
||||
HLE_FORCE("_ZN2nn5album8FinalizeEv", "capture album", void, h_album_Fini, void) {}
|
||||
HLE_FORCE("_ZN2nn5album14SaveScreenshotEPKvmNS0_9ImageSizeENS0_16ImageOrientationENS0_17AlbumReportOptionE", "save a picture to the album: no album, fails", Result, h_album_Save,
|
||||
const void *p, size_t n, int size, int orient, int report) { (void)p; (void)n; (void)size; (void)orient; (void)report; LOG("album::SaveScreenshot (no album)"); return RES_FAIL; }
|
||||
HLE_FORCE("_ZN2nn5album22SaveAndShareScreenshotEPKvmNS0_9ImageSizeENS0_16ImageOrientationENS0_17AlbumReportOptionERKNS_7account3UidEPKc", "save and share a picture: no album, fails", Result, h_album_Share,
|
||||
const void *p, size_t n, int size, int orient, int report, const void *uid, const char *text) { (void)p; (void)n; (void)size; (void)orient; (void)report; (void)uid; (void)text; return RES_FAIL; }
|
||||
HLE_FORCE("_ZN2nn5album14ShowAlbumFilesEv", "album applet: not shown", Result, h_album_Show, void) { return RES_FAIL; }
|
||||
+81
-17
@@ -10,6 +10,7 @@
|
||||
// Game side (main 0xc6deac) sets isUseUtf8 = 1 after MakePreset and reads back as a C string (0x1b66640 strlen); UTF-16 output is supported as well per the original convention.
|
||||
// No window and DM_KBD unset: return as cancel (no popup, no hang). DM_KBD="text": no popup, return that text directly as OK (windowless testing / lockstep comparison).
|
||||
#include "hle.h"
|
||||
#include <stddef.h>
|
||||
#include <unistd.h>
|
||||
#include "front.h"
|
||||
|
||||
@@ -98,10 +99,20 @@ HLE_FORCE("_ZN2nn5swkbd12ShowKeyboardEPNS0_6StringERKNS0_15ShowKeyboardArgE", "s
|
||||
// ---------- InlineKeyboard (Super Mario 3D World + Bowser's Fury) ----------
|
||||
// On hardware Initialize starts a resident keyboard applet (swkbd::detail::InlineKeyboardImpl::Launch -> applet::CreateLibraryApplet); after that Calc collects its results every frame and calls the callbacks the game registered,
|
||||
// Appear / Disappear / SetInputText / SetCursorPos send requests through applet storage, and GetImage takes the keyboard picture from shared memory. The host has no applet proxy: entering the save select after the character select, Initialize crashed on a null pointer in CreateLibraryApplet.
|
||||
// Every entry that touches the applet is taken over here and behaves as a keyboard the player always cancels right away: Appear arms one cancel, and the next Calc (on hardware the callbacks are called from Calc too) calls the game's cancel callback; no picture (GetImage reports none).
|
||||
// Every entry that touches the applet is taken over here. Appear arms one request; the next Calc (on hardware the callbacks are called from Calc too) asks for the text the way ShowKeyboard does
|
||||
// (DM_KBD, else the front end's input box) and hands it to the game's changed-string and enter callbacks, or calls its cancel callback (no callbacks, no input box, or the player cancelled).
|
||||
// No picture (GetImage reports none). Monster Hunter Rise names the hunter, the Palico and the Palamute through it; with every Appear counted as a cancel the name box stayed empty for good
|
||||
// The constructor, GetRequiredWorkBufferSize / GetImageMemoryRequirement / GetRequiredStringBufferSize and InputText::Set are pure computations and keep running in the SDK
|
||||
typedef void (*ikb_cb0)(void);
|
||||
static struct { const void *kb; ikb_cb0 cancel; int pending; } g_ikb[4];
|
||||
// fin_init / released: callbacks the applet makes from Calc once Initialize / UnsetUserWordInfo have finished (Monster Hunter Rise, SDK 11); the host makes them on the next Calc
|
||||
// The arguments of the changed-string and enter callbacks, as the game reads them (Monster Hunter Rise's callbacks main+0x6fb3910 / main+0x6fb3afc): the text inline, then its length;
|
||||
// the changed-string one adds three cursor positions. AppearArg: textMaxLength at +0x20 (as libnx's SwkbdAppearArg)
|
||||
typedef struct { uint16_t str[502]; int32_t length, dic_start, dic_end, cursor; } ikb_str_arg;
|
||||
_Static_assert(offsetof(ikb_str_arg, length) == 0x3ec && offsetof(ikb_str_arg, cursor) == 0x3f8, "swkbd ChangedStringArg / DecidedEnterArg");
|
||||
typedef void (*ikb_cb1)(const ikb_str_arg *);
|
||||
// state: what Calc returns (nn::swkbd::State as Monster Hunter Rise reads it at main+0x6fb3e6c: it finishes an entered name only when Calc returns 1 after a non-zero value):
|
||||
// 0 until initialization has finished, 1 ready / hidden, 3 shown. The hook used to return nothing, leaving w0 as whatever the host last held
|
||||
static struct { const void *kb; ikb_cb0 cancel, fin_init, released, finished; ikb_cb1 enter, changed; int pending, fin_init_due, released_due, finished_due, max_len, state; } g_ikb[4]; // finished: called on the Calc after Disappear (the keyboard has closed)
|
||||
static pthread_mutex_t g_ikb_mu = PTHREAD_MUTEX_INITIALIZER;
|
||||
static int ikb_slot(const void *kb) { // the caller holds the lock
|
||||
for (int i = 0; i < 4; i++) if (g_ikb[i].kb == kb) return i;
|
||||
@@ -109,8 +120,8 @@ static int ikb_slot(const void *kb) { // the caller holds the lock
|
||||
return -1;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard10InitializeEPv", "inline keyboard: no applet started (the host has no applet proxy), reports success", unsigned, h_ikb_Init, void *kb, void *work) {
|
||||
(void)work; pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); pthread_mutex_unlock(&g_ikb_mu);
|
||||
LOG("swkbd: inline keyboard Initialize %p (the host starts no applet; Appear always counts as the player cancelling)%s", kb, i < 0 ? "[RED] registration table full" : "");
|
||||
(void)work; pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].fin_init_due = 1; pthread_mutex_unlock(&g_ikb_mu);
|
||||
LOG("swkbd: inline keyboard Initialize %p (the host starts no applet; Appear asks for the text the way ShowKeyboard does)%s", kb, i < 0 ? "[RED] registration table full" : "");
|
||||
return 1;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard8FinalizeEv", "inline keyboard: no applet started, nothing to close", void, h_ikb_Fin, void *kb) {
|
||||
@@ -123,23 +134,76 @@ HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard24SetDecidedCancelCallbackEPFvvE", "inlin
|
||||
#define IKB_IGNORE_CB(mangled, cname) \
|
||||
HLE_FORCE(mangled, "inline keyboard: callbacks are recorded, never called (the keyboard never confirms / edits text / moves the cursor)", void, cname, void *kb, void *fn) { (void)kb; (void)fn; }
|
||||
IKB_IGNORE_CB("_ZN2nn5swkbd14InlineKeyboard22SetMovedCursorCallbackEPFvPKNS0_14MovedCursorArgEE", h_ikb_SetMoved)
|
||||
IKB_IGNORE_CB("_ZN2nn5swkbd14InlineKeyboard23SetDecidedEnterCallbackEPFvPKNS0_15DecidedEnterArgEE", h_ikb_SetEnter)
|
||||
IKB_IGNORE_CB("_ZN2nn5swkbd14InlineKeyboard24SetChangedStringCallbackEPFvPKNS0_16ChangedStringArgEE", h_ikb_SetChanged)
|
||||
IKB_IGNORE_CB("_ZN2nn5swkbd14InlineKeyboard27SetFinishedKeyboardCallbackEPFvvE", h_ikb_SetFinished)
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard6AppearERKNS0_9AppearArgE", "inline keyboard appears: the host does not show it, the next Calc counts as the player cancelling", void, h_ikb_Appear, void *kb, const void *arg) {
|
||||
(void)arg; pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].pending = 1; pthread_mutex_unlock(&g_ikb_mu);
|
||||
LOG("swkbd: inline keyboard Appear %p → cancel on the next tick", kb);
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard23SetDecidedEnterCallbackEPFvPKNS0_15DecidedEnterArgEE", "inline keyboard: records the enter callback", void, h_ikb_SetEnter, void *kb, ikb_cb1 fn) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].enter = fn; pthread_mutex_unlock(&g_ikb_mu);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard9DisappearEv", "inline keyboard hidden: the host never showed it", void, h_ikb_Disappear, void *kb) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].pending = 0; pthread_mutex_unlock(&g_ikb_mu);
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard24SetChangedStringCallbackEPFvPKNS0_16ChangedStringArgEE", "inline keyboard: records the changed-string callback", void, h_ikb_SetChanged, void *kb, ikb_cb1 fn) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].changed = fn; pthread_mutex_unlock(&g_ikb_mu);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard4CalcEv", "inline keyboard per-frame update: calls the game's cancel callback when a cancel is pending", void, h_ikb_Calc, void *kb) {
|
||||
ikb_cb0 fn = NULL; pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb);
|
||||
if (i >= 0 && g_ikb[i].pending) { g_ikb[i].pending = 0; fn = g_ikb[i].cancel; }
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard27SetFinishedKeyboardCallbackEPFvvE", "inline keyboard: records the finished callback, called on the next Calc after Disappear", void, h_ikb_SetFinished, void *kb, ikb_cb0 fn) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].finished = fn; pthread_mutex_unlock(&g_ikb_mu);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard6AppearERKNS0_9AppearArgE", "inline keyboard appears: the next Calc asks for the text (DM_KBD or the front end's input box)", void, h_ikb_Appear, void *kb, const void *arg) {
|
||||
int32_t max = 0; if (arg) memcpy(&max, (const uint8_t *)arg + 0x20, 4);
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) { g_ikb[i].pending = 1; g_ikb[i].state = 3; g_ikb[i].max_len = max > 0 && max <= 500 ? max : 0; } pthread_mutex_unlock(&g_ikb_mu);
|
||||
LOG("swkbd: inline keyboard Appear %p (at most %d characters) → text asked for on the next tick", kb, max);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard9DisappearEv", "inline keyboard hidden: the finished callback follows on the next Calc", void, h_ikb_Disappear, void *kb) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) { g_ikb[i].pending = 0; g_ikb[i].finished_due = 1; if (g_ikb[i].state) g_ikb[i].state = 1; } pthread_mutex_unlock(&g_ikb_mu);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard4CalcEv", "inline keyboard per-frame update: a pending Appear gets the text and the game's callbacks; returns the keyboard state", int, h_ikb_Calc, void *kb) {
|
||||
ikb_cb0 fn = NULL, init = NULL, rel = NULL, fin = NULL; ikb_cb1 enter = NULL, changed = NULL; int ask = 0, max = 0;
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb);
|
||||
if (i >= 0 && g_ikb[i].fin_init_due) { if (!g_ikb[i].state) g_ikb[i].state = 1; if (g_ikb[i].fin_init) { g_ikb[i].fin_init_due = 0; init = g_ikb[i].fin_init; } }
|
||||
if (i >= 0 && g_ikb[i].released_due && g_ikb[i].released) { g_ikb[i].released_due = 0; rel = g_ikb[i].released; }
|
||||
if (i >= 0 && g_ikb[i].finished_due) { g_ikb[i].finished_due = 0; fin = g_ikb[i].finished; }
|
||||
if (i >= 0 && g_ikb[i].pending) { g_ikb[i].pending = 0; ask = 1; fn = g_ikb[i].cancel; enter = g_ikb[i].enter; changed = g_ikb[i].changed; max = g_ikb[i].max_len; }
|
||||
pthread_mutex_unlock(&g_ikb_mu);
|
||||
if (fn) fn(); // guest code, called on the guest thread that called Calc (as on hardware)
|
||||
if (init) init(); // guest code, called on the guest thread that called Calc (as on hardware)
|
||||
if (rel) rel();
|
||||
if (fin) fin();
|
||||
if (!ask) return i >= 0 ? g_ikb[i].state : 0;
|
||||
char text[502 * 3 + 1]; int r = 0;
|
||||
if (enter) { // as swkbd_show: DM_KBD answers without a box; the guest clock stands still while the box is up
|
||||
const char *script = getenv("DM_KBD");
|
||||
if (script) { snprintf(text, sizeof text, "%s", script); r = 1; }
|
||||
else { guest_clock_freeze_for(2, 1); r = front_text_input("", "", "", "", max, text, sizeof text); guest_clock_freeze_for(2, 0); }
|
||||
}
|
||||
LOG("swkbd: inline keyboard %p → %s%s%s", kb, r > 0 ? "OK '" : enter ? "cancel" : "no enter callback, cancel", r > 0 ? text : "", r > 0 ? "'" : "");
|
||||
pthread_mutex_lock(&g_ikb_mu); if (g_ikb[i].kb == kb) g_ikb[i].state = 1; pthread_mutex_unlock(&g_ikb_mu); // the keyboard closes once the text is decided or cancelled
|
||||
if (r <= 0) { if (fn) fn(); return 1; }
|
||||
static ikb_str_arg a; memset(&a, 0, sizeof a); // one keyboard request at a time; static keeps 1 KB off the guest stack
|
||||
size_t n = swkbd_u8_to_u16(text, max > 0 ? (size_t)max : 500, a.str, 502);
|
||||
a.length = (int32_t)n; a.cursor = (int32_t)n;
|
||||
if (changed) changed(&a);
|
||||
enter(&a);
|
||||
return 1;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard29SetFinishedInitializeCallbackEPFvvE", "inline keyboard: records the finished-initialize callback, called on the next Calc after Initialize", void, h_ikb_SetFinInit, void *kb, ikb_cb0 fn) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].fin_init = fn; pthread_mutex_unlock(&g_ikb_mu);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard31SetReleasedUserWordInfoCallbackEPFvvE", "inline keyboard: records the released-user-words callback, called on the next Calc after UnsetUserWordInfo", void, h_ikb_SetReleased, void *kb, ikb_cb0 fn) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].released = fn; pthread_mutex_unlock(&g_ikb_mu);
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard15SetUserWordInfoERKNS0_12UserWordInfoE", "inline keyboard user words (sent through applet storage): the host keeps none", Result, h_ikb_SetWords, void *kb, const void *info) { (void)kb; (void)info; return RES_OK; }
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard17UnsetUserWordInfoEv", "inline keyboard user words released: the released callback follows on the next Calc", Result, h_ikb_UnsetWords, void *kb) {
|
||||
pthread_mutex_lock(&g_ikb_mu); int i = ikb_slot(kb); if (i >= 0) g_ikb[i].released_due = 1; pthread_mutex_unlock(&g_ikb_mu); return RES_OK;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard25SetCustomizedDictionariesERKNS0_23CustomizedDictionarySetE", "inline keyboard dictionaries (applet storage): the host keeps none", Result, h_ikb_SetDict, void *kb, const void *d) { (void)kb; (void)d; return RES_OK; }
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard27UnsetCustomizedDictionariesEv", "inline keyboard dictionaries: none kept", Result, h_ikb_UnsetDict, void *kb) { (void)kb; return RES_OK; }
|
||||
// Appearance setters (sent to the applet with the next request): the host shows no keyboard, so they are ignored
|
||||
#define IKB_IGNORE(mangled, cname, ...) HLE_FORCE(mangled, "inline keyboard appearance setting (the host does not show the keyboard, ignored)", void, cname, void *kb, __VA_ARGS__) { (void)kb; }
|
||||
IKB_IGNORE("_ZN2nn5swkbd14InlineKeyboard13SetWindowModeENS1_10WindowModeE", h_ikb_SetWinMode, int m)
|
||||
IKB_IGNORE("_ZN2nn5swkbd14InlineKeyboard14SetFooterAlphaEf", h_ikb_SetFooterA, float a)
|
||||
IKB_IGNORE("_ZN2nn5swkbd14InlineKeyboard14SetKeytopAlphaEf", h_ikb_SetKeyA, float a)
|
||||
IKB_IGNORE("_ZN2nn5swkbd14InlineKeyboard14SetKeytopScaleEf", h_ikb_SetKeyScale, float a)
|
||||
IKB_IGNORE("_ZN2nn5swkbd14InlineKeyboard18SetKeytopTranslateEff", h_ikb_SetKeyMove, float x, float y)
|
||||
IKB_IGNORE("_ZN2nn5swkbd14InlineKeyboard15SetTouchEnabledEb", h_ikb_SetTouch, unsigned on)
|
||||
IKB_IGNORE("_ZN2nn5swkbd14InlineKeyboard26SetHardwareKeyboardEnabledEb", h_ikb_SetHwKbd, unsigned on)
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard26IsUsedTouchPointByKeyboardEii", "inline keyboard touch hit test: the keyboard is never shown", unsigned, h_ikb_TouchHit, void *kb, int x, int y) { (void)kb; (void)x; (void)y; return 0; }
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard12GetMaxHeightEv", "inline keyboard height on screen: 400 of the 720 lines (the keyboard is never shown)", int, h_ikb_MaxH, void *kb) { (void)kb; return 400; }
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard12SetInputTextERKNS0_9InputTextE", "inline keyboard: set text (the host does not show it, ignored)", void, h_ikb_SetText, void *kb, const void *t) { (void)kb; (void)t; }
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard12SetCursorPosEi", "inline keyboard: set cursor (the host does not show it, ignored)", void, h_ikb_SetCursor, void *kb, int pos) { (void)kb; (void)pos; }
|
||||
HLE_FORCE("_ZN2nn5swkbd14InlineKeyboard8GetImageEPvm", "inline keyboard image: none (returns false)", unsigned, h_ikb_GetImage, void *kb, void *buf, size_t n) { (void)kb; (void)buf; (void)n; return 0; }
|
||||
HLE_FORCE("_ZNK2nn5swkbd14InlineKeyboard13GetWindowSizeEPiS2_", "inline keyboard window size: 0 × 0", void, h_ikb_WinSize, const void *kb, int *w, int *h) { (void)kb; if (w) *w = 0; if (h) *h = 0; }
|
||||
// The keyboard draws into a 1280 × 720 picture on hardware; Monster Hunter Rise makes three textures of this size for it (a 0 × 0 answer made its texture creation return null, then a null read at main+0x7713044)
|
||||
HLE_FORCE("_ZNK2nn5swkbd14InlineKeyboard13GetWindowSizeEPiS2_", "inline keyboard window size: 1280 × 720 as on hardware (no picture is ever delivered)", void, h_ikb_WinSize, const void *kb, int *w, int *h) { (void)kb; if (w) *w = 1280; if (h) *h = 720; }
|
||||
@@ -78,6 +78,12 @@ HLE_FORCE("_ZN2nn4time14ToCalendarTimeEPNS0_12CalendarTimeEPNS0_22CalendarAdditi
|
||||
ai->dst = tm.tm_isdst > 0; ai->offset = (int32_t)tm.tm_gmtoff; }
|
||||
return RES_OK;
|
||||
}
|
||||
// Explicit time zone rules (Monster Hunter Rise, SDK 11): LoadTimeZoneRule reads the rule from the time service; the host keeps the game's rule buffer untouched and converts with localtime_r as above
|
||||
HLE_FORCE("_ZN2nn4time16LoadTimeZoneRuleEPNS0_12TimeZoneRuleERKNS0_12LocationNameE", "time zone rule from the time service; the host converts with its own zone, the rule stays unused", Result, h_time_LoadRule,
|
||||
void *rule, const char *name) { (void)rule; LOG("time::LoadTimeZoneRule %.36s (host time zone used instead)", name ? name : "?"); return RES_OK; }
|
||||
HLE_FORCE("_ZN2nn4time14ToCalendarTimeEPNS0_12CalendarTimeEPNS0_22CalendarAdditionalInfoERKNS0_9PosixTimeERKNS0_12TimeZoneRuleE", "conversion with an explicit time zone rule; same as the device zone (host localtime_r)",
|
||||
Result, h_time_ToCalendarTimeRule, CalendarTime *ct, CalendarAdditionalInfo *ai, const int64_t *pt, const void *rule) { (void)rule; return h_time_ToCalendarTime(ct, ai, pt); }
|
||||
HLE_FORCE("_ZN2nn4time26StandardNetworkSystemClock20IsAccuracySufficientEv", "network clock accuracy (time service): offline, never sufficient", unsigned, h_time_NetAccurate, void) { return 0; }
|
||||
// Clock snapshot (Tears of the Kingdom: CreateWithStandardSystemClock asks the time service, which the host never opens → abort in sm::Initialize). 0xD0 bytes, laid out
|
||||
// as Ryujinx lays out the service's ClockSnapshot and as this SDK's constructor clears it (its last store is the byte at +0xcd): +0x00 user clock context {offset,
|
||||
// steady time point}, +0x20 network clock context, +0x40 user time, +0x48 network time, +0x50 / +0x58 user / network calendar, +0x60 / +0x78 their additional info,
|
||||
|
||||
Vendored
+1
@@ -5,6 +5,7 @@ Three projects from Ryujinx 1.3.2: `Ryujinx.Graphics.Shader`, `Ryujinx.Common` a
|
||||
- `Ryujinx.Graphics.Shader/CodeGen/Spirv/Instructions.cs`, `GenerateTextureSample`:
|
||||
- Texel offsets that are not constants (Maxwell AOFFI / TLD4.PTP offsets read from a register) are built with `OpCompositeConstruct` and passed as `Offset`. A four-offset gather becomes four one-offset gathers, each giving component w. Upstream wrote them as an `OpConstantComposite` with `ConstOffset(s)`, which is invalid SPIR-V; SPIRV-Cross threw `nullptr` and the shader never built (2026-10-18: 8 shaders in Wonder, Link's Awakening, Tears of the Kingdom and Age of Calamity). Marked `nxnative:` in the code.
|
||||
- Texture coordinates that are NaN are replaced by 0 (the `nfZero` select). This came with the first import.
|
||||
- `Ryujinx.Graphics.Shader/Translation/Optimizations/BindlessElimination.cs`, `TryGetOperation` / `IsSameOperand`: a bindless handle that reaches the texture instruction through a phi is followed through the phis that feed it (a handle kept in a loop variable comes back through the loop's own phi), and two `constant buffer | constant` handles count as the same value. Upstream compared only the phi's direct inputs, so such handles fell to the texture-pool array of 32 (`*_t2d_cb0_40000000`), which cannot be bound; 218 pixel shaders in Monster Hunter Rise (2026-10-20). Marked `nxnative:` in the code.
|
||||
|
||||
The shader translation tool in `dm/gfx/tools/dmsh_xlate` references them to translate the original Switch shaders to SPIR-V.
|
||||
License: MIT, see `LICENSE.txt`. Upstream: <https://github.com/Ryubing>.
|
||||
Vendored
+50
-7
@@ -359,17 +359,49 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
|
||||
{
|
||||
// If we have a phi, let's check if all inputs are effectively the same value.
|
||||
// If so, we can "see through" the phi and pick any of the inputs (since they are all the same).
|
||||
// nxnative: the inputs are gathered through the phis they come from, as a loop makes (a handle kept in a loop variable
|
||||
// reaches the phi from the loop's own phi). Upstream compared the phi's direct inputs only and gave up on such handles.
|
||||
|
||||
Operand firstSrc = phi.GetSource(0);
|
||||
Operand firstSrc = null;
|
||||
HashSet<PhiNode> visited = new();
|
||||
Stack<PhiNode> pending = new();
|
||||
pending.Push(phi);
|
||||
|
||||
for (int index = 1; index < phi.SourcesCount; index++)
|
||||
while (pending.Count > 0)
|
||||
{
|
||||
if (!IsSameOperand(firstSrc, phi.GetSource(index)))
|
||||
{
|
||||
outOperation = null;
|
||||
PhiNode current = pending.Pop();
|
||||
|
||||
return false;
|
||||
if (!visited.Add(current))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
for (int index = 0; index < current.SourcesCount; index++)
|
||||
{
|
||||
Operand src = current.GetSource(index);
|
||||
|
||||
if (src.AsgOp is PhiNode srcPhi)
|
||||
{
|
||||
pending.Push(srcPhi);
|
||||
}
|
||||
else if (firstSrc == null)
|
||||
{
|
||||
firstSrc = src;
|
||||
}
|
||||
else if (!IsSameOperand(firstSrc, src))
|
||||
{
|
||||
outOperation = null;
|
||||
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (firstSrc == null)
|
||||
{
|
||||
outOperation = null;
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
asgOp = firstSrc.AsgOp;
|
||||
@@ -389,14 +421,20 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
|
||||
|
||||
private static bool IsSameOperand(Operand x, Operand y)
|
||||
{
|
||||
if (x == y)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
if (x.Type == y.Type && x.Type == OperandType.LocalVariable)
|
||||
{
|
||||
// nxnative: the second source may also be the same constant (a constant sampler ORed into a texture handle).
|
||||
return x.AsgOp is Operation xOp &&
|
||||
y.AsgOp is Operation yOp &&
|
||||
xOp.Inst == Instruction.BitwiseOr &&
|
||||
yOp.Inst == Instruction.BitwiseOr &&
|
||||
AreBothEqualConstantBuffers(xOp.GetSource(0), yOp.GetSource(0)) &&
|
||||
AreBothEqualConstantBuffers(xOp.GetSource(1), yOp.GetSource(1));
|
||||
(AreBothEqualConstantBuffers(xOp.GetSource(1), yOp.GetSource(1)) || AreBothEqualConstants(xOp.GetSource(1), yOp.GetSource(1)));
|
||||
}
|
||||
|
||||
return false;
|
||||
@@ -407,6 +445,11 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
|
||||
return x.Type == y.Type && x.Value == y.Value && x.Type == OperandType.ConstantBuffer;
|
||||
}
|
||||
|
||||
private static bool AreBothEqualConstants(Operand x, Operand y)
|
||||
{
|
||||
return x.Type == y.Type && x.Value == y.Value && x.Type == OperandType.Constant;
|
||||
}
|
||||
|
||||
private static Operand GetSourceForMaskedHandle(Operation asgOp, uint mask)
|
||||
{
|
||||
// Assume it was already checked that the operation is bitwise AND.
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
// --allow-damaged writes those files as read instead and only lists them (Super Mario Galaxy's base NSP with 147 MB zeroed: its two prologue WebM movies).
|
||||
// Add-on content: nx_extract <prod.keys> --aoc <DLC NSP> <output dir> writes the RomFS of the DLC's data NCA to <output dir> (the packing script puts it at aoc/<index>/, the host mounts it through DM_AOC_DIR).
|
||||
// Title keys come from the common tickets (.tik) inside the NSPs, so only prod.keys is needed; key values are never printed.
|
||||
// The base game may also be a cartridge dump (.xci, Monster Hunter Rise): its secure partition holds the NCAs, which carry their keys in the key area (no ticket).
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
@@ -84,10 +85,17 @@ static class P {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Open an NSP (PFS0), import its common tickets, list all NCAs
|
||||
// Open an NSP (PFS0) or the secure partition of an XCI, import the common tickets, list all NCAs
|
||||
static List<Nca> OpenNsp(KeySet ks, string path) {
|
||||
var pfs = new PartitionFileSystem();
|
||||
pfs.Initialize(new LocalStorage(path, FileAccess.Read)).ThrowIfFailure();
|
||||
IFileSystem pfs; var st0 = new LocalStorage(path, FileAccess.Read);
|
||||
byte[] magic = new byte[4]; st0.Read(0x100, magic).ThrowIfFailure();
|
||||
if (System.Text.Encoding.ASCII.GetString(magic) == "HEAD") { // XCI: the card header's magic at 0x100
|
||||
var xci = new Xci(ks, st0);
|
||||
if (!xci.HasPartition(XciPartitionType.Secure)) throw new Exception($"no secure partition in {path}");
|
||||
pfs = xci.OpenPartition(XciPartitionType.Secure);
|
||||
} else {
|
||||
var p = new PartitionFileSystem(); p.Initialize(st0).ThrowIfFailure(); pfs = p;
|
||||
}
|
||||
var ncas = new List<Nca>();
|
||||
foreach (DirectoryEntryEx e in pfs.EnumerateEntries("*", SearchOptions.Default)) {
|
||||
using var f = new UniqueRef<IFile>();
|
||||
|
||||
+18
-1
@@ -26,6 +26,11 @@ csz = struct.unpack_from('<III', d, 0x60)
|
||||
# The key page stays after bss (the patch area reaches it through ADRP, ±4 GB). Modules below the limit produce output byte-identical to before
|
||||
NEAR_PATCH_OFF = ((((segs[2][1] + segs[2][2] + bss + 0x3FFF) & ~0x3FFF) + 0x4000 + 0xFFFF) & ~0xFFFF) # where an appended patch area would start (as in the else branch below)
|
||||
FAR, FRONT = NEAR_PATCH_OFF > 0x7000000, 0x2000000
|
||||
# A text segment that ends within 32 MB of the ±128 MB reach (Monster Hunter Rise: 125.3 MB of text) puts the patch area right before the base instead. Its stubs follow the sites in address
|
||||
# order, so the last site's stub ends up about as far from it as the text is long. The first pass tries the gap that keeps every site in reach; when the patch area does not fit there
|
||||
# (Monster Hunter Rise: 3.6 MB), the rewriter runs again with exactly the size it needs (DM_PATCH_FRONT, set only by that rerun) and B() checks every branch
|
||||
BIG_TEXT = FAR and segs[0][2] + FRONT > 0x8000000 - 0x10000
|
||||
if BIG_TEXT: FRONT = int(os.environ.get('DM_PATCH_FRONT', '0'), 0) or (0x8000000 - 0x10000 - segs[0][2]) & ~0xFFFF
|
||||
IMG = bytearray(segs[2][1] + segs[2][2] + (0 if FAR else bss))
|
||||
for i, (fo, mo, sz) in enumerate(segs):
|
||||
raw = d[fo:fo + csz[i]]
|
||||
@@ -782,9 +787,17 @@ def jump_adr_offsets(insl, j):
|
||||
ok = [tb for tb in sorted(cands) if (t := targets(tb)) and all(fa <= x < fb and x % 4 == 0 for x in t)]
|
||||
return targets(ok[0]) if len(ok) == 1 else None
|
||||
return None
|
||||
def disasm_fn(fa, fb):
|
||||
"""A function's instructions. Capstone stops at the first word it cannot decode, so go on after it: Monster Hunter Rise main+0x6b34540 is a trap word (0xe7ffdefe)
|
||||
in the middle of a function, and the br x18 jump table after it (main+0x6b346d0) was never seen"""
|
||||
out, a = [], fa
|
||||
while a < fb:
|
||||
l = list(md.disasm(bytes(IMG[a:fb]), a))
|
||||
out += l; a = (l[-1].address + 4 if l else a) + 4
|
||||
return out
|
||||
funcs = sorted({func_of(a) for a, (k, _) in kinds.items() if k == 'x18'})
|
||||
for fa, fb in funcs:
|
||||
insl = list(md.disasm(bytes(IMG[fa:fb]), fa))
|
||||
insl = disasm_fn(fa, fb)
|
||||
ws = find_windows(insl)
|
||||
for wseg in ws:
|
||||
windows.append(wseg); done |= {x.address for x in wseg}
|
||||
@@ -1061,6 +1074,10 @@ if RULE_RUN:
|
||||
os.makedirs(outdir, exist_ok=True)
|
||||
P = lambda ext: os.path.join(outdir, f'{NAME}.{ext}')
|
||||
if FAR: # patch area before the base: file = text + ro + data (padded to 16K) + patch area
|
||||
if BIG_TEXT and len(patch) > FRONT - 0x4000 and 'DM_PATCH_FRONT' not in os.environ:
|
||||
need = (len(patch) + 0x4000 + 0xFFFF) & ~0xFFFF
|
||||
print(f'{NAME}: patch area of {len(patch):#x} bytes does not fit in {FRONT:#x} before the base; rewriting again with {need:#x}', flush=True)
|
||||
os.execve(sys.executable, [sys.executable] + sys.argv, dict(os.environ, DM_PATCH_FRONT=hex(need)))
|
||||
assert len(patch) <= FRONT - 0x4000, f'patch area of {len(patch):#x} bytes does not fit in the {FRONT:#x} before the base'
|
||||
open(P('patched.bin'), 'wb').write(IMG + bytearray(FILE_IMAGE_LEN - len(IMG)) + patch)
|
||||
else:
|
||||
|
||||
@@ -25,6 +25,7 @@ ht -t nca --exefsdir=$W/data/v0/ExeFS --romfsdir=$W/data/v0/RomFS $W/build/nsp/<
|
||||
- Update NSPs: the update's Program NCA is a patch NCA; extract it with `--basenca=<base program nca>` so RomFS is the merged view. Keep base and update in separate directories (`data/v0`, `data/v<ver>`); the rewriter and the host take one ExeFS at a time.
|
||||
- Newer patch NCAs (seen: Mario 1.2.1, key generation 17) make hactool stop with `Invalid BKTR layout!`. `dm/tools/nx_extract` (C#, LibHac 0.19) handles them: `nx_extract <prod.keys> <base.nsp> <update.nsp> <out dir> [<base RomFS dir>]`. It reads the NCAs straight out of both NSPs (no 3+ GB intermediate NCA), imports the common tickets itself, prints the display version and module build IDs, and with a base RomFS dir writes identical files as APFS clones.
|
||||
- After an update, find every address the host or rewriter hard-codes for the old build (`CUTS` / `OBS` / `PARAMS` in `patch_nso.py`, `->base + 0x…` in the host) and carry it over with `dm/tools/nso_fnmap.py <old.orig.bin> <new.orig.bin> 0x…`: it matches functions with address bits masked and maps globals through their `adrp` references; check each result by disassembly (`sdis.py`). Also diff the imported SDK symbols: new imports that reach a system service need host implementations, or the real SDK aborts after connecting to `sm:`.
|
||||
- **A cartridge dump (`.xci`) as the base game** (Monster Hunter Rise): the card header's magic `HEAD` sits at 0x100, and its root HFS0 holds the `update`, `logo`, `normal` and `secure` partitions; the game's NCAs are in `secure` (here the 7 GB program NCA, control, legal information, HTML manual and the meta NCA). Cartridge NCAs carry their key in the key area, so there is no ticket. `nx_extract` opens either kind as the base game (`OpenNsp`), and the NSP script takes `*.xci` beside `*.nsp`; the update and DLC stay NSPs.
|
||||
- Decompressed NSO images for analysis: `dm/tools/nso.py <nso> <out.bin>` (lz4; use a venv that has `lz4`). `dm/tools/nso2elf.py` makes an ELF for Ghidra / REA / objdump.
|
||||
|
||||
**A damaged NSP** (an incomplete copy, a broken download) shows as a hash error while the RomFS is read. `nx_extract` lists every file whose data fails the NCA's hash check and exits with code 3; `--allow-damaged` (the NSP script's `ALLOW_DAMAGED` / `--allow-damaged`) writes them as read and only lists them. To check a whole NCA, its file name is the first 16 bytes of its SHA-256: Super Mario Galaxy's base program NCA hashed differently, with 12 MB runs of zeros near its end (147 MB of damaged RomFS: the two WebM prologue movies), while its update and the small NCAs matched their names.
|
||||
@@ -54,5 +55,6 @@ The work folder `$W` sits in the game's NSP folder, beside the NSP script's `<ti
|
||||
- Newer EPD titles store most files as `*.zs` (plain zstd, no dictionary in Mario; Zelda TotK needs `ZsDic.pack` dictionaries). Check for `*dic*` files before assuming plain zstd.
|
||||
- Older EPD titles (Super Mario 3D World + Bowser's Fury, a Wii U port) store almost everything as `*.szs`: Yaz0-compressed SARC. `shader_split.py` decompresses Yaz0 itself (`yaz0`, pure Python, about 20–45 MB/s per core; the whole 4.7 GB decompressed RomFS scans in about a minute). One of its SARCs (`SystemData/LayoutShader.szs`) is big-endian (BOM `FE FF`), which the SARC reader does not handle; it holds no shader container the game uses.
|
||||
- `*.sarc` / `*.pack` / `*.blarc` / `*.bfarc` are SARC archives; `*.byml` / `*.bgyml` are BYML (version 7 in Mario). A minimal reader for both is short enough to write inline when needed (see `shader_split.py` for SARC). Unity titles: `dm/gfx/tools/unityfs.py` reads UnityFS bundles, serialized files and objects by type tree (`bundle_files`, `serialized_objects`).
|
||||
- Capcom's RE Engine (Monster Hunter Rise) keeps everything in PAK archives: `re_chunk_000.pak` plus `re_chunk_000.pak.patch_001.pak` … `patch_016` (19.6 GB in all). A PAK is `KPKA`, u8 version 4, u8 0, u16 features (0: a plain entry table), u32 count, u32 fingerprint, then one 0x30-byte entry per file: u32 murmur hashes of the lower- and upper-case UTF-16 path, u64 offset, u64 stored length, u64 length, u64 attributes (bits 0–3: 0 stored, 1 raw deflate, 2 zstd), u64 checksum. File names are only hashes. The game reads all of them as one view: a path in a later patch replaces the same hash in earlier PAKs (`kpka_jobs` in `shader_split.py` takes only the newest copy). The first bytes of each file name its format (`MESH`, `TEX\0`, `SDF\0` shader definitions, `BKHD` / `AKPK` Wwise, `USR\0`, `PFB\0`, …).
|
||||
- Koei Tecmo's KTGL (Hyrule Warriors: Age of Calamity) keeps everything in RDB archives: `asset/<name>.rdb` is the index (`_DRK0000` header, then one `IDRK0000` record per file), and `asset/<name>.rdb.bin*` plus `asset/data/0x<KTID>.file` hold the data as a run of `IDRK0000` entries: u64 entry length, u64 stored length, u64 length, u32 0, u32 file KTID, u32 type KTID, u32 flags, a few parameter words, then the stored data at the end of the entry. Entries follow each other directly, some after padding to 64 KB. Flag 0x100000 = zlib in chunks: u32 stored length + a zlib stream each, up to 16 KB inflated per chunk, a zero length ends. The first bytes of the inflated data name the format (`GT1G` textures, `_M1G` models, `_S2G` shaders, `KTSR` / `TSRS` / `ASRS` sound). The DLC folders add `.rdb.bin3_0` files of the same layout.
|
||||
- The older KTGL of Fire Emblem: Three Houses uses a LINKDATA archive instead: `DATA0.bin` is an index of 32-byte entries `{u64 offset, u64 length, u64 stored length, u64 compressed}` into `DATA1.bin` (6.9 GB), and `patch1`–`patch4` hold loose overrides listed by `patchN/INFO0.bin` (0x120-byte entries ending in a `rom:/patchN/…` path). Its compressed entries and `*.gz` files are another chunked zlib: u32 chunk size (0x10000), u32 chunk count, u32 length, the stored length of each chunk, then the chunks at 0x80 alignment, each a u32 length + one zlib stream, or the plain bytes when the stored length equals the chunk's length (`kt_gz` in `shader_split.py`).
|
||||
@@ -40,6 +40,8 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **Checking a rewriter change**: rewrite every extracted module of every title with the old and the new `patch_nso.py` and compare all outputs; only `build_src.sha256` (the rewriter's own hash) may differ for titles the change is not meant for.
|
||||
- **SIGILL / trap on a system register read** → newer SDK reads a register macOS traps (`CTR_EL0` in SDK 16's `FlushDataCache`) → rewrite to a constant.
|
||||
- **`B()` range assertion in `patch_nso.py` (`site_plain`), only for one module** (Metroid Dread `main`) → its bss is 1.8 GB, the engine's static heap, so a patch area appended after the image starts 1.9 GB from the code and no `B` (±128 MB) reaches it. When an appended patch area would start beyond 112 MB, the rewriter puts it 32 MB *before* the module base (negative `PATCH_OFF`). `patched.bin` then holds only text + ro + data (`FILE_IMAGE_LEN`) followed by the patch bytes, and the loader maps the bss and the key page as zero pages (`IMAGE_SIZE`). The Mac loader, the iOS embedding (`gen_embed.py`: no bss in the executable) and `QueryMemory` handle both layouts; Mario's and Engage's output is byte-identical to before.
|
||||
- **`B()` range assertion, then `patch area of … bytes does not fit in the … before the base`** (Monster Hunter Rise `main`: 125.3 MB of text) → with the patch area 32 MB before the base, sites near the end of the text are more than 128 MB away. When the text ends within 32 MB of the reach (`BIG_TEXT`), the patch area goes right before the base: the stubs follow the sites in address order, so each site's stub is about as far from it as the text is long. The first pass tries the gap that keeps every site in reach (2.6 MB here); when the patch area is larger (3.6 MB, 34,322 sites), the rewriter runs itself again with exactly that size (`DM_PATCH_FRONT`, set only by the rerun) and `B()` checks every branch.
|
||||
- **`KeyError` at `site_br` for a `br x18` that is an ordinary byte jump table** (Monster Hunter Rise `main+0x6b346d0`) → capstone stops at the first word it cannot decode, and the function held a trap word (`0xe7ffdefe`, `main+0x6b34540`) before the jump table, so the table was never analysed. Function disassembly now goes on after undecodable words (`disasm_fn`).
|
||||
|
||||
- **Every pointer in `.data` / the GOT is a module offset, crashes everywhere** (SDK 20: The Legend of Zelda: Breath of the Wild 1.9.0) → the modules pack their relative relocations as `DT_RELR` (tag 36, size tag 35): `main` has 223,994 of them and its `DT_RELA` only symbol relocations. Each 8-byte entry is either an even offset (relocate it, move to the next slot) or an odd bitmap whose bits 1–63 stand for the following 63 slots; the addend is the slot's own value (`relr_apply` in `loader.c`; `DM_LOADHASH` and `nvn_slots_got.py` read it too). The loader's log line counts them as `relr=`.
|
||||
|
||||
@@ -52,7 +54,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **Sizing the reservation for a title that uses virtual address memory (Engage)** → such a title may never call `SetHeapSize`: everything comes from `MapPhysicalMemory` in the alias region. The `[mem]` log lines report the alias high-water mark (one line per 256 MB), the total actually mapped (page bitmap) and every heap growth. What fills the alias region first is address space, not memory: StandardAllocator reserves the whole free pool (pool − 64 MB, 1 GB aligned) but Engage commits only 0.58 GB of it, and Unity's own regions commit 0.97 GB. So Engage's profile reports a 2.5 GB pool (`app_pool`) and uses a 4.375 GB alias region in a 5 GB block (reserved address space peaks at 3.47 GB of 4.31 GB). iOS without the extended-virtual-addressing entitlement allows a single 6 GB reservation, but Metal needs address space too; 5 GB is the total proven on a phone. To measure commits, probe `VammManager::AllocateMemoryPages` / `FreeMemoryPages` (x1 address, x2 size) and split them by region.
|
||||
- **Large resources silently skipped, later a null deref where the resource is used** → the game reserves "virtual address memory" (Mario: 1.75 GB in `HeapDef.byml`) and maps it into the alias region with `MapPhysicalMemory` on demand; the alias region left after the node area was too small. Also visible as `SetHeapSize` asking for much less than the pool. → repartition the guest block (`hle_svc.c` `mem_init`): Mario uses alias 2.875 GB, stacks 256 MB, heap 1.625 GB (it asks for 1.47 GB), 5 GB in total. Measure before sizing (`DM_SVC=1`: the `SetHeapSize` request and the highest `MapPhysicalMemory` address). On iOS without the extended-virtual-addressing entitlement a single 6 GB reservation succeeds and 7 GB fails, but Metal and the rest of the process need address space too; 5 GB is the total proven on a phone (D2R).
|
||||
- **`MapPhysicalMemory` faults on part of a page** → round to 16 KB host pages.
|
||||
- **`nnMain` returns right after `nv::InitializeGraphics`, with no error** (Hyrule Warriors: Age of Calamity) → the game's `nninitStartup` sizes its heap as total − used memory (`QueryMemoryInfo`), calls `SetMemoryHeapSize` and hands the block to its own malloc. The host reports 64 MB used beside the heap, but `SetHeapSize` checks a request against the real size of the loaded modules (66.6 MB here), so `SetHeapSize(pool − 64 MB)` failed with ResultLimitReached, `AllocateMemoryBlock` returned 0x1003, malloc was never set up, the 866 MB graphics heap allocation came back null and `nnMain` returned. Probe `DM_SVC=1`'s `svc 0x1` request and the `AllocateMemoryBlock` result. The profile key `mem_used` sets the used-memory figure (72 MB for this game: the image plus stacks).
|
||||
- **`nnMain` returns right after `nv::InitializeGraphics`, with no error** (Hyrule Warriors: Age of Calamity) → the game's `nninitStartup` sizes its heap as total − used memory (`QueryMemoryInfo`), calls `SetMemoryHeapSize` and hands the block to its own malloc. The host reports 64 MB used beside the heap, but `SetHeapSize` checks a request against the real size of the loaded modules (66.6 MB here), so `SetHeapSize(pool − 64 MB)` failed with ResultLimitReached, `AllocateMemoryBlock` returned 0x1003, malloc was never set up, the 866 MB graphics heap allocation came back null and `nnMain` returned. Probe `DM_SVC=1`'s `svc 0x1` request and the `AllocateMemoryBlock` result. The profile key `mem_used` sets the used-memory figure (72 MB for this game: the image plus stacks). Monster Hunter Rise's modules take 320 MB (`main` alone 314 MB), so its `SetHeapSize(pool − 64 MB − bss)` failed the same way (`[mem] SetHeapSize … + modules … > app pool`); 328 MB gives a 2.88 GB heap.
|
||||
- **`SetHeapSize` asks for almost the whole application pool and fails at boot** (Metroid Dread: 0xc9400000, larger than the heap region) → the SDK sizes the heap as total − used memory, and the host reported "used" as heap + 64 MB. That 64 MB covers ordinary modules (Mario's two are about 74 MB, and its heap request matches hardware), but Dread's `main` carries a 1.8 GB bss that the Switch counts as used. Used memory now adds the bss of modules in the front-patch layout (`big_bss_bytes` in `hle_svc.c`); Dread then asks for 1.38 GB.
|
||||
- **Some textures are re-uploaded every frame (several MB per frame)** → the dirty-page tracker registers each memory pool rounded to 16 KB host pages, but NVN pools are only 4 KB aligned, so neighbouring pools share their first / last host page and one address can lie in several registered ranges. Taking the first range that contains an address judged textures in the second pool untracked (always dirty), released only one range on a write fault, and could unregister the neighbour when a pool was destroyed. `nvn_dirty.c` now uses the range that holds the whole span, releases every range covering a faulting page, and removes a range by the pool's exact start (`t_dirty_slots` covers it).
|
||||
- **A null write as soon as the main thread starts recording a command buffer** (13 Sentinels: a constructor at `main+0x2fa9a0` runs on `this = 0` when the prologue enters its first battle) → the object came from one of the engine's ring heaps (`main+0x2d83ec`), and that heap had base 0. At startup the engine takes one GPU memory pool, sized as exactly the sum of three ring heaps, and cuts the heaps out at 64 KB alignment *relative to the pool's GPU address* (`main+0x30c118`). On the Switch a pool's GPU address is big-page aligned; the host's GPU address is the CPU address, and the storage came from the game's `malloc` rounded only to 4 KB. Up to 60 KB was lost at the front, the third heap (16 MB) did not fit, and the sub-allocator returned 0. The game's `malloc` is `nn::mem::StandardAllocator::Allocate(size)`; on this build the host routes allocations of 64 KB or more to `Allocate(size, 0x10000)` (`hle_os.c`). Probing the heap object at the allocation (`DM_PROBE=main:2bec60@x0+8`) showed the zero base directly.
|
||||
@@ -85,10 +87,14 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **SDK 12 names** → older SDKs export other names for the same calls (`ListAudioDeviceName` / `GetActiveAudioDeviceName`, singular `hid::GetNpadState`, `OpenDefaultAudioOut`); hook both generations.
|
||||
- **SDK 11 names** (Metroid Dread) → `fs::CommitSaveData(mount)` is the older name of `fs::Commit`; `fs::DeleteDirectory` (an empty directory) was missing too.
|
||||
- **More SDK 11 calls that reach a service** (13 Sentinels) → `hid::GetMouseState` (the first boot aborted in it: the SDK's mouse resource was never initialized), `fs::CleanDirectoryRecursively`, `fs::EnableGlobalFileDataCache`, `oe::SetUserInactivityDetectionTimeExtendedUnsafe`, `time::ToPosixTime` (host `mktime`) and `audio::GetAudioOutName`. Finding them before the first boot: list `main`'s `nn::` imports that no host hook names (the survey's import table, filtered against `HLE_FORCE` names); most `nn::os` / `nn::mem` imports run natively in `sdk` and need nothing.
|
||||
- **SDK 11 calls of Monster Hunter Rise** (one boot each; the import list against the hooks and the other SDK 11 games found most of them before the first boot): `oe::IsCpuOverclockEnabled` (a null read in `sdk+0x35258`, apm's session, the first call in `nnMain`), `album::Initialize` (abort in `sm::Initialize` under `capsrv::InitializeForApplication`; saving to the album now fails, `hle_offline.c`), `oe::GetDefaultDisplayResolution` and its change event, `oe::SendServerMaintenanceOverlayNotification`, `time::LoadTimeZoneRule` and the `ToCalendarTime` overload with a rule (the host's zone is used), `time::StandardNetworkSystemClock::IsAccuracySufficient` (offline: no), `hid::GetKeyboardStates`, `fs::GetSaveDataSize` / `ExtendSaveData` and the one-argument `fs::MountCacheStorage`. `nvnDeviceGetTexelFetchHandle` returns the texture ID with sampler 0.
|
||||
- **Texture creation returns null right after the inline keyboard's `Initialize`, then a null read at `main+0x7713044`** (Monster Hunter Rise) → the game makes three textures of `swkbd::InlineKeyboard::GetWindowSize`, and the host answered 0 × 0 (its creation function returns null for a size of 0, `main+0x77112b8`). It is 1280 × 720 as on hardware now. SDK 11 adds keyboard setters (window mode, key top alpha / scale / position, touch, hardware keyboard, user words, dictionaries; ignored) and two callbacks the applet makes from `Calc`: finished-initialize after `Initialize` and released-user-words after `UnsetUserWordInfo`. The host makes them on the next `Calc`.
|
||||
- **A null-pointer crash early in `nnMain`, right after NVN setup should have happened** (Super Mario 3D World + Bowser's Fury, `main+0x759e08`: the device singleton `main+0x1a28ea0` is still null) → the game checks `nvnDeviceGetInteger(API_MAJOR / API_MINOR)` for major == 53 and minor ≥ 313 (`main+0x711c88`) and quietly skips creating its device otherwise. The reported version is the profile key `device_api` (default 55.15, Mario's requirement). NVN 53 also brings `nvnQueueAcquireTexture` (the older window acquire: the queue waits instead of a sync object), `nvnSamplerCompare` and the debug group calls (no-ops).
|
||||
- **Crash after the shared fonts load, looking up the "?" glyph (`main+0x6c73c0`, a font cache lookup returning null)** (3D World) → SDK 10's `ScalableFontEngine::LoadFont` failed with error 403 and `SetFont` with 301 on both stand-in fonts. This game records shared fonts as type 0 (`main+0x975658`), calls no `ScalableFontEngineHelper::Decode` and hands the `GetSharedFontAddress` pointer straight to `LoadFont`, so the pointer must be the plain TTF; Mario decodes the same pointer and needs the 8-byte BFTTF header in front. The profile key `shared_font` (`bfttf` default, `ttf` for 3D World) chooses what the address points at (`hle_sdk16.c`). Found by probing the engine's error code after each call (`DM_PROBE` on the instruction after `GetError`).
|
||||
- **More SDK 10 calls that reach a service** (3D World): `settings::LanguageCode::Make` and the reversed `operator==(LanguageCode, Language)` (the first comparison in `nnMain` aborted in `sm`), `account::ListAllUsers`, `oe::GetPerformanceConfiguration`, `oe::SetAlbumImageOrientation`, the `nifm` IP queries with SDK 10's `socket::InAddr` / `in_addr` mangling, `prepo::AddSessionId` / `AddInternetConnectionStatus` (entering the save select), and `vi::GetIndirectImageMemoryRequirement`.
|
||||
- **Null read in `applet::CreateLibraryApplet` under `swkbd::InlineKeyboard::Initialize`** (3D World, entering the save select after the character select) → the inline keyboard is a resident library applet whose results arrive through `Calc`. `hle_swkbd.c` takes over every entry that touches the applet and behaves as a keyboard the player always cancels: `Appear` arms a cancel and the next `Calc` calls the game's cancel callback; `GetImage` reports no picture. `GetImageMemoryRequirement` is not a pure computation: it asks vi, hence the vi hook above.
|
||||
- **Null read in `applet::CreateLibraryApplet` under `swkbd::InlineKeyboard::Initialize`** (3D World, entering the save select after the character select) → the inline keyboard is a resident library applet whose results arrive through `Calc`. `hle_swkbd.c` takes over every entry that touches the applet: `Appear` arms a request, and the next `Calc` asks for the text the way `ShowKeyboard` does (`DM_KBD`, else the front end's input box) and hands it to the game's changed-string and enter callbacks, or calls its cancel callback; `GetImage` reports no picture. `GetImageMemoryRequirement` is not a pure computation: it asks vi, hence the vi hook above.
|
||||
- **The hunter's name box stays empty, then stays open with the name in it** (Monster Hunter Rise, character creation) → the inline keyboard used to count every `Appear` as a cancel. Now the text arrives through the enter callback, whose argument is the text inline (`char16_t[502]`, the length at +0x3EC; the changed-string argument adds three cursor positions at +0x3F0..+0x3F8: `ikb_str_arg`, read off the game's callbacks at `main+0x6fb3910` / `main+0x6fb3afc`). The game then finishes the entry only when `Calc` returns 1 (ready / hidden) after a non-zero value (`main+0x6fb3e6c`); the hook returned nothing. `Calc` now returns 0 until initialization finishes, 3 while shown and 1 otherwise. `SetFinishedKeyboardCallback` is called on the `Calc` after `Disappear`. AppearArg's `textMaxLength` (+0x20) limits the text.
|
||||
- **`CRASH sig=10` at a heap address, called from `main+0x6bca160` right after the name was entered** (Monster Hunter Rise) → `nn::ngc::ProfanityFilter` is polymorphic and the game masks the name through its vtable (slot 8, `MaskProfanityWordsInText(int *, char16_t *, uint32_t)`). The constructor hook did nothing, so the object kept what the heap held. It now gets a host vtable (`ngc_vtable` in `hle_misc.c`): each slot of the SDK's `_ZTVN2nn3ngc15ProfanityFilterE` is matched by address with the class's methods, and slots the loader already bound to forced take-overs keep them. Nothing is masked or reported.
|
||||
- **Abort in `nn::sm::Initialize` under `nn::oe::FinishStartupLogo`** (Metroid Dread, right after the first frames) → it reaches the applet proxy through `am::GetApplicationFunctions`. The host has no startup logo: a no-op hook (`hle_oe.c`).
|
||||
- **Abort in `nn::sm::Initialize` under `nn::hid::VibrationNode::Update`** (Metroid Dread) → the SDK's vibration graph flushes queued values with `hid::detail::SendVibrationValues`, which opens the hid service. The game sets handles on its `VibrationTarget`s even when the host reports no vibration device. The flush calls it through the PLT, so an `HLE_FORCE` on that symbol catches it and returns success (`hle_hid.c`).
|
||||
- **An Atk task aborts in a `switch` default (`UnexpectedDefaultImpl`, then `AbortImpl`) while a stream sound loads** (Metroid Dread, `main+0x1225a8c`) → the stream path was `rom:/system/snd/../../sounds/streams/…`, and the host answered "not found" for any `..` component. Horizon's path normalizer resolves `.` and `..` lexically and fails only above the mount root; `host_path_ex` now does the same (`norm_rest` in `hle_fs.c`).
|
||||
@@ -123,6 +129,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **The game never reads the controller** (the title's "Press L+R" never sees a press; no `GetNpadState` call at all) → it waits for the event bound by `hid::BindNpadStyleSetUpdateEvent` before reading. The host now signals it once when it is bound for the connected controller.
|
||||
- **`nvnWindowGetPresentInterval`, `nvnCommandBufferSetCopyImageStride`** → implemented (the latter is the layer stride of texture ↔ buffer copies).
|
||||
- **Movie sound** → the decoder is asked for type 5 (Vorbis) with output format 1 (PCM16); see the movie entries above and `hle_movie.c`.
|
||||
- **Monster Hunter Rise's movies** → H.264 (type 1) into the client's NVN texture (output 3 = RGBA, mode 2), with AAC 5.1 sound (type 4, PCM16). VideoToolbox decodes to BGRA for that mode (`mavc_set_rgba`), and AAC goes through AudioToolbox's converter with an MPEG-4 ES descriptor built from the AudioSpecificConfig in `csd-0` (as FFmpeg's AudioToolbox decoder does). The game ends a stream with time −1 and the whole 4 MB input buffer as its length: that packet carries no data (decoding it gave VideoToolbox −12909 and AudioToolbox −50). It also deletes its decoders (`D0`), so the host frees the object with the game's `operator delete` after its own state.
|
||||
- **SDK 9.6 (Hyrule Warriors: Age of Calamity)**:
|
||||
- **Null read in `oe::InitializeCopyrightFrameBuffer`** (right after the window setup) → the copyright frame buffer is an overlay the system draws only into screenshots and recordings; the SDK hands it to am as transfer memory. `InitializeCopyrightFrameBuffer`, `SetCopyrightImage` and `SetCopyrightVisibility` are host no-ops (`hle_oe.c`).
|
||||
- **`HardwareOpusMultiStreamDecoder`** → the game builds it with the imported constructor and then calls it through its vtable. Its 12 slots match the SDK's software `OpusMultiStreamDecoder` one by one, so the constructor hook builds the software decoder in place (0x38 of the 0x48 bytes), as for the single-stream `HardwareOpusDecoder`; `GetHardwareOpusMultiStreamDecoderWorkBufferSize` returns the software size (`hle_codec.c`). Both destructors are a bare `ret`.
|
||||
@@ -132,6 +139,8 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **File-not-found noise** → games probe many optional paths (debug parameters, other-language voice files); with `DM_FS=1` these show as `fs … not found`. Only a missing file followed by a failure matters.
|
||||
|
||||
- **A buffer is created with a null pool right after a scene change** → NVN *virtual* memory pools (flag 0x200) have no storage until `nvnMemoryPoolMapVirtual` maps physical pool pages in; if the host returns failure, the game treats the pool as unusable. Mario maps its single 128 MB virtual pool onto one physical pool in one request, so the host aliases it (`hle_nvn.c`). True sparse mappings are still unimplemented.
|
||||
- **`[RED] MapVirtual: non-contiguous mapping (true sparse) not implemented`** (Monster Hunter Rise) → RE Engine's GPU heap is one 64 GB virtual pool built from separately allocated 4 MB physical pools, each mapped at a virtual offset of its own (45 at boot, over the first 180 MB). A virtual pool larger than the application pool cannot be one alias of a physical pool, so it gets an address range of its own at `MemoryPoolInitialize` (reserved, inaccessible: 64 GB on the Mac, at most 1 GB on iOS), and each mapping request `vm_remap`s the physical pages over its part of the range, so virtual addresses read and write the same memory, as the GPU's page table does. Each run of mapped pages is registered with the replay like a pool of its own and rebuilt when a request joins or splits runs (`vpool_*` in `hle_nvn.c`). Mario's 128 MB pool, mapped whole onto one physical pool, keeps the alias. The log reports the mapped high-water mark per 128 MB.
|
||||
- **`CRASH sig=10` in `memmove` from `pkg_upload` → `nvn_deswizzle`, at an address 32 GB into a sparse virtual pool** (Monster Hunter Rise) → the game's streamed textures (flag 0x20, packaged layout) live 32 GB into its virtual pool; it writes their tiles on the CPU into 4 MB physical pools and then maps them, with no GPU copy. NVN maps at 4 KB and the host aliases at its 16 KB page, so a request whose offsets differ modulo 16 KB cannot be aliased: the ASTC 6x6 textures' mip tail starts 0x81000 into the texture. Every request also goes into a 4 KB page table (`vpt_*` in `hle_nvn.c`: the physical page behind each virtual page and the generation of the request that set it). The replay reads a sparse pool's textures through it (`nvn_sparse_copy`, unmapped pages as 0) and uploads one again only when a mapping in its range changed or the CPU wrote a physical page behind it (`nvn_sparse_since`; the physical pools are dirty-tracked). Before that the untracked alias counted as written on every look: 32 textures, 18 MB, were uploaded again every frame, and those with unaliased tails came out magenta. Requests beyond the reserved range (iOS reserves 1 GB: the buffers stay under 0x20100000 through the opening) are kept in the table only, like unaligned ones: textures can read them, the GPU cannot address them.
|
||||
- **A null-pool buffer even with virtual pools working** → the game's own heap ran out. Trace it with probes ([05](05-debugging.md)): the gsys GPU allocator at `main+0x4290c0` and the heap's `tryAlloc` (vtable slot +0x30) show which heap, its size and every allocation into it; the `!xN` heap walk shows the free and used blocks at the moment of failure.
|
||||
- Mario: each resource gets its own `sead::ExpHeap` sized *resource-size-table value + 0x1008* (`main+0x1e603c` → `ExpHeap::create` `main+0x4aac38`). The table values are Nintendo's measured peaks with almost no margin (Yoshi's animation file uses 99 % of its budget). `Model/NpcCommonerA.bfres` in the opening course came up about 0x70 bytes short: its last GPU chunk (0x9000 bytes, 4 KB aligned) fit, but the chunk's 0x598-byte descriptor did not. Every allocation size before it is fixed by game code and data, so the difference from hardware is still unknown.
|
||||
- Fix: a registered code cut gives every resource heap +16 KB (`CUTS` in `dm/tools/patch_nso.py`, keyed by the module build ID, original bytes verified). Per-title code patches go there, with the reason.
|
||||
|
||||
@@ -14,6 +14,7 @@ Do not stop at the obvious archive. In Mario the main `bfsha` held 4,888 of the
|
||||
- **GLSLC outputs in `main`** (13 Sentinels, Vanillaware's engine: no shader file in RomFS, whose six CRI CPK archives hold textures, scripts, CRI audio and USM movies): 838 GLSLC compiler outputs sit in `main`'s read-only segment, each referenced from a `{pointer, size, 0}` table in `.data`. An output starts with the magic `0x19866891`, its total size is at +0x44, and it holds exactly one control section with the code area right after it (control +0x20 = control length); a debug section with the NV assembly and the GLSL source follows. Engage's `main` also has 11 of them, so `shader_split.py` reads them only when the profile sets `shader_main_glslc on` (and accepts only outputs that pass the control / code / length check). The stage comes from SPH word 0, as for BAHS.
|
||||
- **G2S in RDB archives** (Hyrule Warriors: Age of Calamity, Koei Tecmo's KTGL): each material's shaders are a G2S container (`_S2G3000`, type KTID `0x7bcd279f`; 23,108 entries in `asset/system.rdb.bin*`, 2.3 GB inflated) of GLSLC outputs, as in 13 Sentinels' `main`, inside zlib-chunked IDRK entries ([01](01-extract.md)); one `_DOK0000` container (type `0x20a6a0bb`, the engine's system resources) holds 151 more bare GLSLC outputs, 21 of them drawn in the first battle. `shader_split.py` maps the RDB files and inflates only entries that start with those two magics (`idrk_shaders`), with `shader_main_glslc on` for the 117 outputs in `main`. Many outputs carry 0x850-byte control sections and the compute ones have no code header, so `glslc_shaders` accepts controls from 0x760 bytes and takes the stage of headerless code from the thread group size at control +0x748. 40,726 shaders (37,043 pixel, 3,440 vertex, 241 compute, 2 geometry), Mario's texture handle layout (`0x20 + 8 × slot`); pairs come from `shader_separable on`. The older KTGL of Three Houses packs the same G2S containers differently (next item).
|
||||
- **SPKG in LINKDATA** (Fire Emblem: Three Houses, the older KTGL): shader packages (`SPKG`: `patch*/nx/shader/*.spkg` and 132 entries of `DATA1.bin`) of G2S containers as above, one `_SPK4100` package without G2S (the post effects, `DATA1.bin` entry 549), and map and model archives carrying packages of their own. `kt_shaders` inflates every LINKDATA entry and loose `*.gz` / `*.spkg` file ([01](01-extract.md)), takes every GLSLC output that passes the triple check, and scans its SARCs for BNSH; `DATA1.bin` is split into 64 scan jobs by entry range (`linkdata_jobs`). Its GLSLC (output version 0x1d.2) writes the 0x848 / 0x850-byte controls and headerless compute code described above, which `glslc_shaders` used to drop: 24 shaders before, 1,355 after. A few outputs of version 0x1d.0 keep something else at control +0x20 (odd values such as 0x136b), so a control length must be a multiple of 4. Nine more shaders exist only at run time (Seen in Fire Emblem: Three Houses, below).
|
||||
- **SDF in RE Engine PAKs** (Monster Hunter Rise): shader definitions (`SDF\0`, 2,685 files in the PAK view, [01](01-extract.md)): u16 version 1, u16 entry count, u64, then 0x108-byte entries from +0x10. Entry +0x10 holds six u64 offsets (vertex, hull, domain, geometry, pixel, compute; 0 = none), each to `{u32 control length, u32 code length}` followed by the control section (0x880 bytes) and the code area; compute code has no header. An entry's stages are one program, so its vertex / pixel stages are a pair. Empty definitions are 16 bytes of version 0. `kpka_jobs` maps the PAKs, takes the newest copy of each path, peeks at 4 decompressed bytes and inflates only SDF entries (`sdf_shaders`): 63,111 shaders (30,233 vertex, 21,475 pixel, 11,271 compute, 53 geometry, 33 hull, 46 domain) and 102,345 pairs, in 5 s with 12 jobs.
|
||||
- **BAHS** (agl `sharcb`, engine utility shaders): code region at header +0x1c (size +0x18). Binary entries in the table area are 16 bytes `{code offset, 0, control size (0x880/0x88c; 0x848 in BAHS version 9, Super Mario 3D World), signed offset from this entry to the control section}`. Control sections are shared between programs, so pair by entry, never by searching for code near a control.
|
||||
- **Shorter control sections (SDK 10, Super Mario 3D World + Bowser's Fury)**: the control section stores its own length at +0x20. Mario's, Engage's, Dread's and 13 Sentinels' are all at least 0x880 bytes; 3D World has 2,888 of 0x850 bytes (its ShaderData bfsha and six layouts) and 0x848 in BAHS version 9, each followed directly by other data. The key therefore hashes the first `min(0x880, control +0x20)` bytes (`ctl_len` in `shader_split.py`, `keys_hash` in `hle_nvn.c`); the other games' tables came out byte-identical.
|
||||
- **Compute code without a header (SDK 10)**: 3D World's particle compute shaders start their instructions at code +0 (like D2R), not after a `0x12345679` header. `shader_split.py` accepts both; `dmsh_build.py` strips only a header that is there.
|
||||
@@ -41,6 +42,8 @@ The NVN driver puts binding tables in driver constant banks, and the layout chan
|
||||
|
||||
Metroid Dread (SDK 11.4.3) uses Mario's texture handle layout, `0x20 + 8 × slot`: the `TEX` / `TEXS` handle immediates in its shader files' own SASS text are words 0x8–0x22 in steps of 2. Its full build leaves no unresolved bindings and no unfillable driver-bank reads; the other rows are not measured separately.
|
||||
|
||||
Monster Hunter Rise (RE Engine, SDK 11.4.3) uses both tables at once: combined handles from `nvnCommandBufferBindTexture` at `0x20 + 8 × slot` and separate textures from `BindSeparateTexture(s)` at `0x168 + 8 × slot`, with separate samplers at 0x568, and binds slot 0 of both in the same frames. A census of 23,478 translated shaders' texture names: 33,813 combined (`fp_t_tcb_1E`), 29,342 separate textures with a constant sampler ID (handle type 3, `fp_t_cb65538_3041005C`: texture at c2 word 0x5C, sampler 0x104 ORed in), 8,265 separate texture + separate sampler (type 1, `fp_t_cb196610_1017815A`), 5,227 separate textures whose own handle carries the sampler (`fp_t_tcb_5A`), about 1,500 uniform-buffer handles with a constant sampler (`fp_t_cb65541_30410028`), and 121 fully dynamic handles (type 4, `fp_t2d_cb0_40000000`, not supported). The profile keys `nvn_septex_base` / `nvn_septex_slots` give the separate table: separate binds go to slots 128 + n of the replay's table and their handles decode to 128 + n; a constant sampler is the binding kind `samp_id` (`NVN_RB_SAMPI`, reg = the sampler ID).
|
||||
|
||||
The profile's `nvn_*` keys hold these constants; `dmsh_build.py` decodes the bindings from them (`decode_tex`). `Program.cs` still has Mario's `TexBase` built in, but uses it only to look up per-slot texture / image types in the facts table, so a profile with other bases builds as long as its facts table has no such entries (Engage's has none; `dmsh_build.py` stops otherwise); the replay layer fills driver banks with the same layout: the texture, image and separate-sampler tables of `c2` (`NVN_RB_DRV2` in `encode_args`). Some shaders read `c2` rows as data at run time (Engage: 243, the texture and sampler handles of slots 0–11); the build lists reads outside those tables as `cannot fill`. After a full build, the summary must say `shaders with a binding whose slot was not decoded 0`.
|
||||
|
||||
## 4. Bindless textures
|
||||
@@ -213,6 +216,18 @@ A first-frame bring-up that worked for Mario: screenshot is black → `TEXDUMP`
|
||||
- **Indirect draws and dispatches**: `nvnCommandBufferDrawArraysIndirect`'s record `{count, instances, first, base instance}` has the layout of `MTLDrawPrimitivesIndirectArguments` (and `DrawElementsIndirect`'s `{count, instances, first index, base vertex, base instance}` that of `MTLDrawIndexedPrimitivesIndirectArguments`), so the replay issues Metal indirect draws from the pool buffer: the GPU writes the record earlier in the frame, so the CPU cannot read it at replay. The game's GPU-driven effects are indirect quads, which Metal has no primitive for. `ixc_run` in `nvn_mtl.m` converts such records (quads, fans, and indexed ones with 8-bit indices) with a compute pass into indexed indirect arguments in the frame arena, and the draw indexes the shared quad / fan index buffers (indexed ones get a 32-bit list expanded on the GPU). A compute pass cannot run inside a render pass. So when the first such draw is met, it converts every such draw up to the next buffer write outside a render pass (copy, update, fill, texture → buffer, dispatch; they bump `g_wr_epoch`). Each record is then read after its last writer, and only a draw met inside an already open render pass splits it. `DispatchComputeIndirect` → `dispatchThreadgroupsWithIndirectBuffer`; when the shader reads the driver segment, the GPU copies the counts into c0 + 0x200 first. `DM_NVN_IXCLOG=<n>` logs the first n conversions (record → arguments) once the GPU has run them.
|
||||
- **Movie frames through flushed CPU writes**: the decoder's NV12 output is copied by the game into a 1280×720 R8 and a 640×360 RG8 texture in a CPU-visible pool and published with `nvnMemoryPoolFlushMappedRange`; the profile turns on `tex_flush_upload`. Buffer textures (a 1,048,576-texel R32F one) stay out of that path: they are views onto the pool buffer, and an upload would have sent them past Metal's 2D size limit.
|
||||
|
||||
### Seen in Monster Hunter Rise
|
||||
|
||||
- **Subgroup operations in vertex shaders** (about 4 % of them): `SPIRV-Cross threw an exception: Subgroup builtins are not available in this type of function`. Metal has no SIMD-group functions in vertex functions, and SPIRV-Cross's own subgroup emulation covers only `Elect`. RE Engine scalarizes uniform loads: ballot the active lanes, shuffle the first lane's value to all and take a fast path when all agree. With a one-lane subgroup that is exact, so `dmsh_build.py` rewrites the vertex shaders' SPIR-V that way before SPIRV-Cross (`spv_single_lane`: the ballot is the lane's own bit, shuffles and broadcasts return its own value, all / any are the predicate, all-equal and elect are true, lane ID 0, size 1).
|
||||
- **Geometry shaders** (53) do not translate: SPIRV-Cross writes `unknown main0_out main0()` for the geometry stage, and Metal has none.
|
||||
- **Indirect-count multi-draws** (`nvnCommandBufferMultiDrawElementsIndirectCount` / `MultiDrawArraysIndirectCount`): the scene is GPU-driven. Compute culling writes the draw records (stride 20) and a u32 count, then up to `max` draws (1–49 per call) are made from them. Both were empty, and the G-buffer pass had no draws: the title screen showed only the UI. Metal has no draw count read by the GPU, so each call is recorded as `max` ordinary indirect draws that carry the count's address and their index (capped at 1,024). The replay reads the count once the GPU has written it: at the first such draw after a GPU write it commits the work so far and waits for it (`mdi_wait` in `nvn_mtl.m`), then skips draws past the count. That costs one wait per pass that uses it; hoisting the count into a GPU mask (as `ixc_run` does for index ranges) would remove it.
|
||||
- **Textures in the sparse virtual pool hold what the CPU wrote into physical pages before mapping them**: the streamed textures (flag 0x20, packaged layout) and plain sampled ones with no packaged layout. The replay took the plain ones for GPU textures, so their Metal copies stayed 0: the hunter's innerwear (ASTC 6x6) came out magenta, the ASTC error colour, and the face black. A sampled texture in a sparse pool (not a render target, image, linear or buffer texture) is now uploaded like a packaged one, through the pool's page table (03, sparse pools).
|
||||
- **Sparse textures are sampled only where the game has mapped them**: the translator drops TEX's LOD clamp and its residency predicate, so streamed textures were sampled at levels the game had not mapped yet, which read 0 (magenta on the title screen's clothes). A sparse texture is bound as a view starting at its first level whose pages, and those of every smaller level, are all mapped (`sparse_view` in `nvn_mtl.m`). That samples exactly as a minimum LOD would; the view follows each upload.
|
||||
- **Bindless handles in uniform buffers written earlier in the same submission**: the replay reads handles on the CPU, and a write queued earlier in the submission has not landed yet. Inline uniform updates keep where their data sits, so the handle is read from the newest one covering it (`pend_read`). The game queues more than the old limit of 256 writes per submission, after which every address counted as written; pending writes are now indexed by 4 KB page (8,192 per submission).
|
||||
- **Clear of a texture made without render-target use** (a 32x80 packaged one): a whole-level clear upgrades the texture to render-target use first, as format views are upgraded (`tex_upgrade`).
|
||||
- **Storage buffer addresses in the game's own uniform buffer** (about 30 compute shaders: c3 + 0x38): Ryujinx reports such a buffer with `cb_slot` 3, which had no NVN slot. The build marks it `sbuf_ubo` (uniform slot `cb_slot - 3`, byte offset), and the replay reads the 64-bit address there at bind time, as it reads bindless handles.
|
||||
- **Bindless handles through a loop phi** (218 pixel shaders): the handle is `c2[0x1A0] | 0x10500000` (separate texture slot 7, constant sampler 0x105), kept in a loop variable. Ryujinx's bindless elimination compared only the phi's direct inputs, gave up, and turned the access into an index into a 32-entry texture-pool array (`fp_t2d_cb0_40000000`, handle type Direct), which the replay cannot bind (`no resolved slot number`, the shader does not load). The vendored translator now follows the phis that feed a phi, and treats two `constant buffer | constant` handles as the same value ([third_party/ryujinx/README.md](../../dm/third_party/ryujinx/README.md)). Those accesses become ordinary type-3 handles.
|
||||
|
||||
### Seen in Super Mario Galaxy (Vulkan)
|
||||
|
||||
The game renders through Vulkan (`dm/gfx/vk/hle_vk.m`, written for Hades) and needed these additions; each was found from a red line or a screenshot on the route to the Star Festival:
|
||||
|
||||
+9
-1
@@ -12,7 +12,7 @@ The README covers packing and playing. This page is for working on nxnative itse
|
||||
| Audio | Games that carry the SDK's audio renderer (Mario) run it in-process; games that use AudioOut directly (Engage: Wwise) feed it straight. Output goes through Core Audio (default on iOS, `DM_AUDIO=1` on the Mac) |
|
||||
| Input | iOS: the touch gamepad and physical controllers. Mac: scripted input (`DM_KEYS=<seconds>:<button>,…`, with `DM_KBD=<text>` answering the system keyboard) or `DM_WINDOW=1` for a window with keyboard input |
|
||||
| iOS app | One unsigned IPA for iPhone and iPad. The rewritten code is embedded in the executable; the shader libraries, the RomFS and the game icon are inside the app |
|
||||
| Known gaps | Conditional render is always drawn; sparse virtual-pool mappings; `MultiDraw*IndirectCount`. TBDR optimizations are off by default (they don't understand bindless textures yet). No amiibo. The bundled prewarm lists are empty. On the Mac, Mario's menu and dialogue text render blank (fine on the iPhone) |
|
||||
| Known gaps | Conditional render is always drawn; `MultiDraw*IndirectCount` waits on the CPU for the GPU-written count; geometry shaders do not translate. TBDR optimizations are off by default (they don't understand bindless textures yet). No amiibo. The bundled prewarm lists are empty. On the Mac, Mario's menu and dialogue text render blank (fine on the iPhone) |
|
||||
|
||||
## The packing script in detail
|
||||
|
||||
@@ -234,6 +234,14 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- **Not yet:** GX points and lines (the geometry shaders), sampler LOD bias (GX uses it; Metal samplers have none, red line). The base NSP in hand was damaged (147 MB zeroed near its end: the two WebM prologue movies, which the storybook prologue does not use); it was packed with `--allow-damaged`.
|
||||
- **Mac input:** from a fresh save, `DM_KEYS="f1500:A,f1950:T0.373x0.70,f2300:A,f2700:A,f3200:A,f3600:LRight,f3800:A,f4200:T0.247x0.426,f4500:A,f4800:LRight,f5000:A,f5400:T0.659x0.366,f5800:A,f6200:LRight,f6400:A,f7000:A,f8000:A,f9000:A,f10000:A,f11000:A"` creates file 1 (Mario icon, Regular Mode) and plays the storybook prologue into the Star Festival (about frame 12,000). Screenshots of the handheld picture need `DM_NVN_SHOT_OUT=1280x720` (the raw swapchain image is 1920x1080 with the picture in its top-left 1280x720).
|
||||
|
||||
**Monster Hunter Rise (`0100B04011742000`)**
|
||||
- **Engine:** Capcom's RE Engine, SDK 11.4.3, update 16.0.2 (v2228224, Sunbreak). The base game is an XCI (cartridge dump; nx_extract reads its secure partition), the update an NSP, and its 252 DLC NSPs are licences only. `main` has 125 MB of text: the rewriter puts its patch area right before the base (`BIG_TEXT`, a second pass with `DM_PATCH_FRONT`).
|
||||
- **Files and shaders:** everything sits in KPKA PAK archives, later patch PAKs overriding earlier ones by path hash. The 63,111 shaders are in SDF files (`shader_split.py` `kpka_jobs`), with combined and separate texture tables at once (profile key `nvn_septex_base`).
|
||||
- **GPU heap:** one 64 GB sparse virtual pool built from 4 MB physical pools. The host aliases mapped pages (`vm_remap`) and keeps a 4 KB page table the replay reads textures through (streamed textures 32 GB in, mip tails at 4 KB offsets). The scene is GPU-driven (`MultiDrawElementsIndirectCount`).
|
||||
- **Keyboard:** the hunter, Palamute and Palico are named through the inline keyboard (`DM_KBD` on the Mac, the input box on the iPhone); the name is masked through `ngc::ProfanityFilter`'s vtable, which the host fills.
|
||||
- **Frame rate:** 30 fps, as on the Switch. On the Mac (M4 Pro) the title screen takes about 17 ms of GPU per frame docked.
|
||||
- **Mac input:** a fresh save (with the first-boot notices accepted) reaches character creation with `DM_KEYS="f4100:A~0.3,f6300:A~0.3,f7000:A~0.3,f7600:A~0.3"`. With `DM_KBD=Hunter`, `DM_KEYS="f4100:A~0.3,f6300:A~0.3,f7000:A~0.3,f7600:A~0.3,f8000:A~0.3,f8150:Down~0.25*6/40,f8450:A~0.3,f8650:A~0.3,f8900:A~0.3,f9100:A~0.3,f9500:A~0.3,f9800:Down~0.25*9/40,f10250:A~0.3,f10450:A~0.3,f10700:A~0.3,f10950:A~0.3,f11300:A~0.3,f11800:Down~0.25*10/40,f12250:A~0.3,f12450:A~0.3,f12700:A~0.3,f12950:A~0.3,f13300:A~0.3,f16000:A~0.3,f16300:A~0.3"` goes on through creation (hunter, Palamute, Palico, each named) and saves to slot 1. Holds under 0.25 s are not seen, and 0.5 s repeats in lists. From a save made that way, `f4100:A~0.3,f6300:A~0.3,f6800:A~0.3,f7100:A~0.3,f7500:A~0.3` continues into the opening.
|
||||
|
||||
## Tools (`dm/tools`)
|
||||
|
||||
| Tool | What it does |
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 164 KiB |
@@ -314,7 +314,7 @@ def version_of(path):
|
||||
def scan_nsps(nsp_dir):
|
||||
"""Group by game: {application title ID: {'base': path, 'updates': [(version, path)], 'dlc': [(title ID, version, path)]}}"""
|
||||
fams, unknown = {}, []
|
||||
for p in sorted(nsp_dir.glob('*.nsp')):
|
||||
for p in sorted([*nsp_dir.glob('*.nsp'), *nsp_dir.glob('*.xci')]): # a base game may be a cartridge dump (.xci: Monster Hunter Rise); nx_extract reads its secure partition
|
||||
if p.stat().st_size == 0: # an empty file left by an incomplete download (the Zelda BotW folder holds a 0-byte DLC NSP next to the complete (1).nsp)
|
||||
say(f'skipping an empty file: {p.name}'); continue
|
||||
tid = title_of(p)
|
||||
|
||||
Reference in new issue
Block a user