Advance Wars 1+2: Re-Boot Camp (0100300012F2A000, base game 1.0.0, Unity 2020.2.1f3 IL2CPP on SDK 10.4.1): game profile, the SDK 10.4.1 sdk build's address-region cuts and a shader split reader for control sections loose in main. Plays on the iPhone (IPA 7.1 GB, user). On the Mac from the boot through the Bink opening movie, the title, Nell's introduction, a new casual campaign and the first training battle (Field Training, Troop Orders): 60 fps in the menus, 30 fps in battle by the game's own limit (present interval 1, the frame's work 33 ms with CPU and GPU mostly idle), FMOD sound through AudioOut, about 2.5 GB resident and 1.6-1.9 GB of Metal allocations, level over a 15-minute run with no red line. sdk_patches.py: the sdk build is The Great Ace Attorney Chronicles' (no virtual address memory there, so it never needed a cut); this game's NPDM declares 16 MB of system resource and nninitStartup (main+0x365310) hands pool - used to StandardAllocator, whose Initialize (sdk+0x2edf90) aborted at boot. The AddressRegionManager constructor (sdk+0x340e90) loads the 1 GB node area with orr w8 / w9, wzr, #0x3fe00000 (the SDK 7 / 8 form, +0x340ea0 / +0x340f3c), cut to 62 MB, and AllocateAddressRegion's 1 GB alignment of regions of 1 GB and more (+0x34109c) to 32 MB, as on DRAGON QUEST XI S's SDK 7.3.2 build. Profile: the NSP script's heap check set a 1 GB pool, as for the other Unity games with virtual address memory, but the game's malloc then ran dry at the first loading screen and Unity's GPU pool allocation (main+0x96f430) halved its aligned_alloc request down to 0 and spun there (frames stopped 17 s in; found with sample on the Mac host); with 2 GB the alias region is mapped up to 2.26 GB (high-water mark +3.27 GB of 4.375) in Engage's 5 GB block. shader_split.py raw_shaders: Bink 2's Unity plugin builds its programs from 32 control sections and code areas in main's read-only segment that are in no GLSLC output and no BNSH file (2 vertex, 22 pixel, 8 compute), so the movies drew white (two shaders not in the map and a program without a vertex shader as the opening movie started). 26 are 0x20-byte records in .data ({code pointer, control pointer, u32 code length, u32 control length}, found through the relative relocations, RELA or RELR: rel_targets); the code takes the other 6 by address, their code area right after the control section's own length or right before it, and two share one; both neighbours can pass the header and length checks (main+0x4a2c670, a compute shader, would have taken the first 0x100 bytes of the next vertex shader's code), so only a code area whose address the code builds with adrp + add nearest to the control section's is taken (adrp_targets). Compute code areas have no header (SDK 10), so the stage is glslc_stage's, and a code area may not hold a control section. Control sections another reader took from main are left alone (Celeste's MonoGame effect). Of the 25 mains on the game drive the reader adds shaders only to this one, so every other profile's tables stay as they were (Celeste and the other games on the second drive not checked: not mounted). 20,424 shaders, all build for the Mac and iOS. Checked: the NSP script packs it from the NSP alone (Mac and iOS), the tool tests pass. README (games table, a Mac picture of the first battle), AGENTS.md (52 -> 53), dm/games/README.md (the heap check's 1 GB is a starting point), docs/development.md and docs/agents 03 / 04 say so
This commit is contained in:
seedds authored and MacMini committed
1 parent
7861e7c1a9
commit
6d42bc9815
16 files changed
+30834
-4
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Notes for agents
|
||||
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Fifty-two games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`), Monster Train 2 (update 1.1.1, SDK 18.3.5, `010051701FB46000`), Raging Loop (base game 1.0.0, SDK 7.3.2, `010020400E1C2000`), DARK SOULS: REMASTERED (update 1.0.3, SDK 4.5.0, `01004AB00A260000`), Paper Mario: The Thousand-Year Door (update 1.0.1, SDK 15.3.2, `0100ECD018EBE000`), Hollow Knight: Silksong (update 1.0.30000, SDK 20.5.8, `010013C00E930000`), Catherine: Full Body (update 1.0.1a, SDK 8.2.1, `0100BF00112C0000`), Digimon Story Cyber Sleuth: Complete Edition (update 1.0.3, SDK 6.4.0, `010014E00DB56000`), Animal Crossing: New Horizons (update 3.0.3, SDK 20.5.4, `01006F8002326000`), DRAGON QUEST VII Reimagined (update 1.2.0, SDK 19.3.1, `0100A9D01C446000`), FINAL FANTASY RESONANCE (update 1.0.1, SDK 20.5.17, `0100606026F12000`), Luigi's Mansion 3 (update 1.4.0, SDK 7.3.2, `0100DCA0064A6000`), AI: THE SOMNIUM FILES (update 1.0.1, SDK 6.4.2, `010089B00D09C000`), ANIMAL WELL (update 1.0.10, SDK 20.5.7, `010020D01AD24000`), AI: THE SOMNIUM FILES - nirvanA Initiative (update 1.0.2, SDK 13.3.5, `0100713016CCC000`), Mario Kart 8 Deluxe (update 4.0.0, SDK 4.4.0, `0100152000022000`), Balatro (update 1.1.3, SDK 18.3.2, `0100CD801CE5E000`), ROMANCE OF THE THREE KINGDOMS 8 REMAKE (update 1.1.3, SDK 17.5.3, `0100C2A01A730000`), Taiko Risshiden V DX (update 1.1.1, SDK 12.3.3, `0100346017304000`), Super Smash Bros. Ultimate (update 13.0.5, SDK 8.2.1, `01006A800016E000`) and Bayonetta 3 (update 1.2.0, SDK 13.3.2, `01004A4010FEA000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`FRAME_STEP` in their profiles' `patches.py`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout, and its `sdk` build (Breath of the Wild's) has the SDK's address-region node area cut (`sdk_patches.py`): without it the resource heaps ran out of alias region at the first descent from the Great Sky Island and the game played its memory-shortage Blood Moon, which put Link back at the start (its log shows `MSF`). Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK aLine truncated
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Fifty-three games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`), Monster Train 2 (update 1.1.1, SDK 18.3.5, `010051701FB46000`), Raging Loop (base game 1.0.0, SDK 7.3.2, `010020400E1C2000`), DARK SOULS: REMASTERED (update 1.0.3, SDK 4.5.0, `01004AB00A260000`), Paper Mario: The Thousand-Year Door (update 1.0.1, SDK 15.3.2, `0100ECD018EBE000`), Hollow Knight: Silksong (update 1.0.30000, SDK 20.5.8, `010013C00E930000`), Catherine: Full Body (update 1.0.1a, SDK 8.2.1, `0100BF00112C0000`), Digimon Story Cyber Sleuth: Complete Edition (update 1.0.3, SDK 6.4.0, `010014E00DB56000`), Animal Crossing: New Horizons (update 3.0.3, SDK 20.5.4, `01006F8002326000`), DRAGON QUEST VII Reimagined (update 1.2.0, SDK 19.3.1, `0100A9D01C446000`), FINAL FANTASY RESONANCE (update 1.0.1, SDK 20.5.17, `0100606026F12000`), Luigi's Mansion 3 (update 1.4.0, SDK 7.3.2, `0100DCA0064A6000`), AI: THE SOMNIUM FILES (update 1.0.1, SDK 6.4.2, `010089B00D09C000`), ANIMAL WELL (update 1.0.10, SDK 20.5.7, `010020D01AD24000`), AI: THE SOMNIUM FILES - nirvanA Initiative (update 1.0.2, SDK 13.3.5, `0100713016CCC000`), Mario Kart 8 Deluxe (update 4.0.0, SDK 4.4.0, `0100152000022000`), Balatro (update 1.1.3, SDK 18.3.2, `0100CD801CE5E000`), ROMANCE OF THE THREE KINGDOMS 8 REMAKE (update 1.1.3, SDK 17.5.3, `0100C2A01A730000`), Taiko Risshiden V DX (update 1.1.1, SDK 12.3.3, `0100346017304000`), Super Smash Bros. Ultimate (update 13.0.5, SDK 8.2.1, `01006A800016E000`), Bayonetta 3 (update 1.2.0, SDK 13.3.2, `01004A4010FEA000`) and Advance Wars 1+2: Re-Boot Camp (base game 1.0.0, SDK 10.4.1, `0100300012F2A000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`FRAME_STEP` in their profiles' `patches.py`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout, and its `sdk` build (Breath of the Wild's) has the SDK's address-region node area cut (`sdk_patches.py`): without it the resource heaps ran out of alias region at the first descent from the Great Sky Island and the game played its memory-shortage Blood Moon, which put Link back at the start (Line truncated
|
||||
- Each game's constants and shader tables form a game profile in `dm/games/<id>/`, where `<id>` is the base game's title ID: `game.tsv` plus the tables (and `patches.py` when the rewriter has patches for its builds). Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)).
|
||||
- Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it.
|
||||
- Per-build knowledge is keyed by the NSO build ID: rewriter cuts, observation gates and frame-step stubs in the profile's `patches.py` (a game's own modules) or `dm/tools/sdk_patches.py` (Nintendo's `sdk` modules, which several games ship), which `dm/tools/patch_nso.py` (the general rewriter) loads, the profile's `build` rows (the builds the game was brought up on, and their display versions; the NSP script packs any other version from a derived profile, without those builds' fixes), and host behaviour that uses fixed module addresses checks `module_t.build_id` (from `fixups.tsv`; `dm_main_build()`). When a new game build appears, carry addresses over with `dm/tools/nso_fnmap.py` and check them by disassembly; never let fixed addresses run on a build they were not found in.
|
||||
|
||||
@@ -52,6 +52,7 @@ Needs iOS / iPadOS 26 or later on an iPhone 15 Pro or newer (A17 Pro chip or lat
|
||||
| <br>Mario Kart 8 Deluxe<br><sub>IPA ~11.9 GB · update 4.0.0 · Mii heads not drawn (no firmware)</sub> | <br>Balatro<br><sub>IPA ~82 MB · update 1.1.3 · played by touch</sub> |
|
||||
| <br>ROMANCE OF THE THREE KINGDOMS 8 REMAKE<br><sub>IPA ~16.8 GB · update 1.1.3 · controller only</sub> | <br>Taiko Risshiden V DX<br><sub>IPA ~3.3 GB · update 1.1.1 · Japanese · played by touch</sub> |
|
||||
| <br>Super Smash Bros. Ultimate<br><sub>IPA ~17.5 GB · update 13.0.5 · offline</sub> | <br>Bayonetta 3<br><sub>IPA ~14.2 GB · update 1.2.0</sub> |
|
||||
| <br>Advance Wars 1+2: Re-Boot Camp<br><sub>IPA ~7.1 GB</sub> | |
|
||||
|
||||
## Pack an IPA
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
# generated
|
||||
|
@@ -0,0 +1,48 @@
|
||||
# Game profile of Advance Wars 1+2: Re-Boot Camp (Switch, base game v0 = 1.0.0, the only version). Plays on the Mac and on the iPhone
|
||||
# Format: see dm/games/README.md
|
||||
# Unity 2020.2.1f3 IL2CPP on SDK 10.4.1 (the sdk and movie module builds of The Great Ace Attorney Chronicles), NVN, FMOD (sound through AudioOut), Bink 2 movies (.bk2) decoded by Bink's own
|
||||
# Unity plugin in main, Pia / NEX for online play. Everything is in Addressables bundles under Data/StreamingAssets/aa/Switch, read as Engage's (unityfs.py).
|
||||
# Shaders: 20,392 from the bundles and Unity's 11 GLSLC outputs in main, and the Bink plugin's 32 (2 vertex, 22 pixel, 8 compute), whose control sections and code areas lie loose in main
|
||||
# (shader_split.py raw_shaders); without them the opening movie and every other movie drew white
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 0100300012F2A000
|
||||
program_id 0100300012F2A000
|
||||
|
||||
# ---- App ----
|
||||
# Game name: only a label for profile listings and the Mac window title; the iOS app name is the game name in the base NSP, and the bundle ID comes from the base game title ID (build_ipa.sh / NSP script)
|
||||
name Advance Wars 1+2: Re-Boot Camp
|
||||
ipa AdvanceWars12ReBootCamp.ipa
|
||||
app_category public.app-category.strategy-games
|
||||
|
||||
# ---- Versions ----
|
||||
build v0 1.0.0 main=320a17744aefd67fec5738ca0c629795 subsdk0=84e50460493a12f26100e5150fbff271 sdk=d9add76fc264863d85fbbd1d70076c76
|
||||
package v0
|
||||
shader_versions v0
|
||||
|
||||
# ---- Files ----
|
||||
# load order as on Horizon: main, subsdk*, sdk (subsdk0 = NintendoSDK_movie, which main imports from; the movies seen so far, the opening among them, are Bink's)
|
||||
modules main subsdk0 sdk
|
||||
romfs_marker Data/StreamingAssets/music.bank
|
||||
zip_store .assets:.bank:.bk2:.bundle:.dat:.ress:.metallib
|
||||
|
||||
# ---- Memory ----
|
||||
# The NPDM declares 16 MB of system resource (virtual address memory): nninitStartup (main+0x365310) hands pool − used to StandardAllocator as one address region, and main sets no heap;
|
||||
# Unity reserves its own regions beside it. Engage's 5 GB block (4.375 GB alias region), with the SDK's address-region node area cut and regions of 1 GB and more aligned to 32 MB on the
|
||||
# SDK 10.4.1 sdk build (dm/tools/sdk_patches.py). The NSP script's heap check (the hardware pool used the alias region up to +4.25 GB within seconds) set a 1 GB pool, as for the other
|
||||
# Unity games with virtual address memory, but this game's malloc then failed at the first loading screen: Unity's GPU pool allocation (main+0x96f430) halves its request after each failed
|
||||
# aligned_alloc down to a size of 0 and spins there for good (frames stopped 17 s in). With 2 GB: 2.26 GB of the alias region mapped and its high-water mark +3.27 GB through 15 minutes of
|
||||
# the title, the opening movie, the menus and the first training battle; the process holds about 2.5 GB on the Mac, 1.6–1.9 GB of it Metal allocations
|
||||
mem_reserve 0x140000000
|
||||
mem_alias 0x118000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0x80000000
|
||||
sys_resource 0x1000000
|
||||
|
||||
# ---- Input ----
|
||||
# The map cursor moves with the D-pad (the tutorial battle was played with it on the Mac); the 3D layout has one
|
||||
pad_layout full
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_groups /Data/=data
|
||||
shader_note Advance Wars 1+2: Re-Boot Camp shaders
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# Prewarm seed list (shipped in the app, next to shader_map.tsv). Rows: S<TAB>shader key<TAB>stage / R<TAB>render pipeline key (pso_skey hex) / C<TAB>compute shader key.
|
||||
# Provisional profile: empty until recorded on the Mac (DM_PSO_MANIFEST=off DM_PSO_RECORD).
|
||||
|
@@ -0,0 +1,103 @@
|
||||
# Answer table for translation queries (generated by dm/gfx/tools/shader_split.py, do not edit): key <TAB> fact JSON. This game has no PC version to pair with: only the thread group / shared memory of compute shaders (taken from the control section); other queries get the translator defaults
|
||||
00bd3e13d5c0ae84 {"local":[64,1,1],"shared":2240}
|
||||
0b519c896f541066 {"local":[32,32,1],"shared":0}
|
||||
0b81eb0a7f07b193 {"local":[64,1,1],"shared":0}
|
||||
1098e6f731304a9c {"local":[16,16,1],"shared":1024}
|
||||
111832f4ee0993af {"local":[32,32,1],"shared":0}
|
||||
143ffb8648af4032 {"local":[4,4,4],"shared":0}
|
||||
1540f38a33e501a4 {"local":[32,32,1],"shared":0}
|
||||
1796b1640057be4f {"local":[64,1,1],"shared":0}
|
||||
194164fb95d69b6d {"local":[16,16,1],"shared":0}
|
||||
1c264b099c42dc63 {"local":[8,8,1],"shared":3072}
|
||||
1d05ce56509811ff {"local":[4,4,4],"shared":0}
|
||||
1fc72af66f799ddc {"local":[64,1,1],"shared":0}
|
||||
20d00bc1eec3ee1e {"local":[64,1,1],"shared":0}
|
||||
275959166e6bda74 {"local":[64,1,1],"shared":0}
|
||||
2a05e8fcde2b6c64 {"local":[16,16,1],"shared":512}
|
||||
2aa6030f24894b9c {"local":[64,1,1],"shared":3200}
|
||||
2da0e082e41e95a0 {"local":[64,1,1],"shared":0}
|
||||
2f77f78a70575f8c {"local":[64,1,1],"shared":0}
|
||||
31805059cb151042 {"local":[32,32,1],"shared":0}
|
||||
34729b985ab19523 {"local":[8,8,1],"shared":3072}
|
||||
38c40704e8082fe4 {"local":[64,1,1],"shared":0}
|
||||
3ab48be6cee19625 {"local":[16,16,1],"shared":0}
|
||||
44c9e985c05a3c5c {"local":[8,8,1],"shared":6144}
|
||||
457b84b81d291a61 {"local":[32,32,1],"shared":0}
|
||||
4df1d33620adb21f {"local":[64,1,1],"shared":0}
|
||||
50233252b7d9a8fe {"local":[8,8,1],"shared":3072}
|
||||
52b1bbefe273e27b {"local":[8,8,1],"shared":0}
|
||||
546f2662a79af6fa {"local":[64,1,1],"shared":0}
|
||||
552bbc8ffc2459e4 {"local":[4,4,4],"shared":0}
|
||||
5a7a07d9a4abbbaa {"local":[64,1,1],"shared":0}
|
||||
5be253c503dabe56 {"local":[64,1,1],"shared":0}
|
||||
600f8412e31c5177 {"local":[32,32,1],"shared":0}
|
||||
608a1828446ebe28 {"local":[64,1,1],"shared":0}
|
||||
60a02babe0de0814 {"local":[64,1,1],"shared":8192}
|
||||
627f4bb2bbfa7a72 {"local":[8,8,1],"shared":0}
|
||||
628bf7cf471c47f8 {"local":[8,8,1],"shared":0}
|
||||
65eaac88f4268018 {"local":[32,32,1],"shared":0}
|
||||
690188350775b8d4 {"local":[32,32,1],"shared":0}
|
||||
699a1edec58c7e4f {"local":[64,1,1],"shared":0}
|
||||
699be469db01c15e {"local":[16,1,1],"shared":0}
|
||||
6b6b7330d718c025 {"local":[8,8,1],"shared":2048}
|
||||
6df2140eaeb210aa {"local":[64,1,1],"shared":128}
|
||||
6e9d314c02162a85 {"local":[64,1,1],"shared":0}
|
||||
735dbd6813474cad {"local":[64,1,1],"shared":0}
|
||||
7458b4f9ba171258 {"local":[32,32,1],"shared":0}
|
||||
7825ef009c990012 {"local":[64,1,1],"shared":0}
|
||||
78846d02ee6b62df {"local":[8,8,1],"shared":6144}
|
||||
7bcf043fab0b1717 {"local":[16,16,1],"shared":0}
|
||||
82c0830ca84f26ac {"local":[32,32,1],"shared":0}
|
||||
82ce2327a39e22ba {"local":[64,1,1],"shared":2144}
|
||||
82dcec9ffea8d4c5 {"local":[64,1,1],"shared":0}
|
||||
859d5bff6fa8f7b3 {"local":[64,1,1],"shared":0}
|
||||
8d9a8e32d3305994 {"local":[64,1,1],"shared":4320}
|
||||
9080b165cf0b7589 {"local":[64,1,1],"shared":0}
|
||||
91cb7f8dbb7c5301 {"local":[64,1,1],"shared":0}
|
||||
91cd1fc372a67748 {"local":[16,16,1],"shared":256}
|
||||
93436d1ee11f161e {"local":[16,16,1],"shared":8192}
|
||||
935e1617d132fe5a {"local":[8,8,1],"shared":0}
|
||||
936a7bdc19bd9c80 {"local":[32,32,1],"shared":0}
|
||||
93d1fd29dd917b81 {"local":[64,1,1],"shared":0}
|
||||
95e6058d717c0976 {"local":[4,4,4],"shared":0}
|
||||
9bf881bff46b9aad {"local":[8,8,1],"shared":2048}
|
||||
9dbff10c6dd1e398 {"local":[8,8,1],"shared":6144}
|
||||
a09db01872573f5d {"local":[8,8,1],"shared":6144}
|
||||
a1e18eb6aa20a872 {"local":[64,1,1],"shared":0}
|
||||
a29e42d08bb8b442 {"local":[32,32,1],"shared":0}
|
||||
a9b1520f7531c33f {"local":[16,16,1],"shared":4096}
|
||||
a9cbcbbaaae1e2f5 {"local":[4,4,4],"shared":0}
|
||||
aaf064f70e07e9c9 {"local":[64,1,1],"shared":0}
|
||||
ac2819060036c62a {"local":[64,1,1],"shared":0}
|
||||
b2c4da0e4b1eca51 {"local":[16,8,1],"shared":512}
|
||||
b4f38bf08fe41433 {"local":[8,8,1],"shared":3072}
|
||||
b830ae06a715f6ed {"local":[8,8,1],"shared":1024}
|
||||
bbefd4aa50fb7a43 {"local":[32,32,1],"shared":0}
|
||||
bd647d3dc9fe7aaf {"local":[1,256,1],"shared":0}
|
||||
be881be7f00a399e {"local":[64,1,1],"shared":0}
|
||||
bece43eed7871290 {"local":[64,1,1],"shared":0}
|
||||
c395ca75d10d7eb5 {"local":[64,1,1],"shared":0}
|
||||
c4145fbd8e5c6132 {"local":[64,1,1],"shared":0}
|
||||
c481407f55f75a8f {"local":[4,4,4],"shared":0}
|
||||
c4e213a39f9e9852 {"local":[8,8,1],"shared":3072}
|
||||
c5d7742328c0214e {"local":[8,8,1],"shared":2048}
|
||||
cfc03ec0b71ae644 {"local":[8,8,1],"shared":0}
|
||||
d59053ad67bd05eb {"local":[8,8,1],"shared":3072}
|
||||
d5de4986bd207c70 {"local":[256,1,1],"shared":0}
|
||||
d77f114cd5be80c5 {"local":[32,32,1],"shared":0}
|
||||
d8b89a09d64cd124 {"local":[4,4,4],"shared":0}
|
||||
d92a53c2ccd3d4d4 {"local":[32,32,1],"shared":0}
|
||||
db402ef0cd4be5de {"local":[64,1,1],"shared":0}
|
||||
dd35e9a6ac76d497 {"local":[64,1,1],"shared":0}
|
||||
e1f552dd188b2f28 {"local":[64,1,1],"shared":0}
|
||||
e24fbbfad685128d {"local":[64,1,1],"shared":256}
|
||||
ebe11d32a5a24fcb {"local":[64,1,1],"shared":0}
|
||||
ecad0f8f52365a39 {"local":[64,1,1],"shared":0}
|
||||
ee32e29fb1341aa6 {"local":[64,1,1],"shared":0}
|
||||
ee9a9ca2597b750e {"local":[16,8,1],"shared":512}
|
||||
eef08b58a7a1392a {"local":[32,32,1],"shared":0}
|
||||
f11fc6a615d16aee {"local":[64,1,1],"shared":0}
|
||||
f3b359713819fc19 {"local":[16,1,1],"shared":0}
|
||||
f8be37a0384a71bb {"local":[64,1,1],"shared":0}
|
||||
ff2880a09c099144 {"local":[8,8,1],"shared":1024}
|
||||
ff7d2074a896cad5 {"local":[64,1,1],"shared":0}
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 19.
|
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
# kind: evidence control-section key conflict rulings: empty
|
||||
dm_ctl_sha16 chosen_pc class candidates records
|
||||
|
File diff suppressed because it is too large.
Load diff
+1
-1
@@ -94,7 +94,7 @@ These are not profile keys: the tools read them from the game files each time, b
|
||||
|
||||
`python3 dm/games/profile_guess.py --check` runs these rules on every profile in `dm/games` (with the extractions under `/Volumes/T7/nsp`) and lists the keys where the guess differs.
|
||||
|
||||
After the rewrite, `mem_used` is set when the rewritten modules exceed the default 64 MB (a game without a system resource); keep it when the profile moves into `dm/games` (Cuphead without it: a 0.03 GB heap and an abort at the first allocation). A game with a system resource that sets a heap then gets a **heap check**: it boots in the Mac host (headless, no shader libraries) until its first heap request, or 5 s into `nnMain` without one, at most 30 s (`HEAP_CHECK_SECS`). A request the heap region cannot hold (`SetHeapSize OOM`) sets the alias / heap split (Tears of the Kingdom asks for 0x86000000, which gives its hand-made 0x98000000 alias region; Wonder's 1.44 GB gives 0xc0000000), and a pool check that fails for good sets `mem_used`. A run with no heap set whose alias region is used up to within 0.25 GB of its end sets a 1 GB pool: `nninitStartup` handed pool − used to StandardAllocator as one address region and the game reserves regions of its own beside it (Unity 2020 and 2022: AI: THE SOMNIUM FILES - nirvanA Initiative reached +4.26 GB of 4.38 in 5 s with the hardware pool; Unity 2018.3's Raging Loop maps only inside StandardAllocator's region and keeps it). The profile gets a `# heap check:` line, and later runs keep its keys (delete the line to check again; `--no-heap-check` / `HEAP_CHECK = False` skips it). Other games skip the check, so it costs nothing for them and a few seconds plus a Mac host build for the rest.
|
||||
After the rewrite, `mem_used` is set when the rewritten modules exceed the default 64 MB (a game without a system resource); keep it when the profile moves into `dm/games` (Cuphead without it: a 0.03 GB heap and an abort at the first allocation). A game with a system resource that sets a heap then gets a **heap check**: it boots in the Mac host (headless, no shader libraries) until its first heap request, or 5 s into `nnMain` without one, at most 30 s (`HEAP_CHECK_SECS`). A request the heap region cannot hold (`SetHeapSize OOM`) sets the alias / heap split (Tears of the Kingdom asks for 0x86000000, which gives its hand-made 0x98000000 alias region; Wonder's 1.44 GB gives 0xc0000000), and a pool check that fails for good sets `mem_used`. A run with no heap set whose alias region is used up to within 0.25 GB of its end sets a 1 GB pool: `nninitStartup` handed pool − used to StandardAllocator as one address region and the game reserves regions of its own beside it (Unity 2020 and 2022: AI: THE SOMNIUM FILES - nirvanA Initiative reached +4.26 GB of 4.38 in 5 s with the hardware pool; Unity 2018.3's Raging Loop maps only inside StandardAllocator's region and keeps it). The 1 GB is a starting point, not a measurement: Advance Wars 1+2: Re-Boot Camp (Unity 2020.2) ran its malloc dry at its first loading screen with it and needs 2 GB. The profile gets a `# heap check:` line, and later runs keep its keys (delete the line to check again; `--no-heap-check` / `HEAP_CHECK = False` skips it). Other games skip the check, so it costs nothing for them and a few seconds plus a Mac host build for the rest.
|
||||
|
||||
No short run decides `render_cond`, `tex_flush_upload`, `tex_dirty_upload`, `core_preempt`, `gpu_timestamps`, `feedback_snap`, `alpha_from_red`, `vconv_range` or `present_interval`, nor `op_mode` / `pad_layout` outside the presets: they keep their defaults for a developer. A minute's boot without input counted Wonder's clears under render conditions (65,536 or more; it needs no `render_cond`) as readily as Tears of the Kingdom's, saw Breath of the Wild flush 32 CPU-filled textures with data (it needs no `tex_flush_upload`) and Age of Calamity none (its movie frames come later), and Metroid Dread's movie crash shows in about half the runs. On the iPhone, `Documents/env.txt` can still set `DM_OPMODE`, `DM_NVN_GPU_TS` and `DM_NVN_FB_SNAP`, and the pad layout switches in game.
|
||||
|
||||
|
||||
@@ -28,6 +28,7 @@
|
||||
# · data.arc (Super Smash Bros. Ultimate, smash_arc.py): GLSLC outputs in its SSBH shader databases and BNSH in its effects and UI layouts, each of its current files (zstd) scanned, see smash_arc_jobs
|
||||
# · PKZ (PlatinumGames, Bayonetta 3): shader objects (VSO / PSO / CSO / HSO / DSO, a control section and a code area each) in the DAT archives of the PKZ entries (Oodle Kraken, decoded by the
|
||||
# game's own decoder in the Mac host), see pkz_entries, pgdat_members and pg_shader
|
||||
# · control sections loose in main, in no GLSLC output or BNSH file (Advance Wars 1+2: Re-Boot Camp's Bink 2 shaders): paired with their code areas by data records or adjacency, see raw_shaders
|
||||
# Code area: 0x30 header + (graphics) 0x50 SPH + instructions; header magic 0x12345678 for graphics, 0x12345679 for compute. The control section starts with 0x98761234, +0x704 = code area length,
|
||||
# +0x700 = start of the c1 constant segment within the code area; for compute +0x748 thread group xyz (u32 ×3), +0x754 shared memory bytes (checked one by one against PC reflection on D2R's 22 compute shaders).
|
||||
# Stage: BNSH uses the stage slot; BAHS uses bits 10..13 of SPH word 0 (1 vertex 2 hull 3 domain 4 geometry 5 pixel), and a compute header means compute.
|
||||
@@ -713,6 +714,75 @@ def mgfx_shaders(d, label):
|
||||
if label.endswith('.xnb') and b'EffectReader' in d[:0x200] and not out: red(f'{label}: an XNB effect without an MGFX version 8 effect of NVN shaders')
|
||||
return out
|
||||
|
||||
def rel_targets(img):
|
||||
"""A module image's relative relocations: {slot: target}, both module offsets. RELA's R_AARCH64_RELATIVE entries (type 0x403) carry the target as their addend; DT_RELR (SDK 20, tag 36,
|
||||
size tag 35) packs slots whose own 8 bytes are the target: an even entry is a slot (the next slot follows it), an odd one a bitmap of the 63 slots after the last (as loader.c relr_apply)"""
|
||||
m = struct.unpack_from('<I', img, 4)[0]; o = m + struct.unpack_from('<i', img, m + 4)[0]; t = {}
|
||||
while o + 16 <= len(img):
|
||||
tag, v = struct.unpack_from('<qQ', img, o); o += 16
|
||||
if tag == 0: break
|
||||
t.setdefault(tag, v)
|
||||
out = {}
|
||||
for i in range(0, t.get(8, 0) if 7 in t else 0, 24):
|
||||
off, info, add = struct.unpack_from('<QQq', img, t[7] + i)
|
||||
if info & 0xffffffff == 0x403: out[off] = add
|
||||
nxt = 0
|
||||
for i in range(0, t.get(35, 0) if 36 in t else 0, 8):
|
||||
e = struct.unpack_from('<Q', img, t[36] + i)[0]
|
||||
if not e & 1: out[e] = struct.unpack_from('<Q', img, e)[0]; nxt = e + 8; continue
|
||||
for b in range(63):
|
||||
if e >> (b + 1) & 1: s = nxt + 8 * b; out[s] = struct.unpack_from('<Q', img, s)[0]
|
||||
nxt += 8 * 63
|
||||
return out
|
||||
|
||||
def adrp_targets(d, lo, hi):
|
||||
"""Addresses in [lo, hi) that the code builds with adrp + add on the same register (the add within the next 8 instructions) → the offsets of those adds, all module offsets. Every word of
|
||||
the image is tried (a data word that decodes so only adds a candidate)"""
|
||||
w, out = memoryview(d)[:len(d) & ~3].cast('I'), collections.defaultdict(list)
|
||||
for i, x in enumerate(w):
|
||||
if x & 0x9f000000 != 0x90000000: continue
|
||||
rd = x & 31; page = (i * 4 & ~0xfff) + ((((x >> 5 & 0x7ffff) << 2 | x >> 29 & 3) << 44 >> 44) << 12)
|
||||
if not lo - 0x1000 <= page < hi: continue
|
||||
for j, y in enumerate(w[i + 1:i + 9]):
|
||||
if y & 0xffc00000 == 0x91000000 and y >> 5 & 31 == rd:
|
||||
t = page + (y >> 10 & 0xfff)
|
||||
if lo <= t < hi: out[t].append((i + 1 + j) * 4)
|
||||
break
|
||||
return out
|
||||
|
||||
def raw_shaders(d, label, known=()):
|
||||
"""Control sections in main that are in no GLSLC output and no BNSH file: a shader compiled offline whose control section and code area the code hands to nvnShaderData itself (Advance Wars 1+2:
|
||||
Re-Boot Camp, Unity 2020.2 with Bink 2: the movie player's 32: 2 vertex, 22 pixel and 8 compute shaders; without them its movies drew white). Most are 0x20-byte records in data {code area
|
||||
pointer, control section pointer, u32 code length, u32 control length} (two tables, main+0x49eae50 and +0x49fb5c0), found as two relocated slots, the second pointing at the control section and
|
||||
the first at a code area of the length its control +0x704 gives. The rest the code uses by address (main+0x1546b4c, +0x1547efc): adrp + add to the control section and to a code area it copies
|
||||
from, which lies right after the control section's own length (control +0x20, at 16-byte alignment) or right before it. Only a code area the code builds the address of is taken: both neighbours
|
||||
can look right, so the one whose address the code builds nearest to the control section's (within 1 KB) is taken: main+0x4a2c670 is a compute shader whose code is the 0x100 bytes before it (adds 0x20 apart,
|
||||
main+0x1548014 / +0x1548034); after it lies another vertex shader's code area, whose first 0x100 bytes pass the header and length checks, and that area is the vertex shader main+0x4a2d290 takes from
|
||||
before itself (main+0x1548358 / +0x1548374, 0x324 bytes from the first). Compute code areas have no header (SDK 10), so the stage is glslc_stage's, and a headerless one is checked by its length only, never
|
||||
across a control section. known: the control sections the other readers took from this main (Celeste's MonoGame effect in main holds some outside any GLSLC output or BNSH file), left out.
|
||||
Of 25 mains on hand only that one has such control sections, so the tables of every other game stay as they were"""
|
||||
skip = []
|
||||
for m in re.finditer(b'\x91\x68\x86\x19', d):
|
||||
size = struct.unpack_from('<I', d, m.start() + 0x44)[0]
|
||||
if 0x760 < size <= len(d) - m.start(): skip.append((m.start(), m.start() + size))
|
||||
for m in re.finditer(b'BNSH\0\0\0\0', d): skip.append((m.start(), m.start() + struct.unpack_from('<I', d, m.start() + 0x1c)[0]))
|
||||
ctls = [c for m in re.finditer(re.escape(CTL_MAGIC), d) if not (c := m.start()) & 3 and not any(a <= c < b for a, b in skip) and d[c:c + struct.unpack_from('<I', d, c + 0x20)[0]] not in known]
|
||||
if not ctls: return []
|
||||
rel, code_of, cset = rel_targets(d), {}, set(ctls)
|
||||
for s, c in rel.items():
|
||||
if c in cset and s - 8 in rel and c not in code_of and struct.unpack_from('<I', d, s + 8)[0] == struct.unpack_from('<I', d, c + 0x704)[0]: code_of[c] = rel[s - 8]
|
||||
built = adrp_targets(d, min(ctls) - 0x100000, max(ctls) + 0x100000) if any(c not in code_of for c in ctls) else {}
|
||||
out = []
|
||||
for c in ctls:
|
||||
nctl, n = struct.unpack_from('<I', d, c + 0x20)[0], struct.unpack_from('<I', d, c + 0x704)[0]
|
||||
ctl, why = d[c:c + nctl], 'no code area: no data record points at one, and the code builds the address of none right after or right before it'
|
||||
near = lambda k: min(abs(a - b) for a in built[k] for b in built[c]) # how close the code builds this code area's address to the control section's
|
||||
for k in ([code_of[c]] if c in code_of else sorted((k for k in ((c + nctl + 15) & ~15, c - n) if k in built and c in built and near(k) < 0x400), key=near)):
|
||||
code = d[k:k + n]; st = glslc_stage(ctl, code) if k >= 0 and CTL_MAGIC not in code else None
|
||||
if st and not check(label, st, ctl, code): out.append((f'{label}#RAW@{c:#x}', st, ctl, code)); why = None; break
|
||||
if why: print(f'[RED] {label} control section @{c:#x}: {why}; left out', flush=True)
|
||||
return out
|
||||
|
||||
ONEPACK = b'ONEPACK\0'
|
||||
def fb_field(d, t, i):
|
||||
"""FlatBuffers: absolute offset of field i of the table at t, None when the field is absent"""
|
||||
@@ -1045,9 +1115,10 @@ def ps_inputs(code):
|
||||
"""Generic attributes a pixel shader reads: SPH words 6–13 are the input map, 8 bits for each of the 32 attributes (Ryujinx ShaderHeader.ImapTypes), non-zero = read; the translated MSL inputs are exactly user(locn<attribute>)"""
|
||||
return {i for i in range(32) if struct.unpack_from('<I', code, 0x30 + 4 * (6 + (i >> 2)))[0] >> ((i & 3) * 8) & 0xff}
|
||||
|
||||
SINGLE_STAGE = ('#GLSLC@', '#UE', '#MTSPK@', '#MGFX@', '#NLG', '#PGSO') # sources whose every shader is a program of one stage: GLSLC outputs, Unreal's shader records, MT Framework's shader package (BNSH variations, Unity programs, MSHD and SDF pair theirs);
|
||||
SINGLE_STAGE = ('#GLSLC@', '#UE', '#MTSPK@', '#MGFX@', '#NLG', '#PGSO', '#RAW@') # sources whose every shader is a program of one stage: GLSLC outputs, Unreal's shader records, MT Framework's shader package (BNSH variations, Unity programs, MSHD and SDF pair theirs);
|
||||
# MonoGame's effects, whose passes may name no vertex shader and keep the one bound (Celeste: 10 of its 13 effects draw through SpriteBatch's vertex shader, which is in main); Next Level Games'
|
||||
# archives (Luigi's Mansion 3 makes 5,317 programs of one stage each at boot); and PlatinumGames' shader objects (Bayonetta 3: one stage each, pg_shader)
|
||||
# archives (Luigi's Mansion 3 makes 5,317 programs of one stage each at boot); PlatinumGames' shader objects (Bayonetta 3: one stage each, pg_shader); and control sections loose in main
|
||||
# (Advance Wars 1+2: Re-Boot Camp's Bink shaders, raw_shaders), which no file pairs
|
||||
def separable_pairs(keys):
|
||||
"""Single-stage programs (13 Sentinels: each GLSLC output becomes a program with one stage, and BindProgram's stage bits combine vertex and pixel programs per draw; Unreal combines its shaders per draw
|
||||
from the pipeline state) have their real pairs only at draw time. vs_pad uses only the union of the locations the paired pixel shaders read, so every single-stage vertex shader is paired with
|
||||
@@ -1190,6 +1261,7 @@ def main():
|
||||
except ValueError: common = None # not a Unity game
|
||||
found = []; scan_blob(img, 'main' + ('@' + tag if tag else ''), found)
|
||||
found += glslc_shaders(img, 'main' + ('@' + tag if tag else ''), need=False)
|
||||
found += raw_shaders(img, 'main' + ('@' + tag if tag else ''), {c for _, _, c, _ in found})
|
||||
# GLSLC outputs the game compiles at run time, captured on the Mac (hle_glslc.c): <version dir>/glslc_capture, or DM_GLSLC_CAPTURE. A game needs them when main holds GLSL sources and
|
||||
# calls glslcCompile in a module it loads (Fire Emblem: Three Houses; profile_guess.py shader_sources). Captain Toad: Treasure Tracker imports it with agl's GLSL sources, but no module defines it
|
||||
cap = os.environ.get('DM_GLSLC_CAPTURE') or os.path.join(os.path.dirname(os.path.abspath(romfs)), 'glslc_capture')
|
||||
|
||||
@@ -117,6 +117,16 @@ CUTS = {
|
||||
(0x336bf0, 'e9230b32', '097ca052', 'AddressRegionManager constructor: length of the node area registered as an occupied region orr w9,wzr,#0x3fe00000 → mov w9,#0x3e00000'),
|
||||
(0x336d5c, 'ea030232', '0a40a052', 'AllocateAddressRegion: alignment of regions of 1 GB and more orr w10,wzr,#0x40000000 → mov w10,#0x2000000 (32 MB, as from 32 MB up)'),
|
||||
],
|
||||
# SDK 10.4.1 sdk d9add76fc264 (Advance Wars 1+2: Re-Boot Camp 1.0.0, Unity 2020.2.1f3; The Great Ace Attorney Chronicles 1.0.0, which has no virtual address memory and never makes an
|
||||
# address region): the node area cut, and regions of 1 GB and more aligned to 32 MB, as DRAGON QUEST XI S's SDK 7.3.2 build, whose AddressRegionManager this is at other offsets.
|
||||
# nninitStartup (main+0x365310) calls StandardAllocator::Initialize(nullptr, pool − used) with virtual address memory, which aborted at its CentralHeap::Init (sdk+0x2ee060), and that region
|
||||
# is more than 1 GB. The constructor is at sdk+0x340e90 (the limit at +0x340ea0 from orr w8,wzr,#0x3fe00000, the occupied length at +0x340f3c from orr w9,wzr,#0x3fe00000);
|
||||
# VammManager::AllocateAddressRegion (sdk+0x341030) aligns a region by its size: 64 KB under 2 MB, 2 MB under 4 MB, 4 MB under 32 MB, 32 MB under 1 GB, else 1 GB (+0x34109c)
|
||||
'd9add76fc264863d85fbbd1d70076c76': [
|
||||
(0x340ea0, 'e8230b32', '087ca052', 'AddressRegionManager constructor: node area limit orr w8,wzr,#0x3fe00000 → mov w8,#0x3e00000'),
|
||||
(0x340f3c, 'e9230b32', '097ca052', 'AddressRegionManager constructor: length of the node area registered as an occupied region orr w9,wzr,#0x3fe00000 → mov w9,#0x3e00000'),
|
||||
(0x34109c, 'ea030232', '0a40a052', 'AllocateAddressRegion: alignment of regions of 1 GB and more orr w10,wzr,#0x40000000 → mov w10,#0x2000000 (32 MB, as from 32 MB up)'),
|
||||
],
|
||||
# Octopath Traveler 0 1.0.5 (v327680) sdk (SDK 19.3.13): the AddressRegionManager node area 0x3fe00000 → 0x3e00000 (62 MB), as Engage's SDK 12 (above). The same constructor (sdk+0x1423ec): +0x1423fc
|
||||
# sets the node area limit, +0x142468 registers it as an occupied region. Unreal's region (-SwitchVirtualAddressSpaceSize, the profile's game_args) then starts 62 MB into the alias region, and the guest
|
||||
# block stays small enough for an iPhone next to the Unreal heap (hle_uemalloc.c), which lives outside it
|
||||
|
||||
@@ -70,6 +70,8 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **`StandardAllocator::Initialize` aborts in `nninitStartup` on SDK 17, then the alias region fills in minutes** (Ace Attorney Investigations Collection, Unity 2022.3, SDK 17.5.3) → the same node area as Engage's SDK: `CentralHeap::Init(nullptr, pool − used)` got no 1 GB-aligned region for 3.14 GB after the 1 GB node area. The constructor is at `sdk+0x139458` on that build and takes the same cut. With Engage's 2.5 GB pool the game then ran, but probes on `VammManager::AllocateAddressRegion`'s success path (`sdk+0x1395a8`: x21 size, x22 address) and `FreeAddressRegion` (`sdk+0x139df0`) showed StandardAllocator's 2.44 GB reservation beside Unity's pools (1.06 GB live, from `main+0x40f1d4`) ending 4.37 GB into the 4.375 GB alias region after 140 s, while the `MapPhysicalMemory` log showed StandardAllocator committing 0.19 GB. The profile reports a 1 GB pool, a little less than the 1.02 GB the game maps at its peak (the host does not enforce the pool, and only `nninitStartup` reads it); ten minutes then peak at +2.85 GB of address space, against +3.91 GB with a 1.5 GB pool.
|
||||
- **The same abort on SDK 16.2.4** (Brain Memory Training Academy, Unity 2022.3.5f1: `nn::diag` abort at `sdk+0x126490`, `StandardAllocator::Initialize`, called from `nninitStartup` at `main+0x73adcc`; the NSP script's heap check reports "the host stopped before nnMain") → `nninitStartup` is Ace Attorney's, `Initialize(nullptr, pool − used)` with virtual address memory and no `SetMemoryHeapSize`. The `sdk` build is Unicorn Overlord's, whose NPDM declares no system resource, so it never needed the cut; the constructor (`AddressRegionManagerC2`, `sdk+0x134768`) takes it at `+0x134778` / `+0x13480c`. With Ace Attorney's 1 GB pool and 5 GB block the game boots to its title in 5 s with no red line; StandardAllocator ends at +1.0 GB, and five minutes of level loads take the alias high-water mark to +2.25 GB with at most 0.5 GB mapped.
|
||||
- **The same abort on SDK 13.3.7** (Patrick's Parabox, Unity 2020.3.30f1: `StandardAllocator::Initialize` at `sdk+0x324dec`, called from `nninitStartup` at `main+0x1cd080`) → the same `nninitStartup` and the same cut, on this build's constructor (`sdk+0x37c3a0`: `+0x37c3b0` / `+0x37c44c`). With Ace Attorney's 1 GB pool and 5 GB block (in place of the provisional profile's hardware pool) it boots to its title with no red line, and the alias region is mapped up to +1.5 GB with 0.27 GB in use. To find the constructor on a new `sdk` build, search its text for the two `mov w8 / w9, #0x3fe00000` words (`08fca752`, `09fca752`); `sdis.py --fn` names the function `AddressRegionManagerC2Emm` from its dynamic symbols.
|
||||
- **The same abort on SDK 10.4.1** (Advance Wars 1+2: Re-Boot Camp, Unity 2020.2.1f3: `StandardAllocator::Initialize` at `sdk+0x2edf90`, called from `nninitStartup` at `main+0x365310`; the heap check reports "the host stopped before nnMain") → the same `nninitStartup`. The `sdk` build is The Great Ace Attorney Chronicles', which has no virtual address memory; its constructor (`sdk+0x340e90`) loads the node area with `orr w8 / w9, wzr, #0x3fe00000` (`e8230b32` at `+0x340ea0`, `e9230b32` at `+0x340f3c`), the SDK 7 / 8 form, so the `mov` search above finds nothing there. `AllocateAddressRegion` (`sdk+0x341030`) aligns a region of 1 GB or more to 1 GB (`+0x34109c`), as DRAGON QUEST XI S's SDK 7.3.2 build, and takes the same 32 MB cut.
|
||||
- **Then the frames stop at the first loading screen, a graphics worker at 100% CPU** (17 s in, `UnityGfxDeviceWorker` running, the main thread waiting on a condition variable, no red line) → the heap check's 1 GB pool (the hardware pool had used the alias region up to +4.25 GB) left the game's malloc too little: Unity's GPU memory pool block allocation (`main+0x96f430`) calls `aligned_alloc` through `main+0x866810`, halves the request after each failure, and once the request rounds to 0 it calls on for ever with a size of 0. `sample <pid>` on the Mac host shows such a spin (the guest frames are `???` in the host's image list; subtract the module base from the log's `base` lines); it worked here where lldb does not. With a 2 GB pool the game maps up to 2.26 GB of the alias region through its menus and first battle.
|
||||
- **Unity 6 takes its memory with no StandardAllocator and no heap** (Hollow Knight: Silksong, Unity 6000.0.50f1 on SDK 20.5.8) → `nninitStartup` (`main+0x118cb70`) only calls `QueryMemoryInfo` and `SetMemoryAllocatorForThreadLocal`; Unity's own allocator reserves address regions with `nn::os::AllocateAddressRegion` and commits pages with `AllocateMemoryPages`, so nothing aborts at startup as with 2022.3. `main` imports no `SetMemoryHeapSize`, and the provisional profile took that for a game living in its bss (a 1 GB block, 256 MB alias region) while Unity maps 1.5 GB in the first room; `profile_guess.py` now gives every game with an NPDM system resource the 5 GB block (Absolum and Pokémon Legends: Z-A, heapless too, came out right with it). The node area still sits at the start of the alias region: without the cut the high-water mark was +1.00 GB at `nninitStartup` and +3.00 GB in the first room, with it (`sdk+0x145bfc` / `+0x145c68`) +2.26 GB.
|
||||
- **`nn::diag` abort from `StandardAllocator::Initialize` long after `nninitStartup`, right after the first Vulkan query pool** (Super Mario Galaxy 2, SDK 18.3.5: `sdk+0x12dad0` ← `main+0x62c52c`) → the movie player initializes its own allocator with virtual address memory, `Initialize(nullptr, 384 MB)` (`main+0x62c280`); Galaxy, on the same SDK, allocates that block from its main allocator first. The guessed 256 MB alias region holds no region after the 1 GB node area. The same node area cut for that `sdk` build (`sdk+0x13be4c` / `+0x13bee0`) and a 512 MB alias region: the region is 32 MB aligned, so it starts 64 MB in.
|
||||
- **`nn::diag` abort at `sdk+0x12dad0` in `nninitStartup`, called from `main+0x756310`** (Absolum, .NET NativeAOT on SDK 18.3.2) → malloc's `StandardAllocator::Initialize(nullptr, 0xa00000000)`: 40 GB of address space (`main+0x28bff4`). To see what such a game really needs, boot it once on the Mac with a 64 GB alias region (`mem_reserve 0x1030000000`, `mem_alias 0x1008000000`) and probe `VammManager::AllocateAddressRegion`'s success path (`sdk+0x13bf8c`: x21 size, x22 address) with `DM_SVC=1`'s map log. Absolum's regions: malloc's (2.19 GB used into the first level, 1.75 GB of it FNA3D's GPU heap, allocated whole at `FNA3D_CreateDevice`), the .NET GC's region range (3.75 GB) and a few small ones. The GC sizes its range as 5 × its hard limit (768 MB = 75% of the 1 GB its `GetPhysicalMemoryLimit`, `main+0x28bffc`, reports), and it reads `DOTNET_GCRegionRange` through `getenv` first. The host now answers `getenv` from the profile key `game_env` (`hle_os.c`; every other name goes to the SDK's own `getenv`, an empty environment), which sets 1.5 GB. The GC then uses its range from both ends, 756 MB from the start and 748 MB from the end, where it used only the first 1.14 GB of 3.75. A cut on `main` gives malloc 2.5 GB. One more thing kept it from fitting: SDK 18's `AllocateAddressRegion` aligns a region of 1 GB or more to 1 GB in absolute address (`sdk+0x13c03c`; 32 MB from 32 MB up). The two large regions, with a 4 KB one between them, lost up to 1 GB each to alignment, depending on where `mmap` put the guest block. A cut on that build aligns them to 32 MB as well, beside the node-area cut. The regions then end 4.13 GB into the 4.375 GB alias region of Engage's 5 GB block.
|
||||
|
||||
@@ -26,6 +26,7 @@ Do not stop at the obvious archive. In Mario the main `bfsha` held 4,888 of the
|
||||
- **BNSH in Trinity packs** (Pokémon Legends: Z-A, Game Freak's engine): every shader is a BNSH inside the Oodle-compressed files of `arc/data.trpfs` ([01](01-extract.md)). For any game with Trinity packs (`ONEPACK` files), `trpfs_jobs` / `trinity_shaders` in `shader_split.py` name the packs from `data.trpfd`, decode every file with the game's own Oodle in the Mac host (`DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script) and scan it like a loose file. 28,603 shaders (11,633 vertex, 16,845 pixel, 125 compute) and 26,051 variation pairs: the material shaders in `system_resource/gfx2/shader/NX64/env_ikkaku` (`material/standard.bnsh`, `material/transparent.bnsh`, `system/light.bnsh`, and the `special/imgui.trsha` pack with the engine's system shaders), particle effects (`.ptcl` / `.trtml` in `ik_effectdata`, 19,842 of the shaders), UI layouts (`__ArchiveShader.bnsh` in the `.truiv` packs of `uidata`) and the 40 BNSH in `main`. The layout archives' control sections are 104 KB long, and their stage descriptors run 456 bytes past the control's own length (control +0x20), with bytes that differ between copies of one shader; `same_shader` compares copies only up to the control's own length.
|
||||
- **BNSH in Bezel Engine archives** (Big Brain Academy: Brain vs. Brain, NintendoWare Bezel Engine 1.56): the shaders are BNSH in the `.bfsha` entries of the `.bea` archives (zstd entries, [01](01-extract.md)), most of them the engine's own in `System.nx.bea` (`_BezelSystemResources/<system>/shader/*.bfsha`: PBR shadow maps 846, post effects 770, …), the rest in the layouts' `__ArchiveShader.bnsh` and the per-archive shader packs (`.bnbshpk`), plus one BNSH in `main`. `shader_split.py` scans each archive entry as a file of its own (`bea_members`): 2,125 shaders in about 90 s.
|
||||
- **SPK in an MT Framework ARC** (The Great Ace Attorney Chronicles, Capcom's MT Framework): the engine's one shader package, `sc\NX_64\root` (`SPK\0`, type hash `0x02358e1a`) in `nativeNXx64/sa/NX_64/root.arc` ([01](01-extract.md)). After the package's tables every shader is a code area followed directly by its control section, back to back to the end of the file: 230 vertex shaders, then 230 pixel shaders. Each control section names its code length (+0x704), so the code is the bytes right before it, and every block must start where the previous one ended (`mtspk_shaders`). The tables that say which vertex and pixel shaders the engine combines are not read: the vertex shaders are paired with the single-stage cover (one pixel shader reads every input any of them reads). `CustomShaderPackage.mfx` (`MFX\0`, the effect descriptions) holds no shader code, and neither `main` nor any other ARC entry does. All 460 build, with Mario's binding layout; every binding decodes.
|
||||
- **Control sections loose in `main`** (Advance Wars 1+2: Re-Boot Camp, Unity 2020.2 with Bink 2's Unity plugin): the movie player builds its programs from 32 control sections and code areas in `main`'s read-only segment that are in no GLSLC output and no BNSH file. 26 are 0x20-byte records in two tables in `.data` (`{code area pointer, control section pointer, u32 code length, u32 control length}`, both pointers relative relocations); the code takes the other 6 by address (`adrp` + `add` to the control section and, a few instructions away, to the code area it copies with `memcpy`), the code area right after the control section's own length (+0x20, 16-byte aligned) or right before it, and two control sections share one code area. Both neighbours can pass the header and length checks, so the reader (`raw_shaders`) takes only a code area the code builds the address of, the nearest one to the control section's. The 8 compute shaders' code areas have no header (SDK 10). Without them the shaders were not in the map (two `[RED] shader … is not in the shader map` lines and `program has no vertex shader` as the opening movie starts) and every movie drew white. Of 25 `main`s on hand no other has such control sections.
|
||||
- **MonoGame effects** (Celeste: MonoGame for the Switch, its C# compiled to C++ by Sickhead's BRUTE): the 13 XNB files in `Content/Effects` (XNB platform `S`, uncompressed, read by `EffectReader`) each hold one MGFX effect, and `main` holds SpriteBatch's built-in one; no other file and no GLSLC output. MGFX version 8 is MonoGame's own layout: `MGFX`, u8 version, u8 profile, u32 effect key, the constant buffers, then each shader as u8 vertex (1) / pixel (0), u32 length, the bytecode (u32 control length, the control section, u32 code length, the code area: GLSLC's output without its header), the samplers, the constant buffer indices and an attribute list (as MonoGame's OpenGL backend writes it), then the parameters and the techniques with their passes (`mgfx_effect`). The reader parses every effect to its end, which checks the layout; in `main`, four bytes `MGFX` count only when version 8 and a control section follow. A pass names its vertex and pixel shader by index, 255 for none, and 10 of the 13 effects have no vertex shader: SpriteBatch binds its own and the effect's pass only replaces the pixel shader. Vertex and pixel shaders therefore combine across effects, and they are paired as single-stage programs. 26 shaders (20 pixel, 6 vertex), all with Mario's binding layout; every binding decodes.
|
||||
- **GLSLC outputs in FromSoftware's dvdbnd archives** (Dark Souls Remastered): RomFS is two archives, `dvdbnd0.bdt` and `dvdbnd1.bdt` (3.6 GB), each indexed by the `.bhd5` beside it: `BHD5`, 0xff (little-endian), u32 1, u32 file size, u32 bucket count, u32 0, u64 bucket table offset; each 16-byte bucket `{u32 count, u32, u64 offset}` points at 16-byte entries `{u32 path hash, u32 size, u64 offset}` (paths only hashed). 5,184 entries are DCX containers (`DCX\0`, big-endian header, `DFLT` = a zlib stream at +0x4c), most of them BND3 archives (28-byte entries `{u8 flags, i32 size, u64 offset, i32 ID, u32 name offset, i32 unpacked size}` in formats 0x5c / 0x7c). Six BND3 archives in `dvdbnd0.bdt` hold the shaders, one GLSLC output per entry (`.vpo` vertex, `.fpo` pixel, named `Shader/FRPG_Sfx/NVNShaderBinary/ps/…` or by base name in the two FLVER material archives, 2,602 + 486 entries): `bhd5_entries`, `undcx`, `bnd3_members`, `bdt_shaders` in `shader_split.py`, in 64 jobs by entry range, about 10 s. Only archives that contain a GLSLC magic are parsed (the others use format 0x78). SDK 4's GLSLC needs its section headers read (Dark Souls Remastered in section 6).
|
||||
- **BAHS** (agl `sharcb`, engine utility shaders): code region at header +0x1c (size +0x18). Binary entries in the table area are 16 bytes `{code offset, 0, control size (0x880/0x88c; 0x848 in BAHS version 9, Super Mario 3D World), signed offset from this entry to the control section}`. Control sections are shared between programs, so pair by entry, never by searching for code near a control.
|
||||
|
||||
@@ -688,6 +688,12 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- **Host changes, all general:** the newest GPU code version the NVN layer reports now follows the SDK (1.16 from SDK 13): the engine refused its built-in shaders at 1.14 and `nnMain` returned at once ([03](agents/03-bringup.md)); `nvnWindowGetCrop` and `nvnSamplerBuilderGetLodClamp`.
|
||||
- **Input:** the 3D pad layout (the camera is on the right stick).
|
||||
- **Mac input:** the work folder's `run/ctl.sh` drives a run started with `DM_CTL` (`CTL=<file>` picks another run's control file), `run/step.sh <shot> <presses…>` presses and takes a screenshot, `run/snapseq.sh` boots and takes one every few seconds, and `run/route.sh <run>` boots on a fresh save and drives the route above: A at the title, A through the logo, A (Standard), A, B (closes the Naive Angel Mode help), A (Off), A (brightness), A (the auto-save notice), then Plus, Down, A skips the opening movie.
|
||||
**Advance Wars 1+2: Re-Boot Camp (`0100300012F2A000`)**
|
||||
- **Status:** plays on the iPhone (IPA 7.1 GB) from the base game 1.0.0, the only version. On the Mac: the boot, the Bink opening movie, the title, Nell's introduction, a new casual campaign and the first training battle (Field Training, Troop Orders); 60 fps in the menus, 30 fps in battle by the game's own limit (present interval 1, the frame's work 33 ms with the CPU and GPU mostly idle), sound, about 2.5 GB resident and 1.6–1.9 GB of Metal allocations, level over 15 minutes with no red line.
|
||||
- **Engine:** Unity 2020.2.1f3 IL2CPP on SDK 10.4.1 (The Great Ace Attorney Chronicles' `sdk` and movie module builds), NVN; FMOD sound through `AudioOut`; Bink 2 movies (`.bk2`) decoded by Bink's Unity plugin in `main`. The NPDM declares virtual address memory: `nninitStartup` hands pool − used to StandardAllocator and sets no heap, so the `sdk` build takes the node-area cut and the 1 GB alignment cut (`sdk_patches.py`), and the profile reports a 2 GB pool in Engage's 5 GB block: with the heap check's 1 GB the game's malloc ran dry at the first loading screen ([03](agents/03-bringup.md)).
|
||||
- **Shaders:** 20,424: the Addressables bundles (`Data/StreamingAssets/aa/Switch`), Unity's 11 GLSLC outputs in `main`, and Bink's 32 (2 vertex, 22 pixel, 8 compute), whose control sections and code areas lie loose in `main` (`shader_split.py` `raw_shaders`, [04](agents/04-graphics.md)); without them every movie drew white. All build.
|
||||
- **Input:** the 3D pad layout (the map cursor moves with the D-pad).
|
||||
- **Mac input:** the work folder's `run/step.sh <shot> <wait> <presses…>` drives a run started with `DM_CTL`. About 22 s into a boot the opening movie plays: A skips it, `L+R~0.5` opens the main menu (the first time Nell's introduction, A about 25 times, then the file select); A (Campaign), A, A (Advance Wars, the file), A, A, A, A (the mission, Field Training, Troop Orders) start the first battle.
|
||||
|
||||
## Tools (`dm/tools`)
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 187 KiB |
Reference in new issue
Block a user