Balatro (0100CD801CE5E000, update 1.1.3, LÖVE 11.4 "love-nx" with LuaJIT and SDL on SDK 18.3.2): game profile and the GL layer LÖVE needs, plus four general changes. Plays on the iPhone by touch; on the Mac from the intro and the title into a run and its tutorial, at 60 fps with sound, about 0.65 GB resident and flat over three minutes. The game is Lua source in the RomFS and renders with NVIDIA's desktop OpenGL through EGL, as Digimon does. GL layer (dm/gfx/gl/hle_gl.m): LÖVE reads GL_CONTEXT_PROFILE_MASK and takes its core-profile paths only when it says core (with a compatibility profile it streams vertices from client memory, which Metal cannot draw from), so the layer answers core and implements what those paths use, about 150 functions in all against Digimon's 79: persistently mapped buffers (glBufferStorage, glMapBufferRange: the shared MTLBuffer's own memory), glBufferSubData in command order, fences (a fence is the serial of the command buffer being recorded; a wait on one still recording commits it), glBindAttribLocation (the game's attribute index of each Metal input, set at the link) and current attribute values (glVertexAttrib4f: LÖVE's VertexColor and ConstantColor are disabled arrays), glGetActiveUniform and the uniform read-backs, every glUniform* form, glTexStorage2D / 3D with their level counts, 3D and 2D array textures (LÖVE makes a default of each type on 96 units), GL_TEXTURE_SWIZZLE_* (its font glyphs are RG8 read as R, R, R, G), glGenerateMipmap as a blit (the game's images are mipmapped), instanced and base-vertex draws, separate read and draw framebuffers, glBlitFramebuffer (colour, same size), glReadPixels, glClearBufferfv, glFrontFace and the EGL calls SDL makes; stack arguments of the functions with more than eight are 64-bit, as AAPCS64 lays them out. DM_GL_STUBS=1 gives every GL name the layer lacks a stub of its own that reports its first call, instead of a null pointer. Shaders: LÖVE puts its 35 GLSL sources together at run time in Lua, so the NSP script captures them on the Mac as it does Super Mario Galaxy's SPIR-V (capture_shaders now serves both, profile gl_capture.tsv; profile_guess.py's gl_capture rule: an OpenGL game with no GLSL source in its RomFS or main), and gl_shaders.py keeps a bool uniform's declared type (a uint in SPIR-V; LÖVE's Shader:send takes only a boolean for a bool) and takes from main only strings that start with '#version <number>' (Balatro's main has glslang's messages). General: profile_guess.py leaves vkGetInstanceProcAddr, SDL's Vulkan loader, out of its Vulkan count (Balatro's provisional profile said Vulkan; Hades and both Galaxies import 99 or 100 Vulkan functions); OpenAudioRenderer without an event now starts the host's render thread too (hle_audren.c ar_open: only the form with an event did, and such a game was silent); loader_census.py follows registers in a straight line (adrp, add, loads through relocated GOT entries, mov, csel), for LÖVE's glad, whose pointers are globals its code keeps the addresses of in registers; and the iPhone gamepad has a third layout, Tap (profile pad_layout touch, which needs op_mode handheld; DM_GAME_PAD_LAYOUT replaces DM_GAME_PAD_FULL): no control but the layout button, every touch to the game, offered in the layout cycle only in handheld mode. Profile: op_mode handheld (the Switch build plays by touch there), pad_layout touch, gl_capture.tsv. A Lua error makes Balatro quit (its Switch error handler), and love-nx's video worker is then deleted twice (main+0x339c00): the bring-up notes say how the errors were found. Checked: the NSP script packs it from the NSPs alone (the capture in under a second, 35 shaders for the Mac and iOS, an 82 MB IPA), tool tests pass, profile_guess.py --check changes no other profile; Digimon was not rerun (its files are not on hand). README (games table, a Mac picture of the title), AGENTS.md (47 → 48), docs/development.md, docs/agents and the READMEs of dm/games and dm/gfx say so
This commit is contained in:
1 parent
4e91a3d053
commit
c392d56af0
26 files changed
+900
-210
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Notes for agents
|
||||
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Forty-seven games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`), Monster Train 2 (update 1.1.1, SDK 18.3.5, `010051701FB46000`), Raging Loop (base game 1.0.0, SDK 7.3.2, `010020400E1C2000`), DARK SOULS: REMASTERED (update 1.0.3, SDK 4.5.0, `01004AB00A260000`), Paper Mario: The Thousand-Year Door (update 1.0.1, SDK 15.3.2, `0100ECD018EBE000`), Hollow Knight: Silksong (update 1.0.30000, SDK 20.5.8, `010013C00E930000`), Catherine: Full Body (update 1.0.1a, SDK 8.2.1, `0100BF00112C0000`), Digimon Story Cyber Sleuth: Complete Edition (update 1.0.3, SDK 6.4.0, `010014E00DB56000`), Animal Crossing: New Horizons (update 3.0.3, SDK 20.5.4, `01006F8002326000`), DRAGON QUEST VII Reimagined (update 1.2.0, SDK 19.3.1, `0100A9D01C446000`), FINAL FANTASY RESONANCE (update 1.0.1, SDK 20.5.17, `0100606026F12000`), Luigi's Mansion 3 (update 1.4.0, SDK 7.3.2, `0100DCA0064A6000`), AI: THE SOMNIUM FILES (update 1.0.1, SDK 6.4.2, `010089B00D09C000`), ANIMAL WELL (update 1.0.10, SDK 20.5.7, `010020D01AD24000`), AI: THE SOMNIUM FILES - nirvanA Initiative (update 1.0.2, SDK 13.3.5, `0100713016CCC000`) and Mario Kart 8 Deluxe (update 4.0.0, SDK 4.4.0, `0100152000022000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`FRAME_STEP` in their profiles' `patches.py`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout, and its `sdk` build (Breath of the Wild's) has the SDK's address-region node area cut (`sdk_patches.py`): without it the resource heaps ran out of alias region at the first descent from the Great Sky Island and the game played its memory-shortage Blood Moon, which put Link back at the start (its log shows `MSF`). Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK archives (`shader_split.py` `kpka_jobs`, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (`hle_nvn.c` `vpool_*`, a 4 KB page table the replay reads textures through); it runs at 60 fps (30 on the Switch). Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into `main`), renders Line truncated
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Forty-eight games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile `010015100B514000`), Fire Emblem Engage (update 2.0.0, `0100A6301214E000`), Metroid Dread (update 2.1.0, `010093801237C000`), 13 Sentinels: Aegis Rim (update 1.10, `01003FC01670C000`), Super Mario 3D World + Bowser's Fury (update 1.2.2, `010028600EBDA000`), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, `01007EF00011E000`), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, `01006BB00C6F0000`) Super Mario Odyssey (update 1.4.1, SDK 7, `0100000000010000`), Hades (update 1.0.38233, SDK 9, `0100535012974000`), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, `01002B00111A2000`), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, `010055D009F78000`), Super Mario Galaxy (update 1.3.1, SDK 18, `010099C022B96000`) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, `0100F2C0115B6000`) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, `0100B04011742000`), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, `010005501E68C000`), Hades II (base game, SDK 19.3.9, `0100A00019DE0000`), Unicorn Overlord (update 1.05, SDK 16.2.4, `010069401ADB8000`), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, `0100D9F01D474000`), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, `01005270232F2000`), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, `0100F43008C44000`), Cuphead (update 1.3.7, SDK 5.5.0, `0100A5C00D162000`), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, `0100FD8022DAA000`), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, `010038B015560000`), Absolum (update 1.1.1, SDK 18.3.2, `0100A83022A22000`), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, `01009BF0072D4000`) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, `0100F5D01DA3A000`), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, `010036E00FB20000`), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, `01008BC024FDE000`), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, `0100620012D6E000`), Patrick's Parabox (update 1.0.1, SDK 13.3.7, `0100E44016C22000`), Celeste (update 1.4.1.1, SDK 15.3.2, `01002B30028F6000`), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, `01006C300E9F0000`), Monster Train 2 (update 1.1.1, SDK 18.3.5, `010051701FB46000`), Raging Loop (base game 1.0.0, SDK 7.3.2, `010020400E1C2000`), DARK SOULS: REMASTERED (update 1.0.3, SDK 4.5.0, `01004AB00A260000`), Paper Mario: The Thousand-Year Door (update 1.0.1, SDK 15.3.2, `0100ECD018EBE000`), Hollow Knight: Silksong (update 1.0.30000, SDK 20.5.8, `010013C00E930000`), Catherine: Full Body (update 1.0.1a, SDK 8.2.1, `0100BF00112C0000`), Digimon Story Cyber Sleuth: Complete Edition (update 1.0.3, SDK 6.4.0, `010014E00DB56000`), Animal Crossing: New Horizons (update 3.0.3, SDK 20.5.4, `01006F8002326000`), DRAGON QUEST VII Reimagined (update 1.2.0, SDK 19.3.1, `0100A9D01C446000`), FINAL FANTASY RESONANCE (update 1.0.1, SDK 20.5.17, `0100606026F12000`), Luigi's Mansion 3 (update 1.4.0, SDK 7.3.2, `0100DCA0064A6000`), AI: THE SOMNIUM FILES (update 1.0.1, SDK 6.4.2, `010089B00D09C000`), ANIMAL WELL (update 1.0.10, SDK 20.5.7, `010020D01AD24000`), AI: THE SOMNIUM FILES - nirvanA Initiative (update 1.0.2, SDK 13.3.5, `0100713016CCC000`), Mario Kart 8 Deluxe (update 4.0.0, SDK 4.4.0, `0100152000022000`) and Balatro (update 1.1.3, SDK 18.3.2, `0100CD801CE5E000`). Hades renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; `main` imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (`shader_split.py` `idrk_shaders`). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its `main` holds GLSL sources and imports `glslcCompile`). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (`FRAME_STEP` in their profiles' `patches.py`, `nvn_fstep.h`, profile keys `fps_target` and `frame_step`); Tears of the Kingdom has a 5 GB guest layout, and its `sdk` build (Breath of the Wild's) has the SDK's address-region node area cut (`sdk_patches.py`): without it the resource heaps ran out of alias region at the first descent from the Great Sky Island and the game played its memory-shortage Blood Moon, which put Link back at the start (its log shows `MSF`). Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK archives (`shader_split.py` `kpka_jobs`, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (`hle_nvn.c` `vpool_*`, a 4 KB page table the replay reads textures through); it runs at 60 fps (30 on the Switch). Super Mario Galaxy is the Wii game on NERD's HaLine truncated
|
||||
- Each game's constants and shader tables form a game profile in `dm/games/<id>/`, where `<id>` is the base game's title ID: `game.tsv` plus the tables (and `patches.py` when the rewriter has patches for its builds). Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)).
|
||||
- Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it.
|
||||
- Per-build knowledge is keyed by the NSO build ID: rewriter cuts, observation gates and frame-step stubs in the profile's `patches.py` (a game's own modules) or `dm/tools/sdk_patches.py` (Nintendo's `sdk` modules, which several games ship), which `dm/tools/patch_nso.py` (the general rewriter) loads, the profile's `build` rows (the builds the game was brought up on, and their display versions; the NSP script packs any other version from a derived profile, without those builds' fixes), and host behaviour that uses fixed module addresses checks `module_t.build_id` (from `fixups.tsv`; `dm_main_build()`). When a new game build appears, carry addresses over with `dm/tools/nso_fnmap.py` and check them by disassembly; never let fixed addresses run on a build they were not found in.
|
||||
|
||||
@@ -49,7 +49,7 @@ Needs iOS / iPadOS 26 or later on an iPhone 15 Pro or newer (A17 Pro chip or lat
|
||||
| <br>DRAGON QUEST VII Reimagined<br><sub>IPA ~5.9 GB with update 1.2.0</sub> | <br>FINAL FANTASY RESONANCE<br><sub>IPA ~10 GB · 60 fps (30 on Switch) with update 1.0.1</sub> |
|
||||
| <br>Luigi's Mansion 3<br><sub>IPA ~7.5 GB · 60 fps (30 on Switch) with update 1.4.0</sub> | <br>AI: THE SOMNIUM FILES<br><sub>IPA ~8.3 GB</sub> |
|
||||
| <br>ANIMAL WELL<br><sub>IPA ~54 MB · update 1.0.10</sub> | <br>AI: THE SOMNIUM FILES - nirvanA Initiative<br><sub>IPA ~11.6 GB · update 1.0.2</sub> |
|
||||
| <br>Mario Kart 8 Deluxe<br><sub>IPA ~11.9 GB · update 4.0.0 · Mii heads not drawn (no firmware)</sub> | |
|
||||
| <br>Mario Kart 8 Deluxe<br><sub>IPA ~11.9 GB · update 4.0.0 · Mii heads not drawn (no firmware)</sub> | <br>Balatro<br><sub>IPA ~82 MB · update 1.1.3 · played by touch</sub> |
|
||||
|
||||
## Pack an IPA
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
# generated
|
||||
|
@@ -0,0 +1,48 @@
|
||||
# Game profile of Balatro (Switch, update 1.1.3 = v851968). Plays on the iPhone; on the Mac through the title into a run (touch: the mouse), with sound
|
||||
# Format: see dm/games/README.md
|
||||
# LÖVE 11.4 ("Mysterious Mysteries", love-nx) with LuaJIT 2.1.0-beta3 in its interpreter, on SDL, SDK 18.3.2: the game itself is Lua source in the RomFS (main.lua, game.lua, engine/,
|
||||
# functions/), its pictures PNG, its sound Ogg Vorbis. It renders with NVIDIA's desktop OpenGL 4.5 through EGL (subsdk0, not loaded): main gets its GL functions from LÖVE's glad
|
||||
# loader (eglGetProcAddress) and calls about 150 of them (dm/tools/loader_census.py), which the host's GL layer implements on Metal (dm/gfx/gl/hle_gl.m), on the core profile's paths.
|
||||
# main imports vkGetInstanceProcAddr only for SDL's Vulkan loader, which nothing uses (subsdk1 is NVIDIA's Vulkan driver, not loaded either)
|
||||
# The shaders are GLSL that LÖVE puts together at run time, in Lua, from the game's 19 resources/shaders/*.fs files and its own built-in code: no game file holds them, so the NSP
|
||||
# script captures them by booting the game on the Mac (gl_capture.tsv lists them) and dm/gfx/tools/gl_shaders.py translates them offline; the host finds each by the hash of its text
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 0100CD801CE5E000
|
||||
program_id 0100CD801CE5E000
|
||||
|
||||
# ---- App ----
|
||||
# Game name: only a label for profile listings and the Mac window title; the iOS app name is the game name in the base NSP, and the bundle ID comes from the base game title ID (build_ipa.sh / NSP script)
|
||||
name Balatro
|
||||
ipa Balatro.ipa
|
||||
app_category public.app-category.card-games
|
||||
# handheld: the Switch build plays by touch there (SDL turns the touch screen into LÖVE touches and mouse presses: tap a card, drag it), which on the iPhone are the touches no
|
||||
# gamepad control takes, on the Mac the mouse; docked it reads only the pads. Its window is then the 1280x720 handheld display
|
||||
op_mode handheld
|
||||
# touch: the virtual gamepad starts in its Tap layout, every touch to the game (the pads' corners would cover Run Info, Options and the deck); its layout button brings back 2D / 3D
|
||||
pad_layout touch
|
||||
|
||||
# ---- Versions ----
|
||||
build v851968 1.1.3 main=294d920907e888814f9040550861fdf5 sdk=f118ae1c2e7728471aaf1a0d58484968
|
||||
package v851968
|
||||
shader_versions v851968
|
||||
|
||||
# ---- Files ----
|
||||
# load order as on Horizon: main, subsdk*, sdk. subsdk0 / subsdk1 are NVIDIA's OpenGL and Vulkan drivers, which program the GPU through NvRm: never loaded, the GL layer answers
|
||||
# main's EGL imports
|
||||
modules main sdk
|
||||
romfs_marker resources/fonts/NotoSansSC-Bold.ttf
|
||||
zip_store .ogg:.png:.ttf:.metallib
|
||||
|
||||
# ---- Memory ----
|
||||
# The NPDM declares no system resource, so the game has no virtual address memory: nninitStartup gives its malloc a heap of the pool − used memory, which it maps only as it touches
|
||||
# it; main's 264 MB bss counts as used memory
|
||||
mem_reserve 0xf0000000
|
||||
mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
sys_resource 0x0
|
||||
|
||||
# ---- Shader split ----
|
||||
# no NVN shaders: the NVN tables stay empty (gl_shaders.py writes the GL library and its table, not the profile)
|
||||
shader_note Balatro has no NVN shaders (its GLSL shaders go through the GL layer)
|
||||
|
Can't render this file because it contains an unexpected character in line 3 and column 15.
|
@@ -0,0 +1,38 @@
|
||||
# GLSL shaders Balatro puts together at run time (the Mac host's DM_GL_CAPTURE, booting to the title screen): LÖVE's built-in ones and those of the game's 19 resources/shaders files,
|
||||
# all made by Game:start_up before the first frame; <key>.vert / <key>.frag, key = sha256(GLSL)[:16]. The NSP script's GL shader capture runs until every one here is captured;
|
||||
# gl_shaders.py translates the capture with the RomFS
|
||||
025a79df0e26f6ac.frag
|
||||
0e85b14e40ab14af.frag
|
||||
153a84611e5a520a.frag
|
||||
1688bb2aa8983a13.frag
|
||||
1b304327da2df290.vert
|
||||
220675a48d6335db.vert
|
||||
2e65932217333729.vert
|
||||
3bc7e1b81e471167.vert
|
||||
47edfdacbd78c512.frag
|
||||
4b655edbb3470c5a.frag
|
||||
4c23d6b983ddb7d7.vert
|
||||
5d7917d7f983df9b.frag
|
||||
6365d3674b5631c3.vert
|
||||
71d85f3763912406.frag
|
||||
84f2c2b173b6f948.vert
|
||||
8feaf8cecebfafbf.frag
|
||||
916ab7fdd35c3a95.frag
|
||||
94552b2eaf21f593.frag
|
||||
955ac2558c587086.frag
|
||||
98d3ebf757a82902.vert
|
||||
9e343d31b09c61b7.vert
|
||||
9f536d3d12f3edf1.vert
|
||||
a59000fda1e1e00b.vert
|
||||
a8aada4f37f45489.frag
|
||||
ba1a64df36dc291b.frag
|
||||
bb83fe70011ceebb.frag
|
||||
bfc21d9532c99682.frag
|
||||
d28b725d43fc106e.vert
|
||||
e7f9b54e2d2d7fbb.frag
|
||||
ed2e2b4a7c901145.vert
|
||||
f20b9c7c0322241d.frag
|
||||
f2314b010f695639.vert
|
||||
f6cfca5c3f6901e4.frag
|
||||
f8034c074d4e8965.vert
|
||||
fc956bad781f15af.frag
|
||||
|
@@ -0,0 +1 @@
|
||||
# Prewarm seed list (shipped in the app, next to shader_map.tsv). Rows: S<TAB>shader key<TAB>stage / R<TAB>render pipeline key (pso_skey hex) / C<TAB>compute shader key.
|
||||
|
@@ -0,0 +1 @@
|
||||
# Answer table for translation queries (generated by dm/gfx/tools/shader_split.py, do not edit): key <TAB> fact JSON. This game has no PC version to pair with: only the thread group / shared memory of compute shaders (taken from the control section); other queries get the translator defaults
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# kind: evidence Balatro has no NVN shaders: the table stays empty (its GLSL shaders go through the GL layer)
|
||||
package index stage dm_ctl_sha16 source
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# kind: evidence control-section key conflict rulings: empty
|
||||
dm_ctl_sha16 chosen_pc class candidates records
|
||||
|
@@ -0,0 +1 @@
|
||||
# kind: evidence vertex / pixel shader pairs within BNSH variations (generated by dm/gfx/tools/shader_split.py, do not edit): vertex key <TAB> pixel key
|
||||
|
+3
-2
@@ -7,6 +7,7 @@ Each game the port knows has a folder `dm/games/<id>/`, named after the base gam
|
||||
- **`shader_facts_manual.tsv`** (optional): hand-written translation facts in `shader_facts.tsv`'s row format, for what the machine code cannot tell (Link's Awakening: a vertex input the integer-input guess gets wrong; Breath of the Wild and Tears of the Kingdom: integer fragment outputs, which a game without a PC version has no other source for). `shader_split.py` merges it over the generated rows; since the table is on the chain, changing it changes only that profile's chain version.
|
||||
- **`pso_manifest.tsv`**: the prewarm seed list.
|
||||
- **`vk_capture.tsv`** (a Vulkan game that generates its shaders at run time, Super Mario Galaxy): the keys of the SPIR-V modules it generates.
|
||||
- **`gl_capture.tsv`** (an OpenGL game that puts its GLSL together at run time, Balatro): the files (`<key>.vert` / `<key>.frag`) of the shaders it compiles.
|
||||
- **`bf_want.tsv`**.
|
||||
- **`patches.py`** (optional): the rewriter's patches of the game's own builds (`main`), keyed by NSO build ID: `CUTS` (byte cuts, each original asserted), `OBS` (observation gates) and `FRAME_STEP` (frame-step stubs). `dm/tools/patch_nso.py` loads every profile's file and applies those of a module's build, wherever it comes from; a build claimed twice stops the rewrite. Cuts of Nintendo's `sdk` modules, which several games ship, are in `dm/tools/sdk_patches.py`. A game without patches has no file, and a derived profile never has one. `dm/tools/tests/test_patches.py` checks that a file names only its own profile's builds.
|
||||
|
||||
@@ -57,7 +58,7 @@ None of them has a default game: without one they stop and list the profiles. To
|
||||
| `device_api` | Optional, decided by the SDK version without it: the NVN API version `major.minor` that `nvnDeviceGetInteger` reports, 55.15 for SDK 11 and later (what Mario requires) and 53.313 for SDK 10 and older (`hle_nvn.c` `nvn_api`). Games check it and skip creating their device on a mismatch: Super Mario 3D World + Bowser's Fury (SDK 10) requires major 53 and minor ≥ 313. No profile sets it. `DM_NVN_API` overrides. Not part of the shader chain version (only `shader_*` keys are) |
|
||||
| `op_mode` | The operation mode the game sees at start: `console` (the default, docked) or `handheld`. Super Mario Galaxy and Captain Toad: Treasure Tracker use `handheld`: docked, their pointer is the Joy-Con gyro, which the host does not have; handheld, it is the touch screen, which the iPhone front end feeds with the touches no gamepad control takes (in the 3D layout, outside the right third of the screen, which is the right stick; `DmGamepad.m`). Handheld also renders at 720p. `DM_OPMODE=0` / `1` overrides, also from `Documents/env.txt` |
|
||||
| `perf_mode` | `follow` (the default): the performance mode follows the operation mode (Boost docked, Normal handheld); `boost`: Boost whatever the operation mode (`hle_oe.c`), a combination a Switch 1 never reports, so a profile choice. Rhythm Heaven Groove plays handheld with Boost: docked means a TV to the game, which then asks for a latency calibration, and only Boost gives its 1920x1080 draw size (handheld Normal draws 1280x720) |
|
||||
| `pad_layout` | The iPhone virtual gamepad's layout on first launch: `simple` (the default) = 2D (left stick, A/B/X/Y, shoulders, − / +), `full` = 3D (adds a D-pad, a right stick and L3 / R3; the right stick is not drawn: a touch landing in the right third of the screen on no control holds it, centred where it lands). The layout button at the top centre switches between them in game and the app remembers the player's choice (`DmGamepad.m`) |
|
||||
| `pad_layout` | The iPhone virtual gamepad's layout on first launch: `simple` (the default) = 2D (left stick, A/B/X/Y, shoulders, − / +), `full` = 3D (adds a D-pad, a right stick and L3 / R3; the right stick is not drawn: a touch landing in the right third of the screen on no control holds it, centred where it lands), `touch` = Tap (no control but the layout button: every touch goes to the game's touch screen; needs `op_mode handheld`, Balatro). The layout button at the top centre goes to the next layout in game (Tap only in handheld mode) and the app remembers the player's choice (`DmGamepad.m`) |
|
||||
| `feedback_snap` | Fragment shader keys (prefixes) whose draws sample a colour attachment of their own render pass and must read a snapshot instead (`fb_prepare` in `nvn_mtl.m`). Writing and sampling one attachment in a render pass is undefined in Metal: the Mac usually happens to read the old content, the iPhone sometimes reads whole 16×16 blocks as magenta. Each such draw costs a pass split and a copy, so only the shaders seen to break are listed (Super Mario 3D World + Bowser's Fury: its final composite; Captain Toad: Treasure Tracker: the same agl program, whose blocks showed on the Mac too). `DM_NVN_FB_SNAP=0` / `all` / `<keys>` overrides, also from `Documents/env.txt` |
|
||||
| `alpha_from_red` | Fragment shader keys (prefixes, up to 48) that read a single-channel render target whose channel mapping is exactly (R, 0, 0, 1) as (R, 0, 0, R) (`tex_alpha_red` in `nvn_mtl.m`). It was a workaround for NintendoWare layout captures: the game rebuilds each capture with its own mapping ((1, 1, 1, R) for one tagged `A8`), and the textures kept (R, 0, 0, 1) only while the host's `nvnTextureCompare` ignored the mapping (fixed, see [04](../../docs/agents/04-graphics.md)). Breath of the Wild no longer lists it; Super Mario Odyssey still lists its 26 layout shaders. `DM_NVN_ALPHA_RED=0` / `<keys>` overrides, also from `Documents/env.txt` |
|
||||
| `tex_flush_upload` | `on` / `off` (the default): a texture the game fills from the CPU in place (no packaged layout, no `WriteTexels`) and publishes with `nvnMemoryPoolFlushMappedRange` is uploaded from guest memory like a packaged one, and re-uploaded when its pages change (`ft_add` in `hle_nvn.c`). Tears of the Kingdom streams all its textures this way; without it the 3D scene is black. Off by default because a flushed range in another game could cover an image a shader writes, whose GPU content the upload would replace. Without the key, every game still gets a texture's storage once when the game flushed it before the texture's first use (`nvn_flushed` in `hle_nvn.c`: Breath of the Wild's sky tables); the key is for textures the CPU writes again after that |
|
||||
@@ -73,7 +74,7 @@ None of them has a default game: without one they stop and list the profiles. To
|
||||
These are not profile keys: the tools read them from the game files each time, by rules checked against every profile (`dm/games/profile_guess.py` `shader_sources`).
|
||||
|
||||
- **Graphics API.** A game whose `main` imports Vulkan functions and not `nvnBootstrapLoader` renders with Vulkan, which the host implements on Metal (`dm/gfx/vk/hle_vk.m`; Hades, Super Mario Galaxy). Its shaders are its SPIR-V files, translated by `dm/gfx/tools/vk_shaders.py <RomFS> <shader dir> [--ios <dir>]` into `<dir>/vk/`, and `build_ipa.sh` ships `$MSL_IOS/vk` instead of the NVN chain libraries (`GFXAPI`, which the NSP script passes; by hand the shader libraries there tell). The NVN shader tables stay empty.
|
||||
- **SPIR-V generated at run time.** A Vulkan game without SPIR-V files in its RomFS generates its shaders at run time (Super Mario Galaxy: Hagi's GX shaders, 3,089 modules created from its pipeline database while it shows "Initializing system..."). The NSP script captures them on the Mac with `DM_VK_CAPTURE=<dir>` (`dm/gfx/vk/hle_vk.m`), booting the game in the Mac host until every key the profile's `vk_capture.tsv` lists is there, and `vk_shaders.py` translates the capture folder with the RomFS.
|
||||
- **SPIR-V generated at run time.** A Vulkan game without SPIR-V files in its RomFS generates its shaders at run time (Super Mario Galaxy: Hagi's GX shaders, 3,089 modules created from its pipeline database while it shows "Initializing system..."). The NSP script captures them on the Mac with `DM_VK_CAPTURE=<dir>` (`dm/gfx/vk/hle_vk.m`), booting the game in the Mac host until every key the profile's `vk_capture.tsv` lists is there, and `vk_shaders.py` translates the capture folder with the RomFS. An OpenGL game with no GLSL source in its RomFS or `main` (Balatro: LÖVE puts its shaders together in Lua) is captured the same way with `DM_GL_CAPTURE=<dir>` (`dm/gfx/gl/hle_gl.m`) until every file in `gl_capture.tsv` is there, and `gl_shaders.py` translates the capture.
|
||||
- **Shaders compiled at run time.** A game whose `main` holds GLSL sources and calls `glslcCompile` in a module it loads compiles some shaders at run time with its own GLSLC (Fire Emblem: Three Houses: 9 built-in KTGL shaders at boot), so no game file holds them. The NSP script captures them on the Mac with `DM_GLSLC_CAPTURE=<dir>` (`dm/host/hle_glslc.c`), booting the game in the Mac host until every capture file the profile's `shader_map.tsv` names is there, and `shader_split.py` reads `<version dir>/glslc_capture/*.glslc` (beside that version's `RomFS`) or `$DM_GLSLC_CAPTURE`; without the folder it prints a red line, those shaders get no translation and the tables differ from the profile's.
|
||||
- **Unreal paks.** An Unreal Engine game keeps its shaders in the shader code archives of its paks (`*.ushaderbytecode`; a pak is Unreal's when its footer has Unreal's magic, `uepak.py` `unreal_paks`), each shader Oodle-compressed, which only the game's own Oodle decodes. The NSP script (`decode_ue_shaders`) takes the archives out of the paks with `dm/gfx/tools/uepak.py` (the pak index is AES-encrypted with a key it finds in `main`), boots the game in the Mac host with `DM_OODLE_JOB` (`dm/host/hle_oodle.c`: the game's `OodleLZ_Decompress`, found as the one function that references the library's own name string, decodes every shader right before `nnMain`, then the host exits) and writes the decoded archives to `<version dir>/ue_shaders`, where `shader_split.py` reads them (`DM_UE_SHADERS` overrides; without them it stops). Octopath Traveler 0: 10,596 shaders in 2 s. Unreal Engine 4.26 and later compress shaders with LZ4 unless the project picks another format (DRAGON QUEST VII Reimagined, 4.27: 64,949 shaders, its packages in IoStore containers beside the pak); `uepak.py` decodes those itself (`lz4_shaders`) and the NSP script skips the host run.
|
||||
- **Trinity packs.** Game Freak's Trinity engine keeps nearly every file in Oodle-compressed packs (`arc/data.trpfs`, files starting `ONEPACK`; Pokémon Legends: Z-A). `shader_split.py` decodes them with the game's own `OodleLZ_Decompress`, run by the Mac host in its tool mode (`dm/host/hle_oodle.c`, `DM_OODLE_SERVE`). The split needs `DM_OODLE_HOST` (the Mac host) and `DM_OODLE_MODULES` (the rewritten modules); the NSP script builds the host and sets both when the split runs.
|
||||
|
||||
+4
-3
@@ -101,8 +101,8 @@ def header(g):
|
||||
if aba not in ('on', 'off'): sys.exit(f'{g.id}: alloc_big_align must be on or off: {aba}')
|
||||
sf = g.get('shared_font', 'bfttf') # what the shared-font address points at (hle_sdk16.c): bfttf = 8-byte header + TTF, ttf = the TTF alone
|
||||
if sf not in ('bfttf', 'ttf'): sys.exit(f'{g.id}: shared_font must be bfttf or ttf: {sf}')
|
||||
pl = g.get('pad_layout', 'simple') # the virtual gamepad's first layout (DmGamepad.m; afterwards whatever the player switched to in game is kept): simple = 2D, full = 3D (D-pad, right stick, L3 / R3)
|
||||
if pl not in ('simple', 'full'): sys.exit(f'{g.id}: pad_layout must be simple or full: {pl}')
|
||||
pl = g.get('pad_layout', 'simple') # the virtual gamepad's first layout (DmGamepad.m; afterwards whatever the player switched to in game is kept): simple = 2D, full = 3D (D-pad, right stick, L3 / R3),
|
||||
if pl not in ('simple', 'full', 'touch'): sys.exit(f'{g.id}: pad_layout must be simple, full or touch: {pl}') # touch = Tap (no control: every touch goes to the game's touch screen)
|
||||
fbs = g.words('feedback_snap', '') # fragment shader key prefixes whose same-pass feedback sampling gets a snapshot first (nvn_mtl.m fb_prepare); none by default
|
||||
if any(not re.fullmatch(r'[0-9a-f]{8,16}', k) for k in fbs): sys.exit(f'{g.id}: feedback_snap takes fragment shader keys (8-16 lowercase hex digits): {fbs}')
|
||||
afr = g.words('alpha_from_red', '') # fragment shader key prefixes that read single-channel textures mapped (R, 0, 0, 1) with alpha = red (nvn_mtl.m tex_alpha_red); none by default
|
||||
@@ -123,6 +123,7 @@ def header(g):
|
||||
if gts not in ('submission', 'segment'): sys.exit(f'{g.id}: gpu_timestamps must be submission or segment: {gts}')
|
||||
om = g.get('op_mode', 'console') # the operation mode reported at start (hle_oe.c dm_opmode; DM_OPMODE overrides): console = docked, handheld = touch screen in use (Super Mario Galaxy's pointer)
|
||||
if om not in ('console', 'handheld'): sys.exit(f'{g.id}: op_mode must be console or handheld: {om}')
|
||||
if pl == 'touch' and om != 'handheld': sys.exit(f'{g.id}: pad_layout touch needs op_mode handheld (the touch screen reaches the game only in handheld mode)')
|
||||
pm = g.get('perf_mode', 'follow') # the performance mode reported (hle_oe.c): follow = the operation mode's (Boost docked, Normal handheld), boost = Boost always
|
||||
if pm not in ('follow', 'boost'): sys.exit(f'{g.id}: perf_mode must be follow or boost: {pm}')
|
||||
api = g.get('device_api', '55.15') # the NVN API major.minor version nvnDeviceGetInteger reports (hle_nvn.c)
|
||||
@@ -152,7 +153,7 @@ def header(g):
|
||||
f'#define DM_GAME_FPS_TARGET {int(ft)}',
|
||||
f'#define DM_GAME_FPS_HOLD {int(fh == "on")}',
|
||||
*([f'#define DM_GAME_SHARED_FONT_TTF {int(sf == "ttf")}'] if g.get('shared_font', '') else []), # without the key the host decides by the SDK version (hle_sdk16.c)
|
||||
f'#define DM_GAME_PAD_FULL {int(pl == "full")}',
|
||||
f'#define DM_GAME_PAD_LAYOUT {("simple", "full", "touch").index(pl)}',
|
||||
f'#define DM_GAME_OPMODE {int(om == "console")}',
|
||||
f'#define DM_GAME_PERF_BOOST {int(pm == "boost")}',
|
||||
f'#define DM_GAME_FEEDBACK_SNAP {q(" ".join(fbs))}',
|
||||
|
||||
@@ -24,6 +24,8 @@ MARKERS = [(b'nvnBootstrapLoader', 'NVN'), (b'vkCreateInstance', 'Vulkan'), (b'e
|
||||
(b'sead::', 'Nintendo sead'), (b'agl::', 'Nintendo agl'), (b'gsys::', 'Nintendo gsys'), (b'Unreal', 'Unreal'), (b'UE4', 'Unreal 4'), (b'Havok', 'Havok'), (b'CRIWARE', 'CRIWARE'), (b'Wwise', 'Wwise'),
|
||||
(b'MonoGame.Framework', 'MonoGame'), (b'FMOD mixer', 'FMOD')]
|
||||
GLSLC_MAGIC = b'\x91\x68\x86\x19' # GLSLC compiler outputs (shader_split.py glslc_shaders)
|
||||
GL_SRC = rb'\0#version [0-9]' # a GLSL source string as dm/gfx/tools/gl_shaders.py takes it from main (Balatro's main has none, only glslang's messages that start with '#version:')
|
||||
GL_FILES = ('.mvgl', '.vert', '.frag') # the RomFS files gl_shaders.py reads GLSL from
|
||||
APP_POOL = 0xCD500000 # the application memory pool of a 4 GB Switch (3285 MiB), what every profile but Engage's reports
|
||||
MiB = 1 << 20
|
||||
|
||||
@@ -70,7 +72,7 @@ def npdm(path): # (system resource size, program ID) from main.npdm; a non-zer
|
||||
def facts_of(exefs, romfs):
|
||||
"""What the profile rules (and the survey report) need from a game's code modules and RomFS"""
|
||||
exefs = Path(exefs)
|
||||
F = {'mods': {}, 'markers': {}, 'sdkver': set(), 'comps': set(), 'unity': set(), 'glslc': 0, 'glsl_src': 0, 'gl_names': 0, 'sysres': 0, 'program': None}
|
||||
F = {'mods': {}, 'markers': {}, 'sdkver': set(), 'comps': set(), 'unity': set(), 'glslc': 0, 'glsl_src': 0, 'gl_names': 0, 'gl_src': 0, 'sysres': 0, 'program': None}
|
||||
for p in sorted(exefs.iterdir()):
|
||||
with open(p, 'rb') as f:
|
||||
if f.read(4) != b'NSO0': continue
|
||||
@@ -87,6 +89,7 @@ def facts_of(exefs, romfs):
|
||||
if p.name == 'main':
|
||||
F['unity'] = {v.decode() for v in re.findall(rb'20[12][0-9]\.[0-9]+\.[0-9]+f[0-9]', img)}
|
||||
F['glslc'], F['glsl_src'], F['gl_names'] = img.count(GLSLC_MAGIC), img.count(b'#version '), img.count(b'glShaderSource')
|
||||
F['gl_src'] = len(re.findall(GL_SRC, img))
|
||||
F['main_markers'] = {name for pat, name in MARKERS if pat in img}
|
||||
m['kind'] = {'main': 'the game', 'sdk': 'nnSdk', 'rtld': 'rtld'}.get(p.name) or kind(m)
|
||||
F['sysres'], F['program'] = npdm(exefs / 'main.npdm')
|
||||
@@ -167,20 +170,23 @@ GPU_DRIVERS = {'NintendoSDK_OpenGL', 'NintendoSDK_Vulkan'}
|
||||
|
||||
|
||||
# ---- Shader sources: decided from the game files at every build, not kept in the profile (scripts/build_ipa_from_nsp.py; shader_split.py reads what those steps write) ----
|
||||
def shader_sources(imports, glsl_src, glslc_mods, spirv, unreal_paks, trinity_packs=0, legacy_paks=0, gl_names=0):
|
||||
def shader_sources(imports, glsl_src, glslc_mods, spirv, unreal_paks, trinity_packs=0, legacy_paks=0, gl_names=0, gl_src=0, gl_files=0):
|
||||
"""{name: (value, reason)}: graphics (vulkan when main imports Vulkan functions and not nvnBootstrapLoader; gl when it imports eglCreateWindowSurface and names glShaderSource, so it compiles
|
||||
GLSL through OpenGL (gl_names: how often main has that name; Absolum imports the same EGL functions for SDL3 but renders with NVN and names no GL function); else nvn), vk_capture (a Vulkan game without SPIR-V files generates its shaders at
|
||||
run time), glslc_capture (main holds GLSL sources and calls glslcCompile in a module the game loads: it compiles some shaders at run time), ue_decode (Unreal paks keep the shaders Oodle-compressed
|
||||
GLSL through OpenGL (gl_names: how often main has that name; Absolum imports the same EGL functions for SDL3 but renders with NVN and names no GL function); else nvn; vkGetInstanceProcAddr
|
||||
alone is SDL's Vulkan loader, which Balatro's LÖVE links and never uses), vk_capture (a Vulkan game without SPIR-V files generates its shaders at run time), gl_capture (an OpenGL game with no
|
||||
GLSL source where gl_shaders.py looks, gl_src strings in main and gl_files files in the RomFS: it puts its shaders together at run time, LÖVE from its Lua), glslc_capture (main holds GLSL sources and calls glslcCompile in a module the game loads: it compiles some shaders at run time), ue_decode (Unreal paks keep the shaders Oodle-compressed
|
||||
in their shader code archives, which only the game's own Oodle decodes), oodle_packs (Trinity packs, Oodle-compressed file by file, which the shader split decodes with the game's own decoder in
|
||||
the Mac host's tool mode), ue_legacy (legacy Unreal paks, version 4: the shaders are inline in the cooked packages, which the shader split reads through the game's own decoder the same way).
|
||||
unreal_paks counts the version 11 paks, legacy_paks the older ones. Checked against every profile: Hades and Super Mario Galaxy Vulkan, Galaxy alone with no SPIR-V file, Three Houses alone compiling
|
||||
shaders (13 Sentinels holds 106 GLSL sources but does not import glslcCompile; Captain Toad: Treasure Tracker imports it with agl's GLSL sources, but no module defines it), Octopath Traveler 0
|
||||
alone with version 11 Unreal paks, DRAGON QUEST XI S alone with legacy ones, Pokémon Legends: Z-A alone with Trinity packs, Digimon Story Cyber Sleuth alone with OpenGL (of the 22 games extracted)"""
|
||||
vk = sum(1 for s in imports if re.match(r'vk[A-Z]', s)); vulkan = bool(vk) and 'nvnBootstrapLoader' not in imports
|
||||
alone with version 11 Unreal paks, DRAGON QUEST XI S alone with legacy ones, Pokémon Legends: Z-A alone with Trinity packs, Digimon Story Cyber Sleuth and Balatro alone with OpenGL (of the 23 games
|
||||
extracted; Hades and both Galaxies import 99 or 100 Vulkan functions, Balatro only vkGetInstanceProcAddr), Balatro alone composing its GLSL"""
|
||||
vk = sum(1 for s in imports if re.match(r'vk[A-Z]', s) and s != 'vkGetInstanceProcAddr'); vulkan = bool(vk) and 'nvnBootstrapLoader' not in imports
|
||||
gl = not vulkan and 'eglCreateWindowSurface' in imports and bool(gl_names)
|
||||
return {'graphics': ('vulkan', f'main imports {vk} Vulkan functions and not nvnBootstrapLoader: the host\'s Vulkan layer (dm/gfx/vk/hle_vk.m)') if vulkan else
|
||||
('gl', 'main imports EGL and names glShaderSource: it renders with OpenGL and compiles GLSL, the host\'s GL layer (dm/gfx/gl/hle_gl.m)') if gl else ('nvn', 'main imports nvnBootstrapLoader'),
|
||||
'vk_capture': (vulkan and not spirv, 'no SPIR-V file in the RomFS: the game generates its shaders at run time (Super Mario Galaxy), captured by booting it on the Mac'),
|
||||
'gl_capture': (gl and not gl_src and not gl_files, 'no GLSL source in the RomFS or main: the game puts its shaders together at run time (LÖVE: Balatro), captured by booting it on the Mac'),
|
||||
'glslc_capture': (not vulkan and not gl and bool(glsl_src) and 'glslcCompile' in imports and bool(glslc_mods),
|
||||
f'main holds {glsl_src} GLSL sources and calls glslcCompile in {", ".join(glslc_mods) or "no module it loads"}: shaders compiled at run time (Fire Emblem: Three Houses), captured by booting it on the Mac'),
|
||||
'ue_decode': (not vulkan and not gl and bool(unreal_paks), f'{unreal_paks} Unreal pak(s): the shaders are Oodle-compressed in their shader code archives (Octopath Traveler 0), decoded by booting the game on the Mac'),
|
||||
@@ -191,7 +197,7 @@ def shader_sources(imports, glsl_src, glslc_mods, spirv, unreal_paks, trinity_pa
|
||||
def build_sources(exefs, romfs, modules):
|
||||
"""shader_sources from an extraction (its ExeFS and RomFS folders) and the modules the profile loads"""
|
||||
exefs = Path(exefs); main = nso(exefs / 'main')[0]
|
||||
imports = {s for s, defined, _ in nso_syms.dynsyms(main) if not defined}; glsl_src = main.count(b'#version '); gl_names = main.count(b'glShaderSource')
|
||||
imports = {s for s, defined, _ in nso_syms.dynsyms(main) if not defined}; glsl_src = main.count(b'#version '); gl_names = main.count(b'glShaderSource'); gl_src = len(re.findall(GL_SRC, main))
|
||||
glslc_mods = [m for m in modules if m != 'main' and glsl_src and 'glslcCompile' in imports and (exefs / m).exists()
|
||||
and any(s == 'glslcCompile' and defined for s, defined, _ in nso_syms.dynsyms(nso(exefs / m)[0]))]
|
||||
spirv = 0
|
||||
@@ -203,7 +209,8 @@ def build_sources(exefs, romfs, modules):
|
||||
for n in names:
|
||||
if n.endswith('.trpfs'):
|
||||
with open(os.path.join(d, n), 'rb') as f: trinity += f.read(8) == b'ONEPACK\0'
|
||||
return shader_sources(imports, glsl_src, glslc_mods, spirv, len(uepak.unreal_paks(str(romfs), legacy=False)), trinity, len(uepak.unreal_paks(str(romfs), legacy=True)), gl_names)
|
||||
gl_files = sum(1 for _, _, names in os.walk(romfs) for n in names if n.lower().endswith(GL_FILES)) if 'eglCreateWindowSurface' in imports else 0
|
||||
return shader_sources(imports, glsl_src, glslc_mods, spirv, len(uepak.unreal_paks(str(romfs), legacy=False)), trinity, len(uepak.unreal_paks(str(romfs), legacy=True)), gl_names, gl_src, gl_files)
|
||||
|
||||
|
||||
def va_heap(sysres, imports): # virtual address memory (an NPDM system resource) and a heap: the alias / heap split is a starting point the heap check corrects (Wonder, Tears of the Kingdom; Engage's pool is its own measurement)
|
||||
|
||||
+1
-1
@@ -81,7 +81,7 @@ Binding conventions followed during translation (the NVN driver's layout, the sa
|
||||
| `nvn/names.txt` | The NVN API name list. The per-game tables described above live in the game profile `dm/games/<id>/` (named after the base game's title ID; tools pick it with `DM_GAME`, make with `GAME`; there is no default game) |
|
||||
| `tex/` | Texture format mapping, deswizzling, BC decompression, see [tex/README.md](tex/README.md) |
|
||||
| `vk/hle_vk.m` | Vulkan on Metal, for games that render with Vulkan (Hades, Super Mario Galaxy); their SPIR-V shaders are translated by `tools/vk_shaders.py` |
|
||||
| `gl/hle_gl.m` | OpenGL through EGL on Metal, for games that render with NVIDIA's GL driver (Digimon Story Cyber Sleuth: Complete Edition); their GLSL sources are translated by `tools/gl_shaders.py` (glslang, then SPIRV-Cross) |
|
||||
| `gl/hle_gl.m` | OpenGL through EGL on Metal, for games that render with NVIDIA's GL driver (Digimon Story Cyber Sleuth: Complete Edition; Balatro, LÖVE on its core-profile paths); their GLSL sources are translated by `tools/gl_shaders.py` (glslang, then SPIRV-Cross) |
|
||||
| `metal/` | A standalone minimal Metal offscreen example, not compiled into the host, see [metal/README.md](metal/README.md) |
|
||||
| `tools/` | Offline tools, see below |
|
||||
| `tests/` | Unit tests |
|
||||
|
||||
+576
-111
File diff suppressed because it is too large.
Load diff
@@ -35,20 +35,14 @@ def text_of(b):
|
||||
return b[:z] if z >= 0 else b
|
||||
|
||||
def nso_strings(path):
|
||||
"""'#version' strings of an NSO's read-only segment (lz4 segments, as dm/tools/nso.py decompresses them)"""
|
||||
"""'#version <number>' strings of an NSO's read-only segment (lz4 segments, as dm/tools/nso.py decompresses them); a main that links glslang (Balatro's LÖVE) also has its messages that
|
||||
start with '#version:'"""
|
||||
import lz4.block, struct
|
||||
d = open(path, 'rb').read()
|
||||
flags = struct.unpack_from('<I', d, 0xc)[0]; fo, mo, sz = struct.unpack_from('<III', d, 0x20); csz = struct.unpack_from('<I', d, 0x64)[0]
|
||||
img = d[fo:fo + csz]
|
||||
if flags >> 1 & 1: img = lz4.block.decompress(img, uncompressed_size=sz)
|
||||
out, i = [], 0
|
||||
while True:
|
||||
i = img.find(b'#version', i)
|
||||
if i < 0: break
|
||||
if i == 0 or img[i - 1] == 0:
|
||||
out.append(text_of(img[i:i + 0x10000]))
|
||||
i += 8
|
||||
return out
|
||||
return [text_of(img[m.start():m.start() + 0x10000]) for m in re.finditer(rb'(?:^|(?<=\0))#version [0-9]', img)]
|
||||
|
||||
def sources(roots):
|
||||
"""(source bytes, stage or None, where) for every shader in the input paths"""
|
||||
@@ -130,6 +124,8 @@ def translate(src, stage, key, tmp):
|
||||
for v in refl.get('inputs', []): f.append(f'a:{v["name"]}={v["location"]}:{v["type"]}')
|
||||
else:
|
||||
for v in refl.get('outputs', []): f.append(f'o:{v["name"]}={v["location"]}')
|
||||
# a bool uniform is a uint in the SPIR-V block (it has no bool members), but the game asks GL for its declared type: LÖVE's Shader:send takes only a boolean for a GLSL bool
|
||||
bools = {n for n in re.findall(r'\buniform\s+(?:(?:lowp|mediump|highp)\s+)?bool\s+(\w+)\s*;', pre)}
|
||||
for u in refl.get('ubos', []):
|
||||
t = refl['types'][u['type']]
|
||||
if t['name'] != 'gl_DefaultUniformBlock': return f'uniform block {t["name"]} is not the default block'
|
||||
@@ -138,6 +134,7 @@ def translate(src, stage, key, tmp):
|
||||
arr = mb.get('array', [])
|
||||
if len(arr) > 1: return f'uniform {mb["name"]}: arrays of arrays'
|
||||
ty = glsl_type(mb['type'], refl)
|
||||
if ty == 'uint' and mb['name'] in bools: ty = 'bool'
|
||||
if ty not in VEC_TYPES: return f'uniform {mb["name"]}: type {ty} (structs are not implemented)'
|
||||
f.append(f'u:{mb["name"]}={mb["offset"]}:{ty}:{arr[0] if arr else 0}:{mb.get("array_stride", 0)}')
|
||||
for t in refl.get('textures', []):
|
||||
|
||||
+14
-14
@@ -38,13 +38,6 @@ HLE_FORCE("_ZN2nn5audio30GetAudioRendererWorkBufferSizeERKNS0_22AudioRendererPar
|
||||
{ static int once; if (!once++) LOG("audren: work buffer %#zx", n); }
|
||||
return n;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleERKNS0_22AudioRendererParameterEPvm", "open the renderer: switched to CPU rendering + manual execution, then the SDK in-process renderer",
|
||||
Result, h_ar_Open0, void **handle, const uint8_t *param, void *work, size_t sz) {
|
||||
static Result (*orig)(void **, const uint8_t *, void *, size_t);
|
||||
if (!orig) orig = gsym("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleERKNS0_22AudioRendererParameterEPvm");
|
||||
uint8_t p[P_SIZE]; cpu_manual(p, param); return orig(handle, p, work, sz);
|
||||
}
|
||||
|
||||
// ---------- rendering and output ----------
|
||||
static void *g_handle[2]; // AudioRendererHandle (renderer object, client state)
|
||||
static _Atomic int g_open;
|
||||
@@ -86,21 +79,28 @@ static void render_main(void *arg) {
|
||||
cq_popped(); // an update waiting for queue space (catch-up pacing) sees g_open cleared
|
||||
}
|
||||
|
||||
HLE_FORCE("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleEPNS_2os11SystemEventERKNS0_22AudioRendererParameterEPvm", "open the renderer (with event): CPU rendering + manual execution, the host starts a render thread",
|
||||
Result, h_ar_Open, void **handle, void *ev, const uint8_t *param, void *work, size_t sz) {
|
||||
static Result (*orig)(void **, void *, const uint8_t *, void *, size_t);
|
||||
if (!orig) orig = gsym("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleEPNS_2os11SystemEventERKNS0_22AudioRendererParameterEPvm");
|
||||
// Both OpenAudioRenderer forms: CPU rendering + manual execution, then the host's render thread. With the event the game waits on it, which each render signals; without it the
|
||||
// game paces its updates itself (Balatro: LÖVE's OpenAL Soft) and still needs the renderer to run every 5 ms: the form without an event used to start no render thread, so only the
|
||||
// game's updates rendered (catch-up frames, which nothing played) and the game was silent
|
||||
static Result ar_open(void **handle, void *ev, const uint8_t *param, void *work, size_t sz) {
|
||||
static Result (*orig_ev)(void **, void *, const uint8_t *, void *, size_t), (*orig)(void **, const uint8_t *, void *, size_t);
|
||||
if (!orig_ev) orig_ev = gsym("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleEPNS_2os11SystemEventERKNS0_22AudioRendererParameterEPvm");
|
||||
if (!orig) orig = gsym("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleERKNS0_22AudioRendererParameterEPvm");
|
||||
uint8_t p[P_SIZE]; cpu_manual(p, param);
|
||||
LOG("audren: OpenAudioRenderer rate %d samples per frame %d mix buffers %d submixes %d voices %d sinks %d effects %d splitters %d (original device %d mode %d → CPU / manual) work buffer %p+%#zx",
|
||||
*(int32_t *)param, *(int32_t *)(param + 4), *(int32_t *)(param + 8), *(int32_t *)(param + 0xc), *(int32_t *)(param + 0x10), *(int32_t *)(param + 0x14),
|
||||
LOG("audren: OpenAudioRenderer%s rate %d samples per frame %d mix buffers %d submixes %d voices %d sinks %d effects %d splitters %d (original device %d mode %d → CPU / manual) work buffer %p+%#zx",
|
||||
ev ? "" : " (no event)", *(int32_t *)param, *(int32_t *)(param + 4), *(int32_t *)(param + 8), *(int32_t *)(param + 0xc), *(int32_t *)(param + 0x10), *(int32_t *)(param + 0x14),
|
||||
*(int32_t *)(param + 0x18), *(int32_t *)(param + 0x24), *(int32_t *)(param + P_DEVICE), *(int32_t *)(param + P_MODE), work, sz);
|
||||
Result r = orig(handle, ev, p, work, sz);
|
||||
Result r = ev ? orig_ev(handle, ev, p, work, sz) : orig(handle, p, work, sz);
|
||||
if (r) { LOG("[RED] audren: SDK OpenAudioRenderer failed %#x", r); return r; }
|
||||
if (atomic_load(&g_open)) { LOG("[RED] audren: a second renderer (the host drives only one)"); return r; }
|
||||
g_handle[0] = handle[0]; g_handle[1] = handle[1]; atomic_store(&g_open, 1);
|
||||
g_render_tt = dm_guest_thread_spawn(render_main, NULL, 9, 0x10000);
|
||||
return r;
|
||||
}
|
||||
HLE_FORCE("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleEPNS_2os11SystemEventERKNS0_22AudioRendererParameterEPvm", "open the renderer (with event): CPU rendering + manual execution, the host starts a render thread",
|
||||
Result, h_ar_Open, void **handle, void *ev, const uint8_t *param, void *work, size_t sz) { return ar_open(handle, ev, param, work, sz); }
|
||||
HLE_FORCE("_ZN2nn5audio17OpenAudioRendererEPNS0_19AudioRendererHandleERKNS0_22AudioRendererParameterEPvm", "open the renderer (no event): CPU rendering + manual execution, the host starts a render thread",
|
||||
Result, h_ar_Open0, void **handle, const uint8_t *param, void *work, size_t sz) { return ar_open(handle, NULL, param, work, sz); }
|
||||
HLE_FORCE("_ZN2nn5audio18CloseAudioRendererENS0_19AudioRendererHandleE", "close the renderer: stop the host render thread first, then hand over to the SDK", void, h_ar_Close, void *obj, void *state) {
|
||||
static void (*orig)(void *, void *); if (!orig) orig = gsym("_ZN2nn5audio18CloseAudioRendererENS0_19AudioRendererHandleE");
|
||||
// Join the render thread first (it checks g_open every round and at most every 10 ms while waiting for the output): it may be inside exec(), which uses the renderer the SDK is about to free.
|
||||
|
||||
+18
-12
@@ -1,5 +1,6 @@
|
||||
// Virtual gamepad (Nintendo button positions) + physical controllers (GameController). Both are merged and handed to the host through dm_input_set (hle_hid.c, bit layout in dm_input.h).
|
||||
// Two layouts; one tap on the layout button at the top centre (labelled with the current layout, "2D" / "3D") switches them, and the choice is kept in NSUserDefaults (each game is its own app, so per game); the first launch uses the game profile's pad_layout (DM_GAME_PAD_FULL).
|
||||
// Two layouts, three in handheld mode; one tap on the layout button at the top centre (labelled with the current layout, "2D" / "3D" / "Tap") goes to the next, and the choice is kept in NSUserDefaults (each game is its own app, so per game); the first launch uses the game profile's pad_layout (DM_GAME_PAD_LAYOUT).
|
||||
// Tap (touch): no control but the layout button, every other touch goes to the game's touch screen; offered only while the game is in handheld mode, the only one in which it reads the touch screen (Balatro, played by touch).
|
||||
// 2D (simple): top corners ZL above L (one column), ZR above R likewise, teardrops with the tip toward the screen corner; − and + side by side at the top centre; the left stick in the bottom-left corner (fixed ring, tilt measured from the ring centre, small dead zone);
|
||||
// the A (right) B (bottom) X (top) Y (left) diamond in the bottom-right corner, sliding a finger between keys switches keys. Each bottom corner holds a single control, so it sits close in (reaching halfway into the side safe-area inset).
|
||||
// 3D (full, Pro Controller style): everything in 2D (left stick and diamond keep their place and size), plus a D-pad (inward of and below the left stick, 8-way), a right stick that is not drawn
|
||||
@@ -88,7 +89,8 @@ static const CGFloat
|
||||
K_RSWIPE = 1.0 / 3, // right stick area: this fraction of the screen width from the right edge (not in u)
|
||||
K_CLICK_X = 102, K_CLICK_R = 17; // L3 / R3: centre from the outer edge (level with ZL / ZR), radius
|
||||
static const int k_btns[] = {B_ZL, B_L, B_ZR, B_R, B_MINUS, B_PLUS, B_LS, B_RS}; // keys locked at touch-down (L3 / R3 only in the 3D layout)
|
||||
static NSString *const k_layout_key = @"DmPadLayout"; // NSUserDefaults: 1 = 3D, 0 = 2D
|
||||
static NSString *const k_layout_key = @"DmPadLayout"; // NSUserDefaults: 0 = 2D, 1 = 3D, 2 = Tap (DM_GAME_PAD_LAYOUT's numbers)
|
||||
int dm_opmode(void); // hle_oe.c: 0 = handheld
|
||||
|
||||
@interface DmGamepad : UIView
|
||||
@end
|
||||
@@ -104,7 +106,7 @@ static __weak DmGamepad *g_view;
|
||||
CAShapeLayer *_ring, *_knob; // the left stick (the right one is not drawn)
|
||||
CAShapeLayer *_swap; CATextLayer *_swapLabel; BOOL _swapHeld;
|
||||
BOOL _stickShown; // the left stick was held at the last refresh
|
||||
BOOL _full; // the 3D layout is current
|
||||
BOOL _full, _tap; // the 3D layout / the Tap layout is current
|
||||
CALayer *_controls; // holds every control layer: hidden while a physical controller is connected (the view itself stays, for game touches)
|
||||
BOOL _padOff; // a physical controller is connected: every touch goes to the game
|
||||
UITouch *_game[16]; // touches on the game's touch screen by finger slot
|
||||
@@ -146,7 +148,9 @@ static BOOL fullOnly(int b) { return b == B_LS || b == B_RS || b >= B_LEFT; }
|
||||
}
|
||||
_swap = shape(_controls); _swapLabel = label(_swap, @"");
|
||||
NSNumber *saved = [NSUserDefaults.standardUserDefaults objectForKey:k_layout_key];
|
||||
_full = saved ? saved.boolValue : DM_GAME_PAD_FULL;
|
||||
int l = saved ? (int)saved.integerValue : DM_GAME_PAD_LAYOUT;
|
||||
if (l == 2 && dm_opmode() != 0) l = 0; // Tap only in handheld mode (env.txt's DM_OPMODE may have changed it)
|
||||
_full = l == 1; _tap = l == 2;
|
||||
[self applyLayout];
|
||||
NSNotificationCenter *nc = NSNotificationCenter.defaultCenter;
|
||||
[nc addObserver:self selector:@selector(padsChanged) name:GCControllerDidConnectNotification object:nil];
|
||||
@@ -158,15 +162,16 @@ static BOOL fullOnly(int b) { return b == B_LS || b == B_RS || b >= B_LEFT; }
|
||||
}
|
||||
- (void)applyLayout { // show / hide the controls for the current layout
|
||||
[CATransaction begin]; [CATransaction setDisableActions:YES];
|
||||
for (int b = 0; b < B_N; b++) _key[b].hidden = fullOnly(b) && !_full;
|
||||
_swapLabel.string = _full ? @"3D" : @"2D";
|
||||
for (int b = 0; b < B_N; b++) _key[b].hidden = _tap || (fullOnly(b) && !_full);
|
||||
_ring.hidden = _knob.hidden = _tap;
|
||||
_swapLabel.string = _tap ? @"Tap" : _full ? @"3D" : @"2D";
|
||||
[CATransaction commit];
|
||||
}
|
||||
- (void)setFull:(BOOL)full { // switch layouts: release every touch first (no key stays held), save, lay out again
|
||||
if (full == _full) return;
|
||||
- (void)nextLayout { // 2D → 3D (→ Tap in handheld mode) → 2D: release every touch first (no key stays held), save, lay out again
|
||||
int l = _tap ? 0 : _full ? (dm_opmode() == 0 ? 2 : 0) : 1;
|
||||
[self reset];
|
||||
_full = full;
|
||||
[NSUserDefaults.standardUserDefaults setObject:@(full) forKey:k_layout_key];
|
||||
_full = l == 1; _tap = l == 2;
|
||||
[NSUserDefaults.standardUserDefaults setObject:@(l) forKey:k_layout_key];
|
||||
[self applyLayout]; [self setNeedsLayout];
|
||||
}
|
||||
|
||||
@@ -278,9 +283,10 @@ static CGFloat dist(CGPoint a, CGPoint b) { return hypot(a.x - b.x, a.y - b.y);
|
||||
BOOL game = NO;
|
||||
for (UITouch *t in ts) {
|
||||
if (_padOff) { game |= [self gameBegin:t]; continue; }
|
||||
CGPoint p = [t locationInView:self]; int hit = [self btnAt:p];
|
||||
CGPoint p = [t locationInView:self]; int hit = _tap ? -1 : [self btnAt:p];
|
||||
if (hit >= 0) { [_owner setObject:@(C_BTN | (uint64_t)hit << 8) forKey:t]; continue; }
|
||||
if ([self inSwap:p]) { [_owner setObject:@(C_SWAP) forKey:t]; continue; }
|
||||
if (_tap) { game |= [self gameBegin:t]; continue; }
|
||||
// diamond, D-pad, left stick: take the one with the smallest distance / grab radius, if it is <= 1
|
||||
CGFloat best = 1; int kind = C_NONE, idx = 0;
|
||||
CGFloat f = dist(p, self.faceCenter) / (self.faceGap + self.faceR * 1.6); if (f <= best) { best = f; kind = C_FACE; }
|
||||
@@ -318,7 +324,7 @@ static CGFloat dist(CGPoint a, CGPoint b) { return hypot(a.x - b.x, a.y - b.y);
|
||||
}
|
||||
if (game) [self pushGame];
|
||||
[self recompute];
|
||||
if (swap) [self setFull:!_full];
|
||||
if (swap) [self nextLayout];
|
||||
}
|
||||
- (void)touchesEnded:(NSSet<UITouch *> *)ts withEvent:(UIEvent *)e { [self endTouches:ts tapped:YES]; }
|
||||
- (void)touchesCancelled:(NSSet<UITouch *> *)ts withEvent:(UIEvent *)e { [self endTouches:ts tapped:NO]; }
|
||||
|
||||
@@ -41,7 +41,7 @@ class XcodePipelineTests(unittest.TestCase):
|
||||
profile = SimpleNamespace(id=registered.id, dir=root / 'profile' / registered.id,
|
||||
get=registered.get, words=registered.words,
|
||||
builds=lambda: {'v0': {'display': '1.0.0', 'main': 'other-build'}})
|
||||
sources = {key: (False, '') for key in ('glslc_capture', 'vk_capture', 'ue_decode', 'oodle_packs', 'ue_legacy')}
|
||||
sources = {key: (False, '') for key in ('glslc_capture', 'vk_capture', 'gl_capture', 'ue_decode', 'oodle_packs', 'ue_legacy')}
|
||||
sources['graphics'] = (graphics, 'fixture')
|
||||
fam = {'base': base, 'updates': [] if derived else [(327680, update)], 'dlc': []}
|
||||
version = 'v0' if derived else 'v327680'
|
||||
|
||||
+92
-13
@@ -28,6 +28,84 @@ def addimm(w): return (w & 31, (w >> 5) & 31, (w >> 10) & 0xfff) if (w & 0xffc00
|
||||
def ldr64(w): return (w & 31, (w >> 5) & 31, ((w >> 10) & 0xfff) * 8) if (w & 0xffc00000) == 0xf9400000 else None
|
||||
def is_call(w): return (w & 0xfc000000) == 0x94000000 or (w & 0xfffffc1f) == 0xd63f0000 # bl, blr
|
||||
def is_str_x0(w, base): return (w & 0xffc003ff) == 0xf9000000 | (base << 5) and ((w >> 10) & 0xfff) == 0 # str x0, [base]
|
||||
def str_x0(w): return ((w >> 5) & 31, ((w >> 10) & 0xfff) * 8) if (w & 0xffc0001f) == 0xf9000000 else None # str x0, [base, #off] → (base, off)
|
||||
|
||||
def got_values(img):
|
||||
"""{address: the address it holds once relocated} for the image's relative relocations (RELA R_AARCH64_RELATIVE, and RELR, whose addend is in place)"""
|
||||
out = {}
|
||||
mod0 = struct.unpack_from('<I', img, 4)[0]
|
||||
if img[mod0:mod0 + 4] != b'MOD0': return out
|
||||
dyn = mod0 + struct.unpack_from('<i', img, mod0 + 4)[0]; tags = {}
|
||||
for k in range(4096):
|
||||
t, v = struct.unpack_from('<qQ', img, dyn + k * 16)
|
||||
if t == 0: break
|
||||
tags.setdefault(t, v)
|
||||
for k in range(tags.get(8, 0) // 24): # DT_RELA / DT_RELASZ: RELATIVE, and ABS64 / GLOB_DAT of a symbol the module defines (Balatro's glad pointers are exported)
|
||||
off, info, add = struct.unpack_from('<QQq', img, tags[7] + k * 24)
|
||||
if info & 0xffffffff == 1027: out[off] = add
|
||||
elif info & 0xffffffff in (257, 1025) and 6 in tags:
|
||||
shndx, value = struct.unpack_from('<HQ', img, tags[6] + (info >> 32) * 24 + 6)
|
||||
if shndx: out[off] = value + add
|
||||
where = 0
|
||||
for k in range(tags.get(35, 0) // 8): # DT_RELR / DT_RELRSZ
|
||||
e = struct.unpack_from('<Q', img, tags[36] + k * 8)[0]
|
||||
if not e & 1: where = e; out[where] = struct.unpack_from('<Q', img, where)[0]; where += 8
|
||||
else:
|
||||
for b in range(63):
|
||||
if e >> (b + 1) & 1: a = where + b * 8; out[a] = struct.unpack_from('<Q', img, a)[0]
|
||||
where += 63 * 8
|
||||
return out
|
||||
|
||||
def known_loads(words, got=None):
|
||||
"""(index, address) of every 64-bit load whose base register holds a known address: adrp, adds to it, mov and csel of known ones, followed in a straight line (a return forgets
|
||||
them all; a call or a branch keeps only x19–x28, which a function's hoisted pointers stay in; any instruction that may write a register forgets that one); a load from a slot in
|
||||
got (address → the address it holds) gives its register that address. Digimon reaches a function through its GOT slot (adrp + ldr); LÖVE's glad keeps its pointers in globals
|
||||
that the code reaches through GOT entries and keeps in registers (Balatro: adrp + ldr x22 of the GOT entry, ldr x8, [x22], blr x8)"""
|
||||
got = got or {}
|
||||
val = {} # register → the addresses it may hold (two after a csel of known ones: LÖVE picks glEnable or glDisable, glEnableVertexAttribArray or glDisableVertexAttribArray so)
|
||||
for i, w in enumerate(words):
|
||||
r = adrp(w, i * 4)
|
||||
if r: val[r[0]] = (r[1],); continue
|
||||
a = addimm(w)
|
||||
if a:
|
||||
if a[1] in val: val[a[0]] = tuple(x + a[2] for x in val[a[1]])
|
||||
else: val.pop(a[0], None)
|
||||
continue
|
||||
l = ldr64(w)
|
||||
if l:
|
||||
ats = tuple(x + l[2] for x in val.get(l[1], ()))
|
||||
for at in ats: yield i, at
|
||||
held = tuple(got[at] for at in ats if at in got) # a GOT slot holds a global's address: the next load off it reads the global
|
||||
if held: val[l[0]] = held
|
||||
else: val.pop(l[0], None)
|
||||
continue
|
||||
mov, csel = (w & 0xffe0ffe0) == 0xaa0003e0, (w & 0xffe00c00) == 0x9a800000
|
||||
if mov or csel: # mov xd, xm; csel xd, xn, xm
|
||||
src = [(w >> 16) & 31] + ([(w >> 5) & 31] if csel else [])
|
||||
if all(k in val for k in src): val[w & 31] = tuple(x for k in src for x in val[k])
|
||||
else: val.pop(w & 31, None)
|
||||
continue
|
||||
if (w & 0xfc000000) == 0x94000000 or (w & 0xfffffc1f) == 0xd63f0000: # bl, blr: x0–x18 and x30 are the callee's
|
||||
for k in [*range(19), 30]: val.pop(k, None)
|
||||
continue
|
||||
if (w & 0xfc000000) == 0x14000000: # b: the next block is reached from elsewhere, where x19–x28 still hold what the function hoisted into them (a loop's GL pointers)
|
||||
for k in [*range(19), 29, 30]: val.pop(k, None)
|
||||
continue
|
||||
if w == 0xd65f03c0 or (w & 0xfffffc1f) == 0xd61f0000: # ret, br: the function ends
|
||||
val.clear(); continue
|
||||
if (w & 0x0a000000) == 0x08000000: # loads and stores
|
||||
if (w & 0x3a000000) == 0x28000000: # pairs: a load writes both, pre / post index writes the base
|
||||
if (w >> 22) & 1: val.pop(w & 31, None); val.pop((w >> 10) & 31, None)
|
||||
if (w >> 23) & 1 or not (w >> 24) & 1: val.pop((w >> 5) & 31, None)
|
||||
else:
|
||||
if (w & 0x3b000000) == 0x39000000: load = (w >> 22) & 3 != 0 # unsigned offset
|
||||
else:
|
||||
load = (w >> 22) & 3 != 0 or (w & 0x3f000000) in (0x08000000, 0x18000000, 0x58000000, 0x98000000)
|
||||
if (w & 0x3b200000) == 0x38000000 and (w >> 10) & 1: val.pop((w >> 5) & 31, None) # pre / post index
|
||||
if load: val.pop(w & 31, None)
|
||||
continue
|
||||
if (w & 0x1c000000) == 0x14000000 and (w & 0xfff00000) != 0xd5300000: continue # branches, compares, system (but mrs) write no register
|
||||
val.pop(w & 31, None) # data processing and the rest: Rd
|
||||
|
||||
def main():
|
||||
a = sys.argv[1:]
|
||||
@@ -40,27 +118,28 @@ def main():
|
||||
e = img.find(b'\0', p, p + 256)
|
||||
try: return img[p:e].decode('ascii') if e > p else None
|
||||
except UnicodeDecodeError: return None
|
||||
slot, loader = {}, set()
|
||||
slot, loader, page = {}, set(), {}
|
||||
for i in range(4, n - 3):
|
||||
r = adrp(words[i], i * 4); a = addimm(words[i])
|
||||
if r: page[r[0]] = r[1] # the last address each register was given by adrp (+ add): LÖVE's glad keeps a few globals' addresses in x19–x28 across its loader
|
||||
elif a and a[1] in page: page[a[0]] = page[a[1]] + a[2]
|
||||
if not is_call(words[i]): continue
|
||||
name = None
|
||||
for k in range(i - 4, i - 1): # the name: adrp + add into x0 or x1 just before the call
|
||||
r = adrp(words[k], k * 4); s = addimm(words[k + 1])
|
||||
if r and s and r[0] in (0, 1) and s[1] == r[0] and s[0] == r[0]: name = cstr(r[1] + s[2])
|
||||
r = adrp(words[i + 1], (i + 1) * 4); l = ldr64(words[i + 2])
|
||||
if not (name and r and l and l[1] == r[0] and is_str_x0(words[i + 3], l[0])): continue
|
||||
slot[r[1] + l[2]] = name
|
||||
if not name: continue
|
||||
r = adrp(words[i + 1], (i + 1) * 4); l = ldr64(words[i + 2]); s = str_x0(words[i + 1]); s2 = str_x0(words[i + 2])
|
||||
if r and l and l[1] == r[0] and is_str_x0(words[i + 3], l[0]): at = r[1] + l[2] # through the GOT: adrp + ldr of the slot's address, str x0 to it (Digimon)
|
||||
elif r and s2 and s2[0] == r[0]: at = r[1] + s2[1] # straight into the global: adrp + str x0, [page, #off] (LÖVE's glad: Balatro)
|
||||
elif s and s[0] in page and s[0] not in (0, 1): at = page[s[0]] + s[1] # str x0, [base, #off], base a page set earlier
|
||||
else: continue
|
||||
if name not in slot.setdefault(at, []): slot[at].append(name) # a slot two names were stored into keeps both
|
||||
loader.update(range((i - 4) * 4, (i + 4) * 4, 4))
|
||||
slot = {at: ' / '.join(sorted(names, key=len)) for at, names in slot.items()}
|
||||
refs, sites = collections.Counter(), collections.defaultdict(list)
|
||||
for i in range(n - 1):
|
||||
r = adrp(words[i], i * 4)
|
||||
if not r or i * 4 in loader: continue
|
||||
for j in range(i + 1, min(i + 6, n)):
|
||||
l = ldr64(words[j])
|
||||
if l and l[1] == r[0]:
|
||||
s = r[1] + l[2]
|
||||
if s in slot: refs[slot[s]] += 1; sites[slot[s]].append(i * 4)
|
||||
break
|
||||
for i, at in known_loads(words, got_values(img)):
|
||||
if at in slot and i * 4 not in loader: refs[slot[at]] += 1; sites[slot[at]].append(i * 4)
|
||||
fams = collections.Counter(nm[:2] if nm.startswith('gl') else nm[:3] for nm in slot.values())
|
||||
print(f'{len(slot)} slots stored by a loader ({", ".join(f"{k}* {v}" for k, v in fams.most_common(6))}); {len(refs)} loaded by other code:')
|
||||
for nm in sorted(refs): print(f' {refs[nm]:4d} {nm}')
|
||||
|
||||
@@ -48,7 +48,7 @@ The tables below are what to check by hand afterwards.
|
||||
|
||||
## NVN surface
|
||||
|
||||
- First check that the game draws with NVN at all. Importing `nvnBootstrapLoader` is not enough: Digimon Story Cyber Sleuth: Complete Edition imports it and EGL, makes an NVN device for a helper beside its movie player, and renders everything with NVIDIA's desktop OpenGL (`eglBindAPI(EGL_OPENGL_API)`, a glad loader asking `eglGetProcAddress` for 2,956 names). Read the graphics setup that calls `nvnBootstrapLoader` / `eglGetDisplay` / `vkCreateInstance`, and count which loaded entry points the code really reaches: for a glad or NVN C loader that stores each pointer in a global, the GOT slots of those globals that code outside the loader loads (Digimon: 79 GL functions, 30 NVN ones, all in that helper). `profile_guess.py` decides `graphics gl` when `main` imports `eglCreateWindowSurface` and names `glShaderSource`; the host's GL layer is `dm/gfx/gl/hle_gl.m` ([04](04-graphics.md)).
|
||||
- First check that the game draws with NVN at all. Importing `nvnBootstrapLoader` is not enough: Digimon Story Cyber Sleuth: Complete Edition imports it and EGL, makes an NVN device for a helper beside its movie player, and renders everything with NVIDIA's desktop OpenGL (`eglBindAPI(EGL_OPENGL_API)`, a glad loader asking `eglGetProcAddress` for 2,956 names). Read the graphics setup that calls `nvnBootstrapLoader` / `eglGetDisplay` / `vkCreateInstance`, and count which loaded entry points the code really reaches: for a glad or NVN C loader that stores each pointer in a global, the GOT slots of those globals that code outside the loader loads (Digimon: 79 GL functions, 30 NVN ones, all in that helper; Balatro's LÖVE: about 150 GL functions, whose globals its code reaches through registers, which `dm/tools/loader_census.py` follows). `profile_guess.py` decides `graphics gl` when `main` imports `eglCreateWindowSurface` and names `glShaderSource`; the host's GL layer is `dm/gfx/gl/hle_gl.m` ([04](04-graphics.md)).
|
||||
- Find how the game obtains NVN entry points. Some games write the slots with `adrp`-to-slot code; Mario's loader (`main+0x1070910`) calls `GetProcAddress` per name and stores through the GOT (`dm/tools/nvn_slots_got.py <main.bin> <loader> <text end> <out tsv>`). The output lists every fetched function and every call site.
|
||||
- Mario fetched 534 functions and called 236 of them; 79 were new to the host (CommandBuffer 31, Texture 15, Buffer 10, Sampler 10). Names the host doesn't know at all go into `dm/gfx/nvn/names.txt` → `gen_protos.py`.
|
||||
|
||||
|
||||
@@ -202,6 +202,11 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **SDK 6.4.0 (Digimon Story Cyber Sleuth: Complete Edition, Media.Vision's MVGL engine on OpenGL), one boot each**:
|
||||
- **`UNIMPL eglGetDisplay`, then `Break` right after `nv::InitializeGraphics` and `vi::OpenDefaultDisplay`** (`main+0x5ed9d0`) → the game renders with NVIDIA's desktop OpenGL through EGL; its NVN device (made just before) serves only a helper beside the movie player. The host's GL layer (`dm/gfx/gl/hle_gl.m`) answers the EGL imports and `eglGetProcAddress` ([04](04-graphics.md)).
|
||||
- **SIGBUS in `__vfprintf` under `aoc_load`, from `h_aoc_Count`** (a store to `0x…67ff0`, right below a page of the guest thread's stack) → the first add-on content query lists `DM_AOC_DIR` and logs, a few KB of stack, and the game asks from a thread with little left. `aoc::CountAddOnContent` and `ListAddOnContent` run on the big host stack (`HLE_FORCE_BIG`, `hle_misc.c`).
|
||||
- **SDK 18.3.2 (Balatro, LÖVE 11.4 on OpenGL with SDL and LuaJIT), one boot each**:
|
||||
- **The NSP script's provisional profile said Vulkan, and the capture boot crashed in the GL layer** → `main` imports `vkGetInstanceProcAddr` for SDL's Vulkan loader, which nothing uses; the game draws with EGL and GL. `profile_guess.py` now leaves that import out of its Vulkan count ([04](04-graphics.md)).
|
||||
- **A jump to address 0 right after `glGetFloatv(0xb12)`** (`lr=main+0x2d58c4`) → a GL function the layer did not have (here `glVertexAttrib4fv`, LÖVE's default colours). `DM_GL_STUBS=1` hands out a stub per missing name that reports its first call; `loader_census.py` lists what the code calls before the first boot.
|
||||
- **A crash in `main+0x339c00` a second or two into the title, in love-nx's video module teardown** (`love::video::theora::Video` → `~Worker`; its thread released the worker that the closing module deletes too) → the Lua state was closing: on the Switch, Balatro's `love.errhand` returns at once, so any Lua error quits. A `print` at the top of `love.errhand` in a copy of the RomFS (`DM_ROMFS=<copy>`) gave the error: `Shader uniform 'distortion_fac' does not exist` (the capture run had no shader library, so every program linked with no uniforms), then `bad argument #2 to 'send' (number expected, got boolean)` (a `bool` uniform reported as `GL_UNSIGNED_INT`). Probing the destructors (`DM_PROBE`) and logging every object the Lua GC released (`DM_PROBE=main:0x18b920@x0+0`: the module proxies all at once) showed it was a shutdown, not a premature collection.
|
||||
- **Silent** (the log has `audren: OpenAudioRenderer (no event)` and no `render thread started`) → see [Audio](#audio).
|
||||
- **SDK 8 (Link's Awakening, 8.2.0)**:
|
||||
- **Abort in `os::WaitSystemEvent` under `oe::PopNotificationMessage`** → the blocking pop waits on the system event `oe::Initialize` would have created, and the host's `Initialize` is a no-op. The host now implements it on its notification queue (a condition variable, then the `TryPop` path).
|
||||
- **SIGSEGV in `vi::SetLayerCrop`** → the SDK's version dereferences vi service objects the host never made; `SetLayerCrop`, `DestroyLayer`, `CloseDisplay` and `vi::Finalize` are host no-ops.
|
||||
@@ -284,6 +289,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **The CPU processor drops the final mix.** `dsp::SendCommandBuffer`, `dsp::Wait` and `dsp::Signal` are stubs in the application build, and DeviceSink (command 0x13) is a no-op in `CommandListProcessor::Process`. Take the output after `Process` from the mix buffers the DeviceSink command names (layout in `hle_audren.c`).
|
||||
- **Generation needs the renderer's lock.** `AudioRenderSystem::Update` holds a mutex in `AudioRenderSystem`; `GenerateCommand` does not take it, so the host does (the auto-mode system thread does the same in `SendCommandToDsp`). Which mutex depends on the SDK: `nn::os::LockMutex(MutexType *)` at +0x1d0 in SDK 4.4 and at +0x1d8 in 6.4–8.2, `SdkMutexType::Lock` at +0x1d8 in 10.4–11.4, +0x1f0 in 13.3–16.2 and +0x1f8 in 17.5 (the `sdk` builds of 24 games on hand), inlined in 19 and 20 (+0x2d0, the Span layout above). The host took SDK 16's for every SDK before 20, which wrote a lock word into another field and excluded nothing; Mario Kart 8 Deluxe (4.4) was the first game it was plainly wrong for. `update_lock` in `hle_audren.c` reads it off the SDK's own `Update`: the register the prologue adds the offset to and moves into x0 before its first `bl`, whose PLT slot must hold `LockMutex` or `SdkMutexType::Lock`; anything else keeps SDK 16's. Checked with sound on Big Brain Academy (11.4, +0x1d8), DRAGON QUEST XI S (7.3, `MutexType` +0x1d8) and Super Mario Odyssey (7.3, which is silent for its first minute with either lock).
|
||||
- **At least one rendered frame between two updates.** An effect's "needs init" parameter state is set only in the update where the game flags it new; the next update copies the game's value over it. Mario updates twice in a row at start-up; with a real audio device the first frame came after both, and the light limiter later crashed on a null state pointer (`ApplyLightLimiterEffect`, `sdk+0x228494`). The `AudioRenderSystem::Update` hook renders a catch-up frame first when nothing was rendered since the last update. `DM_AUDREN_TRACE=1` logs every update with the number of frames rendered since the previous one.
|
||||
- **A renderer opened without an event** (Balatro: LÖVE's OpenAL Soft on `OpenAudioRenderer(handle, parameter, work, size)`, the form with no system event). The game paces its own updates, and the renderer must still run every 5 ms; the host used to start its render thread only for the form with an event, so only the game's updates rendered (catch-up frames that nothing played) and the game was silent. Both forms now go through one open that starts the thread (`hle_audren.c` `ar_open`).
|
||||
- **A sound driver that never waits on the renderer event** (DRAGON QUEST XI S, Square Enix's `SQEX::Sd::Driver::Core::CoreAudioOut`, its loop at `main+0x442c5c0`). It appends a wave buffer, mixes the next block on the same thread, calls `RequestUpdate` and takes the next released buffer. It waits on the renderer event only when no buffer is released. Every update there finds no frame rendered since the last one and renders a catch-up frame, which releases the buffer just appended, so the loop never waits. When a full catch-up queue discarded the frame, the loop ran 11,000 updates a second on one core, the sound ran about 50 times faster than the output. An update with a full queue now waits (at most 50 ms) for the render thread to play one queued frame, so catch-up frames keep real time. The render thread then only plays queued frames, and the driver's updates render every frame. A queued frame does not signal the renderer event (the SDK signals it in each render): signalled, an ordinary game woke, updated with nothing rendered since and rendered the next catch-up frame, so every frame became one (Big Brain Academy: Brain vs. Brain, about 38,000 catch-up renders instead of 37). `audren: rendered 12000 frames, N of them not silent` shows whether a game's sound reaches the output.
|
||||
- **Crackling on iOS only** → RemoteIO asks for as many frames per callback as the audio session's I/O buffer duration, about 21 ms by default (1024 frames at 48 kHz), while the renderer keeps 4 blocks × 5 ms = 20 ms in flight. Every callback came up 1.3 ms short and was padded with zeros, about 47 gaps per second. Reproduced on the Mac by forcing 1024-frame callbacks (`DM_DBG_AU_FRAMES=1024` with `DM_AUDIO=1`: 1999 underruns in 2000 callbacks). Fix: `AVAudioSession` `setPreferredIOBufferDuration:0.005` (and 48 kHz) before the game opens audio and again on return to the foreground (`front_ios_audio_session` in `front_ios.m`); the device then calls back with 240 frames and nothing underruns. When checking a new device, read the host's `audio #N device callbacks` line, logged every 2000 callbacks: frames per callback, queued blocks, underruns.
|
||||
- **Several outputs at once.** Engage opens one AudioOut from Wwise (`main+0x177d924`, no release event, the game polls). On the phone it opened a second one later; the Mac runs never did. Unity's FMOD also has an output (`main+0x4aec0c` / `0x4aec4c`). The host logs each output as `audio #N`: the open with its call site, Start / Stop / Close (the first 32 per output), and the callback line every 2000 of that output's callbacks. Combined numbers mislead: one fully fed output plus one fed by nobody look like "every other callback underruns, 2.0 blocks queued".
|
||||
|
||||
@@ -403,12 +403,25 @@ Media.Vision's MVGL engine on SDK 6.4.0 renders with **NVIDIA's desktop OpenGL 4
|
||||
- **Shaders are GLSL ES 3.20 source**, compiled at run time with `glShaderSource` / `glCompileShader`: 1,682 `.shad` files in the MVGL archives (615 distinct) and 37 strings in `main` (post effects, FXAA, shadows, the movie's YUV). `gl_shaders.py` translates them offline: glslang (`-V -R`: relaxed Vulkan rules gather the loose uniforms into one `gl_DefaultUniformBlock`, whose member offsets the table lists) → SPIR-V → SPIRV-Cross. The host finds each shader by the hash of its text and its stage. Varyings are matched by name, as GL links them: `[[user(locnN)]]` becomes `[[user(<name>)]]`, so a vertex and a fragment shader translated apart still meet. `--fixup-clipspace` maps GL's clip z to Metal's.
|
||||
- **What NVIDIA's compiler accepts and glslang does not** (fix-ups on the preprocessed text): the GLSL ES 1.0 functions `textureCube` / `texture2D` in ES 3.20 code (89 fragment shaders; with `-V` glslang reads `textureCube` as Vulkan's texture type, so a macro cannot rename it), `out vec4 gl_Position` redeclared through a macro (the built-in sprite shader), and `textureOffset` with an offset that is not a constant (FXAA), which becomes a plain sample at P + offset ÷ level 0 size. A string in `main` says nothing of its stage, so it is translated as each stage that compiles (one fragment shader defines `vPosition` as `gl_Position`).
|
||||
- **Orientation**: the layer renders as GL does, bottom row first, through the NVN layer's lower-left origin (`mtl_set_origin_lower_left`): flipped viewports, reversed winding, a flipped present. Scissor rectangles and `gl_FragCoord` need no change, and render targets read back with GL's texture coordinates.
|
||||
- **State**: uniforms are CPU copies per stage, set as bytes at a draw after they change; pipelines are cached per program by vertex layout, attachment formats, blend and colour mask; disabled vertex arrays read GL's default (0, 0, 0, 1) through a constant-step buffer. The game's attribute index is the Metal one: `glGetAttribLocation` returns the shader's own attribute numbers.
|
||||
- **State**: uniforms are CPU copies per stage, set as bytes at a draw after they change; pipelines are cached per program by vertex layout, attachment formats, blend and colour mask; disabled vertex arrays read the attribute's current value (GL's default (0, 0, 0, 1) until `glVertexAttrib4f`) through a constant-step buffer. Without `glBindAttribLocation` the game's attribute index is the Metal one: `glGetAttribLocation` returns the shader's own attribute numbers (Balatro below binds its own).
|
||||
- **Textures** are made on first use from the levels given so far (the loader defines them level by level). An upload into a texture a pending command buffer may still read is a blit in command order; any other is a CPU write. Metal reads a channel a format lacks as 0 (alpha 1) as GL does, so only luminance / alpha formats get a swizzle: Metal refuses a swizzle on a texture that can be a render target (`Texture swizzling is not compatable with MTLTextureUsageRenderTarget`, the first story scene).
|
||||
- **Buffers**: `glBufferData` on a buffer the GPU may still read gets a new `MTLBuffer`, as GL drivers orphan it. **Clears**: a whole-framebuffer clear becomes the next pass's load action; a scissored or masked one is a quad.
|
||||
- **Present**: three window textures through the NVN layer's present path (`mtl_ext_tex`, as the Vulkan swapchain), paced by the swap interval in display-link ticks. The game sets the interval to 60 ÷ its frame rate before each swap: 1 at the title and in the DigiLab, 2 from the story's opening on.
|
||||
- **Movies**: the USM movies' H.264 is decoded through the SDK's movie library (the host's decoder) into NV12 buffers (3,110,400 bytes a 1920x1080 frame); the game uploads the planes into a `GL_RED` and a `GL_RG` texture (two `glTexSubImage2D` a frame) and converts YUV in its own shader (in `main`, `SamplerY` / `SamplerUV`).
|
||||
|
||||
### Seen in Balatro (LÖVE 11.4 on OpenGL)
|
||||
|
||||
LÖVE ("love-nx", with SDL and LuaJIT) renders with the same NVIDIA desktop GL 4.5 through EGL; its graphics module is LÖVE's own OpenGL backend, so its choices follow what the context reports.
|
||||
|
||||
- **Telling the API apart**: `main` imports the EGL functions and `vkGetInstanceProcAddr`, SDL's Vulkan loader, which nothing calls (`subsdk1` is NVIDIA's Vulkan driver). `profile_guess.py` leaves `vkGetInstanceProcAddr` out of its Vulkan count: Hades and both Galaxies import 99 or 100 Vulkan functions, `vkCreateInstance` among them.
|
||||
- **Which GL functions**: LÖVE's glad stores each pointer straight into a global of its own (no GOT slot per name), and the code reaches those globals through GOT entries that hold their addresses, which it often keeps in callee-saved registers across a loop, or picks between with `csel` (`glEnableVertexAttribArray` or `glDisableVertexAttribArray`). `loader_census.py` follows registers in a straight line for that (adrp, add, loads through relocated GOT entries, `mov`, `csel`): 2,133 names asked, about 150 called. `DM_GL_STUBS=1` gives every name the layer lacks a stub of its own that reports its first call instead of a null pointer (a jump to address 0 names nothing).
|
||||
- **Core profile**: LÖVE reads `GL_CONTEXT_PROFILE_MASK`; on a core profile it streams vertices through persistently mapped buffers (`glBufferStorage` + `glMapBufferRange` of a 3-frame ring, a `glFenceSync` per frame and `glClientWaitSync` three frames later), on a compatibility one from client memory, which Metal cannot draw from. The layer answers core: the mapping is the shared `MTLBuffer`'s memory, a fence is the serial of the command buffer being recorded, and a wait on a fence of that command buffer commits it first. The shaders then come as `#version 330 core`.
|
||||
- **Attributes**: LÖVE binds VertexPosition, VertexTexCoord, VertexColor and ConstantColor to 0–3 with `glBindAttribLocation` before each link, keeps VertexColor and ConstantColor as disabled arrays and sets their values with `glVertexAttrib4f` (the draw colour); the layer maps each Metal input to the game's index at the link.
|
||||
- **Uniforms**: LÖVE lists them with `glGetActiveUniform` (types as GL enums, arrays as `name[0]`) and reads their initial values back with `glGetUniformfv` / `iv`. A GLSL `bool` is a `uint` in the SPIR-V block; `gl_shaders.py` keeps the declared type, as LÖVE's `Shader:send` takes only a boolean for a bool.
|
||||
- **Textures**: made with `glTexStorage2D` (every level up front: the game's images are mipmapped, filled by `glGenerateMipmap` as a blit) or `glTexStorage3D`: LÖVE makes a 1x1 default texture of each type, 3D and 2D array among them, and the layer's units hold all four targets (96 of them, as `GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS` says). Font glyphs are RG8 with a `GL_TEXTURE_SWIZZLE_*` of R, R, R, G, applied when the texture is made (a swizzled texture cannot be a render target).
|
||||
- **Draws and framebuffers**: batched quads share one 16-bit index buffer through `glDrawElementsBaseVertex`; canvases bind GL_FRAMEBUFFER (read and draw), check completeness, discard depth and stencil after use (`glInvalidateFramebuffer`) and clear with `glClearBufferfv`. `glBlitFramebuffer` (colour, same size) and `glReadPixels` (RGBA8) are there for multisample resolves and `Canvas:newImageData`, which the game does not use.
|
||||
- **Shaders are put together at run time**: LÖVE builds each source in Lua from its header, its built-in code and the game's `.fs` file; the 35 are all made before the first frame. The NSP script captures them on the Mac (`gl_capture.tsv`). A capture run has no library: every program then links with no uniforms, the game's first `Shader:send` raises a Lua error and, on the Switch, its error handler quits ([03](03-bringup.md)).
|
||||
|
||||
### Seen in DRAGON QUEST VII Reimagined
|
||||
|
||||
Unreal Engine 4.27's desktop renderer with forward shading (one RG11B10 colour pass draws every object, lit per pixel from the light grid, the volumetric lightmap and the screen-space shadow mask), HEXA DRIVE's toon lighting on top. 62,901 shaders, LZ4 in the shader code archives (section 1); the game has no PC version, so the facts table holds only compute group sizes.
|
||||
|
||||
+13
-2
@@ -58,7 +58,8 @@ What the script does:
|
||||
3. It rewrites the code modules, checking each build ID against the profile. A version the profile does not register (the base game alone when the profile was brought up on an update, another update, or files whose build IDs differ), or one whose shader tables need a version that is not in the folder, is built from a profile derived from it in `<id>_build/profile/<id>/`: the profile's keys, that version's build row, and shader tables, chain header, `c1_full.tsv` and `cb_extent.tsv` made from that version's own files (as for a provisional profile, without the heap check). The rewriter's cuts and the host's fixes for a registered build do not reach a module of another build; the script says which modules those are. A provisional profile (a game without one in `dm/games`) then gets `mem_used` from the rewritten modules, and, when the game has virtual address memory and sets a heap, a heap check: the Mac host is built into `<id>_build/tools/dmhost` and the game boots in it until its first heap request (a few seconds, log `<id>_build/logs/heap_check.log`), which sizes the heap region (see [dm/games/README.md](../dm/games/README.md), Adding a game).
|
||||
4. For a game whose `main` holds GLSL sources and calls `glslcCompile` (Three Houses), it captures the shaders the game compiles at run time, which no game file holds. It builds the Mac host into `<id>_build/tools/dmhost` and boots the game in it, headless and silent, with `DM_GLSLC_CAPTURE` pointing at `<id>_build/extract/<version>/glslc_capture`. The save, temporary storage and log all stay in `<id>_build/`. It stops the game as soon as every capture file named in the profile's shader map is there (Three Houses: 9, about a second after launch), and gives up after 30 s. A folder that already holds them all is reused.
|
||||
A Vulkan game without SPIR-V files in its RomFS (Super Mario Galaxy) gets the same kind of run: the game boots in the Mac host with `DM_VK_CAPTURE` pointing at `<id>_build/extract/<version>/vk_capture` until every SPIR-V key in the profile's `vk_capture.tsv` is there (Galaxy: 3,089, about 7 s after launch), and gives up after 120 s.
|
||||
5. It splits out the shaders and checks them against the profile's tables (a Vulkan profile: it translates the RomFS's SPIR-V and the capture with `vk_shaders.py` instead).
|
||||
So does an OpenGL game with no GLSL source in its RomFS or `main` (Balatro: LÖVE puts its shaders together at run time, in Lua): `DM_GL_CAPTURE` at `<id>_build/extract/<version>/gl_capture` until every file in the profile's `gl_capture.tsv` is there (Balatro: 35, made before its first frame). Without the profile's tables (a provisional profile) a capture stops once nothing has been added for 20 s.
|
||||
5. It splits out the shaders and checks them against the profile's tables (a Vulkan profile: it translates the RomFS's SPIR-V and the capture with `vk_shaders.py` instead; an OpenGL one: the GLSL sources and the capture with `gl_shaders.py`).
|
||||
6. It builds the iOS shader libraries, with the Mac's own Metal compiler and spirv-cross when they differ from those in `dmsh_chain.lock` (`dm/gfx/tools/own_compilers.py`; one line in the output names them). If a registered profile's `dmsh_chain.h` does not hold the version the source tree gives (a chain change left it behind), the script writes it again before any host is built, so it shows as changed in the checkout (`dmsh_build.py` would stop otherwise). Then the script makes the profile's `c1_full.tsv` and `cb_extent.tsv` from this build's Mac outputs when they are missing or name another chain. These two tables are not committed and git ignores them; `build_ipa.sh` stops without a current `c1_full.tsv`. Each line in the output names the file.
|
||||
7. It calls `build_ipa.sh`. Xcode builds the app in the game's own build folder (`<id>_build/ipa/build/ios/pkg/dd`: `build_ipa.sh` sets `SYMROOT` and `OBJROOT`, which a custom build location in Xcode's settings does not override), so a repeat build compiles only what changed and two games built at once never write the same app. Then `dm/ios/tools/pack_ipa.py` packs it with the shader libraries, the add-on content and the RomFS straight from where they are. It reads and deflates files on all cores and writes at about the T7's write speed (300 MB/s): Breath of the Wild's 16 GB IPA takes 57 s, Info-ZIP's copy and zip took 2 min 50 s.
|
||||
|
||||
@@ -632,6 +633,16 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
- **Host changes, all general:** the Mii database and the MiiModel stand-in; the add-on content cache sized as Horizon's RomFS working memory (its licence is decrypted in a 0x3130-byte heap); device save data as a save of its own for a game that mounts account save data too (every boot started a new save); SDK 4's DeviceSink layout and the audio renderer's update lock read off each SDK's `Update` (it was SDK 16's for every SDK before 20); and the zero cube for a one-layer 2D array in a cube slot ([03](agents/03-bringup.md), [04](agents/04-graphics.md)).
|
||||
- **Mac input:** the work folder's `run/ctl.sh` drives a run started with `DM_CTL`; `run/race.sh` boots from a copy of a save past the Mii pick into a 50cc Grand Prix (Mario, Mushroom Cup) with frame-timed keys and holds accelerate (`DM_KEYS=f960:L+R~0.3,f1260:A,f1500:A,f1740:A,f2000:A,f2240:A,f2400:A,f2560:A,f2800:A,f3040:A,f4300:A~70`). From a fresh save, L+R at the title opens "Select a Mii" (Right, A takes the first default Mii).
|
||||
|
||||
**Balatro (`0100CD801CE5E000`)**
|
||||
- **Status:** brought up on the Mac from update 1.1.3 (`v851968`): the intro, the title, a new run by touch and the tutorial's first steps, at 60 fps with sound; about 0.65 GB resident and 0.2 GB of Metal allocations, flat over three minutes in a run. Plays on the iPhone (IPA about 82 MB).
|
||||
- **Engine:** LÖVE 11.4 ("love-nx") with LuaJIT 2.1.0-beta3 in its interpreter and SDL, on SDK 18.3.2. The game is Lua source in the RomFS (`main.lua`, `game.lua`, `engine/`, `functions/`), its pictures PNG, its sound Ogg Vorbis through OpenAL Soft on the audio renderer. It renders with **OpenGL**: NVIDIA's desktop GL 4.5 through EGL (`subsdk0`, not loaded; `subsdk1` is NVIDIA's Vulkan driver, and `main` imports `vkGetInstanceProcAddr` only for SDL's Vulkan loader, which `profile_guess.py` now leaves out of its Vulkan count). LÖVE's glad loader asks for 2,133 GL names and its code calls about 150 of them (`loader_census.py`).
|
||||
- **GL layer:** LÖVE takes its core-profile paths when `GL_CONTEXT_PROFILE_MASK` says core (the layer now answers it): its vertex streams are persistently mapped buffers (`glBufferStorage`, `glMapBufferRange`) fenced per frame (`glFenceSync`, `glClientWaitSync`); with a compatibility profile it would draw from client memory. Also new for it: attribute bindings (`glBindAttribLocation`: LÖVE fixes VertexPosition, VertexTexCoord, VertexColor and ConstantColor to 0–3) and current attribute values (`glVertexAttrib4f`: VertexColor and ConstantColor are disabled arrays), `glGetActiveUniform` and the uniform read-backs, every `glUniform*` form, texture storage with its levels, 3D and array textures (LÖVE makes a default of each), texture swizzles (its font glyphs are RG8 read as R, R, R, G), `glGenerateMipmap` (the game's images are mipmapped), instanced and base-vertex draws, separate read and draw framebuffers, blits and `glReadPixels` ([04](agents/04-graphics.md)).
|
||||
- **Shaders:** LÖVE puts each shader's GLSL together at run time, in Lua, from its own header and built-in code and the game's 19 `resources/shaders/*.fs`: 35 sources (`#version 330 core`), all made by `Game:start_up` before the first frame. No game file holds them, so the NSP script captures them on the Mac (`gl_capture.tsv`) and `gl_shaders.py` translates them. A GLSL `bool` uniform is a `uint` in SPIR-V; `gl_shaders.py` now reports the declared type, since LÖVE's `Shader:send` takes only a boolean for a bool (the game stopped at its first card otherwise).
|
||||
- **Sound:** the game opens the audio renderer without an event, and the host started its render thread only for the form with one, so only the game's own updates rendered, and nothing played them: silent. Both forms now start it (`hle_audren.c`).
|
||||
- **Input and display:** `op_mode handheld`: the Switch build plays by touch there (SDL turns the touch screen into LÖVE touches and mouse presses), the iPhone's touches outside the gamepad controls, the Mac's mouse; docked it reads only the pads. Its window is the 1280x720 handheld display. `pad_layout touch`: the virtual gamepad starts in its new Tap layout (only the layout button; every touch goes to the game), since the 2D layout's corners cover Run Info, Options and the deck; the layout button brings back 2D and 3D.
|
||||
- **When the game stops:** on the Switch, Balatro's error handler quits at once, and love-nx's video worker is then deleted twice (its thread releases the worker that the closing video module also deletes), which crashes in `main+0x339c00`. A Mac run that reaches this has hit a Lua error first: a copy of the RomFS with a `print` at the top of `love.errhand` in `main.lua` shows it (a capture run without the shader library stops at the first `Shader:send` so, after every shader is captured).
|
||||
- **Mac input:** in handheld mode, `DM_KEYS="f1300:T0.35x0.86,f1700:T0.35x0.86,f2100:T0.5x0.68"` taps Play at the title, Play in the run setup and the tutorial's Next.
|
||||
|
||||
## Tools (`dm/tools`)
|
||||
|
||||
| Tool | What it does |
|
||||
@@ -643,4 +654,4 @@ The game's own work and its frame rate stay as they are; the host avoids work of
|
||||
| `nso_code.py` | Function ranges from `.eh_frame` and the data blocks in a text segment, which the rewriter and the census leave out |
|
||||
| `nso_syms.py` | Compares the SDK import surface of two games, or of one game (`-` for the reference) with the host hooks |
|
||||
| `nvn_slots_got.py` | Builds the NVN fetch table and call sites (GOT-indirect loaders) |
|
||||
| `loader_census.py` | Which of the functions a name loader fetches (glad's GL loader over `eglGetProcAddress`, NVN's C proc loader) the code really calls: the GOT slots the loader stores, loaded by other code |
|
||||
| `loader_census.py` | Which of the functions a name loader fetches (glad's GL loader over `eglGetProcAddress`, NVN's C proc loader) the code really calls: the GOT slots or globals the loader stores, loaded by other code (followed through registers, relocated GOT entries and `csel`, for LÖVE's glad) |
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 125 KiB |
@@ -509,12 +509,14 @@ def survey(nsp_dir, app, fam, keys, repo, py, env):
|
||||
G = pg.guess(F) # the provisional profile's keys (write_profile writes the same)
|
||||
keep = G['modules'][0].split() # how the build will get the shaders (decided again from the extraction at build time; the survey has only a RomFS manifest, so Unreal paks are guessed from main's markers)
|
||||
S = pg.shader_sources(mods['main']['imports'], F['glsl_src'], [m for m in keep if 'glslcCompile' in mods[m]['exports']], rom['mags']['SPIR-V'],
|
||||
rom['exts'].get('.pak', 0) if F.get('main_markers', set()) & {'Unreal', 'Unreal 4'} else 0, rom['mags']['Trinity pack'], gl_names=F.get('gl_names', 0))
|
||||
rom['exts'].get('.pak', 0) if F.get('main_markers', set()) & {'Unreal', 'Unreal 4'} else 0, rom['mags']['Trinity pack'], gl_names=F.get('gl_names', 0),
|
||||
gl_src=F.get('gl_src', 0), gl_files=sum(rom['exts'].get(e, 0) for e in pg.GL_FILES))
|
||||
vulkan, gl = S['graphics'][0] == 'vulkan', S['graphics'][0] == 'gl'
|
||||
notes = []
|
||||
if not vulkan and not gl and 'nvnBootstrapLoader' not in mods['main']['imports']: notes.append('main imports neither nvnBootstrapLoader nor Vulkan functions nor OpenGL through EGL: the host implements only NVN, Vulkan and OpenGL')
|
||||
if vulkan: notes.append(f'{S["graphics"][1]}; the NVN shader steps are skipped' + (', and its shaders are captured by booting it on the Mac (no SPIR-V file in the RomFS)' if S['vk_capture'][0] else ''))
|
||||
if gl: notes.append(f'{S["graphics"][1]}; the NVN shader steps are skipped: gl_shaders.py translates its GLSL sources (MVGL archives, main\'s strings)')
|
||||
if gl: notes.append(f'{S["graphics"][1]}; the NVN shader steps are skipped: gl_shaders.py translates its GLSL sources (MVGL archives, main\'s strings'
|
||||
+ (', and the shaders it puts together at run time, captured by booting it on the Mac)' if S['gl_capture'][0] else ')'))
|
||||
notes += [S[k][1] for k in ('glslc_capture',) if S[k][0]]
|
||||
if S['ue_decode'][0]: # the manifest has no pak footers, so both kinds are named; the build tells them apart by the footer's version (uepak.py)
|
||||
notes.append(f'{rom["exts"].get(".pak", 0)} Unreal pak(s): version 11 paks keep the shaders Oodle-compressed in shader code archives (Octopath Traveler 0), decoded by booting the game on the Mac; '
|
||||
@@ -603,7 +605,7 @@ def write_profile(pdir, app, f):
|
||||
|
||||
|
||||
# ---- Derived profile (a version the game's profile was not brought up on) ----
|
||||
DERIVED_SKIP = {'game.tsv', 'shader_map.tsv', 'shader_facts.tsv', 'shader_pairs.tsv', 'c1_full.tsv', 'cb_extent.tsv', 'vk_capture.tsv', 'patches.py'} # made again from the packed version's own files (patches.py: the rewriter's patches of the registered builds)
|
||||
DERIVED_SKIP = {'game.tsv', 'shader_map.tsv', 'shader_facts.tsv', 'shader_pairs.tsv', 'c1_full.tsv', 'cb_extent.tsv', 'vk_capture.tsv', 'gl_capture.tsv', 'patches.py'} # made again from the packed version's own files (patches.py: the rewriter's patches of the registered builds)
|
||||
|
||||
|
||||
def derive_profile(g, pdir, ver, d, mods):
|
||||
@@ -976,53 +978,59 @@ def decode_ue_shaders(repo, dirs, pkg_ver, patched, work, env, logs, g):
|
||||
PROG.end(f'{total:,} shaders in {hms(time.time() - t0)}')
|
||||
|
||||
|
||||
VK_CAPTURE_SECS = 120 # the longest Vulkan capture run: Super Mario Galaxy creates its 3,089 modules within 6 s of its first one (about 15 s after boot on an M4 Pro)
|
||||
CAPTURE_SECS = 120 # the longest run-time shader capture: Super Mario Galaxy creates its 3,089 Vulkan modules within 6 s of its first one (about 15 s after boot on an M4 Pro)
|
||||
# Run-time shader captures, by graphics API: the host's switch, the profile's table of the keys a capture must hold, and a capture file's name → its key (None: not a whole file;
|
||||
# the host writes each under a temporary name and renames it). Vulkan: <16 hex>.spv (dm/gfx/vk/hle_vk.m); GL: <16 hex>.vert / .frag (dm/gfx/gl/hle_gl.m sh_capture)
|
||||
CAPTURES = {'vk': dict(name='Vulkan', env='DM_VK_CAPTURE', table='vk_capture.tsv', what='SPIR-V shaders it generates', key=lambda n: n[:-4] if n.endswith('.spv') and len(n) == 20 else None),
|
||||
'gl': dict(name='GL', env='DM_GL_CAPTURE', table='gl_capture.tsv', what='GLSL shaders it puts together',
|
||||
key=lambda n: n if re.fullmatch(r'[0-9a-f]{16}\.(vert|frag)', n) else None)}
|
||||
|
||||
|
||||
def vk_wanted(g): # the SPIR-V keys the profile's vk_capture.tsv lists; None without the table (a provisional profile)
|
||||
try: keys = {l.strip() for l in g.table('vk_capture.tsv').read_text().splitlines() if l.strip() and not l.startswith('#')}
|
||||
def capture_wanted(g, api): # the keys the profile's capture table lists; None without the table (a provisional profile)
|
||||
try: keys = {l.strip() for l in g.table(CAPTURES[api]['table']).read_text().splitlines() if l.strip() and not l.startswith('#')}
|
||||
except OSError: return None
|
||||
return keys or None
|
||||
|
||||
|
||||
def vk_done(d): # keys of the whole SPIR-V files in a capture folder (the host writes each under a temporary name and renames it)
|
||||
def capture_done(d, api): # keys of the whole files in a capture folder
|
||||
try: names = os.listdir(d)
|
||||
except OSError: return set()
|
||||
return {n[:-4] for n in names if n.endswith('.spv') and len(n) == 20}
|
||||
return {k for k in map(CAPTURES[api]['key'], names) if k}
|
||||
|
||||
|
||||
def capture_vk(repo, d, patched, work, env, logs, g):
|
||||
"""A Vulkan game without SPIR-V files (Super Mario Galaxy): it generates its SPIR-V shaders at run time (Hagi's GX shaders, from its pipeline database), so no game file holds them.
|
||||
The game boots in the Mac host (headless and silent, no shader library) with DM_VK_CAPTURE (dm/gfx/vk/hle_vk.m), which writes each module to <version dir>/vk_capture, where vk_shaders.py
|
||||
reads them with the RomFS; the modules are the same on every run. The run stops once every key the profile's vk_capture.tsv lists is there; without the table, once none has been added for 20 s.
|
||||
A folder that already holds every listed key is reused"""
|
||||
want, out = vk_wanted(g), d / 'vk_capture'
|
||||
PROG.stage('Vulkan shader capture')
|
||||
if want and vk_done(out) >= want:
|
||||
def capture_shaders(repo, d, patched, work, env, logs, g, api):
|
||||
"""Shaders no game file holds, made at run time: a Vulkan game without SPIR-V files (Super Mario Galaxy: Hagi's GX shaders, from its pipeline database) or an OpenGL game whose GLSL is put
|
||||
together at run time (Balatro: LÖVE adds its header and its built-in code to each of the game's shader files, in Lua). The game boots in the Mac host (headless and silent, no shader
|
||||
library) with DM_VK_CAPTURE (dm/gfx/vk/hle_vk.m) or DM_GL_CAPTURE (dm/gfx/gl/hle_gl.m), which writes each shader to <version dir>/vk_capture or gl_capture, where vk_shaders.py /
|
||||
gl_shaders.py read them with the RomFS; the shaders are the same on every run. The run stops once every key the profile's capture table lists is there; without the table, once none has
|
||||
been added for 20 s. A folder that already holds every listed key is reused"""
|
||||
C = CAPTURES[api]; want, out = capture_wanted(g, api), d / f'{api}_capture'
|
||||
PROG.stage(f'{C["name"]} shader capture')
|
||||
if want and capture_done(out, api) >= want:
|
||||
say(f'reusing the captured run-time shaders: {out}'); PROG.end('reused'); return out
|
||||
PROG.set(step='building the Mac host')
|
||||
host = build_mac_host(repo, work / 'tools/dmhost', env, logs, g)
|
||||
cap, rd, log = d / 'vk_capture.partial', work / 'tmp/vk_run', logs / 'vk_capture.log'
|
||||
cap, rd, log = d / f'{api}_capture.partial', work / f'tmp/{api}_run', logs / f'{api}_capture.log'
|
||||
rmtree(cap); rmtree(rd); cap.mkdir(parents=True)
|
||||
for x in ('save', 'temp', 'msl'): (rd / x).mkdir(parents=True)
|
||||
# everything the run writes stays in <id>_build (DM_ROOT is already there): a fresh save and temporary storage, an empty shader library folder (pipelines fail, the game still creates every module)
|
||||
henv = dict(env, DM_ROMFS=str(d / 'RomFS'), DM_SAVE=str(rd / 'save'), DM_TEMP=str(rd / 'temp'), DM_MSL_DIR=str(rd / 'msl'), DM_VK_CAPTURE=str(cap))
|
||||
# everything the run writes stays in <id>_build (DM_ROOT is already there): a fresh save and temporary storage, an empty shader library folder (pipelines fail, the game still creates every shader)
|
||||
henv = dict(env, DM_ROMFS=str(d / 'RomFS'), DM_SAVE=str(rd / 'save'), DM_TEMP=str(rd / 'temp'), DM_MSL_DIR=str(rd / 'msl'), **{C['env']: str(cap)})
|
||||
PROG.set(step='booting the game', done=0, total=len(want) if want else None, unit='shaders')
|
||||
say(f'booting the game in the Mac host to capture the SPIR-V shaders it generates at run time → {out}')
|
||||
say(f'booting the game in the Mac host to capture the {C["what"]} at run time → {out}')
|
||||
t0 = time.time(); last = {'n': 0, 't': t0}
|
||||
def done(): # every listed key is there; without the table, none has been added for 20 s
|
||||
got = vk_done(cap); n = len(got & want if want else got); PROG.set(done=n)
|
||||
got = capture_done(cap, api); n = len(got & want if want else got); PROG.set(done=n)
|
||||
if n != last['n']: last['n'], last['t'] = n, time.time()
|
||||
return bool((want and got >= want) or (not want and n and time.time() - last['t'] > 20))
|
||||
rc = boot_mac(host, patched, henv, rd, log, VK_CAPTURE_SECS, done)
|
||||
got = vk_done(cap)
|
||||
rc = boot_mac(host, patched, henv, rd, log, CAPTURE_SECS, done)
|
||||
got = capture_done(cap, api)
|
||||
if rc is not None and not (want and got >= want): host_stopped(log, rc, time.time() - t0)
|
||||
if not got or (want and not got >= want):
|
||||
PROG.end(ok=False)
|
||||
PROG.say(f'---- last lines of {log} ----\n' + ''.join(collections.deque(open(log, errors='replace'), maxlen=40)).rstrip('\n'))
|
||||
raise BuildError(f'captured only {len(got)} run-time shaders'
|
||||
+ (f', the profile needs {len(want)}' if want else '') + f' (host output: {log})')
|
||||
if want and got - want: say(f'[RED] captured shaders the profile\'s vk_capture.tsv does not list: {len(got - want)}')
|
||||
if want and got - want: say(f'[RED] captured shaders the profile\'s {C["table"]} does not list: {len(got - want)}')
|
||||
swap_in(cap, out); rmtree(rd)
|
||||
PROG.end(f'{len(got)} shaders in {hms(time.time() - t0)}')
|
||||
return out
|
||||
@@ -1586,13 +1594,14 @@ def main(args=None):
|
||||
src = pg.build_sources(game / 'ExeFS', game / 'RomFS', g.words('modules'))
|
||||
gfx = src['graphics'][0] # nvn, vulkan (Hades: the host's Vulkan layer, SPIR-V shaders) or gl (Digimon Story Cyber Sleuth: the host's GL layer, GLSL sources)
|
||||
glslc = gfx == 'nvn' and src['glslc_capture'][0] # shaders the game compiles at run time, captured in the Mac host (capture_glslc)
|
||||
vkcap = gfx == 'vulkan' and src['vk_capture'][0] # SPIR-V shaders the game generates at run time, captured in the Mac host (capture_vk)
|
||||
vkcap = gfx == 'vulkan' and src['vk_capture'][0] # SPIR-V shaders the game generates at run time, captured in the Mac host (capture_shaders)
|
||||
glcap = gfx == 'gl' and src['gl_capture'][0] # GLSL the game puts together at run time (LÖVE), captured in the Mac host (capture_shaders)
|
||||
uedec = gfx == 'nvn' and src['ue_decode'][0] # Unreal Engine shaders, Oodle-compressed in the paks' shader code archives, decoded in the Mac host (decode_ue_shaders)
|
||||
if gfx == 'vulkan': say(f'Vulkan game ({src["graphics"][1]}): vk_shaders.py translates the shaders instead of the NVN shader chain')
|
||||
if gfx == 'gl': say(f'OpenGL game ({src["graphics"][1]}): gl_shaders.py translates the shaders instead of the NVN shader chain')
|
||||
for on, k in ((glslc, 'glslc_capture'), (vkcap, 'vk_capture'), (uedec, 'ue_decode'), (gfx == 'nvn' and src['oodle_packs'][0], 'oodle_packs'), (gfx == 'nvn' and src['ue_legacy'][0], 'ue_legacy')):
|
||||
for on, k in ((glslc, 'glslc_capture'), (vkcap, 'vk_capture'), (glcap, 'gl_capture'), (uedec, 'ue_decode'), (gfx == 'nvn' and src['oodle_packs'][0], 'oodle_packs'), (gfx == 'nvn' and src['ue_legacy'][0], 'ue_legacy')):
|
||||
if on: say(src[k][1])
|
||||
PROG.add(glslc + vkcap + uedec + (gfx == 'nvn'))
|
||||
PROG.add(glslc + vkcap + glcap + uedec + (gfx == 'nvn'))
|
||||
aoc = extract_aoc(nx, keys, repo, work, app, fam, logs)
|
||||
base = next((dirs[v] for v in versions if picks[v][1] is None), dirs[versions[0]]) # the folder the base game was extracted to alone
|
||||
# the app name and bundle ID both come from the base NSP: the game name in its control data, and a bundle ID from its title ID; a setting / the command line overrides them
|
||||
@@ -1625,10 +1634,11 @@ def main(args=None):
|
||||
if uedec: decode_ue_shaders(repo, {v: dirs[v] for v in sv}, pkg_ver, patched, work, env, logs, g)
|
||||
sh = work / 'shader'
|
||||
if gfx == 'vulkan': # the packed version's merged RomFS holds the SPIR-V files the game loads; a capture holds those it generates
|
||||
cap = capture_vk(repo, game, patched, work, env, logs, g) if vkcap else None
|
||||
cap = capture_shaders(repo, game, patched, work, env, logs, g, 'vk') if vkcap else None
|
||||
ios = build_vk_shaders(py, repo, [game / 'RomFS'] + ([cap] if cap else []), sh, env, logs, g, reuse=not (provisional or derived or ALLOW_DAMAGED), mac=mac)
|
||||
elif gfx == 'gl': # the GLSL sources in the packed version's archives and in its main (the game compiles them at run time; the host finds each by the hash of its text)
|
||||
ios = build_gl_shaders(py, repo, [game / 'RomFS', game / 'ExeFS/main'], sh, env, logs, g, reuse=not (provisional or derived or ALLOW_DAMAGED), mac=mac)
|
||||
elif gfx == 'gl': # the GLSL sources in the packed version's archives and in its main (the game compiles them at run time; the host finds each by the hash of its text), and a capture of those it puts together
|
||||
cap = capture_shaders(repo, game, patched, work, env, logs, g, 'gl') if glcap else None
|
||||
ios = build_gl_shaders(py, repo, [game / 'RomFS', game / 'ExeFS/main'] + ([cap] if cap else []), sh, env, logs, g, reuse=not (provisional or derived or ALLOW_DAMAGED), mac=mac)
|
||||
else:
|
||||
# Trinity packs and legacy Unreal paks: the split decodes them with the packed version's own decoder (Oodle, LZ5), in the Mac host built for the profile
|
||||
oodle = (lambda: {'DM_OODLE_HOST': str(build_mac_host(repo, work / 'tools/dmhost', env, logs, g)), 'DM_OODLE_MODULES': str(patched)}) if src['oodle_packs'][0] or src['ue_legacy'][0] else None
|
||||
|
||||
Reference in new issue
Block a user