Commit Graph
62 Commits
Author SHA1 Message Date
seedds 6034fcad89 Bayonetta 3 (01004A4010FEA000, update 1.2.0, PlatinumGames' engine on SDK 13.3.2): game profile plus general changes to the Oodle tool mode, the shader split and the NVN layer. Plays on the iPhone (IPA 14.2 GB). On the Mac from the boot through the title, a new game (Standard, Naive Angel Mode off, brightness, the auto-save notice), the VP9 opening movie and the prologue's first real-time scene and play: 60 fps in play (frame time p50 16.67 / p95 17.5 ms, GPU about 5 ms a frame on an M6), 30 where the game asks for it (title, loading screens), sound, about 2 GB resident and 3.1 GB of Metal allocations, flat after the first scene; the title is up about 15 s after launch with a warm shader cache. hle_oodle.c: everything is in 72 PKZ archives (14 GB) of DAT archives whose entries are Oodle Kraken or stored, and this main reads them only through Oodle's streaming decoder (main+0xcbc698 / +0xcbc7f8), so OodleLZ_Decompress was never linked in and its name string is not there; when a main has no such name the tool mode now finds OodleLZDecoder_MemorySizeNeeded / Create / Destroy / DecodeSome (the two functions that pass oodlelzcompressors.cpp to Oodle's error handler, the first also taking the legacy-compressor message, and the two functions between them, checked) and decodes each request with them; games with OodleLZ_Decompress take the same path as before. shader_split.py: PKZ archives (pkz_entries, pkz_jobs: every entry, decoded by the game's own decoder in the Mac host), DAT archives (pgdat_members; a file that only starts with DAT\0 but whose tables do not fit is scanned as before) and PlatinumGames' shader objects (pg_shader: VSO / PSO / CSO / HSO / DSO, a control section and a code area each, single-stage programs): 8,024 objects, all in root_000.pkz, 6,424 distinct keys; all build but two domain shaders (subgroup operations), which are never drawn. profile_guess.py turns on oodle_packs for Kraken PKZ archives, so the NSP script hands the split the Oodle host. hle_nvn.c: the engine checks each shader object's GPU code version against the driver's range (main+0xd0da20) and refused its first built-in shader, hwfillcolor.vso at 1.15, against the 1.14 the host reported (Super Mario Odyssey's SDK 7 table), so nnMain's start-up failed and the shutdown read an uninitialized pool (main+0xb8462c); a driver runs the code of its own SDK's GLSLC, so the newest version reported now follows the SDK: 1.15 from SDK 8, 1.16 from SDK 13, and at least a GLSLC the game carries, as before. Also nvnWindowGetCrop and nvnSamplerBuilderGetLodClamp, red stubs before. Profile: pad_layout full, mem_used 116 MB, shader groups core / eff, a prewarm list recorded on the Mac over the route above (678 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS), the host unit tests and the six tool test files pass, and profile_guess.py --check gives the same results as before. README (games table, a Mac picture of the prologue), AGENTS.md (51 → 52), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-24 13:21:44 +08:00
seedds ff64352885 Super Smash Bros. Ultimate (01006A800016E000, update 13.0.5, SDK 8.2.1): game profile, NRO modules on iOS, data.arc shaders. Plays on the iPhone (IPA 17.5 GB, all 99 DLC licences)
On the Mac from update 13.0.5 (v2031616): the opening movie, the title, the first launch's DLC unlocks and Challenges board, the main menu, rulesets, stage and fighter select, 2-player battles (Frigate Orpheon, Big Blue, Great Bay, Find Mii, Prism Tower, Kongo Jungle 64) at 60 fps with about 2.7 ms of GPU per frame and 2 GB resident (1.2 GB of Metal allocations), quitting to the results and a second battle whose fighters' NROs the game loaded again; no red line in battle.

- Profile: modules main subsdk0 sdk (Catherine: Full Body's sdk and movie builds), the 5 GB guest layout, shader groups for data.arc's shader databases, effects and UI layouts, 35,012 shaders.
- patches.py: the engine's allocator (main+0x37c9a20, a static constructor) reserves 62 GB of address space with AllocateAddressRegion and aborted at main+0x37c9bd0. Booted once with a 64 GB alias region it commits from the region's start up (1.73 GB at the title, up to 2.5 GB in a battle) and keeps its page tables in the top 14 MB; all 11 instructions derived from the size are cut to a 4 GB region.
- sdk_patches.py: this SDK 8.2.1 sdk build (5fd4f531, Catherine's too) has the 1 GB AddressRegionManager node area and the 1 GB alignment of large regions (DRAGON QUEST XI S's SDK 7.3.2 layout at sdk+0x336b40 / +0x336d5c); both cut, so the region starts 64 MB into the 4.375 GB alias region.
- NVN: nvnSetExternalDebuggerEnabledNVX, fetched with nvnDeviceGetProcAddress(NULL, …) before the device and called unchecked (pc=0 from main+0x37fe92c), is a no-op in names.txt / hle_nvn.c, as docs/agents/03 already described.
- iOS NRO embedding (the part d979c34 left out): gen_embed.py packs the NROs nro.tsv lists into three sections (__TEXT,__dm_nro_x0, __DM_NRO_W,__img, __TEXT,__dm_nro_x1) with a table of their pieces (__DATA,__dm_nro_tab), since a Mach-O file has at most 255 sections and three per NRO would take 294; loader.c map_image maps an NRO from there (nro_pieces); build_ipa.sh puts nro.tsv beside the rewriter tables. The 98 NROs add 157 MB of code and 51 MB of data to the executable.
- shader_split.py reads data.arc through smash_arc.py (smash_arc_jobs / smash_arc_shaders: each current file scanned, labelled /data.arc/<format>#<hash40>); smash_arc.files(regions=True) gives every region's data of a regional file, without which 128 effect shaders were missing (34,884 of 35,012).
- Docs: README (games table, a Mac picture of Great Bay), AGENTS.md (50 → 51), docs/development.md (game notes, the NRO paragraph, nro_collect.py), docs/agents/01 (data.arc) and 03 (the 4 GB cut and its measurements, iOS NRO embedding).

Checked: the NSP script packs it from the NSPs alone, for the Mac and for iOS; tool tests (41), host unit tests and the NRO loader test pass. Played on an iPhone with this port's build from before it moved onto d979c34's NRO loader; the merged build was checked on the Mac (battles, an NRO loaded again) and builds for iOS.
2026-10-24 13:08:34 +08:00
d979c346b9 Integrate demand-first PSOs, far-base tracing and NRO loading
Integrate PRs #75, #90 and #91 with focused correctness fixes:
- Prioritize demanded render pipelines and bound speculative compilation.
- Follow copied jump-table bases and reject W/X register clobbers.
- Collect and rewrite registered NROs, load them on the Mac, and preserve
  runtime activation order through unload/reload and symbol rebinding.
- Preserve refused-only manifests and invalidate collection caches when
  archive readers or symbol parsing change.

Add pipeline scheduling, register tracing, NRO packaging and loader
lifecycle regression coverage. Document that executable NRO embedding
on iOS remains outside this integration.

Validation: 41 Python tests; 48 pipeline scheduling, 102 NVN and 95
geometry checks; three pipeline-key serialization cases; full host test
suite; Mac host build; iOS loader syntax checks; git diff --check.
Gameplay and controlled performance A/B measurements were not run.

Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-24 11:38:12 +08:00
Kyle Pyne 0633e56de4 Fix nested f-string quoting in NSP build script 2026-10-24 09:12:59 +08:00
seedds c392d56af0 Balatro (0100CD801CE5E000, update 1.1.3, LÖVE 11.4 "love-nx" with LuaJIT and SDL on SDK 18.3.2): game profile and the GL layer LÖVE needs, plus four general changes. Plays on the iPhone by touch; on the Mac from the intro and the title into a run and its tutorial, at 60 fps with sound, about 0.65 GB resident and flat over three minutes. The game is Lua source in the RomFS and renders with NVIDIA's desktop OpenGL through EGL, as Digimon does. GL layer (dm/gfx/gl/hle_gl.m): LÖVE reads GL_CONTEXT_PROFILE_MASK and takes its core-profile paths only when it says core (with a compatibility profile it streams vertices from client memory, which Metal cannot draw from), so the layer answers core and implements what those paths use, about 150 functions in all against Digimon's 79: persistently mapped buffers (glBufferStorage, glMapBufferRange: the shared MTLBuffer's own memory), glBufferSubData in command order, fences (a fence is the serial of the command buffer being recorded; a wait on one still recording commits it), glBindAttribLocation (the game's attribute index of each Metal input, set at the link) and current attribute values (glVertexAttrib4f: LÖVE's VertexColor and ConstantColor are disabled arrays), glGetActiveUniform and the uniform read-backs, every glUniform* form, glTexStorage2D / 3D with their level counts, 3D and 2D array textures (LÖVE makes a default of each type on 96 units), GL_TEXTURE_SWIZZLE_* (its font glyphs are RG8 read as R, R, R, G), glGenerateMipmap as a blit (the game's images are mipmapped), instanced and base-vertex draws, separate read and draw framebuffers, glBlitFramebuffer (colour, same size), glReadPixels, glClearBufferfv, glFrontFace and the EGL calls SDL makes; stack arguments of the functions with more than eight are 64-bit, as AAPCS64 lays them out. DM_GL_STUBS=1 gives every GL name the layer lacks a stub of its own that reports its first call, instead of a null pointer. Shaders: LÖVE puts its 35 GLSL sources together at run time in Lua, so the NSP script captures them on the Mac as it does Super Mario Galaxy's SPIR-V (capture_shaders now serves both, profile gl_capture.tsv; profile_guess.py's gl_capture rule: an OpenGL game with no GLSL source in its RomFS or main), and gl_shaders.py keeps a bool uniform's declared type (a uint in SPIR-V; LÖVE's Shader:send takes only a boolean for a bool) and takes from main only strings that start with '#version <number>' (Balatro's main has glslang's messages). General: profile_guess.py leaves vkGetInstanceProcAddr, SDL's Vulkan loader, out of its Vulkan count (Balatro's provisional profile said Vulkan; Hades and both Galaxies import 99 or 100 Vulkan functions); OpenAudioRenderer without an event now starts the host's render thread too (hle_audren.c ar_open: only the form with an event did, and such a game was silent); loader_census.py follows registers in a straight line (adrp, add, loads through relocated GOT entries, mov, csel), for LÖVE's glad, whose pointers are globals its code keeps the addresses of in registers; and the iPhone gamepad has a third layout, Tap (profile pad_layout touch, which needs op_mode handheld; DM_GAME_PAD_LAYOUT replaces DM_GAME_PAD_FULL): no control but the layout button, every touch to the game, offered in the layout cycle only in handheld mode. Profile: op_mode handheld (the Switch build plays by touch there), pad_layout touch, gl_capture.tsv. A Lua error makes Balatro quit (its Switch error handler), and love-nx's video worker is then deleted twice (main+0x339c00): the bring-up notes say how the errors were found. Checked: the NSP script packs it from the NSPs alone (the capture in under a second, 35 shaders for the Mac and iOS, an 82 MB IPA), tool tests pass, profile_guess.py --check changes no other profile; Digimon was not rerun (its files are not on hand). README (games table, a Mac picture of the title), AGENTS.md (47 → 48), docs/development.md, docs/agents and the READMEs of dm/games and dm/gfx say so 2026-10-24 01:50:19 +08:00
seeddsandv0lumehi 98a91a6749 NSP script: --platform mac (PLATFORM = 'mac') builds only what running a game on this Mac needs, from PR #82
A development option; the Mac is the development host, not a platform for players. The same steps up to the
shaders (extraction, DLC, rewrite, run-time captures and decodes, heap check, shader split), then only the
Mac's: dmsh_build.py without IOSOUT (an inherited one is dropped), a Vulkan or GL game's library without --ios,
the Mac host for the profile into <id>_build/tools/dmhost (build_mac_host, through DM_GAMES_DIR /
DM_GAMES_FIRST for a provisional or derived profile) and the button-icon font into <id>_build/font. No iOS
shaders, Xcode project or IPA; the extracted files are kept whatever KEEP_EXTRACTED says, and the last line is
the command that runs the game (its save in <id>_build/run/save, DM_NVN_SHMAP for an NVN game, DM_AOC_DIR
with DLC, the --env-file switches). It refuses --prepare-xcode and an Intel Mac.

Reuse: the Mac libraries have a key of their own (shader/mac-b-<chain>.key beside the cache folder,
shader/vk.key / gl.key), which an iOS build writes too, so a Mac-only build after an IPA reuses its shaders;
either key only while the Mac libraries carry a full build's stamp (inputs.tsv). An iOS build after a Mac-only
one translates again (dmsh_build.py's per-shader keys include the iOS compile convention) and takes the Mac
libraries from the Metal compile cache. build_mac_host now has a reuse key too (the compile line, the sources and
headers of the folders it compiles, the generated profile header, libvpx, the third-party headers, clang and the
SDK), which also saves the rebuild for the Oodle / LZ5 decodes and the run-time captures. build_vk_shaders and
build_gl_shaders share layer_shaders. The default (ios) path builds as before.

Checked: dm/tools/tests 34 of 34 (11 in the new test_nsp_mac.py); Hades in Mac mode ran the Mac stages only,
printed the run command and exited 0, and a second run reused every stage, the host included (2 s); the host it
built boots Hades from that command. Breath of the Wild (NVN, with DLC) in Mac mode restored its 19,592 shaders
from the cache, printed a command with DM_AOC_DIR and DM_NVN_SHMAP, and the host booted it from that command into
play. Not run end to end here: a GL, provisional or derived game in Mac mode, and an IPA after a Mac-only build
(unit tests with the tools stubbed).

Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
2026-10-24 01:40:58 +08:00
seedds e98a11562a shader_split.py reads GLSLC outputs of several stages: Baba Is You (01002CD00A51C000, no profile) keeps its shaders as 120 GLSLC outputs in RomFS/Assets.dat, each a pixel and a vertex shader compiled together, then a reflection section, and glslc_shaders took only outputs of one control section, so the NSP script's split found 0 shaders and the pack stopped one step later with "no iOS shader libraries". An output with more than one control section is now read by its section headers (glslc_section returns every GPU code section, glslc_program checks each stage and needs distinct stages); its stages are labelled #GLSLC@<offset>#var, so var_pairs pairs them with each other and separable_pairs leaves them out of the single-stage cover. The NSP script stops at the split when it finds no shader, with the reason, instead of at the iOS shader step. Checked: Baba Is You splits into 131 shaders (115 pixel, 16 vertex) and 115 pairs, and its pack built BabaIsYou.ipa from the provisional profile with 131 of 131 shaders ready (not run on an iPhone); the split's tables for 13 Sentinels, AI: THE SOMNIUM FILES, Raging Loop, Metroid Dread and Age of Calamity are byte-identical to their profiles', and no main on the game drive holds an output of several stages. docs/agents/04-graphics.md says so 2026-10-24 00:47:48 +08:00
seedds 86d9ba1239 One translation chain version for every game: DMSH_CHAIN in dm/gfx/nvn/dmsh_chain.h replaces the 47 dm/games/<id>/dmsh_chain.h, which every chain change rewrote and some were left behind (GitHub issue #78). The version hashed each profile's shader tables and nvn_* / shader_* keys too, once the only thing that kept the host off a library built from older tables; each shader's cache key (dmsh_build.py shader_key) has covered them since, so the version now covers only the chain's tools (dmsh_chain.py, dmsh_build.py, dmsh_fetch.py, dmsh_vin.py, dmsh_geometry.py, dmsh_facts.py, the translator's source, ios_metal.py, the lock) and the options, and needs no DM_GAME. What the per-game part still did is done by the library itself: a full dmsh_build.py run stamps b-<V>/inputs.tsv (Mac and iOS) with the game and the digest of the profile's inputs (dmsh_chain.py inputs / --inputs: the four shader tables, bf_want.tsv, which the variants use but the version never counted, and the keys), starts from no shader outputs (a partial one from none of the keys it builds), so a key that left the map, a shader that no longer builds or a variant no longer made keeps no old library, refuses a directory stamped for another game, and writes the profile's c1_full.tsv and cb_extent.tsv from its Mac outputs; a partial build keeps the stamp only while it names the current inputs. c1_guard.py / cb_extent.py head their table with the library's chain version and stamp (table_head), and the host takes a table only beside that stamp, logs the stamp and reports one of another game; build_ipa.sh reads the one header and stops when the iOS libraries' stamp is not this profile's current inputs or c1_full.tsv names another. NSP script: chain_header writes the one header when stale, before any host is built; provisional and derived profiles no longer write one; chain_tables makes a table again when it does not name the reused library's stamp (False without one). Makefiles take the header beside nvn_mtl.m; Xcode projects prepared earlier point at a profile header that is gone and are told to prepare again. Tests: test_nsp_chain.py (the committed header is current, stale and missing ones are written, tables against the stamp). Checked on The Great Ace Attorney Chronicles: a full build gave 460 of 460 shaders and both tables (0 and 409 entries, as before), the Mac host booted with the stamp and both tables past frame 900; one added facts row rebuilt one shader under the same chain version with a new digest and new tables, and the host refused the old tables beside the new stamp; a full build removed planted outputs, another game's build into that directory stopped, a partial build after an input change dropped the stamp; build_ipa.sh stopped on a stale stamp, a missing one and a stale or old-form c1_full.tsv; three NSP packs built (no table lines: dmsh_build made them), reused everything, and remade deleted tables from the reused build. Graphics unit tests give the same results as before (on this Mac t_refl needs ~/dm-work's libraries and several GPU tests fail on the previous commit too). AGENTS.md's chain procedure (header once after a tool change; a full shader build makes the tables), dm/games/README.md, dm/gfx/README.md and docs/development.md say so 2026-10-23 23:37:03 +08:00
seedds f4f0a85d6e patch_nso.py split into the general rewriter and per-build patch files (GitHub issue #63): every game's bring-up edited the one file, so PRs for unrelated games conflicted. dm/tools/patch_nso.py (1,711 → about 1,100 lines) keeps the general passes and the driver; the instruction encoders move unchanged to dm/tools/nso_asm.py; a game's own patches (CUTS, OBS and the frame-step stubs, now FRAME_STEP = {build ID: stub(ctx)}) move verbatim with their comments to dm/games/<id>/patches.py (16 profiles: Tears of the Kingdom, Breath of the Wild, Age of Calamity and Paper Mario: The Thousand-Year Door with their stubs), and the cuts of Nintendo's sdk module builds, 6 of the 15 shipped by 2 to 4 games, to dm/tools/sdk_patches.py. The rewriter loads every patch file, whatever the game, and applies those of a module's build ID at the points it did before (cuts before the scan, gates after the rewrite, stubs after the gates); a build ID claimed twice stops it, and its command line needs no DM_GAME. build_src.sha256 lists the rewriter's files and every patch file (the same for every module). NSP script: the rewrite step's reuse key counts nso_asm.py and only the patch files that name one of the game's builds (patch_files), profile_files leaves patches.py out (only the rewrite step reads it), and a derived profile gets none. dm/tools/tests/test_patches.py: no build claimed twice, the entries' forms, a game's file names only its own profile's non-sdk builds, sdk_patches.py only sdk builds some profile ships, and the reuse key's patch files. Checked byte-identical: every extracted module on the game drive (131 builds) rewritten with the previous and the new rewriter gives the same outputs but build_src.sha256, 28 of the 30 patched builds among them (all four frame-step mains), and 7 fail alike (rtld, and New Super Mario Bros. U Deluxe's, which no profile rewrites); the other two (Super Mario Bros. Wonder's mains and Absolum's, no game files here) by their CUTS / OBS data, equal to the previous tables. AGENTS.md, docs, dm/games/README.md, the profiles' comments and the host comments that named a game's cut in patch_nso.py now name its patches.py or sdk_patches.py 2026-10-23 23:36:21 +08:00
seedds 4c7adf4619 c1_full.tsv and cb_extent.tsv are no longer committed: each checkout makes them from its own shader build's Mac outputs (git ignores them in dm/games/*/), as the NSP script does after its iOS shader build. They follow from the MSL this Mac built, every chain change rewrote about 90 of them, and 56704a0 could only carry most over unchecked; 81 files, 18.7 MB. The NSP script makes a missing one as a stale one (chain_tables now says whether both name the chain: with no Mac outputs a reuse builds again and a build stops), and no step's reuse key counts them (profile_files), so the run after they were made does not split again. build_ipa.sh stops for an NVN game whose c1_full.tsv is missing or names another chain, with the c1_guard.py command, before generating anything (the host would give every c1 its own 64 KiB). Checked on The Great Ace Attorney Chronicles from a checkout without them: the first pack made both (0 and 409 entries, as committed), the second reused the split and the iOS shaders with no table lines, a third with them deleted made them from the reused build's Mac outputs; build_ipa.sh stops on a missing and a stale c1_full.tsv. AGENTS.md's chain procedure, dm/games/README.md, dm/gfx/README.md and docs/development.md say so 2026-10-23 22:24:10 +08:00
seedds a2e6f4a18c NSP script: a registered profile's chain files that a chain change left behind are made again (GitHub issue #78: after bb585cd, 45 of 47 profiles' dmsh_chain.h no longer held the version their source gives, and Breath of the Wild's pack stopped at dmsh_build). chain_header writes dmsh_chain.h again, before any host is built, when it does not hold the version dmsh_chain.py gives (it follows from the source alone; a current one is left untouched, so the host is not rebuilt). chain_tables makes c1_full.tsv (c1_guard.py) or cb_extent.tsv (cb_extent.py) again from the build's Mac outputs when its header names another chain, which the host would ignore (every c1 its own 64 KiB, whole uniform bindings copied), also when the libraries are reused; provisional and derived profiles make both as before. The two tables are the shader step's outputs, so its reuse key no longer counts them. Tests: dm/tools/tests/test_nsp_chain.py (stale, missing and current headers and tables), and chain_tables with the real tools on Breath of the Wild's Mac cache (47 s, the committed list's 2 entries) 2026-10-23 22:11:09 +08:00
seedds 3f73710bd6 NSP script: cli() restores os.environ after each run. main points DM_GAMES_DIR (and, for --provisional or a derived profile, DM_GAMES_FIRST) at the run's own profile folder for its tools, and Spyder runs the next F5 in the same process: the game lookup (game.py by_title) then found that profile and took it for a registered one. ASTRAL CHAIN's second run printed its profile without (provisional), skipped provisional_prep and the chain header, took the split tables as checked, and stopped at the iOS shaders with FileNotFoundError on the profile's dmsh_chain.h; after a derived build the next run would have packed from the derived profile in place of the game's own. Two runs in one process now both take the provisional path and leave the environment as it was; the progress tests pass 2026-10-23 15:59:45 +08:00
seedds 1822616144 Intel Macs (GitHub issue #54): the shader translator's launcher dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate is a universal binary (arm64 and x86_64, the same .NET 10.0.13 host joined with lipo); it was arm64 only, and the NSP script stopped at dmsh_build with Bad CPU type in executable. The lock does not cover the launcher, so the chain version is unchanged. The x86_64 translator under Rosetta gives byte-identical output (Raging Loop, 452 shaders; Ryujinx's constant folding uses only exactly rounded arithmetic). The Mac host runs the game's own arm64 code, so on an Intel Mac a step that boots the game there (a run-time shader capture, a decode with the game's own Oodle or LZ5) stops with a plain error, and a provisional profile's heap check is skipped (build_ipa_from_nsp.py apple_silicon) 2026-10-23 14:42:50 +08:00
seedds b3d9b24be7 NSP script: the iOS shaders build with the Mac's own Metal compiler and spirv-cross (dm/gfx/tools/own_compilers.py), one note line naming any that differ from dmsh_chain.lock; the translator must still match, and the lock and chain version stay as they are, so a player on another Xcode (GitHub issue #26: Xcode 27.2 beta, metal 32023.922) or a newer spirv-cross needs no --update-lock; a Metal toolchain that does not run is reported before the build (xcodebuild -downloadComponent MetalToolchain) 2026-10-23 13:33:10 +08:00
seedds 2000f9274a FINAL FANTASY RESONANCE plays on the Mac at 60 fps: profile 0100606026F12000 (update 1.0.1, SDK 20.5.17)
- Engine: Unreal Engine 5.6.1 (Square Enix, the desktop renderer), the newest Unreal so far; everything in one IoStore container beside one pak; the sdk and subsdk0 builds are STEINS;GATE RE:BOOT's
- Rewriter, for every module: code before the first unwind entry is one function (each site there was a function of one instruction, so a byte jump table at main+0x219234 went unseen), and a function longer than 256 KB is analysed in pieces cut after a ret / br / b that no conditional branch or b within 64 KB crosses, with far branches and jump table targets checked against windows and retry blocks afterwards (this main's 82 MB without unwind entries took 100 GB of memory as one function; now 2.7 GB resident). Checked: DRAGON QUEST VII Reimagined's main byte-identical to its rewrite by the previous patch_nso.py; FINAL FANTASY RESONANCE's, Octopath Traveler 0's and DRAGON QUEST XI S's byte-identical between two different cut placements; only DQ VII's main among the 120 modules of 33 games on hand has other sites before its first unwind entry (none in a window or jump table)
- Shaders: no archive in the pak; the two libraries are IoStore shader library chunks whose tables and shader groups are Oodle-compressed, read through the game's own Oodle in the Mac host's tool mode with no boot (uepak.py Toc / io_archive / OodleHost; library names from CityHash64 of the manifest's archive names) and written as decoded version 2 archives; the NSP script uses them for any game with IoStore shader libraries. The pak's single-block files store block size 0 (Pak.entry). 73,496 unique shaders; 11 do not build (3 geometry, 8 compute with 3D image atomics), none met so far. 2,077 particle vertex shaders pick their texture slot at run time (tex_dyn_slots); CopyStencilToLightingChannelsPS writes integers to an R8Uint target (shader_facts_manual.tsv; without it the sprites were dark silhouettes)
- Host, for every game: a shader with run-time texture handles in a game without tex_dyn_slots freed its reflection twice and aborted the host (nvn_mtl.m); oe::MuteRecording / UnmuteRecording (no-ops)
- Memory: the FMallocBinned3 constructor call is cut and the host's allocator runs Unreal's heap (hle_uemalloc.c layout UE5_6: UE 5.5+ FMalloc has 32 slots, with MallocZeroed and two more TLS cache calls); the SDK's node area is cut to 62 MB. The engine's own page allocator only grows its use of the engine region (2.03 GB of address space over seven minutes, about 1 GB mapped), and a 2 GB region aborted in the first battle, so the region is 4064 MB in Engage's 5 GB block (this sdk build aligns regions to 2 MB, no alignment cut needed)
- Frame rate: 60 fps (30 on the Switch): the host writes rhi.SyncInterval 1 (hle_fpslock.c rows for this build; profile fps_target 60); the game steps by real time (DM_NVN_REGLUM traces at 30 and 60 fps match from the load through a walk)
- Mac: first-boot settings, the title, a new game, the opening cutscene, the first field and its autosave, Continue, and the first two battles at 59.7 fps, 2.3-2.6 GB resident, flat over seven minutes, no red line apart from a few texture type mismatches at one moment of the new game's load in some runs. Docs: docs/agents 01, 03, 04 and development.md
2026-10-23 07:22:09 +08:00
seeddsandAnimax Deng 6da1882871 NSP script: --prepare-xcode makes a per-game Xcode project to sign and run with your own developer account (GitHub PR #59)
Each game gets its own project in <NSP folder>/<title ID>_build/xcode, registered in a shared NxNative.xcworkspace; a target phase copies the RomFS, shader libraries and add-on content into the app before Xcode signs it, and preparing again keeps the bundle ID and team chosen in Xcode. The unsigned IPA build is unchanged.
On top of the PR: the resource phase stops when the profile's shader chain is no longer the prepared shaders' (the host loads only its own chain), prepared projects ask for the Increased Memory Limit capability (DM_ENTITLEMENTS overrides), the source links are made again on every preparation, and scheme names keep non-ASCII letters. The PR's TOML launcher is left out.

Co-authored-by: Animax Deng <Animax.deng@gmail.com>
2026-10-23 00:37:52 +08:00
seedds e72a06510e DRAGON QUEST VII Reimagined plays on the Mac: profile 0100A9D01C446000 (update 1.2.0, SDK 19.3.1)
- Files and shaders: Unreal Engine 4.27 with IoStore containers beside one pak (version 11, its footer magic 0xcc bytes from the end, uepak.py magic_back); the 62,901 shaders of the pak's two shader code archives are LZ4 blocks, which uepak.py decodes itself (lz4_shaders), so the NSP script runs no Oodle host when no record is Oodle. One global shader writes integers into an R8Uint target (shader_facts_manual.tsv)
- Shader chain, for every game: a texture query on an image handle (c2 0x120 + 8 r: Unreal's GetDimensions on the RWTexture3D a compute shader writes) had no type in a facts table built without PC shaders, and the 2D default took a 3D image's depth from its width; the translator now translates such a shader again with the image types of its first translation (dmsh_xlate ImageTypesOf; dmsh_xlate.dll rebuilt, dmsh_chain.lock updated). Host, for every game: a texture binding of slot 32-39 reads the image bound in the image slot that shares its c2 word (nvn_mtl.m tex_slot_h; Hades II's texture slot 34 without an image still reads the texture). Unreal's volumetric lightmap indirection patch (7b958a7b21e72801) read the empty texture slot 32's 1x1 size and wrote only cell (0, 0, 0), so in the opening cutscene every character and prop sampled the atlas origin, a daytime level's bricks baked 2^15 brighter (white figures about 10^4 too bright). Monster Hunter Rise has four such queries. Every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (the change touches no c1 or uniform read); DRAGON QUEST VII Reimagined rebuilt for Mac and iOS
- Host, for every game: stencil-mode texture builders (nvnTextureBuilderSetDepthStencilMode STENCIL) read the stencil of their depth-stencil storage (the hero was drawn as a dithered silhouette); the audio renderer at 32 kHz on the CPU path (the SDK makes its upsampler only for DSP rendering: UpsamplerManager::Allocate returns null and the host resamples 3:2); GetAudioDeviceOutputVolume; the shop and Nintendo Account calls of the title (ec Finalize / Cancel / GetErrorCode / ShowShopSubscriptionDetails, GetAuthorizationCode / GetState); kpr::GetKeyCodeMap; the movie output format's nv12-width / nv12-height beside nv12-uv-stride (Sofdec's VP9 movies were grey with bands)
- Memory: MallocBinned3 reserves 37 GB of pools in the engine region; its creation is cut and an observation gate hands the engine the host's malloc zone (patch_nso.py cut + gate OBS_UE_MALLOC, hle_uemalloc.c layout UE4_27; patch_nso.py places a gate where the scratch registers it uses are dead, scratch_dead); the region is 2 GB (game_args) with the SDK's node-area cut on this sdk build, in a 3.75 GB guest block; about 2.6 GB resident on the Mac
- Frame rate: this build's t.MaxFPS and rhi.SyncInterval in hle_fpslock.c; the profile keeps the game's 30 fps
- Diagnostics: DM_CTL (Mac) drives a running game line by line: presses, shots, pass and texture dumps, texlist (textures of a target with storage and block-linear layout), memdump, peek, Unreal console variables (cvarf / cvari), env; the pass dump prints buffer-to-texture copies and buffer textures' format, size and storage, and DM_NVN_CBDUMP saves the source bytes of 3D uploads (docs/agents 04, 05)
- Mac: the logos and opening movie, the title, a new adventure log, the opening cutscene (the well, the night scene with Maribel by the boat) into play at Pilchard Bay, and a saved log continued, at 30 fps with saves; no red line at run time apart from 62 of the 63 geometry shaders (one-pass cube shadow maps), which do not build. Unit tests: the graphics tests that build pass (t_refl's two sample libraries are not on this machine); 13 that compile nvn_mtl.m with hle_nvn.c already failed to build on main (g_ft defined in both, hle_nvn.c symbols missing from their stubs)
2026-10-22 23:13:12 +08:00
seedds 26231779dd Digimon Story Cyber Sleuth: Complete Edition: the chain header for the current chain version; the survey's notes know OpenGL games 2026-10-22 20:48:44 +08:00
seedds 298fa96f0b Digimon Story Cyber Sleuth: Complete Edition plays on the Mac: profile 010014E00DB56000 (update 1.0.3, SDK 6.4.0), an OpenGL layer on Metal (dm/gfx/gl), its GLSL translated offline (gl_shaders.py), MVGL archives (mvgl.py) 2026-10-22 20:43:37 +08:00
seedds afa99e5831 NSP script: the update is optional and any version builds (a derived profile for versions the profile was not brought up on)
- The newest version in the folder is packed (-p / PACKAGE picks another). A registered version with every shader_versions version present builds from the profile as before
- Any other version (the base game alone, another update, other build IDs) builds from a profile derived from it in <id>_build/profile: the profile's keys, that version's build row, and shader tables made from its own files; a red line names the modules whose per-build fixes do not apply
- Host prewarm skips shader keys the shader map does not have (a list of another game version), counted in the "prewarm done" line
- Tested with Hollow Knight: Silksong: the base game alone (1.0.27776) packs with its own 2,244 shaders and plays the opening on the Mac; base + 1.0.30000 builds as before
- README: the update is optional; Silksong's row no longer names it
2026-10-22 18:25:12 +08:00
seedds 33aa059ade Hollow Knight: Silksong plays on the Mac: profile 010013C00E930000 (update 1.0.30000, SDK 20.5.8)
- Unity 6000.0.50f1 IL2CPP with FMOD; the bundles carry their type trees, so unityfs.py reads Unity 6's Shader objects with no layout written out (2,450 shaders, the one geometry shader does not build)
- Host: nn::hid::GetDebugPadStates, which Unity 6 polls every frame and the SDK's own aborts on (Result 0x2aca)
- Rewriter: the address-region node area cut on this sdk build (alias high-water mark +3.00 GB → +2.26 GB in the first room)
- profile_guess.py: every game with an NPDM system resource gets the 5 GB block; heapless ones (Silksong, Absolum, Pokémon Legends: Z-A) used to get a 256 MB alias region
- Prewarm list recorded from boot to the first two rooms of Moss Grotto
2026-10-22 17:51:13 +08:00
seeddsandnocmt f1c4a6bd67 NSP script: --progress json for a program that shows the build (from PR #49)
stdout carries one JSON object per line: stage, step, progress (done, total,
unit, est, note, frac, eta, elapsed; null while unknown), stage_end and say,
which carries every message, errors and a failed step's log tail included.
A count change goes out at most every 0.5 s, any other change at once; the
ticker now runs in json mode too and sends a count that was held back, so a
stage whose count moves rarely (the rewrite) no longer shows a stale count
until it ends. json mode never prompts; close() stops the ticker by an event
of its own and keeps the mode, so a second build in one Spyder session gets
its redraws again.

dm/tools/tests/test_nsp_progress.py covers the events, the held-back count,
the failed step's tail, the CLI error and the bar / lines / off output,
with no game files, keys or Xcode.

Co-authored-by: nocmt <nocmtone@gmail.com>
2026-10-22 13:59:35 +08:00
seedds f7b8fede8f NSP script: the fps counter is on by default (SHOW_FPS = True) 2026-10-22 13:51:15 +08:00
seedds abcdbbdd4d Raging Loop plays on the Mac: profile 010020400E1C2000 (base game 1.0.0, SDK 7.3.2)
Unity 2018.3.10f1 IL2CPP with FMOD; its sdk and movie module are DRAGON QUEST XI S's builds, so
that build's rewriter cuts apply as they are, and no host change was needed. Brought up on the Mac
through the opening movie, the title, options, the tutorial and the prologue (choices, menu,
scenario chart, log) at 60 fps as on the Switch, with sound and saves; no red line at run time,
0.6 GB resident at the title and 0.77 GB after ten minutes of story.

- unityfs.py: Unity 2018.3's Shader layout (SHADER_2018_3, type hash 8db76007...). The game has no
  bundles and its serialized files carry no type trees, so the shader split stopped. Written from
  AssetRipper's TypeTreeDumps for 2018.3.10f1 and checked node for node against the dump (1,325
  nodes with the alignment flags); all 48 Shader objects read to their exact end (450 subprograms,
  452 shaders with main's GLSLC outputs). Other profiles' hashes are untouched.
- Memory: the hardware pool (3.2 GB) in Engage's 5 GB block. Unity 2018.3 reserves no address
  regions of its own, so every mapped page lies in StandardAllocator's region (377 MB at the peak).
- Input: the game reads no touch, so it stays docked with the 2D pad; the left stick moves the menu
  cursor.

Development notes, the graphics catalogue and the survey's list of Unity versions read.
2026-10-22 12:21:03 +08:00
seedds 26a72dd562 NSP script: DLC and iOS shaders reused across reruns, one Xcode build folder per game
A rerun after an interrupted IPA pack did a lot of the build again:

- The iOS shaders were keyed on the shader split's key, and that covers every dm/gfx/tools/*.py.
  A pull that changed only the Unity readers (shader_split.py, unityfs.py) made Monster Hunter
  Rise split again and then restore all 63,111 shaders from the cache (9 minutes). The iOS shaders
  are now keyed on the shader package's content and on what dmsh_build.py loads (the repository
  modules it imports, at any depth, and the chain lock). A split that runs again and writes the
  same package leaves them reused; hashing Rise's 510 MB package takes 0.3 s.
- The DLC was extracted on every run (Rise: 252 DLC, 16-30 s). It is now reused while the DLC NSPs
  (names, sizes, times) and nx_extract's sources are unchanged (aoc.key).
- Xcode's custom build location setting overrode -derivedDataPath, so every game's app was built
  in one shared ~/Library/Developer/Xcode/DerivedData/Build: a build after another game's compiled
  all 129 files, and two games built at once wrote the same app. build_ipa.sh sets SYMROOT and
  OBJROOT under the game's work folder; a repeat build there compiled nothing (1 s).

Checked: Celeste with its split forced to run again (same package, iOS shaders reused), Rise's DLC
twice (15.9 s, then 0.1 s), and an Xcode build into a path with spaces and an apostrophe.
Each game's next build runs its iOS shader step once more, to write the new key.
2026-10-22 11:33:33 +08:00
seedds df0820eb68 DRAGON QUEST XI S plays on the Mac: profile 01006C300E9F0000 (update 1.0.4, SDK 7.3.2)
New profile for Square Enix's Unreal Engine 4.18 game: from the opening movie and the title
through the prologue into free play at Cobblestone Tor, 30 fps with sound and saves, no red
line. An IPA builds from the NSP folder; it has not run on an iPhone yet.

- Files and shaders: Unreal 4.18 legacy (version 4) paks with LZ5 blocks, decoded on the Mac
  with the game's own LZ5_decompress_safe (uepak.py, hle_oodle.c serving after nninitStartup);
  38,892 shaders inline in the cooked packages as zlib-compressed LSLG records (shader_split.py
  ue_legacy_jobs, ue_inline_shaders; the NSP script and profile_guess know both pak kinds).
- Memory: Unreal's address region capped at 4064 MB (game_args, MB up to 4095); SDK 7.3.2 cuts
  in patch_nso.py (node area 62 MB, 32 MB alignment for large regions; the build Odyssey shares,
  which boots unchanged) for Engage's 5 GB block. Rewriter: a jump table whose bounds check sits
  right before the load (check_before_copy).
- Host, for every game: audio catch-up frames wait for queue space (Square Enix's sound driver
  never waits on the renderer event: 11,000 updates a second before); movie decoder events read
  at signal time as the SDK does, audio decoders keep 32 outputs (the player plays only once
  every track holds one: the opening movie never started in half the boots), and no input after
  an end-of-stream packet until Flush / Stop (a skipped movie mapped 2.25 GB of end-of-stream
  samples); ec shop accessors, friends, auto-sleep, album, file data cache and movie hooks.
- Graphics, for every game: a geometry shader that only picks the layer (Unreal's
  FWriteToSliceGS, the colour-grading LUT) folded into the vertex shader's layer entry points
  (dmsh_build.py gs_slice / vs_layer_entry, nvn_mtl.m prog_fold_gs); cube-map array depth counted
  in layer-faces in the Metal texture, the upload and the guest layout (t_nvn checks it);
  nvnTextureGetTextureAddress and the command memory size / free getters.
- Shader chain changed (dmsh_build.py): DQ11 rebuilt for Mac and iOS; every other profile's
  dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (no c1 or uniform read
  touched). Mac checks on this host: Big Brain Academy (movie loops, 23 catch-up renders as
  before), Monster Hunter Rise (movies), Super Mario Odyssey (boot on the cut sdk).
2026-10-22 09:32:16 +08:00
seedds 00de15f095 Celeste plays on the Mac: profile 01002B30028F6000 (update 1.4.1.1, SDK 15.3.2)
- MonoGame for the Switch, its C# and the game's compiled to C++ by Sickhead's BRUTE into main; FMOD Studio through AudioOut; only main and sdk. No host change: the logos, the title, a new file and the prologue at 60 fps with music and sound, no red line, about 1.65 GB resident, flat over five minutes. Not yet tested on an iPhone
- The shaders are MonoGame effects (MGFX version 8): 13 in Content/Effects/*.xnb and SpriteBatch's built-in one in main. shader_split.py reads them (mgfx_effect), each effect parsed to its end; a shader's bytecode is u32 control length, the control section, u32 code length, the code area. 26 shaders (20 pixel, 6 vertex), every binding decodes. A pass may name no vertex shader and keep SpriteBatch's (10 of the 13 effects), so they pair as single-stage programs. Cuphead's and 13 Sentinels' tables split again unchanged
- NSP script: a provisional profile never got the sys_resource the survey lists, so the host answered 32 MB of system resource for every new game; write_profile writes it now (Celeste: 0, as its NPDM declares)
- Survey: XNB files and the MonoGame and FMOD markers (profile_guess.py), and a note on where a MonoGame game's shaders are
- Prewarm list recorded through boot, the title, a new file and the prologue
2026-10-22 09:09:55 +08:00
seedds 64cdee954c Big Brain Academy: Brain vs. Brain plays on the Mac: profile 0100620012D6E000 (update 1.2.0, SDK 11.4.3)
- NintendoWare Bezel Engine 1.56: nearly all of RomFS is in Bezel Engine archives (.bea, magic SCNE, every entry a zstd frame). shader_split.py reads them (bea_members): 2,125 shaders, most of them the engine's own in System.nx.bea. The 12 geometry shaders of Bezel's line renderer do not build (the main menu chart's outlines are missing); ten pixel shaders that choose their texture handle between two of the game's uniform buffers at run time do not load. The survey names Bezel Engine archives (profile_guess.py, the NSP script's notes)
- nninitStartup picks virtual address memory (a 60 GB StandardAllocator region) or a heap by IsVirtualAddressMemoryEnabled, which the host answered yes for every game. The new profile key sys_resource reports the NPDM's system resource (0 here); the survey writes it for new games, profiles without it keep the 32 MB answer
- SDK 11 calls that reach a service: GetHardwareOpusDecoderWorkBufferSize(int, int), the IR camera (every camera unconnected), LoadCachedNintendoAccountInfo, Enable / DisableIndividualFileDataCache, ldn::GetNetworkConfig and the socket overloads with plain sockaddr
- Movie decoder: the activity previews loop with Flush and Start from four threads. An input index the game held across a flush went back to the free list and was handed out twice; the fifth pending packet overwrote the decoder's mutex and the process died in 2 runs of 5. A send now claims such an index, held indices are withheld for 250 ms after a flush, pend[] cannot overflow, a flush keeps each slot's last picture and an output with no picture leaves the client texture as it was (the black flash at every loop). DM_MOVIE_TRACE=1 logs every decoder call
- The crash handler takes SIGABRT; a command buffer with no GPU start time no longer counts the whole clock as GPU time
- Handheld with Boost (op_mode, perf_mode): handheld, the game reads the touch screen; Boost gives 1920x1080
- 60 fps, sound, previews and saves; the title, enrollment, the menus and activities of every category. About 1.5 GB resident at the first activity, 1.86 GB after 15 minutes of activities (the game's pools as each activity loads). Prewarm list recorded through enrollment and a full activity. Not yet tested on an iPhone
2026-10-22 00:54:39 +08:00
seedds 54eac07b0e NSP script: game language option (--language / LANGUAGE, English by default), checked against the languages in the game's control data and packed as DMLanguage in Info.plist 2026-10-21 23:08:03 +08:00
seedds 85e6dc18e5 The Great Ace Attorney Chronicles plays on the Mac and the iPhone: profile 010036E00FB20000 (base game 1.0.0, SDK 10.4.1)
- Capcom's MT Framework: ARC archives (version 7, zlib entries) under nativeNXx64. Its 460 shaders (230 vertex, 230 pixel) are in one shader package (SPK, sc\NX_64\root in sa/NX_64/root.arc), each a code area followed by its control section; shader_split.py reads it (mtarc_members, mtspk_shaders) and pairs the vertex shaders with the single-stage cover. All build, every binding decodes (Mario's combined layout)
- SDK 10 calls that reach a service, from the import list against the hooks: oe::DisableAlbumImageTakenNotification (the first boot's crash), oe::IsInStoreDemoModeEnabled, vi::ListDisplays, fs::RenameDirectory, account save thumbnails, the ec::ShowShop overloads without an ApplicationId, friends::Start*FriendRequest, and ngc::ProfanityFilter's UTF-8 masking, SetMaskMode and Finalize called by name. nvnDeviceSetIntermediateShaderCache and nvnCommandBufferDiscardColor / DiscardDepthStencil are no-ops
- Black screen: the engine copies its textures into a CPU-cached pool with the CPU after creating them and never flushes (main has no cache maintenance at all): tex_flush_upload and tex_dirty_upload, as FINAL FANTASY TACTICS
- 30 fps (the game's own pacing), sound, H.264 movies, saves, the DLC reported in Special Contents; about 1.1 GB resident, flat over 14 minutes from the title to the first trial
- Prewarm list recorded through the first-boot notices, the title, the opening movie and the first episode's opening scenes
- The survey names MT Framework ARC archives (profile_guess.py, the NSP script's notes)
2026-10-21 20:37:40 +08:00
seedds e63b8eca43 Absolum plays on the Mac into its first level: profile 0100A83022A22000 (update 1.1.1, SDK 18.3.2)
- FNA on .NET NativeAOT with SDL3 and FNA3D's NVN driver; 162 GLSLC outputs in shaders.nx.bin
- Rewriter: data blocks in the text segment are left alone (dm/tools/nso_code.py; main starts with 1.7 MB of data that read as x18 sites); the census applies the same rule. The 52 modules of the 19 other games on hand rewrite byte-identically
- Upside-down picture: nvnCommandBufferSetViewportSwizzles is recorded and a NEGATIVE_Y swizzle flips the viewport
- Logo movie: an H.264 decoder made at Configure (parameter sets in csd-0 / csd-1) now gives RGBA in the native texture mode, not NV12
- SDK calls: vi::CreateLayer with a size; AudioIn (no microphone); the EOS NRO stays unloaded and the game plays offline
- Memory for a 5 GB block: getenv answers the new profile key game_env (DOTNET_GCRegionRange: the GC range 3.75 → 1.5 GB); build cuts for malloc's address space (40 → 2.5 GB), the SDK node area and the SDK's 1 GB alignment of large regions
- Modules: NVIDIA's OpenGL and Vulkan drivers are never loaded (profile_guess.py); the survey's hook list reads dm/gfx too
- dm/host/Makefile: a build for another game always relinks (two builds in the same second kept the first game's host)
2026-10-21 18:44:33 +08:00
seedds 745f39d778 FINAL FANTASY TACTICS - The Ivalice Chronicles plays on the Mac into its first battle: profile 010038B015560000 (update 1.4.0, SDK 19.3.9)
- Shaders: 257 GLSLC outputs in its technique files (resources_nx/.../shader/*.tec); shader_split.py reads GLSLC outputs in any loose file no other reader takes (no other profile's tables change)
- Boot hang in 4 of 6 runs: two job-pool workers share one auto-clear event and a wake reached the wrong one; a cut on this build makes the woken worker pass the wake on (patch_nso.py)
- UI textures written by the CPU into uncached pools: tex_flush_upload + tex_dirty_upload
- Translucent UI yellow: an image the shader reads as uint / int gets a view of that class (img_fit; the texture is upgraded to format views before the next pass)
- Dialog boxes with holes: a game barrier with ORDER_FRAGMENTS ends a draw pass that has shader writes (Metal orders nothing between draws of one pass); reflection marks written images rw
- New NVN entry points: CommandBufferBindTextures, SamplerPoolRegisterSamplerBuilder, TextureWriteTexelsStrided, device item 0x67; a clear mask covering all of a format's channels is whole
- NSP script: the title ID comes from the ticket first (this base NSP's file name is tagged ...0001); thread dumps print each condition waiter's arrival order
2026-10-21 16:20:57 +08:00
seedds 8360a5da9d Shader split runs on Python 3.12: zstd from compression.zstd (3.14+) or backports.zstd
kpka_data (Monster Hunter Rise's PAKs) imported compression.zstd with no fallback; one helper, zstd_mod, now serves it and the Tears of the Kingdom dictionary paths. The NSP script checks for either module up front, and the README's pip line installs backports.zstd only before Python 3.14.
2026-10-21 14:39:13 +08:00
seedds f5646efc47 Uniform updates that cover part of what a draw reads no longer end the render pass; clears with no draw ride on the next pass
Outdoors in Breath of the Wild (the user's Great Plateau save, Mac M4 Pro) a frame had about 386
render passes and loaded and stored about 900 MB of attachments, which an iPhone at 60 fps can
hardly move (about 55 GB/s). 220 of those passes ended only because the game updates the first
64 bytes of a 64 KB vertex uniform block before each draw group (nvnCommandBufferUpdateUniformBuffer
inside the pass) while its shaders read up to 544 bytes of it. A deferred update that covers only
part of what a draw reads used to end the pass.

- Partly covered blocks (nvn_mtl.m ubp_get): the shadow starts from memory and takes every inline
  update queued in the submission, newest last, incrementally per range. It is refused, and the
  pass ends as before, when something else may have written the range by the time the pass runs:
  a copy, fill or feedback write queued in the submission, a storage buffer a shader may write or
  a texture->buffer copy over it (sbw, by 4 KB page), or a GPU write or counter report of an
  earlier submission still running (gw). A counter report of the same submission does not count:
  the host writes reports on the CPU after the whole submission completes (Fire Emblem Engage
  writes its reports into a uniform block's page). DM_NVN_UB_PARTIAL=0 turns it off;
  DM_NVN_UBP_CHECK=1 lands the updates, waits for the GPU and compares every shadow with memory.
- cb_extent.tsv (new tool dm/gfx/tools/cb_extent.py, c1_guard.py's interval arithmetic over the
  translated MSL): per shader, how far into each uniform slot it can read, and the storage slots
  it never writes. Like c1_full.tsv, the host loads it only for its chain version; build_ipa.sh
  copies it, the NSP script writes it for provisional profiles, and the chain procedure
  regenerates it. Without it bindings are copied whole (up to 64 KB). Ten profiles have one.
- Route B's reflection never says whether a storage buffer is written, so every one read as
  read-only and its writes went unnoticed: a storage buffer now counts as written unless
  SPIRV-Cross typed it const (nvn_refl.m sbuf_rw) or cb_extent.tsv proves the slot unwritten.
- Lazy clears (lc_*): a set of targets that gets only clears before the game sets other targets
  no longer opens a pass just to store them; the next pass that attaches the same texture level
  and layer through the same view clears it as its load action, and the rest land as before
  before anything else could see them. Breath of the Wild's four clear-only passes before the
  G-buffer fold into its first pass. Off in TBDR mode; DM_NVN_LAZY_CLEAR=0 turns it off.
- The residency set line counts shadows from memory, refusals by reason, and lazy clears.

On the Mac, old (DM_NVN_UB_PARTIAL=0 DM_NVN_LAZY_CLEAR=0) and new side by side on each game's
route; attachment loads + stores per frame:
- Fire Emblem Engage battles: about 1,850 -> 360 MB, 88 -> 38 passes.
- Tears of the Kingdom: about 1,100 -> 700 MB, 124 -> 116 passes.
- Breath of the Wild outdoors: 386 -> about 158 passes, GPU p50 about 9.3 -> 6.5 ms, submit
  5.7 -> 4.2-4.9 ms; where the old host fell to 30 or 20 fps under load, the new held 60.
- Link's Awakening, 13 Sentinels: about 15% less. Super Mario 3D World: 5% less (lazy clears).
- Odyssey, Three Houses: 2-8% less. Mario Wonder, Metroid Dread: unchanged.

No new red lines in any of them. The shadow check found every shadow equal to the GPU's memory
(BotW 346,000, Engage 284,000, TOTK 86,000), and t_nvn passes 68/68. Screenshots differ only by
timing. Tried and dropped: DM_NVN_TBDR=1 (it turns the deferred updates off), DM_NVN_BARRIER=game
(no gain), and same-target continuation outside TBDR mode (BotW's decals end the pass anyway).

What still splits BotW's big passes: the GPU particles' compute dispatches (about 190 MB a frame)
and decals that sample the G-buffer. Mac shader libraries were built for the current chain of
TOTK, Engage, Wonder, Dread, Odyssey, Link's Awakening, Three Houses and 13 Sentinels (c1_full.tsv
headers already matched). The IPA ZeldaBreathOfTheWild-2026-10-21-passes.ipa (an earlier state of
these changes plus the uncommitted frame-step probes) awaits an iPhone test.
2026-10-21 12:57:48 +08:00
seedds 81b908b904 Frame-rate counter on the iPhone: SHOW_FPS / --show-fps in the NSP script
The app shows the frames put on screen each second in the top row (front_ios.m, from mtl_present_count, so NVN and Vulkan games alike).
build_ipa.sh's DM_SHOW_FPS=1 sets DMShowFPS in Info.plist; DM_SHOW_FPS=0 / 1 in Documents/env.txt overrides it without a rebuild.
2026-10-21 11:12:37 +08:00
seedds 56b806155f One NVN driver layout for every game; shader sources read from the game files
The NVN driver's constant segments have the same layout in every game, so the
nvn_* profile keys are gone. Combined textures are at c2 0x20 + 8 r (a slot past
31 shares the image table; Hades II binds 34), images at 0x120, separate textures
at 0x168 (now always the host's slots 128 + n, as Monster Hunter Rise had them),
samplers at 0x568, and the storage buffer tables at c0 0x110 / 0x310 / 0x510.
dmsh_build.py decodes with that layout, the replay fills both texture tables,
and the translator numbers slots the same way (rebuilt; its SPIR-V and
reflection are byte-identical on 1,491 sample shaders, lock updated).

The shader_* switches and graphics are gone too; the build reads them from the
files (profile_guess.py shader_sources):
- Vulkan when main imports Vulkan and not NVN; its capture when the RomFS has
  no SPIR-V.
- A GLSLC capture when main holds GLSL sources and imports glslcCompile.
- An Oodle decode for Unreal paks, and the tool-mode decoder for Trinity packs.
- GLSLC outputs from every main.
- A cover pairing for each single-stage vertex shader (GLSLC, Unreal).
build_ipa.sh takes GFXAPI from the NSP script.

Also removed: the last D2R pieces (the embedded-shader path in dmsh_build.py,
dmsh_facts.py --freeze / --check, maxwell_refs.py, ref/hle_hid_d2r.c).

Chain: every profile's dmsh_chain.h is regenerated. The c1_full lists are
carried over: the change touches no c1 read, and the rebuilt Octopath
Traveler 0, Hades II, Unicorn Overlord and Rhythm Heaven Groove gave the same
lists, identical tables and the same reflection apart from separate texture
slots + 128.

Still to do:
- Run the chain procedure for the other NVN profiles.
- Engage's and Three Houses' shader tables need a new split: Engage gains
  Unity's 11 GLSLC shaders in main, and Three Houses' 17 BNSH vertex shaders
  lose their cover pairs.
- Ace Attorney's new tables are in, but its libraries are not rebuilt.
2026-10-21 07:23:10 +08:00
seedds d34e0446aa Pokémon Legends: Z-A plays on the Mac: Trinity packs through the game's Oodle, transform feedback, stencil views, the base instance in the driver segment
- Profile 0100F43008C44000 (2.0.2, SDK 19.3.11): 5 GB layout with two cuts (StandardAllocator 12 → 4 GB, node area), mem_map_limit (the pool enforced as on Horizon), shader_oodle.
- Shaders: Trinity packs (arc/data.trpfs, FlatBuffers, Oodle Mermaid) decoded with the game's own OodleLZ_Decompress in the Mac host; 28,603 shaders.
- SDK 19: GetAudioOutBufferCount, crypto random bytes, ToPosixTime with a rule, NVN state getters, CommandBufferFenceSync, the 64-bit WaitForAddress type.
- Graphics: transform feedback (vertex capture entry point, rasterizer discard), layered passes (the colour grading LUT), point size dropped beside a layer output,
  function constants specialized, a stencil-only view for STENCIL-mode texture views (striped lighting), sampler min / max reduction (Metal 26),
  the draw's base instance at c0 + 0x644 (occlusion boxes; characters' legs folded back from the hips), the zero cube for a 2D texture in a cube slot.
- Shader chain changed (dmsh_build.py): every profile's dmsh_chain.h rewritten; the other profiles' c1_full lists carried over (that change touches no c1 read).
2026-10-21 03:47:16 +08:00
seedds 9923c5a032 NVN API version and shared-font layout follow the SDK; Oodle found by its name string
- The loader reads the SDK version from the sdk module's NintendoSDK_nnSdk string
  (dm_sdk_major). SDK 10 and older report NVN API 53.313 and point the shared-font
  address at the TTF itself; later SDKs 55.15 and the BFTTF header. That is what
  every profile set, so device_api and shared_font leave the five that had them
  (Odyssey, 3D World, Link's Awakening, Age of Calamity, Three Houses) and
  profile_guess.py no longer guesses them; a profile can still override.
- DM_OODLE_JOB finds OodleLZ_Decompress as the one function in main that takes
  the address of the library's own name string, with its start from
  .eh_frame_hdr, instead of an address per main build: Octopath Traveler 0
  main+0x4f77e10 and Pokemon Legends: Z-A main+0x1a9c9e0. Octopath's 10,596
  shaders decode byte-identical to before.
- Rows that only restated defaults are gone (pad_layout simple, graphics nvn,
  present_interval game), and the NSP script writes no default rows into
  provisional profiles.

No chain version changes. All 19 profiles build, the host tests pass; Mac boots
of 3D World (SDK 10.4.0: API 53.313) and Engage (SDK 12.3.7) read the version.
2026-10-21 02:34:46 +08:00
seedds 79d82f7b64 Remove the D2R code and the legacy frame step
D2R (Diablo II Resurrected) has no profile and its code could not run here: the
host-call svcs its cuts emitted only died, the touch-run-v4 rule set needed a
missing dm/PRODUCT, and the host files its hooks called were never in this
repository. Removed with it:
- rewriter: D2R cuts, observation gates and parameter slots, the touch-run-v4
  rule set and private gates, the boot list, the cut scope column and the
  KRES / KASPECT key slots (fixups.tsv keeps the four thread slots; no
  params.tsv any more); dm/tools/product.sh
- host: D2R save edits, add-on content, render-scale controller, time()
  takeover and Terror Zone clock, crash decoding at D2R offsets, host-call
  registry and stubs, i18n, UI-screen, shell-app loader, zero-stripped
  archives, boot profiling hooks with no implementation; D2R headers, tests
  and scripts; dm/ios/Sources/dm_touch_controls; dm/third_party/libdeflate
- graphics: runtime extrapolation, world freeze, iPad UI shift, shop drag
  image, brightness LUT, visual ruler, bloom / FXAA shader hashes, DM_GFXUSE,
  the eco-fps setting, the loading-screen gate; shader_map.py,
  nometal_run.py, nvn_slots.py, nvn_slots.tsv and the PC-texture samples

The legacy frame step (frame_step legacy, DM_FRAME_STEP) is retired: the
Breath of the Wild and Tears of the Kingdom stubs lose their mode test.
Breath of the Wild's interval-controller cuts stay: the framework keeps the
controller's interval while the game's VFR override holds.

patch_nso.py output is byte-identical for all 15 games on this machine except
the two frame-step stubs; all 18 profiles build; the host tests pass (their
Makefile now builds the general tests only).
2026-10-21 02:09:49 +08:00
seedds 9d941f8604 Rhythm Heaven Groove plays into its first rhythm game on the Mac: SDK 21 calls and audio, pool initial content, NVN block height
New profile 0100D9F01D474000 (base game 1.0.0, SDK 21.4.0, Nintendo's own 2D engine; one BNSH of 35 shaders,
Mario's binding layout). On the Mac it plays from a first boot through the setup, the main menu and Play Solo into
Hoop Trundling at 60 fps with sound and no red line. Not yet run on an iPhone.
- SDK 21 calls: nn::pl::GetSharedFont (every shared font at once; the SDK opened the pl service and aborted in
  sm::Initialize) returns the stand-in TTFs past the BFTTF header, as the service's offsets point there and the game
  hands them straight to ScalableFontEngine::LoadFont; vi::GetDefaultDisplayPanelIdentifier (SIGSEGV through the vi
  service object) answers an all-zero panel.
- Audio: SDK 21's DSP command header has no magic (+0 enabled, +1 command ID, +2 size) and DeviceSink is 0x140
  bytes with its fields 8 bytes earlier. capture() recognizes that layout on the first non-empty command list (the
  walk bounded by the generated list's length); the game was silent.
- Pool initial content (all games): a new texture in a CPU_NO_ACCESS pool whose storage holds data is filled from it
  once, as NVN keeps a pool's storage content at nvnMemoryPoolInitialize (tex_init_fill); it stays a GPU texture
  that can be rendered into. Linear textures are made on the pool buffer in any pool. The game reads each texture
  file into memory and makes a CPU_NO_ACCESS pool on it; every texture read 0 and the screen stayed black.
  COMPRESSIBLE, multisample, depth / stencil and sparse textures and storage of zeros are left out.
- NVN block height (all games): the automatic block height truncates height in blocks x 1.5 / 8 (MacNativeGames'
  finding, confirmed on this game's 40-block-row BC7 textures and by Touchable on 16 of its files); rounding up
  scrambled the calibration screen's beat numbers. Only data a game lays out itself without a packaged layout
  changes (Tears of the Kingdom's flushed and Monster Hunter Rise's sparse textures were not re-checked).
- Profile key perf_mode (follow by default): boost reports Boost whatever the operation mode. This game plays
  handheld with Boost: docked means a TV to it, which asks for a latency calibration, and Boost alone picks its
  1920x1080 draw size.
- The NSP script's survey no longer calls nn::ro missing (hle_ro.c answers it; it now counts the .nro files);
  sdis.py --callers takes imported names and scans to the end of the PLT.
The dm/gfx unit tests give the same results as before; Unicorn Overlord renders the same with the old and new host
(no texture takes the initial-content path there).
2026-10-21 01:21:39 +08:00
seedds 476b1135b4 Octopath Traveler 0 boots on the Mac: Unreal pak and Oodle shader decode, the host's allocator for Unreal
Octopath Traveler 0 1.0.5 (Unreal Engine 4.26 / 4.27 mobile renderer, SDK 19.3.13) plays from boot through character
creation into the story on the Mac (15 minutes with the iPhone memory layout, no crash); not yet run on an iPhone.
- Shaders: the 10,324 are in the two shader code archives of the game's pak, each one Oodle-compressed. uepak.py reads
  the version 11 pak (AES-256 index; the key is the window of main that decrypts it, never printed or written; AES
  through CommonCrypto) and the archives; the Mac host decodes them with the game's own OodleLZ_Decompress
  (DM_OODLE_JOB, hle_oodle.c, build-gated) and shader_split.py reads the decoded LSLG records (profile key
  shader_ue_decode; the NSP script's decode_ue_shaders stage). Bindings measured: Engage's separate texture layout.
- Memory: Unreal's nninitStartup reserves a 62 GB address region and its FMallocBinned3 37 GB of it. A cut skips the
  allocator's constructor and the host puts its own FMalloc vtable on a malloc zone in the object (hle_uemalloc.c);
  the region is capped at 2 GB with the game's own -SwitchVirtualAddressSpaceSize (new profile key game_args, also
  for the iOS app) and SDK 19's node area is cut to 62 MB as Engage's, for a 3.75 GB guest block.
- Host: game arguments are set before nninitStartup, as on hardware; an undefined weak import nothing defines binds
  to 0, as rtld binds it (Tears of the Kingdom boots the same with the old and new hosts: its weak glslc* imports
  now resolve to 0); nvnCommandBufferFenceSync (OP_FENCE_SYNC), the texture / sampler pool getters,
  vi::GetLatestFrameNumber / ListFrameHistory; DM_ALIAS_WINDOWS for sizing fixed-window allocators.
- Rewriter: a byte jump table whose add sits 17 instructions before its br (jump_bytes_far, tried last).
- build_ipa.sh quotes FONT_PY (a python path with spaces failed the font step).
Open: 61 vertex shaders pick a texture slot at run time (dynamic handle), 16 compute shaders with 3D image atomics
do not build. The profile (dm/games/01005270232F2000) holds the shader tables, translation chain b-ba76ba138e78.
2026-10-20 23:01:20 +08:00
seeddsandarmyja e437e80ab6 Ace Attorney Investigations Collection boots on the Mac: Unity 2022 shaders, the C library's file checks and clock
New profile 010005501E68C000 (update 1.0.0.2, Unity 2022.3.17f1 IL2CPP, SDK 17.5.3; the base game an XCI):
- Rewriter: two IL2CPP thunks end in an indirect tail call, ldr x18, [x0, #0x18]; ...; br x18 (main+0x1d271bc,
  +0x204ebec), which asserted as an unknown jump table. When neither table form is found, the last write of x18 is a
  plain load and the br ends the function, it becomes x16 = guest x18; br x16 (tail_call / site_br_call). Tried only
  where the rewriter used to stop, so modules it accepted before come out byte-identical.
- sdk cut: the AddressRegionManager node area 1 GB -> 62 MB on this build (sdk+0x139468 / +0x1394fc), as for Engage;
  StandardAllocator::Initialize aborted in nninitStartup without it.
- Shaders (unityfs.py, shader_split.py): Unity 2022 keeps each program's parameters as blob entries of their own
  (m_ParameterBlobIndices; the programs are m_PlayerSubPrograms[tier][].m_BlobIndex), and subprogram version
  202012090 (from 2021.2) has no local keyword list. The player's files borrow the unitybuiltinshaders bundle's Shader
  type tree. 4,951 shaders, 4,950 build (one geometry shader); Engage's layout of driver segments decodes every binding.
  UnityPy's type trees for 2020.3.18 and 2021.3 have neither field, so Engage reads every entry as before.
- hle_fs.c: nnmusl (open / stat / opendir) checks every path with nn::fs::IsPathMounted, GetPathLengthMax,
  GetUtf16PathLengthMax, GetEntryNameLengthMax and GetUtf16EntryNameLengthMax, which the SDK answered from its empty
  mount table: File.Exists failed with no nn::fs call, Addressables never found settings.json and the game crashed on
  a null asset (main+0x31100ec). Answered from the host's mount table with Horizon's 0x300-byte limits.
- hle_time.c: clock_gettime(CLOCK_REALTIME) reads the source nntimeStartup selects (sdk+0x8e414),
  nn::time::GetStandardUserSystemClockCurrentTimespec, which asked the time service; system_clock::now() then called
  std::terminate after the first dialog. It and the adjustable clock's getter are the host clock now.
- Memory: Engage's 5 GB block with a 1 GB application pool. StandardAllocator commits 0.19 GB; with a 2.5 GB pool
  its reservation and Unity's own pools reached the end of the 4.375 GB alias region after 140 s (probes on
  VammManager::AllocateAddressRegion / FreeAddressRegion). 25 minutes now peak at +2.81 GB.
On the Mac: first-boot settings, title, Investigations 1's Episode 1 into the first investigation, 60 fps, FMOD sound,
25 minutes with no red line, 1.55 GB resident and 1.25 GB of Metal allocations, flat.
IPA AceAttorneyInvestigations.ipa (7.0 GB) built from HEAD + these changes; awaits an iPhone test.

Also: UNIMPL stubs log once (the seen counter was a byte and wrapped every 256 calls); build_ipa.sh quotes FONT_PY
(a Python path with spaces broke the font step); profile_guess.py marks IL2CPP by global-metadata.dat (Unity 2022's
main has no il2cpp_ names). Not needed in the end: pull requests #15 (this main's string table is intact) and #16
(nninitStartup sizes no heap with virtual address memory).

Much of the bring-up was found first by armyja (GitHub issues #17, #18, #19, pull requests #15, #16).

Co-authored-by: armyja <546323405@qq.com>
2026-10-20 19:55:55 +08:00
seedds c01fad8cdb IPA builds: parallel packer, reuse of unchanged stages
- dm/ios/tools/pack_ipa.py replaces the Payload/ copy and Info-ZIP: it packs the app, the shader
  libraries, the add-on content and the RomFS straight from their folders, reading and deflating on
  all cores (zip64, modes, symlinks, the profile's zip_store as before). Metroid Dread 156 s -> 12 s,
  Breath of the Wild 171 s -> 57 s (the T7 writes at about 310 MB/s).
- The shader libraries no longer go through Xcode (Gen/msl left the project): no 1.5 GB detour
  through the home disk.
- NSP script: the rewrite, the shader split and the iOS / Vulkan shaders are reused when a sha256
  of their inputs matches the one stored beside their output; the modules are rewritten in
  parallel; J = the core count; the extraction is kept by default (--no-keep-extracted).
  A Metroid Dread repeat build: about 8 min -> 19 s.
2026-10-20 09:17:17 +08:00
seedds 8d7f957fa2 Monster Hunter Rise: plays on the Mac into the opening; translator phi fix, all chains rebuilt
- Profile 0100B04011742000 (update 16.0.2, RE Engine): KPKA PAK shaders (shader_split.py kpka_jobs), XCI base game in nx_extract and the NSP script, BIG_TEXT patch area in patch_nso.py.
- Sparse virtual pool: a 4 KB page table (vpt_*) that the replay reads textures through, dirty tracking per mapped page, texture views from the first mapped level.
- MultiDraw*IndirectCount with a CPU wait for the count; pending GPU writes indexed by page; bindless handles read from queued inline updates.
- sbuf_ubo binding kind, integer vertex formats into float inputs, clears on textures that are not render targets.
- Inline keyboard text, state and finished callback; ngc::ProfanityFilter host vtable.
- Ryujinx BindlessElimination follows phi webs (218 MHR pixel shaders); dmsh_xlate_bin and dmsh_chain.lock updated; dmsh_chain.h and c1_full.tsv rebuilt for every Maxwell profile.
- README: Monster Hunter Rise under being ported, gallery image, XCI base games, Ryujinx local fixes.
2026-10-20 07:17:31 +08:00
seedds 6563bb1990 NSP script: provisional profiles read their switches off the game files; a short heap check
dm/games/profile_guess.py decides the profile keys a game's files show, each written with its reason:
graphics vulkan and the modules to load (a subsdk only when main takes something from it the host does
not supply), shader_vk_capture / shader_main_glslc / shader_separable / shader_glslc_capture, the memory
partition (giant bss, no heap, virtual address memory), the SDK 8 / 10 font and NVN API, Hagi's handheld
pointer. --check compares the guess with every profile in dm/games: all 13 agree on these keys.

After the rewrite the script sets mem_used from the rewritten modules (Age of Calamity's 72 MB), writes
the empty shader tables and chain header a provisional Vulkan game lacked, and boots a game with virtual
address memory that sets a heap until its heap request to size the heap region (about 7 s; Tears of the
Kingdom gets its 0x98000000 alias region). A minute's boot could not tell which games need render_cond,
tex_flush_upload or core_preempt, so they stay with the developer.

--provisional builds a game that has a profile from a guessed one (DM_GAMES_FIRST in game.py and the host
Makefile, whose profile header is now rewritten only when it changes); --no-heap-check skips the check.
A capture run whose host stops early now says so. hle_svc.c logs a heap request the pool check refuses.
2026-10-19 18:50:50 +08:00
seedds c5955d42a6 English only: no Chinese left in code, scripts, logs, tables or docs
- Log messages, script output, test labels, Makefile errors and comments are English. Red log lines start with "[RED] " (was 【红】).
- Shader map stage column: vertex / pixel / compute / geometry / hull / domain (generators, dmsh_build / dmsh_facts and nvn_mtl.m stage_of follow).
- The NSP script's regexes match the new shader_split.py / dmsh_build.py summary lines; patch_nso cut scopes are all / host-only.
- Host UI text table (dm_i18n_tab.h) is English only; dm_lang_zh() removed.
- Text that matches game data (t_swkbd.c, t_textbox.c) is written as \x escapes.
- Shader chain: the chain files lose their bilingual comments, so every profile has a new chain version. dmsh_xlate.dll rebuilt from the translated Program.cs (SPIR-V byte-identical on 4,500 shaders of Wonder, TOTK and Age of Calamity), lock and dmsh_chain.h headers regenerated. c1_full.tsv restamped for Dread, 13 Sentinels and Wonder; the other profiles follow when their rebuild finishes.
- AGENTS.md and docs/agents: the language rule now covers logs; quoted log lines updated.
2026-10-19 15:34:57 +08:00
seedds c162b95a93 Super Mario Galaxy: profile 010099C022B96000 (1.3.1, SDK 18); plays on the Mac
The Wii game on NERD's Hagi layer (PowerPC code recompiled into main), rendering
with Vulkan through Hades' layer. Title screen, file creation and the storybook
prologue into the Star Festival at about 58 fps on the Mac; the IPA awaits an
iPhone test.

- Vulkan layer (hle_vk.m): push constants, descriptor arrays, dual-source
  blending variants, copies inside a render pass (cut and resume with deferred
  store actions), linear images over their memory, timestamp and occlusion
  queries, pipeline caches, vkCmdDraw, image copies and read-backs, USCALED /
  SSCALED and three-component vertex formats, viewport and scissor kept across
  passes, the INHERIT swapchain transform, the vi layer crop at present.
- vk_shaders.py: ArrayStride on struct members, push constants, array counts,
  `_d` dual-source variants, extra SPIR-V input dirs; non-vert/frag stages
  skipped with a red line.
- Run-time SPIR-V capture: DM_VK_CAPTURE in the layer, profile key
  shader_vk_capture and vk_capture.tsv (3,089 keys), and the NSP script's
  Vulkan shader capture stage.
- Handheld pointer: profile key op_mode, touch hooks fill touches only in
  handheld mode, InitializeTouchScreen(int,int), DM_KEYS taps, iOS gamepad
  forwards untaken touches to the game.
- SleepThread under 20 us takes the yield back-off (Hagi's 100 ns poll kept a
  core at 100%).
- nx_extract / NSP script: --allow-damaged for NSPs whose files fail the NCA
  hash check.
2026-10-19 14:35:31 +08:00
seedds dacae6966e README: Hades, Age of Calamity and Three Houses playable on the iPhone; NSP script packs Vulkan games
Hades, Hyrule Warriors: Age of Calamity and Fire Emblem: Three Houses move into the Games table and
the screenshot gallery (Mac captures); Tears of the Kingdom is listed as being ported. The NSP script
now handles a graphics vulkan profile: vk_shaders.py translates the SPIR-V shaders for the Mac and iOS
in place of the NVN shader split and chain build (the change from an uncommitted worktree, ported to
the current script; Hades packed with it: 74 shaders, 3.6 GB IPA). AGENTS.md and the development
notes follow.
2026-10-19 01:28:33 +08:00
seedds 840171d448 TOTK's 3D scene renders; BotW issue #9 memory and 30 fps fixes; in-pass clears; GLSLC capture in the NSP script
Tears of the Kingdom (two profile keys, off for every other game):
- render_cond: NVN conditional rendering is evaluated at replay from the
  timestamp reports' counter 0; before, every conditional clear ran and the
  lighting passes drew nothing.
- tex_dirty_upload: textures the game fills from the CPU without an NVN flush
  (its 121-layer ASTC terrain arrays) are found through write-protected pages
  and uploaded; before, they stayed empty (magenta in ASTC).

Breath of the Wild, issue #9 (iPhone 16 Pro Max killed for memory in the first
shrine; an 8 GB iPhone gives the app 3,375 MB):
- Finalized textures' serials no longer stay in the serial table for good:
  live serials are marked at nvnTextureInitialize, dead ones resolve through a
  512-entry table (about 10 a frame in BotW before, 276k after 25k frames).
- The memory-pool tracking table grows past 4,096 entries (BotW reached 4,006).
- Most of the growth was the defer_drain leak fixed in 0fc0ce4: the shrine now
  stays at 2.8-2.9 GB resident on the Mac, about 500 MB under the limit.
- Falls back to 30 fps at the right speed when the GPU can't hold 60: two cuts
  make the frame-interval controller always use its GPU-time path, and the
  profile key gpu_timestamps segment gives each timestamp report its
  segment's GPU start / end (the default, submission, is unchanged). Under
  load, interval 2 held at 0.93-0.98 of real time (0.78-0.95 before). The
  check that it still holds 60 on a quiet machine is pending.

NVN: clears inside a pass, with a partial channel mask or a partial scissor
are drawn as a triangle in the open pass instead of dropped, and 8-bit indices
written by the GPU earlier in the submission are widened on the GPU (TOTK red
lines 6 / 7 / 48). Adapted from MacNativeGames (MIT, dm/third_party/macnativegames).

NVN vertex fetch: 8 / 16-bit integer attributes fed to an input of the other
sign are widened by their own sign (CPU conversion, red 96 once); before, the
signedness swap widened them the other way and Three Houses' terrain heights
below 0 left the screen (no ground under close-up cameras). The pass dump's d
line shows polygon offset and depth clamp.

NSP script: a GLSLC capture stage for profiles with shader_glslc_capture
(Three Houses' 9 run-time shaders), so only the NSPs and prod.keys are needed.

AGENTS.md: only the coordinating session runs the multi-game smoke test.

Each part was checked on its own game on the Mac. The multi-game smoke test
runs after this push (Wonder and Engage already passed on the same changes
minus the vertex-fetch fix). The translator fix and its chain rebuild are not
in this commit.
2026-10-18 22:30:11 +08:00
seedds b886c3f0e5 NSP script: one progress line per stage, with a bar and ETA
scripts/build_ipa_from_nsp.py shows [k/n] stage · step. Where the work can be counted, the line has a bar, done/total, percent, elapsed time and ETA: bytes extracted, modules rewritten, shaders (cache check, translation, compile, read from the files dmsh_build.py writes), files compiled by Xcode, and bytes copied and zipped into the IPA. Elsewhere it shows a spinner. In a terminal or Spyder the line is redrawn in place; piped output gets a plain line every BEAT (30) seconds. PROGRESS / --progress auto|bar|lines|off picks the mode. Tool output now goes only to <id>_build/logs/; a failed step prints its log's last 40 lines and the log's path. No chain-hashed file and no build_ipa.sh change.
2026-10-18 16:36:03 +08:00