Commit Graph
166 Commits
Author SHA1 Message Date
seeddsandparsifal295 2db12f7efd Fix vibration handles before controller haptics routing (#246)
GetVibrationDeviceHandles used routed haptics availability to decide whether
handles existed. Games that cache this answer at startup could retain an empty
answer after a controller connected. Return handles for a connected npad style
when rumble is enabled and an output sink is installed; check routed outputs
when values are sent, keeping the existing permission and lifecycle behavior.

Include the PR's rumble diagnostics, count unroutable values individually in
mixed batches, and protect aggregate counters across game threads. Correct the
code comment to describe Unicorn Overlord's observed empty handles and zero
values without claiming restored physical rumble.

Add regressions for cached startup handles delivering a nonzero value after
routing, diagnostic totals and concurrent sends. Include HID in the existing
ThreadSanitizer target and record the mechanism in the service reference.

Checked with GAME=010069401ADB8000: full host suite, 73 HID checks, audio and HID
ThreadSanitizer checks, docs checks, Mac host build, and iOS compilation
(133 files). Physical rumble and iOS runtime remain untested. Broader tool
validation is incomplete because Capstone is missing and iconutil failed.

Co-authored-by: parsifal295 <130349765+parsifal295@users.noreply.github.com>
2026-10-26 09:54:43 +08:00
v0lumehi 037ff69562 Profile keys present_snap and pso_mtl4: a game ships the presentation and pipeline settings it was tested with. present_snap on: present snapshots by default (a ring of 2 host textures; the iOS launcher's Frame buffering defaults to it; DM_PRESENT_SNAP and Documents/env.txt override); pso_mtl4 on: render pipelines through MTL4Compiler on iOS by default, with the pipeline archive, the prepare step and the hybrid pipelines (on by default on the Mac; DM_PSO_MTL4 overrides). Breath of the Wild sets both: they held a steady 60 fps on an iPhone 17 Pro Max (issues #142 / #159; without the snapshots the GPU idled between frames and iOS lowered its clock, 30 fps). README, docs/development.md and docs/reference/frame-rate.md say the new defaults. Checked: the host builds for Breath of the Wild (DM_GAME_PRESENT_SNAP 1, DM_GAME_PSO_MTL4 1) and Mario Wonder (0, 0), dm/tools/tests pass, t_present_snap and t_nvn pass with Breath of the Wild's profile, the iOS compile check passes 2026-10-26 01:43:25 +01:00
v0lumehi f6bd56f5cc The pipeline prebuild runs in the prepare step: with a window (and on the iPhone) the bases of every shader pair are made right after the prewarm, before the game's first frame, behind the progress screen ("Preparing pipelines"), once per chain version, depth mode and pair list (prebuilt.tsv). Built in the background while playing, they took an iPhone 17 Pro Max's GPU from about 13 to 15-18 ms a frame in Breath of the Wild (the compiler's load on the shared power budget: 43-95 dropped frames per 300 against 0-25 without it); without a prepare step the background prebuild stays. Tests: t_nvn 128, t_pso_archive 57, t_pull 346, iOS compile check 2026-10-26 01:43:20 +01:00
v0lumehi 1eeb109b35 docs: the prepare step (docs/development.md "The prepare step", Breath of the Wild's measurements in its note dm/games/01007EF00011E000/README.md; dm/gfx/README.md DM_PSO_PREPARE; README). Mac, M2 Max, runs started at a 1-minute load below 4, Metal cache empty, the prebuilt archive, the Zora's Domain save: with no list, 4 frames over 25 ms with new pipelines and 5 with shader library loads (75 libraries, 168 ms of that work, worst 128 ms) in the 60 s after the last menu press; with the new pso_manifest.tsv none, whether it was built in the background (14.6 and 16.0 s on one worker, done during boot) or by the prepare step (5.2 and 2.2 s on 12 workers, 3.3 s on 8, 872 bases compiled); a second start prepares it in 0.08 s and a player's 5,783-line list in 0.23 s, both with no screen. Hades (Vulkan) is unchanged: it builds its 1,856 pipelines itself while it loads 2026-10-26 01:43:20 +01:00
v0lumehi e0ef4e707e Pipeline prebuild: the pulling base of every known shader pair into the pipeline archive, with the pairs of agl's BAHS program variations (pso_pairs.tsv)
A pulling stand-in is used only when its base is made or archived, so a cold start's draws waited until the prebuild puts the base of every known pair into the archive (pso_prebuild): in the app after the prewarm, on a utility queue on half the performance cores, flushing every 1,000 bases (DM_PSO_PREBUILD=0: off), or as a tool on the Mac (DM_PSO_PREBUILD=tool: build, write, exit; DM_PSO_PREBUILD_DEPTH=zero, DM_PSO_PREBUILD_THREADS=<n>), once per chain version, depth mode and pair list (pso_archive/prebuilt.tsv). The NSP script runs the tool in a --platform mac build (pso_prebuild.log). Files that hold only bases without a vertex layout are named p<seq>.mtl4a; a base with a vertex layout is never looked up in them (each file a lookup misses in costs about 0.06 ms) and compaction leaves them alone.

The pairs: shader_pairs.tsv paired the stages of one BNSH variation and none of the pairs in agl's BAHS shader archives (.sharcb: the agl / gsys / terrain resource packs), and in Breath of the Wild 109 of the 543 pairs drawn on entering Zora's Domain came from those, so they had no stand-in. shader_split.py now reads a BAHS file's binary array (record size, kind: 0 vertex, 1 fragment, 3 compute, the entry bahs_shaders reads) and program array (record size, name length, stage mask, first binary index), and pairs the n-th vertex binary of a program with its n-th fragment binary (bahs_variations); a file whose arrays do not read back gives none. Those pairs go to a new table, pso_pairs.tsv, which only the prebuild reads (DM_PSO_PAIRS: another path): vs_pad keeps reading shader_pairs.tsv, so no shader library, inputs digest or other table changes. Breath of the Wild: 3,012 more pairs, and all 543 pairs drawn are now known; its shader_map / facts / pairs tables split again byte-identical. A profile made before the table existed still packs: the NSP script compares pso_pairs.tsv only where the profile has one, and otherwise keeps it beside the shader package and hands it to the prebuild. Seven other profiles have BAHS files (Odyssey, Wonder, Mario Kart 8 Deluxe, 3D World, Animal Crossing, Captain Toad, Tears of the Kingdom); their tables are made at their next pack.

Breath of the Wild, Mac (M2 Max), Zora's Domain save, empty Metal shader cache, no pso_user.tsv. Prebuild: 15,767 bases, 910 MB in 16 files, 187 s with 7 workers for the first 12,755 and 19 s for the 3,012 more; the files open in 2.2 s cold, 0.25 s from the file cache, and add about 850 MB of footprint. The frame entering the area: 340 ms with pso_pairs.tsv, 830 ms with shader_pairs.tsv alone, 990 ms through the MTLDevice (then frames of 738, 535 and 496 ms); draws that waited: 50 (20 ms), 102 (650 ms) and 512 (3,325 ms). The picture is the same.

Tests: t_pso_archive (41 checks: also a base without a layout flushed into a p file, and a base with a layout not looked up there); the other graphics tests, the host unit tests and test_nsp_mac / progress / chain give the results they give on main. Not run: an iPhone (the prebuild is off there with the Metal 4 path), the NSP script's pack end to end, the other BAHS profiles' splits. docs/development.md, dm/gfx/README.md and dm/games/README.md describe the prebuild and the table; Breath of the Wild's note (dm/games/01007EF00011E000/README.md, Pipelines on a cold start) has its measurements.
2026-10-26 01:43:20 +01:00
v0lumehi 8593d85a9c docs/development.md: a diagram of the hybrid miss path (cache, base and specialization, pulling stand-in with the draw's own pipeline queued first, or a build at once with the bounded prediction) 2026-10-26 01:43:20 +01:00
v0lumehi bc76cab190 Hybrid pipelines: a draw whose pipeline is not built yet draws with its shader pair's pulling pipeline meanwhile, on main's demand-first pipeline builds
On a miss the draw first takes its pipeline without compiling (its base made or in the archive, then a specialization). Failing that it draws with the pair's pulling pipeline (<key>_bp: no vertex descriptor, the streams read through a per-draw layout block at buffer 16, pull_bind), specialized with the draw's colour state, while its own pipeline is built in the background; a later draw takes that one. Both compute the same positions (t_pull_inv), so a draw that switches does not flicker or z-fight. Only a pulling base already made or archived is used; otherwise it is built in the background for the pair's next miss and the draw waits. Merged passes, readable-tile and diagnostic fragments, and layer-writing or transform feedback vertex shaders have no stand-in. DM_PSO_HYBRID=0 turns it off; it needs the Metal 4 path, so it is on by default on the Mac only.

On main's demand-first pipelines (#75): pso_pend_kick stays the speculative admission (at most two running, at most two per prediction, the oldest finished entry reclaimed when the table is full). A draw with a stand-in queues its own pipeline before the prediction runs (pso_pend_demand): outside the speculative limit, on serial queues at utility QoS, one per two performance cores (pp_bg_queue), so a demanded build never waits behind speculation and speculation never waits behind it. The prediction spends its budget only on draws that will have no stand-in, since only those would wait. A draw that would wait for a build still queued makes the pipeline itself at once (pso_pend_take; Breath of the Wild's cold start waited 0.5-0.9 s for one pipeline behind a burst before). The first miss of a submission loads the shader libraries the rest of it binds in parallel (shader_prefetch_rest): 248 libraries in about 210 ms where one by one on the submit thread took about 1 s, in the hybrid and the waiting path alike.

Diagnostics: a "pipelines up to frame" line every 300 frames (submit-thread pipeline, library and prediction time, archive lookups, specializations, base compiles, hybrid draws, background builds, the draws that waited and why), DM_PSO_WAITLOG=1 (each wait over 20 ms with its pair), DM_PSO_PULL_ALL=1 (every draw with a pulling pipeline draws with it), and the longframe line's count of hybrid draws.

Breath of the Wild, Mac (M2 Max), steady state in Zora's Domain: mean GPU p50 over frames 2,400-5,100 was 8.98 ms with the hybrid, 8.98 ms on the Metal 4 path without it, 9.35 ms through the MTLDevice. Cold-start numbers are with the prebuild of the next change.

Tests: t_pull (346 checks, now also the layout block the host fills), t_pull_inv, t_pso_pending (48) and t_pso_archive (32) pass; the other graphics tests and the host unit tests give the results they give on main. docs/development.md ("Hybrid pipelines") and dm/gfx/README.md describe it.
2026-10-26 01:43:20 +01:00
seeddsandAleksandr Svidchenkov 796d592aa8 Test and document NVN prewarm key normalization
Exercise historical manifest rows through the production warmer, shader loader,
pipeline builders and draw-time cache. Check normal and merged passes on both
MTLDevice and Metal 4, with GPU pixels compared against the original descriptors.
The regression fails on the PR base and when the merged-pass guard is removed.

Document the shared normalization rule and keep game measurements in their notes.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:52:46 +08:00
seedds b5c78fbddb Match verified game patches across compatible updates 2026-10-25 23:21:17 +08:00
seedds 7a3b95873b Fix Mac app missing-module checks and quoted alerts 2026-10-25 19:49:34 +08:00
StanandClaude Opus 5.5 4213e5ea79 EoW A-2: docs for the Mac app (--platform mac-app)
docs/development.md: a new section "A Mac app" after "A Mac-only build for development": what --platform mac-app builds and where the app goes (-o), the bundle's layout, APFS clones or copies, where the save, pso_user.tsv, temporary files, env.txt and the log go, signal forwarding and the alert, why the launcher runs the host as a child process (macOS privacy checks on removable volumes: a launcher that exec'ed dmhost got EPERM on every open on the external drive, signed or not), and signing (ad-hoc or --mac-sign, never the hardened runtime, so no notarization). "A Mac-only build for development" no longer says there is no Mac app. The script's examples get one for --platform mac-app --mac-sign, and a line lists the three --platform values. README.md "Pack an IPA": one line pointing at the Mac app.

Docs only; no test ran for this commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 19:49:21 +08:00
seedds 9116109c9f Expand CI with texture, sanitizer and unsigned iOS build checks 2026-10-25 19:08:45 +08:00
seedds 846c54c63c Per-game notes beside each profile: dm/games/<id>/README.md for all 58 games, and the games table in dm/games/README.md. Each note says what the profile needs and why (a table of its game.tsv keys, patches.py cuts and frame-step stubs, sdk_patches.py cuts for its sdk build, manual shader rows), the game's own findings by topic (frame rate, memory, rendering, audio, movies, services, files and shaders, input, Mac routes), each marked current or resolved, a "Found on this game" list of the general host changes first needed there (one line each, linked to where they are explained), and what is still open, including stale profile comments and figures the sources disagree on. The notes gather what was spread over AGENTS.md, docs/development.md (Game notes and Power), docs/agents/01-05 and the profiles' own comments. The Game notes section of docs/development.md (523 lines, 53 games) is now a pointer to the table; development.md goes from 33,300 to 8,600 words. README.md, AGENTS.md and the Breath of the Wild link in 04-graphics.md point to the notes. A README.md in a profile folder changes no shader cache key (dmsh_chain.inputs hashes only the named tables and keys) and no profile listing (game.py lists folders with a game.tsv). Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (five file patterns now appear as the full names they stood for), and every relative link and heading anchor in the repository's Markdown resolves 2026-10-25 12:28:47 +08:00
seedds b7dfba4dd0 Load LZ4 only when decoding NSO files
The memory-profile host test imports profile_guess, which imported nso_syms and required LZ4 even for plain log parsing. The host CI job has no third-party Python packages, so this broke the host suite after cae5ec2. Move the import into load() and document the standard-library-only heap probe.

Validated the exact failing target and full parallel host suite with system Python, the heap probe with site packages disabled, six related tool tests, and synthetic compressed NSO decoding.
2026-10-25 12:27:40 +08:00
seeddsandb4f8abbb24dbf56a7d0280f296e16eeb cae5ec2155 Keep guest memory queries, budgets and stack ownership consistent
Incorporate the active SVC improvements from PR #151 around the existing
alias bitmap and mapping registry. Serialize heap changes, physical maps,
stack ownership and memory queries with one lock; report committed alias
runs and borrowed sources; initialize reused logical pages while preserving
live neighbors and lazy first exposure. Retain PR #127's fail-fast host
protection policy and the log formats parsed by the NSP heap probe.

Game smoke tests exposed two integration requirements: Raging Loop's SDK
extends committed windows across borrowed stack sources, so allow recommit
that preserves existing bytes; Tears of the Kingdom's audio helper needs
guest-backed stacks, so allocate host-created guest-thread stacks through
SDK malloc and free them after thread destruction and copyback.

Keep SDK reservation ownership and existing patches. The unused host
reservation allocator from #151 is not included.

Validation: full host suite; focused UBSan, ASan and ThreadSanitizer tests;
real dirty-tracker concurrency tests; heap-probe parser compatibility; Mac
host builds; iOS arm64 syntax checks; all 60 profile layout checks.
Raging Loop and Tears of the Kingdom passed guest-thread self-checks and
ran to deliberate timeouts after the startup fixes. Their shader caches
are stale for this checkout, so rendering/performance are not validated.
No IPA build or iOS device validation was performed.

Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
2026-10-25 12:23:01 +08:00
seeddsandv0lumehi 66abbc3d30 Cache Metal 4 pipeline bases with per-game archives
Adapt PR #122: compile reusable bases with unspecialized colour state, specialize per draw state, and persist bases between launches. Keep iOS opt-in and the MTLDevice fallback.

Preserve rasterizing depth-only pipelines through the legacy path. Share in-progress base builds, including errors, while unrelated bases compile in parallel. Defer timed flushes during compaction and finish flush/cleanup together; synchronize the pending count with serializer capture.

Checked on Apple M4 Pro: t_pso_archive 48 checks with Metal validation, t_pso_pending 48, t_integer_blend 63, t_geometry 95, and t_pso_skey pass. The Breath of the Wild Mac host builds and nvn_mtl.m passes the iOS arm64 syntax check. Real-game memory usage and iPhone runtime behavior were not measured.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-25 11:31:14 +08:00
seedds 1fbbd1486a Bayonetta (010076F0049A2000, update 1.1 = v65536, PlatinumGames' engine in its Wii U form on SDK 3.6.0, as Bayonetta 2): game profile, no host change. Plays on the iPhone (IPA 8.8 GB, user). Same sdk and subsdk0 builds as Bayonetta 2, so its profile is Bayonetta 2's with its own build row and tables, and the CPK reader and the two NVN fixes made for Bayonetta 2 cover it. On the Mac from a fresh save: the boot, the opening movie (USM; Plus skips it), the title and its attract demo, a new game (PLAY, NEW GAME, the auto-save notice, Normal), the prologue on the falling clock tower, the graveyard cutscene (skipped), the touch-controls tips, the practice tutorial (punch, kick, combos, evade, Witch Time), the graveyard fight of Verse 1 with Witch Time, its autosave and the next loading screen's practice, with CRI ADX2 sound through one 6-channel AudioOut and no red line: 60 fps in play (frame time p50 16.65 ms, p95 17.1-17.5 ms; GPU 2-4 ms a frame on an M6), 30 in its cutscenes, where the game sets present interval 2; about 2.45 GB resident and 1.54 GB of Metal allocations, flat over 15 minutes. No virtual address memory (a 3.11 GB heap in the 4 GB layout, mem_used 102 MB). Files: nine CRI CPK archives (data00.cpk ... data03.cpk, data10.cpk ... data14.cpk, 4.25 GB) of PlatinumGames DAT archives, beside the USM movies and the ADX2 sound. Shaders: 458 BNSH files of one stage each (236 pixel, 222 vertex), the .vso / .pso members of shader.dat in data00.cpk, all built for the Mac and iOS. Profile: the 3D pad layout (the camera is on the right stick; the touch controls are handheld only), a prewarm list recorded on the Mac over the route above (513 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); the Python tool, gfx and iOS tests pass (test_tbdr_budget with DM_GAME set). README (games gallery, a Mac picture of the graveyard fight), AGENTS.md (59 -> 60), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-25 11:03:12 +08:00
seedds 0f99c353d3 Haptics: a Core Haptics engine that fails to start is replaced, not retried for good (general, from PR #143). A Bluetooth Switch Pro controller that vibrates through iOS Settings' Identify stayed silent in a game, which logged Core Haptics -4810 (server init failed): the adapter made each engine once per route and called startAndReturnError on that same broken engine at every later effect, and its stop/reset handlers were installed only after the start. Now haptics_apple.m makes an engine when the first effect needs it (the phone's too, so after the audio session is active), installs the handlers before starting it, and a failure to make or start the engine, make or start its player or send its parameters releases that engine: a fresh game signal makes a new one after a backoff of 1, 2, 4 or 8 s, effects from before the failure are not replayed, and the old engine's callbacks are stale (generation). A controller whose left and right handle engines cannot be made or started falls back to its default locality until it reconnects; a player or parameter failure (a service reset, say) is retried on the same handle. Idle release (1 s of silence), focus loss and engine resets stop an engine but keep it, so the next effect restarts the same engine, as before. Errors name the target, the stage, the error domain and code and the underlying error, and the log names the controller, its localities and the output channels. From PR #143 (nanda86), with two changes: the PR dropped the engine at every stop, so each rumble after a second of silence made a new engine (a new connection to the haptics service on a controller, a new initWithAudioSession: engine on the phone), and any handle failure moved the route to the default locality for good. t_haptics_apple covers each failure stage, the backoff, the fallback (and no fallback on a player failure), stale callbacks, a restarted engine after idle release, a disconnect and a controller without haptics, with fake controllers and engines. Checked: the host unit tests pass (GAME=0100DCA0064A6000), haptics_apple.m compiles for iOS arm64 with -Wall -Werror. Not checked: a physical controller or the iPhone's own haptics 2026-10-25 09:55:44 +08:00
seedds 9c1e449470 Bayonetta 2 (01007960049A0000, update 1.2 = v131072, PlatinumGames' engine in its Wii U form on SDK 3.6.0, the oldest SDK so far): game profile, a CRI CPK reader for the shader split, and two general NVN fixes. Plays on the iPhone (IPA 12.5 GB, user). On the Mac from a fresh save: the boot, the PlatinumGames logo movie (USM), the title, a new game (slot 1, the auto-save notice, Story, Normal), the opening narration (which cannot be skipped on a first play), the prologue cutscene (skipped: Plus, Down, A, A), the tutorial (closed: Plus, Down x4, A), Verse 1 on the jet, Rodin's cutscene and the next fight with an Umbran Climax and a torture attack, with Wwise sound through two 6-channel AudioOuts: 60 fps in play (frame time p50 16.67 ms, p95 about 17.5 ms; GPU 2-3.5 ms a frame on an M6), 30 in its cutscenes as the game asks, about 1.6 GB resident and 1.75 GB of Metal allocations. No host change was needed for SDK 3.6.0 itself (NVN API 53); no virtual address memory (a 3.11 GB heap in the 4 GB layout, mem_used 102 MB). Files: six CPK archives (9.5 GB, 6,091 files, 12 GB of them CRILAYLA-compressed) of PlatinumGames DAT archives. New dm/gfx/tools/cpk.py: the @UTF tables (and the masked ones of Unicorn Overlord's CPKs, the same tables as before once unmasked), the TOC and CRILAYLA (a decoder over a bit string, about 29 MB/s); shader_split.py cpk_jobs splits the CPKs into jobs and scans each entry (a CPK it cannot read is scanned as one file, with a [RED] line). Shaders: 1,856 BNSH files of one stage each (.vso / .pso) in shader.dat, shader2.dat and shadereff.dat of data000.cpk, all built for the Mac and iOS. nvn_mtl.m, both general: a cleared attachment larger than the render pass (Metal's render area is the smallest attachment) is now cleared in a pass of its own and loaded by the pass, as NVN clears the whole target (rp_begin_ex_): the game's 32x32 and 256x256 passes cleared only that corner of its full-size depth target, so the opening narration drew in a corner of a black screen and the opening scenes were speckled; and Downsample of a single-sample texture is a filtered scaling copy of the whole level that may convert the format (copy_t2t, its bloom chain 320x176 -> 80x48 and its 1280x720 BGRA8 frame into an RGBA8 texture), which logged red lines before. build_ipa_from_nsp.py: a survey note for CPK archives. Profile: the 3D pad layout (the camera is on the right stick; the game's touch controls are handheld only), a prewarm list recorded on the Mac over the route above (974 lines). Checked: the graphics run list, the host unit tests and the 49 tool tests pass; Unicorn Overlord's CPKs split as before. Not checked: 13 Sentinels and Catherine: Full Body's CPKs (no files on the drives), the clear fix in other games beyond the unit tests. README (games gallery, a Mac picture of Verse 1), AGENTS.md (58 -> 59), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-25 08:56:02 +08:00
seedds 6a0687721a NSZ and XCZ files in the NSP folder (from PR #140): the NSP script converts them itself instead of a second script. scan_nsps lists *.nsz and *.xcz beside *.nsp and *.xci (an NSZ is a PFS0 with the NSP's ticket, so title_of reads it as it is; a compressed file with an NSP of the same name beside it is skipped), and decompress_nsz converts each once with nsz -D, run from the Python running the script (Spyder does not have its bin folder on PATH), into <title ID>_build/nsp/, before the build ID lookup or the survey reads them. nsz writes into an empty temporary folder and the file is moved into nsp/ only when nsz exits 0 and made it, so a partial file left by a failure or an interruption is never taken for a whole one (with -w left out, nsz itself skips an output whose title and version are already there and exits 0); a conversion is reused while its source has the same name, size and modification time. nsz -D prints an NCA hash mismatch but goes on and exits 0 (Decompressor.decompress passes raiseVerificationException=False), so a damaged file is left to nx_extract's own hash check (--allow-damaged). Without the nsz package the script stops with one line: pip install nsz. README: one line in the packing steps; docs/development.md says how it works. AGENTS.md: only build_ipa_from_nsp.py has to run in Spyder with F5; the other scripts in scripts/ run from a terminal (the user). New test dm/tools/tests/test_nsp_nsz.py (nsz stood in for). Checked: the new test, test_nsp_alias, test_nsp_chain, test_nsp_mac, test_nsp_progress and test_xcode_workflow pass; ANIMAL WELL's base game and update compressed with nsz 5.0.0 -C, then the script on a folder holding only the two NSZ files: both converted (byte for byte the original NSPs), the IPA built (exit 0), and a second run reused both conversions. Not checked: an XCZ, the IPA on the iPhone 2026-10-25 02:34:18 +08:00
seedds 3f4d973ac5 Mario vs. Donkey Kong (0100B99019412000, update 1.0.1 over the cartridge base game, Nintendo's sead / agl / gsys engine on SDK 15.3.0): game profile plus three general host changes. Plays on the iPhone (IPA 1.6 GB, user). On the Mac: the opening movie with sound, the title (Press L+R), file select, the play style choice, the first cutscene, level 1-1 (both colour switches, the spring, a time-out and its restart), the pause menu, leaving the level, level select and the save it shows: 60 fps (frame interval p99 about 17 ms), sound, about 1 GB resident in the level and flat while it runs, CPU about 30% of a core, no red line on that route.
Movie decoder (hle_movie.c, every game), both found on the opening movie (Movie/Cutscene01.mp4, H.264 and AAC through movie::Extractor):
- AAC decoders offer at most 32 outputs (MD_NOUT_AAC; Vorbis keeps 64, video 16). The game's movie player keeps the outputs it holds in a ring of 32 entries per track (main+0xe93d4c: 0x200 bytes) and pushes with no bounds check (main+0xe94140), so with 64 the 33rd AAC output wrote its index (0x3f) and time over the next heap block, and the game crashed in sead::ExpHeap's free a second into the movie, at the same address every boot. The console's AAC decoder cannot offer more than 32, or the game would overflow there too; Monster Train 2 peaked at 30 held.
- An AAC decoder's GetOutputBufferSize is one frame (1024 samples per channel, 2048 with SBR), what each output holds. The player sizes its own AudioOut buffers by it, decodes into them and appends each whole, advancing its audio clock by size / 4 samples (main+0xe92a74); with the 128 KB Vorbis bound every 4 KB AAC frame played as 0.68 s, the clock crawled and the movie froze on its first second. Vorbis keeps the bound (its packets vary in length).

Advanced blend modes (hle_nvn.c, nvn_obj.h, nvn_mtl.m, every game): the blend state keeps the mode, the premultiplied-source flag and the overlap (with their getters), and the pipeline blends the Porter-Duff modes (ZERO, SRC, DST, SRC/DST_OVER, _IN, _OUT, _ATOP, XOR, the PLUS modes) with the fixed-function factors they amount to under uncorrelated overlap (adv_blend); the others keep the state's functions with a red line naming the mode. The game's levels set SRC_OVER on one blend state, which the host used to ignore.

DM_PROBE_POLL takes a value filter (=<value>): every change is logged, the threads are snapshotted and dumped only at changes to that value. That is how the ring overflow was caught: DM_PROBE_WATCH on the heap link made the movie's file reads into the same page fail.

Profile: the provisional profile's guesses held (no system resource, a 3.14 GB heap in the 4 GB layout, modules main subsdk0 sdk), plus render_cond on (the game skips clears under render conditions; without it every level was black behind its HUD). 12,691 shaders (BNSH / BAHS in the zstd RomFS and main), all translated. Pad only (main imports no touch screen), the default 2D layout. A prewarm list recorded on the Mac from a fresh save over the route above (550 lines; with it nothing was built on demand). Checked: the NSP script packs it from the NSPs with the profile in dm/games; the graphics run list and the 49 tool tests pass; after rebasing on 6d34984 (spin waits) the game boots into level 1-1 as before. README (gallery, a Mac picture of level 1-1; 2D layout), AGENTS.md (57 -> 58) and docs/development.md, docs/agents 03 / 04 / 05 say so
2026-10-25 01:19:47 +08:00
seedds 6d34984488 Spin waits that take a spin lock (general, from PR #134, found on Breath of the Wild; fixes #129). In steady play on the Mac, Breath of the Wild had frames with 30-210 ms of CPU work every 1-10 s: the main thread and the two workers spinning in one job queue poll (main+0xef62dc). The first item of some of its jobs is for the thread of core 0; the others wait until it is taken, taking and releasing the queue's lock (an exclusive test-and-set, then a compare-and-clear) and writing a status word on every pass, so the spin wait search (a store or an exclusive store ends a path) did not take it. Polled at full speed by the two workers, the lock went back and forth between their cores, and the main thread, whose item it was, lost nearly every exclusive store: in a 40 s Time Profiler trace it never held the lock while each worker held it in about 750 samples. patch_nso.py "Spin waits": a lock taken and released on the same address on every path back to the head (spin_lock 'acq' / 'rel', counted like the exclusive adds), a test loop on a word inside the region ('tst', whose branch back goes to the original loop and its own gate), and up to four stores whose value and address registers the region never writes ('stc', only in a region that loads something). Such stores write fixed values to fixed addresses, so a loop can change its own state for a few hundred passes at most, well inside the gate's 2 ms of polling, before its passes repeat. A store of a value a pass changes ends its path and the search runs again without it (the path that takes an item stores the new cursor); when the region with stores does not cover the old one's back-edges, the old one is used. Through the gate the workers leave the lock a moment between passes and the main thread takes its item at once. Zora's Domain on an M2 Max at 120 Hz (one host, 42 s of steady play per run, three runs each, before -> after): standing still, frames with more than 30 ms of work 2.8 / 6.3 / 3.5 -> 0 per 30 s, the frame's work p99 20-21 -> 8.3-8.4 ms; turning the camera, CPU hitches 11.9 / 13.3 / 6.3 -> 0, p99 21-25 -> 9-15 ms; process CPU per frame unchanged. Every module on the game drives rewritten with the old and the new search (206: 108 NSOs and Super Smash Bros. Ultimate's 98 NROs): 175 byte for byte as before (Hades, Super Smash Bros. Ultimate, Super Mario Odyssey among them), 7 fail alike under both (every rtld, New Super Mario Bros. U Deluxe), and 24 take new or wider loops, all but Breath of the Wild's queue through the store rule alone: Monster Hunter Rise's main 90 -> 166 (72 of them one inlined countdown whose reset makes the next pass leave), BALL x PIT 87 -> 94, Animal Crossing: New Horizons 162 -> 166, Balatro 9 -> 12, Octopath Traveler 0 75 -> 78, Pikmin 4 80 -> 83, Bayonetta 3 35 -> 37, Breath of the Wild 308 -> 310, one more each in Big Brain Academy, Mario Kart 8 Deluxe, Metroid Dread, Pokemon Legends: Z-A, Stardew Valley, Super Mario Bros. Wonder (both builds) and Tears of the Kingdom; the movie library's thread loop is wider in the subsdk0 of Breath of the Wild, Mario Kart 8 Deluxe, Animal Crossing, BALL x PIT, Monster Train 2 and Octopath Traveler 0, and one loop is wider in the main of AI: THE SOMNIUM FILES and Advance Wars 1+2. No loop of the old search is lost or loses a back-edge. Each distinct shape was read in its disassembly: one-pass resets and state setters (the gate is reached only when another thread changes the word), parser states that leave on the next pass, and real flag waits. Rewriting all 206 takes about 4,100 s of CPU instead of 3,250. Diagnostics: DM_HITCH_DUMP=<ms>[:<s>] logs every guest thread (its wait, last SVC, where it is, x0-x2 and x19-x25 in guest code) when no NVN frame was presented for that long, and each guest thread names its pthread '<handle> <name>' (0x1008 Worker) before it enters guest code, the main thread '0x1001 main thread', so sample and Instruments' Time Profiler show which guest thread a sample belongs to. The graphics tests that link hle_nvn.c without the host define dm_present_ns (t_host_stubs.c weak, t_nvn.c). New test dm/tools/tests/test_spin_waits.py. Checked: the host builds, the host unit tests, the 46 tool tests and the gfx run list pass; Breath of the Wild into Zora's Domain and Mario Kart 8 Deluxe for 85 s from boot with the new rewrite (the PR's author), and Monster Hunter Rise from a fresh save for 180 s headless with each rewrite on one host (e72a065's, the chain of its Mac libraries): 60 fps, frame p95 about 17.5 ms, the same red lines, no hang. Not checked: Paper Mario: The Thousand-Year Door, DARK SOULS: REMASTERED, FINAL FANTASY TACTICS and FINAL FANTASY RESONANCE (no files on the drives), the iPhone. docs/development.md (Spin loops, Breath of the Wild) and docs/agents/05-debugging.md say so 2026-10-25 00:48:24 +08:00
seedds 98bd33a4ef BALL x PIT (010086A022444000, update 1.293, Unity 6000.0.62f1 on SDK 20.5.19): game profile and the SDK 20.5.19 sdk build's node-area cut, no host change. Plays on the iPhone (IPA 1.1 GB, user). On the Mac from a fresh save through the title, a new game, the intro and the descent into the pit, the first dive and its tutorial (moving, aiming with the right stick and shooting with ZR, gems and XP, the enemy waves that overrun the player and end it), and the base-building tutorial (the sidebar, the build menu, a wheat field and a forest placed, the first harvest): 60 fps (frame time p50 16.66 ms, p95 about 16.8 ms; GPU 1-5 ms a frame), FMOD sound through one stereo AudioOut, about 1.6 GB resident and 0.7 GB of Metal allocations, flat over 10 minutes, no red line. sdk_patches.py: like Hollow Knight: Silksong, Unity 6 takes all its memory through AllocateAddressRegion / AllocateMemoryPages with no heap; this sdk build's AddressRegionManager constructor is the SDK 20.5.22 build's at the same offsets (sdk+0x145cfc; the node area limit +0x145d0c and its occupied length +0x145d78 cut from 0x3fe00000 to 0x3e00000): the alias region's high-water mark +2.00 GB of 4.375 GB, 1.26 GB mapped. The provisional profile was otherwise right as generated (Engage's 5 GB block, sys_resource 16 MB; profile_guess.py --check: ok). Shaders: 38,099 (18,913 vertex, 18,917 pixel, 116 compute, 76 hull and domain, one geometry), 11 of them GLSLC outputs of Unity's player in main; the loose serialized files (resources.assets, sharedassets1.assets, the built-in resources) have no type trees and borrow the Shader tree 10 of the 11 Addressables bundles carry (the same type hash, shader_split.py's deferred pass), so unityfs.py needed no layout; all build for the Mac and iOS. The game draws 1280x720 into 1920x1080 window textures with a window crop (nvnWindowSetCrop 0,0 1280x720), which the present already scales. Profile: pad_layout full (the right stick aims in a dive), a prewarm list recorded on the Mac over the route above (297 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); after the rebase onto ded1cff the 44 tool tests pass. README (games gallery, a Mac picture of the first dive), AGENTS.md (56 -> 57), docs/development.md and docs/agents 03 / 04 say so 2026-10-25 00:24:57 +08:00
seedds 02294dda9e Stardew Valley (0100E65002BB8000, update 1.6.15.13, MonoGame on Sickhead's BRUTE, SDK 20.5.27): game profile, no host change. Plays on the iPhone (IPA 0.6 GB, user). On the Mac: the title, a new farm (character creation with the software keyboard, Skip Intro), the farmhouse, the farm and its tools, the bus stop, Pelican Town by day and night, Pierre's shop and a purchase, the game menu, the mail, a night's sleep and its save, Load from the title, and the Co-op menu (Online Communication goes back to the title with no network): 60 fps (frame time p50 16.67 ms, p95 16.8 ms; GPU about 0.6 ms a frame on an M6; CPU 8% of a core), sound, 0.93 GB resident in play and flat over the last 10 of 23 minutes, no red line. About one frame in 800 takes 24-47 ms of the game's own CPU time (no pipeline build, upload or present in it; the main thread otherwise sleeps 95% of each frame in SleepThread, whose host sleep is exact), probably BRUTE's Boehm collector, not checked further. The two changes this game needed came before it from PR #120 (compressed XNB shaders, the Mac text box releasing the buttons that opened it). Profile: the provisional profile's guesses held (no system resource, a 2.96 GB heap in the 4 GB layout, mem_used 218 MB for the rewritten modules); 65 shaders (64 MonoGame effects in main, one in the LZ4 XNB Content/Effects/ShadowRemove.xnb); pad_layout full (main imports no touch screen or mouse; the D-pad in menus, the right stick for the cursor); a prewarm list recorded on the Mac with prewarm off over the route above (6 lines: SpriteBatch's two shaders and four pipelines; with it nothing was built on demand). Checked: the NSP script packs it from the NSPs with the profile in dm/games; the 41 tool tests pass. README (gallery, a Mac picture of the farm), AGENTS.md (55 -> 56) and docs/development.md say so 2026-10-24 23:20:25 +08:00
seedds ce173001ce NSP script: a title ID no profile has is looked up by its main build (general, from issue #114). A regional release of a build a profile registers has another title ID (Advance Wars 1+2: Re-Boot Camp's main 320a17744aefd67fec5738ca0c629795 under 0100300012FFF000 as well as 0100300012F2A000), so game.py by_title found nothing and the script surveyed it as an unknown game with a provisional profile. same_build now extracts only the code modules (nx_extract --manifest: no game files written) of the base game with its newest update, then of the base game alone, and game.py by_build looks main's build ID up in every profile's build rows; a match packs from a profile derived from that one (derive_profile), with the folder's title ID and main.npdm's program ID (the host reports that program ID and answers save data queries for it) and shaders split from the title's own files, since its RomFS may differ. The closing message says the derived build is the profile's registered build under another title ID instead of a version it was not brought up on. No two of the 56 registered main builds are in more than one profile. New test test_nsp_alias.py (by_build, the update-then-base order, the derived profile's IDs); the 44 tool tests pass. Checked: the script on Advance Wars' NSP with its title ID made to read as 0100300012FFF000 found profile 0100300012F2A000's v0 in 2 s, packed for the Mac (20,424 shaders) and the build ran 90 s headless with no red line; the real title packs as before, with no lookup. docs/development.md says so 2026-10-24 23:05:05 +08:00
seedds 7a029f66d9 Pikmin 4 (0100B7C00933A000, update 1.1.1, Unreal Engine 4.26.2 on SDK 10.4.8): game profile, the allocator and engine-region cuts, the SDK 10.4.8 sdk build's address-region cuts, two rewriter cases and a general nifm fix. Plays on the iPhone (IPA 10.8 GB, user). On the Mac from the boot through the title (a VP9 movie behind the logo), the main menu, a new game (the auto-save notice, players, creature activity level), the opening and its cutscenes into the Olimar prologue in play, and Continue from a save: 30 fps as on the Switch (the game steps a fixed frame time, bUseFixedFrameRate=True, and asks for rhi.SyncInterval=2), Wwise sound through its own AudioOut sink, about 2.6-2.8 GB resident and 1.7 GB of Metal allocations, no red line. patches.py: Unreal's FMallocBinned3 is made on the first allocation as in 4.27 (GCreateMalloc main+0x353f770 mallocs 0x1c38 bytes, constructor main+0x35ba610, the one store of the vtable main+0x8da4758 outside its two destructors, found from the Binned3 error strings through the relocated slots); the constructor call is cut and observation gate OBS_UE_MALLOC on the ldp x29, x30 after it hands the object the host allocator (hle_uemalloc.c, the 21-slot UE4_27 layout). This nninitStartup reserves the engine region only when -SwitchVirtualAddressSpaceSize= is absent (ldr x8,[x22,#0x130]; cbnz x8 past the default and its AllocateAddressRegion, main+0x353f564): with the argument the base stayed 0 and the first commit, AllocateMemoryPages(0, 2 MB) as the RHI started (main+0x353ff8c, 0xe03), went into Unreal's fatal error. The two orr #0xf80000000 (62 GB) are cut to mov #0xfe000000 (4064 MB) and the profile has no game_args, in Engage's 5 GB block. sdk_patches.py: the SDK 10.4.8 sdk build's AddressRegionManager is the 10.4.1 build's at the same offsets; the node area is cut to 62 MB and the 1 GB alignment of large regions to 32 MB. patch_nso.py (general): an adrp inside an exclusive window that uses x18 (main+0x43da14c: ldaxr; cmp; b.ne; adrp x10 + add; mov x18,x26; stlxr) is re-encoded for the copy's address (window_adrp), and scratch_dead accepts a ret as it does a bl (the gate sits on an epilogue, and the check reads the image after the cut, so the bl before it is a nop); both only apply where the rewrite used to stop, so every other module rewrites byte for byte as before. hle_net.c (general): a submitted nifm request stays on hold for 50 ms, then settles as unavailable, as Horizon's asynchronous request does. Answered at once, about one boot in six (3 in 4 with four Mac hosts at once) waited at the black loading screen for good: the boot state machine (main+0x1f2f0e0) sat in state 7 polling the BCAT runnable's status, whose constructor (main+0x203f3b0) starts FBCATThread and only then stores 1, and whose Run (main+0x203f510) submits the request, waits while on hold and stores 6; a thread that ran first finished before the constructor's store, which overwrote the 6. With the hold 12 of 12 boots under load reached the title. Profile: shader groups as DRAGON QUEST VII Reimagined's (15,727 distinct shaders, every one an LZ4 block in the pak's two shader code archives; all build for the Mac and iOS), pad_layout full, a prewarm list recorded on the Mac over the route above (806 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); after the rebase onto 0045c1d the 41 tool tests pass and the host builds. README (games table, a Mac picture of the prologue), AGENTS.md (54 -> 55), docs/development.md and docs/agents 01 / 03 say so 2026-10-24 20:16:15 +08:00
seedds 0045c1dfb6 Pokémon Sword (0100ABF008968000, update 1.3.2, Game Freak's engine on SDK 7.3.2): game profile plus general changes to the shader split, the shader build, the NVN layer and HID / offline services. Plays on the iPhone (IPA 12.4 GB, user). On the Mac from a fresh save through the first boot's choices, the opening, Postwick, Hop's house, the walk to Wedgehurst and Leon's arrival, the starter choice (Scorbunny) and the first battle against Hop (won), and from a save in the bedroom through the title and out of the front door: 30 fps as on the Switch (frame time p50 33.3 / p95 about 35 ms, GPU 4-9 ms a frame on an M6, CPU about half a core), sound, 1.8 GB resident after the title and 2.4 GB after 17 minutes of new areas, 1.9 GB of Metal allocations, no red line on that route. The sdk and movie module (subsdk0) builds are Super Mario Odyssey 1.4.1's; subsdk1 is GLSLC, which main imports but never calls; no virtual address memory (a 3.13 GB heap in the 4 GB layout, mem_used 76 MB). shader_split.py: everything is in 7,699 GFPAK archives (GFLXPACK: file count at +0x10, 0x18-byte file entries of compression, length, stored length and offset; every entry LZ4, zlib and stored read too, Oodle a red line), whose BNSH entries the split now reads (gfpak_members): 3,126 shaders before, 38,361 after with the loose BNSH files and two in main. The 38 shaders still in no map at the title were Silicon Studio's YEBIS post effects: a .tfxo file (3,155 shaders, control sections each followed by its code area of control length + 0x704) in a 39.5 MB zlib stream in main (+0x1c98ed9), which tfxo_streams finds by its zlib header and the inflated file's magic (the header check inflates 4,096 bytes; 64 did not get past the first block header) and tfxo_shaders reads, single-stage programs: 41,516 shaders in all. Of the other games' mains the stream reader finds nothing, and main's loose control section reader (raw_shaders) finds nothing in this one, so every other profile's tables stay as they were. dmsh_build.py / nvn_mtl.m: the G-buffer pixel shaders write RGBA16Uint at colour 1 between two float targets, and the all-integer entry point (fs_uint, main0_u) made Metal refuse the pipeline for the float ones; a pixel shader whose outputs are partly integer bits now also has main0_m, which returns only those as integers (uint_mask_m in the reflection, 336 shaders here), and pso_desc takes it when the bound targets' integer formats are exactly those outputs (pso_mixed_uint). Chain version dec5ee141a9f. nvn_mtl.m tex_by_id: a cube view whose layer count is not a multiple of 6 (a view of one layer, as the player leaves the house) aborted in Metal; it now counts its layers in cubes when they fit, and is read as a 2D array with a red line otherwise. Packaged textures: the game writes beside its textures in the 16 KB host pages they share with other data, so each such write re-uploaded the texture, 73k re-uploads over a few minutes and frames from 33 to about 75 ms; the bytes of a texture's storage that lie in its shared first and last pages are kept after each upload (pkg_edges_take), and a dirty texture whose interior pages were not written and whose edge bytes are unchanged is re-protected without a re-upload (pkg_edges_clean; the packaged-textures log line counts them): 0 re-uploads on the route since. New t_nvn case 6c1 (writes beside a texture, then a real write), 110 checks. hle_nvn.c: nvnDeviceGetWindowOriginMode and nvnDeviceGetDepthMode, which the game reads without setting either. hle_hid.c: the Poké Ball Plus (Palma): boost mode a no-op, connection enable / disable succeed, GetPalmaConnectionHandle and every handle call fail with an invalid handle (the first boot crashed at SetPalmaBoostMode). hle_offline.c: pdm::QueryApplicationPlayStatistics answers each asked title as never played (the game asks about the Let's Go games; it crashed there next), err::ShowNetworkServiceError is logged. Profile: pad_layout full (the camera is on the right stick), shader groups bin / systemresource, a prewarm list recorded on the Mac over the route above (3,273 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); after the rebase onto 6d42bc9 the host unit tests, the 41 tool tests, t_nvn, t_dirty_slots, t_dirty_race, t_integer_blend, t_upgrade_alias, t_clear_ds and t_stencil_view (under validation) pass, and the route out of the house runs as before. README (games table, a Mac picture of Postwick), AGENTS.md (53 -> 54), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-24 18:55:11 +08:00
seedds 6d42bc9815 Advance Wars 1+2: Re-Boot Camp (0100300012F2A000, base game 1.0.0, Unity 2020.2.1f3 IL2CPP on SDK 10.4.1): game profile, the SDK 10.4.1 sdk build's address-region cuts and a shader split reader for control sections loose in main. Plays on the iPhone (IPA 7.1 GB, user). On the Mac from the boot through the Bink opening movie, the title, Nell's introduction, a new casual campaign and the first training battle (Field Training, Troop Orders): 60 fps in the menus, 30 fps in battle by the game's own limit (present interval 1, the frame's work 33 ms with CPU and GPU mostly idle), FMOD sound through AudioOut, about 2.5 GB resident and 1.6-1.9 GB of Metal allocations, level over a 15-minute run with no red line. sdk_patches.py: the sdk build is The Great Ace Attorney Chronicles' (no virtual address memory there, so it never needed a cut); this game's NPDM declares 16 MB of system resource and nninitStartup (main+0x365310) hands pool - used to StandardAllocator, whose Initialize (sdk+0x2edf90) aborted at boot. The AddressRegionManager constructor (sdk+0x340e90) loads the 1 GB node area with orr w8 / w9, wzr, #0x3fe00000 (the SDK 7 / 8 form, +0x340ea0 / +0x340f3c), cut to 62 MB, and AllocateAddressRegion's 1 GB alignment of regions of 1 GB and more (+0x34109c) to 32 MB, as on DRAGON QUEST XI S's SDK 7.3.2 build. Profile: the NSP script's heap check set a 1 GB pool, as for the other Unity games with virtual address memory, but the game's malloc then ran dry at the first loading screen and Unity's GPU pool allocation (main+0x96f430) halved its aligned_alloc request down to 0 and spun there (frames stopped 17 s in; found with sample on the Mac host); with 2 GB the alias region is mapped up to 2.26 GB (high-water mark +3.27 GB of 4.375) in Engage's 5 GB block. shader_split.py raw_shaders: Bink 2's Unity plugin builds its programs from 32 control sections and code areas in main's read-only segment that are in no GLSLC output and no BNSH file (2 vertex, 22 pixel, 8 compute), so the movies drew white (two shaders not in the map and a program without a vertex shader as the opening movie started). 26 are 0x20-byte records in .data ({code pointer, control pointer, u32 code length, u32 control length}, found through the relative relocations, RELA or RELR: rel_targets); the code takes the other 6 by address, their code area right after the control section's own length or right before it, and two share one; both neighbours can pass the header and length checks (main+0x4a2c670, a compute shader, would have taken the first 0x100 bytes of the next vertex shader's code), so only a code area whose address the code builds with adrp + add nearest to the control section's is taken (adrp_targets). Compute code areas have no header (SDK 10), so the stage is glslc_stage's, and a code area may not hold a control section. Control sections another reader took from main are left alone (Celeste's MonoGame effect). Of the 25 mains on the game drive the reader adds shaders only to this one, so every other profile's tables stay as they were (Celeste and the other games on the second drive not checked: not mounted). 20,424 shaders, all build for the Mac and iOS. Checked: the NSP script packs it from the NSP alone (Mac and iOS), the tool tests pass. README (games table, a Mac picture of the first battle), AGENTS.md (52 -> 53), dm/games/README.md (the heap check's 1 GB is a starting point), docs/development.md and docs/agents 03 / 04 say so 2026-10-24 17:13:12 +08:00
seedds 14b68357a3 Run-time shader captures on a Mac without Metal 4 (GitHub issue #112: Balatro could not be packed on GitHub's macos-26 runner, a virtual Mac). The GL and Vulkan layers draw on classic Metal queues of their own and take only the device and the present path from the NVN layer, but mtl_device ran the NVN layer's whole start-up, which aborted on the paravirtualized GPU at the MTLResidencySet (and, past it, at the MTL4CommandQueue), so the GL capture boot stopped at eglGetDisplay and captured nothing. nvn_mtl.m mtl_init: a device without the Metal 4 GPU family, or one where the residency set or the Metal 4 queue cannot be made, now starts only the classic part (queue, shared event) when the caller came through mtl_device_classic (hle_gl.m, hle_vk.m), says so in one red line, and drops every present (present_enqueue gives ticket 0, so nothing waits on the present path), which a headless capture does not need; an NVN game still stops there, with a message naming the device and the reason. DM_TEST_NO_METAL4=1 takes the same path on any Mac. Checked on this Mac (M6): the NSP script for Balatro (--platform mac) with DM_TEST_NO_METAL4=1 captured the 35 shaders gl_capture.tsv lists in 1 s, translated them and built the Mac host; booted headless for 20 s without the switch, the host starts Metal 4 as before (35 shader table entries, no red line, graceful exit), and with the switch and the shader library the game runs the 20 s with presents dropped and exits gracefully. Not checked: the Vulkan capture (Super Mario Galaxy, Galaxy 2; same call, no NSPs here) and a real virtual Mac. docs/development.md (build step 4) says so 2026-10-24 15:33:22 +08:00
seedds 6034fcad89 Bayonetta 3 (01004A4010FEA000, update 1.2.0, PlatinumGames' engine on SDK 13.3.2): game profile plus general changes to the Oodle tool mode, the shader split and the NVN layer. Plays on the iPhone (IPA 14.2 GB). On the Mac from the boot through the title, a new game (Standard, Naive Angel Mode off, brightness, the auto-save notice), the VP9 opening movie and the prologue's first real-time scene and play: 60 fps in play (frame time p50 16.67 / p95 17.5 ms, GPU about 5 ms a frame on an M6), 30 where the game asks for it (title, loading screens), sound, about 2 GB resident and 3.1 GB of Metal allocations, flat after the first scene; the title is up about 15 s after launch with a warm shader cache. hle_oodle.c: everything is in 72 PKZ archives (14 GB) of DAT archives whose entries are Oodle Kraken or stored, and this main reads them only through Oodle's streaming decoder (main+0xcbc698 / +0xcbc7f8), so OodleLZ_Decompress was never linked in and its name string is not there; when a main has no such name the tool mode now finds OodleLZDecoder_MemorySizeNeeded / Create / Destroy / DecodeSome (the two functions that pass oodlelzcompressors.cpp to Oodle's error handler, the first also taking the legacy-compressor message, and the two functions between them, checked) and decodes each request with them; games with OodleLZ_Decompress take the same path as before. shader_split.py: PKZ archives (pkz_entries, pkz_jobs: every entry, decoded by the game's own decoder in the Mac host), DAT archives (pgdat_members; a file that only starts with DAT\0 but whose tables do not fit is scanned as before) and PlatinumGames' shader objects (pg_shader: VSO / PSO / CSO / HSO / DSO, a control section and a code area each, single-stage programs): 8,024 objects, all in root_000.pkz, 6,424 distinct keys; all build but two domain shaders (subgroup operations), which are never drawn. profile_guess.py turns on oodle_packs for Kraken PKZ archives, so the NSP script hands the split the Oodle host. hle_nvn.c: the engine checks each shader object's GPU code version against the driver's range (main+0xd0da20) and refused its first built-in shader, hwfillcolor.vso at 1.15, against the 1.14 the host reported (Super Mario Odyssey's SDK 7 table), so nnMain's start-up failed and the shutdown read an uninitialized pool (main+0xb8462c); a driver runs the code of its own SDK's GLSLC, so the newest version reported now follows the SDK: 1.15 from SDK 8, 1.16 from SDK 13, and at least a GLSLC the game carries, as before. Also nvnWindowGetCrop and nvnSamplerBuilderGetLodClamp, red stubs before. Profile: pad_layout full, mem_used 116 MB, shader groups core / eff, a prewarm list recorded on the Mac over the route above (678 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS), the host unit tests and the six tool test files pass, and profile_guess.py --check gives the same results as before. README (games table, a Mac picture of the prologue), AGENTS.md (51 → 52), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-24 13:21:44 +08:00
seedds ff64352885 Super Smash Bros. Ultimate (01006A800016E000, update 13.0.5, SDK 8.2.1): game profile, NRO modules on iOS, data.arc shaders. Plays on the iPhone (IPA 17.5 GB, all 99 DLC licences)
On the Mac from update 13.0.5 (v2031616): the opening movie, the title, the first launch's DLC unlocks and Challenges board, the main menu, rulesets, stage and fighter select, 2-player battles (Frigate Orpheon, Big Blue, Great Bay, Find Mii, Prism Tower, Kongo Jungle 64) at 60 fps with about 2.7 ms of GPU per frame and 2 GB resident (1.2 GB of Metal allocations), quitting to the results and a second battle whose fighters' NROs the game loaded again; no red line in battle.

- Profile: modules main subsdk0 sdk (Catherine: Full Body's sdk and movie builds), the 5 GB guest layout, shader groups for data.arc's shader databases, effects and UI layouts, 35,012 shaders.
- patches.py: the engine's allocator (main+0x37c9a20, a static constructor) reserves 62 GB of address space with AllocateAddressRegion and aborted at main+0x37c9bd0. Booted once with a 64 GB alias region it commits from the region's start up (1.73 GB at the title, up to 2.5 GB in a battle) and keeps its page tables in the top 14 MB; all 11 instructions derived from the size are cut to a 4 GB region.
- sdk_patches.py: this SDK 8.2.1 sdk build (5fd4f531, Catherine's too) has the 1 GB AddressRegionManager node area and the 1 GB alignment of large regions (DRAGON QUEST XI S's SDK 7.3.2 layout at sdk+0x336b40 / +0x336d5c); both cut, so the region starts 64 MB into the 4.375 GB alias region.
- NVN: nvnSetExternalDebuggerEnabledNVX, fetched with nvnDeviceGetProcAddress(NULL, …) before the device and called unchecked (pc=0 from main+0x37fe92c), is a no-op in names.txt / hle_nvn.c, as docs/agents/03 already described.
- iOS NRO embedding (the part d979c34 left out): gen_embed.py packs the NROs nro.tsv lists into three sections (__TEXT,__dm_nro_x0, __DM_NRO_W,__img, __TEXT,__dm_nro_x1) with a table of their pieces (__DATA,__dm_nro_tab), since a Mach-O file has at most 255 sections and three per NRO would take 294; loader.c map_image maps an NRO from there (nro_pieces); build_ipa.sh puts nro.tsv beside the rewriter tables. The 98 NROs add 157 MB of code and 51 MB of data to the executable.
- shader_split.py reads data.arc through smash_arc.py (smash_arc_jobs / smash_arc_shaders: each current file scanned, labelled /data.arc/<format>#<hash40>); smash_arc.files(regions=True) gives every region's data of a regional file, without which 128 effect shaders were missing (34,884 of 35,012).
- Docs: README (games table, a Mac picture of Great Bay), AGENTS.md (50 → 51), docs/development.md (game notes, the NRO paragraph, nro_collect.py), docs/agents/01 (data.arc) and 03 (the 4 GB cut and its measurements, iOS NRO embedding).

Checked: the NSP script packs it from the NSPs alone, for the Mac and for iOS; tool tests (41), host unit tests and the NRO loader test pass. Played on an iPhone with this port's build from before it moved onto d979c34's NRO loader; the merged build was checked on the Mac (battles, an NRO loaded again) and builds for iOS.
2026-10-24 13:08:34 +08:00
seedds 066f2afc76 Pokémon Legends: Z-A's GPU page fault and crashes (GitHub issue #101): image atomics that are not followed go to a scratch buffer as large as the image, and the transform feedback draw ends the copy encoder before it opens its own
On the Mac, the fresh-save route of the docs crashed at about frame 3390 and from about frame 5300 page-faulted or crashed on corrupted pointers, with the encoding thread on and off; issue #101 reports an iPhone 17 freezing on kIOGPUCommandBufferCallbackErrorPageFault.

- tf_draw opened its render encoder while the copy encoder (g_blit) was open: ubd_land's inline updates leave it open, as does a copy or a dispatch just before. Metal validation: "Command buffer already has an open command encoder"; the driver then crashed in -[AGXG18GFamilyComputeContext_mtlnext performEndEncoding] (or its dealloc on the encoding thread). Every other helper pass ends g_blit before it opens its own encoder; tf_draw now does too. A host built from before PR #62 crashed the same way.
- A block-linear image's atomics went to the 64 KB dummy buffer, but SPIRV-Cross indexes them by the image bound in the same slot (spvImage2DAtomicCoord: x + y × its width). Z-A's atomic max into a 480x270 r32ui 2D array (texture ID 302, fragment shader 2bd5037003980716) ran about 1.4 MB past the dummy: into zero-copy pools over the game's heap (pointers in the game's objects and the host's program objects had their upper 4 bytes replaced by packed values, rewritten every frame) and into unmapped addresses (the page fault, after which macOS ignores the process's command buffers). MTL_SHADER_VALIDATION=1, which stops out-of-bounds accesses, let the scene play. Such atomics now go to a scratch buffer at least as large as the image (atom_scratch: width × height × layers × texel size, at least the NVN storage size; 3 MB here), grown when a larger image needs it; the results are discarded as before. An unbound image keeps the 64 KB dummy, since its slot holds the 1x1 dummy image. The bring-up host (d34e044) with today's shaders page-faulted the same way.

Three headless Mac runs of the whole route (encoding thread on, off, on) reached frame 12,300 with no GPU error and no crash; before, every run crashed or page-faulted between frames 5230 and 5480. t_nvn 102/102, t_ub 47/47, t_geometry 95/95, t_vbuf 43/43. docs/agents/04-graphics.md and Z-A's notes in docs/development.md say so.
2026-10-24 12:43:39 +08:00
d979c346b9 Integrate demand-first PSOs, far-base tracing and NRO loading
Integrate PRs #75, #90 and #91 with focused correctness fixes:
- Prioritize demanded render pipelines and bound speculative compilation.
- Follow copied jump-table bases and reject W/X register clobbers.
- Collect and rewrite registered NROs, load them on the Mac, and preserve
  runtime activation order through unload/reload and symbol rebinding.
- Preserve refused-only manifests and invalidate collection caches when
  archive readers or symbol parsing change.

Add pipeline scheduling, register tracing, NRO packaging and loader
lifecycle regression coverage. Document that executable NRO embedding
on iOS remains outside this integration.

Validation: 41 Python tests; 48 pipeline scheduling, 102 NVN and 95
geometry checks; three pipeline-key serialization cases; full host test
suite; Mac host build; iOS loader syntax checks; git diff --check.
Gameplay and controlled performance A/B measurements were not run.

Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-24 11:38:12 +08:00
seedds f596c75012 Taiko Risshiden V DX (0100346017304000, update 1.1.1, Koei Tecmo's KTGL on SDK 12.3.3): game profile plus two general host changes. Plays on the iPhone by touch (IPA 3.3 GB, Japanese; the game has no English). On the Mac from the notice, the logos and the opening movie through the title, a new game (1560, Kinoshita Tokichiro) with its opening story, the town, a council with a battle proposal, the territory map, a rest at home, a save and a load, and the field battle tutorial, docked with buttons and in handheld mode by touch: 60 fps (frame time p50 16.66 / p95 16.85 ms in the town), sound, about 0.7 GB resident and 1.6 GB of Metal allocations, flat over 15 minutes, no red line on that route. hle_oe.c: oe::SetCpuOverclockEnabled, the first boot's crash (nnMain at main+0x2e1e4c; the SDK's setter reads the apm session the host never opened, null read at sdk+0x362b8, as IsCpuOverclockEnabled did for Monster Hunter Rise); the host keeps the flag and the query reports it. hle_touch.c / hle_hid.c: gestures for hid::GetGestureStates, which answered no entries: the game takes its taps, drags and flicks only from there (main+0x42f1ac acts on types 5-9; its GetTouchScreenState read only tracks the finger), so no tap registered in handheld mode. The host now recognizes them from the same touch points, only in handheld mode as touches, by yuzu's rules (Touch, Press after 0.5 s, Pan, Pinch / Rotate, Cancel on a change in the finger count, then Tap with IsDoubleTap within 0.35 s, Swipe above 400 px/s or Complete, and Idle) plus an 8 px slop before a touch counts as moving, since a fingertip on a phone wobbles; DM_TOUCH_LOG=1 logs the entries, and a DM_CTL line drag <x0> <y0> <x1> <y1> [seconds] moves a finger for testing pans and swipes on the Mac. New unit test t_gesture (43 checks: tap, double tap, slop, slow pan, swipe, press, finger-count cancel, pinch, a press and lift between two reads). Profile: op_mode handheld and pad_layout touch (the Tap layout: the gamepad's corners would cover the cards, the menu icons and the on-screen B button; the layout button brings it back), shaders from eff/ (167 in the G2S container KTGLEffect.g1s, _S2G3000, and 45 GLSLC outputs in main, read by the existing splitter), a prewarm list recorded on the Mac over the route above (18 lines). patches.py: KTGL's frame end (main+0x443e8c) slept in 2 ms rounds until 1/60 s had passed since the frame started, which with about 2 ms of work ended at 18.2 ms: 55 fps in the menus and the town, every 11th frame on screen two vsyncs. The game presents at interval 1, so a cut on this build skips that wait (b.hs → b at main+0x443fd4) and the host's present pacing holds 60; wakeups 2,050 → 320 a second, CPU energy 1.7 → 0.7 mJ a frame. Checked: the NSP script packs it from the NSPs alone (Mac and iOS), the host unit tests and the four tool test files pass. README (games table, a Mac picture of the town), AGENTS.md (49 → 50), docs/development.md and docs/agents/03-bringup.md say so 2026-10-24 11:31:10 +08:00
seedds 3eb1d76f5d Add game rumble for iPhone and Apple game controllers
Preserve per-player motor state and SDK vibration handles, apply batches atomically, and route Core Haptics output through existing controller assignments. Add independent launcher controls and lifecycle tests.

Supersedes #86. Physical feel and controller compatibility still require device testing.
2026-10-24 11:02:21 +08:00
seedds 6de59702ec ROMANCE OF THE THREE KINGDOMS 8 REMAKE (0100C2A01A730000, update 1.1.3, Koei Tecmo's KTGL on SDK 17.5.3): game profile plus three general changes. Plays on the iPhone with a controller (IPA 16.8 GB); on the Mac from the logo movie, the intro and the title through a new game (the recommended setup, the opening story), the council, espionage, a month's missions, the 3D city view, a visit, a practice duel in 3D and a save and load, at 30 fps as on the Switch with sound, about 2 GB resident in the council and 2.5 GB in the duel, flat on a screen left idle. nx_extract: the update is a multi-program patch, the game's patch (…0000) beside a whole second program (…0001, 34 MB, its own Control NCA) that main starts with nn::oe::ExecuteProgram(1); nx_extract took the first Program NCA, the second program, and stopped with "NCA section 1 does not exist" layering it over the base game. It now takes the base NSP's lowest title ID (program index 0) and the update's Program and Control NCAs of that title ID (Balatro and Mario Kart 8 Deluxe extract as before; the host runs program 0 only). shader_split.py: a newer RDB than Age of Calamity's, 296 .fdata files (PDRK0000 and u64 0x10, then IDRK entries) beside data/*.file; flag 0x400000 marks chunks of a new form (u16 stored length, 8 bytes, the stream; a zero length ends; 16 KB inflated at most), LZ4 blocks with 0x100000 too, zlib otherwise; every other flag value is read as before. All 51,655 entries (49,620 LZ4, 2,026 zlib, 9 stored) inflate to the length in their header. The shaders are G2S (_S2G5000 / 6000 / 7000) and _DOK0000 containers again: 59,214 GLSLC outputs there and 149 in main (149 before the change), all built for the Mac and iOS. hle_audren.c: SDK 17.5.3's OpenAudioRenderer with an event calls the form without one through the PLT (sdk+0x1e489c), which the loader binds to the host's hook, so ar_open ran twice for one renderer: the inner call started the render thread before the event existed and the outer one logged a second renderer; the inner call (t_in_open) now goes straight to the SDK. Profile: pad_layout full (the dialogue boxes' auto and log buttons are the right stick and R3; the game reads a USB mouse, not the touch screen), mem_used 80 MB, a prewarm list recorded on the Mac over the route above (451 entries). Left as is: three shaders get textures of another type on some slots (texture type mismatch, once each), with nothing wrong on screen. Checked: the NSP script packs it from the NSPs alone (Mac and iOS shaders, 1 h 19 min and 1 h 29 min on a 12-core M6), the four tool test files pass (34 tests), profile_guess.py gives the profile's memory keys. README (games table, a Mac picture of the duel), AGENTS.md (48 → 49), docs/development.md and docs/agents say so 2026-10-24 08:29:17 +08:00
seedds c392d56af0 Balatro (0100CD801CE5E000, update 1.1.3, LÖVE 11.4 "love-nx" with LuaJIT and SDL on SDK 18.3.2): game profile and the GL layer LÖVE needs, plus four general changes. Plays on the iPhone by touch; on the Mac from the intro and the title into a run and its tutorial, at 60 fps with sound, about 0.65 GB resident and flat over three minutes. The game is Lua source in the RomFS and renders with NVIDIA's desktop OpenGL through EGL, as Digimon does. GL layer (dm/gfx/gl/hle_gl.m): LÖVE reads GL_CONTEXT_PROFILE_MASK and takes its core-profile paths only when it says core (with a compatibility profile it streams vertices from client memory, which Metal cannot draw from), so the layer answers core and implements what those paths use, about 150 functions in all against Digimon's 79: persistently mapped buffers (glBufferStorage, glMapBufferRange: the shared MTLBuffer's own memory), glBufferSubData in command order, fences (a fence is the serial of the command buffer being recorded; a wait on one still recording commits it), glBindAttribLocation (the game's attribute index of each Metal input, set at the link) and current attribute values (glVertexAttrib4f: LÖVE's VertexColor and ConstantColor are disabled arrays), glGetActiveUniform and the uniform read-backs, every glUniform* form, glTexStorage2D / 3D with their level counts, 3D and 2D array textures (LÖVE makes a default of each type on 96 units), GL_TEXTURE_SWIZZLE_* (its font glyphs are RG8 read as R, R, R, G), glGenerateMipmap as a blit (the game's images are mipmapped), instanced and base-vertex draws, separate read and draw framebuffers, glBlitFramebuffer (colour, same size), glReadPixels, glClearBufferfv, glFrontFace and the EGL calls SDL makes; stack arguments of the functions with more than eight are 64-bit, as AAPCS64 lays them out. DM_GL_STUBS=1 gives every GL name the layer lacks a stub of its own that reports its first call, instead of a null pointer. Shaders: LÖVE puts its 35 GLSL sources together at run time in Lua, so the NSP script captures them on the Mac as it does Super Mario Galaxy's SPIR-V (capture_shaders now serves both, profile gl_capture.tsv; profile_guess.py's gl_capture rule: an OpenGL game with no GLSL source in its RomFS or main), and gl_shaders.py keeps a bool uniform's declared type (a uint in SPIR-V; LÖVE's Shader:send takes only a boolean for a bool) and takes from main only strings that start with '#version <number>' (Balatro's main has glslang's messages). General: profile_guess.py leaves vkGetInstanceProcAddr, SDL's Vulkan loader, out of its Vulkan count (Balatro's provisional profile said Vulkan; Hades and both Galaxies import 99 or 100 Vulkan functions); OpenAudioRenderer without an event now starts the host's render thread too (hle_audren.c ar_open: only the form with an event did, and such a game was silent); loader_census.py follows registers in a straight line (adrp, add, loads through relocated GOT entries, mov, csel), for LÖVE's glad, whose pointers are globals its code keeps the addresses of in registers; and the iPhone gamepad has a third layout, Tap (profile pad_layout touch, which needs op_mode handheld; DM_GAME_PAD_LAYOUT replaces DM_GAME_PAD_FULL): no control but the layout button, every touch to the game, offered in the layout cycle only in handheld mode. Profile: op_mode handheld (the Switch build plays by touch there), pad_layout touch, gl_capture.tsv. A Lua error makes Balatro quit (its Switch error handler), and love-nx's video worker is then deleted twice (main+0x339c00): the bring-up notes say how the errors were found. Checked: the NSP script packs it from the NSPs alone (the capture in under a second, 35 shaders for the Mac and iOS, an 82 MB IPA), tool tests pass, profile_guess.py --check changes no other profile; Digimon was not rerun (its files are not on hand). README (games table, a Mac picture of the title), AGENTS.md (47 → 48), docs/development.md, docs/agents and the READMEs of dm/games and dm/gfx say so 2026-10-24 01:50:19 +08:00
seeddsandv0lumehi 76545a58a7 Mac: each controller is a player of its own, for local co-op (GitHub issue #74), from PR #84
Every GameController fed the same input path (front_mac.m), so the last one moved won, and hle_hid.c reported
exactly one npad whatever the game supports. Now the first controller is player 1, with the keyboard, the next
player 2 and so on up to 8 (pad_slots.h; a removed controller frees its player and gets it back when a
controller of the same name reconnects). Player 1 stays on No1 docked and Handheld in handheld mode, player n is
on No n (No1 stays empty in handheld mode: games take Handheld and No1 both for player 1). The game's supported
npad ids decide how many play: a one-player game ({No1, Handheld}) gets every controller as player 1, a
controller past a game's last player is not used, as on the Switch. A connect or disconnect signals that npad's
style-set event through hle_signal_bound_system_event (hle_thread.c), which signals only an event still bound,
from the macOS main thread where the notifications arrive; player LEDs follow the Switch's patterns; the
controller-support applet reports the connected players, capped at the game's maximum; only player 1 shakes.
DM_PAD_MERGE=1 merges every controller into player 1 as before; DM_KBD_PLAYER=own makes the keyboard player 1 on
its own. One controller, or the keyboard alone, behaves as before. iOS is unchanged: DmGamepad.m still feeds
player 1 and never reports further players.

Rebased onto a678703 (front_mac.m's includes, the Input row of docs/development.md). Checked: host unit tests
pass with the new t_hid (44 checks: slots and reconnects, npads docked and handheld, one- and four-player games,
events, LEDs, the applet); the Mac host builds for Breath of the Wild and Mario Kart 8 Deluxe with no new warning;
hle_hid.c and hle_thread.c pass an iOS SDK syntax check; Hades in a window with the keyboard and DM_KEYS reads
the supported ids (0x10f), and its scripted presses take it past the title. Two controllers were not run here;
the PR's author played Mario Kart 8 Deluxe in 2-player split screen with an Xbox Wireless Controller and a
DUALSHOCK 4, each its own kart, and a controller disconnected and reconnected during play got its slot back.

Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
2026-10-24 01:40:58 +08:00
seeddsandv0lumehi 98a91a6749 NSP script: --platform mac (PLATFORM = 'mac') builds only what running a game on this Mac needs, from PR #82
A development option; the Mac is the development host, not a platform for players. The same steps up to the
shaders (extraction, DLC, rewrite, run-time captures and decodes, heap check, shader split), then only the
Mac's: dmsh_build.py without IOSOUT (an inherited one is dropped), a Vulkan or GL game's library without --ios,
the Mac host for the profile into <id>_build/tools/dmhost (build_mac_host, through DM_GAMES_DIR /
DM_GAMES_FIRST for a provisional or derived profile) and the button-icon font into <id>_build/font. No iOS
shaders, Xcode project or IPA; the extracted files are kept whatever KEEP_EXTRACTED says, and the last line is
the command that runs the game (its save in <id>_build/run/save, DM_NVN_SHMAP for an NVN game, DM_AOC_DIR
with DLC, the --env-file switches). It refuses --prepare-xcode and an Intel Mac.

Reuse: the Mac libraries have a key of their own (shader/mac-b-<chain>.key beside the cache folder,
shader/vk.key / gl.key), which an iOS build writes too, so a Mac-only build after an IPA reuses its shaders;
either key only while the Mac libraries carry a full build's stamp (inputs.tsv). An iOS build after a Mac-only
one translates again (dmsh_build.py's per-shader keys include the iOS compile convention) and takes the Mac
libraries from the Metal compile cache. build_mac_host now has a reuse key too (the compile line, the sources and
headers of the folders it compiles, the generated profile header, libvpx, the third-party headers, clang and the
SDK), which also saves the rebuild for the Oodle / LZ5 decodes and the run-time captures. build_vk_shaders and
build_gl_shaders share layer_shaders. The default (ios) path builds as before.

Checked: dm/tools/tests 34 of 34 (11 in the new test_nsp_mac.py); Hades in Mac mode ran the Mac stages only,
printed the run command and exited 0, and a second run reused every stage, the host included (2 s); the host it
built boots Hades from that command. Breath of the Wild (NVN, with DLC) in Mac mode restored its 19,592 shaders
from the cache, printed a command with DM_AOC_DIR and DM_NVN_SHMAP, and the host booted it from that command into
play. Not run end to end here: a GL, provisional or derived game in Mac mode, and an IPA after a Mac-only build
(unit tests with the tools stubbed).

Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
2026-10-24 01:40:58 +08:00
seeddsandv0lumehi a6787033eb Mac, in fullscreen: a screen whose refresh rate is not a multiple of the game's frame rate switches to one that is (144 → 120 Hz), from PR #79
60 fps on a 144 Hz screen cannot be paced evenly: the display link counts a tick on the refresh nearest each 1/60 s
slot (link_range's virtual ticks), so frames stay up two and three refreshes in turn (13.9 / 20.8 ms) and Apple's HUD
shows the frame interval jumping between them. link_sync now tells the front end the link's rate (60, 30, or 120
with DM_PRESENT_LINK=2; front_screen_match), and in fullscreen front_mac.m switches a screen whose mode is not a
multiple of it to the mode of the same size with the highest rate that is, with kCGConfigureForAppOnly, so macOS
switches it back when the host quits; leaving fullscreen or that screen switches it back at once. A window keeps
the screen's rate and the virtual ticks: the switch changes the whole screen for every app and most external
screens go black for a second or more. DM_SCREEN_MATCH=0 keeps the rate in fullscreen too.

Changes to the PR: the switch only in fullscreen, with the way back on leaving it (the PR switched in a window
too); no switch without a window (the windowless host's main thread runs the guest); the rate to match is
atomic; the link takes a rate change only once its object exists, so one made while its thread starts is not lost,
and sets the range directly instead of logging a pacing mode -1; the screen this host switched takes its rate
from its mode, which maximumFramesPerSecond may not show yet. The link now also sets its range again when the
window moves to another screen, which kept the first screen's range before.

Checked on this Mac's built-in 120 Hz screen (a multiple of every link rate) with a harness that drives
front_mac.m asked for 50 fps: no switch in the window, 120 → 50 Hz in fullscreen, 120 Hz again on leaving it, and
120 Hz after quitting in fullscreen; at 60 fps and with DM_SCREEN_MATCH=0 nothing switched. Hades windowed and
fullscreen (DM_FULLSCREEN=1) paced at 60 as before, the unchanged host alike. nvn_mtl.m passes an iOS SDK
syntax check. The link's new range after a switch has not run here (it needs a 144 or 165 Hz screen); the PR's
author measured the windowed switch on a 144 Hz XB273U at 2560x1440: 120 Hz, the HUD's frame interval steady at
16.7 ms, 144 Hz again after quitting.

Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
2026-10-24 00:39:05 +08:00
seedds 86d9ba1239 One translation chain version for every game: DMSH_CHAIN in dm/gfx/nvn/dmsh_chain.h replaces the 47 dm/games/<id>/dmsh_chain.h, which every chain change rewrote and some were left behind (GitHub issue #78). The version hashed each profile's shader tables and nvn_* / shader_* keys too, once the only thing that kept the host off a library built from older tables; each shader's cache key (dmsh_build.py shader_key) has covered them since, so the version now covers only the chain's tools (dmsh_chain.py, dmsh_build.py, dmsh_fetch.py, dmsh_vin.py, dmsh_geometry.py, dmsh_facts.py, the translator's source, ios_metal.py, the lock) and the options, and needs no DM_GAME. What the per-game part still did is done by the library itself: a full dmsh_build.py run stamps b-<V>/inputs.tsv (Mac and iOS) with the game and the digest of the profile's inputs (dmsh_chain.py inputs / --inputs: the four shader tables, bf_want.tsv, which the variants use but the version never counted, and the keys), starts from no shader outputs (a partial one from none of the keys it builds), so a key that left the map, a shader that no longer builds or a variant no longer made keeps no old library, refuses a directory stamped for another game, and writes the profile's c1_full.tsv and cb_extent.tsv from its Mac outputs; a partial build keeps the stamp only while it names the current inputs. c1_guard.py / cb_extent.py head their table with the library's chain version and stamp (table_head), and the host takes a table only beside that stamp, logs the stamp and reports one of another game; build_ipa.sh reads the one header and stops when the iOS libraries' stamp is not this profile's current inputs or c1_full.tsv names another. NSP script: chain_header writes the one header when stale, before any host is built; provisional and derived profiles no longer write one; chain_tables makes a table again when it does not name the reused library's stamp (False without one). Makefiles take the header beside nvn_mtl.m; Xcode projects prepared earlier point at a profile header that is gone and are told to prepare again. Tests: test_nsp_chain.py (the committed header is current, stale and missing ones are written, tables against the stamp). Checked on The Great Ace Attorney Chronicles: a full build gave 460 of 460 shaders and both tables (0 and 409 entries, as before), the Mac host booted with the stamp and both tables past frame 900; one added facts row rebuilt one shader under the same chain version with a new digest and new tables, and the host refused the old tables beside the new stamp; a full build removed planted outputs, another game's build into that directory stopped, a partial build after an input change dropped the stamp; build_ipa.sh stopped on a stale stamp, a missing one and a stale or old-form c1_full.tsv; three NSP packs built (no table lines: dmsh_build made them), reused everything, and remade deleted tables from the reused build. Graphics unit tests give the same results as before (on this Mac t_refl needs ~/dm-work's libraries and several GPU tests fail on the previous commit too). AGENTS.md's chain procedure (header once after a tool change; a full shader build makes the tables), dm/games/README.md, dm/gfx/README.md and docs/development.md say so 2026-10-23 23:37:03 +08:00
seedds f4f0a85d6e patch_nso.py split into the general rewriter and per-build patch files (GitHub issue #63): every game's bring-up edited the one file, so PRs for unrelated games conflicted. dm/tools/patch_nso.py (1,711 → about 1,100 lines) keeps the general passes and the driver; the instruction encoders move unchanged to dm/tools/nso_asm.py; a game's own patches (CUTS, OBS and the frame-step stubs, now FRAME_STEP = {build ID: stub(ctx)}) move verbatim with their comments to dm/games/<id>/patches.py (16 profiles: Tears of the Kingdom, Breath of the Wild, Age of Calamity and Paper Mario: The Thousand-Year Door with their stubs), and the cuts of Nintendo's sdk module builds, 6 of the 15 shipped by 2 to 4 games, to dm/tools/sdk_patches.py. The rewriter loads every patch file, whatever the game, and applies those of a module's build ID at the points it did before (cuts before the scan, gates after the rewrite, stubs after the gates); a build ID claimed twice stops it, and its command line needs no DM_GAME. build_src.sha256 lists the rewriter's files and every patch file (the same for every module). NSP script: the rewrite step's reuse key counts nso_asm.py and only the patch files that name one of the game's builds (patch_files), profile_files leaves patches.py out (only the rewrite step reads it), and a derived profile gets none. dm/tools/tests/test_patches.py: no build claimed twice, the entries' forms, a game's file names only its own profile's non-sdk builds, sdk_patches.py only sdk builds some profile ships, and the reuse key's patch files. Checked byte-identical: every extracted module on the game drive (131 builds) rewritten with the previous and the new rewriter gives the same outputs but build_src.sha256, 28 of the 30 patched builds among them (all four frame-step mains), and 7 fail alike (rtld, and New Super Mario Bros. U Deluxe's, which no profile rewrites); the other two (Super Mario Bros. Wonder's mains and Absolum's, no game files here) by their CUTS / OBS data, equal to the previous tables. AGENTS.md, docs, dm/games/README.md, the profiles' comments and the host comments that named a game's cut in patch_nso.py now name its patches.py or sdk_patches.py 2026-10-23 23:36:21 +08:00
seedds 4c7adf4619 c1_full.tsv and cb_extent.tsv are no longer committed: each checkout makes them from its own shader build's Mac outputs (git ignores them in dm/games/*/), as the NSP script does after its iOS shader build. They follow from the MSL this Mac built, every chain change rewrote about 90 of them, and 56704a0 could only carry most over unchecked; 81 files, 18.7 MB. The NSP script makes a missing one as a stale one (chain_tables now says whether both name the chain: with no Mac outputs a reuse builds again and a build stops), and no step's reuse key counts them (profile_files), so the run after they were made does not split again. build_ipa.sh stops for an NVN game whose c1_full.tsv is missing or names another chain, with the c1_guard.py command, before generating anything (the host would give every c1 its own 64 KiB). Checked on The Great Ace Attorney Chronicles from a checkout without them: the first pack made both (0 and 409 entries, as committed), the second reused the split and the iOS shaders with no table lines, a third with them deleted made them from the reused build's Mac outputs; build_ipa.sh stops on a missing and a stale c1_full.tsv. AGENTS.md's chain procedure, dm/games/README.md, dm/gfx/README.md and docs/development.md say so 2026-10-23 22:24:10 +08:00
seedds a2e6f4a18c NSP script: a registered profile's chain files that a chain change left behind are made again (GitHub issue #78: after bb585cd, 45 of 47 profiles' dmsh_chain.h no longer held the version their source gives, and Breath of the Wild's pack stopped at dmsh_build). chain_header writes dmsh_chain.h again, before any host is built, when it does not hold the version dmsh_chain.py gives (it follows from the source alone; a current one is left untouched, so the host is not rebuilt). chain_tables makes c1_full.tsv (c1_guard.py) or cb_extent.tsv (cb_extent.py) again from the build's Mac outputs when its header names another chain, which the host would ignore (every c1 its own 64 KiB, whole uniform bindings copied), also when the libraries are reused; provisional and derived profiles make both as before. The two tables are the shader step's outputs, so its reuse key no longer counts them. Tests: dm/tools/tests/test_nsp_chain.py (stale, missing and current headers and tables), and chain_tables with the real tools on Breath of the Wild's Mac cache (47 s, the committed list's 2 entries) 2026-10-23 22:11:09 +08:00
seedds 6ec201c756 Mario Kart 8 Deluxe (0100152000022000, update 4.0.0, Nintendo's sead / agl / gsys on SDK 4.4.0, the oldest SDK so far): game profile and five general changes. Plays on the Mac and an iPhone from boot through the title's demo race, the Mii pick and the menus into Grand Prix races at 60 fps with sound, and a second boot loads the save. The Mii database (hle_mii.c: the six default Miis every console has, which the game lists at boot without reading the count, none of the console's own, random Miis for the demo race, the editor cancelled) and a MiiModel system data stand-in (0100000000000802 mounted as MiiResource:, written by the host with the layout main's Mii renderer reads: no textures and every shape one degenerate triangle, because the faceline is drawn without a check; the firmware is not the host's to ship, so Mii heads draw nothing). The add-on content cache sized as Horizon's RomFS working memory (its licence is decrypted in a 0x3130-byte heap that also holds the 4 KB the host used to answer). Device save data as a save of its own when main also imports MountSaveData (at boot the game deletes CommonSave:/userdata.dat, which in the shared directory was UserSave's; Animal Crossing, device save only, keeps its island where it was). SDK 4's DeviceSink (command 0xf, 0x130 bytes, byte input indices) and the audio renderer's update lock read off each SDK's own AudioRenderSystem::Update (LockMutex +0x1d0 / +0x1d8, SdkMutexType +0x1d8 / +0x1f0 / +0x1f8: it was SDK 16's for every SDK before 20; sound checked on Big Brain Academy and DRAGON QUEST XI S). The zero cube for a 2D array of fewer than six layers in a cube slot. Profile: render_cond on, a recorded prewarm list from boot into the first race; Breath of the Wild and Animal Crossing boot with their DLC caches resized, host unit tests pass 2026-10-23 20:36:56 +08:00
seedds bb585cd5f2 Add compute-backed geometry shader replay
Capture vertex outputs, run geometry shaders as compute, and rasterize compacted primitives through a general NVN pipeline. Rebuild the pinned translator and AI shader chain, add geometry regression coverage, and document supported behavior and remaining limits.
2026-10-23 16:29:24 +08:00
seedds aea97ec65f Add ANIMAL WELL port verified on iPhone 2026-10-23 15:15:42 +08:00
seedds fa7ad177cf AI: THE SOMNIUM FILES (010089B00D09C000, update 1.0.1, Unity 2017.4.17f1 IL2CPP with FMOD, SDK 6.4.2): game profile (no system resource, a 3.11 GB heap in the 4 GB layout, mem_used for the rewritten modules, docked, 3D pad layout for the investigation's view on the right stick, 13,057 shaders from its AssetBundles with Cuphead's Unity 2017.4 layout, a prewarm list from boot to the first investigation) and two general host fixes. Movies: a released output slot also goes on with an H.264 drain, and the end-of-stream packet's own output takes the first queued picture (hle_movie.c); Unity held all 16 video outputs when the Spike Chunsoft logo movie's end-of-stream packet came, the last pictures and the end-of-stream flag never came out, and the game waited before its title for good (Raging Loop's opening movie still ends and reaches its title). Per-stage programs: each restore of the replay state takes the combined program again from the per-stage bindings (nvn_mtl.m prog_resolve); the precompile prediction over a 280-draw submission reused the ring slot the saved state named, and the draws after it ran another pair's shaders (a pipeline Metal refused, VertexLit's shadow vertex shader with BG_AlphaTest's pixel shader, and a cube / 2D texture mismatch; 2 of 2 old-host runs from one save, 0 of 2 fixed). Plays on the Mac from boot through a new game into the first investigation at 60 fps, 2.27 GB resident, and on the iPhone 2026-10-23 15:04:08 +08:00
seedds 1822616144 Intel Macs (GitHub issue #54): the shader translator's launcher dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate is a universal binary (arm64 and x86_64, the same .NET 10.0.13 host joined with lipo); it was arm64 only, and the NSP script stopped at dmsh_build with Bad CPU type in executable. The lock does not cover the launcher, so the chain version is unchanged. The x86_64 translator under Rosetta gives byte-identical output (Raging Loop, 452 shaders; Ryujinx's constant folding uses only exactly rounded arithmetic). The Mac host runs the game's own arm64 code, so on an Intel Mac a step that boots the game there (a run-time shader capture, a decode with the game's own Oodle or LZ5) stops with a plain error, and a provisional profile's heap check is skipped (build_ipa_from_nsp.py apple_silicon) 2026-10-23 14:42:50 +08:00
seedds 7deea8eedc Tears of the Kingdom: the SDK's address-region node area cut from ~1 GB to 62 MB on the sdk build it shares with Breath of the Wild 1.9.0 (patch_nso.py). The game's resource heaps are address regions in the 2.375 GB alias region; at the first descent from the Great Sky Island it was used up to +2.25 GB with 1.00 GB mapped, and the game took a memory shortage (MSF / R / Rf) and played the memory-shortage Blood Moon, which put Link back at the start (GitHub issue #70; 7 of 8 Mac runs from the issue's save). With the cut: +1.75 GB, no shortage in 3.7 minutes on the island. Probe tools: the write watch reports exclusive and atomic stores, and DM_PROBE_POLL snapshots every thread when a word changes 2026-10-23 14:13:52 +08:00