f171088e2e
Fire Emblem: Three Houses: record why its two pacing cuts stay per-build. Checked on the Mac (host at 7753b7b, headless boot to the title, 100 s per run, each cut undone in the rewritten main): without the frame-limiter cut the general yield back-off holds 59.7-59.8 fps (frame p50 16.73 ms against 16.68) at 0.9-1.2 W of CPU energy against 0.14-0.42 W (about 9,700 plain yields a frame before the back-off starts at 2 ms); without the render-thread timeout cut, 53,000-55,000 wakeups a second against 1,450 and 0.7-1.0 W. The 16 us timeout is the game's own: the render thread passes (1/fps) x 1000 = 16 to main+0xaa90e0, which multiplies it by 1000 into the TimeSpan (nanoseconds) of nn::os::TimedWaitEvent. Of the 206 modules on hand, the limiter (main+0x5be4b4) is the only loop whose only calls are YieldThread and GetSystemTick, so a rewriter pass for such loops would match this one site. Docs: the game note; docs tests pass
seedds2026-10-26 13:29:13 +08:00
dac7ba6338
Function-matched cuts: same-name rules are one patch's implementations; Taiko Risshiden V DX's frame-limiter cut reaches the base game and later builds. nso_match.resolve groups rules by name: exactly one implementation must match (the log names it), a red line only when none or more than one does. Before, each implementation of a changed function logged a red line on every build it did not fit. Taiko's MATCH_CUTS has two implementations of the frame end under one name: 1.1.1's (main+0x443e8c, 364 bytes, cut at +288) and the base game's (v0 main 47d7baf6..., main+0x43e140, 172 bytes: 1.1.1 added the frame-skip branch; the same 20 bytes at +104 = main+0x43e1a8). In v0 N is likewise a global (GOT 0xa1be28, data 0x9ec55c, 1) set to 2 only by CPrivateBattleWindow / CPrivateBattleTutorialWindow, and the start tick is read after the update; nso_fnmap.py finds no exact match for 1.1.1's function in v0, found by disassembly. Checked through patch_nso.py: v0 main takes the cut at 0x43e1a8 (implementation 2, no red line); a 1.1.1 main copy with another build ID takes implementation 1, output byte-identical to the registered cut; 1.1.1 keeps its exact CUTS. A scan of the 42 distinct mains on the drives found this limiter only in Taiko. The base game has not been run. Docs: rewriter.md (same-name implementations, procedure step 2, Found on), the game note. Checked: test_nso_match.py (new test fails on the old matcher), all 19 tool test files, docs tests
seedds2026-10-26 13:24:01 +08:00
7753b7b685
Taiko Risshiden V DX: keep the frame limiter for N = 2, cut it only at N = 1, and record its cause. KTGL's frame end (main+0x443e8c) presents, acquires, then waits until N / 60 s since the frame's start tick in 2 ms SleepThread rounds; the tick is read after the scene's update (main+0x443d44), so every frame lasts N / 60 s plus the time to the next start, rounded up: never one vsync, on any hardware. That is the game's code, not the host (SleepThread and the tick are exact; a model of the loop gives 54 fps with the host's acquire and with the Switch's double-buffered one). N (GOT 0xa1c088, data 0x9ec764) is the game's vsyncs per frame: 2 in CPrivateBattleWindow / CPrivateBattleTutorialWindow (main+0xf02d0, main+0xec4cc), whose code counts 60 / N frames a second, and the game presents at interval 1 everywhere. The earlier cut (b.hs -> b at main+0x443fd4) skipped the wait for every N: with N forced to 2 it presented 60 frames a second against the game's 28.8. The new cut at main+0x443fac computes the threshold with udiv by 60 and sets it to 0 only when N = 1, where the host's present pacing holds one vsync. Checked on the Mac (host at d50ecc1, Mac libraries for chain 8393d85a4eae): title N = 1 54.6 -> 60.0 fps (frame p50 16.67 ms, wakeups 2,050 -> 300/s), N forced to 2 28.9 fps (no cut 28.8, old cut 60.0); town on the rewriter's own output 60.0 fps, p50 16.66 / p95 17.08 ms, no red line; patch_nso.py applies the cut with its byte checks. The CPrivateBattle scenes (small unit battles, PBBATTLE) were not reached: open in the game note. Docs: the game note and the frame-rate reference entry give the corrected cause; tool tests (17 files) and docs tests pass
seedds2026-10-26 13:04:04 +08:00
fb20bce70e
Add GPU diagnostics and close periodic profiles per frame (#225)
seeddsandv0lumehi2026-10-26 12:09:07 +08:00
3f22526e87
Add repeatable Mac benchmarks and harden report generation (#191)
seeddsandv0lumehi2026-10-26 12:08:47 +08:00
d50ecc1f84
Support masked-predecessor jump tables and external patches (#237)
seeddsandAmost2026-10-26 11:03:47 +08:00
c4b74a62f9
Add thermal frame-rate cap with synchronized recovery (#220)
seeddsandv0lumehi2026-10-26 10:13:42 +08:00
c76f679834
Remove alpha_from_red: Super Mario Odyssey's layout captures need only the nvnTextureCompare fix. The game creates each capture with its per-format default mapping (an R8 one reads (R, 0, 0, 1)), rebuilds it on the same storage with the capture's own, (1, 1, 1, R), and replaces the registered descriptor only on a difference (main+0x982320); while the host's comparison ignored the mapping the captures kept alpha 1 (white coin cards, a dark box behind hints), which the 26-shader alpha_from_red list papered over. Checked on the Mac from the "Look Around" hint with a probe host on the comparison (a c3a0880^ host for the profile's chain dec5ee141a9f libraries): 40 mapping-only rebuilds in 11,700 frames (R8 captures 640x80 down to 48x57 to (1, 1, 1, R), a 320x180 RGBA one to (R, G, B, A)); ignoring the mapping again brings back the box (about 12,800 pixels from frame 4,200), the old list hid it by converting the 378x80 capture, and with the fix the list converted nothing and changed no pixel of frames 4,200-6,600. The comparison and the views have no platform branch, so the iPhone gets the same rebuilt mapping (not run on a device). With no profile left using it, the key, its host code (tex_alpha_red, afr_want, DM_NVN_ALPHA_RED) and its docs go; treg_drop_ar becomes treg_drop_im (the image-slot view it also drops stays). Odyssey's note has the A/B runs. Checked: the host builds for Odyssey, the iOS compile check (133 files), the tool tests (111, 14 skipped)
seedds2026-10-26 10:01:20 +08:00
16061e0399
Rule: fix a bug at its root cause, not around it. AGENTS.md says to find why a bug happens (the faulting instruction and its decompilation, the bad value and what wrote it, the semantics the host or translator gets wrong) and fix it there; a change that only hides the symptom (stub, skipped call, clamp, retry, sleep or timeout, cache flush, profile key that turns a feature off) is a workaround, used only when the cause lies where it cannot be fixed (the game's own code, changed deliberately by a rewriter cut, or a platform limit) or the user asks for a stopgap, and then recorded as open with the cause or what is unknown. docs/agents/README.md has the same as a ground rule. Checked: dm/tools/tests/test_docs.py passes (AGENTS.md 1,419 words)
seedds2026-10-26 09:42:56 +08:00
c13ddcf531
Add opt-in MetalFX spatial upscaling and a single-screen launcher (#215)
seeddsandv0lumehi2026-10-26 09:36:56 +08:00
3aa706a3fd
Pokémon Shield (01008DB008C2C000, update 1.3.2 = v458752, Game Freak's engine on SDK 7.3.2): game profile, no host change. Sword's twin: the same sdk, subsdk0 (movie) and subsdk1 (GLSLC) builds, only main is its own (a1680262…), and the shader split gives Sword's 41,516 shaders with the same data rows in all three tables; the profile is Sword's with Shield's title, name, IPA name, build row and tables (prewarm list copied). The game note (dm/games/01008DB008C2C000/README.md) has what differs, the Mac route (a fresh save needs DM_KBD for the name; Sword's route out of the house works unchanged) and the shader build with DMSH_PREFILL from Sword's libraries (21,974 Mac / 21,684 iOS filled, 176 / 175 compiled). Plays on the iPhone (IPA 12.4 GB, user) and on the Mac at 30 fps with no red line from the first boot through the opening and Postwick to Hop. README gallery entry with a Mac screenshot, a row in the games table, a pointer from Sword's note, AGENTS.md's count to sixty-seven. Checked: the tool tests pass (111, 14 skipped)
seedds2026-10-26 09:19:04 +08:00
3dd5d25a51
Add Echoes of Wisdom 1.1.0 port and correct cube fallback diagnostic
seedds2026-10-26 09:08:12 +08:00
037ff69562
Profile keys present_snap and pso_mtl4: a game ships the presentation and pipeline settings it was tested with. present_snap on: present snapshots by default (a ring of 2 host textures; the iOS launcher's Frame buffering defaults to it; DM_PRESENT_SNAP and Documents/env.txt override); pso_mtl4 on: render pipelines through MTL4Compiler on iOS by default, with the pipeline archive, the prepare step and the hybrid pipelines (on by default on the Mac; DM_PSO_MTL4 overrides). Breath of the Wild sets both: they held a steady 60 fps on an iPhone 17 Pro Max (issues #142 / #159; without the snapshots the GPU idled between frames and iOS lowered its clock, 30 fps). README, docs/development.md and docs/reference/frame-rate.md say the new defaults. Checked: the host builds for Breath of the Wild (DM_GAME_PRESENT_SNAP 1, DM_GAME_PSO_MTL4 1) and Mario Wonder (0, 0), dm/tools/tests pass, t_present_snap and t_nvn pass with Breath of the Wild's profile, the iOS compile check passes
v0lumehi2026-10-25 18:36:28 +01:00
f6bd56f5cc
The pipeline prebuild runs in the prepare step: with a window (and on the iPhone) the bases of every shader pair are made right after the prewarm, before the game's first frame, behind the progress screen ("Preparing pipelines"), once per chain version, depth mode and pair list (prebuilt.tsv). Built in the background while playing, they took an iPhone 17 Pro Max's GPU from about 13 to 15-18 ms a frame in Breath of the Wild (the compiler's load on the shared power budget: 43-95 dropped frames per 300 against 0-25 without it); without a prepare step the background prebuild stays. Tests: t_nvn 128, t_pso_archive 57, t_pull 346, iOS compile check
v0lumehi2026-10-25 18:42:06 +01:00
1eeb109b35
docs: the prepare step (docs/development.md "The prepare step", Breath of the Wild's measurements in its note dm/games/01007EF00011E000/README.md; dm/gfx/README.md DM_PSO_PREPARE; README). Mac, M2 Max, runs started at a 1-minute load below 4, Metal cache empty, the prebuilt archive, the Zora's Domain save: with no list, 4 frames over 25 ms with new pipelines and 5 with shader library loads (75 libraries, 168 ms of that work, worst 128 ms) in the 60 s after the last menu press; with the new pso_manifest.tsv none, whether it was built in the background (14.6 and 16.0 s on one worker, done during boot) or by the prepare step (5.2 and 2.2 s on 12 workers, 3.3 s on 8, 872 bases compiled); a second start prepares it in 0.08 s and a player's 5,783-line list in 0.23 s, both with no screen. Hades (Vulkan) is unchanged: it builds its 1,856 pipelines itself while it loads
v0lumehi2026-10-24 18:53:17 +02:00
d9aa864d0d
Breath of the Wild: a prewarm seed list (pso_manifest.tsv, 1,789 lines: 960 render pipelines, 819 shaders, 10 compute shaders), so a first start has something to prepare. Recorded on the Mac with prewarm off (DM_PSO_MANIFEST=off DM_PSO_RECORD) over update 1.9.0: boot, title, New Game and the Shrine of Resurrection until Link can move (1,298 lines); then boot, title, a save load into Zora's Domain and a minute of play, whose new lines are appended (491). The lines name shaders by their Switch content key and pipelines by NVN state (pso_skey), which no translation chain version changes: the save-load part was recorded on chain b7b9aa970b37 and every line resolves on 97e2f301eb73
v0lumehi2026-10-24 18:53:17 +02:00
7a47d1c5eb
The prepare step: with a window, the game's first submission waits until the prewarm lists (pso_manifest.tsv, then pso_user.tsv) are built, on every core at user-initiated QoS, behind a "Preparing shaders" screen that shows only past 0.5 s and with 0.3 s or more still to go. nvn_mtl.m: the prewarm reads its lists into tasks (warm_parse): a shader, a compute pipeline, or all render pipelines of one shader pair, which share their bases, so no two workers compile the same base; shaders go into one dictionary the workers share, and a vertex shader's pulling entry is loaded with it (as shader_prefetch_rest does), so neither a draw nor its hybrid stand-in loads a library the lists name. warm_prepare runs the tasks on hw.activecpu workers (DM_PSO_PREPARE_THREADS=<n>) from mtl_sub_begin's first call, before anything of that submission is encoded, and gives the front end done of total every 50 ms once 0.5 s have passed, unless the rate so far leaves less than 0.3 s; the shaders and pipelines go into the caches the draws use at once (warm_claim), which keep them for the run. It runs at the first submission and not before the game starts because the vertex entry each pipeline is built with follows the depth mode the game sets (nvnDeviceSetDepthMode). It runs at every start: with the Metal cache and the pipeline archive warm it takes well under a second and shows nothing, and a stamp that skipped it would skip it exactly when the system has dropped the Metal cache (an OS update, storage pressure on iOS). Without a window, or with DM_PSO_PREPARE=0, the prewarm runs in the background on one worker as before; DM_PSO_PREPARE=1 makes a windowless run wait too. The log line: pipeline prepare: N lines (shaders, render and compute pipelines) in s on N workers before the first frame, progress screen shown or not; bases compiled, from the archive, reused. front.h front_progress: front_mac.m draws the screen with AppKit views over the game's view, front_ios.m with UIKit views over the game view and the pad (a dark view, the title, a bar and a count); nvn_mtl.m keeps a weak empty one for the unit tests. NVN only: the Vulkan and GL layers keep no prewarm lists
v0lumehi2026-10-24 18:53:17 +02:00
9607f51022
iOS app: shader_pairs.tsv and pso_pairs.tsv go into the bundle beside the shader map (build_ipa.sh), where the pipeline prebuild reads them; without them an iPhone made no pulling bases and hybrid draws never had a stand-in (Breath of the Wild's log: 'pipeline prebuild: no …/dm/shader_pairs.tsv, nothing to build', 0 draws with a hybrid pipeline in 15,600 frames)
v0lumehi2026-10-25 17:24:32 +01:00
e0ef4e707e
Pipeline prebuild: the pulling base of every known shader pair into the pipeline archive, with the pairs of agl's BAHS program variations (pso_pairs.tsv)
v0lumehi2026-10-24 13:01:23 +02:00
8593d85a9c
docs/development.md: a diagram of the hybrid miss path (cache, base and specialization, pulling stand-in with the draw's own pipeline queued first, or a build at once with the bounded prediction)
v0lumehi2026-10-24 13:07:46 +02:00
bc76cab190
Hybrid pipelines: a draw whose pipeline is not built yet draws with its shader pair's pulling pipeline meanwhile, on main's demand-first pipeline builds
v0lumehi2026-10-24 12:54:51 +02:00
e6b800a56d
Pulling vertex shader variants in the translation chain (<key>_bp, dmsh_pull.py with its decoder dmsh_pull.metal)
v0lumehi2026-10-24 14:43:00 +02:00
7dc250e2f1
Bound NVN upload staging and retire it on upload completion
seedds2026-10-26 01:05:41 +08:00
2ef0a17e15
Pipeline keys: the write mask and blend bit of colour targets a normal pass lacks leave the key too, and prewarm rows are normalized when read (general, from MacNativeGames)
Aleksandr Svidchenkov2026-10-25 20:18:40 +04:00
5a81e13f24
NVN on Metal: no autorelease in ReadTexels and arena_alloc, and a registry lock that lends priority (general, from MacNativeGames)
Aleksandr Svidchenkov2026-10-25 20:18:26 +04:00
be5b197640
Port Persona 5 Royal with Needle shaders and NVN texture fixes
seedds2026-10-25 23:47:28 +08:00
8ca9e9af93
Widen iOS launcher settings and remove list heading
seedds2026-10-25 19:34:06 +08:00
9116109c9f
Expand CI with texture, sanitizer and unsigned iOS build checks
seedds2026-10-25 19:08:45 +08:00
d62ddbbd25
Use native navigation list for iOS launcher settings
seedds2026-10-25 18:59:47 +08:00
641799d51b
Fix isolated SVC test linking after Unreal heap support
seedds2026-10-25 18:45:56 +08:00
a1886130a6
Add opt-in frame buffering with safe snapshot pacing and suspend
seeddsandv0lumehi2026-10-25 18:39:57 +08:00
cbbb600866
OCTOPATH TRAVELER II: the IPA with the geometry split (geo_split, three-dispatch index compaction) and the Unreal-heap thread stacks (16f4889) plays on the iPhone (user); the game note says so and drops the open item
seedds2026-10-25 17:46:41 +08:00
7d368c4249
Ghost Trick: Phantom Detective (010029B018432000, base game 1.0.0 = v0, Capcom's RE Engine on SDK 13.3.0, with the Bonus Content add-on): game profile, and finalized pool buffers no longer kept by a guest thread's autorelease pool (every game). Plays on the iPhone (IPA 3.0 GB, user)
seedds2026-10-25 17:42:38 +08:00
16f4889192
OCTOPATH TRAVELER II (0100A3501946E000, update 1.1.1 = v131072, Unreal Engine 4.27 on SDK 12.3.2): game profile, inline Unreal shaders in version 11 paks, and five general host fixes. Plays on the iPhone (IPA 5.9 GB, user; that IPA predates the geometry and allocator changes below). On the Mac: the boot, the title, a new game, the traveler selection, Hikari's opening into his first battle (won) and the scene after it, and Castti's opening (the flashback, the ship, free play on the deck), at 60 fps (frame time p50 16.7 ms, p95 about 17 ms) with CRIWARE sound, about 2.8 GB resident. Shaders: the project ships no shader code archives (bShareMaterialShaderCode off), so each material's shaders are inline in its .uexp and the global ones in Engine/GlobalShaderCache-SF_GLSL_SWITCH.bin, each an entry of FShaderMapResourceCode (an LZ4 block, its raw size, its frequency): uepak.py inline_shaders sends a version 11 pak with cooked packages and no .ushaderbytecode to shader_split.py's inline reader (ue_legacy_jobs; Zlib and LZ4 pak blocks decoded in Python, Pak.read), ue_inline_shaders finds each LZ4 block by its first literals (LSLG), and profile_guess.py / the NSP script tell the two kinds apart; 13,980 shaders in 13 s. Profile: Unreal's FMallocBinned3 cut with an OBS_UE_MALLOC gate as DRAGON QUEST VII Reimagined's (GCreateMalloc main+0xbecbf0, a 20-slot vtable, hle_uemalloc.c), a 2 GB engine region (game_args) with the node-area cut on this SDK 12.3.2 build (sdk_patches.py, sdk+0x346190 / +0x34622c), fps_target 60 through t.MaxFPS / rhi.SyncInterval (hle_fpslock.c; the same route at 30 and 60 fps reaches the same scenes at the same times), pad_layout full (the battle menu picks weapons with the D-pad), a prewarm list of 843 lines. General fixes: (1) a shader may bind 96 resources instead of 64 (NVN_REFL_MAXB: the traveler selection's 3D map shader binds 66, and the map was black); (2) the parallel precompile look-ahead (pso_predict_rest) restores the scissor, which state_op keeps in R, not S: with pipelines built on demand the picture went bright green and white from Castti's flashback and Hikari's battlefield on (bisected with simultaneous runs; DM_NVN_PREDICT=0 turns the look-ahead off); (3) a geometry draw above the 256 MiB capture scratch bound is drawn in runs instead of skipped (nvn_geometry.inc geo_split: one instance at a time, then runs of at most 4,096 list primitives whose index range fits 64 MB, one read of the indices, one GPU wait), the index compaction after the geometry kernel runs as count / offset / write dispatches instead of one thread (each frame drawing Unreal's 129,032-triangle point light shadow took about 300 ms with it), and scratch arenas above 32 MB are released after 600 idle submissions (arena_new takes the smallest that fits); (4) MapMemory takes a thread stack from the host's Unreal heap (dm_ue_heap_owns: a range outside the guest block the host maps read-write, vm_region_64): with cae5ec2's stricter source check every Unreal game on the host heap aborted in MapAliasStack at nnMain (checked: this game and Pikmin 4 boot); (5) the host Makefile rebuilds on a change to dm/gfx/nvn/*.inc. Scripted input takes partly tilted sticks (L<x>/<y>): the traveler selection opens a card only while its quill cursor rests on a traveler, and full-tilt presses swept past them. Open: 2 of 87 Mac boots crashed at the title in the game's CRI cue sheet loader (main+0x1d8054, a record copied while one of its TArrays held count 1 and no data); the writer is not found yet (the game note). Checked: 57 tool tests, the gfx run list (t_geometry 95 checks) and the docs check pass; the NSP script packs the IPA from the NSPs (Mac and iOS). README (games gallery, a Mac picture of Hikari's first battle), the game note, the games table, docs/reference shaders / memory, docs/agents/04 and the symptom index say so
seedds2026-10-25 15:43:48 +08:00
18414bb5ff
Fix resource heap slack for Super Mario Bros. Wonder 1.0.1
seedds2026-10-25 15:33:15 +08:00
93578abad6
13 Sentinels: Aegis Rim, Asian release (01008D7016438000, base game 1.00 = v0, Japanese / Korean / Chinese): game profile, no host change (PR #161 by lx7720)
seedds2026-10-25 13:59:43 +08:00
cdd5619226
Docs check: new dm/tools/tests/test_docs.py, which the CI job runs with the other tool tests. It fails when a relative link or #anchor in the repository's own Markdown does not resolve (GitHub's heading anchors, repeated headings numbered; files from git ls-files, third_party and node_modules left out), when a profile with a game.tsv has no README.md or no row in the games table of dm/games/README.md (or the table lists a folder with no profile), when docs/agents/symptoms.md is not what dm/tools/docs_symptoms.py writes from docs/reference, or when AGENTS.md passes 1,500 words or a numbered guide in docs/agents 6,000. AGENTS.md says so. Checked: the test passes (88 Markdown files, 2,293 links, 60 profiles), and fails as it should with a link to a missing file, a link to a missing heading, a Symptom bullet changed without regenerating the index, and AGENTS.md 400 words longer
seedds2026-10-25 13:07:24 +08:00
0df4dbe280
A short AGENTS.md: what the repository is (the native-execution model, the host and its three graphics layers, sixty games, each with its note in dm/games/<id>/README.md and a row in the games table), where to start for each kind of task (a crash, abort, hang, [RED] line or wrong picture: the symptom index, then the reference entry and its games; a game short of its frame loop: 03; rendering: 04 and the rendering, shaders and NVN objects references; one game or its profile: its note; another title: the NSP script and docs/agents; packing, building, prewarm, power: development.md), and where a new finding goes, once (a fact about one game in its note; a general mechanism as an entry in its reference, the symptom index regenerated and a line in the note's Found on this game; this file only for a rule or the count). The rules are as before. The game list of 4,500 words, one sentence per game, is gone: each sentence is in its game's note. New dm/tools/docs_symptoms.py writes docs/agents/symptoms.md from the references' Symptom bullets (--check reports whether it is current). The NSP script's survey points at docs/reference/nvn-objects.md for SDK object sizes and at the symptom index; Balatro's note names capture_shaders. AGENTS.md goes from 6,235 to 1,137 words. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 on main still appears in the repository's Markdown (besides the five file patterns, nn::hid::GetGestureStates and nn::oe::SetCpuOverclockEnabled appear without the nn:: prefix in Taiko Risshiden V DX's note), every relative link and anchor resolves, and the tool tests and test_xcode_workflow pass
seedds2026-10-25 12:56:14 +08:00
50ad443cb1
Bayonetta and Bayonetta 2 in the new layout: their game notes (dm/games/010076F0049A2000/README.md, dm/games/01007960049A0000/README.md) and rows in the games table, from their Game notes entries in docs/development.md, their AGENTS.md sentences, their profiles' comments and their commits; the SDK 3.6.0 entry of 03's catalogue becomes services.md "SDK 3.6.0 answered as SDK 4", and 04's two Seen-in sections become shaders.md "BNSH files in PlatinumGames DAT archives inside CRI CPK archives" (cpk.py, cpk_jobs) and rendering.md "Clears of attachments larger than the render pass" (rp_begin_ex_) and "Downsample of a single-sample texture" (copy_t2t), each with the games it was found on. Mario Kart 8 Deluxe's note no longer calls SDK 4.4.0 the oldest. Checked: every profile with a game.tsv has a note and a table row (60), every relative link and anchor in the repository's Markdown resolves, and every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 on main (4,669 items) still appears (the same five file patterns as before)
seedds2026-10-25 12:44:03 +08:00
3a993fa3fe
Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in <game>" subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games/<id>/README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv)
seedds2026-10-25 12:24:24 +08:00
846c54c63c
Per-game notes beside each profile: dm/games/<id>/README.md for all 58 games, and the games table in dm/games/README.md. Each note says what the profile needs and why (a table of its game.tsv keys, patches.py cuts and frame-step stubs, sdk_patches.py cuts for its sdk build, manual shader rows), the game's own findings by topic (frame rate, memory, rendering, audio, movies, services, files and shaders, input, Mac routes), each marked current or resolved, a "Found on this game" list of the general host changes first needed there (one line each, linked to where they are explained), and what is still open, including stale profile comments and figures the sources disagree on. The notes gather what was spread over AGENTS.md, docs/development.md (Game notes and Power), docs/agents/01-05 and the profiles' own comments. The Game notes section of docs/development.md (523 lines, 53 games) is now a pointer to the table; development.md goes from 33,300 to 8,600 words. README.md, AGENTS.md and the Breath of the Wild link in 04-graphics.md point to the notes. A README.md in a profile folder changes no shader cache key (dmsh_chain.inputs hashes only the named tables and keys) and no profile listing (game.py lists folders with a game.tsv). Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (five file patterns now appear as the full names they stood for), and every relative link and heading anchor in the repository's Markdown resolves
seedds2026-10-25 03:25:48 +08:00
b7dfba4dd0
Load LZ4 only when decoding NSO files
seedds2026-10-25 12:27:40 +08:00
66abbc3d30
Cache Metal 4 pipeline bases with per-game archives
seeddsandv0lumehi2026-10-25 11:31:14 +08:00
9ba44da80a
Fail fast when SetHeapSize protection changes fail
seeddsandnocmt2026-10-25 11:19:22 +08:00
1fbbd1486a
Bayonetta (010076F0049A2000, update 1.1 = v65536, PlatinumGames' engine in its Wii U form on SDK 3.6.0, as Bayonetta 2): game profile, no host change. Plays on the iPhone (IPA 8.8 GB, user). Same sdk and subsdk0 builds as Bayonetta 2, so its profile is Bayonetta 2's with its own build row and tables, and the CPK reader and the two NVN fixes made for Bayonetta 2 cover it. On the Mac from a fresh save: the boot, the opening movie (USM; Plus skips it), the title and its attract demo, a new game (PLAY, NEW GAME, the auto-save notice, Normal), the prologue on the falling clock tower, the graveyard cutscene (skipped), the touch-controls tips, the practice tutorial (punch, kick, combos, evade, Witch Time), the graveyard fight of Verse 1 with Witch Time, its autosave and the next loading screen's practice, with CRI ADX2 sound through one 6-channel AudioOut and no red line: 60 fps in play (frame time p50 16.65 ms, p95 17.1-17.5 ms; GPU 2-4 ms a frame on an M6), 30 in its cutscenes, where the game sets present interval 2; about 2.45 GB resident and 1.54 GB of Metal allocations, flat over 15 minutes. No virtual address memory (a 3.11 GB heap in the 4 GB layout, mem_used 102 MB). Files: nine CRI CPK archives (data00.cpk ... data03.cpk, data10.cpk ... data14.cpk, 4.25 GB) of PlatinumGames DAT archives, beside the USM movies and the ADX2 sound. Shaders: 458 BNSH files of one stage each (236 pixel, 222 vertex), the .vso / .pso members of shader.dat in data00.cpk, all built for the Mac and iOS. Profile: the 3D pad layout (the camera is on the right stick; the touch controls are handheld only), a prewarm list recorded on the Mac over the route above (513 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); the Python tool, gfx and iOS tests pass (test_tbdr_budget with DM_GAME set). README (games gallery, a Mac picture of the graveyard fight), AGENTS.md (59 -> 60), docs/development.md and docs/agents 01 / 03 / 04 say so
seedds2026-10-25 11:03:12 +08:00
787523f2cc
Mac front end: input held when the game window loses focus is released (general, from PR #128). A window that is no longer key sees no key-up, mouse-up or controller release, so a key, a click or a controller button held when the window lost focus (another app, the software keyboard's text box, minimizing) stayed pressed in the game. Now front_mac.m releases what the front end itself pressed when the window resigns key or is minimized (input_release): the keyboard buttons and the WASD stick, the touch only while the mouse holds it (g_mouse_down), and the pad of each player a controller holds (in DM_PAD_MERGE mode player 1's, when a controller is connected); while out of focus the keyboard, the mouse and the controllers' value changes are ignored (g_input_on), and when the window is key again each held controller is read as it is now (input_resume), so a button still held or a stick still deflected counts again. From PR #128 (nocmt), with two changes: the PR released player 1's pad and the touch in every case, which are also DM_KEYS' and DM_CTL's paths (hle_hid.c), so a scripted hold or drag ended early at a focus change during a Mac run; and it took input up again only if NSApp.isActive held when the window became key, so if AppKit makes the window key before the app counts as active, the keyboard, the mouse and the controllers stayed off until the next focus change. Also left out: the handler queue set to the main queue (already GameController's default), the focus state 3 on resigning key while minimized (DidMiniaturize sets it), the release on deminiaturizing (nothing is held then), and the t_hid test, which checked only that dm_input_set / dm_pad_set_player with zeros clear the state, not the focus handling. Checked: the host unit tests pass (GAME=010015100B514000), the Mac host builds for Mario vs. Donkey Kong, and a DM_WINDOW run boots it with DM_CTL presses reaching the game. Not checked: an actual focus change (in that run the window never became key or resigned it: the shell's activation requests and Finder's activation left no focus line), a physical controller, the software keyboard's text box by hand
seeddsandnocmt2026-10-25 10:41:17 +08:00
7afd90b0b2
Tests: allow CI scheduling delays in audio stall recovery
seedds2026-10-25 10:31:57 +08:00
0f99c353d3
Haptics: a Core Haptics engine that fails to start is replaced, not retried for good (general, from PR #143). A Bluetooth Switch Pro controller that vibrates through iOS Settings' Identify stayed silent in a game, which logged Core Haptics -4810 (server init failed): the adapter made each engine once per route and called startAndReturnError on that same broken engine at every later effect, and its stop/reset handlers were installed only after the start. Now haptics_apple.m makes an engine when the first effect needs it (the phone's too, so after the audio session is active), installs the handlers before starting it, and a failure to make or start the engine, make or start its player or send its parameters releases that engine: a fresh game signal makes a new one after a backoff of 1, 2, 4 or 8 s, effects from before the failure are not replayed, and the old engine's callbacks are stale (generation). A controller whose left and right handle engines cannot be made or started falls back to its default locality until it reconnects; a player or parameter failure (a service reset, say) is retried on the same handle. Idle release (1 s of silence), focus loss and engine resets stop an engine but keep it, so the next effect restarts the same engine, as before. Errors name the target, the stage, the error domain and code and the underlying error, and the log names the controller, its localities and the output channels. From PR #143 (nanda86), with two changes: the PR dropped the engine at every stop, so each rumble after a second of silence made a new engine (a new connection to the haptics service on a controller, a new initWithAudioSession: engine on the phone), and any handle failure moved the route to the default locality for good. t_haptics_apple covers each failure stage, the backoff, the fallback (and no fallback on a player failure), stale callbacks, a restarted engine after idle release, a disconnect and a controller without haptics, with fake controllers and engines. Checked: the host unit tests pass (GAME=0100DCA0064A6000), haptics_apple.m compiles for iOS arm64 with -Wall -Werror. Not checked: a physical controller or the iPhone's own haptics
seedds2026-10-25 09:55:44 +08:00
ca658c9dde
NVN BufferInitialize: a buffer past the end of its pool is a one-line note, not a red line (general, found on Breath of the Wild). Breath of the Wild creates one 256 KB buffer (offset 0) over a 144 KB pool when the pause menu opens and near a save commit (Mac and iPhone: the same line, once each time). The NVN driver does not check a buffer's range against its pool, and the host never reads a buffer's size to address memory: every GPU access takes an address and a length from its command and resolves it in the pool's memory (mtl_addr / pool_find), which reports a range that leaves a pool on its own (RED_ONCE). The buffer's only use is BufferGetAddress and one texture-to-buffer copy (nvnCommandBufferCopyTextureToBuffer, called from main+0xec2b80 on the unpatched module) of the screen picture's own extent, which resolved inside the pool (no "destination is in no memory pool" line), then it is finalized in the same function; it is never mapped. Now a non-red [nvn] line, once per run. Checked on the Mac (BotW, a copy of the save): the pause menu opens with the inventory drawn as before and the line is a note; the host tests and the gfx run list pass. From PR #145 (v0lumehi). Also checked on current main: the host unit tests and the gfx run list pass. Side finding of the PR, not changed here: in the world, Breath of the Wild's uniform buffer binds of 0x750 bytes at +0xa00 of one-page pools straddle the pool end (pool_find returns -1 for them)
seedds2026-10-25 09:36:51 +08:00
188c1d8f99
DM_HITCH_DUMP for every graphics API, and safe against exiting threads and the dyld lock (follow-up to #134's review). The present time dm_present_ns was written only in the NVN present (hle_nvn.c), so the dump never fired for the Vulkan and GL games; it is now written in present_enqueue (nvn_mtl.m), which mtl_present (NVN), mtl_present_ext (Vulkan: hle_vk.m; GL: hle_gl.m) all reach, and the NVN-only write is gone. hitch_main took thr_t pointers under G, released G, and read the threads afterwards, so a thread that exits and is freed during a dump was a use-after-free; and it called dladdr while the target thread was suspended, which deadlocks if that thread holds the dyld lock. Now hitch_grab copies what a line needs (handle, name, wait state, svc, registers, the raw pc / lr / frame-pointer chain) into a hitch_t while holding G (thread exit and thr_free need G, so no listed thread is freed or leaves), and between suspend and resume only mach calls and memory reads run; hitch_log then looks the symbols up (module table, dladdr) and logs after every thread runs again. The tests' weak dm_present_ns definitions (t_host_stubs.c, t_nvn.c) stay as they are. Checked: the host unit tests and the gfx run list pass (GAME=01007EF00011E000); the Mac host builds for Breath of the Wild (NVN) and Hades (Vulkan). From PR #137 (v0lumehi). Also checked on current main: the host unit tests and the gfx run list pass; on the Mac with DM_HITCH_DUMP=20:2, Mario vs. Donkey Kong (NVN) logged two dumps in its opening movie and Balatro (GL, which never dumped before: no present time was written outside NVN) one, every guest thread with its guest frames and host symbols, and both games ran on to the end of the run. Not checked: a Vulkan game (none on the drives; Vulkan presents reach present_enqueue through mtl_present_ext as GL's do)
seedds2026-10-25 09:36:51 +08:00
9c1e449470
Bayonetta 2 (01007960049A0000, update 1.2 = v131072, PlatinumGames' engine in its Wii U form on SDK 3.6.0, the oldest SDK so far): game profile, a CRI CPK reader for the shader split, and two general NVN fixes. Plays on the iPhone (IPA 12.5 GB, user). On the Mac from a fresh save: the boot, the PlatinumGames logo movie (USM), the title, a new game (slot 1, the auto-save notice, Story, Normal), the opening narration (which cannot be skipped on a first play), the prologue cutscene (skipped: Plus, Down, A, A), the tutorial (closed: Plus, Down x4, A), Verse 1 on the jet, Rodin's cutscene and the next fight with an Umbran Climax and a torture attack, with Wwise sound through two 6-channel AudioOuts: 60 fps in play (frame time p50 16.67 ms, p95 about 17.5 ms; GPU 2-3.5 ms a frame on an M6), 30 in its cutscenes as the game asks, about 1.6 GB resident and 1.75 GB of Metal allocations. No host change was needed for SDK 3.6.0 itself (NVN API 53); no virtual address memory (a 3.11 GB heap in the 4 GB layout, mem_used 102 MB). Files: six CPK archives (9.5 GB, 6,091 files, 12 GB of them CRILAYLA-compressed) of PlatinumGames DAT archives. New dm/gfx/tools/cpk.py: the @UTF tables (and the masked ones of Unicorn Overlord's CPKs, the same tables as before once unmasked), the TOC and CRILAYLA (a decoder over a bit string, about 29 MB/s); shader_split.py cpk_jobs splits the CPKs into jobs and scans each entry (a CPK it cannot read is scanned as one file, with a [RED] line). Shaders: 1,856 BNSH files of one stage each (.vso / .pso) in shader.dat, shader2.dat and shadereff.dat of data000.cpk, all built for the Mac and iOS. nvn_mtl.m, both general: a cleared attachment larger than the render pass (Metal's render area is the smallest attachment) is now cleared in a pass of its own and loaded by the pass, as NVN clears the whole target (rp_begin_ex_): the game's 32x32 and 256x256 passes cleared only that corner of its full-size depth target, so the opening narration drew in a corner of a black screen and the opening scenes were speckled; and Downsample of a single-sample texture is a filtered scaling copy of the whole level that may convert the format (copy_t2t, its bloom chain 320x176 -> 80x48 and its 1280x720 BGRA8 frame into an RGBA8 texture), which logged red lines before. build_ipa_from_nsp.py: a survey note for CPK archives. Profile: the 3D pad layout (the camera is on the right stick; the game's touch controls are handheld only), a prewarm list recorded on the Mac over the route above (974 lines). Checked: the graphics run list, the host unit tests and the 49 tool tests pass; Unicorn Overlord's CPKs split as before. Not checked: 13 Sentinels and Catherine: Full Body's CPKs (no files on the drives), the clear fix in other games beyond the unit tests. README (games gallery, a Mac picture of Verse 1), AGENTS.md (58 -> 59), docs/development.md and docs/agents 01 / 03 / 04 say so
seedds2026-10-25 08:49:26 +08:00
6a0687721a
NSZ and XCZ files in the NSP folder (from PR #140): the NSP script converts them itself instead of a second script. scan_nsps lists *.nsz and *.xcz beside *.nsp and *.xci (an NSZ is a PFS0 with the NSP's ticket, so title_of reads it as it is; a compressed file with an NSP of the same name beside it is skipped), and decompress_nsz converts each once with nsz -D, run from the Python running the script (Spyder does not have its bin folder on PATH), into <title ID>_build/nsp/, before the build ID lookup or the survey reads them. nsz writes into an empty temporary folder and the file is moved into nsp/ only when nsz exits 0 and made it, so a partial file left by a failure or an interruption is never taken for a whole one (with -w left out, nsz itself skips an output whose title and version are already there and exits 0); a conversion is reused while its source has the same name, size and modification time. nsz -D prints an NCA hash mismatch but goes on and exits 0 (Decompressor.decompress passes raiseVerificationException=False), so a damaged file is left to nx_extract's own hash check (--allow-damaged). Without the nsz package the script stops with one line: pip install nsz. README: one line in the packing steps; docs/development.md says how it works. AGENTS.md: only build_ipa_from_nsp.py has to run in Spyder with F5; the other scripts in scripts/ run from a terminal (the user). New test dm/tools/tests/test_nsp_nsz.py (nsz stood in for). Checked: the new test, test_nsp_alias, test_nsp_chain, test_nsp_mac, test_nsp_progress and test_xcode_workflow pass; ANIMAL WELL's base game and update compressed with nsz 5.0.0 -C, then the script on a folder holding only the two NSZ files: both converted (byte for byte the original NSPs), the IPA built (exit 0), and a second run reused both conversions. Not checked: an XCZ, the IPA on the iPhone
seedds2026-10-25 02:34:18 +08:00
