Commit Graph
100 Commits
Author SHA1 Message Date
seeddsandparsifal295 40b643bbe6 Keep Mac audio buffers near 5 ms across output changes
Apply PR #262's device-buffer fix for clicks caused by audio underruns.
Reconfigure on both output-device and hardware sample-rate changes, respect
the device's frame range, and read back the actual buffer size. Avoid
rewriting unchanged sizes when buffer changes notify the format listener.
Remove listeners before teardown, including failed initialization, so they
cannot configure a device that the AudioUnit has already released.

Add hardware-independent regression coverage for switching, rate changes,
scope filtering, limits, synchronous notifications, diagnostic overrides,
absent devices, and listener cleanup. Record the original gameplay evidence
and the additional silent Core Audio validation in the audio reference.

Validation: full host tests, Mac host build, docs checks, and the iOS source
compile check (178 files) passed. Silent USB/speaker switching and speaker
rates of 44.1/96/48 kHz verified 221/480/240 frames. HDMI switching and
gameplay listening after these listener changes remain unvalidated.

Co-authored-by: parsifal295 <130349765+parsifal295@users.noreply.github.com>
2026-10-27 00:41:13 +08:00
seeddsandsmartLanny 06b7b3e563 Share finite SyncWait budget across event and feedback waits (#274)
Use only the remaining finite timeout after the fast-event wait instead of
starting a second full wait for feedback. Preserve polling, CPU completion
ordering and the existing infinite-wait stage caps.

Add deterministic regressions against the production wait function across
all three fast-wake sites and the four fast-wake/wait-ledger combinations.

Co-authored-by: smartLanny <51014844+smartLanny@users.noreply.github.com>
2026-10-27 00:29:26 +08:00
seedds 53843cc7d5 Port Donkey Kong Country Returns HD 1.1.0 2026-10-27 00:13:42 +08:00
seeddsandSuraj Mandal 09f3778c36 Make repeated IPA and Mac app builds incremental (#279)
Reuse unchanged IPA entries through the packing index and APFS clones,
and keep Mac app and derived-profile shader outputs while their inputs
match. Preserve main's shader regression checks and .NET discovery.

Count uncompressed bytes in streamed files so a size change aborts the
pack without replacing the previous IPA or index. Cover both deflated
and stored streams with growth and shrinkage regression tests, and
record the cause and fix in the packing reference.

Co-authored-by: Suraj Mandal <surajmandalcell@gmail.com>
2026-10-27 00:10:35 +08:00
seeddsandJose Rosendo 89744da753 Discover Homebrew .NET runtimes for standalone build tools
Derive DOTNET_ROOT from dotnet's Microsoft.NETCore.App runtime listing and
export it to both explicit and inherited build environments. Preserve caller
overrides, use the build environment for discovery, and leave discovery
unchanged on probe failures instead of treating Homebrew's wrapper directory
as a runtime root.

Add regression coverage and document the packing failure and its resolution.
Based on PR #284.

Co-authored-by: Jose Rosendo <josemrb99@gmail.com>
2026-10-26 23:54:32 +08:00
seedds b117c91317 Check translation chain changes on games their author does not own (#272)
One translation chain serves every game, so a translator or host change was checked
only on its author's games; on another game the first sign was a crash at run time
(c3a0880 broke the image atomics of SHIN MEGAMI TENSEI V, Yoshi's Crafted World and
FINAL FANTASY RESONANCE).

- CI: the tool-tests job installs .NET 10, spirv-cross and spirv-tools and points
  DMSH_TEST_XLATE at the committed translator, so the translator's synthetic-shader
  tests (test_image_atomic_bounds, test_dmsh_geometry) run on every change.
  test_dmsh_geometry imports its sibling module when run from the repository root.
- dm/gfx/tools/shader_regressions.py: lists the shaders that translated in the
  game's previous b-*/ and fail in the current chain's, from dmsh_build.py's build
  reports (not part of the chain). The NSP script runs it after its shader step and
  stops on a regression unless ALLOW_SHADER_REGRESSIONS / --allow-shader-regressions;
  docs/development.md describes it.
- FINAL FANTASY RESONANCE: the c3a0880 regression is resolved since 2e655b7 (image
  atomics with a used result): chain 39e4d9585b1d builds 73,488 of 73,496 shaders
  (the 8 3D image atomics as before), shader_regressions.py finds 0 against the
  earlier build, and a new game from an empty save plays the opening cutscene on
  the Mac with no red line; the two shaders that failed and aborted run their
  atomics on their buffer textures.

Checked: every CI test file as the job runs it (DMSH_TEST_XLATE set), the docs tests.
2026-10-26 23:43:09 +08:00
seeddsandb1tdr3amer24 99d0aae38f Keep bindless reads through mixed handle phis; read misaligned buffer textures through an aligned copy (#168)
Two general causes of FINAL FANTASY RESONANCE's missing fires, smoke and muzzle
flashes, found and first fixed by b1tdr3amer24 (GitHub issue #168).

- Translator (vendored BindlessElimination.cs, TryTableHandle): a texture read
  whose handle phi mixes c2 handle words with values that are not handles
  (undefined, or float values the compiler kept in the register on paths where
  the predicated read does not run) was removed and its result set to 0:
  58,122 reads in FINAL FANTASY RESONANCE, mostly Niagara particle vertex
  factories, whose meshes collapsed to a point. The read now takes the c2
  handle among the inputs, or reads each candidate and picks by the phi's run-
  time value; candidates read elsewhere as another texture shape are left out
  (a second sweep, ResourceManager.GetTextureTypes). Only phis that
  GenerateBindlessAccess refuses come here, so tex_dyn_slots arrays keep their
  path: the other 69,449 shaders' SPIR-V is unchanged. Several candidates are
  never used for an image store or atomic. dmsh_xlate's Log now reaches the
  notes as "translator: ..." and the build summary counts those apart.
- Host (nvn_mtl.m, tbuf_shadow): a texture buffer view at an offset that is
  not 16-byte aligned reads from the boundary below on an M4 Pro (t_tbuf_align:
  56 of 56 views wrong; the M6 measurement in the old comment does not hold).
  Such views are read through a per-submission aligned copy; aligned views are
  untouched. A workaround for the Metal limit, its gaps (GPU writes in the same
  submission, CPU_NO_ACCESS pools, image writes) recorded as open.

Translator rebuilt; lock and chain header updated. Checked: all FFR shaders
translated before and after; FFR on the Mac (fires and smoke column render,
steady; with the copy off they flicker); gfx, chain and docs tests.

Co-authored-by: b1tdr3amer24 <240231594+b1tdr3amer24@users.noreply.github.com>
2026-10-26 23:43:09 +08:00
seeddsandNishikori-Yui a0841d6b3c Isolate save failures by backing transaction
Keep save failure state and write timing with the canonical backing save, and
retain that identity on open files and mount aliases. Coordinate commits with
in-flight writes and file-size changes, preserve taint when recovery fails,
and mark the affected saves after a failed rename rollback.

Initialize shared write-injection settings with pthread_once so simultaneous
first writes do not race. Expand regressions for independent saves, aliases,
overlapping I/O and commits, failed recovery, and concurrent first writes.

Validation: 98 save regression checks, filesystem ThreadSanitizer, the full
host unit suite, and documentation checks pass.

Refs: https://github.com/seedds/nxnative/pull/288

Co-authored-by: Nishikori-Yui <165007678+Nishikori-Yui@users.noreply.github.com>
2026-10-26 23:25:17 +08:00
seedds a2e1c1d8a6 Refresh host test fixtures for heap and audio event changes
Update the heap-probe regression for main's corrected SetHeapSize result
(0xD001). Keep invalid mapping and heap-shrink expectations at 0xD401.

Supply the audio fixture's SDK DestroySystemEvent symbol and verify the
local event is destroyed before kernel binding, once per event open,
including the nested open route.
2026-10-26 22:44:57 +08:00
seeddsandhuajiaoshu520 0963959f2f Fix startup deadlock in blocking notification waits
Publish native HLE condition waits to the guest scheduler and end inherited
same-core handoffs at the blocking boundary. Restore runnable state on wake.

Exercise real thread creation, startup and exit in the notification regression,
including spurious wakes and both scheduling modes. Record the real-game Mac
startup comparison and the remaining visual gameplay and iPhone validation.

Co-authored-by: huajiaoshu520 <huajiaoshu@jasonml.eu.org>
2026-10-26 22:41:52 +08:00
seedds 4db4c69f0b Port Donkey Kong Country: Tropical Freeze 1.0.2
Profile 0100C1F0051B6000 (SDK 4.4.0, Retro Studios' engine, NVN): plays on the Mac from boot
through the menus and the intro cinematic into the first level at 60 fps. General changes:

- svc SetHeapSize: a request beyond the heap region returns ResultOutOfMemory, as Horizon
  (the game binary-searches its heap size between 3 and 6 GB)
- shader_split.py: SNVN program records in Retro's RFRM forms (MaterialArchive.arc, the packs'
  MTRL resources) and the LZSS-compressed startup material archive in main
- nvn_mtl.m: a texture in a CPU_UNCACHED pool starts with its storage at first use (title logo,
  menu panels); transform feedback captures draws that rasterize too
- dmsh_build.py: transform feedback buffers holding several outputs (DK's fur simulation);
  chain header regenerated
- hle_audren.c: the renderer event is the kernel event the audio service gives, not the SDK's
  local one (the main thread was starved at the first level's load)
- hle_oe.c: oe::GetUserInactivityDetectionTimeExtensionMode
2026-10-26 22:38:18 +08:00
seedds 1402ded2d9 BotW 1.9.0: held-button counts step by the VFR ratio (menu repeat in game time at 60 fps); archive build folders never taken at a recapture; unsupported sample counts not made
- sead::ControllerBase::updateDerivativeParams_ added a constant 1.0 per frame to held counts, so menus
  repeated twice as fast at 60 (0.3 s went 5 entries, 2 at 30). A FRAME_STEP stub at main+0xeb0f88
  takes the VFR's ratio (+0x3c); checked in game: 0.2 / 0.3 / 0.4 s move 1 / 2 / 3 entries at 60
  (MacNativeGames 866beb9).
- ga_capture: a recapture through lookup archives makes no folder at its compile, so it no longer
  lists the folder (it could take another program's folder made meanwhile).
- tex_desc: a multisample texture of a sample count the GPU lacks is not made instead of being made
  with fewer samples (shaders could read sample indices past it).
2026-10-26 21:49:19 +08:00
seedds 933b67590c Port OCTOPATH TRAVELER 1.0.5 (Unreal Engine 4.18, SDK 3.5.1)
Profile 010057D006492000 (no virtual address memory: the 4 GB heap layout; tex_flush_upload and tex_dirty_upload for the textures Unreal writes from the CPU into uncached pools after creating them) and its note. Brought up on the Mac through the boot, the language notice, the title, the traveler selection and Ophilia's and H'aanit's openings (free play in the cathedral and in S'warkii) at 30 fps; battles, sound and the iPhone not yet tested.

General changes found on it:
- patch_nso.py: a jump table whose index is bounds-checked with a 64-bit cmp (main+0x1676f58), accepted only in the last pass so earlier rewrites are byte-identical.
- profile_guess.py: Unreal's 64 GB alias layout only with virtual address memory (this game sets a 3.08 GB heap instead).
- NVN: nvnTextureBuilderGetSparseTileLayout / nvnTextureGetSparseTileLayout (Unreal sizes a sparse texture's mip tail with it; unimplemented, it asked for 1.75 GB twice); sparse 2D textures without a packaged layout take the block height of their 64 KB tile (the world map's tiles were shuffled); every virtual pool gets an address range of its own until its first mapping decides between the alias (a whole-pool mapping, Mario's pool) and the sparse page table (Unreal's one-texture pools, mapped piece by piece: streamed textures were black and later mappings refused).
- Audio renderer: SdkMutexType is looked up only when the SDK uses it (SDK 3.5.1 takes LockMutex at +0x1f8).

README: the gallery entry (Mac screenshot of S'warkii, IPA ~3.6 GB, all 5,434 shaders built for iOS); AGENTS.md: seventy-one games.

Checked on the rebased tree: the tool tests (15 skipped), the docs tests, a Mac boot into the traveler selection; before the rebase the graphics tests (make run, exit 0).
2026-10-26 21:25:17 +08:00
seedds ce9b30b010 Port Sea of Stars 2.0.59001 at 60 fps
Unity 2021.3.25f1 on SDK 16.2.0 (the sdk build of Super Mario Bros. Wonder 1.0.0).
The sdk build takes the node-area cut and SDK 20's 2 MB region alignment: Unity's
N MB + 252 KB regions each took the next size class's slot, which exhausted the
4.375 GB alias region after about six minutes. Profile with a 1.5 GB pool, note,
games table row, README gallery entry and reference entry.
2026-10-26 19:40:09 +08:00
seedds 2e655b79bf Port SHIN MEGAMI TENSEI V: Vengeance 1.0.3 at 60 fps
Profile 010069C01AB82000 (Atlus, Unreal Engine 4.27, SDK 15.3.2, NVN). Opening gameplay
tested on the Mac: new game, classroom, saves and reload in a fresh process, Tokyo map,
Shinagawa Station and its event. iOS and combat untested.

- Memory: the FMallocBinned3 constructor cut (main+0x1f2f63c, gate +0x1f2f644) with the
  measured UE4 vtable in hle_uemalloc.c; -SwitchVirtualAddressSpaceSize=2048 and a 3.75 GB
  block. profile_guess.py now leaves 128 MB for the initial heap in provisional Unreal
  layouts (test_profile_guess.py).
- Shaders (general, chain 86c8282f6fb4): returning IGN image atomics keep the old value with
  bounded access (Ryujinx translator, t_atomic_return.m); buffer-image reflection keeps its
  Metal texture type (nvn_refl.m, t_refl.m). All 19,746 shaders build for Mac and iOS.
- 60 fps: hle_fpslock.c writes the game's own Project.FrameRateLimit (main+0x6d6d5d0, which
  GameUserSettings applies to t.MaxFPS) and rhi.SyncInterval, gated on main 0a4f14a5...
  Mac (M4 Pro): 59.2-60.1 fps at the station, frame p95 17.5-20.3 ms; a 1 s walk ends at
  the same place at 30 and 60 fps. DM_FPS_TARGET=30 keeps 30.

Docs: the game note, games table, README gallery, memory.md, shaders.md, symptom index;
AGENTS count 68 -> 69. Checked: tool tests (186, 6 skips), docs tests.
2026-10-26 19:35:03 +08:00
seedds e72991d8ca NVN on Metal: remove Metal's archive build folders, cap texture fields Metal refuses, 8-byte hash keys as one word (general, from MacNativeGames)
- Pipeline archive: the Metal 4 serializer leaves a com.apple.gpuarchiver/gpuarchiver-* folder per flush
  (456 folders, 2.5 GB on this Mac). The first capture after a flush takes the one folder it made, the
  flush removes it (and the one a recapture-only flush makes), exit removes an unflushed one; iOS clears
  earlier runs' folders at start (MacNativeGames 1eff989, 110e6c6).
- tex_desc: levels past what the size holds, arrays of no or more than 2048 layers, sample counts the GPU
  lacks no longer abort the process; copies and ReadTexels at a missing level are skipped (f864eb0).
- nvn_hm_get compares 8-byte keys as one word: 5.9 -> 1.5 ns a lookup (7431eb7).
- frame-rate.md: segment size cannot change BotW's 60 fps frame on the Mac (fence wait 0.02 ms).
2026-10-26 19:32:49 +08:00
seedds eb858ec51a Shader split: check a size-prefixed frame's own size first, bound TextAsset reads (#203, #238)
#203 as submitted stopped the shader split of Tears of the Kingdom with a
TypeError: its 1,202 Cave/.../C.crbin.<hash>/*.quad files are a 4-byte hash
(the one in their folder name), then a zstd frame, so they pass the gate,
and on the length mismatch the error path took len() of None. The PR checked
Wonder and Yoshi only; of the RomFS of 39 games on hand, only those files
have a frame at +4.

- unsized: a frame whose header records a size other than the prefix is not
  such a container and is read as it is, without decompressing it
  (frame_size). A frame that records none (Kirby's LZ4 frames) must
  decompress to exactly the prefix's size: the whole output is compared,
  where the PR's max_length cut a longer frame at the prefix and let it pass.
- The dictionary handling is shared with unzs (zs_dict_id, unzs_dict). A
  dictionary ZDICTS does not have is still reported, not fatal, as in the PR:
  the ID may be any bytes after a magic at +4.
- unityfs.text_assets: an object shorter than the two length fields, or one
  whose name padding leaves no room for the script's length, is left out
  instead of reading past its end (struct.error, which stopped the split).

Checked:
- Kirby and the Forgotten Land (base and update 1.1.0): 121 shaders before,
  27,288 after (27,302 with the update), no [RED] line. 5,121 zstd frames
  record their size, equal to the prefix; 2,080 + 354 LZ4 frames record none
  and decompress to the prefix's size. The PR's "383 in lyt/" is 378.
- Tears of the Kingdom: the full split exits 1 with #203 alone; with this
  commit its package and tables are byte-identical to main's, no [RED] line.
- main's split and this branch's give byte-identical packages and tables on
  Monster Train 2, Ace Attorney Investigations Collection, AI: THE SOMNIUM
  FILES, nirvanA Initiative, BALL x PIT, Advance Wars 1+2, Brain Memory
  Training Academy (Unity 2017.4 to 6000) and Super Mario Bros. Wonder.
- New test_size_prefixed_frames.py; test_unity_text_glslc.py covers short
  objects (fails on #238's unityfs.py). All of dm/tools/tests pass.
- Not checked here: FINAL FANTASY CRYSTAL CHRONICLES (#238's game) is not on
  hand; Kirby has not been run.
2026-10-26 19:04:04 +08:00
seeddsandClaude Code 7c5b0b2a78 Shader split: open HAL's .cmp containers
HAL keeps its assets in .cmp containers: a 4-byte little-endian uncompressed
size, then a frame (zstd, or LZ4 in the ubin/ map data). The split
decompressed a zstd file by its magic at offset 0, so a frame that starts at
offset 4 was read raw, and the shaders inside the BFRES archives of those
containers were never seen.

On Kirby and the Forgotten Land this is the difference between 121 shaders
found and 27,288, and between a game that plays and one that shows a black
screen: the game asked for 900 shaders the map did not have, and the log
filled with "[RED] shader ... is not in the shader map".

The split now opens a size-prefixed frame (unzs_bytes for zstd, unlz4_bytes
for LZ4). The size prefix is checked against the decompressed length, so a
file that is not a container is left alone, and a container that does not
open is reported instead of being passed over in silence.

- Found on: Kirby and the Forgotten Land (01004D300C5AE000, in bring-up):
  7,201 .cmp files, 5,121 zstd frames with 19,364 BNSH inside (11.2 GB
  decompressed) and 2,080 LZ4 frames (0.3 GB, no BNSH). The split now finds
  27,288 shaders: 18,476 in gfx/, 8,313 in ptcl/, 383 in lyt/, 67 in the
  loose shader files, 54 in main.
- No other game on hand has such a file: Super Mario Bros. Wonder (v0 and
  v65536) and Yoshi's Crafted World (v1.0.1) have none.
- The chain version is unchanged: shader_split.py is not one of the chain's
  tools (dm/gfx/nvn/dmsh_chain.h needs no rewrite).
- The docs entry is in docs/reference/shaders.md; the symptom index is
  regenerated. All 15 test files in dm/tools/tests pass.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-26 19:04:04 +08:00
4639c5114c Pack shader pairs and the offline pipeline archive into Mac apps (#241, #260)
The Mac app packer left out shader_pairs.tsv and pso_pairs.tsv, which the
host's pipeline prebuild reads beside the shader map and build_ipa.sh already
copies into the IPA. Every NVN game's Mac app therefore built no pulling
bases, had no hybrid stand-ins, and compiled first-seen pipelines on the
submit thread. Copy both tables, and the shader split's pso_pairs.tsv for a
profile without one.

An NVN game's Mac app also carries its pipeline archive made offline for this
Mac's GPU (pso_offline.py --platform macos), handed to the host as
DM_PSO_BUNDLE as on iOS; --gpu none leaves it out. Integrated with three
fixes: the archive is validated and counted in the free-space check before
anything is copied; a plan whose pairs all need function constants packs the
app without an archive instead of aborting; and a PSO_GPUS value that does
not name this Mac's GPU is reported.

Validation: test_nsp_mac_app 22 (pair tables, split copy, bad archive, free
space, GPU note, no-archive plan; the table and space tests fail without the
fixes), the tool suite, iOS packaging tests and test_docs. A Breath of the
Wild Mac app on an M4 Pro (applegpu_g16s, 629.4 MB archive): 16 of 16 probed
bases found, no pair base built, no red line; the run stayed in the title.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>
2026-10-26 16:46:44 +08:00
seedds ecbeae164c Port Pokemon Brilliant Diamond 1.3.0 at 60 fps
Add the registered profile, Unity 2019 compute shaders, VP8 decoding, depth initialization and AudioOut stop fixes. Include build-gated frame limiting, regression coverage and Mac validation notes.
2026-10-26 15:40:36 +08:00
seedds ceb5046b3f Apply Pokémon Legends: Z-A's allocator cut to other builds by function match (#266)
The base game 1.0.0 (main 7222e13e…) aborted in nninitStartup: the 12 GB → 4 GB
StandardAllocator cut was registered for 2.0.2's main only. The same function
(an exact fingerprint match, main+0x19f8480 in v0) now gets it through MATCH_CUTS.
2026-10-26 15:34:21 +08:00
58f74014f4 Seed TotK prewarm from the recorded M2 iPad route (#261)
Add 1,537 shader, 2,512 render-pipeline and 43 compute-pipeline rows from
the update 1.4.3 capture, preserving first-use order. Document the
author-reported route and results, and limit the coverage claim to
that route.

This profile-only capture is separate from the shared CPU and loader
optimizations. Validation checks the row counts, uniqueness and exact
data/order preservation; no new cold-start or gameplay run here.

Co-authored-by: draco-cong <71372970+draco-cong@users.noreply.github.com>
Co-authored-by: laobeibei <laobeibei@users.noreply.github.com>
2026-10-26 15:28:59 +08:00
5632203274 Optimize CPU deswizzle copies and host symbol lookup (#261)
Hoist block-linear row and slice addressing out of segment copies, and
index HLE, forced-takeover and missing-import names while retaining
existing duplicate precedence and stub identity.

Add byte-address differential tests and direct collision, duplicate,
growth and reuse checks. Correct the ASTC-tail and loader-serialization
comments and document independent CPU microbenchmarks. Keep TotK
prewarm data in a separate change.

Validation: 5,334 texture checks; symbol-index and NRO lifecycle tests;
AddressSanitizer/UBSan; documentation checks; Mac host build and changed
iOS source compilation. No new gameplay validation.

Co-authored-by: draco-cong <71372970+draco-cong@users.noreply.github.com>
Co-authored-by: laobeibei <laobeibei@users.noreply.github.com>
2026-10-26 15:28:52 +08:00
f7345d9a00 Same-core handoffs wait only for the threads their wake made runnable (#247, #259 section 6)
A wake's handoff waited until every runnable higher-priority thread pinned to the caller's core was blocked, not only the ones the wake made runnable. The host runs a lower-priority thread beside such threads at all other times, and a core's busy loaders, workers and pollers are seldom all blocked at once, so the wait had no bound. Tears of the Kingdom 1.4.3 (issue #247) stayed on an empty scene after the boot logos: on 82ab25b, 84 functions / 771 NVN calls a frame, no packaged texture uploaded, 250,008 ms of handoff waits in a 180 s run; a temporary log showed BootupBackgroundWorkThread (priority 58) waiting 200-290 ms at a time for ARes Loading 1, Presentation Thread and BaseProcExecutor. Donkey Kong Country: Tropical Freeze's priority-58 job thread waited the same way for pollers (issue #259, section 6).

A wake now records the woken thread in the waker's current handoff and in those the waker belongs to (note_runnable, at most four), so Metroid Dread's main thread still waits for its movie dispatcher and for the worker the dispatcher wakes. Blocking, a sleep, YieldToAnyThread (SleepThread -2, which on Horizon takes the thread off its core so that lower-priority threads run there), exit, suspension and host GPU work end a thread's hold; a handoff ends when its caller's thread SVC returns (hand_epoch). There is still no timeout, and core_preempt keeps checking every runnable peer.

Checked on the Mac, without shader libraries for chain 8393d85a4eae (draws skipped): TotK on the issue #70 route for 180 s, 1,221 ms of handoff waits and the same per-frame NVN call counts and texture first uploads as 9feac2f (before the unbounded handoff). Dread, fresh saves, DM_KEYS=4:A*20/0.7, 60 s: a control host with the handoff off crashed in 1 of 3 runs at sdk+0x347eb4 from main+0x225338; the limited handoff passed 12 of 12 runs, each through three decoders and two teardowns with about 12,400 handoffs. t_preempt adds three handoff-scope cases: a busy peer nobody woke (main's code never returns), a worker woken by the woken dispatcher (fails without the recorded chain), a woken thread yielding to any thread (never returns without that rule). Host tests, docs tests and an iOS syntax check of hle_thread.c pass. Gameplay with shaders, iOS and DKC itself are not checked here.

Co-authored-by: HougeLangley <1161594+HougeLangley@users.noreply.github.com>
Co-authored-by: aurlight <120410928+aurlight@users.noreply.github.com>
2026-10-26 15:00:32 +08:00
seeddsandchengjie 2019ff866b Recognize framed pointer tail calls safely (#256)
Preserve the existing terminal-tail recognizer and accept a contiguous
framed method-pointer thunk followed by its separate null-error call.
Reject stores that overwrite saved FP/LR, unproven store addresses and
stack/frame/link-register clobbers, including WSP. Stores are limited to
fixed caller stack-argument slots above the saved frame.

Keep the fallback narrower than #188's proposed unframed block recognition.
Add synthetic AArch64 regressions and document the rule beside the other
tail-call findings; no game-specific IDs, fixed addresses or profile changes.

Validation: 179 tool tests run, 174 passed and 5 skipped; 10 tail-call tests
pass; the legacy terminal recognizer is AST-identical to main; git diff
--check passes. Real game modules were not rewritten for this integration.

Co-authored-by: chengjie <chengjie177@gmail.com>
2026-10-26 14:46:10 +08:00
seeddsandRAHUL REX 4acde72a6e Recover the frame step after movie and thermal caps (#253)
Keep the proven frame cost and GPU reference for each cap, and probe the
shorter hold when a movie or thermal cap ends. Otherwise a GPU-bound game
can fall straight back to 30 fps without a usable recovery reference.

Save an uninterrupted sustained run when a cap begins, even if no earlier
hitch populated cref. Keep the earlier reference through short runs and
update it after another sustained run. Count cap-release probes and give
them all 30 grace frames: the completed frame at the old cap consumes none.

Add movie and thermal recovery regressions, including first movies,
repeated caps, full grace and failed-probe backoff. Document the mechanism
and link it from both Zelda game notes.

Validated: 64/64 frame-step checks, including ASan/UBSan; six checks fail
against the original PR. Production thermal tests and ThreadSanitizer,
docs tests and diff checks pass; the BotW Mac host builds. Device clock
recovery remains modelled, without a new gameplay observation.

Co-authored-by: RAHUL REX <75500427+RahulRex0@users.noreply.github.com>
2026-10-26 14:22:03 +08:00
seeddsandRAHUL REX 82ab25b40b Reuse a present allocator only after its own present completes (#254)
present_now cycles three Metal 4 command allocators by ticket, and before resetting one it waited for g_pq_done to reach the ticket that last used it. g_pq_done is not GPU completion: every display-link callback sets it to ticket - 1 before presenting (the previous present was committed, not run), and a present that was not submitted sets it to its own ticket. So a display-link present never waited there: with three presents in flight, or two after a dropped frame, the allocator was reset and its command buffer and UV buffer encoded again under a present the GPU had not run. Each allocator now keeps the newest ticket its own commit feedback reported (g_present_alloc_done), and its reuse waits for that. The display-link callback can now block in present_now until the GPU has run that allocator's previous present; it still moves g_pq_done before it waits, so the game's swapchain acquire is unchanged. General: every game presents through this path, display-link mode is the default.

t_present_alloc holds the Metal 4 queue behind a shared event and calls the display-link callback by hand, each present with a drawable of its own offscreen layer: a dropped frame (tickets 1, 3, then 4 on allocator 1 again) and one frame per tick (5, 6, 7, then 8 on allocator 2 again). Checked on an Apple M6 at 6062518: without the production change 4 of 13 checks fail in 10 of 10 runs, the earlier drawable reads back 0 in all 64 pixels, and every run logs [RED] display command buffer error with kIOGPUCommandBufferCallbackErrorPageFault; with it 13 of 13 pass in 20 of 20 runs under MTL_DEBUG_LAYER=1 with no red line. The whole dm/gfx/tests run list passes, dmhost builds for Mario, docs tests pass. Not checked: a game run that hits it, the iPhone, the not-submitted path. Docs: the frame-rate reference entry (with the page-fault symptom) and the symptom index.

Co-Authored-By: RAHUL REX <75500427+RahulRex0@users.noreply.github.com>
2026-10-26 14:06:20 +08:00
seeddsandv0lumehi 60625182c4 Harden GPU fence wake and pass scheduling (#158, #213)
Integrate fast shared-event wakeups, per-segment publication, array-slice
pass merging, and independent compute dispatch hoisting from the PRs.
Serialize CPU completion effects before publishing completed fences and
validate recycled post records. Preserve submission order for early waits.

Guard scheduling against attachment aliases, first-draw and bindless
reads, indirect ranges, stencil writes, and explicit barriers. Add direct
Metal regression coverage and a gated CPU completion race test with TSAN.

Record final BotW and Metroid Dread gameplay validation with optimizations
on and off. BotW retains fewer render passes, but the measured 60 Hz M4
Pro runs do not establish a timing speedup. Keep the original higher-load
and iOS performance claims explicitly unverified.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 13:33:02 +08:00
seedds f171088e2e Fire Emblem: Three Houses: record why its two pacing cuts stay per-build. Checked on the Mac (host at 7753b7b, headless boot to the title, 100 s per run, each cut undone in the rewritten main): without the frame-limiter cut the general yield back-off holds 59.7-59.8 fps (frame p50 16.73 ms against 16.68) at 0.9-1.2 W of CPU energy against 0.14-0.42 W (about 9,700 plain yields a frame before the back-off starts at 2 ms); without the render-thread timeout cut, 53,000-55,000 wakeups a second against 1,450 and 0.7-1.0 W. The 16 us timeout is the game's own: the render thread passes (1/fps) x 1000 = 16 to main+0xaa90e0, which multiplies it by 1000 into the TimeSpan (nanoseconds) of nn::os::TimedWaitEvent. Of the 206 modules on hand, the limiter (main+0x5be4b4) is the only loop whose only calls are YieldThread and GetSystemTick, so a rewriter pass for such loops would match this one site. Docs: the game note; docs tests pass 2026-10-26 13:29:13 +08:00
seedds dac7ba6338 Function-matched cuts: same-name rules are one patch's implementations; Taiko Risshiden V DX's frame-limiter cut reaches the base game and later builds. nso_match.resolve groups rules by name: exactly one implementation must match (the log names it), a red line only when none or more than one does. Before, each implementation of a changed function logged a red line on every build it did not fit. Taiko's MATCH_CUTS has two implementations of the frame end under one name: 1.1.1's (main+0x443e8c, 364 bytes, cut at +288) and the base game's (v0 main 47d7baf6..., main+0x43e140, 172 bytes: 1.1.1 added the frame-skip branch; the same 20 bytes at +104 = main+0x43e1a8). In v0 N is likewise a global (GOT 0xa1be28, data 0x9ec55c, 1) set to 2 only by CPrivateBattleWindow / CPrivateBattleTutorialWindow, and the start tick is read after the update; nso_fnmap.py finds no exact match for 1.1.1's function in v0, found by disassembly. Checked through patch_nso.py: v0 main takes the cut at 0x43e1a8 (implementation 2, no red line); a 1.1.1 main copy with another build ID takes implementation 1, output byte-identical to the registered cut; 1.1.1 keeps its exact CUTS. A scan of the 42 distinct mains on the drives found this limiter only in Taiko. The base game has not been run. Docs: rewriter.md (same-name implementations, procedure step 2, Found on), the game note. Checked: test_nso_match.py (new test fails on the old matcher), all 19 tool test files, docs tests 2026-10-26 13:24:01 +08:00
seedds 7753b7b685 Taiko Risshiden V DX: keep the frame limiter for N = 2, cut it only at N = 1, and record its cause. KTGL's frame end (main+0x443e8c) presents, acquires, then waits until N / 60 s since the frame's start tick in 2 ms SleepThread rounds; the tick is read after the scene's update (main+0x443d44), so every frame lasts N / 60 s plus the time to the next start, rounded up: never one vsync, on any hardware. That is the game's code, not the host (SleepThread and the tick are exact; a model of the loop gives 54 fps with the host's acquire and with the Switch's double-buffered one). N (GOT 0xa1c088, data 0x9ec764) is the game's vsyncs per frame: 2 in CPrivateBattleWindow / CPrivateBattleTutorialWindow (main+0xf02d0, main+0xec4cc), whose code counts 60 / N frames a second, and the game presents at interval 1 everywhere. The earlier cut (b.hs -> b at main+0x443fd4) skipped the wait for every N: with N forced to 2 it presented 60 frames a second against the game's 28.8. The new cut at main+0x443fac computes the threshold with udiv by 60 and sets it to 0 only when N = 1, where the host's present pacing holds one vsync. Checked on the Mac (host at d50ecc1, Mac libraries for chain 8393d85a4eae): title N = 1 54.6 -> 60.0 fps (frame p50 16.67 ms, wakeups 2,050 -> 300/s), N forced to 2 28.9 fps (no cut 28.8, old cut 60.0); town on the rewriter's own output 60.0 fps, p50 16.66 / p95 17.08 ms, no red line; patch_nso.py applies the cut with its byte checks. The CPrivateBattle scenes (small unit battles, PBBATTLE) were not reached: open in the game note. Docs: the game note and the frame-rate reference entry give the corrected cause; tool tests (17 files) and docs tests pass 2026-10-26 13:04:04 +08:00
seeddsandv0lumehi fb20bce70e Add GPU diagnostics and close periodic profiles per frame (#225)
Add periodic encoder profiles with attachment traffic and formats, alternating A/B measurement phases, bounded Metal trace capture, and gpuprof.py log summaries. Preserve main's offline pipeline-archive integration while adding the capture build setting.

Close periodic samples at frame boundaries through the last committed shared-event fence, so every:1 reports without mixing frames or reusing an unresolved heap. Add 28 CPU checks of production profiling code and include them in CI; document the mechanism and update the symptom index.

Validated with all 31 CI-style Python test files, the profiling regression, a Mac host build, and all 133 iOS source compiles. Profiling completion and heap ownership were checked with synthetic counter/event objects; no new real-GPU or gameplay validation.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 12:09:07 +08:00
seeddsandv0lumehi 3f22526e87 Add repeatable Mac benchmarks and harden report generation (#191)
Run profile-defined routes from independent copies of one save and report frame-time, presentation and GPU percentiles across repeated variants. Preserve the display-link behavior for targets at or below 60 fps.

Resolve relative paths before launching in the run folder, preserve the archive-disable sentinel, aggregate optional metrics from the runs that supplied them, and release the measurement lock when launch fails. Regression tests exercise an actual synthetic child host and leave the source save intact.

Validated with benchmark regressions, the CI-style Python test suite, and documentation checks. The three new benchmark regression cases reject the original PR code.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 12:09:07 +08:00
seeddsandAmost d50ecc1f84 Support masked-predecessor jump tables and external patches (#237)
Recognize offset-table dispatches whose predecessor masks the index while
the dispatch checks a masked copy of the same source. Reject calls and
resolved table paths that bypass masking, undecodable instruction gaps or
targets, and writes that invalidate the table base. Keep the existing
recognizers ahead of this fallback.

Load build-gated patches from DM_GAMES_DIR and include them in rewrite reuse
keys. Preserve repository matching rules for derived profiles, select the
first external profile for provisional matching rules, and deduplicate
patch paths while retaining conflicts between distinct build claims.

Validation on current main: Python tool tests (156, 14 skipped), iOS tests
(6), graphics tests (44, 2 skipped), documentation index and diff checks.
Regression cases fail on the original implementation and pass with these
guards; both supported base-loading forms remain accepted. Real game
modules and gameplay were not revalidated.

Co-authored-by: Amost <6445689+Amost-@users.noreply.github.com>
2026-10-26 11:03:47 +08:00
seeddsandv0lumehi 92fc2f2711 Build and ship offline Metal pipeline archives (#226)
Compile eligible shader-pair bases on the Mac with metal-tt, package one
archive per selected iPhone GPU, and use a matching bundle before falling
back to on-device preparation. Carry the profile depth mode through packing
and include the pulling libraries required by the bundled bases.

Integrate PR #226 with four confirmed fixes: validate archive container
bounds before invoking Metal; require at least one actual successful bundle
probe; hash consumed library bytes into offline cache keys; and emit device
remainder lists for constant-only plans instead of aborting packaging.
Keep game measurements in the game note and document the general findings.

Validation on current main: 52 real Metal bundle checks, 57 archive checks,
79 prewarm checks on each pipeline path, 73 Python regressions covering
offline tools, packaging, documentation and chain checks, and compilation
of all 133 iOS source files. No new iPhone gameplay validation. The archive guard validates
container structure, not private Metal payloads.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 10:52:50 +08:00
seeddsand7splay c6c4fac5c3 Exclude Mac shader intermediates and respect ARG_MAX (#240)
Pack only runtime shader libraries, reflection JSON and TSV tables, excluding translated Metal sources and _b_build.json from the app and its free-space estimate.

The original 2,000-file batching still exceeded the macOS exec limit with valid long paths. Budget encoded arguments, inherited environment and pointer overhead instead, with regression coverage for long and multibyte paths, clone flags and an exhausted argument budget.

Validation: reproduced E2BIG before the fix; all 17 Mac app tests and 4 documentation tests pass on current main. Tests use synthetic files and a stand-in host; a full large-game packaging run remains untested.

Co-authored-by: 7splay <107197512+7splay@users.noreply.github.com>
2026-10-26 10:40:10 +08:00
seeddsandFernando Messias 62a640b037 Fix custom key loading for compressed game inputs (#204)
nsz 4.6.1 rejects --keys and loads keys.txt beside argv[0] during import.
Run the converter in its temporary output folder with the selected key
file linked there before import, using absolute input and output paths.
Remove the temporary link on both successful and failed conversions.

Integrate PR #204 with XCZ and relative-path regressions, rename the test
flag that shadowed unittest.TestCase.fail, and qualify the documentation
for the verified nsz release. Record the packing finding and provenance.

Validation: reproduced the exact parser failure using real nsz 4.6.1;
a synthetic NSZ conversion verifies selected-key lookup and link cleanup;
52 NSP tests, four documentation tests and git diff --check pass.
The synthetic payload is unencrypted and uses dummy keys. Real-game
decryption, actual XCZ conversion and a full IPA build remain unverified.

Co-authored-by: Fernando Messias <fernandomessias95@gmail.com>
2026-10-26 10:36:23 +08:00
seedds 435a231051 Preserve Horizon audio renderer work-buffer requirements
DSP callers were charged the local SDK CPU server budget, whose command representation is larger than the Horizon service representation. For Age of Calamity the service requires 0x61d000 while the local SDK requires 0x760000, exhausting the fixed sound heap.

Calculate the service requirement through renderer revision 15 and supply larger local backend storage separately, with cleanup on failed open and after renderer close. Remove the two game heap cuts and add sizing, nested-open, failure and lifetime regression coverage.

Validation: 360 seeded cases match isolated Horizon machine-code execution across REV1-REV15; the original game heaps initialize all five players and produce audio; the host suite, documentation and patch checks, SDK 8/20 smoke checks and iOS C compilation pass. iOS runtime and long battles remain untested.
2026-10-26 10:18:29 +08:00
seeddsandv0lumehi c4b74a62f9 Add thermal frame-rate cap with synchronized recovery (#220)
Hold the host frame step at 30 fps while iOS reports serious or critical
thermal pressure, and restore its configured cap after 30 calm seconds.
Keep the existing movie cap and DM_THERMAL_CAP=0 override.

Sample thermal state independently of movies and configured 30 fps targets
so heating resets recovery and cooling counts during playback. Serialize
the sample, cached cap and recovery timer across acquire/present readers.
The original PR failed five timing checks and ThreadSanitizer detected a
race through concurrent production cap getters.

Add controlled-clock production-path regressions and CI coverage, including
disabled/inactive caps and ThreadSanitizer. Record the shared mechanism
and validation limits in the frame-rate reference and BotW note.

Validated on latest main with the BotW profile: 24 thermal checks normally
and under ThreadSanitizer, 51 frame-step checks, 154 NVN checks, the
present-snapshot suite, 21 tool/docs checks, Mac host build, and iOS compile
check (133 sources). No new device gameplay or thermal recovery run.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 10:13:42 +08:00
seedds c76f679834 Remove alpha_from_red: Super Mario Odyssey's layout captures need only the nvnTextureCompare fix. The game creates each capture with its per-format default mapping (an R8 one reads (R, 0, 0, 1)), rebuilds it on the same storage with the capture's own, (1, 1, 1, R), and replaces the registered descriptor only on a difference (main+0x982320); while the host's comparison ignored the mapping the captures kept alpha 1 (white coin cards, a dark box behind hints), which the 26-shader alpha_from_red list papered over. Checked on the Mac from the "Look Around" hint with a probe host on the comparison (a c3a0880^ host for the profile's chain dec5ee141a9f libraries): 40 mapping-only rebuilds in 11,700 frames (R8 captures 640x80 down to 48x57 to (1, 1, 1, R), a 320x180 RGBA one to (R, G, B, A)); ignoring the mapping again brings back the box (about 12,800 pixels from frame 4,200), the old list hid it by converting the 378x80 capture, and with the fix the list converted nothing and changed no pixel of frames 4,200-6,600. The comparison and the views have no platform branch, so the iPhone gets the same rebuilt mapping (not run on a device). With no profile left using it, the key, its host code (tex_alpha_red, afr_want, DM_NVN_ALPHA_RED) and its docs go; treg_drop_ar becomes treg_drop_im (the image-slot view it also drops stays). Odyssey's note has the A/B runs. Checked: the host builds for Odyssey, the iOS compile check (133 files), the tool tests (111, 14 skipped) 2026-10-26 10:01:20 +08:00
seeddsandparsifal295 2db12f7efd Fix vibration handles before controller haptics routing (#246)
GetVibrationDeviceHandles used routed haptics availability to decide whether
handles existed. Games that cache this answer at startup could retain an empty
answer after a controller connected. Return handles for a connected npad style
when rumble is enabled and an output sink is installed; check routed outputs
when values are sent, keeping the existing permission and lifecycle behavior.

Include the PR's rumble diagnostics, count unroutable values individually in
mixed batches, and protect aggregate counters across game threads. Correct the
code comment to describe Unicorn Overlord's observed empty handles and zero
values without claiming restored physical rumble.

Add regressions for cached startup handles delivering a nonzero value after
routing, diagnostic totals and concurrent sends. Include HID in the existing
ThreadSanitizer target and record the mechanism in the service reference.

Checked with GAME=010069401ADB8000: full host suite, 73 HID checks, audio and HID
ThreadSanitizer checks, docs checks, Mac host build, and iOS compilation
(133 files). Physical rumble and iOS runtime remain untested. Broader tool
validation is incomplete because Capstone is missing and iconutil failed.

Co-authored-by: parsifal295 <130349765+parsifal295@users.noreply.github.com>
2026-10-26 09:54:43 +08:00
seedds 16061e0399 Rule: fix a bug at its root cause, not around it. AGENTS.md says to find why a bug happens (the faulting instruction and its decompilation, the bad value and what wrote it, the semantics the host or translator gets wrong) and fix it there; a change that only hides the symptom (stub, skipped call, clamp, retry, sleep or timeout, cache flush, profile key that turns a feature off) is a workaround, used only when the cause lies where it cannot be fixed (the game's own code, changed deliberately by a rewriter cut, or a platform limit) or the user asks for a stopgap, and then recorded as open with the cause or what is unknown. docs/agents/README.md has the same as a ground rule. Checked: dm/tools/tests/test_docs.py passes (AGENTS.md 1,419 words) 2026-10-26 09:42:57 +08:00
seeddsandv0lumehi c13ddcf531 Add opt-in MetalFX spatial upscaling and a single-screen launcher (#215)
Scale the valid presented image with MetalFX's spatial scaler on the shared
Metal 4 path, preserving crop, letterboxing, origin and gamma-encoded colours.
Keep Standard as the default and fall back when scaling is unsupported.

Use the same path for asynchronous whole-window captures and diagnostic
screenshots, retaining resources until GPU completion. Bound scaler recreation
during resizing and report presentation GPU time separately by scaling mode.

Put all launcher settings on one screen with Game, Controls and Performance
sections, inline menus and sliders, and a reset to tested defaults. Extract the
settings model for native persistence tests, validate saved ranges, share HUD
setup, and document the measured costs without claiming an iPhone baseline.

Validated with Metal presentation, NVN and snapshot regressions; tool and iOS
tests; the Mac host; and an unsigned iOS smoke build. No new device gameplay run.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 09:38:05 +08:00
seedds 3aa706a3fd Pokémon Shield (01008DB008C2C000, update 1.3.2 = v458752, Game Freak's engine on SDK 7.3.2): game profile, no host change. Sword's twin: the same sdk, subsdk0 (movie) and subsdk1 (GLSLC) builds, only main is its own (a1680262…), and the shader split gives Sword's 41,516 shaders with the same data rows in all three tables; the profile is Sword's with Shield's title, name, IPA name, build row and tables (prewarm list copied). The game note (dm/games/01008DB008C2C000/README.md) has what differs, the Mac route (a fresh save needs DM_KBD for the name; Sword's route out of the house works unchanged) and the shader build with DMSH_PREFILL from Sword's libraries (21,974 Mac / 21,684 iOS filled, 176 / 175 compiled). Plays on the iPhone (IPA 12.4 GB, user) and on the Mac at 30 fps with no red line from the first boot through the opening and Postwick to Hop. README gallery entry with a Mac screenshot, a row in the games table, a pointer from Sword's note, AGENTS.md's count to sixty-seven. Checked: the tool tests pass (111, 14 skipped) 2026-10-26 09:20:23 +08:00
seedds 3dd5d25a51 Add Echoes of Wisdom 1.1.0 port and correct cube fallback diagnostic 2026-10-26 09:14:43 +08:00
seeddsandv0lumehi 577df01c39 Fix iOS pulling shader packaging for profile defaults (#211)
Include pulling vertex libraries when the compiled game profile enables the
Metal 4 path, preserving explicit IOS_PULL overrides. Keep the pack options
array nonempty so inclusion also works with macOS Bash 3.2 and set -u.

Exercise the actual archive step with tiny shader fixtures for Breath of
the Wild and Wonder, checking both defaults and explicit overrides. Record
the packaging failures and fixes in the rendering reference and game note.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 08:56:50 +08:00
seeddsandAleksandr Svidchenkov 1a93086e2d Fix service-event and vsync worker lifetimes (#202)
Apply PR #202's host-thread skip and duplicate vsync suppression. Give
bindings unique lifetime IDs so a retired worker cannot signal another
service's event after guest storage and kernel handles are reused. Replace
the eight-entry worker table with allocated records so rapid replacement
does not abort while retired workers wait for their next tick.

Add deterministic regression cases through real event binding and SVCs,
and document the mechanism with an updated symptom index. Both the
capacity abort and stale signal reproduce against the original PR.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:35:19 +08:00
seedds 8b6f907fc1 Merge pull request #243 from seedds/codex/thread-start-handoff
Honor same-core priority handoffs when starting threads
2026-10-26 08:31:38 +08:00
seedds 02e40ba753 Record threading audit and retain verified game workarounds 2026-10-26 08:22:50 +08:00
seedds c188131824 Honor same-core priority handoff when starting threads 2026-10-26 08:22:50 +08:00
seeddsandAleksandr Svidchenkov 4896c78d8a Fix packaged texture dimensions and linear storage sizing (#201)
Decode packaged byte 0 as separate block-height and block-depth nibbles,
including explicit zero dimensions, and size linear compressed textures
by pixel rows. Port the driver-derived rules from PR #201 and preserve
its attribution to digitalhivehub's PR #11 measurements.

Integrate the layout test with current main, correct its BC4 label, add
independently addressed CPU markers and packaged GPU mip coverage, and
record the mechanisms in the reference docs and symptom index.

Validation: 9 driver shapes, 189 CPU texture checks (also ASan/UBSan),
17 GPU deswizzle cases, 154 NVN checks, documentation checks, Mac host
build, and iOS compilation of 133 sources pass. No new gameplay or
iPhone runtime validation.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:19:08 +08:00
seeddsandAleksandr Svidchenkov c1b7eff61f Fix Metal buffer alias visibility and sparse copy ordering (#200)
Carry ResourceAlias visibility on copy encoders containing buffer copies,
inline updates or fills, using the contribution from PR #200. Operations
through sparse virtual mappings also need ordering against surrounding
operations: disjoint guest addresses can name the same physical bytes,
so guest-range dependency detection alone misses their hazards.

Add a regression using distinct no-copy Metal buffers and real sparse
mappings. The original host failed 37 of 101 checks on an M4 Pro and
PR #200 alone left 24 failures; the complete fix passes all 101 checks
with both direct and deferred encoding under Metal validation. Ordinary
independent buffer operations retain range-based batching.

Validation: t_nvn passes 154 checks; the Mac host builds; all 133 iOS
source compile checks and the documentation checks pass. Monster Hunter
Rise gameplay, iPhone execution and sparse-barrier performance were not
retested.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:28:09 +08:00
seedds 31b5944eb4 Port NieR Automata with 60 fps and geometry fixes 2026-10-26 07:13:22 +08:00
seeddsandAleksandr Svidchenkov a5513d4ae0 Fix Depth16 load-clears to match fragment depth writes (#199)
Apple GPUs truncate Depth16Unorm load-clears while rounding fragment depth,
so clearing to 0.5 stores 32767 instead of the fragment's 32768 and fails
an EQUAL comparison. Apply the format-specific correction to both
attachment and whole-texture clears, preserving other depth formats.

Exercise both production command paths against independent GPU fragment
writes for 1,042 float depths, including half-bin boundaries and adjacent
floats. Removing either correction produces 518 mismatches on M4 Pro;
with both corrections enabled, neither path differs. Record the mechanism
and the pending TOTK highlight and iPhone validation in the documentation.

Validated on M4 Pro with Metal validation: t_depth16_clear, t_clear_ds
(43 checks), t_clear_tex (49 checks), and the documentation tests pass.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:06:32 +08:00
seedds 9a56ecf0c1 Order same-core wake handoffs and remove Dread movie gates
Use guest runnable state after a higher-priority same-core wake instead of Mach dispatch state or a 5 ms cutoff. Honour wake handoffs for every game and remove Metroid Dread request-ring gates and its core_preempt setting.

Validate event, mutex and condition-variable handoffs with delayed host dispatch; 13 valid Dread movie-transition runs passed. Host tests, docs and shader-chain checks passed, along with an iOS syntax check. Record Wonder boot comparison and pending iOS runtime validation.
2026-10-26 02:27:03 +08:00
seeddsandAleksandr Svidchenkov 9feac2f1e7 Rescue stalled display links with bounded retries (#198)
Switch active-app presents to drawable acquisition after 500 ms of display-link
silence, then retry with exponential backoff capped at 60 seconds. Preserve the
requested pacing mode, synchronize rescue state, and prevent focus timestamp
underflow from bypassing the foreground grace period.

Add CPU regression coverage in CI, a real Metal layer ownership check, and the
mechanism reference with upstream provenance and validation limits.

Validated: 43 CPU rescue checks; 47 checks with real Metal drawable acquisition;
34 present/crop and 10 pacing checks; Wonder Mac host build; all 133 iOS source
compile checks; documentation tests. Windowed gameplay recovery and iPhone
validation remain outstanding.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 02:25:06 +08:00
seeddsandAleksandr Svidchenkov 5b0c11a4f3 Fix NVN dirty-page upload races (#197)
Take upload stamps before protection, initialize cached checks from those
stamps, and wait out pages another thread is unprotecting before retrying.

Publish page generations before advancing the clock again in both the fault
handler and batched host pre-write path. A shared stamp_page helper closes
the remaining batched publication race in the original PR.

Add deterministic publication regressions for both paths and document the
mechanism and the limits of the game-level evidence. The regressions fail
on main, and the batched case also fails with the original PR applied.

Validation: full graphics suite, Wonder Mac host build, all 133 iOS source
compile checks, and documentation tests pass. No fresh gameplay or iPhone
runtime validation.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 02:06:27 +08:00
seeddsandAleksandr Svidchenkov 1fa521958a Merge PR #196: deduplicate normalized NVN pipeline keys
Drop unused normal-pass colour-target blend and mask state from pipeline keys.
Normalize historical prewarm rows and deduplicate their parsed keys before
compilation, preserving merged passes' logical target state.

Include production-path GPU regression coverage and the reference/game notes.
Validated on the latest main: Wonder host build, key serialization tests,
48 pending-pipeline checks, 48 archive checks, 158 prewarm GPU checks across
MTLDevice and Metal 4, and documentation checks. The regression fails on the
original base and detects wrong pixels if the merged-pass guard is removed.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:54:19 +08:00
seeddsandAleksandr Svidchenkov 796d592aa8 Test and document NVN prewarm key normalization
Exercise historical manifest rows through the production warmer, shader loader,
pipeline builders and draw-time cache. Check normal and merged passes on both
MTLDevice and Metal 4, with GPU pixels compared against the original descriptors.
The regression fails on the PR base and when the merged-pass guard is removed.

Document the shared normalization rule and keep game measurements in their notes.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:52:46 +08:00
seedds 6b947d9504 Fix Xcode workflow mock for the rewrite profile return
Return the fixture profile from the rewrite mock so the preparation pipeline keeps a valid game profile after the Portal change.

Validation: all three tests in dm/ios/tests/test_xcode_workflow.py pass; git diff --check passes.
2026-10-26 01:46:07 +08:00
seedds a8e76bf601 Port Portal: Source engine NROs on NVN, shaders from game.zip
- profile 01007BB017812000 (1.0.3, SDK 13.3.0): plays on the Mac at 60 fps
- provisional profiles add the subsdks the NROs import (profile_guess nro_modules)
- an NRO importing nvnBootstrapLoader makes an NVN game; NROs calling glslcCompile get the run-time GLSLC capture
- shader_split reads zip archive members (Portal's nxcontent/nvsb GLSLC outputs)
- ro::LookupModuleSymbol for a name a loaded module lacks is not red (Horizon fails it too)
- AGENTS.md: a Mac gameplay test is enough to count a game as playable (65)
2026-10-26 01:28:35 +08:00
seeddsandAleksandr Svidchenkov 6d4202b562 Merge PR #195: fix NVN autorelease ownership and registry contention
Keep the latest upload-staging and compact-binding fixes from main. Add a real-Metal readback lifetime regression and reference documentation for all three general fixes.

Validation: nine Metal test runs pass, including direct and deferred upload-budget tests; Wonder host build, 133-file iOS compile check, and documentation checks pass.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:27:04 +08:00
seedds 2d344f050c Test NVN readback lifetime and document general Metal fixes 2026-10-26 01:25:40 +08:00
seedds caee62a853 Compact NVN buffer bindings and remove Wonder heap workaround 2026-10-26 01:21:10 +08:00
seedds 7dc250e2f1 Bound NVN upload staging and retire it on upload completion
Split GPU upload batches at 128 MiB and retire overflow and CPU staging when the upload completes, rather than retaining it until the main rendering fence. Scope weak-reference diagnostics to avoid temporary staging retention.

Add a real-Metal regression covering memory bounds, upload pixels and an open render pass in direct and deferred encoding modes. Both modes pass 261 checks; documentation checks and the TotK host build pass.

Refs #147. Reporter-device validation and the full-device reboot investigation remain open.
2026-10-26 01:05:44 +08:00
seedds be5b197640 Port Persona 5 Royal with Needle shaders and NVN texture fixes 2026-10-25 23:49:24 +08:00
seeddsandb4f8abbb24dbf56a7d0280f296e16eeb c3a0880304 Guard Metal image atomics and reject unsupported bindings
Apply the generalized image-atomic bounds and backing checks from PR #178.
Also reject unsupported descriptor formats before integer fallback and fail
unresolved bindless atomics instead of silently deleting them. Rebuild the
translator and record chain 467efdfd0b62, with regression tests and findings.

Validation: compiler/helper and sanitizer tests, 96 Metal compile/link checks,
ten GPU probes, all 28,603 Z-A shaders for Mac and iPhone, and a 240-second
Mac station-area run reaching 6,987 frames without a freeze or GPU fault.
Reported iPhone cases and the station-to-world transition remain unverified.

Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
2026-10-25 23:37:28 +08:00
seeddsandXc1Ym 4f29560fac Merge PR #166: match verified game patches across updates
Integrate the Wonder 1.0.1 resource-heap fix and shared opt-in function matching for compatible game updates.

Validated with 95 tool tests and byte-identical Wonder 1.2.1 rewrite outputs for registered and synthetic unregistered build IDs.

Co-authored-by: Xc1Ym <29765332+Xc1Ym@users.noreply.github.com>
2026-10-25 23:26:44 +08:00
seedds be30c4aa55 Merge original PR 166 history into the generalized patch update 2026-10-25 23:21:58 +08:00
seedds b5c78fbddb Match verified game patches across compatible updates 2026-10-25 23:21:17 +08:00
seedds ec4ea16da0 Enable ASTRAL CHAIN 60 fps with corrected game timing 2026-10-25 20:59:04 +08:00
seedds 7a3b95873b Fix Mac app missing-module checks and quoted alerts 2026-10-25 19:49:34 +08:00
seedds ab45d05f3f Keep game gallery captions to noteworthy details 2026-10-25 19:48:24 +08:00
seedds da9d3b3ef6 Include Mac-tested games in README gallery
Replace the separate ASTRAL CHAIN bring-up notice with a gameplay screenshot and notes. Record that passing Mac gameplay testing is sufficient for gallery inclusion, with validation scope kept in each game note.
2026-10-25 19:43:14 +08:00
seedds 38006ef601 Remove descriptions from launcher haptics settings 2026-10-25 19:41:20 +08:00
seedds 1159ccc675 Add ASTRAL CHAIN profile and fix PlatinumGames startup compatibility
Support PKZ v1 archives and advertise the translator's GPU-code range independently of SDK version. Register the 1.0.1 profile, shader tables, regression checks and Mac bring-up findings. Retain 30 fps; document the unshipped 60 fps experiment and outstanding iPhone validation.
2026-10-25 19:36:05 +08:00
seedds 8ca9e9af93 Widen iOS launcher settings and remove list heading 2026-10-25 19:34:06 +08:00
seedds 9116109c9f Expand CI with texture, sanitizer and unsigned iOS build checks 2026-10-25 19:08:45 +08:00
seedds d62ddbbd25 Use native navigation list for iOS launcher settings 2026-10-25 18:59:47 +08:00
seedds 641799d51b Fix isolated SVC test linking after Unreal heap support
The dispatcher gained a dm_ue_heap_owns dependency in 16f4889. Supply the no-Unreal-allocator stub in both isolated test fixtures, alongside their existing host dependency stubs.

Validated the full host unit suite with the CI game profile and all nine CI CPU-only graphics tests.
2026-10-25 18:45:56 +08:00
seeddsandv0lumehi a1886130a6 Add opt-in frame buffering with safe snapshot pacing and suspend
Integrate the present snapshots and frame-step work from #87 together with
the encode-thread support and wait ledger from #157. Keep snapshots off
by default on both platforms pending wider device validation.

Offer Frame buffering in the iOS launcher for NVN builds. The existing
launcher remembers the choice and preserves env.txt overrides. Pass the
packer's detected graphics API to the settings generator.

Wait for snapshot capacity at acquire; a present reaching a busy slot
falls back to its original texture ticket. Drain snapshot commits and GPU
completion during suspend even when display tickets were discarded.
Regression tests cover both waits, fallback tickets and copied pixels.

Validated on M4 Pro: full gfx run list, 38 snapshot lifecycle checks,
219 checks with encoding and snapshots enabled, Mac host build, 133-file
iOS compile check, and eight launcher/docs/packing tests. The host unit
suite cannot link t_heap_size because dm_ue_heap_owns is missing, reproduced
on unchanged main cbbb600. No iPhone runtime performance claim.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-25 18:39:57 +08:00
seedds cbbb600866 OCTOPATH TRAVELER II: the IPA with the geometry split (geo_split, three-dispatch index compaction) and the Unreal-heap thread stacks (16f4889) plays on the iPhone (user); the game note says so and drops the open item 2026-10-25 17:46:41 +08:00
seedds 7d368c4249 Ghost Trick: Phantom Detective (010029B018432000, base game 1.0.0 = v0, Capcom's RE Engine on SDK 13.3.0, with the Bonus Content add-on): game profile, and finalized pool buffers no longer kept by a guest thread's autorelease pool (every game). Plays on the iPhone (IPA 3.0 GB, user)
The profile is the NSP script's provisional one with tex_flush_upload and tex_dirty_upload: RE Engine writes its UI textures (ASTC) into CPU-visible pools in place, as Monster Hunter Rise does, and without both keys the language screen's background and buttons were magenta (with tex_flush_upload alone they still were). 7,744 shaders from the PAK's SDF definitions and main; 7,739 build. The other 5 are compute shaders with atomics on a 3D image, which SPIRV-Cross indexes with an int3, as Octopath Traveler 0's 16; none was requested on the routes below. PR #138 (Nishikori-Yui) built them by flattening the coordinate in dmsh_build.py, a chain change that would rebuild every game's libraries, so it is left out until a run requests one (the game note). PR #138's profile is the same as this one (the same keys and shader tables).

Host fix: the memory line's "freed but alive" count only grew in most games (Animal Crossing: New Horizons 3,113 after 27,900 frames, Pokémon Sword 2,050, Super Smash Bros. Ultimate 1,279; Ghost Trick 125 to 146 buffers, 6.6 to 8.7 GB of zero-copy buffers, so GPU allocated read 9.9 to 11.7 GB against 2.2 GB resident). A temporary census of every host table that can hold a texture over a pool buffer found no holder: mtl_pool_del runs on the guest thread that finalizes the pool, whose implicit autorelease pool never drains, and adding the buffer to the weak table g_dead_pools left it autoreleased there. containsObject: / removeObject: on a weak NSHashTable do the same (a standalone test), and res_add / res_remove call them on guest threads too. mtl_pool_del, res_add_at and res_remove now have their own @autoreleasepool. Ghost Trick: 0 freed but alive, GPU allocated 1.7 GB instead of 8.3 GB 300 frames into the boot and 2.9 GB in Chapter 1, resident memory, frame times and pictures unchanged; Pokémon Sword: 8 to 0 at the title after 90 s, the same frame pacing.

On the Mac: the opening movies, the language screen, the first-boot notices, save creation, the Bonus Content notices and Chapter 1's opening in the junkyard (the Ghost World, the first trick), and a second boot that loads the save, at 60 fps (frame interval p99 about 17 to 19 ms), about 2.3 GB resident, no red line; the Wwise sound plays (a muted run with DM_AUDIO_DUMP). Checked: the tool tests (57, the docs check among them), t_nvn (124 checks), a boot of the rebased host. README (games gallery, a Mac picture of the junkyard), AGENTS.md's count (sixty-two: the gallery's cells), the game note, the games table, docs/reference/nvn-objects.md and the symptom index say so
2026-10-25 17:44:41 +08:00
seedds 16f4889192 OCTOPATH TRAVELER II (0100A3501946E000, update 1.1.1 = v131072, Unreal Engine 4.27 on SDK 12.3.2): game profile, inline Unreal shaders in version 11 paks, and five general host fixes. Plays on the iPhone (IPA 5.9 GB, user; that IPA predates the geometry and allocator changes below). On the Mac: the boot, the title, a new game, the traveler selection, Hikari's opening into his first battle (won) and the scene after it, and Castti's opening (the flashback, the ship, free play on the deck), at 60 fps (frame time p50 16.7 ms, p95 about 17 ms) with CRIWARE sound, about 2.8 GB resident. Shaders: the project ships no shader code archives (bShareMaterialShaderCode off), so each material's shaders are inline in its .uexp and the global ones in Engine/GlobalShaderCache-SF_GLSL_SWITCH.bin, each an entry of FShaderMapResourceCode (an LZ4 block, its raw size, its frequency): uepak.py inline_shaders sends a version 11 pak with cooked packages and no .ushaderbytecode to shader_split.py's inline reader (ue_legacy_jobs; Zlib and LZ4 pak blocks decoded in Python, Pak.read), ue_inline_shaders finds each LZ4 block by its first literals (LSLG), and profile_guess.py / the NSP script tell the two kinds apart; 13,980 shaders in 13 s. Profile: Unreal's FMallocBinned3 cut with an OBS_UE_MALLOC gate as DRAGON QUEST VII Reimagined's (GCreateMalloc main+0xbecbf0, a 20-slot vtable, hle_uemalloc.c), a 2 GB engine region (game_args) with the node-area cut on this SDK 12.3.2 build (sdk_patches.py, sdk+0x346190 / +0x34622c), fps_target 60 through t.MaxFPS / rhi.SyncInterval (hle_fpslock.c; the same route at 30 and 60 fps reaches the same scenes at the same times), pad_layout full (the battle menu picks weapons with the D-pad), a prewarm list of 843 lines. General fixes: (1) a shader may bind 96 resources instead of 64 (NVN_REFL_MAXB: the traveler selection's 3D map shader binds 66, and the map was black); (2) the parallel precompile look-ahead (pso_predict_rest) restores the scissor, which state_op keeps in R, not S: with pipelines built on demand the picture went bright green and white from Castti's flashback and Hikari's battlefield on (bisected with simultaneous runs; DM_NVN_PREDICT=0 turns the look-ahead off); (3) a geometry draw above the 256 MiB capture scratch bound is drawn in runs instead of skipped (nvn_geometry.inc geo_split: one instance at a time, then runs of at most 4,096 list primitives whose index range fits 64 MB, one read of the indices, one GPU wait), the index compaction after the geometry kernel runs as count / offset / write dispatches instead of one thread (each frame drawing Unreal's 129,032-triangle point light shadow took about 300 ms with it), and scratch arenas above 32 MB are released after 600 idle submissions (arena_new takes the smallest that fits); (4) MapMemory takes a thread stack from the host's Unreal heap (dm_ue_heap_owns: a range outside the guest block the host maps read-write, vm_region_64): with cae5ec2's stricter source check every Unreal game on the host heap aborted in MapAliasStack at nnMain (checked: this game and Pikmin 4 boot); (5) the host Makefile rebuilds on a change to dm/gfx/nvn/*.inc. Scripted input takes partly tilted sticks (L<x>/<y>): the traveler selection opens a card only while its quill cursor rests on a traveler, and full-tilt presses swept past them. Open: 2 of 87 Mac boots crashed at the title in the game's CRI cue sheet loader (main+0x1d8054, a record copied while one of its TArrays held count 1 and no data); the writer is not found yet (the game note). Checked: 57 tool tests, the gfx run list (t_geometry 95 checks) and the docs check pass; the NSP script packs the IPA from the NSPs (Mac and iOS). README (games gallery, a Mac picture of Hikari's first battle), the game note, the games table, docs/reference shaders / memory, docs/agents/04 and the symptom index say so 2026-10-25 15:43:48 +08:00
seedds 18414bb5ff Fix resource heap slack for Super Mario Bros. Wonder 1.0.1 2026-10-25 15:33:15 +08:00
seedds 93578abad6 13 Sentinels: Aegis Rim, Asian release (01008D7016438000, base game 1.00 = v0, Japanese / Korean / Chinese): game profile, no host change (PR #161 by lx7720)
The Asian release has its own title ID. Its subsdk0 and sdk are the western builds, but its main (31194702…) is not the western 1.10's (4b632573…), so the NSP script's lookup by main build does not find the western profile, and its RomFS marker is ROBO_CN.CPK. The profile is the NSP script's provisional one with the western profile's alloc_big_align (the engine's ring heaps are cut from a pool at 64 KB alignment) and the same 4 GB layout. Its shader tables have the same keys as the western ones (all 751 shader_map rows; shader_pairs, shader_facts and bf_want identical), at other offsets in main.

Ported onto main with its game note (dm/games/01008D7016438000/README.md), its row in the games table and a line in the western note pointing to it. The game.tsv header is in English (the PR had the Japanese and Korean titles as characters). README's gallery and AGENTS.md's count are unchanged: it is the same game and is not tested on the iPhone.

Checked (PR #161): on the Mac the NSP script packs it from the base NSP alone and it plays at 60 fps with sound, saving and loading. Here: the tool tests in dm/tools/tests pass, test_docs among them (61 profiles), and game.py lists the profile and writes its header. Not checked: iOS (no Asian NSP here).
2026-10-25 13:59:43 +08:00
seedds cdd5619226 Docs check: new dm/tools/tests/test_docs.py, which the CI job runs with the other tool tests. It fails when a relative link or #anchor in the repository's own Markdown does not resolve (GitHub's heading anchors, repeated headings numbered; files from git ls-files, third_party and node_modules left out), when a profile with a game.tsv has no README.md or no row in the games table of dm/games/README.md (or the table lists a folder with no profile), when docs/agents/symptoms.md is not what dm/tools/docs_symptoms.py writes from docs/reference, or when AGENTS.md passes 1,500 words or a numbered guide in docs/agents 6,000. AGENTS.md says so. Checked: the test passes (88 Markdown files, 2,293 links, 60 profiles), and fails as it should with a link to a missing file, a link to a missing heading, a Symptom bullet changed without regenerating the index, and AGENTS.md 400 words longer 2026-10-25 13:07:24 +08:00
seedds 0df4dbe280 A short AGENTS.md: what the repository is (the native-execution model, the host and its three graphics layers, sixty games, each with its note in dm/games/<id>/README.md and a row in the games table), where to start for each kind of task (a crash, abort, hang, [RED] line or wrong picture: the symptom index, then the reference entry and its games; a game short of its frame loop: 03; rendering: 04 and the rendering, shaders and NVN objects references; one game or its profile: its note; another title: the NSP script and docs/agents; packing, building, prewarm, power: development.md), and where a new finding goes, once (a fact about one game in its note; a general mechanism as an entry in its reference, the symptom index regenerated and a line in the note's Found on this game; this file only for a rule or the count). The rules are as before. The game list of 4,500 words, one sentence per game, is gone: each sentence is in its game's note. New dm/tools/docs_symptoms.py writes docs/agents/symptoms.md from the references' Symptom bullets (--check reports whether it is current). The NSP script's survey points at docs/reference/nvn-objects.md for SDK object sizes and at the symptom index; Balatro's note names capture_shaders. AGENTS.md goes from 6,235 to 1,137 words. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 on main still appears in the repository's Markdown (besides the five file patterns, nn::hid::GetGestureStates and nn::oe::SetCpuOverclockEnabled appear without the nn:: prefix in Taiko Risshiden V DX's note), every relative link and anchor resolves, and the tool tests and test_xcode_workflow pass 2026-10-25 12:56:14 +08:00
seedds 50ad443cb1 Bayonetta and Bayonetta 2 in the new layout: their game notes (dm/games/010076F0049A2000/README.md, dm/games/01007960049A0000/README.md) and rows in the games table, from their Game notes entries in docs/development.md, their AGENTS.md sentences, their profiles' comments and their commits; the SDK 3.6.0 entry of 03's catalogue becomes services.md "SDK 3.6.0 answered as SDK 4", and 04's two Seen-in sections become shaders.md "BNSH files in PlatinumGames DAT archives inside CRI CPK archives" (cpk.py, cpk_jobs) and rendering.md "Clears of attachments larger than the render pass" (rp_begin_ex_) and "Downsample of a single-sample texture" (copy_t2t), each with the games it was found on. Mario Kart 8 Deluxe's note no longer calls SDK 4.4.0 the oldest. Checked: every profile with a game.tsv has a note and a table row (60), every relative link and anchor in the repository's Markdown resolves, and every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 on main (4,669 items) still appears (the same five file patterns as before) 2026-10-25 12:44:03 +08:00
seedds 3a993fa3fe Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in <game>" subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games/<id>/README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv) 2026-10-25 12:31:58 +08:00
seedds 846c54c63c Per-game notes beside each profile: dm/games/<id>/README.md for all 58 games, and the games table in dm/games/README.md. Each note says what the profile needs and why (a table of its game.tsv keys, patches.py cuts and frame-step stubs, sdk_patches.py cuts for its sdk build, manual shader rows), the game's own findings by topic (frame rate, memory, rendering, audio, movies, services, files and shaders, input, Mac routes), each marked current or resolved, a "Found on this game" list of the general host changes first needed there (one line each, linked to where they are explained), and what is still open, including stale profile comments and figures the sources disagree on. The notes gather what was spread over AGENTS.md, docs/development.md (Game notes and Power), docs/agents/01-05 and the profiles' own comments. The Game notes section of docs/development.md (523 lines, 53 games) is now a pointer to the table; development.md goes from 33,300 to 8,600 words. README.md, AGENTS.md and the Breath of the Wild link in 04-graphics.md point to the notes. A README.md in a profile folder changes no shader cache key (dmsh_chain.inputs hashes only the named tables and keys) and no profile listing (game.py lists folders with a game.tsv). Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (five file patterns now appear as the full names they stood for), and every relative link and heading anchor in the repository's Markdown resolves 2026-10-25 12:28:47 +08:00
seedds b7dfba4dd0 Load LZ4 only when decoding NSO files
The memory-profile host test imports profile_guess, which imported nso_syms and required LZ4 even for plain log parsing. The host CI job has no third-party Python packages, so this broke the host suite after cae5ec2. Move the import into load() and document the standard-library-only heap probe.

Validated the exact failing target and full parallel host suite with system Python, the heap probe with site packages disabled, six related tool tests, and synthetic compressed NSO decoding.
2026-10-25 12:27:40 +08:00
seeddsandb4f8abbb24dbf56a7d0280f296e16eeb cae5ec2155 Keep guest memory queries, budgets and stack ownership consistent
Incorporate the active SVC improvements from PR #151 around the existing
alias bitmap and mapping registry. Serialize heap changes, physical maps,
stack ownership and memory queries with one lock; report committed alias
runs and borrowed sources; initialize reused logical pages while preserving
live neighbors and lazy first exposure. Retain PR #127's fail-fast host
protection policy and the log formats parsed by the NSP heap probe.

Game smoke tests exposed two integration requirements: Raging Loop's SDK
extends committed windows across borrowed stack sources, so allow recommit
that preserves existing bytes; Tears of the Kingdom's audio helper needs
guest-backed stacks, so allocate host-created guest-thread stacks through
SDK malloc and free them after thread destruction and copyback.

Keep SDK reservation ownership and existing patches. The unused host
reservation allocator from #151 is not included.

Validation: full host suite; focused UBSan, ASan and ThreadSanitizer tests;
real dirty-tracker concurrency tests; heap-probe parser compatibility; Mac
host builds; iOS arm64 syntax checks; all 60 profile layout checks.
Raging Loop and Tears of the Kingdom passed guest-thread self-checks and
ran to deliberate timeouts after the startup fixes. Their shader caches
are stale for this checkout, so rendering/performance are not validated.
No IPA build or iOS device validation was performed.

Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
2026-10-25 12:23:01 +08:00
seeddsandv0lumehi 66abbc3d30 Cache Metal 4 pipeline bases with per-game archives
Adapt PR #122: compile reusable bases with unspecialized colour state, specialize per draw state, and persist bases between launches. Keep iOS opt-in and the MTLDevice fallback.

Preserve rasterizing depth-only pipelines through the legacy path. Share in-progress base builds, including errors, while unrelated bases compile in parallel. Defer timed flushes during compaction and finish flush/cleanup together; synchronize the pending count with serializer capture.

Checked on Apple M4 Pro: t_pso_archive 48 checks with Metal validation, t_pso_pending 48, t_integer_blend 63, t_geometry 95, and t_pso_skey pass. The Breath of the Wild Mac host builds and nvn_mtl.m passes the iOS arm64 syntax check. Real-game memory usage and iPhone runtime behavior were not measured.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-25 11:31:14 +08:00
seeddsandnocmt 9ba44da80a Fail fast when SetHeapSize protection changes fail
Apply PR #127 and require the regression test's abort handler to observe a validated production fatal path. Unrelated assertion failures can no longer pass the injected-failure cases.

Co-authored-by: nocmt <nocmtone@gmail.com>
2026-10-25 11:19:22 +08:00
seedds 1fbbd1486a Bayonetta (010076F0049A2000, update 1.1 = v65536, PlatinumGames' engine in its Wii U form on SDK 3.6.0, as Bayonetta 2): game profile, no host change. Plays on the iPhone (IPA 8.8 GB, user). Same sdk and subsdk0 builds as Bayonetta 2, so its profile is Bayonetta 2's with its own build row and tables, and the CPK reader and the two NVN fixes made for Bayonetta 2 cover it. On the Mac from a fresh save: the boot, the opening movie (USM; Plus skips it), the title and its attract demo, a new game (PLAY, NEW GAME, the auto-save notice, Normal), the prologue on the falling clock tower, the graveyard cutscene (skipped), the touch-controls tips, the practice tutorial (punch, kick, combos, evade, Witch Time), the graveyard fight of Verse 1 with Witch Time, its autosave and the next loading screen's practice, with CRI ADX2 sound through one 6-channel AudioOut and no red line: 60 fps in play (frame time p50 16.65 ms, p95 17.1-17.5 ms; GPU 2-4 ms a frame on an M6), 30 in its cutscenes, where the game sets present interval 2; about 2.45 GB resident and 1.54 GB of Metal allocations, flat over 15 minutes. No virtual address memory (a 3.11 GB heap in the 4 GB layout, mem_used 102 MB). Files: nine CRI CPK archives (data00.cpk ... data03.cpk, data10.cpk ... data14.cpk, 4.25 GB) of PlatinumGames DAT archives, beside the USM movies and the ADX2 sound. Shaders: 458 BNSH files of one stage each (236 pixel, 222 vertex), the .vso / .pso members of shader.dat in data00.cpk, all built for the Mac and iOS. Profile: the 3D pad layout (the camera is on the right stick; the touch controls are handheld only), a prewarm list recorded on the Mac over the route above (513 lines). Checked: the NSP script packs it from the NSPs alone (Mac and iOS); the Python tool, gfx and iOS tests pass (test_tbdr_budget with DM_GAME set). README (games gallery, a Mac picture of the graveyard fight), AGENTS.md (59 -> 60), docs/development.md and docs/agents 01 / 03 / 04 say so 2026-10-25 11:03:12 +08:00
seeddsandnocmt 787523f2cc Mac front end: input held when the game window loses focus is released (general, from PR #128). A window that is no longer key sees no key-up, mouse-up or controller release, so a key, a click or a controller button held when the window lost focus (another app, the software keyboard's text box, minimizing) stayed pressed in the game. Now front_mac.m releases what the front end itself pressed when the window resigns key or is minimized (input_release): the keyboard buttons and the WASD stick, the touch only while the mouse holds it (g_mouse_down), and the pad of each player a controller holds (in DM_PAD_MERGE mode player 1's, when a controller is connected); while out of focus the keyboard, the mouse and the controllers' value changes are ignored (g_input_on), and when the window is key again each held controller is read as it is now (input_resume), so a button still held or a stick still deflected counts again. From PR #128 (nocmt), with two changes: the PR released player 1's pad and the touch in every case, which are also DM_KEYS' and DM_CTL's paths (hle_hid.c), so a scripted hold or drag ended early at a focus change during a Mac run; and it took input up again only if NSApp.isActive held when the window became key, so if AppKit makes the window key before the app counts as active, the keyboard, the mouse and the controllers stayed off until the next focus change. Also left out: the handler queue set to the main queue (already GameController's default), the focus state 3 on resigning key while minimized (DidMiniaturize sets it), the release on deminiaturizing (nothing is held then), and the t_hid test, which checked only that dm_input_set / dm_pad_set_player with zeros clear the state, not the focus handling. Checked: the host unit tests pass (GAME=010015100B514000), the Mac host builds for Mario vs. Donkey Kong, and a DM_WINDOW run boots it with DM_CTL presses reaching the game. Not checked: an actual focus change (in that run the window never became key or resigned it: the shell's activation requests and Finder's activation left no focus line), a physical controller, the software keyboard's text box by hand
Co-authored-by: nocmt <26828323+nocmt@users.noreply.github.com>
2026-10-25 10:41:24 +08:00
seedds 7afd90b0b2 Tests: allow CI scheduling delays in audio stall recovery
Raise the recovery test's elapsed-time ceiling from 2.6 to 5 seconds while retaining buffer ordering, minimum playback duration, and resumed callback checks. The original CI run recovered correctly in 2686 ms and passed unchanged on rerun.

Validated with the full host unit suite and all nine CPU-only graphics tests using GAME=010015100B514000.
2026-10-25 10:31:57 +08:00
seedds 0f99c353d3 Haptics: a Core Haptics engine that fails to start is replaced, not retried for good (general, from PR #143). A Bluetooth Switch Pro controller that vibrates through iOS Settings' Identify stayed silent in a game, which logged Core Haptics -4810 (server init failed): the adapter made each engine once per route and called startAndReturnError on that same broken engine at every later effect, and its stop/reset handlers were installed only after the start. Now haptics_apple.m makes an engine when the first effect needs it (the phone's too, so after the audio session is active), installs the handlers before starting it, and a failure to make or start the engine, make or start its player or send its parameters releases that engine: a fresh game signal makes a new one after a backoff of 1, 2, 4 or 8 s, effects from before the failure are not replayed, and the old engine's callbacks are stale (generation). A controller whose left and right handle engines cannot be made or started falls back to its default locality until it reconnects; a player or parameter failure (a service reset, say) is retried on the same handle. Idle release (1 s of silence), focus loss and engine resets stop an engine but keep it, so the next effect restarts the same engine, as before. Errors name the target, the stage, the error domain and code and the underlying error, and the log names the controller, its localities and the output channels. From PR #143 (nanda86), with two changes: the PR dropped the engine at every stop, so each rumble after a second of silence made a new engine (a new connection to the haptics service on a controller, a new initWithAudioSession: engine on the phone), and any handle failure moved the route to the default locality for good. t_haptics_apple covers each failure stage, the backoff, the fallback (and no fallback on a player failure), stale callbacks, a restarted engine after idle release, a disconnect and a controller without haptics, with fake controllers and engines. Checked: the host unit tests pass (GAME=0100DCA0064A6000), haptics_apple.m compiles for iOS arm64 with -Wall -Werror. Not checked: a physical controller or the iPhone's own haptics 2026-10-25 09:55:44 +08:00
seedds ca658c9dde NVN BufferInitialize: a buffer past the end of its pool is a one-line note, not a red line (general, found on Breath of the Wild). Breath of the Wild creates one 256 KB buffer (offset 0) over a 144 KB pool when the pause menu opens and near a save commit (Mac and iPhone: the same line, once each time). The NVN driver does not check a buffer's range against its pool, and the host never reads a buffer's size to address memory: every GPU access takes an address and a length from its command and resolves it in the pool's memory (mtl_addr / pool_find), which reports a range that leaves a pool on its own (RED_ONCE). The buffer's only use is BufferGetAddress and one texture-to-buffer copy (nvnCommandBufferCopyTextureToBuffer, called from main+0xec2b80 on the unpatched module) of the screen picture's own extent, which resolved inside the pool (no "destination is in no memory pool" line), then it is finalized in the same function; it is never mapped. Now a non-red [nvn] line, once per run. Checked on the Mac (BotW, a copy of the save): the pause menu opens with the inventory drawn as before and the line is a note; the host tests and the gfx run list pass. From PR #145 (v0lumehi). Also checked on current main: the host unit tests and the gfx run list pass. Side finding of the PR, not changed here: in the world, Breath of the Wild's uniform buffer binds of 0x750 bytes at +0xa00 of one-page pools straddle the pool end (pool_find returns -1 for them) 2026-10-25 09:36:51 +08:00