Commit Graph
27 Commits
Author SHA1 Message Date
seeddsandAleksandr Svidchenkov 1a93086e2d Fix service-event and vsync worker lifetimes (#202)
Apply PR #202's host-thread skip and duplicate vsync suppression. Give
bindings unique lifetime IDs so a retired worker cannot signal another
service's event after guest storage and kernel handles are reused. Replace
the eight-entry worker table with allocated records so rapid replacement
does not abort while retired workers wait for their next tick.

Add deterministic regression cases through real event binding and SVCs,
and document the mechanism with an updated symptom index. Both the
capacity abort and stale signal reproduce against the original PR.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:35:19 +08:00
seedds 02e40ba753 Record threading audit and retain verified game workarounds 2026-10-26 08:22:50 +08:00
seedds c188131824 Honor same-core priority handoff when starting threads 2026-10-26 08:22:50 +08:00
seeddsandAleksandr Svidchenkov 4896c78d8a Fix packaged texture dimensions and linear storage sizing (#201)
Decode packaged byte 0 as separate block-height and block-depth nibbles,
including explicit zero dimensions, and size linear compressed textures
by pixel rows. Port the driver-derived rules from PR #201 and preserve
its attribution to digitalhivehub's PR #11 measurements.

Integrate the layout test with current main, correct its BC4 label, add
independently addressed CPU markers and packaged GPU mip coverage, and
record the mechanisms in the reference docs and symptom index.

Validation: 9 driver shapes, 189 CPU texture checks (also ASan/UBSan),
17 GPU deswizzle cases, 154 NVN checks, documentation checks, Mac host
build, and iOS compilation of 133 sources pass. No new gameplay or
iPhone runtime validation.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:19:08 +08:00
seeddsandAleksandr Svidchenkov c1b7eff61f Fix Metal buffer alias visibility and sparse copy ordering (#200)
Carry ResourceAlias visibility on copy encoders containing buffer copies,
inline updates or fills, using the contribution from PR #200. Operations
through sparse virtual mappings also need ordering against surrounding
operations: disjoint guest addresses can name the same physical bytes,
so guest-range dependency detection alone misses their hazards.

Add a regression using distinct no-copy Metal buffers and real sparse
mappings. The original host failed 37 of 101 checks on an M4 Pro and
PR #200 alone left 24 failures; the complete fix passes all 101 checks
with both direct and deferred encoding under Metal validation. Ordinary
independent buffer operations retain range-based batching.

Validation: t_nvn passes 154 checks; the Mac host builds; all 133 iOS
source compile checks and the documentation checks pass. Monster Hunter
Rise gameplay, iPhone execution and sparse-barrier performance were not
retested.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:28:09 +08:00
seedds 31b5944eb4 Port NieR Automata with 60 fps and geometry fixes 2026-10-26 07:13:22 +08:00
seeddsandAleksandr Svidchenkov a5513d4ae0 Fix Depth16 load-clears to match fragment depth writes (#199)
Apple GPUs truncate Depth16Unorm load-clears while rounding fragment depth,
so clearing to 0.5 stores 32767 instead of the fragment's 32768 and fails
an EQUAL comparison. Apply the format-specific correction to both
attachment and whole-texture clears, preserving other depth formats.

Exercise both production command paths against independent GPU fragment
writes for 1,042 float depths, including half-bin boundaries and adjacent
floats. Removing either correction produces 518 mismatches on M4 Pro;
with both corrections enabled, neither path differs. Record the mechanism
and the pending TOTK highlight and iPhone validation in the documentation.

Validated on M4 Pro with Metal validation: t_depth16_clear, t_clear_ds
(43 checks), t_clear_tex (49 checks), and the documentation tests pass.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:06:32 +08:00
seedds 9a56ecf0c1 Order same-core wake handoffs and remove Dread movie gates
Use guest runnable state after a higher-priority same-core wake instead of Mach dispatch state or a 5 ms cutoff. Honour wake handoffs for every game and remove Metroid Dread request-ring gates and its core_preempt setting.

Validate event, mutex and condition-variable handoffs with delayed host dispatch; 13 valid Dread movie-transition runs passed. Host tests, docs and shader-chain checks passed, along with an iOS syntax check. Record Wonder boot comparison and pending iOS runtime validation.
2026-10-26 02:27:03 +08:00
seeddsandAleksandr Svidchenkov 9feac2f1e7 Rescue stalled display links with bounded retries (#198)
Switch active-app presents to drawable acquisition after 500 ms of display-link
silence, then retry with exponential backoff capped at 60 seconds. Preserve the
requested pacing mode, synchronize rescue state, and prevent focus timestamp
underflow from bypassing the foreground grace period.

Add CPU regression coverage in CI, a real Metal layer ownership check, and the
mechanism reference with upstream provenance and validation limits.

Validated: 43 CPU rescue checks; 47 checks with real Metal drawable acquisition;
34 present/crop and 10 pacing checks; Wonder Mac host build; all 133 iOS source
compile checks; documentation tests. Windowed gameplay recovery and iPhone
validation remain outstanding.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 02:25:06 +08:00
seeddsandAleksandr Svidchenkov 5b0c11a4f3 Fix NVN dirty-page upload races (#197)
Take upload stamps before protection, initialize cached checks from those
stamps, and wait out pages another thread is unprotecting before retrying.

Publish page generations before advancing the clock again in both the fault
handler and batched host pre-write path. A shared stamp_page helper closes
the remaining batched publication race in the original PR.

Add deterministic publication regressions for both paths and document the
mechanism and the limits of the game-level evidence. The regressions fail
on main, and the batched case also fails with the original PR applied.

Validation: full graphics suite, Wonder Mac host build, all 133 iOS source
compile checks, and documentation tests pass. No fresh gameplay or iPhone
runtime validation.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 02:06:27 +08:00
seeddsandAleksandr Svidchenkov 1fa521958a Merge PR #196: deduplicate normalized NVN pipeline keys
Drop unused normal-pass colour-target blend and mask state from pipeline keys.
Normalize historical prewarm rows and deduplicate their parsed keys before
compilation, preserving merged passes' logical target state.

Include production-path GPU regression coverage and the reference/game notes.
Validated on the latest main: Wonder host build, key serialization tests,
48 pending-pipeline checks, 48 archive checks, 158 prewarm GPU checks across
MTLDevice and Metal 4, and documentation checks. The regression fails on the
original base and detects wrong pixels if the merged-pass guard is removed.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:54:19 +08:00
seeddsandAleksandr Svidchenkov 796d592aa8 Test and document NVN prewarm key normalization
Exercise historical manifest rows through the production warmer, shader loader,
pipeline builders and draw-time cache. Check normal and merged passes on both
MTLDevice and Metal 4, with GPU pixels compared against the original descriptors.
The regression fails on the PR base and when the merged-pass guard is removed.

Document the shared normalization rule and keep game measurements in their notes.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:52:46 +08:00
seedds a8e76bf601 Port Portal: Source engine NROs on NVN, shaders from game.zip
- profile 01007BB017812000 (1.0.3, SDK 13.3.0): plays on the Mac at 60 fps
- provisional profiles add the subsdks the NROs import (profile_guess nro_modules)
- an NRO importing nvnBootstrapLoader makes an NVN game; NROs calling glslcCompile get the run-time GLSLC capture
- shader_split reads zip archive members (Portal's nxcontent/nvsb GLSLC outputs)
- ro::LookupModuleSymbol for a name a loaded module lacks is not red (Horizon fails it too)
- AGENTS.md: a Mac gameplay test is enough to count a game as playable (65)
2026-10-26 01:28:35 +08:00
seeddsandAleksandr Svidchenkov 6d4202b562 Merge PR #195: fix NVN autorelease ownership and registry contention
Keep the latest upload-staging and compact-binding fixes from main. Add a real-Metal readback lifetime regression and reference documentation for all three general fixes.

Validation: nine Metal test runs pass, including direct and deferred upload-budget tests; Wonder host build, 133-file iOS compile check, and documentation checks pass.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 01:27:04 +08:00
seedds 2d344f050c Test NVN readback lifetime and document general Metal fixes 2026-10-26 01:25:40 +08:00
seedds caee62a853 Compact NVN buffer bindings and remove Wonder heap workaround 2026-10-26 01:21:10 +08:00
seedds 7dc250e2f1 Bound NVN upload staging and retire it on upload completion
Split GPU upload batches at 128 MiB and retire overflow and CPU staging when the upload completes, rather than retaining it until the main rendering fence. Scope weak-reference diagnostics to avoid temporary staging retention.

Add a real-Metal regression covering memory bounds, upload pixels and an open render pass in direct and deferred encoding modes. Both modes pass 261 checks; documentation checks and the TotK host build pass.

Refs #147. Reporter-device validation and the full-device reboot investigation remain open.
2026-10-26 01:05:44 +08:00
seedds be5b197640 Port Persona 5 Royal with Needle shaders and NVN texture fixes 2026-10-25 23:49:24 +08:00
seeddsandb4f8abbb24dbf56a7d0280f296e16eeb c3a0880304 Guard Metal image atomics and reject unsupported bindings
Apply the generalized image-atomic bounds and backing checks from PR #178.
Also reject unsupported descriptor formats before integer fallback and fail
unresolved bindless atomics instead of silently deleting them. Rebuild the
translator and record chain 467efdfd0b62, with regression tests and findings.

Validation: compiler/helper and sanitizer tests, 96 Metal compile/link checks,
ten GPU probes, all 28,603 Z-A shaders for Mac and iPhone, and a 240-second
Mac station-area run reaching 6,987 frames without a freeze or GPU fault.
Reported iPhone cases and the station-to-world transition remain unverified.

Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
2026-10-25 23:37:28 +08:00
seedds b5c78fbddb Match verified game patches across compatible updates 2026-10-25 23:21:17 +08:00
seedds ec4ea16da0 Enable ASTRAL CHAIN 60 fps with corrected game timing 2026-10-25 20:59:04 +08:00
seedds 1159ccc675 Add ASTRAL CHAIN profile and fix PlatinumGames startup compatibility
Support PKZ v1 archives and advertise the translator's GPU-code range independently of SDK version. Register the 1.0.1 profile, shader tables, regression checks and Mac bring-up findings. Retain 30 fps; document the unshipped 60 fps experiment and outstanding iPhone validation.
2026-10-25 19:36:05 +08:00
seeddsandv0lumehi a1886130a6 Add opt-in frame buffering with safe snapshot pacing and suspend
Integrate the present snapshots and frame-step work from #87 together with
the encode-thread support and wait ledger from #157. Keep snapshots off
by default on both platforms pending wider device validation.

Offer Frame buffering in the iOS launcher for NVN builds. The existing
launcher remembers the choice and preserves env.txt overrides. Pass the
packer's detected graphics API to the settings generator.

Wait for snapshot capacity at acquire; a present reaching a busy slot
falls back to its original texture ticket. Drain snapshot commits and GPU
completion during suspend even when display tickets were discarded.
Regression tests cover both waits, fallback tickets and copied pixels.

Validated on M4 Pro: full gfx run list, 38 snapshot lifecycle checks,
219 checks with encoding and snapshots enabled, Mac host build, 133-file
iOS compile check, and eight launcher/docs/packing tests. The host unit
suite cannot link t_heap_size because dm_ue_heap_owns is missing, reproduced
on unchanged main cbbb600. No iPhone runtime performance claim.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-25 18:39:57 +08:00
seedds 7d368c4249 Ghost Trick: Phantom Detective (010029B018432000, base game 1.0.0 = v0, Capcom's RE Engine on SDK 13.3.0, with the Bonus Content add-on): game profile, and finalized pool buffers no longer kept by a guest thread's autorelease pool (every game). Plays on the iPhone (IPA 3.0 GB, user)
The profile is the NSP script's provisional one with tex_flush_upload and tex_dirty_upload: RE Engine writes its UI textures (ASTC) into CPU-visible pools in place, as Monster Hunter Rise does, and without both keys the language screen's background and buttons were magenta (with tex_flush_upload alone they still were). 7,744 shaders from the PAK's SDF definitions and main; 7,739 build. The other 5 are compute shaders with atomics on a 3D image, which SPIRV-Cross indexes with an int3, as Octopath Traveler 0's 16; none was requested on the routes below. PR #138 (Nishikori-Yui) built them by flattening the coordinate in dmsh_build.py, a chain change that would rebuild every game's libraries, so it is left out until a run requests one (the game note). PR #138's profile is the same as this one (the same keys and shader tables).

Host fix: the memory line's "freed but alive" count only grew in most games (Animal Crossing: New Horizons 3,113 after 27,900 frames, Pokémon Sword 2,050, Super Smash Bros. Ultimate 1,279; Ghost Trick 125 to 146 buffers, 6.6 to 8.7 GB of zero-copy buffers, so GPU allocated read 9.9 to 11.7 GB against 2.2 GB resident). A temporary census of every host table that can hold a texture over a pool buffer found no holder: mtl_pool_del runs on the guest thread that finalizes the pool, whose implicit autorelease pool never drains, and adding the buffer to the weak table g_dead_pools left it autoreleased there. containsObject: / removeObject: on a weak NSHashTable do the same (a standalone test), and res_add / res_remove call them on guest threads too. mtl_pool_del, res_add_at and res_remove now have their own @autoreleasepool. Ghost Trick: 0 freed but alive, GPU allocated 1.7 GB instead of 8.3 GB 300 frames into the boot and 2.9 GB in Chapter 1, resident memory, frame times and pictures unchanged; Pokémon Sword: 8 to 0 at the title after 90 s, the same frame pacing.

On the Mac: the opening movies, the language screen, the first-boot notices, save creation, the Bonus Content notices and Chapter 1's opening in the junkyard (the Ghost World, the first trick), and a second boot that loads the save, at 60 fps (frame interval p99 about 17 to 19 ms), about 2.3 GB resident, no red line; the Wwise sound plays (a muted run with DM_AUDIO_DUMP). Checked: the tool tests (57, the docs check among them), t_nvn (124 checks), a boot of the rebased host. README (games gallery, a Mac picture of the junkyard), AGENTS.md's count (sixty-two: the gallery's cells), the game note, the games table, docs/reference/nvn-objects.md and the symptom index say so
2026-10-25 17:44:41 +08:00
seedds 16f4889192 OCTOPATH TRAVELER II (0100A3501946E000, update 1.1.1 = v131072, Unreal Engine 4.27 on SDK 12.3.2): game profile, inline Unreal shaders in version 11 paks, and five general host fixes. Plays on the iPhone (IPA 5.9 GB, user; that IPA predates the geometry and allocator changes below). On the Mac: the boot, the title, a new game, the traveler selection, Hikari's opening into his first battle (won) and the scene after it, and Castti's opening (the flashback, the ship, free play on the deck), at 60 fps (frame time p50 16.7 ms, p95 about 17 ms) with CRIWARE sound, about 2.8 GB resident. Shaders: the project ships no shader code archives (bShareMaterialShaderCode off), so each material's shaders are inline in its .uexp and the global ones in Engine/GlobalShaderCache-SF_GLSL_SWITCH.bin, each an entry of FShaderMapResourceCode (an LZ4 block, its raw size, its frequency): uepak.py inline_shaders sends a version 11 pak with cooked packages and no .ushaderbytecode to shader_split.py's inline reader (ue_legacy_jobs; Zlib and LZ4 pak blocks decoded in Python, Pak.read), ue_inline_shaders finds each LZ4 block by its first literals (LSLG), and profile_guess.py / the NSP script tell the two kinds apart; 13,980 shaders in 13 s. Profile: Unreal's FMallocBinned3 cut with an OBS_UE_MALLOC gate as DRAGON QUEST VII Reimagined's (GCreateMalloc main+0xbecbf0, a 20-slot vtable, hle_uemalloc.c), a 2 GB engine region (game_args) with the node-area cut on this SDK 12.3.2 build (sdk_patches.py, sdk+0x346190 / +0x34622c), fps_target 60 through t.MaxFPS / rhi.SyncInterval (hle_fpslock.c; the same route at 30 and 60 fps reaches the same scenes at the same times), pad_layout full (the battle menu picks weapons with the D-pad), a prewarm list of 843 lines. General fixes: (1) a shader may bind 96 resources instead of 64 (NVN_REFL_MAXB: the traveler selection's 3D map shader binds 66, and the map was black); (2) the parallel precompile look-ahead (pso_predict_rest) restores the scissor, which state_op keeps in R, not S: with pipelines built on demand the picture went bright green and white from Castti's flashback and Hikari's battlefield on (bisected with simultaneous runs; DM_NVN_PREDICT=0 turns the look-ahead off); (3) a geometry draw above the 256 MiB capture scratch bound is drawn in runs instead of skipped (nvn_geometry.inc geo_split: one instance at a time, then runs of at most 4,096 list primitives whose index range fits 64 MB, one read of the indices, one GPU wait), the index compaction after the geometry kernel runs as count / offset / write dispatches instead of one thread (each frame drawing Unreal's 129,032-triangle point light shadow took about 300 ms with it), and scratch arenas above 32 MB are released after 600 idle submissions (arena_new takes the smallest that fits); (4) MapMemory takes a thread stack from the host's Unreal heap (dm_ue_heap_owns: a range outside the guest block the host maps read-write, vm_region_64): with cae5ec2's stricter source check every Unreal game on the host heap aborted in MapAliasStack at nnMain (checked: this game and Pikmin 4 boot); (5) the host Makefile rebuilds on a change to dm/gfx/nvn/*.inc. Scripted input takes partly tilted sticks (L<x>/<y>): the traveler selection opens a card only while its quill cursor rests on a traveler, and full-tilt presses swept past them. Open: 2 of 87 Mac boots crashed at the title in the game's CRI cue sheet loader (main+0x1d8054, a record copied while one of its TArrays held count 1 and no data); the writer is not found yet (the game note). Checked: 57 tool tests, the gfx run list (t_geometry 95 checks) and the docs check pass; the NSP script packs the IPA from the NSPs (Mac and iOS). README (games gallery, a Mac picture of Hikari's first battle), the game note, the games table, docs/reference shaders / memory, docs/agents/04 and the symptom index say so 2026-10-25 15:43:48 +08:00
seedds 50ad443cb1 Bayonetta and Bayonetta 2 in the new layout: their game notes (dm/games/010076F0049A2000/README.md, dm/games/01007960049A0000/README.md) and rows in the games table, from their Game notes entries in docs/development.md, their AGENTS.md sentences, their profiles' comments and their commits; the SDK 3.6.0 entry of 03's catalogue becomes services.md "SDK 3.6.0 answered as SDK 4", and 04's two Seen-in sections become shaders.md "BNSH files in PlatinumGames DAT archives inside CRI CPK archives" (cpk.py, cpk_jobs) and rendering.md "Clears of attachments larger than the render pass" (rp_begin_ex_) and "Downsample of a single-sample texture" (copy_t2t), each with the games it was found on. Mario Kart 8 Deluxe's note no longer calls SDK 4.4.0 the oldest. Checked: every profile with a game.tsv has a note and a table row (60), every relative link and anchor in the repository's Markdown resolves, and every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 on main (4,669 items) still appears (the same five file patterns as before) 2026-10-25 12:44:03 +08:00
seedds 3a993fa3fe Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in <game>" subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games/<id>/README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv) 2026-10-25 12:31:58 +08:00