Apply PR #202's host-thread skip and duplicate vsync suppression. Give
bindings unique lifetime IDs so a retired worker cannot signal another
service's event after guest storage and kernel handles are reused. Replace
the eight-entry worker table with allocated records so rapid replacement
does not abort while retired workers wait for their next tick.
Add deterministic regression cases through real event binding and SVCs,
and document the mechanism with an updated symptom index. Both the
capacity abort and stale signal reproduce against the original PR.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Decode packaged byte 0 as separate block-height and block-depth nibbles,
including explicit zero dimensions, and size linear compressed textures
by pixel rows. Port the driver-derived rules from PR #201 and preserve
its attribution to digitalhivehub's PR #11 measurements.
Integrate the layout test with current main, correct its BC4 label, add
independently addressed CPU markers and packaged GPU mip coverage, and
record the mechanisms in the reference docs and symptom index.
Validation: 9 driver shapes, 189 CPU texture checks (also ASan/UBSan),
17 GPU deswizzle cases, 154 NVN checks, documentation checks, Mac host
build, and iOS compilation of 133 sources pass. No new gameplay or
iPhone runtime validation.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Carry ResourceAlias visibility on copy encoders containing buffer copies,
inline updates or fills, using the contribution from PR #200. Operations
through sparse virtual mappings also need ordering against surrounding
operations: disjoint guest addresses can name the same physical bytes,
so guest-range dependency detection alone misses their hazards.
Add a regression using distinct no-copy Metal buffers and real sparse
mappings. The original host failed 37 of 101 checks on an M4 Pro and
PR #200 alone left 24 failures; the complete fix passes all 101 checks
with both direct and deferred encoding under Metal validation. Ordinary
independent buffer operations retain range-based batching.
Validation: t_nvn passes 154 checks; the Mac host builds; all 133 iOS
source compile checks and the documentation checks pass. Monster Hunter
Rise gameplay, iPhone execution and sparse-barrier performance were not
retested.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Apple GPUs truncate Depth16Unorm load-clears while rounding fragment depth,
so clearing to 0.5 stores 32767 instead of the fragment's 32768 and fails
an EQUAL comparison. Apply the format-specific correction to both
attachment and whole-texture clears, preserving other depth formats.
Exercise both production command paths against independent GPU fragment
writes for 1,042 float depths, including half-bin boundaries and adjacent
floats. Removing either correction produces 518 mismatches on M4 Pro;
with both corrections enabled, neither path differs. Record the mechanism
and the pending TOTK highlight and iPhone validation in the documentation.
Validated on M4 Pro with Metal validation: t_depth16_clear, t_clear_ds
(43 checks), t_clear_tex (49 checks), and the documentation tests pass.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Use guest runnable state after a higher-priority same-core wake instead of Mach dispatch state or a 5 ms cutoff. Honour wake handoffs for every game and remove Metroid Dread request-ring gates and its core_preempt setting.
Validate event, mutex and condition-variable handoffs with delayed host dispatch; 13 valid Dread movie-transition runs passed. Host tests, docs and shader-chain checks passed, along with an iOS syntax check. Record Wonder boot comparison and pending iOS runtime validation.
Switch active-app presents to drawable acquisition after 500 ms of display-link
silence, then retry with exponential backoff capped at 60 seconds. Preserve the
requested pacing mode, synchronize rescue state, and prevent focus timestamp
underflow from bypassing the foreground grace period.
Add CPU regression coverage in CI, a real Metal layer ownership check, and the
mechanism reference with upstream provenance and validation limits.
Validated: 43 CPU rescue checks; 47 checks with real Metal drawable acquisition;
34 present/crop and 10 pacing checks; Wonder Mac host build; all 133 iOS source
compile checks; documentation tests. Windowed gameplay recovery and iPhone
validation remain outstanding.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Take upload stamps before protection, initialize cached checks from those
stamps, and wait out pages another thread is unprotecting before retrying.
Publish page generations before advancing the clock again in both the fault
handler and batched host pre-write path. A shared stamp_page helper closes
the remaining batched publication race in the original PR.
Add deterministic publication regressions for both paths and document the
mechanism and the limits of the game-level evidence. The regressions fail
on main, and the batched case also fails with the original PR applied.
Validation: full graphics suite, Wonder Mac host build, all 133 iOS source
compile checks, and documentation tests pass. No fresh gameplay or iPhone
runtime validation.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Drop unused normal-pass colour-target blend and mask state from pipeline keys.
Normalize historical prewarm rows and deduplicate their parsed keys before
compilation, preserving merged passes' logical target state.
Include production-path GPU regression coverage and the reference/game notes.
Validated on the latest main: Wonder host build, key serialization tests,
48 pending-pipeline checks, 48 archive checks, 158 prewarm GPU checks across
MTLDevice and Metal 4, and documentation checks. The regression fails on the
original base and detects wrong pixels if the merged-pass guard is removed.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Exercise historical manifest rows through the production warmer, shader loader,
pipeline builders and draw-time cache. Check normal and merged passes on both
MTLDevice and Metal 4, with GPU pixels compared against the original descriptors.
The regression fails on the PR base and when the merged-pass guard is removed.
Document the shared normalization rule and keep game measurements in their notes.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
- profile 01007BB017812000 (1.0.3, SDK 13.3.0): plays on the Mac at 60 fps
- provisional profiles add the subsdks the NROs import (profile_guess nro_modules)
- an NRO importing nvnBootstrapLoader makes an NVN game; NROs calling glslcCompile get the run-time GLSLC capture
- shader_split reads zip archive members (Portal's nxcontent/nvsb GLSLC outputs)
- ro::LookupModuleSymbol for a name a loaded module lacks is not red (Horizon fails it too)
- AGENTS.md: a Mac gameplay test is enough to count a game as playable (65)
Keep the latest upload-staging and compact-binding fixes from main. Add a real-Metal readback lifetime regression and reference documentation for all three general fixes.
Validation: nine Metal test runs pass, including direct and deferred upload-budget tests; Wonder host build, 133-file iOS compile check, and documentation checks pass.
Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
Split GPU upload batches at 128 MiB and retire overflow and CPU staging when the upload completes, rather than retaining it until the main rendering fence. Scope weak-reference diagnostics to avoid temporary staging retention.
Add a real-Metal regression covering memory bounds, upload pixels and an open render pass in direct and deferred encoding modes. Both modes pass 261 checks; documentation checks and the TotK host build pass.
Refs #147. Reporter-device validation and the full-device reboot investigation remain open.
Apply the generalized image-atomic bounds and backing checks from PR #178.
Also reject unsupported descriptor formats before integer fallback and fail
unresolved bindless atomics instead of silently deleting them. Rebuild the
translator and record chain 467efdfd0b62, with regression tests and findings.
Validation: compiler/helper and sanitizer tests, 96 Metal compile/link checks,
ten GPU probes, all 28,603 Z-A shaders for Mac and iPhone, and a 240-second
Mac station-area run reaching 6,987 frames without a freeze or GPU fault.
Reported iPhone cases and the station-to-world transition remain unverified.
Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
docs/development.md: a new section "A Mac app" after "A Mac-only build for development": what --platform mac-app builds and where the app goes (-o), the bundle's layout, APFS clones or copies, where the save, pso_user.tsv, temporary files, env.txt and the log go, signal forwarding and the alert, why the launcher runs the host as a child process (macOS privacy checks on removable volumes: a launcher that exec'ed dmhost got EPERM on every open on the external drive, signed or not), and signing (ad-hoc or --mac-sign, never the hardened runtime, so no notarization). "A Mac-only build for development" no longer says there is no Mac app. The script's examples get one for --platform mac-app --mac-sign, and a line lists the three --platform values. README.md "Pack an IPA": one line pointing at the Mac app.
Docs only; no test ran for this commit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the separate ASTRAL CHAIN bring-up notice with a gameplay screenshot and notes. Record that passing Mac gameplay testing is sufficient for gallery inclusion, with validation scope kept in each game note.
Support PKZ v1 archives and advertise the translator's GPU-code range independently of SDK version. Register the 1.0.1 profile, shader tables, regression checks and Mac bring-up findings. Retain 30 fps; document the unshipped 60 fps experiment and outstanding iPhone validation.
Integrate the present snapshots and frame-step work from #87 together with
the encode-thread support and wait ledger from #157. Keep snapshots off
by default on both platforms pending wider device validation.
Offer Frame buffering in the iOS launcher for NVN builds. The existing
launcher remembers the choice and preserves env.txt overrides. Pass the
packer's detected graphics API to the settings generator.
Wait for snapshot capacity at acquire; a present reaching a busy slot
falls back to its original texture ticket. Drain snapshot commits and GPU
completion during suspend even when display tickets were discarded.
Regression tests cover both waits, fallback tickets and copied pixels.
Validated on M4 Pro: full gfx run list, 38 snapshot lifecycle checks,
219 checks with encoding and snapshots enabled, Mac host build, 133-file
iOS compile check, and eight launcher/docs/packing tests. The host unit
suite cannot link t_heap_size because dm_ue_heap_owns is missing, reproduced
on unchanged main cbbb600. No iPhone runtime performance claim.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
The profile is the NSP script's provisional one with tex_flush_upload and tex_dirty_upload: RE Engine writes its UI textures (ASTC) into CPU-visible pools in place, as Monster Hunter Rise does, and without both keys the language screen's background and buttons were magenta (with tex_flush_upload alone they still were). 7,744 shaders from the PAK's SDF definitions and main; 7,739 build. The other 5 are compute shaders with atomics on a 3D image, which SPIRV-Cross indexes with an int3, as Octopath Traveler 0's 16; none was requested on the routes below. PR #138 (Nishikori-Yui) built them by flattening the coordinate in dmsh_build.py, a chain change that would rebuild every game's libraries, so it is left out until a run requests one (the game note). PR #138's profile is the same as this one (the same keys and shader tables).
Host fix: the memory line's "freed but alive" count only grew in most games (Animal Crossing: New Horizons 3,113 after 27,900 frames, Pokémon Sword 2,050, Super Smash Bros. Ultimate 1,279; Ghost Trick 125 to 146 buffers, 6.6 to 8.7 GB of zero-copy buffers, so GPU allocated read 9.9 to 11.7 GB against 2.2 GB resident). A temporary census of every host table that can hold a texture over a pool buffer found no holder: mtl_pool_del runs on the guest thread that finalizes the pool, whose implicit autorelease pool never drains, and adding the buffer to the weak table g_dead_pools left it autoreleased there. containsObject: / removeObject: on a weak NSHashTable do the same (a standalone test), and res_add / res_remove call them on guest threads too. mtl_pool_del, res_add_at and res_remove now have their own @autoreleasepool. Ghost Trick: 0 freed but alive, GPU allocated 1.7 GB instead of 8.3 GB 300 frames into the boot and 2.9 GB in Chapter 1, resident memory, frame times and pictures unchanged; Pokémon Sword: 8 to 0 at the title after 90 s, the same frame pacing.
On the Mac: the opening movies, the language screen, the first-boot notices, save creation, the Bonus Content notices and Chapter 1's opening in the junkyard (the Ghost World, the first trick), and a second boot that loads the save, at 60 fps (frame interval p99 about 17 to 19 ms), about 2.3 GB resident, no red line; the Wwise sound plays (a muted run with DM_AUDIO_DUMP). Checked: the tool tests (57, the docs check among them), t_nvn (124 checks), a boot of the rebased host. README (games gallery, a Mac picture of the junkyard), AGENTS.md's count (sixty-two: the gallery's cells), the game note, the games table, docs/reference/nvn-objects.md and the symptom index say so
The memory-profile host test imports profile_guess, which imported nso_syms and required LZ4 even for plain log parsing. The host CI job has no third-party Python packages, so this broke the host suite after cae5ec2. Move the import into load() and document the standard-library-only heap probe.
Validated the exact failing target and full parallel host suite with system Python, the heap probe with site packages disabled, six related tool tests, and synthetic compressed NSO decoding.
Incorporate the active SVC improvements from PR #151 around the existing
alias bitmap and mapping registry. Serialize heap changes, physical maps,
stack ownership and memory queries with one lock; report committed alias
runs and borrowed sources; initialize reused logical pages while preserving
live neighbors and lazy first exposure. Retain PR #127's fail-fast host
protection policy and the log formats parsed by the NSP heap probe.
Game smoke tests exposed two integration requirements: Raging Loop's SDK
extends committed windows across borrowed stack sources, so allow recommit
that preserves existing bytes; Tears of the Kingdom's audio helper needs
guest-backed stacks, so allocate host-created guest-thread stacks through
SDK malloc and free them after thread destruction and copyback.
Keep SDK reservation ownership and existing patches. The unused host
reservation allocator from #151 is not included.
Validation: full host suite; focused UBSan, ASan and ThreadSanitizer tests;
real dirty-tracker concurrency tests; heap-probe parser compatibility; Mac
host builds; iOS arm64 syntax checks; all 60 profile layout checks.
Raging Loop and Tears of the Kingdom passed guest-thread self-checks and
ran to deliberate timeouts after the startup fixes. Their shader caches
are stale for this checkout, so rendering/performance are not validated.
No IPA build or iOS device validation was performed.
Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
Adapt PR #122: compile reusable bases with unspecialized colour state, specialize per draw state, and persist bases between launches. Keep iOS opt-in and the MTLDevice fallback.
Preserve rasterizing depth-only pipelines through the legacy path. Share in-progress base builds, including errors, while unrelated bases compile in parallel. Defer timed flushes during compaction and finish flush/cleanup together; synchronize the pending count with serializer capture.
Checked on Apple M4 Pro: t_pso_archive 48 checks with Metal validation, t_pso_pending 48, t_integer_blend 63, t_geometry 95, and t_pso_skey pass. The Breath of the Wild Mac host builds and nvn_mtl.m passes the iOS arm64 syntax check. Real-game memory usage and iPhone runtime behavior were not measured.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Movie decoder (hle_movie.c, every game), both found on the opening movie (Movie/Cutscene01.mp4, H.264 and AAC through movie::Extractor):
- AAC decoders offer at most 32 outputs (MD_NOUT_AAC; Vorbis keeps 64, video 16). The game's movie player keeps the outputs it holds in a ring of 32 entries per track (main+0xe93d4c: 0x200 bytes) and pushes with no bounds check (main+0xe94140), so with 64 the 33rd AAC output wrote its index (0x3f) and time over the next heap block, and the game crashed in sead::ExpHeap's free a second into the movie, at the same address every boot. The console's AAC decoder cannot offer more than 32, or the game would overflow there too; Monster Train 2 peaked at 30 held.
- An AAC decoder's GetOutputBufferSize is one frame (1024 samples per channel, 2048 with SBR), what each output holds. The player sizes its own AudioOut buffers by it, decodes into them and appends each whole, advancing its audio clock by size / 4 samples (main+0xe92a74); with the 128 KB Vorbis bound every 4 KB AAC frame played as 0.68 s, the clock crawled and the movie froze on its first second. Vorbis keeps the bound (its packets vary in length).
Advanced blend modes (hle_nvn.c, nvn_obj.h, nvn_mtl.m, every game): the blend state keeps the mode, the premultiplied-source flag and the overlap (with their getters), and the pipeline blends the Porter-Duff modes (ZERO, SRC, DST, SRC/DST_OVER, _IN, _OUT, _ATOP, XOR, the PLUS modes) with the fixed-function factors they amount to under uncorrelated overlap (adv_blend); the others keep the state's functions with a red line naming the mode. The game's levels set SRC_OVER on one blend state, which the host used to ignore.
DM_PROBE_POLL takes a value filter (=<value>): every change is logged, the threads are snapshotted and dumped only at changes to that value. That is how the ring overflow was caught: DM_PROBE_WATCH on the heap link made the movie's file reads into the same page fail.
Profile: the provisional profile's guesses held (no system resource, a 3.14 GB heap in the 4 GB layout, modules main subsdk0 sdk), plus render_cond on (the game skips clears under render conditions; without it every level was black behind its HUD). 12,691 shaders (BNSH / BAHS in the zstd RomFS and main), all translated. Pad only (main imports no touch screen), the default 2D layout. A prewarm list recorded on the Mac from a fresh save over the route above (550 lines; with it nothing was built on demand). Checked: the NSP script packs it from the NSPs with the profile in dm/games; the graphics run list and the 49 tool tests pass; after rebasing on 6d34984 (spin waits) the game boots into level 1-1 as before. README (gallery, a Mac picture of level 1-1; 2D layout), AGENTS.md (57 -> 58) and docs/development.md, docs/agents 03 / 04 / 05 say so
