Copy byte-aligned NVN streams into aligned submission-owned storage.
Use CPU snapshots only when host visibility is proven safe; otherwise
encode ordered GPU copies before regular, feedback and geometry draws.
Invalidate copies after overlapping writes and pool remaps while keeping
them reusable across unrelated uniform updates. Restore the encoder after
deferred replay when a stream copy split the pass.
Extend t_vbuf to cover all nonzero address residues, copy reuse, queued
copies, deferred updates, fills, shader writes, previous submissions and
pool replacement. Record the general mechanism and Arkham Knight finding.
The per-stream copy limit remains 64 MiB.
Validated on Apple M4 Pro: 84 GPU checks in each of normal, cache-disabled
and encoder-thread modes with Metal validation; full graphics and host
suites; 70 graphics Python tests; 298 tool tests; documentation tests;
iOS compile check (178 files). Full cross-game validation is incomplete:
the harness reported unsupported OpenGL chain checking and its first NVN
shader rebuild did not finish alongside other active builds. No local
Arkham Knight gameplay run.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Reuse unchanged IPA entries through the packing index and APFS clones,
and keep Mac app and derived-profile shader outputs while their inputs
match. Preserve main's shader regression checks and .NET discovery.
Count uncompressed bytes in streamed files so a size change aborts the
pack without replacing the previous IPA or index. Cover both deflated
and stored streams with growth and shrinkage regression tests, and
record the cause and fix in the packing reference.
Co-authored-by: Suraj Mandal <surajmandalcell@gmail.com>
(cherry picked from commit 09f3778c36)
Derive DOTNET_ROOT from dotnet's Microsoft.NETCore.App runtime listing and
export it to both explicit and inherited build environments. Preserve caller
overrides, use the build environment for discovery, and leave discovery
unchanged on probe failures instead of treating Homebrew's wrapper directory
as a runtime root.
Add regression coverage and document the packing failure and its resolution.
Based on PR #284.
Co-authored-by: Jose Rosendo <josemrb99@gmail.com>
(cherry picked from commit 89744da753)
One translation chain serves every game, so a translator or host change was checked
only on its author's games; on another game the first sign was a crash at run time
(c3a0880 broke the image atomics of SHIN MEGAMI TENSEI V, Yoshi's Crafted World and
FINAL FANTASY RESONANCE).
- CI: the tool-tests job installs .NET 10, spirv-cross and spirv-tools and points
DMSH_TEST_XLATE at the committed translator, so the translator's synthetic-shader
tests (test_image_atomic_bounds, test_dmsh_geometry) run on every change.
test_dmsh_geometry imports its sibling module when run from the repository root.
- dm/gfx/tools/shader_regressions.py: lists the shaders that translated in the
game's previous b-*/ and fail in the current chain's, from dmsh_build.py's build
reports (not part of the chain). The NSP script runs it after its shader step and
stops on a regression unless ALLOW_SHADER_REGRESSIONS / --allow-shader-regressions;
docs/development.md describes it.
- FINAL FANTASY RESONANCE: the c3a0880 regression is resolved since 2e655b7 (image
atomics with a used result): chain 39e4d9585b1d builds 73,488 of 73,496 shaders
(the 8 3D image atomics as before), shader_regressions.py finds 0 against the
earlier build, and a new game from an empty save plays the opening cutscene on
the Mac with no red line; the two shaders that failed and aborted run their
atomics on their buffer textures.
Checked: every CI test file as the job runs it (DMSH_TEST_XLATE set), the docs tests.
(cherry picked from commit b117c91317)
The Mac app packer left out shader_pairs.tsv and pso_pairs.tsv, which the
host's pipeline prebuild reads beside the shader map and build_ipa.sh already
copies into the IPA. Every NVN game's Mac app therefore built no pulling
bases, had no hybrid stand-ins, and compiled first-seen pipelines on the
submit thread. Copy both tables, and the shader split's pso_pairs.tsv for a
profile without one.
An NVN game's Mac app also carries its pipeline archive made offline for this
Mac's GPU (pso_offline.py --platform macos), handed to the host as
DM_PSO_BUNDLE as on iOS; --gpu none leaves it out. Integrated with three
fixes: the archive is validated and counted in the free-space check before
anything is copied; a plan whose pairs all need function constants packs the
app without an archive instead of aborting; and a PSO_GPUS value that does
not name this Mac's GPU is reported.
Validation: test_nsp_mac_app 22 (pair tables, split copy, bad archive, free
space, GPU note, no-archive plan; the table and space tests fail without the
fixes), the tool suite, iOS packaging tests and test_docs. A Breath of the
Wild Mac app on an M4 Pro (applegpu_g16s, 629.4 MB archive): 16 of 16 probed
bases found, no pair base built, no red line; the run stayed in the title.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>
(cherry picked from commit 4639c5114c)
An incomplete startup command list consumed both layout detection and the
one-time DeviceSink scan, leaving valid SDK 4 audio frames permanently
silent. Keep both decisions pending until a supported layout walks the
whole list, as proposed in PR #331.
Extend the reproducer with valid-first and incomplete-startup stereo PCM
checks for SDK 4, 6, 16 and 21, and document the finding.
Co-authored-by: aldomrodriguez <rodriguez.aldo.91@hotmail.com>
From PR #329 (stacked on #328): a game's usage manifest, dm/games/<id>/usage.tsv,
written by its owner with regress.py usage (shader features of its chain build,
and the NVN / HLE functions, SVCs and present path of one smoke-route run;
names and counts only), and regress.py impact <base>..<head>, which maps the
functions around the changed lines through the host's callers and the rules of
dm/tools/impact_map.tsv to manifest rows. Manifests for Breath of the Wild,
Tears of the Kingdom and Hades.
Fixed on top of the PR:
- impact checked the head out as a git worktree under the harness root, so it
stopped (PermissionError on /Volumes/NX) for exactly the contributors it is
for, who have no games and no harness root. The call graph is now read from
the git object store at the head revision, and the report is printed and
written only to --out or an existing harness root.
- A hunk mapped only to the function at its first line: a hunk spanning two
functions named one. Every added line is mapped now, and a pure deletion maps
to the function around it (it was moved one line up).
- Functions defined in headers were not in the call graph: the frame step
(nvn_fstep.h, static inline, called from nvnQueuePresentTexture) reached no
entry point, so 4acde72 named no game. Headers are scanned, and their
functions are visible to every file. 4acde72 now names both Zeldas through
nvnQueuePresentTexture.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Give DM_KEYS and DM_CTL their own buttons, press latch and stick axes,
merged with player 1's keyboard/touch and physical controller state.
Controller updates no longer replace scripted holds, and script release
preserves physical input. Include the scripted path in shake detection.
Exercise the real scripted publisher with controller updates during its
hold, including repeated reads after the press latch is consumed. Cover
release, short presses, handheld axes and player isolation, and update the
input documentation.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Add Monolith archive and shader extraction, a build-gated compact memory layout, vertex storage writes under rasterizer discard, primitive ordering and interleaved geometry feedback. Correct native decoder/audio wait accounting and the oe calls used after movies.
Validated opening combat and movement on Mac with real audio, the full Mac/iOS shader build, GPU regressions, and the unsigned IPA archive. iOS runtime, later areas and DLC gameplay remain untested.
Compare revision shader builds and pipeline interfaces across local games,
and run available smoke routes with crash, hang, timing and image checks.
Add the developer loop and share the measuring lock with bench.py.
Keep all local games in static checks and honor --no-smoke. Include shader
bytes and fallback pairs in cache keys, recheck interfaces on reuse, and
reuse smoke baselines only for matching inputs and run settings. Detect
failed prebuilds, missing or intermittently bad screenshots, and Vulkan
build failures even when no MSL was emitted. Keep game-derived files and
writable profile copies inside each game's NSP work folder.
Validation: tool tests and the lint ratchet pass. Full shader rebuilds and
gameplay comparisons remain unverified; the 26 discovered Mac builds have
no smoke routes in this checkout.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Back compatible GPU-only image families at one NVN pool offset with exact-sized placement heaps. Keep descriptor dimensions, initialize fresh storage, register parent-heap residency and retire backing with its textures. Preserve ordinary allocations for single-description storage and exclude unsupported storage classes.
Add synthetic allocation, pixel, indirect-read, depth/stencil and lifetime regressions. Record the LM3 live-allocation census, measured Mac memory reduction and remaining validation limits. Refs #257 and #100.
Register GL and GLB entries in the shared HLE import table so direct imports and eglGetProcAddress reach the same implementations, retaining big-stack wrappers. Add regression and CI coverage for all 156 GL entries and document issue #228.
Display mode at 120 Hz previously waited two display-link ticks per frame
and stayed at 60 fps. Count its acquire waits in the link's own ticks so
120 fps takes one tick. Carry fractional ticks across frames so requests
such as 90 fps on a 120 Hz link wait four ticks per three frames.
Keep game-interval and frame-step pacing in 60 Hz vsync units. Extend
t_link_ticks through the production acquire path, add it to CPU-only CI,
and document the behavior in the frame-rate reference and symptom index.
Validation: 20 link-tick checks, 154 NVN integration checks, pacing and
frame-step tests, Metal snapshot and display-link rescue tests, Mac host
build, iOS source compilation, and documentation checks passed. Physical
display gameplay and iPhone gameplay were not retested.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Below MSL 3.1, SPIRV-Cross emulates an image atomic with a raw device pointer
beside the texture, laid out like a linear image. A block-linear image's Metal
texture is a deswizzled copy, so its atomics went to a scratch buffer nothing
read (Pokemon Legends: Z-A's atomic max into texture 302), and a 3D image's
pointer was indexed with the int3 coordinate itself, which does not compile
(16 shaders of Octopath Traveler 0, 5 of Ghost Trick, 8 of FINAL FANTASY
RESONANCE).
- dmsh_atomic.py: shaders with image atomics (OpImageTexelPointer) are written
at MSL 3.1, so the atomics run on the image's own texture; every other
shader keeps its MSL 2.3 source byte for byte. MSL that still holds the
pointer form fails the build.
- Translator (TranslationFlags.MetalImageAtomics, renamed from
Msl23ImageAtomics): admits 3D images and SUATOM.CAS (Metal's texture
compare-exchange); the IGN coordinate guard is unchanged.
- Host: the IMGA pointer binding, the scratch buffer and the layout admission
are gone. NVN images of R32Uint / R32Sint take MTLTextureUsageShaderAtomic
(img_usage, kept by tex_upgrade: Metal leaves that bit out of the usage
property); any other texture in an atomic slot is replaced by an
atomic-capable write sink with a red line.
- Chain a4e2088c2e69 (with main's vertex-output padding). Tested on
chain 0ce4b5e8aefb, this change before that rebase: Octopath Traveler 0
builds all 10,324 shaders, Z-A all
28,603, Ghost Trick's five 3D-atomic shaders build. Mac runs of both games
under the Metal API validation layer showed no validation error (Octopath
Traveler 0 to frame 14,400, its particle atomics on buffer texture 2049;
Z-A to frame 12,000 through the atomic max on texture 302). Synthetic
atomics and CAS run on buffer, 2D, 2D-array and 3D textures of a Mac GPU
(test_image_atomic_bounds.py, t_atomic_return.m).
GitHub issue #290: at 40b643b Octopath Traveler 0 already built 10,308 of
10,324 shaders (2e655b7 fixed the returning atomics); the 16 left were the 3D
ones, which build now. Not checked: an iPhone, FINAL FANTASY RESONANCE's
library, the picture of Z-A's atomic scene.
Add pinned ruff and pyright checks, the existing-findings baseline, and a
workflow that rejects new findings and requires fixed findings to be removed.
Compare checker pins numerically and permit baseline growth only for the
checker whose release version increased. Comments, unrelated dependency
edits and downgrades retain the guard. Add 10 regression tests using Git
trees and document the policy.
Validation: 14 guard/documentation tests pass; pinned lint reports 414
existing findings, zero new findings and zero stale baseline entries.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
HAL Laboratory's engine on NintendoWare, SDK 11.4.0, NVN. Plays on the Mac at 30 fps, as on
the Switch: online and difficulty prompts, the 1080p opening movie, Point of Arrival with its
guide movies, the city reveal, the Sword ability and fights, title and Continue; a 15-minute
soak with no crash. No game code is cut; the profile is the memory layout and the shader groups.
All 27,297 shaders build for Mac and iOS; the IPA (6.4 GB) is untested on a device.
- Threads (general): a guest thread waiting for a host lock that HLE code holds while it runs
guest code is blocked, not runnable (dm_host_lock, a native HLE wait). The movie decoder held
its decode lock across SignalEvent; the signal's same-core handoff waited for the movie input
thread, which waited for that lock, and the game froze at its first guide movie. t_preempt
has the case.
Docs: the game note, games table, README gallery, threads.md, shaders.md, symptom index;
AGENTS count 73 -> 74. Checked: host tests, preempt, tool tests (253, 15 skips), docs tests;
a Mac run on chain 5cddc47253ab through the first guide movie.
Read both shader pair tables, including the generated BAHS fallback, for
padding, build order and shader cache dependencies. Include the optional
profile table in the input stamp and the generated table in step reuse;
regenerate a missing fallback before reusing the shader split.
Add native Metal and cache/order regressions, update the shared chain,
and document full validation on TotK and Odyssey. All 301 affected pairs
compile for g13s and g16s, both full Mac shader/host builds pass, all
57,039 pair interfaces match, and TotK's 24,153-base archive succeeds.
The separate Odyssey Max Unstruct ID compiler limit remains open.
Fixes#283
Refs #276, #298
Co-authored-by: marco <6410852+marcolaux@users.noreply.github.com>
DM_FRAME_STEP was removed in 79d82f7b, but frame-rate.md and Breath of the
Wild's note still said DM_FRAME_STEP=legacy ran the old interval-controller
method. Both now say it was removed and that the controller cuts stay, for
the interval the framework keeps while the game's VFR override holds.
rendering.md names c76f6798, which removed DM_NVN_ALPHA_RED.
Every doc mention of a name no code reads now says it was removed.
Validation: test_docs and test_env_switches pass.
Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
Add docs/env_switches.tsv, one row per switch with its class (player, path,
switch, diagnostic, test) and a one-line meaning: 257 switches, 70 of which
no doc mentioned before. dm/tools/env_switches.py writes it from the sources
and keeps the hand-written class and meaning; test_env_switches.py fails when
the code reads a switch the table does not list, when a row names a switch
nothing reads any more, or when a row has no meaning. It also prints the
names the docs quote that nothing reads.
The check compares only the set of names, so mentioning a switch in a doc,
moving a file or reading a switch in a second place does not fail it. A
macro name stays listed when the code also reads it from the environment
(DM_GAME_ENV overrides the profile's macro of that name).
Remove DM_MOVIE: it set g_flag.movie, which nothing read.
Validation: test_env_switches and test_docs pass; a doc mention of an
undocumented switch passes the check and an unlisted getenv fails it; the
Mac host builds for Mario's profile.
Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
Remove repetitive idle messages, permanent footers, the guide link and normal-run technical logs. Show a compact preparation form, contextual recovery instructions and a ready-to-install result. Send the prepared IPA directly to Sideloadly without signing or starting device installation. Simplify the catalog and fix missing Xcode detection.
Always use the validated normal build pipeline, which reuses unchanged stages. Keep app-only rebuilding as a developer CLI option and reduce the GUI disclosure to diagnostics and help.
Read all checkout game profiles, search names, versions and title IDs without case or accent sensitivity, and link each result to its game notes. Localize the catalog in English and Spanish and verify discovery against the current profiles.
Persist the interface language, translate app-owned labels and dialogs immediately, preserve raw build diagnostics, and replace unverified device and account claims with precise preparation and compatibility wording. Add English and Spanish READMEs and localization regression coverage.
Replace desktop artwork with generic SF Symbols, use explicit ad-hoc Mac signing, document provenance and project scope, and record pre-publication checks. Keep repository visibility private pending the owner decision on documentation artwork.
Provide folder selection, build progress, process-group cancellation, and IPA/RomFS/Sideloadly buttons. Preserve command documentation separately and correct the personal author policy.
Use the edited checkout for every game, persist identity, verify archives, preserve existing outputs, and reject concurrent builds. Include applicability triage for 57 open upstream PRs and 20 commits after the stable base.
Apply PR #262's device-buffer fix for clicks caused by audio underruns.
Reconfigure on both output-device and hardware sample-rate changes, respect
the device's frame range, and read back the actual buffer size. Avoid
rewriting unchanged sizes when buffer changes notify the format listener.
Remove listeners before teardown, including failed initialization, so they
cannot configure a device that the AudioUnit has already released.
Add hardware-independent regression coverage for switching, rate changes,
scope filtering, limits, synchronous notifications, diagnostic overrides,
absent devices, and listener cleanup. Record the original gameplay evidence
and the additional silent Core Audio validation in the audio reference.
Validation: full host tests, Mac host build, docs checks, and the iOS source
compile check (178 files) passed. Silent USB/speaker switching and speaker
rates of 44.1/96/48 kHz verified 221/480/240 frames. HDMI switching and
gameplay listening after these listener changes remain unvalidated.
Co-authored-by: parsifal295 <130349765+parsifal295@users.noreply.github.com>
Use only the remaining finite timeout after the fast-event wait instead of
starting a second full wait for feedback. Preserve polling, CPU completion
ordering and the existing infinite-wait stage caps.
Add deterministic regressions against the production wait function across
all three fast-wake sites and the four fast-wake/wait-ledger combinations.
Co-authored-by: smartLanny <51014844+smartLanny@users.noreply.github.com>
Reuse unchanged IPA entries through the packing index and APFS clones,
and keep Mac app and derived-profile shader outputs while their inputs
match. Preserve main's shader regression checks and .NET discovery.
Count uncompressed bytes in streamed files so a size change aborts the
pack without replacing the previous IPA or index. Cover both deflated
and stored streams with growth and shrinkage regression tests, and
record the cause and fix in the packing reference.
Co-authored-by: Suraj Mandal <surajmandalcell@gmail.com>
Derive DOTNET_ROOT from dotnet's Microsoft.NETCore.App runtime listing and
export it to both explicit and inherited build environments. Preserve caller
overrides, use the build environment for discovery, and leave discovery
unchanged on probe failures instead of treating Homebrew's wrapper directory
as a runtime root.
Add regression coverage and document the packing failure and its resolution.
Based on PR #284.
Co-authored-by: Jose Rosendo <josemrb99@gmail.com>
One translation chain serves every game, so a translator or host change was checked
only on its author's games; on another game the first sign was a crash at run time
(c3a0880 broke the image atomics of SHIN MEGAMI TENSEI V, Yoshi's Crafted World and
FINAL FANTASY RESONANCE).
- CI: the tool-tests job installs .NET 10, spirv-cross and spirv-tools and points
DMSH_TEST_XLATE at the committed translator, so the translator's synthetic-shader
tests (test_image_atomic_bounds, test_dmsh_geometry) run on every change.
test_dmsh_geometry imports its sibling module when run from the repository root.
- dm/gfx/tools/shader_regressions.py: lists the shaders that translated in the
game's previous b-*/ and fail in the current chain's, from dmsh_build.py's build
reports (not part of the chain). The NSP script runs it after its shader step and
stops on a regression unless ALLOW_SHADER_REGRESSIONS / --allow-shader-regressions;
docs/development.md describes it.
- FINAL FANTASY RESONANCE: the c3a0880 regression is resolved since 2e655b7 (image
atomics with a used result): chain 39e4d9585b1d builds 73,488 of 73,496 shaders
(the 8 3D image atomics as before), shader_regressions.py finds 0 against the
earlier build, and a new game from an empty save plays the opening cutscene on
the Mac with no red line; the two shaders that failed and aborted run their
atomics on their buffer textures.
Checked: every CI test file as the job runs it (DMSH_TEST_XLATE set), the docs tests.
Two general causes of FINAL FANTASY RESONANCE's missing fires, smoke and muzzle
flashes, found and first fixed by b1tdr3amer24 (GitHub issue #168).
- Translator (vendored BindlessElimination.cs, TryTableHandle): a texture read
whose handle phi mixes c2 handle words with values that are not handles
(undefined, or float values the compiler kept in the register on paths where
the predicated read does not run) was removed and its result set to 0:
58,122 reads in FINAL FANTASY RESONANCE, mostly Niagara particle vertex
factories, whose meshes collapsed to a point. The read now takes the c2
handle among the inputs, or reads each candidate and picks by the phi's run-
time value; candidates read elsewhere as another texture shape are left out
(a second sweep, ResourceManager.GetTextureTypes). Only phis that
GenerateBindlessAccess refuses come here, so tex_dyn_slots arrays keep their
path: the other 69,449 shaders' SPIR-V is unchanged. Several candidates are
never used for an image store or atomic. dmsh_xlate's Log now reaches the
notes as "translator: ..." and the build summary counts those apart.
- Host (nvn_mtl.m, tbuf_shadow): a texture buffer view at an offset that is
not 16-byte aligned reads from the boundary below on an M4 Pro (t_tbuf_align:
56 of 56 views wrong; the M6 measurement in the old comment does not hold).
Such views are read through a per-submission aligned copy; aligned views are
untouched. A workaround for the Metal limit, its gaps (GPU writes in the same
submission, CPU_NO_ACCESS pools, image writes) recorded as open.
Translator rebuilt; lock and chain header updated. Checked: all FFR shaders
translated before and after; FFR on the Mac (fires and smoke column render,
steady; with the copy off they flicker); gfx, chain and docs tests.
Co-authored-by: b1tdr3amer24 <240231594+b1tdr3amer24@users.noreply.github.com>
Keep save failure state and write timing with the canonical backing save, and
retain that identity on open files and mount aliases. Coordinate commits with
in-flight writes and file-size changes, preserve taint when recovery fails,
and mark the affected saves after a failed rename rollback.
Initialize shared write-injection settings with pthread_once so simultaneous
first writes do not race. Expand regressions for independent saves, aliases,
overlapping I/O and commits, failed recovery, and concurrent first writes.
Validation: 98 save regression checks, filesystem ThreadSanitizer, the full
host unit suite, and documentation checks pass.
Refs: https://github.com/seedds/nxnative/pull/288
Co-authored-by: Nishikori-Yui <165007678+Nishikori-Yui@users.noreply.github.com>
Publish native HLE condition waits to the guest scheduler and end inherited
same-core handoffs at the blocking boundary. Restore runnable state on wake.
Exercise real thread creation, startup and exit in the notification regression,
including spurious wakes and both scheduling modes. Record the real-game Mac
startup comparison and the remaining visual gameplay and iPhone validation.
Co-authored-by: huajiaoshu520 <huajiaoshu@jasonml.eu.org>
Profile 0100C1F0051B6000 (SDK 4.4.0, Retro Studios' engine, NVN): plays on the Mac from boot
through the menus and the intro cinematic into the first level at 60 fps. General changes:
- svc SetHeapSize: a request beyond the heap region returns ResultOutOfMemory, as Horizon
(the game binary-searches its heap size between 3 and 6 GB)
- shader_split.py: SNVN program records in Retro's RFRM forms (MaterialArchive.arc, the packs'
MTRL resources) and the LZSS-compressed startup material archive in main
- nvn_mtl.m: a texture in a CPU_UNCACHED pool starts with its storage at first use (title logo,
menu panels); transform feedback captures draws that rasterize too
- dmsh_build.py: transform feedback buffers holding several outputs (DK's fur simulation);
chain header regenerated
- hle_audren.c: the renderer event is the kernel event the audio service gives, not the SDK's
local one (the main thread was starved at the first level's load)
- hle_oe.c: oe::GetUserInactivityDetectionTimeExtensionMode
Erick Sanchez