At -O2, CorrelatedValuePropagation and SimplifyCFG fold a translated
shader's repeated tests of one value into switches whose cases jump past
their own join. Apple's GPU backend then runs out of control-flow IDs
("Max Unstruct ID 758 ... exceeds supported limit", a note: metal-tt
still exits 0) and lowers the function a far slower way. Odyssey's
alRenderMaterialUber took 24.6 minutes and 17.3 MB of applegpu_g13s code;
at -O1 it takes 31 s and 1.3 MB.
Sampled archive bases from seven games compile to code within a few
percent of -O2. Odyssey's full g13s archive builds in 15.6 minutes
(exit 0), its Cap Kingdom frames match -O2 within 3/255 on scattered
pixels, and GPU time per frame is within run-to-run noise.
Document the mechanism, and correct the Odyssey note: the 758 line never
failed the build; the reporter's exit 1 came from the BAHS link errors
fixed in ff44c59.
Refs #276
Copy byte-aligned NVN streams into aligned submission-owned storage.
Use CPU snapshots only when host visibility is proven safe; otherwise
encode ordered GPU copies before regular, feedback and geometry draws.
Invalidate copies after overlapping writes and pool remaps while keeping
them reusable across unrelated uniform updates. Restore the encoder after
deferred replay when a stream copy split the pass.
Extend t_vbuf to cover all nonzero address residues, copy reuse, queued
copies, deferred updates, fills, shader writes, previous submissions and
pool replacement. Record the general mechanism and Arkham Knight finding.
The per-stream copy limit remains 64 MiB.
Validated on Apple M4 Pro: 84 GPU checks in each of normal, cache-disabled
and encoder-thread modes with Metal validation; full graphics and host
suites; 70 graphics Python tests; 298 tool tests; documentation tests;
iOS compile check (178 files). Full cross-game validation is incomplete:
the harness reported unsupported OpenGL chain checking and its first NVN
shader rebuild did not finish alongside other active builds. No local
Arkham Knight gameplay run.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Reuse unchanged IPA entries through the packing index and APFS clones,
and keep Mac app and derived-profile shader outputs while their inputs
match. Preserve main's shader regression checks and .NET discovery.
Count uncompressed bytes in streamed files so a size change aborts the
pack without replacing the previous IPA or index. Cover both deflated
and stored streams with growth and shrinkage regression tests, and
record the cause and fix in the packing reference.
Co-authored-by: Suraj Mandal <surajmandalcell@gmail.com>
(cherry picked from commit 09f3778c36)
Derive DOTNET_ROOT from dotnet's Microsoft.NETCore.App runtime listing and
export it to both explicit and inherited build environments. Preserve caller
overrides, use the build environment for discovery, and leave discovery
unchanged on probe failures instead of treating Homebrew's wrapper directory
as a runtime root.
Add regression coverage and document the packing failure and its resolution.
Based on PR #284.
Co-authored-by: Jose Rosendo <josemrb99@gmail.com>
(cherry picked from commit 89744da753)
One translation chain serves every game, so a translator or host change was checked
only on its author's games; on another game the first sign was a crash at run time
(c3a0880 broke the image atomics of SHIN MEGAMI TENSEI V, Yoshi's Crafted World and
FINAL FANTASY RESONANCE).
- CI: the tool-tests job installs .NET 10, spirv-cross and spirv-tools and points
DMSH_TEST_XLATE at the committed translator, so the translator's synthetic-shader
tests (test_image_atomic_bounds, test_dmsh_geometry) run on every change.
test_dmsh_geometry imports its sibling module when run from the repository root.
- dm/gfx/tools/shader_regressions.py: lists the shaders that translated in the
game's previous b-*/ and fail in the current chain's, from dmsh_build.py's build
reports (not part of the chain). The NSP script runs it after its shader step and
stops on a regression unless ALLOW_SHADER_REGRESSIONS / --allow-shader-regressions;
docs/development.md describes it.
- FINAL FANTASY RESONANCE: the c3a0880 regression is resolved since 2e655b7 (image
atomics with a used result): chain 39e4d9585b1d builds 73,488 of 73,496 shaders
(the 8 3D image atomics as before), shader_regressions.py finds 0 against the
earlier build, and a new game from an empty save plays the opening cutscene on
the Mac with no red line; the two shaders that failed and aborted run their
atomics on their buffer textures.
Checked: every CI test file as the job runs it (DMSH_TEST_XLATE set), the docs tests.
(cherry picked from commit b117c91317)
The Mac app packer left out shader_pairs.tsv and pso_pairs.tsv, which the
host's pipeline prebuild reads beside the shader map and build_ipa.sh already
copies into the IPA. Every NVN game's Mac app therefore built no pulling
bases, had no hybrid stand-ins, and compiled first-seen pipelines on the
submit thread. Copy both tables, and the shader split's pso_pairs.tsv for a
profile without one.
An NVN game's Mac app also carries its pipeline archive made offline for this
Mac's GPU (pso_offline.py --platform macos), handed to the host as
DM_PSO_BUNDLE as on iOS; --gpu none leaves it out. Integrated with three
fixes: the archive is validated and counted in the free-space check before
anything is copied; a plan whose pairs all need function constants packs the
app without an archive instead of aborting; and a PSO_GPUS value that does
not name this Mac's GPU is reported.
Validation: test_nsp_mac_app 22 (pair tables, split copy, bad archive, free
space, GPU note, no-archive plan; the table and space tests fail without the
fixes), the tool suite, iOS packaging tests and test_docs. A Breath of the
Wild Mac app on an M4 Pro (applegpu_g16s, 629.4 MB archive): 16 of 16 probed
bases found, no pair base built, no red line; the run stayed in the title.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>
(cherry picked from commit 4639c5114c)
An incomplete startup command list consumed both layout detection and the
one-time DeviceSink scan, leaving valid SDK 4 audio frames permanently
silent. Keep both decisions pending until a supported layout walks the
whole list, as proposed in PR #331.
Extend the reproducer with valid-first and incomplete-startup stereo PCM
checks for SDK 4, 6, 16 and 21, and document the finding.
Co-authored-by: aldomrodriguez <rodriguez.aldo.91@hotmail.com>
From PR #329 (stacked on #328): a game's usage manifest, dm/games/<id>/usage.tsv,
written by its owner with regress.py usage (shader features of its chain build,
and the NVN / HLE functions, SVCs and present path of one smoke-route run;
names and counts only), and regress.py impact <base>..<head>, which maps the
functions around the changed lines through the host's callers and the rules of
dm/tools/impact_map.tsv to manifest rows. Manifests for Breath of the Wild,
Tears of the Kingdom and Hades.
Fixed on top of the PR:
- impact checked the head out as a git worktree under the harness root, so it
stopped (PermissionError on /Volumes/NX) for exactly the contributors it is
for, who have no games and no harness root. The call graph is now read from
the git object store at the head revision, and the report is printed and
written only to --out or an existing harness root.
- A hunk mapped only to the function at its first line: a hunk spanning two
functions named one. Every added line is mapped now, and a pure deletion maps
to the function around it (it was moved one line up).
- Functions defined in headers were not in the call graph: the frame step
(nvn_fstep.h, static inline, called from nvnQueuePresentTexture) reached no
entry point, so 4acde72 named no game. Headers are scanned, and their
functions are visible to every file. 4acde72 now names both Zeldas through
nvnQueuePresentTexture.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Give DM_KEYS and DM_CTL their own buttons, press latch and stick axes,
merged with player 1's keyboard/touch and physical controller state.
Controller updates no longer replace scripted holds, and script release
preserves physical input. Include the scripted path in shake detection.
Exercise the real scripted publisher with controller updates during its
hold, including repeated reads after the press latch is consumed. Cover
release, short presses, handheld axes and player isolation, and update the
input documentation.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
AudioUnit setup now joins its helper through dm_host_thread_join, which references the graphics host's thread-local counter. Define that counter in the standalone device test, matching the other audio fixtures.
Validated the audio-device test, full host suite, audio/HID/memory ThreadSanitizer checks, display-link tests, OpenGL imports and documentation tests.
Add Monolith archive and shader extraction, a build-gated compact memory layout, vertex storage writes under rasterizer discard, primitive ordering and interleaved geometry feedback. Correct native decoder/audio wait accounting and the oe calls used after movies.
Validated opening combat and movement on Mac with real audio, the full Mac/iOS shader build, GPU regressions, and the unsigned IPA archive. iOS runtime, later areas and DLC gameplay remain untested.
Compare revision shader builds and pipeline interfaces across local games,
and run available smoke routes with crash, hang, timing and image checks.
Add the developer loop and share the measuring lock with bench.py.
Keep all local games in static checks and honor --no-smoke. Include shader
bytes and fallback pairs in cache keys, recheck interfaces on reuse, and
reuse smoke baselines only for matching inputs and run settings. Detect
failed prebuilds, missing or intermittently bad screenshots, and Vulkan
build failures even when no MSL was emitted. Keep game-derived files and
writable profile copies inside each game's NSP work folder.
Validation: tool tests and the lint ratchet pass. Full shader rebuilds and
gameplay comparisons remain unverified; the 26 discovered Mac builds have
no smoke routes in this checkout.
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Erick Sanchez