Commit Graph
451 Commits
Author SHA1 Message Date
Erick Sanchez 585eb02cdf Replace front page with Spanish quick-start and Sideloadly guide
Preserve the previous README byte for byte as README-original.md. List current game profiles, distinguish local device validation, and document build commands, lightweight assets, controls and save-preserving updates.
2026-10-26 11:10:48 -06:00
Erick Sanchez 7c33c508c9 Automate personal unsigned iOS builds and lightweight IPA export
Use the edited checkout for every game, persist identity, verify archives, preserve existing outputs, and reject concurrent builds. Include applicability triage for 57 open upstream PRs and 20 commits after the stable base.
2026-10-26 11:10:48 -06:00
Erick Sanchez 94dfd1655e Count unique archives when estimating extraction space
Adapted from upstream PR #277 (841684012800327d6d6fa6b6fd79d3ab689c9751).
2026-10-26 11:10:48 -06:00
Erick Sanchez 8d29a31d2b Add persistent per-control iOS touch layout editor 2026-10-26 11:10:48 -06:00
seedds ecbeae164c Port Pokemon Brilliant Diamond 1.3.0 at 60 fps
Add the registered profile, Unity 2019 compute shaders, VP8 decoding, depth initialization and AudioOut stop fixes. Include build-gated frame limiting, regression coverage and Mac validation notes.
2026-10-26 15:40:36 +08:00
seedds ceb5046b3f Apply Pokémon Legends: Z-A's allocator cut to other builds by function match (#266)
The base game 1.0.0 (main 7222e13e…) aborted in nninitStartup: the 12 GB → 4 GB
StandardAllocator cut was registered for 2.0.2's main only. The same function
(an exact fingerprint match, main+0x19f8480 in v0) now gets it through MATCH_CUTS.
2026-10-26 15:34:21 +08:00
58f74014f4 Seed TotK prewarm from the recorded M2 iPad route (#261)
Add 1,537 shader, 2,512 render-pipeline and 43 compute-pipeline rows from
the update 1.4.3 capture, preserving first-use order. Document the
author-reported route and results, and limit the coverage claim to
that route.

This profile-only capture is separate from the shared CPU and loader
optimizations. Validation checks the row counts, uniqueness and exact
data/order preservation; no new cold-start or gameplay run here.

Co-authored-by: draco-cong <71372970+draco-cong@users.noreply.github.com>
Co-authored-by: laobeibei <laobeibei@users.noreply.github.com>
2026-10-26 15:28:59 +08:00
5632203274 Optimize CPU deswizzle copies and host symbol lookup (#261)
Hoist block-linear row and slice addressing out of segment copies, and
index HLE, forced-takeover and missing-import names while retaining
existing duplicate precedence and stub identity.

Add byte-address differential tests and direct collision, duplicate,
growth and reuse checks. Correct the ASTC-tail and loader-serialization
comments and document independent CPU microbenchmarks. Keep TotK
prewarm data in a separate change.

Validation: 5,334 texture checks; symbol-index and NRO lifecycle tests;
AddressSanitizer/UBSan; documentation checks; Mac host build and changed
iOS source compilation. No new gameplay validation.

Co-authored-by: draco-cong <71372970+draco-cong@users.noreply.github.com>
Co-authored-by: laobeibei <laobeibei@users.noreply.github.com>
2026-10-26 15:28:52 +08:00
f7345d9a00 Same-core handoffs wait only for the threads their wake made runnable (#247, #259 section 6)
A wake's handoff waited until every runnable higher-priority thread pinned to the caller's core was blocked, not only the ones the wake made runnable. The host runs a lower-priority thread beside such threads at all other times, and a core's busy loaders, workers and pollers are seldom all blocked at once, so the wait had no bound. Tears of the Kingdom 1.4.3 (issue #247) stayed on an empty scene after the boot logos: on 82ab25b, 84 functions / 771 NVN calls a frame, no packaged texture uploaded, 250,008 ms of handoff waits in a 180 s run; a temporary log showed BootupBackgroundWorkThread (priority 58) waiting 200-290 ms at a time for ARes Loading 1, Presentation Thread and BaseProcExecutor. Donkey Kong Country: Tropical Freeze's priority-58 job thread waited the same way for pollers (issue #259, section 6).

A wake now records the woken thread in the waker's current handoff and in those the waker belongs to (note_runnable, at most four), so Metroid Dread's main thread still waits for its movie dispatcher and for the worker the dispatcher wakes. Blocking, a sleep, YieldToAnyThread (SleepThread -2, which on Horizon takes the thread off its core so that lower-priority threads run there), exit, suspension and host GPU work end a thread's hold; a handoff ends when its caller's thread SVC returns (hand_epoch). There is still no timeout, and core_preempt keeps checking every runnable peer.

Checked on the Mac, without shader libraries for chain 8393d85a4eae (draws skipped): TotK on the issue #70 route for 180 s, 1,221 ms of handoff waits and the same per-frame NVN call counts and texture first uploads as 9feac2f (before the unbounded handoff). Dread, fresh saves, DM_KEYS=4:A*20/0.7, 60 s: a control host with the handoff off crashed in 1 of 3 runs at sdk+0x347eb4 from main+0x225338; the limited handoff passed 12 of 12 runs, each through three decoders and two teardowns with about 12,400 handoffs. t_preempt adds three handoff-scope cases: a busy peer nobody woke (main's code never returns), a worker woken by the woken dispatcher (fails without the recorded chain), a woken thread yielding to any thread (never returns without that rule). Host tests, docs tests and an iOS syntax check of hle_thread.c pass. Gameplay with shaders, iOS and DKC itself are not checked here.

Co-authored-by: HougeLangley <1161594+HougeLangley@users.noreply.github.com>
Co-authored-by: aurlight <120410928+aurlight@users.noreply.github.com>
2026-10-26 15:00:32 +08:00
seeddsandchengjie 2019ff866b Recognize framed pointer tail calls safely (#256)
Preserve the existing terminal-tail recognizer and accept a contiguous
framed method-pointer thunk followed by its separate null-error call.
Reject stores that overwrite saved FP/LR, unproven store addresses and
stack/frame/link-register clobbers, including WSP. Stores are limited to
fixed caller stack-argument slots above the saved frame.

Keep the fallback narrower than #188's proposed unframed block recognition.
Add synthetic AArch64 regressions and document the rule beside the other
tail-call findings; no game-specific IDs, fixed addresses or profile changes.

Validation: 179 tool tests run, 174 passed and 5 skipped; 10 tail-call tests
pass; the legacy terminal recognizer is AST-identical to main; git diff
--check passes. Real game modules were not rewritten for this integration.

Co-authored-by: chengjie <chengjie177@gmail.com>
2026-10-26 14:46:10 +08:00
seeddsandRAHUL REX 4acde72a6e Recover the frame step after movie and thermal caps (#253)
Keep the proven frame cost and GPU reference for each cap, and probe the
shorter hold when a movie or thermal cap ends. Otherwise a GPU-bound game
can fall straight back to 30 fps without a usable recovery reference.

Save an uninterrupted sustained run when a cap begins, even if no earlier
hitch populated cref. Keep the earlier reference through short runs and
update it after another sustained run. Count cap-release probes and give
them all 30 grace frames: the completed frame at the old cap consumes none.

Add movie and thermal recovery regressions, including first movies,
repeated caps, full grace and failed-probe backoff. Document the mechanism
and link it from both Zelda game notes.

Validated: 64/64 frame-step checks, including ASan/UBSan; six checks fail
against the original PR. Production thermal tests and ThreadSanitizer,
docs tests and diff checks pass; the BotW Mac host builds. Device clock
recovery remains modelled, without a new gameplay observation.

Co-authored-by: RAHUL REX <75500427+RahulRex0@users.noreply.github.com>
2026-10-26 14:22:03 +08:00
seeddsandRAHUL REX 82ab25b40b Reuse a present allocator only after its own present completes (#254)
present_now cycles three Metal 4 command allocators by ticket, and before resetting one it waited for g_pq_done to reach the ticket that last used it. g_pq_done is not GPU completion: every display-link callback sets it to ticket - 1 before presenting (the previous present was committed, not run), and a present that was not submitted sets it to its own ticket. So a display-link present never waited there: with three presents in flight, or two after a dropped frame, the allocator was reset and its command buffer and UV buffer encoded again under a present the GPU had not run. Each allocator now keeps the newest ticket its own commit feedback reported (g_present_alloc_done), and its reuse waits for that. The display-link callback can now block in present_now until the GPU has run that allocator's previous present; it still moves g_pq_done before it waits, so the game's swapchain acquire is unchanged. General: every game presents through this path, display-link mode is the default.

t_present_alloc holds the Metal 4 queue behind a shared event and calls the display-link callback by hand, each present with a drawable of its own offscreen layer: a dropped frame (tickets 1, 3, then 4 on allocator 1 again) and one frame per tick (5, 6, 7, then 8 on allocator 2 again). Checked on an Apple M6 at 6062518: without the production change 4 of 13 checks fail in 10 of 10 runs, the earlier drawable reads back 0 in all 64 pixels, and every run logs [RED] display command buffer error with kIOGPUCommandBufferCallbackErrorPageFault; with it 13 of 13 pass in 20 of 20 runs under MTL_DEBUG_LAYER=1 with no red line. The whole dm/gfx/tests run list passes, dmhost builds for Mario, docs tests pass. Not checked: a game run that hits it, the iPhone, the not-submitted path. Docs: the frame-rate reference entry (with the page-fault symptom) and the symptom index.

Co-Authored-By: RAHUL REX <75500427+RahulRex0@users.noreply.github.com>
2026-10-26 14:06:20 +08:00
seeddsandv0lumehi 60625182c4 Harden GPU fence wake and pass scheduling (#158, #213)
Integrate fast shared-event wakeups, per-segment publication, array-slice
pass merging, and independent compute dispatch hoisting from the PRs.
Serialize CPU completion effects before publishing completed fences and
validate recycled post records. Preserve submission order for early waits.

Guard scheduling against attachment aliases, first-draw and bindless
reads, indirect ranges, stencil writes, and explicit barriers. Add direct
Metal regression coverage and a gated CPU completion race test with TSAN.

Record final BotW and Metroid Dread gameplay validation with optimizations
on and off. BotW retains fewer render passes, but the measured 60 Hz M4
Pro runs do not establish a timing speedup. Keep the original higher-load
and iOS performance claims explicitly unverified.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 13:33:02 +08:00
seedds f171088e2e Fire Emblem: Three Houses: record why its two pacing cuts stay per-build. Checked on the Mac (host at 7753b7b, headless boot to the title, 100 s per run, each cut undone in the rewritten main): without the frame-limiter cut the general yield back-off holds 59.7-59.8 fps (frame p50 16.73 ms against 16.68) at 0.9-1.2 W of CPU energy against 0.14-0.42 W (about 9,700 plain yields a frame before the back-off starts at 2 ms); without the render-thread timeout cut, 53,000-55,000 wakeups a second against 1,450 and 0.7-1.0 W. The 16 us timeout is the game's own: the render thread passes (1/fps) x 1000 = 16 to main+0xaa90e0, which multiplies it by 1000 into the TimeSpan (nanoseconds) of nn::os::TimedWaitEvent. Of the 206 modules on hand, the limiter (main+0x5be4b4) is the only loop whose only calls are YieldThread and GetSystemTick, so a rewriter pass for such loops would match this one site. Docs: the game note; docs tests pass 2026-10-26 13:29:13 +08:00
seedds dac7ba6338 Function-matched cuts: same-name rules are one patch's implementations; Taiko Risshiden V DX's frame-limiter cut reaches the base game and later builds. nso_match.resolve groups rules by name: exactly one implementation must match (the log names it), a red line only when none or more than one does. Before, each implementation of a changed function logged a red line on every build it did not fit. Taiko's MATCH_CUTS has two implementations of the frame end under one name: 1.1.1's (main+0x443e8c, 364 bytes, cut at +288) and the base game's (v0 main 47d7baf6..., main+0x43e140, 172 bytes: 1.1.1 added the frame-skip branch; the same 20 bytes at +104 = main+0x43e1a8). In v0 N is likewise a global (GOT 0xa1be28, data 0x9ec55c, 1) set to 2 only by CPrivateBattleWindow / CPrivateBattleTutorialWindow, and the start tick is read after the update; nso_fnmap.py finds no exact match for 1.1.1's function in v0, found by disassembly. Checked through patch_nso.py: v0 main takes the cut at 0x43e1a8 (implementation 2, no red line); a 1.1.1 main copy with another build ID takes implementation 1, output byte-identical to the registered cut; 1.1.1 keeps its exact CUTS. A scan of the 42 distinct mains on the drives found this limiter only in Taiko. The base game has not been run. Docs: rewriter.md (same-name implementations, procedure step 2, Found on), the game note. Checked: test_nso_match.py (new test fails on the old matcher), all 19 tool test files, docs tests 2026-10-26 13:24:01 +08:00
seedds 7753b7b685 Taiko Risshiden V DX: keep the frame limiter for N = 2, cut it only at N = 1, and record its cause. KTGL's frame end (main+0x443e8c) presents, acquires, then waits until N / 60 s since the frame's start tick in 2 ms SleepThread rounds; the tick is read after the scene's update (main+0x443d44), so every frame lasts N / 60 s plus the time to the next start, rounded up: never one vsync, on any hardware. That is the game's code, not the host (SleepThread and the tick are exact; a model of the loop gives 54 fps with the host's acquire and with the Switch's double-buffered one). N (GOT 0xa1c088, data 0x9ec764) is the game's vsyncs per frame: 2 in CPrivateBattleWindow / CPrivateBattleTutorialWindow (main+0xf02d0, main+0xec4cc), whose code counts 60 / N frames a second, and the game presents at interval 1 everywhere. The earlier cut (b.hs -> b at main+0x443fd4) skipped the wait for every N: with N forced to 2 it presented 60 frames a second against the game's 28.8. The new cut at main+0x443fac computes the threshold with udiv by 60 and sets it to 0 only when N = 1, where the host's present pacing holds one vsync. Checked on the Mac (host at d50ecc1, Mac libraries for chain 8393d85a4eae): title N = 1 54.6 -> 60.0 fps (frame p50 16.67 ms, wakeups 2,050 -> 300/s), N forced to 2 28.9 fps (no cut 28.8, old cut 60.0); town on the rewriter's own output 60.0 fps, p50 16.66 / p95 17.08 ms, no red line; patch_nso.py applies the cut with its byte checks. The CPrivateBattle scenes (small unit battles, PBBATTLE) were not reached: open in the game note. Docs: the game note and the frame-rate reference entry give the corrected cause; tool tests (17 files) and docs tests pass 2026-10-26 13:04:04 +08:00
seeddsandv0lumehi fb20bce70e Add GPU diagnostics and close periodic profiles per frame (#225)
Add periodic encoder profiles with attachment traffic and formats, alternating A/B measurement phases, bounded Metal trace capture, and gpuprof.py log summaries. Preserve main's offline pipeline-archive integration while adding the capture build setting.

Close periodic samples at frame boundaries through the last committed shared-event fence, so every:1 reports without mixing frames or reusing an unresolved heap. Add 28 CPU checks of production profiling code and include them in CI; document the mechanism and update the symptom index.

Validated with all 31 CI-style Python test files, the profiling regression, a Mac host build, and all 133 iOS source compiles. Profiling completion and heap ownership were checked with synthetic counter/event objects; no new real-GPU or gameplay validation.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 12:09:07 +08:00
seeddsandv0lumehi 3f22526e87 Add repeatable Mac benchmarks and harden report generation (#191)
Run profile-defined routes from independent copies of one save and report frame-time, presentation and GPU percentiles across repeated variants. Preserve the display-link behavior for targets at or below 60 fps.

Resolve relative paths before launching in the run folder, preserve the archive-disable sentinel, aggregate optional metrics from the runs that supplied them, and release the measurement lock when launch fails. Regression tests exercise an actual synthetic child host and leave the source save intact.

Validated with benchmark regressions, the CI-style Python test suite, and documentation checks. The three new benchmark regression cases reject the original PR code.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 12:09:07 +08:00
seeddsandAmost d50ecc1f84 Support masked-predecessor jump tables and external patches (#237)
Recognize offset-table dispatches whose predecessor masks the index while
the dispatch checks a masked copy of the same source. Reject calls and
resolved table paths that bypass masking, undecodable instruction gaps or
targets, and writes that invalidate the table base. Keep the existing
recognizers ahead of this fallback.

Load build-gated patches from DM_GAMES_DIR and include them in rewrite reuse
keys. Preserve repository matching rules for derived profiles, select the
first external profile for provisional matching rules, and deduplicate
patch paths while retaining conflicts between distinct build claims.

Validation on current main: Python tool tests (156, 14 skipped), iOS tests
(6), graphics tests (44, 2 skipped), documentation index and diff checks.
Regression cases fail on the original implementation and pass with these
guards; both supported base-loading forms remain accepted. Real game
modules and gameplay were not revalidated.

Co-authored-by: Amost <6445689+Amost-@users.noreply.github.com>
2026-10-26 11:03:47 +08:00
seeddsandv0lumehi 92fc2f2711 Build and ship offline Metal pipeline archives (#226)
Compile eligible shader-pair bases on the Mac with metal-tt, package one
archive per selected iPhone GPU, and use a matching bundle before falling
back to on-device preparation. Carry the profile depth mode through packing
and include the pulling libraries required by the bundled bases.

Integrate PR #226 with four confirmed fixes: validate archive container
bounds before invoking Metal; require at least one actual successful bundle
probe; hash consumed library bytes into offline cache keys; and emit device
remainder lists for constant-only plans instead of aborting packaging.
Keep game measurements in the game note and document the general findings.

Validation on current main: 52 real Metal bundle checks, 57 archive checks,
79 prewarm checks on each pipeline path, 73 Python regressions covering
offline tools, packaging, documentation and chain checks, and compilation
of all 133 iOS source files. No new iPhone gameplay validation. The archive guard validates
container structure, not private Metal payloads.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 10:52:50 +08:00
seeddsand7splay c6c4fac5c3 Exclude Mac shader intermediates and respect ARG_MAX (#240)
Pack only runtime shader libraries, reflection JSON and TSV tables, excluding translated Metal sources and _b_build.json from the app and its free-space estimate.

The original 2,000-file batching still exceeded the macOS exec limit with valid long paths. Budget encoded arguments, inherited environment and pointer overhead instead, with regression coverage for long and multibyte paths, clone flags and an exhausted argument budget.

Validation: reproduced E2BIG before the fix; all 17 Mac app tests and 4 documentation tests pass on current main. Tests use synthetic files and a stand-in host; a full large-game packaging run remains untested.

Co-authored-by: 7splay <107197512+7splay@users.noreply.github.com>
2026-10-26 10:40:10 +08:00
seeddsandFernando Messias 62a640b037 Fix custom key loading for compressed game inputs (#204)
nsz 4.6.1 rejects --keys and loads keys.txt beside argv[0] during import.
Run the converter in its temporary output folder with the selected key
file linked there before import, using absolute input and output paths.
Remove the temporary link on both successful and failed conversions.

Integrate PR #204 with XCZ and relative-path regressions, rename the test
flag that shadowed unittest.TestCase.fail, and qualify the documentation
for the verified nsz release. Record the packing finding and provenance.

Validation: reproduced the exact parser failure using real nsz 4.6.1;
a synthetic NSZ conversion verifies selected-key lookup and link cleanup;
52 NSP tests, four documentation tests and git diff --check pass.
The synthetic payload is unencrypted and uses dummy keys. Real-game
decryption, actual XCZ conversion and a full IPA build remain unverified.

Co-authored-by: Fernando Messias <fernandomessias95@gmail.com>
2026-10-26 10:36:23 +08:00
seedds 435a231051 Preserve Horizon audio renderer work-buffer requirements
DSP callers were charged the local SDK CPU server budget, whose command representation is larger than the Horizon service representation. For Age of Calamity the service requires 0x61d000 while the local SDK requires 0x760000, exhausting the fixed sound heap.

Calculate the service requirement through renderer revision 15 and supply larger local backend storage separately, with cleanup on failed open and after renderer close. Remove the two game heap cuts and add sizing, nested-open, failure and lifetime regression coverage.

Validation: 360 seeded cases match isolated Horizon machine-code execution across REV1-REV15; the original game heaps initialize all five players and produce audio; the host suite, documentation and patch checks, SDK 8/20 smoke checks and iOS C compilation pass. iOS runtime and long battles remain untested.
2026-10-26 10:18:29 +08:00
seeddsandv0lumehi c4b74a62f9 Add thermal frame-rate cap with synchronized recovery (#220)
Hold the host frame step at 30 fps while iOS reports serious or critical
thermal pressure, and restore its configured cap after 30 calm seconds.
Keep the existing movie cap and DM_THERMAL_CAP=0 override.

Sample thermal state independently of movies and configured 30 fps targets
so heating resets recovery and cooling counts during playback. Serialize
the sample, cached cap and recovery timer across acquire/present readers.
The original PR failed five timing checks and ThreadSanitizer detected a
race through concurrent production cap getters.

Add controlled-clock production-path regressions and CI coverage, including
disabled/inactive caps and ThreadSanitizer. Record the shared mechanism
and validation limits in the frame-rate reference and BotW note.

Validated on latest main with the BotW profile: 24 thermal checks normally
and under ThreadSanitizer, 51 frame-step checks, 154 NVN checks, the
present-snapshot suite, 21 tool/docs checks, Mac host build, and iOS compile
check (133 sources). No new device gameplay or thermal recovery run.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 10:13:42 +08:00
seedds c76f679834 Remove alpha_from_red: Super Mario Odyssey's layout captures need only the nvnTextureCompare fix. The game creates each capture with its per-format default mapping (an R8 one reads (R, 0, 0, 1)), rebuilds it on the same storage with the capture's own, (1, 1, 1, R), and replaces the registered descriptor only on a difference (main+0x982320); while the host's comparison ignored the mapping the captures kept alpha 1 (white coin cards, a dark box behind hints), which the 26-shader alpha_from_red list papered over. Checked on the Mac from the "Look Around" hint with a probe host on the comparison (a c3a0880^ host for the profile's chain dec5ee141a9f libraries): 40 mapping-only rebuilds in 11,700 frames (R8 captures 640x80 down to 48x57 to (1, 1, 1, R), a 320x180 RGBA one to (R, G, B, A)); ignoring the mapping again brings back the box (about 12,800 pixels from frame 4,200), the old list hid it by converting the 378x80 capture, and with the fix the list converted nothing and changed no pixel of frames 4,200-6,600. The comparison and the views have no platform branch, so the iPhone gets the same rebuilt mapping (not run on a device). With no profile left using it, the key, its host code (tex_alpha_red, afr_want, DM_NVN_ALPHA_RED) and its docs go; treg_drop_ar becomes treg_drop_im (the image-slot view it also drops stays). Odyssey's note has the A/B runs. Checked: the host builds for Odyssey, the iOS compile check (133 files), the tool tests (111, 14 skipped) 2026-10-26 10:01:20 +08:00
seeddsandparsifal295 2db12f7efd Fix vibration handles before controller haptics routing (#246)
GetVibrationDeviceHandles used routed haptics availability to decide whether
handles existed. Games that cache this answer at startup could retain an empty
answer after a controller connected. Return handles for a connected npad style
when rumble is enabled and an output sink is installed; check routed outputs
when values are sent, keeping the existing permission and lifecycle behavior.

Include the PR's rumble diagnostics, count unroutable values individually in
mixed batches, and protect aggregate counters across game threads. Correct the
code comment to describe Unicorn Overlord's observed empty handles and zero
values without claiming restored physical rumble.

Add regressions for cached startup handles delivering a nonzero value after
routing, diagnostic totals and concurrent sends. Include HID in the existing
ThreadSanitizer target and record the mechanism in the service reference.

Checked with GAME=010069401ADB8000: full host suite, 73 HID checks, audio and HID
ThreadSanitizer checks, docs checks, Mac host build, and iOS compilation
(133 files). Physical rumble and iOS runtime remain untested. Broader tool
validation is incomplete because Capstone is missing and iconutil failed.

Co-authored-by: parsifal295 <130349765+parsifal295@users.noreply.github.com>
2026-10-26 09:54:43 +08:00
seedds 16061e0399 Rule: fix a bug at its root cause, not around it. AGENTS.md says to find why a bug happens (the faulting instruction and its decompilation, the bad value and what wrote it, the semantics the host or translator gets wrong) and fix it there; a change that only hides the symptom (stub, skipped call, clamp, retry, sleep or timeout, cache flush, profile key that turns a feature off) is a workaround, used only when the cause lies where it cannot be fixed (the game's own code, changed deliberately by a rewriter cut, or a platform limit) or the user asks for a stopgap, and then recorded as open with the cause or what is unknown. docs/agents/README.md has the same as a ground rule. Checked: dm/tools/tests/test_docs.py passes (AGENTS.md 1,419 words) 2026-10-26 09:42:57 +08:00
seeddsandv0lumehi c13ddcf531 Add opt-in MetalFX spatial upscaling and a single-screen launcher (#215)
Scale the valid presented image with MetalFX's spatial scaler on the shared
Metal 4 path, preserving crop, letterboxing, origin and gamma-encoded colours.
Keep Standard as the default and fall back when scaling is unsupported.

Use the same path for asynchronous whole-window captures and diagnostic
screenshots, retaining resources until GPU completion. Bound scaler recreation
during resizing and report presentation GPU time separately by scaling mode.

Put all launcher settings on one screen with Game, Controls and Performance
sections, inline menus and sliders, and a reset to tested defaults. Extract the
settings model for native persistence tests, validate saved ranges, share HUD
setup, and document the measured costs without claiming an iPhone baseline.

Validated with Metal presentation, NVN and snapshot regressions; tool and iOS
tests; the Mac host; and an unsigned iOS smoke build. No new device gameplay run.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 09:38:05 +08:00
seedds 3aa706a3fd Pokémon Shield (01008DB008C2C000, update 1.3.2 = v458752, Game Freak's engine on SDK 7.3.2): game profile, no host change. Sword's twin: the same sdk, subsdk0 (movie) and subsdk1 (GLSLC) builds, only main is its own (a1680262…), and the shader split gives Sword's 41,516 shaders with the same data rows in all three tables; the profile is Sword's with Shield's title, name, IPA name, build row and tables (prewarm list copied). The game note (dm/games/01008DB008C2C000/README.md) has what differs, the Mac route (a fresh save needs DM_KBD for the name; Sword's route out of the house works unchanged) and the shader build with DMSH_PREFILL from Sword's libraries (21,974 Mac / 21,684 iOS filled, 176 / 175 compiled). Plays on the iPhone (IPA 12.4 GB, user) and on the Mac at 30 fps with no red line from the first boot through the opening and Postwick to Hop. README gallery entry with a Mac screenshot, a row in the games table, a pointer from Sword's note, AGENTS.md's count to sixty-seven. Checked: the tool tests pass (111, 14 skipped) 2026-10-26 09:20:23 +08:00
seedds 3dd5d25a51 Add Echoes of Wisdom 1.1.0 port and correct cube fallback diagnostic 2026-10-26 09:14:43 +08:00
seeddsandv0lumehi 577df01c39 Fix iOS pulling shader packaging for profile defaults (#211)
Include pulling vertex libraries when the compiled game profile enables the
Metal 4 path, preserving explicit IOS_PULL overrides. Keep the pack options
array nonempty so inclusion also works with macOS Bash 3.2 and set -u.

Exercise the actual archive step with tiny shader fixtures for Breath of
the Wild and Wonder, checking both defaults and explicit overrides. Record
the packaging failures and fixes in the rendering reference and game note.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 08:56:50 +08:00
v0lumehi 037ff69562 Profile keys present_snap and pso_mtl4: a game ships the presentation and pipeline settings it was tested with. present_snap on: present snapshots by default (a ring of 2 host textures; the iOS launcher's Frame buffering defaults to it; DM_PRESENT_SNAP and Documents/env.txt override); pso_mtl4 on: render pipelines through MTL4Compiler on iOS by default, with the pipeline archive, the prepare step and the hybrid pipelines (on by default on the Mac; DM_PSO_MTL4 overrides). Breath of the Wild sets both: they held a steady 60 fps on an iPhone 17 Pro Max (issues #142 / #159; without the snapshots the GPU idled between frames and iOS lowered its clock, 30 fps). README, docs/development.md and docs/reference/frame-rate.md say the new defaults. Checked: the host builds for Breath of the Wild (DM_GAME_PRESENT_SNAP 1, DM_GAME_PSO_MTL4 1) and Mario Wonder (0, 0), dm/tools/tests pass, t_present_snap and t_nvn pass with Breath of the Wild's profile, the iOS compile check passes 2026-10-26 01:43:25 +01:00
v0lumehi f6bd56f5cc The pipeline prebuild runs in the prepare step: with a window (and on the iPhone) the bases of every shader pair are made right after the prewarm, before the game's first frame, behind the progress screen ("Preparing pipelines"), once per chain version, depth mode and pair list (prebuilt.tsv). Built in the background while playing, they took an iPhone 17 Pro Max's GPU from about 13 to 15-18 ms a frame in Breath of the Wild (the compiler's load on the shared power budget: 43-95 dropped frames per 300 against 0-25 without it); without a prepare step the background prebuild stays. Tests: t_nvn 128, t_pso_archive 57, t_pull 346, iOS compile check 2026-10-26 01:43:20 +01:00
v0lumehi 1eeb109b35 docs: the prepare step (docs/development.md "The prepare step", Breath of the Wild's measurements in its note dm/games/01007EF00011E000/README.md; dm/gfx/README.md DM_PSO_PREPARE; README). Mac, M2 Max, runs started at a 1-minute load below 4, Metal cache empty, the prebuilt archive, the Zora's Domain save: with no list, 4 frames over 25 ms with new pipelines and 5 with shader library loads (75 libraries, 168 ms of that work, worst 128 ms) in the 60 s after the last menu press; with the new pso_manifest.tsv none, whether it was built in the background (14.6 and 16.0 s on one worker, done during boot) or by the prepare step (5.2 and 2.2 s on 12 workers, 3.3 s on 8, 872 bases compiled); a second start prepares it in 0.08 s and a player's 5,783-line list in 0.23 s, both with no screen. Hades (Vulkan) is unchanged: it builds its 1,856 pipelines itself while it loads 2026-10-26 01:43:20 +01:00
v0lumehi d9aa864d0d Breath of the Wild: a prewarm seed list (pso_manifest.tsv, 1,789 lines: 960 render pipelines, 819 shaders, 10 compute shaders), so a first start has something to prepare. Recorded on the Mac with prewarm off (DM_PSO_MANIFEST=off DM_PSO_RECORD) over update 1.9.0: boot, title, New Game and the Shrine of Resurrection until Link can move (1,298 lines); then boot, title, a save load into Zora's Domain and a minute of play, whose new lines are appended (491). The lines name shaders by their Switch content key and pipelines by NVN state (pso_skey), which no translation chain version changes: the save-load part was recorded on chain b7b9aa970b37 and every line resolves on 97e2f301eb73 2026-10-26 01:43:20 +01:00
v0lumehi 7a47d1c5eb The prepare step: with a window, the game's first submission waits until the prewarm lists (pso_manifest.tsv, then pso_user.tsv) are built, on every core at user-initiated QoS, behind a "Preparing shaders" screen that shows only past 0.5 s and with 0.3 s or more still to go. nvn_mtl.m: the prewarm reads its lists into tasks (warm_parse): a shader, a compute pipeline, or all render pipelines of one shader pair, which share their bases, so no two workers compile the same base; shaders go into one dictionary the workers share, and a vertex shader's pulling entry is loaded with it (as shader_prefetch_rest does), so neither a draw nor its hybrid stand-in loads a library the lists name. warm_prepare runs the tasks on hw.activecpu workers (DM_PSO_PREPARE_THREADS=<n>) from mtl_sub_begin's first call, before anything of that submission is encoded, and gives the front end done of total every 50 ms once 0.5 s have passed, unless the rate so far leaves less than 0.3 s; the shaders and pipelines go into the caches the draws use at once (warm_claim), which keep them for the run. It runs at the first submission and not before the game starts because the vertex entry each pipeline is built with follows the depth mode the game sets (nvnDeviceSetDepthMode). It runs at every start: with the Metal cache and the pipeline archive warm it takes well under a second and shows nothing, and a stamp that skipped it would skip it exactly when the system has dropped the Metal cache (an OS update, storage pressure on iOS). Without a window, or with DM_PSO_PREPARE=0, the prewarm runs in the background on one worker as before; DM_PSO_PREPARE=1 makes a windowless run wait too. The log line: pipeline prepare: N lines (shaders, render and compute pipelines) in s on N workers before the first frame, progress screen shown or not; bases compiled, from the archive, reused. front.h front_progress: front_mac.m draws the screen with AppKit views over the game's view, front_ios.m with UIKit views over the game view and the pad (a dark view, the title, a bar and a count); nvn_mtl.m keeps a weak empty one for the unit tests. NVN only: the Vulkan and GL layers keep no prewarm lists 2026-10-26 01:43:20 +01:00
v0lumehi 9607f51022 iOS app: shader_pairs.tsv and pso_pairs.tsv go into the bundle beside the shader map (build_ipa.sh), where the pipeline prebuild reads them; without them an iPhone made no pulling bases and hybrid draws never had a stand-in (Breath of the Wild's log: 'pipeline prebuild: no …/dm/shader_pairs.tsv, nothing to build', 0 draws with a hybrid pipeline in 15,600 frames) 2026-10-26 01:43:20 +01:00
v0lumehi e0ef4e707e Pipeline prebuild: the pulling base of every known shader pair into the pipeline archive, with the pairs of agl's BAHS program variations (pso_pairs.tsv)
A pulling stand-in is used only when its base is made or archived, so a cold start's draws waited until the prebuild puts the base of every known pair into the archive (pso_prebuild): in the app after the prewarm, on a utility queue on half the performance cores, flushing every 1,000 bases (DM_PSO_PREBUILD=0: off), or as a tool on the Mac (DM_PSO_PREBUILD=tool: build, write, exit; DM_PSO_PREBUILD_DEPTH=zero, DM_PSO_PREBUILD_THREADS=<n>), once per chain version, depth mode and pair list (pso_archive/prebuilt.tsv). The NSP script runs the tool in a --platform mac build (pso_prebuild.log). Files that hold only bases without a vertex layout are named p<seq>.mtl4a; a base with a vertex layout is never looked up in them (each file a lookup misses in costs about 0.06 ms) and compaction leaves them alone.

The pairs: shader_pairs.tsv paired the stages of one BNSH variation and none of the pairs in agl's BAHS shader archives (.sharcb: the agl / gsys / terrain resource packs), and in Breath of the Wild 109 of the 543 pairs drawn on entering Zora's Domain came from those, so they had no stand-in. shader_split.py now reads a BAHS file's binary array (record size, kind: 0 vertex, 1 fragment, 3 compute, the entry bahs_shaders reads) and program array (record size, name length, stage mask, first binary index), and pairs the n-th vertex binary of a program with its n-th fragment binary (bahs_variations); a file whose arrays do not read back gives none. Those pairs go to a new table, pso_pairs.tsv, which only the prebuild reads (DM_PSO_PAIRS: another path): vs_pad keeps reading shader_pairs.tsv, so no shader library, inputs digest or other table changes. Breath of the Wild: 3,012 more pairs, and all 543 pairs drawn are now known; its shader_map / facts / pairs tables split again byte-identical. A profile made before the table existed still packs: the NSP script compares pso_pairs.tsv only where the profile has one, and otherwise keeps it beside the shader package and hands it to the prebuild. Seven other profiles have BAHS files (Odyssey, Wonder, Mario Kart 8 Deluxe, 3D World, Animal Crossing, Captain Toad, Tears of the Kingdom); their tables are made at their next pack.

Breath of the Wild, Mac (M2 Max), Zora's Domain save, empty Metal shader cache, no pso_user.tsv. Prebuild: 15,767 bases, 910 MB in 16 files, 187 s with 7 workers for the first 12,755 and 19 s for the 3,012 more; the files open in 2.2 s cold, 0.25 s from the file cache, and add about 850 MB of footprint. The frame entering the area: 340 ms with pso_pairs.tsv, 830 ms with shader_pairs.tsv alone, 990 ms through the MTLDevice (then frames of 738, 535 and 496 ms); draws that waited: 50 (20 ms), 102 (650 ms) and 512 (3,325 ms). The picture is the same.

Tests: t_pso_archive (41 checks: also a base without a layout flushed into a p file, and a base with a layout not looked up there); the other graphics tests, the host unit tests and test_nsp_mac / progress / chain give the results they give on main. Not run: an iPhone (the prebuild is off there with the Metal 4 path), the NSP script's pack end to end, the other BAHS profiles' splits. docs/development.md, dm/gfx/README.md and dm/games/README.md describe the prebuild and the table; Breath of the Wild's note (dm/games/01007EF00011E000/README.md, Pipelines on a cold start) has its measurements.
2026-10-26 01:43:20 +01:00
v0lumehi 8593d85a9c docs/development.md: a diagram of the hybrid miss path (cache, base and specialization, pulling stand-in with the draw's own pipeline queued first, or a build at once with the bounded prediction) 2026-10-26 01:43:20 +01:00
v0lumehi bc76cab190 Hybrid pipelines: a draw whose pipeline is not built yet draws with its shader pair's pulling pipeline meanwhile, on main's demand-first pipeline builds
On a miss the draw first takes its pipeline without compiling (its base made or in the archive, then a specialization). Failing that it draws with the pair's pulling pipeline (<key>_bp: no vertex descriptor, the streams read through a per-draw layout block at buffer 16, pull_bind), specialized with the draw's colour state, while its own pipeline is built in the background; a later draw takes that one. Both compute the same positions (t_pull_inv), so a draw that switches does not flicker or z-fight. Only a pulling base already made or archived is used; otherwise it is built in the background for the pair's next miss and the draw waits. Merged passes, readable-tile and diagnostic fragments, and layer-writing or transform feedback vertex shaders have no stand-in. DM_PSO_HYBRID=0 turns it off; it needs the Metal 4 path, so it is on by default on the Mac only.

On main's demand-first pipelines (#75): pso_pend_kick stays the speculative admission (at most two running, at most two per prediction, the oldest finished entry reclaimed when the table is full). A draw with a stand-in queues its own pipeline before the prediction runs (pso_pend_demand): outside the speculative limit, on serial queues at utility QoS, one per two performance cores (pp_bg_queue), so a demanded build never waits behind speculation and speculation never waits behind it. The prediction spends its budget only on draws that will have no stand-in, since only those would wait. A draw that would wait for a build still queued makes the pipeline itself at once (pso_pend_take; Breath of the Wild's cold start waited 0.5-0.9 s for one pipeline behind a burst before). The first miss of a submission loads the shader libraries the rest of it binds in parallel (shader_prefetch_rest): 248 libraries in about 210 ms where one by one on the submit thread took about 1 s, in the hybrid and the waiting path alike.

Diagnostics: a "pipelines up to frame" line every 300 frames (submit-thread pipeline, library and prediction time, archive lookups, specializations, base compiles, hybrid draws, background builds, the draws that waited and why), DM_PSO_WAITLOG=1 (each wait over 20 ms with its pair), DM_PSO_PULL_ALL=1 (every draw with a pulling pipeline draws with it), and the longframe line's count of hybrid draws.

Breath of the Wild, Mac (M2 Max), steady state in Zora's Domain: mean GPU p50 over frames 2,400-5,100 was 8.98 ms with the hybrid, 8.98 ms on the Metal 4 path without it, 9.35 ms through the MTLDevice. Cold-start numbers are with the prebuild of the next change.

Tests: t_pull (346 checks, now also the layout block the host fills), t_pull_inv, t_pso_pending (48) and t_pso_archive (32) pass; the other graphics tests and the host unit tests give the results they give on main. docs/development.md ("Hybrid pipelines") and dm/gfx/README.md describe it.
2026-10-26 01:43:20 +01:00
v0lumehi e6b800a56d Pulling vertex shader variants in the translation chain (<key>_bp, dmsh_pull.py with its decoder dmsh_pull.metal)
A side library of every vertex shader with stage inputs, as <key>_bv is, whose main0 / main0_z have no [[stage_in]] and read their inputs from a per-draw layout block at buffer 16: per stream the base address, stride and divisor, per NVN attribute its state word, the base instance from [[base_instance]] (so indirect draws are right too). dm_fetch decodes the NVN format at run time into the register words Maxwell's fetch gives the shader, and the input takes them with as_type, so the body runs unchanged. A pipeline built with it has no vertex descriptor and serves every vertex layout the pair is drawn with; the hybrid pipelines (next change) draw with it while a draw's own pipeline is built.

The words are bit-identical to Metal's vertex-descriptor fetch for every format nvn_mtl.m vfmt knows and to the host's CPU conversion for the integer-read-as-float formats: unorm / snorm through the unpack builtins (a plain division under fast math is a unit in the last place off for up to half the values), R11G11B10F NaNs canonical as from Metal, divisors with Metal's rule base + (instance - base) / divisor. Positions are [[invariant]] and built with preserved invariance, so a pulling pipeline and the vertex-descriptor one compute the same positions. Transform feedback, geometry capture and layer entries get no variant; shaders without stage inputs need none. The run-time decode costs GPU time: +65 % of a vertex-bound draw with 8 attributes on an M2 Max, about 1 ms per million vertices, so it is meant only to bridge the frames until the draw's own pipeline is built.

The iOS app does not use the variants yet (the Metal 4 path is off on iOS by default), so build_ipa.sh leaves the _bp libraries out of the IPA unless IOS_PULL=1 (pack_ipa.py --skip-suffix).

Chain version 8393d85a4eae (on main's 20fe3d62b9f6). Breath of the Wild's full Mac build on chain 97e2f301eb73 (25 min, a full retranslation since the code fingerprint changed): 19,592 shaders ok, 8,888 pulling libraries generated (275 MB), none failed, 43 vertex shaders without stage inputs; every existing library is byte-identical to the previous build (19,592 _b.metallib and _b.metal, 6,254 _bf, 1,070 _bv). Other profiles' shaders were not rebuilt.

Tests: t_pull.m (304 checks: the 76 formats vfmt knows, 73 comparisons with Metal's fetch at aligned, 2-byte and stride-1 placements, 24 with the CPU conversion, stride 0 and divisors 1-3 with a base instance), t_pull_inv.m (2,000 random triangles: a depth prepass with one pipeline and depth-equal with the other passes every covered pixel, both ways round) and test_dmsh_pull.py (the generator: entries, ids it adds or keeps, entries left out, shaders declined; compiles for macOS and iOS); pack_ipa.py's --skip-suffix checked by hand. The other graphics tests and the host unit tests give the results they give on main. docs/reference/rendering.md ("Pulling vertex inputs") describes the variant and its cost; AGENTS.md lists the new chain files.
2026-10-26 01:43:14 +01:00
seeddsandAleksandr Svidchenkov 1a93086e2d Fix service-event and vsync worker lifetimes (#202)
Apply PR #202's host-thread skip and duplicate vsync suppression. Give
bindings unique lifetime IDs so a retired worker cannot signal another
service's event after guest storage and kernel handles are reused. Replace
the eight-entry worker table with allocated records so rapid replacement
does not abort while retired workers wait for their next tick.

Add deterministic regression cases through real event binding and SVCs,
and document the mechanism with an updated symptom index. Both the
capacity abort and stale signal reproduce against the original PR.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:35:19 +08:00
seedds 8b6f907fc1 Merge pull request #243 from seedds/codex/thread-start-handoff
Honor same-core priority handoffs when starting threads
2026-10-26 08:31:38 +08:00
seedds 02e40ba753 Record threading audit and retain verified game workarounds 2026-10-26 08:22:50 +08:00
seedds c188131824 Honor same-core priority handoff when starting threads 2026-10-26 08:22:50 +08:00
seeddsandAleksandr Svidchenkov 4896c78d8a Fix packaged texture dimensions and linear storage sizing (#201)
Decode packaged byte 0 as separate block-height and block-depth nibbles,
including explicit zero dimensions, and size linear compressed textures
by pixel rows. Port the driver-derived rules from PR #201 and preserve
its attribution to digitalhivehub's PR #11 measurements.

Integrate the layout test with current main, correct its BC4 label, add
independently addressed CPU markers and packaged GPU mip coverage, and
record the mechanisms in the reference docs and symptom index.

Validation: 9 driver shapes, 189 CPU texture checks (also ASan/UBSan),
17 GPU deswizzle cases, 154 NVN checks, documentation checks, Mac host
build, and iOS compilation of 133 sources pass. No new gameplay or
iPhone runtime validation.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 08:19:08 +08:00
seeddsandAleksandr Svidchenkov c1b7eff61f Fix Metal buffer alias visibility and sparse copy ordering (#200)
Carry ResourceAlias visibility on copy encoders containing buffer copies,
inline updates or fills, using the contribution from PR #200. Operations
through sparse virtual mappings also need ordering against surrounding
operations: disjoint guest addresses can name the same physical bytes,
so guest-range dependency detection alone misses their hazards.

Add a regression using distinct no-copy Metal buffers and real sparse
mappings. The original host failed 37 of 101 checks on an M4 Pro and
PR #200 alone left 24 failures; the complete fix passes all 101 checks
with both direct and deferred encoding under Metal validation. Ordinary
independent buffer operations retain range-based batching.

Validation: t_nvn passes 154 checks; the Mac host builds; all 133 iOS
source compile checks and the documentation checks pass. Monster Hunter
Rise gameplay, iPhone execution and sparse-barrier performance were not
retested.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:28:09 +08:00
seedds 31b5944eb4 Port NieR Automata with 60 fps and geometry fixes 2026-10-26 07:13:22 +08:00
seeddsandAleksandr Svidchenkov a5513d4ae0 Fix Depth16 load-clears to match fragment depth writes (#199)
Apple GPUs truncate Depth16Unorm load-clears while rounding fragment depth,
so clearing to 0.5 stores 32767 instead of the fragment's 32768 and fails
an EQUAL comparison. Apply the format-specific correction to both
attachment and whole-texture clears, preserving other depth formats.

Exercise both production command paths against independent GPU fragment
writes for 1,042 float depths, including half-bin boundaries and adjacent
floats. Removing either correction produces 518 mismatches on M4 Pro;
with both corrections enabled, neither path differs. Record the mechanism
and the pending TOTK highlight and iPhone validation in the documentation.

Validated on M4 Pro with Metal validation: t_depth16_clear, t_clear_ds
(43 checks), t_clear_tex (49 checks), and the documentation tests pass.

Co-authored-by: Aleksandr Svidchenkov <a.svidchenkov@gmail.com>
2026-10-26 07:06:32 +08:00
seedds 9a56ecf0c1 Order same-core wake handoffs and remove Dread movie gates
Use guest runnable state after a higher-priority same-core wake instead of Mach dispatch state or a 5 ms cutoff. Honour wake handoffs for every game and remove Metroid Dread request-ring gates and its core_preempt setting.

Validate event, mutex and condition-variable handoffs with delayed host dispatch; 13 valid Dread movie-transition runs passed. Host tests, docs and shader-chain checks passed, along with an iOS syntax check. Record Wonder boot comparison and pending iOS runtime validation.
2026-10-26 02:27:03 +08:00