Harden GPU fence wake and pass scheduling (#158, #213)

Integrate fast shared-event wakeups, per-segment publication, array-slice
pass merging, and independent compute dispatch hoisting from the PRs.
Serialize CPU completion effects before publishing completed fences and
validate recycled post records. Preserve submission order for early waits.

Guard scheduling against attachment aliases, first-draw and bindless
reads, indirect ranges, stencil writes, and explicit barriers. Add direct
Metal regression coverage and a gated CPU completion race test with TSAN.

Record final BotW and Metroid Dread gameplay validation with optimizations
on and off. BotW retains fewer render passes, but the measured 60 Hz M4
Pro runs do not establish a timing speedup. Keep the original higher-load
and iOS performance claims explicitly unverified.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
This commit is contained in:
seeddsandv0lumehi committed 2026-10-26 13:33:02 +08:00
1 parent f171088e2e
commit 60625182c4
17 files changed
+656 -39

No files matched your search

+3
View File
@@ -51,6 +51,9 @@ jobs:
run: |
mkdir -p logs
cd dm/gfx/tests
make GAME="$GAME" -j3 t_perf_order
./t_perf_order --cpu 2>&1 | tee ../../../logs/t_perf_order.log
make GAME="$GAME" -j3 perf-order-tsan 2>&1 | tee ../../../logs/t_perf_order-tsan.log
tests="t_dirty_slots t_dirty_race t_tex_layout t_pace t_fstep t_hmap t_cover t_pso_skey t_refl t_rt_range_ro t_present_rescue t_gpuprof"
make GAME="$GAME" -j3 $tests 2>&1 | tee ../../../logs/gfx-build.log
failed=0
+2
View File
@@ -31,6 +31,7 @@ No `patches.py`, no `sdk_patches.py` cut for its `sdk` build (`2d369d99…`), no
## Rendering
- **Synchronization menu smoke check (2026-10-26, Mac M4 Pro):** #158 and #213 with review fixes and current `8393d85a4eae` Mac libraries. Isolated copies of the existing save ran 50 s with fast wake/pass optimizations/deferred encoding (2,853 frames) and with all three optimizations disabled/direct encoding (2,968 frames), without a red line or crash. The wait ledger records one samples-passed report and one timestamp report per frame in both modes. Screenshots show the audio-language selection menu; a 65 s extended route still stopped there (3,864 frames). This validates that menu and its report traffic, not battle/gameplay or a performance gain; neither pass optimization activated. Logs, copied saves and shots are in the NSP work folder's `run/perf-order-on/`, `perf-order-off/` and `perf-scene-on/`. See the general [SyncWait completion](../../../docs/reference/nvn-objects.md#the-games-syncwait-wakes-on-the-gpus-own-signal-of-its-fence), [slice merging](../../../docs/reference/rendering.md#passes-into-other-layers-of-one-texture-level-run-as-one-layered-pass) and [compute hoisting](../../../docs/reference/rendering.md#compute-dispatches-between-the-draws-of-a-pass-run-ahead-of-it) entries. iOS runtime validation remains open.
- **Single-stage programs:** the game builds one NVN program per GLSLC output (838 `ProgramSetShaders` at startup) and combines vertex and pixel programs per draw with `BindProgram` stage bits, so the real pairs exist only at draw time. Every pixel shader input is a plain `float4` at locations 1–9 (the SPH input map, words 6–13, matches the translated MSL inputs for all 375 pixel shaders), so the cover that pairs every vertex shader here is one pixel shader.
- Depth mode: the game sets near = 0 (`nvnDeviceSetDepthMode`), as Link's Awakening and Engage do.
- **Resolved: an assertion a minute and a half into the prologue,** `buffer index (31) must be < 31`: the game binds vertex streams 14 and 15, and the host bound NVN stream *n* to Metal vertex buffer 16 + *n*.
@@ -41,6 +42,7 @@ No `patches.py`, no `sdk_patches.py` cut for its `sdk` build (`2d369d99…`), no
General host changes first needed here; each applies to every game:
- **Synchronization review menu check:** [Rendering](#rendering) records the #158/#213 comparison, samples-passed/timestamp traffic and its limits; the general fixes and synthetic regressions are linked there.
- **64 KB-aligned large allocations** (`alloc_big_align`, `hle_os.c`; shared with Unicorn Overlord, [010069401ADB8000](../010069401ADB8000/README.md)): [Ring heaps cut from a pool at 64 KB alignment](../../../docs/reference/memory.md#ring-heaps-cut-from-a-pool-at-64-kb-alignment) and [the profile keys](../README.md#keys).
- **GLSLC outputs in `main`** (`shader_split.py` `glslc_shaders`, the control / code / length check): [GLSLC outputs in `main` and in loose files](../../../docs/reference/shaders.md#glslc-outputs-in-main-and-in-loose-files).
- **Single-stage programs paired with a cover** (`vs_pad`), **a shader cache key that covers the pairs** (`shader_key`, `pair_keyed`), **triangle fans and indexed quads** (a shared fan index buffer; CPU expansion at replay in `nvn_mtl.m`), **vertex streams 14 and 15 at slots 14 and 15** (`VB_SLOT`) and **packaged texture data copied at `TextureInitialize`**: [Single-stage programs paired with a cover](../../../docs/reference/shaders.md#single-stage-programs-paired-with-a-cover), [Shader cache key covering the pairs](../../../docs/reference/shaders.md#shader-cache-key-covering-the-pairs), [Triangle fans and indexed quads](../../../docs/reference/rendering.md#triangle-fans-and-indexed-quads), [Vertex streams 14 and 15 at Metal slots 14 and 15](../../../docs/reference/rendering.md#vertex-streams-14-and-15-at-metal-slots-14-and-15), [Packaged texture data copied at `TextureInitialize`](../../../docs/reference/nvn-objects.md#packaged-texture-data-copied-at-textureinitialize).
+33
View File
@@ -61,6 +61,8 @@ Used at `DM_FPS_TARGET=0` and for the interval the framework keeps while the VFR
- **Submissions in segments.** The game does not submit frame N+1 until frame N's GPU work is done: in the shrine the main thread sat in `nvnSyncWait` 59% of the time and the render thread in `nvnQueueSubmitCommands` 40%. That submit is the host's translation (about 3,600 draws and 347 passes, 7 ms), so the GPU ran 13–14 ms and then sat idle 7.7 ms a frame. The replay now commits every 64 draws between two passes (`sub_cut`, `DM_NVN_CUT`): GPU idle 7.7 → 3.8 ms, most of the rest the game's own work between its fence wait and the next submit (about 2.5 ms).
- **Shrine of Resurrection on the Mac (M4 Pro)**, after the above: docked (1600×900) GPU about 13 ms, frame p50 16.7 ms (60 fps; p90 17.6 ms), was 20.8 ms before segmenting; handheld (`DM_OPMODE=0`, 1280×720) GPU p50 9.3 ms and a steady 60. Host translation is 8 ms a frame (22,000 instructions per draw), spread over Metal's encoder creation and draw calls, argument encoding, retain / release and pool lookups. `DM_NVN_PASSGPU` is off under Metal 4, so GPU time cannot be attributed to passes. The frame loaded 376 MB and stored 432 MB of render-target data (325 of 347 passes load and store everything).
- **Outdoors (2026-10-21, the user's Great Plateau save, Mac M4 Pro).** A frame had about 386 render passes and loaded and stored about 900 MB of attachments (on an iPhone at 60 fps about 55 GB/s, near the memory's bandwidth). 220 passes ended only because the game updates the first 64 bytes of a 64 KB vertex uniform block (`0x70c38c0000`) before each draw group while its shaders read up to 544 bytes of it (the terrain pixel shader `eb0afe555b86da79` indexes the whole block). Partial-cover uniform shadows (`ubp_get`, `DM_NVN_UB_PARTIAL`) and `cb_extent.tsv` keep those passes open: 386 → about 165 passes, GPU p50 about 9.3 → 6.5 ms, submit 5.7 → 4.9 ms; `DM_NVN_UBP_CHECK=1` found every shadow equal to the GPU's memory (968,000 checked). Lazy clears fold the four clear-only passes before the G-buffer into its first pass (about 30 MB less a frame at 1600x900). The frame still loads and stores about 800 MB: the NintendoWare VFX GPU particles run one compute dispatch per emitter between draws (ten 800x450 half-resolution passes and two G-buffer passes, about 190 MB a frame; the G-buffer ones read the depth being drawn), and decals sample the G-buffer right after the game sets it again. Tried and dropped: `DM_NVN_TBDR=1` (10–15% fewer stores, more CPU, red ledger lines, deferred uniform updates off), `DM_NVN_BARRIER=game` (no gain), same-target continuation outside TBDR mode.
- **Cube map filter every tenth frame (current, resolved).** Every tenth frame starts by filtering the environment cube map: 132 one-draw passes into 64x64 to 8x8 faces (22 levels of 6 faces, `f8d9bbda1e0ebb0a` with `2ab186d450f5b32c` / `cfc85e532ccf8dd2`, game barrier `0x12` only between levels), so the G-buffer started 7–9 ms into that frame instead of 1.5 ms and the frame took about 5 ms more GPU time: the regular spike in the Metal HUD's frame-time line. The faces of a level now run as one layered pass ([passes into other layers of one texture level](../../../docs/reference/rendering.md#passes-into-other-layers-of-one-texture-level-run-as-one-layered-pass)): 284 → 173 passes in that frame; Zora's Domain at 900p, FPS 120 (M2 Max), frame p50 10.51 → 10.23 ms, p95 13.16 → 11.11 ms, mean window max 15.6 → 12.1 ms, GPU p95 13.0 → 8.7 ms, 95 → 98 fps.
- **Particle dispatches between effect draws (resolved).** The NintendoWare VFX GPU particles' one dispatch per emitter ended the G-buffer's and the half-resolution passes' effect draws into about 20 extra passes and compute encoders a frame. They now run ahead of the pass ([compute dispatches between the draws of a pass run ahead of it](../../../docs/reference/rendering.md#compute-dispatches-between-the-draws-of-a-pass-run-ahead-of-it)): 28 → 8 compute encoders, 149 → 132 render passes; same scene and switches as the cube map filter above, frame p50 9.92 → 8.97 ms, p95 10.70 → 9.98 ms, GPU p50 7.69 → 6.70 ms, 101 → 111 fps.
- Same-pass feedback: over a thousand draws a frame sample the G-buffer of their own pass (`4d3d5a4408c2e8d9`, `753d0169ae3ce4f0` and others on the 1600×900 G-buffer); snapshotting all of them took the Mac frame time from 16.6 to 36.6 ms, and none has shown magenta blocks on the iPhone so far. The decals that sample targets of their own pass read the stored content on the Mac; `DM_NVN_FB_SNAP=all` would snapshot them at a large cost.
- **Present snapshots (Mac).** The game's two swapchain textures held its CPU work to display ticks and the GPU at 444 MHz on an M2 Max ([present snapshots](../../../docs/reference/frame-rate.md#present-snapshots-the-games-swapchain-texture-free-at-once)). Judged by cost, the frame step stepped to 30 at a GPU time of 0.75 vsyncs. Zora's Domain from a Cemu save (120 Hz), frames on screen longer than 1/60 s by a quarter: 64 of 8,684 with neither, 41 with the GPU filler alone, 4 of 10,800 with snapshots alone (no step to 30 in 3 minutes, at 444 MHz throughout).
@@ -147,18 +149,49 @@ On this Mac the background prewarm also ends before the save loads, so the step
## Mac routes
### Final synchronization and pass validation (2026-10-26, current)
#158 and #213 with the review fixes, on base `c6c4fac`, were tested in outdoor gameplay on a Mac M4 Pro, macOS 27.0.1, at the supported 60 fps target and docked 1600x900 rendering. The supplied outdoor save was copied into an isolated seed; each run copied that seed again. The modules were regenerated with that rewriter, and all 19,592 Mac shaders rebuilt for chain `8393d85a4eae`. Every measured run used one retained host binary and the same source/save hashes. No game or graphics code changed during measurement.
The primary comparison ran **off / on / on / off**: all three switches (`DM_NVN_FAST_WAKE`, `DM_NVN_SLICE_MERGE`, `DM_NVN_CS_HOIST`) at 0 or 1, with encoding thread and present snapshots both 1, cut 64, silent audio, and one shared warmed pipeline archive. Each run reached frame 6,000; the measured window was frames 3,001-5,700 (2,700 frames), including a stationary interval, camera turn, walking and a jump. The windows had no new shader loads or pipelines. Values below average the two runs' means, except the p95 row, which gives the two runs separately:
| Metric | Off | On |
|---|---:|---:|
| Frame interval mean | 16.667 ms | 16.673 ms |
| GPU time mean | 7.051 ms | 7.144 ms |
| GPU time p95, each run | 11.155 / 11.782 ms | 11.708 / 12.018 ms |
| Submission CPU mean | 3.456 ms | 3.653 ms |
| Render passes per frame | 158.55 | 146.94 |
**Result:** pass savings survive (about 7.3%), but these runs do not demonstrate a frame/GPU timing speedup. Submission CPU cost rose about 0.20 ms (5.7%). Both pass optimizations activated; a typical 300-frame on window recorded 2,280 merged passes/layer continuations and roughly 1,400-1,800 moved dispatches. Separate stationary and post-movement windows also showed no consistent GPU win.
Single-switch follow-ups, plus another off baseline, gave GPU means of 6.963 ms (wake only), 7.518 ms (slice only), 6.977 ms (hoist only), and 6.970 ms (off). Hoist-only submission CPU was 3.645 ms versus 3.432 ms off. These are single runs, with a few library/pipeline events in the wake/hoist windows, so they are attribution checks rather than repeated timing proof. A further **direct-encoding, snapshots-off** pair gave GPU means 7.331 → 7.199 ms and submission CPU 4.630 → 4.864 ms, both still 60 fps. Blocking SyncWaits were rare in these 60 fps scenes; sparse diagnostic windows showed wake delay 0.090 → 0.064 ms, insufficient to confirm the original GPU-bound wake gain. Segment publishing remained enabled throughout: it has no runtime off switch.
All ten measured BotW runs exited cleanly, with no red line or crash. Screenshots in both modes showed the outdoor scene and the changed camera/character position without an obvious rendering regression. Wind, animation and second-based input holds cause nonzero image differences, including between repeated runs in the same mode; these are not bit-exact pixel checks. GPU clocks were not fixed. This does **not** reproduce the earlier M2 Max/Zora's Domain/120 fps workload or establish its reported gains, and it is not an iPhone test.
Artifacts in the NSP work folder: `run/perf-final-gameplay-20261026/`, with `summary.json`, per-run `run.json`, `source.json`, `frames.tsv`, `host.log`, `analysis*.json`, screenshots, seed, rewritten modules, retained host and reproduction scripts. The earlier DLC-menu and shrine pilots are excluded from the timing comparison. The general correctness changes are linked under “Found on this game”.
Before publishing, the fixes were integrated with main's pipeline archive and GPU profiling changes (`7753b7b`). The full graphics suite, 37 ordering checks in three encoding/switch modes, 8 CPU completion checks under TSAN, 52 bundle / 57 archive checks, 28 profiler checks, both game host builds, documentation checks and all 133 iOS source compiles passed. Two additional on/off frame-loop checks reached frame 5,000 and exited without a red line or crash. Their display link stalled and dropped frames, and the requested screenshots were not produced; they are execution checks only, excluded from the gameplay/timing evidence above. Their source identities, hosts, saves and logs are retained separately in `run/perf-main-integration-20261026/`. Main's subsequent `dac7ba6` changes the rewriter's Taiko matcher, not these host sources.
### Routes
- From the title menu into the autosave: `DM_KEYS=f2500:A,f2800:A,f3100:A,f3400:A` (control at frame ~3900).
- For the copied outdoor save with the downloaded-DLC notices: `DM_KEYS=f300:A,f500:A,f700:A,f900:A,f1100:B,f1300:A,f1500:A,f1700:A`; gameplay is visible by frame 2,600. The measured route adds `f4100:RRight~0.15,f4250:LUp~0.4,f4400:X~0.15,f4600:LDown~0.4`.
- The frame step under load: `DM_TEST_PRESENT_LAG=1,12` (12 ms added to every frame: 30 fps at 1.000 of real time).
## Found on this game
General host changes first needed here; each applies to every game:
- **Final #158/#213 gameplay validation:** [Current results](#final-synchronization-and-pass-validation-2026-10-26-current) distinguish the retained pass savings from the timing gains not demonstrated on this Mac.
- **Resolved: the IPA omitted pulling libraries with `pso_mtl4 on`, and explicitly including them failed under macOS Bash 3.2.** See [Pulling vertex inputs](../../../docs/reference/rendering.md#pulling-vertex-inputs).
- **Upscaling in the present** (`fx_encode`, `DM_PRESENT_UPSCALE`): [The present upscales through MetalFX](../../../docs/reference/rendering.md#the-present-upscales-through-metalfx).
- **Thermal cap; resolved movie recovery and acquire/present races** (`thermal_capped`, `DM_THERMAL_CAP`): [A thermally serious iPhone holds 30 fps](../../../docs/reference/frame-rate.md#a-thermally-serious-iphone-holds-30-fps).
- **Pipeline keys omit disabled blend factors** (`pso_key_norm`): [Pipeline keys omit unused colour state](../../../docs/reference/shaders.md#pipeline-keys-omit-unused-colour-state).
- **Resolved: GPU readback temporaries retained by a game thread's outer pool** (`mtl_read_texels`): [GPU readback without an autorelease pool](../../../docs/reference/memory.md#gpu-readback-without-an-autorelease-pool). Ported from MacNativeGames; checked here with a synthetic Metal readback lifetime test, without a new gameplay memory measurement.
- **Compute dispatches ahead of their pass** (`cs_hoist_try`, `DM_NVN_CS_HOIST`): [Compute dispatches between the draws of a pass run ahead of it](../../../docs/reference/rendering.md#compute-dispatches-between-the-draws-of-a-pass-run-ahead-of-it).
- **Slice-merged passes** (`sm_plan`, `sm_amp_get`, `DM_NVN_SLICE_MERGE`): [Passes into other layers of one texture level run as one layered pass](../../../docs/reference/rendering.md#passes-into-other-layers-of-one-texture-level-run-as-one-layered-pass).
- **The host frame step in whole vsyncs** (`nvn_fstep.h`, `fstep_tick`, `FRAME_STEP` stubs, `fps_target`), explained in full under [Frame rate](#the-frame-step-current); its probes (`nvn_fstep_gpu_slowed`, `nvn_fstep_same`): [development.md, Power](../../../docs/development.md#power).
- **Real GPU timestamps per segment** (`gpu_timestamps`, `rep_ticks`), **submissions in segments** (`sub_cut`), **deferred texture binds** (`bh_fresh`), **duplicate colour slots** (`rt_dup_slot`), **cube textures as 2D arrays** (`tex_fit`), **compressed render targets through shadows** (`bcrt_attach` / `bcrt_flush`), **pool lookups** (`pool_find`), **memory fixes** (`defer_drain`'s pool, `g_ghost`, growing pool table), **`nvnTextureCompare` with the channel mapping**, **first-use fill of flushed storage** (`nvn_flushed`, `tex_init_fill`), **partial-cover uniform shadows** (`ubp_get`, `cb_extent.tsv`): [GPU timestamps per submission segment](../../../docs/reference/frame-rate.md#gpu-timestamps-per-submission-segment), [Submissions handed to the GPU in segments](../../../docs/reference/frame-rate.md#submissions-handed-to-the-gpu-in-segments), [Deferred texture binds read at execution](../../../docs/reference/shaders.md#deferred-texture-binds-read-at-execution), [The same texture attached at two colour slots](../../../docs/reference/rendering.md#the-same-texture-attached-at-two-colour-slots), [Cube textures held as 6-layer 2D arrays](../../../docs/reference/rendering.md#cube-textures-held-as-6-layer-2d-arrays), [Compressed textures as render targets through an uncompressed shadow](../../../docs/reference/rendering.md#compressed-textures-as-render-targets-through-an-uncompressed-shadow), [Address to memory pool lookups by binary search](../../../docs/reference/frame-rate.md#address-to-memory-pool-lookups-by-binary-search), [Deferred-release drain without an autorelease pool](../../../docs/reference/memory.md#deferred-release-drain-without-an-autorelease-pool), [Dead texture serials kept in a ghost table](../../../docs/reference/nvn-objects.md#dead-texture-serials-kept-in-a-ghost-table), [Memory-pool tracking table grows in blocks](../../../docs/reference/nvn-objects.md#memory-pool-tracking-table-grows-in-blocks), [`nvnTextureCompare` compares the channel mapping](../../../docs/reference/rendering.md#nvntexturecompare-compares-the-channel-mapping), [First-use fill of storage the game flushed](../../../docs/reference/nvn-objects.md#first-use-fill-of-storage-the-game-flushed), [Partial-cover uniform shadows keep passes open](../../../docs/reference/rendering.md#partial-cover-uniform-shadows-keep-passes-open).
- **Display-link waits per present interval** (`mtl_link_wait_tick`): [Frames of a dynamic present interval held on screen and at the acquire](../../../docs/reference/frame-rate.md#frames-of-a-dynamic-present-interval-held-on-screen-and-at-the-acquire).
+20
View File
@@ -65,12 +65,32 @@ The game plays VP9 WebM movies through the SDK's own demuxer and a decoder in na
## Mac routes
### Final gameplay validation (2026-10-26, current)
The final #158/#213 review fixes on base `c6c4fac` were tested in the saved playable cave room, beyond the earlier movie-only smoke coverage. On a Mac M4 Pro (macOS 27.0.1), four isolated copies of `run/save` ran **off / on / on / off** with all three optimization switches at 0 or 1. Encoding thread and present snapshots stayed 1, cut 64, silent audio, current `8393d85a4eae` shaders and one warmed pipeline archive. Host/source/save hashes were identical between modes. Each run reached frame 4,600 and exited cleanly; frames 2,001-4,200 supplied 2,200 timing samples per run, with no new shader loads or pipelines. The route includes standing, walking, shooting and jumping; screenshots verified those actions and both modes' rendered scene.
| Metric (mean of the two runs) | Off | On |
|---|---:|---:|
| Frame interval | 16.689 ms | 16.690 ms |
| GPU time | 4.007 ms | 4.014 ms |
| Submission CPU | 0.569 ms | 0.604 ms |
Neither slice merging nor compute hoisting activated in this room. GPU results varied between repeat runs (on means 3.829 and 4.199 ms) and show no reproducible gain; submission CPU rose about 0.035 ms. Both modes stayed about 59.9 fps without a red line or crash. HUD fading and second-based input holds vary the pass/draw counts and camera position slightly, so screenshot comparisons establish no obvious regression, not pixel identity or an identical workload at every frame. GPU clocks were not fixed. This is broader NVN gameplay coverage, not evidence of a pass-optimization benefit here; [BotW's active-switch comparison](../01007EF00011E000/README.md#final-synchronization-and-pass-validation-2026-10-26-current) records the performance result where both activate. iOS runtime validation remains open.
Artifacts in the NSP work folder's `run/perf-final-gameplay-20261026/`: source/save/host identities, `summary.json`, per-run logs, frame timings, analyses, screenshots and reproduction scripts. An earlier movement pilot's AudioUnit callback warning is excluded; all measured runs use the same silent sink. Reproduction starts with `DM_KEYS=4:A*20/0.7`, then frame-anchored right/shoot/jump/left sequences beginning at frame 3,100 (the complete key string is in each `run.json`).
After integration with main's pipeline archive and profiling changes (`7753b7b`), two additional on/off frame-loop checks reached frame 4,400 and exited without a red line or crash. Display-link stalls and missing requested screenshots exclude these from the gameplay/timing evidence above. Separate source identities, hosts, saves and logs are in `run/perf-main-integration-20261026/`; [BotW's integration note](../01007EF00011E000/README.md#final-synchronization-and-pass-validation-2026-10-26-current) records the passing integration tests and the subsequent unrelated main update.
### Earlier smoke routes
- **Synchronization smoke check (2026-10-26, Mac M4 Pro):** #158 and #213 combined with their review fixes, current `8393d85a4eae` Mac libraries, final movie-handoff modules, isolated fresh saves. A 60 s run with fast wake, both pass optimizations and deferred encoding completed six movie configurations and 3,397 presented frames; the comparison with all three optimizations disabled and direct encoding completed six and 3,334. No red line or crash in either successful run; frame 1,800 was visually checked in the opening movie. This covers movie/render transitions, not gameplay or a performance comparison, and neither pass optimization activated. Logs, saves and shots: `run/movie-sync/perf-order-on-0/` and `perf-order-off-fixed-0/` in the NSP work folder. An earlier comparison aborted because the relocated host lacked `DM_NVN_SHMAP`; the successful comparison sets that absolute path explicitly. General completion and dependency fixes: [SyncWait completion](../../../docs/reference/nvn-objects.md#the-games-syncwait-wakes-on-the-gpus-own-signal-of-its-fence), [slice merging](../../../docs/reference/rendering.md#passes-into-other-layers-of-one-texture-level-run-as-one-layered-pass) and [compute hoisting](../../../docs/reference/rendering.md#compute-dispatches-between-the-draws-of-a-pass-run-ahead-of-it). iOS runtime validation remains open.
- One `DM_KEYS` switch runs through the title, a new game and the story recap ([05, Scripted input and thread scheduling](../../../docs/agents/05-debugging.md#scripted-input-and-thread-scheduling)). The key string itself is not in the docs.
## Found on this game
General host changes first needed here; each applies to every game:
- **Synchronization review gameplay validation:** [Final gameplay validation](#final-gameplay-validation-2026-10-26-current) records the #158/#213 comparison and its limits; the general fixes and synthetic regressions are linked under the earlier smoke routes.
- **Patch areas before the module base** for modules with a giant bss (negative `PATCH_OFF`, `FILE_IMAGE_LEN`, `IMAGE_SIZE`): [Patch area before the module base for a large bss](../../../docs/reference/rewriter.md#patch-area-before-the-module-base-for-a-large-bss).
- **Used memory counts the bss of front-patch modules** (`big_bss_bytes`): [Heap sized from the application pool less used memory](../../../docs/reference/memory.md#heap-sized-from-the-application-pool-less-used-memory).
- **`sleep_precise`** against macOS timer slack, **same-core wake handoffs** (`svc_thread` / `dm_yield_to_core_peers`): [Timer slack in guest sleeps (`sleep_precise`)](../../../docs/reference/threads.md#timer-slack-in-guest-sleeps-sleep_precise), [Same-core wake handoffs](../../../docs/reference/threads.md#same-core-wake-handoffs).
+2
View File
@@ -184,6 +184,8 @@ Comparison and testing:
| `DM_NVN_TBDR=1` | Turns on the pass optimizations of the replay layer (store / load elision, pass merging). They are off by default: they infer which textures a draw reads from its bound slots, which bindless textures defeat ([docs/agents/04-graphics.md](../../docs/agents/04-graphics.md) section 4) |
| `DM_NVN_UNWRITTEN_DC=0` | With `DM_NVN_TBDR=1`, turns off the single item "attachments not written in this pass are not stored" |
| `DM_NVN_FUSE_CHECK=1` | Same-frame check of pass merging; logs the comparison result every time |
| `DM_NVN_CS_HOIST=0` | A compute dispatch between the draws of a pass ends it (by default it runs ahead of the pass when that changes nothing the draws before it read: [reference](../../docs/reference/rendering.md#compute-dispatches-between-the-draws-of-a-pass-run-ahead-of-it)) |
| `DM_NVN_SLICE_MERGE=0` | Passes into other layers of one array or cube texture level stay apart (by default they run as one layered pass: [reference](../../docs/reference/rendering.md#passes-into-other-layers-of-one-texture-level-run-as-one-layered-pass)) |
| `DM_FPS=30` or `60` | Pins the target frame rate |
| `DM_NVN_BARRIER=game`, `DM_NVN_CUT` | Pass ordering and segment commits: see [docs/agents/04-graphics.md](../../docs/agents/04-graphics.md) section 7 |
| `DM_PRESENT_HZ=N` | The frequency of presentation pacing (default 60); 0 means no frame limit and is for performance profiling only |
+7 -4
View File
@@ -1391,10 +1391,13 @@ static void power_line(uint64_t frame) {
void mtl_lc_stats(uint64_t *, uint64_t *); static uint64_t plt, pll; uint64_t lt, ll; mtl_lc_stats(&lt, &ll);
void mtl_ubp_why(uint64_t w[5]); static uint64_t pw5[5]; uint64_t w5[5]; mtl_ubp_why(w5);
uint64_t ld, st, fu, udc = mtl_udc_saved_bytes(); mtl_bw(&ld, &st, &fu);
LOG("[nvn] residency set up to frame %llu: %s; draw-pass entry barriers full %llu / saved %llu; split submits %llu (%llu after buffer → texture copies); passes broken by copies / buffer updates %llu; in-pass uniform updates deferred %llu, passes ended to land them for a read %llu, partly covered blocks read from memory %llu (cached %llu, refused %llu: other GPU writes %llu, earlier submission running %llu, copy or fill queued %llu, size %llu, unreadable %llu); waiting clears taken by a later pass %llu, landed in passes of their own %llu; "
"attachments per frame load %.0f MB / store %.0f MB, load-all store-all passes %.0f, saved by not storing unwritten attachments %.1f MB", frame, rs, (unsigned long long)(bf - pbf), (unsigned long long)(bs - pbs),
(unsigned long long)(cut - pcut), (unsigned long long)(ccut - pccut), (unsigned long long)(bsp - pbsp), (unsigned long long)(ud - pud), (unsigned long long)(uc - puc), (unsigned long long)(pm - ppm), (unsigned long long)(pr - ppr), (unsigned long long)(pf_ - ppf), (unsigned long long)(w5[1] - pw5[1]), (unsigned long long)(w5[2] - pw5[2]), (unsigned long long)(w5[3] - pw5[3]), (unsigned long long)(w5[0] - pw5[0]), (unsigned long long)(w5[4] - pw5[4]), (unsigned long long)(lt - plt), (unsigned long long)(ll - pll),
(ld - pld) / 1048576.0 / nf, (st - pst) / 1048576.0 / nf, (fu - pfu) / nf, (udc - pudc) / 1048576.0 / nf); pbs = bs; pbf = bf; pcut = cut; pccut = ccut; pbsp = bsp; pud = ud; puc = uc; ppm = pm; ppr = pr; ppf = pf_; memcpy(pw5, w5, sizeof pw5); plt = lt; pll = ll;
void mtl_sm_stats(uint64_t *, uint64_t *, uint64_t *); static uint64_t psmp, psmc, psmw; uint64_t smp, smc, smw; mtl_sm_stats(&smp, &smc, &smw);
void mtl_cshoist_stats(uint64_t *, uint64_t *, uint64_t *); static uint64_t pchm, pchd, pchs[8]; uint64_t chm, chd, chs[8]; mtl_cshoist_stats(&chm, &chd, chs);
LOG("[nvn] residency set up to frame %llu: %s; draw-pass entry barriers full %llu / saved %llu; split submits %llu (%llu after buffer → texture copies); passes broken by copies / buffer updates %llu; in-pass uniform updates deferred %llu, passes ended to land them for a read %llu, partly covered blocks read from memory %llu (cached %llu, refused %llu: other GPU writes %llu, earlier submission running %llu, copy or fill queued %llu, size %llu, unreadable %llu); waiting clears taken by a later pass %llu, landed in passes of their own %llu; slice-merged passes %llu (layer switches taken in %llu, waits for an amplified pipeline %llu); compute dispatches moved ahead of their pass %llu (sampling its depth %llu; "
"scans stopped by a draw %llu, a dispatch %llu, a read of its writes %llu, depth written before it %llu, another op %llu); attachments per frame load %.0f MB / store %.0f MB, load-all store-all passes %.0f, saved by not storing unwritten attachments %.1f MB", frame, rs, (unsigned long long)(bf - pbf), (unsigned long long)(bs - pbs),
(unsigned long long)(cut - pcut), (unsigned long long)(ccut - pccut), (unsigned long long)(bsp - pbsp), (unsigned long long)(ud - pud), (unsigned long long)(uc - puc), (unsigned long long)(pm - ppm), (unsigned long long)(pr - ppr), (unsigned long long)(pf_ - ppf), (unsigned long long)(w5[1] - pw5[1]), (unsigned long long)(w5[2] - pw5[2]), (unsigned long long)(w5[3] - pw5[3]), (unsigned long long)(w5[0] - pw5[0]), (unsigned long long)(w5[4] - pw5[4]), (unsigned long long)(lt - plt), (unsigned long long)(ll - pll), (unsigned long long)(smp - psmp), (unsigned long long)(smc - psmc), (unsigned long long)(smw - psmw), (unsigned long long)(chm - pchm), (unsigned long long)(chd - pchd),
(unsigned long long)(chs[1] - pchs[1]), (unsigned long long)(chs[2] - pchs[2]), (unsigned long long)(chs[3] - pchs[3]), (unsigned long long)(chs[7] - pchs[7]), (unsigned long long)(chs[5] - pchs[5]),
(ld - pld) / 1048576.0 / nf, (st - pst) / 1048576.0 / nf, (fu - pfu) / nf, (udc - pudc) / 1048576.0 / nf); pbs = bs; pbf = bf; pcut = cut; pccut = ccut; pbsp = bsp; pud = ud; puc = uc; ppm = pm; ppr = pr; ppf = pf_; memcpy(pw5, w5, sizeof pw5); plt = lt; pll = ll; psmp = smp; psmc = smc; psmw = smw; pchm = chm; pchd = chd; memcpy(pchs, chs, sizeof pchs);
char tc[400]; dm_thread_cpu(tc, sizeof tc, s);
LOG("[nvn] threads up to frame %llu: CPU %s", frame, tc); }
else { char yt[192], rs[400], tc[400]; dm_yield_top(yt, sizeof yt); mtl_res_stats(rs, sizeof rs); dm_thread_cpu(tc, sizeof tc, 1); } // the first call only takes the baselines
+333 -33
View File
@@ -113,11 +113,11 @@ static uint64_t g_pq_read;
static id<MTLResidencySet> g_res;
static int g_res_dirty;
static id<MTLSharedEvent> g_ev;
static uint64_t g_fence; // Highest committed fence value (incremented exclusively by the submit thread)
static _Atomic uint64_t g_fence; // Highest committed fence value (incremented exclusively by the submit thread)
static int g_cut_copy; // a buffer → texture copy was encoded since the last commit: the next pass boundary commits the segment (sub_cut)
static pthread_mutex_t g_mu = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t g_cv = PTHREAD_COND_INITIALIZER;
static uint64_t g_done; // Highest completed fence value (written by the completion callback)
static _Atomic uint64_t g_done; // GPU complete and all CPU-visible effects through this fence published
static uint64_t g_arena_made;
static int g_allocprof = -1; static uint64_t g_ap_bytes[64], g_ap_fn[8];
#define AP_B uint64_t _ap0 = g_allocprof > 0 ? g_dev.currentAllocatedSize : 0
@@ -139,16 +139,25 @@ static inline void ws_put(int site, uint64_t t0, int blocked) {
if (blocked) { atomic_fetch_add_explicit(&g_ws_blk[site], 1, memory_order_relaxed); atomic_fetch_add_explicit(&g_ws_ns[site], ws_t0() - t0, memory_order_relaxed); }
}
// For the game's waits: when each fence's command buffer ended on the GPU and when its feedback handler ran (CACurrentMediaTime), so a wait that blocked tells its wake-up delay
static struct { _Atomic uint64_t fence; double gpu_end, fb; } g_fend[64];
static struct { _Atomic uint64_t fence, wfence; double gpu_end, fb, wake; _Atomic int done; } g_fend[64];
static _Atomic uint64_t g_ws_wake_n, g_ws_wake_ns, g_ws_fb_ns;
static void ws_wake(uint64_t fence) {
unsigned k = fence % 64; if (atomic_load(&g_fend[k].fence) != fence || g_fend[k].gpu_end <= 0) return;
double now = CACurrentMediaTime(); atomic_fetch_add(&g_ws_wake_n, 1);
atomic_fetch_add(&g_ws_wake_ns, (uint64_t)((now - g_fend[k].gpu_end) * 1e9)); atomic_fetch_add(&g_ws_fb_ns, (uint64_t)((g_fend[k].fb - g_fend[k].gpu_end) * 1e9));
static void ws_wake_pair(unsigned k) { // both halves known: the wake-up delay after the GPU end, counted once
if (atomic_load(&g_fend[k].fence) != atomic_load(&g_fend[k].wfence) || atomic_exchange(&g_fend[k].done, 1)) return;
atomic_fetch_add(&g_ws_wake_n, 1); atomic_fetch_add(&g_ws_wake_ns, (uint64_t)(fmax(0, g_fend[k].wake - g_fend[k].gpu_end) * 1e9)); atomic_fetch_add(&g_ws_fb_ns, (uint64_t)(fmax(0, g_fend[k].fb - g_fend[k].gpu_end) * 1e9));
}
static void ws_wake(uint64_t fence) { // a blocking SyncWait on this fence has woken
unsigned k = fence % 64; g_fend[k].wake = CACurrentMediaTime(); atomic_store(&g_fend[k].wfence, fence); ws_wake_pair(k);
static int once; if (!once++) { char nm[64] = ""; pthread_getname_np(pthread_self(), nm, sizeof nm); MLOG("[waits] first blocking SyncWait on the newest fence: thread '%s'", nm); }
}
static void ws_fb(uint64_t fence, double gpu_end) { // the fence's feedback handler
unsigned k = fence % 64; if (atomic_load(&g_fend[k].fence) != fence) { atomic_store(&g_fend[k].done, 0); g_fend[k].gpu_end = gpu_end; g_fend[k].fb = CACurrentMediaTime(); atomic_store(&g_fend[k].fence, fence); }
ws_wake_pair(k);
}
static _Atomic uint64_t g_ws_post[5];
static void ws_report(uint64_t frame, uint64_t frames) {
char line[2048]; size_t o = 0;
{ uint64_t e = atomic_exchange(&g_ws_post[1], 0), sp = atomic_exchange(&g_ws_post[2], 0), ts = atomic_exchange(&g_ws_post[3], 0), oc = atomic_exchange(&g_ws_post[4], 0);
if (e + sp + ts + oc) o += snprintf(line + o, sizeof line - o, " CPU-side writes at GPU end: events %.2f, samples-passed reports %.2f, timestamp reports %.2f, other counter reports %.2f;", (double)e / frames, (double)sp / frames, (double)ts / frames, (double)oc / frames); }
uint64_t wn = atomic_exchange(&g_ws_wake_n, 0), wns = atomic_exchange(&g_ws_wake_ns, 0), fns = atomic_exchange(&g_ws_fb_ns, 0);
if (wn) o += snprintf(line + o, sizeof line - o, " newest-fence SyncWait wakes %.3f ms after the GPU end (feedback handler at %.3f ms);", wns / 1e6 / wn, fns / 1e6 / wn);
for (int i = 0; i < WS_N; i++) { uint64_t n = atomic_exchange(&g_ws_n[i], 0), b = atomic_exchange(&g_ws_blk[i], 0), ns = atomic_exchange(&g_ws_ns[i], 0);
@@ -1352,6 +1361,50 @@ int mtl_gpu_span(uint64_t f, uint64_t *t0, uint64_t *t1) {
static void rp_flush(void);
static const char *cctx_snap(void); // What is in this submission (printed into the failure line on error; defined before mtl_exec)
static void lc_land(void);
// A submission's CPU-side writes at its end (events, samples-passed and counter reports): run once, by its feedback handler or, when a game's SyncWait wakes on the
// GPU's own signal of the fence first, by that waiter (DM_NVN_FAST_WAKE, on by default): the feedback handler comes 0.26 ms after the GPU end on an M2 Max
typedef struct { _Atomic uint64_t fence; _Atomic int claimed; int n; post_op *ops; seg_clk *sc; uint32_t seg_last; uint64_t t_commit; } post_rec; // t_commit: guest ns when it was committed
#define POST_RING 128
static post_rec g_postr[POST_RING];
static pthread_mutex_t g_post_mu = PTHREAD_MUTEX_INITIALIZER;
// A successful claim holds the lock until post_run finishes. Ownership alone
// cannot publish completion: another waiter must see the actual writes first.
static int post_claim(post_rec *r, uint64_t fence) {
pthread_mutex_lock(&g_post_mu);
if (r->fence != fence || r->claimed) { pthread_mutex_unlock(&g_post_mu); return 0; }
return 1;
}
static void post_run(post_rec *r, uint64_t c0, uint64_t c1) {
int n = r->n; post_op *ops = r->ops; seg_clk *sc = r->sc; uint32_t seg_last = r->seg_last;
for (int i = 0; i < n; i++) {
if (ops[i].kind == 1) { if (ops[i].mode == 1) *ops[i].val += ops[i].v; else *ops[i].val = ops[i].v; }
else if (ops[i].kind == 3) { // Samples passed: the sum of all slots since the reset
vis_list *l = ops[i].vis; uint64_t sum = 0;
for (int k = 0; k < l->n; k++) { sum += ((const uint64_t *)((__bridge id<MTLBuffer>)l->s[k].buf).contents)[l->s[k].slot]; CFRelease(l->s[k].buf); }
free(l); uint64_t rec[2] = {sum, gpu_ticks()}; memcpy((void *)ops[i].addr, rec, sizeof rec);
}
else { uint64_t t = rep_ticks(&ops[i], sc, seg_last, c0, c1), rec[2] = {(ops[i].type == 0 || ops[i].type == 0x10) && !DM_GAME_RENDER_COND ? t : 0, t}; memcpy((void *)ops[i].addr, rec, sizeof rec); } // render_cond: a timestamp report's counter field is 0, as on the GPU (see rc_value)
}
free(ops); seg_clk_put(sc); r->ops = NULL; r->sc = NULL;
atomic_store(&r->claimed, 1);
pthread_mutex_unlock(&g_post_mu);
}
// Both GPU feedback and an early waiter complete records in submission order.
// Delayed callbacks carry their fence, never the identity of a recyclable slot.
static void post_complete(uint64_t fence, uint64_t c0, uint64_t c1) {
uint64_t now = guest_ns();
for (uint64_t f = g_done + 1; f <= fence; f++) {
post_rec *r = &g_postr[f % POST_RING];
if (!post_claim(r, f)) continue;
uint64_t start = r->t_commit;
if (r->sc && r->seg_last > 0 && r->seg_last <= SEG_CLK_N) {
uint64_t pe = atomic_load(&r->sc->t[r->seg_last - 1][1]); if (pe > start) start = pe;
}
if (start > now) start = now;
post_run(r, f == fence && c1 ? c0 : seg_ts() ? start : 0, f == fence && c1 ? c1 : seg_ts() ? now : 0);
}
pthread_mutex_lock(&g_mu); if (fence > g_done) g_done = fence; pthread_cond_broadcast(&g_cv); pthread_mutex_unlock(&g_mu);
}
uint64_t mtl_sub_commit(void) {
lc_land(); // clears still waiting (the command list ends with them)
defer_fallback(); // Deferred merged pass: at the end of the submission there are still P draws not replayed (the normal path never gets here)
@@ -1363,7 +1416,8 @@ uint64_t mtl_sub_commit(void) {
gw_commit(fence);
if (g_vis_ring_i >= 0) g_vis_ring[g_vis_ring_i].fence = fence;
// CPU-visible effects after completion: copy the op table into the callback
int n = g_npost; post_op *ops = n ? malloc(n * sizeof *ops) : NULL;
int n = g_npost; post_op *ops = n ? malloc(n * sizeof *ops) : NULL; post_rec *pr = &g_postr[fence % POST_RING];
if (ws_on()) for (int i = 0; i < n; i++) { int k = g_post[i].kind == 2 ? (g_post[i].type == 0 || g_post[i].type == 0x10 ? 3 : 4) : g_post[i].kind == 3 ? 2 : 1; atomic_fetch_add(&g_ws_post[k], 1); } // 1 event, 2 samples passed, 3 timestamp, 4 other counter
if (n) memcpy(ops, g_post, n * sizeof *ops);
NSArray<dispatch_block_t> *commit_blocks = [g_commit_blocks copy];
MTL4CommitOptions *o = [MTL4CommitOptions new];
@@ -1371,22 +1425,16 @@ uint64_t mtl_sub_commit(void) {
uint64_t gpu_f = atomic_load(&g_cur_frame); gpu_tag_begin(gpu_f);
NSString *cctx = @(cctx_snap());
seg_clk *sc = g_sclk; uint32_t seg_last = g_seg; g_sclk = NULL;
pthread_mutex_lock(&g_post_mu);
if (!atomic_load(&pr->claimed) && pr->fence) MDIE("post ring: the writes of fence %llu never ran", (unsigned long long)pr->fence);
pr->n = n; pr->ops = ops; pr->sc = sc; pr->seg_last = seg_last; pr->t_commit = guest_ns(); atomic_store(&pr->claimed, 0); atomic_store(&pr->fence, fence);
pthread_mutex_unlock(&g_post_mu);
[o addFeedbackHandler:^(id<MTL4CommitFeedback> cb) {
gpu_tag_done(gpu_f, cb.GPUStartTime, cb.GPUEndTime);
if (ws_on()) { unsigned fk = fence % 64; g_fend[fk].gpu_end = cb.GPUEndTime; g_fend[fk].fb = CACurrentMediaTime(); atomic_store(&g_fend[fk].fence, fence); }
if (ws_on()) ws_fb(fence, cb.GPUEndTime);
uint64_t c0 = 0, c1 = 0; if (seg_ts()) { c0 = gpu_s_guest_ns(cb.GPUStartTime); c1 = gpu_s_guest_ns(cb.GPUEndTime); }
if (cb.error) { atomic_store(&g_last_commit_error, 1); static int n, nv; int victim = [cb.error.description containsString:@"InnocentVictim"]; if (victim ? nv++ < 1 : n++ < 8) MLOG("[RED] command buffer error details fence=%llu: %s | GPU %.3f→%.3f", (unsigned long long)fence, cb.error.description.UTF8String, cb.GPUStartTime, cb.GPUEndTime); if (!victim) MLOG("[RED] state of the failed submission fence=%llu: %s", (unsigned long long)fence, cctx.UTF8String); MLOG("[RED] command buffer execution error: %s", cb.error.localizedDescription.UTF8String); }
for (int i = 0; i < n; i++) {
if (ops[i].kind == 1) { if (ops[i].mode == 1) *ops[i].val += ops[i].v; else *ops[i].val = ops[i].v; }
else if (ops[i].kind == 3) { // Samples passed: the sum of all slots since the reset
vis_list *l = ops[i].vis; uint64_t sum = 0;
for (int k = 0; k < l->n; k++) { sum += ((const uint64_t *)((__bridge id<MTLBuffer>)l->s[k].buf).contents)[l->s[k].slot]; CFRelease(l->s[k].buf); }
free(l); uint64_t rec[2] = {sum, gpu_ticks()}; memcpy((void *)ops[i].addr, rec, sizeof rec);
}
else { uint64_t t = rep_ticks(&ops[i], sc, seg_last, c0, c1), rec[2] = {(ops[i].type == 0 || ops[i].type == 0x10) && !DM_GAME_RENDER_COND ? t : 0, t}; memcpy((void *)ops[i].addr, rec, sizeof rec); } // render_cond: a timestamp report's counter field is 0, as on the GPU (see rc_value)
}
free(ops); seg_clk_put(sc);
pthread_mutex_lock(&g_mu); if (fence > g_done) g_done = fence; pthread_cond_broadcast(&g_cv); pthread_mutex_unlock(&g_mu);
post_complete(fence, c0, c1);
defer_drain();
}];
void arena_commit(uint64_t fence); arena_commit(fence);
@@ -1407,9 +1455,22 @@ uint64_t mtl_sub_commit(void) {
uint64_t mtl_last_fence(void) { return g_fence; }
int mtl_last_commit_error(void) { return atomic_load(&g_last_commit_error); }
static int fast_wake(void) { static int v = -1; if (v < 0) { const char *e = getenv("DM_NVN_FAST_WAKE"); v = !(e && !strcmp(e, "0")); MLOG("game SyncWaits wake on the GPU's signal of the fence%s (DM_NVN_FAST_WAKE)", v ? "" : ": off, on the feedback handler"); } return v; }
// A SyncWait woken by the GPU's signal: the CPU-side writes of every submission up to the fence that no feedback handler has run yet, run here. The fence's last segment ended
// now (c1); it started at the later of its commit and the previous segment's end (c0), as near as the CPU can tell without its feedback
static void post_early(uint64_t fence) {
post_complete(fence, 0, 0);
}
static int mtl_wait_at(uint64_t fence, uint64_t timeout_ns, int site) {
if (g_done >= fence) { if (ws_on()) ws_put(site, 0, 0); return 1; }
uint64_t w0 = ws_t0();
if ((site == WS_SYNC || site == WS_SYNC_NEWEST || site == WS_ACQ) && fast_wake() && g_ev && fence <= g_fence) { // the game's own wait: wake on the GPU's signal (rec_signal(g_ev, fence) after each submission)
uint64_t ms = timeout_ns / 1000000ull; if (ms > 3600000ull) ms = 3600000ull;
if ([g_ev waitUntilSignaledValue:fence timeoutMS:ms]) post_early(fence);
int ok = g_done >= fence;
if (ws_on()) { ws_put(site, w0, 1); ws_wake(fence); }
if (ok || timeout_ns < 1000000ull) return ok;
}
struct timespec dl; clock_gettime(CLOCK_REALTIME, &dl);
uint64_t ns = (uint64_t)dl.tv_nsec + (timeout_ns > 3600000000000ull ? 3600000000000ull : timeout_ns);
dl.tv_sec += ns / 1000000000ull; dl.tv_nsec = ns % 1000000000ull;
@@ -2374,6 +2435,16 @@ static int enc_open(void) { return g_enc != nil; }
// Target info of the current render pass: determined once from the attachments at rp_begin and shared by all draws in the pass (pipeline key, depth-stencil, viewport / scissor all depend only on it)
static struct { uint32_t cfmt[8], dfmt, sfmt, samples; int hd, hs; NSUInteger tw, th; uint64_t serial, reg_gen, ser_gen, range_dirty; int range_external, range_depth_nonro, layered; __unsafe_unretained id<MTLTexture> range_color, range_depth; } g_pass;
static int g_rp_layered; // the pass about to open is for a vertex shader that writes the layer (draw → rp_begin_ex)
// Slice-merged passes (DM_NVN_SLICE_MERGE, on by default; 0 off): consecutive passes that each draw into another layer of one level of the same array or cube texture,
// with nothing between them but state, uniform updates and draws (no game barrier, clear, copy or dispatch: sm_scan), run as one layered pass. Each draw reaches its layer through a vertex
// amplification view mapping (count 1, renderTargetArrayIndexOffset), which needs pipelines built with maxVertexAmplificationCount 2 (sm_amp_get; a pipeline with the default 1 ignores
// the mapping and every draw lands in the first layer). On Apple GPUs a render pass costs about 35-40 us whatever it draws (M2 Max): Breath of the Wild filters its environment cube map
// every tenth frame in 132 one-draw passes, 22 levels of 6 faces, and that frame's GPU time grew by about 5 ms
static struct { int on, lo, hi; } g_sm_plan; // set by draw() for the pass rp_begin_ex is about to open
static int g_sm_pass, g_sm_lo, g_sm_hi, g_sm_cur, g_sm_applied; // the open pass is slice-merged: its attachment holds layers lo..hi; cur: the layer the game's target names now; applied: the offset set on the encoder (-1 none)
static uint64_t g_sm_npass, g_sm_ncont, g_sm_nwait; // slice-merged passes, layer switches taken into them, draws that left one because their pipeline was not built yet
static int g_pso_amp; // pso_get: the pipeline of a slice-merged pass (sm_amp_get)
void mtl_sm_stats(uint64_t *np, uint64_t *nc, uint64_t *nw) { *np = g_sm_npass; *nc = g_sm_ncont; *nw = g_sm_nwait; }
static id<MTLRenderPipelineState> g_enc_pso; static id<MTLDepthStencilState> g_enc_dss; // Already set on this encoder (not set again if identical)
// Dynamic state already sent on the open render encoder: each draw used to send all nine (stencil reference, viewport, scissor, cull, winding, fill, depth bias, depth clip, blend colour);
// now a value is sent only when it differs. g_ed.ok holds one bit per item and is cleared for every new encoder and after a clear triangle (which sets its own)
@@ -2730,7 +2801,7 @@ static void rp_end(void) { if (g_enc) { AP_B;
int ranged=g_rt_range_active && !g_rt_range_pending;
#endif
sd_finish(); enc_r_end_pass(g_enc); g_gp_bw_l0 = g_bw_load; g_gp_bw_s0 = g_bw_store; // the profile's per-pass ledger counts from the previous pass end, profiled or not
zskip_commit_clears(); g_enc = nil; g_rt_merged = 0; g_rt_range_active = g_rt_range_pending = 0; pg_end_render(); ts_after(); ms_after(); if (g_lp.count) lp_finish(); fz_end();
zskip_commit_clears(); g_enc = nil; g_rt_merged = 0; g_sm_pass = 0; g_rt_range_active = g_rt_range_pending = 0; pg_end_render(); ts_after(); ms_after(); if (g_lp.count) lp_finish(); fz_end();
if (g_nubd && g_nubd >= ubd_lazy()) ubd_flush(); // the inline updates deferred in this pass, in order (see mtl_sub_update); across passes they wait for the next copy or dispatch (blit_raw)
if (g_nbcrt) bcrt_flush(); // blocks drawn into shadows go back into their compressed textures
#ifdef NVN_RT_RANGE_TEST
@@ -2865,7 +2936,7 @@ static void bare_dims(NSUInteger *w, NSUInteger *h);
static id<MTL4RenderCommandEncoder> rp_begin_ex_(int clear_only) {
if (g_enc) return g_enc;
memset(g_zskip_cand_ok, 0, sizeof g_zskip_cand_ok);
g_enc_mode = clear_only; g_pass_swrite = 0;
g_enc_mode = clear_only; g_pass_swrite = 0; g_sm_pass = 0;
if (!R.has_rt) { RED_ONCE(3, "no render target set before the draw"); return nil; }
if (g_imfit_npend) img_fit_upgrade(); // no pass is open here: the upgrade's copy may run (img_fit)
// The version of the attachment-range candidate is taken before capturing any attachment; if it changes during setup, there is no continuation.
@@ -2902,6 +2973,14 @@ static id<MTL4RenderCommandEncoder> rp_begin_ex_(int clear_only) {
if (xe && (xe->state == XV_INVALID || xe->state == XV_PROBE)) { if (pd.colorAttachments[i].loadAction == MTLLoadActionLoad) { xv_ent copy = *xe; xv_violation(&copy, "old content loaded as an attachment"); } else xv_good(xe); } }
pd.colorAttachments[i].storeAction = MTLStoreActionStore;
}
if (g_sm_plan.on && !clear_only && pd.colorAttachments[0].texture && !R.rt.depth) { // slice-merged pass (see g_sm_plan): the attachment holds every layer the draws reach
MTLRenderPassColorAttachmentDescriptor *c = pd.colorAttachments[0];
g_sm_cur = (int)c.slice; c.slice = (NSUInteger)g_sm_plan.lo; if (c.loadAction == MTLLoadActionDontCare) c.loadAction = MTLLoadActionLoad; // the other layers' content stays
pd.renderTargetArrayLength = (NSUInteger)(g_sm_plan.hi - g_sm_plan.lo + 1);
g_sm_pass = 1; g_sm_lo = g_sm_plan.lo; g_sm_hi = g_sm_plan.hi; g_sm_applied = -1; g_sm_npass++;
static int once; if (!once++) MLOG("slice-merged pass: draws into layers %d-%d of one texture level run as one layered pass (first case pass %llu; DM_NVN_SLICE_MERGE)", g_sm_lo, g_sm_hi, g_stat_pass + 1);
PD("L %llu slice-merged pass: layers %d-%d, first draw into layer %d", g_stat_pass + 1, g_sm_lo, g_sm_hi, g_sm_cur);
}
// clear_only: 0 attach all 1 only for clears 2 neither depth nor stencil attached 3 stencil only 4 depth only (3 / 4: the pass_ds_need lookahead proves the whole pass uses only that one)
if (R.rt.depth && clear_only == 3) { // Stencil only: the depth plane of the combined format does not enter the tile and is neither read nor written (Metal allows an empty depth attachment with the combined texture on the stencil attachment)
attach(pd.stencilAttachment, R.rt.depth, &R.rt.dview, R.rt.has_dview);
@@ -4482,9 +4561,10 @@ static NSMutableDictionary<NSString *, id<MTLFunction>> *g_bf_fn_cache;
static NSMutableDictionary<NSString *, id<MTLLibrary>> *g_bf_lib_cache;
static nvn_hmap g_pso = {.klen = sizeof(pso_key)}, g_dss;
static nvn_hmap g_pso_fail = {.klen = sizeof(pso_key)}; // pipelines that failed to build: remembered and never retried (previously every draw recompiled it, over 10,000 times in one iPhone session)
static const char g_tf_mark;
static const char g_tf_mark, g_amp_mark;
static char g_geo_mark;
#define GEO_MARK ((const void *)&g_geo_mark)
#define AMP_MARK ((const void *)&g_amp_mark) // pso_key.fsfn of a slice-merged pass's pipeline (vertex amplification, sm_amp_get)
#define TF_MARK ((const void *)&g_tf_mark) // pso_key.fsfn of a transform feedback capture pipeline (tf_pso)
static int mtl_fmt_uint(MTLPixelFormat f) {
switch (f) { case MTLPixelFormatR8Uint: case MTLPixelFormatR16Uint: case MTLPixelFormatR32Uint: case MTLPixelFormatRG8Uint: case MTLPixelFormatRG16Uint: case MTLPixelFormatRG32Uint:
@@ -4538,6 +4618,7 @@ static MTLRenderPipelineDescriptor *pso_desc(const pso_key *k, NvShader *vs, NvS
if (fs && fs->fn_u && !fsfn && d.fragmentFunction == fs->fn && pso_all_uint(k, fs, merged)) d.fragmentFunction = fs->fn_u; // integer colour targets (dmsh_build.py fs_uint)
else if (fs && fs->fn_m && !fsfn && d.fragmentFunction == fs->fn && pso_mixed_uint(k, fs, merged)) d.fragmentFunction = fs->fn_m; // integer and float colour targets
if (vs->layer_out && k->fsfn != GEO_MARK) d.inputPrimitiveTopology = vs->geo_output == 1 ? MTLPrimitiveTopologyClassPoint : vs->geo_output == 6 ? MTLPrimitiveTopologyClassLine : MTLPrimitiveTopologyClassTriangle;
if (k->fsfn == AMP_MARK) d.maxVertexAmplificationCount = 2; // a slice-merged pass's pipeline: the view mapping's layer offset applies (see g_sm_plan)
if (k->fsfn == TF_MARK || k->fsfn == GEO_MARK) { d.vertexFunction = k->fsfn == GEO_MARK ? vs->geo_fn : vs->tf_fn; d.fragmentFunction = nil; d.rasterizationEnabled = NO; } // transform feedback capture (tf_draw)
d.rasterSampleCount = k->samples ? k->samples : 1;
d.alphaToCoverageEnabled = k->a2c;
@@ -4643,7 +4724,7 @@ static pthread_rwlock_t g_arch_rw = PTHREAD_RWLOCK_INITIALIZER;
static _Atomic uint64_t g_pb_built, g_pb_archived, g_pb_reused, g_pb_spec, g_pb_legacy, g_pb_base_ns;
// Time by kind, all threads (the 'pipelines up to frame' line): archive lookups that found the base / found none, specializations, base compiles; predictions (submit thread)
static _Atomic uint64_t g_pt_hit_n, g_pt_hit_ns, g_pt_miss_n, g_pt_miss_ns, g_pt_spec_ns, g_pt_comp_ns, g_pt_pred_n, g_pt_pred_ns, g_pp_done_n; // bases compiled / taken from the archive / reused, specializations, MTLDevice builds
typedef struct { const void *vf, *ff; uint32_t samples, a2c, rast, topo; uint32_t va[31][3], vl[31][3]; } pbase_key; // what a base depends on: pso_desc's descriptor less the colour attachments
typedef struct { const void *vf, *ff; uint32_t samples, a2c, rast, topo, amp; uint32_t va[31][3], vl[31][3]; } pbase_key; // what a base depends on: pso_desc's descriptor less the colour attachments
static nvn_hmap g_pbase = {.klen = sizeof(pbase_key)};
// Only callers for the same base wait on this group; unrelated bases compile in parallel.
// compiled: its owner asked the compiler (a compile 0 owner only looks in the archive, and a waiter that needs the base builds it itself when that found none).
@@ -4824,6 +4905,7 @@ static MTL4RenderPipelineDescriptor *pso_desc4(MTLRenderPipelineDescriptor *d, i
m.vertexFunctionDescriptor = vf; m.fragmentFunctionDescriptor = ff; m.vertexDescriptor = d.vertexDescriptor;
m.rasterSampleCount = d.rasterSampleCount; m.alphaToCoverageState = d.alphaToCoverageEnabled ? MTL4AlphaToCoverageStateEnabled : MTL4AlphaToCoverageStateDisabled;
m.rasterizationEnabled = d.rasterizationEnabled; m.inputPrimitiveTopology = d.inputPrimitiveTopology;
if (d.maxVertexAmplificationCount > 1) m.maxVertexAmplificationCount = d.maxVertexAmplificationCount;
for (int i = 0; i < 8; i++) {
MTL4RenderPipelineColorAttachmentDescriptor *a = m.colorAttachments[i];
if (!unspec) { pso_col4(a, d.colorAttachments[i]); continue; }
@@ -4836,7 +4918,7 @@ static MTL4RenderPipelineDescriptor *pso_desc4(MTLRenderPipelineDescriptor *d, i
static void pbase_key_of(MTLRenderPipelineDescriptor *d, pbase_key *k) {
memset(k, 0, sizeof *k);
k->vf = (__bridge const void *)d.vertexFunction; k->ff = (__bridge const void *)d.fragmentFunction;
k->samples = (uint32_t)d.rasterSampleCount; k->a2c = d.alphaToCoverageEnabled; k->rast = d.rasterizationEnabled; k->topo = (uint32_t)d.inputPrimitiveTopology;
k->samples = (uint32_t)d.rasterSampleCount; k->a2c = d.alphaToCoverageEnabled; k->rast = d.rasterizationEnabled; k->topo = (uint32_t)d.inputPrimitiveTopology; k->amp = (uint32_t)d.maxVertexAmplificationCount;
MTLVertexDescriptor *vd = d.vertexDescriptor;
for (NSUInteger i = 0; vd && i < 31; i++) {
MTLVertexAttributeDescriptor *a = vd.attributes[i]; MTLVertexBufferLayoutDescriptor *l = vd.layouts[i];
@@ -5242,6 +5324,22 @@ static const char *pso_fout_hint(NvShader *fs, const uint32_t cfmt[8], char *b,
return b;
}
static __unsafe_unretained id<MTLRenderPipelineState> pso_get(NvProg *pr, MTLRenderPassDescriptor *unused) { AP_B; __unsafe_unretained id<MTLRenderPipelineState> r = pso_get_(pr, unused); AP_E(2); return r; }
// The pipelines of slice-merged passes (g_sm_plan): the draw's pipeline built with maxVertexAmplificationCount 2, keyed with AMP_MARK. Built in the background on first use; until it is
// there the draw gets nil and goes to a pass of its own (sm_plan does not merge, draw splits a merged pass), so no frame waits for the compiler. A failed build is not tried again
enum { SM_AMP_MAX = 64 };
static struct { pso_key k; id<MTLRenderPipelineState> ps; int done; } g_sm_amp[SM_AMP_MAX]; static int g_sm_amp_n; static os_unfair_lock g_sm_amp_lock = OS_UNFAIR_LOCK_INIT;
static __unsafe_unretained id<MTLRenderPipelineState> sm_amp_get(const pso_key *k, NvShader *vs, NvShader *fs) {
__unsafe_unretained id<MTLRenderPipelineState> c = hm_get_id(&g_pso, k); if (c) return c;
os_unfair_lock_lock(&g_sm_amp_lock); int i = 0; for (; i < g_sm_amp_n; i++) if (!memcmp(&g_sm_amp[i].k, k, sizeof *k)) break;
if (i < g_sm_amp_n) { id ps = g_sm_amp[i].done ? g_sm_amp[i].ps : nil; os_unfair_lock_unlock(&g_sm_amp_lock); if (ps) { hm_put_id(&g_pso, k, ps); return hm_get_id(&g_pso, k); } return nil; }
if (g_sm_amp_n >= SM_AMP_MAX) { os_unfair_lock_unlock(&g_sm_amp_lock); return nil; }
g_sm_amp[i].k = *k; g_sm_amp[i].ps = nil; g_sm_amp[i].done = 0; g_sm_amp_n++; os_unfair_lock_unlock(&g_sm_amp_lock);
MTLRenderPipelineDescriptor *d = pso_desc(k, vs, fs, nil, 0);
dispatch_async(dispatch_get_global_queue(QOS_CLASS_UTILITY, 0), ^{ NSError *e = nil; id<MTLRenderPipelineState> ps = pso_build(d, &e);
if (!ps) MLOG("[RED] slice-merged pass: no amplified pipeline for vertex %s fragment %s: %s (those passes stay apart)", vs->key, fs ? fs->key : "-", e.localizedDescription.UTF8String);
os_unfair_lock_lock(&g_sm_amp_lock); g_sm_amp[i].ps = ps; g_sm_amp[i].done = 1; os_unfair_lock_unlock(&g_sm_amp_lock); });
return nil;
}
static __unsafe_unretained id<MTLRenderPipelineState> pso_get_(NvProg *pr, MTLRenderPassDescriptor *unused) {
(void)unused;
g_pso_pull = 0; // set again only where a pulling pipeline is returned: a stale 1 made the next draw, whose pipeline came from the cache, bind the layout block instead of its streams
@@ -5250,6 +5348,7 @@ static __unsafe_unretained id<MTLRenderPipelineState> pso_get_(NvProg *pr, MTLRe
pso_key k; pso_key_fill(&k, vs, fs, g_pass.cfmt, g_pass.dfmt, g_pass.sfmt, g_pass.samples, g_pso_lmap != NULL);
if (g_pso_lmap) { memcpy(k.lmap, g_pso_lmap, 8); k.fsfn = (__bridge void *)g_pso_fsfn; }
id<MTLFunction> ms_fn = ms_fs_override(fs); if (ms_fn) k.fsfn = (__bridge void *)ms_fn; // DM_NVN_MAGSCAN phases 2 / 3
if (g_pso_amp) { if (k.fsfn || g_pso_lmap) return nil; k.fsfn = AMP_MARK; return sm_amp_get(&k, vs, fs); } // a slice-merged pass's pipeline: built in the background, nil until then
// DM_PSO_PULL_ALL=1 (diagnostic): every draw that has a pulling pipeline draws with it, never with the pipeline for its layout
{ static int all = -1; if (all < 0) { const char *ev = getenv("DM_PSO_PULL_ALL"); all = ev && !strcmp(ev, "1"); if (all) MLOG("DM_PSO_PULL_ALL=1: every draw with a pulling pipeline draws with it (diagnostic)"); }
if (all && !g_pso_lmap && !ms_fn && hybrid_on()) { __unsafe_unretained id<MTLRenderPipelineState> h = hybrid_get(&k, vs, fs); if (h) { g_pso_pull = 1; return h; } } }
@@ -6421,6 +6520,55 @@ static int rt_same(const p_set_rt *a, const p_set_rt *b) { // Same set of targ
}
return 1;
}
// Slice-merged passes (see g_sm_plan). sm_rt_slice: b names the same single colour target as a (one colour attachment in slot 0, no depth / stencil, the same texture and view)
// except the view's first layer, which goes to *slice
static int sm_on(void) { static int m = -1; if (m < 0) { const char *e = getenv("DM_NVN_SLICE_MERGE"); m = !(e && !strcmp(e, "0")); if (!m) MLOG("DM_NVN_SLICE_MERGE=0: passes into other layers of one texture level stay apart"); } return m; }
static int sm_rt_slice(const p_set_rt *a, const p_set_rt *b, int *slice) {
if (a->depth || b->depth || a->n < 1 || b->n < 1 || !a->color[0] || !b->color[0] || a->color[0]->serial != b->color[0]->serial || !a->has_cview[0] || !b->has_cview[0]) return 0;
for (int k = 1; k < 8; k++) if ((k < a->n && a->color[k]) || (k < b->n && b->color[k])) return 0;
view_t x = a->cview[0], y = b->cview[0];
if (!x.set_layers || !y.set_layers || x.set_format || y.set_format || (x.set_target && x.target != NVN_TEXTURE_TARGET_2D) || (y.set_target && y.target != NVN_TEXTURE_TARGET_2D)) return 0;
x.min_layer = y.min_layer = 0; if (memcmp(&x, &y, sizeof x)) return 0;
*slice = b->cview[0].min_layer; return 1;
}
// From the op after the draw that opens a pass: the layers the following targets switch to while nothing but state, uniform updates (deferred in the pass: ubd_on) and draws comes between.
// Returns the number of switches; lo..hi and the mask of layers drawn into
static int sm_scan(const uint8_t *p, const p_set_rt *rt, int cur, int *lo, int *hi, uint64_t *mask) {
int n = 0; *lo = *hi = cur; *mask = cur < 64 ? 1ull << cur : 0;
for (int guard = 0; guard < 4096 && p; guard++) {
op_hdr h; memcpy(&h, p, sizeof h); const void *a = p + 8;
switch (h.op) {
case OP_JUMP: { const uint8_t *t; memcpy(&t, a, 8); p = t; continue; }
case OP_SET_RT: { int sl; if (!sm_rt_slice(rt, a, &sl) || sl >= 64) return n;
if (sl != cur) { n++; cur = sl; if (sl < *lo) *lo = sl; if (sl > *hi) *hi = sl; *mask |= 1ull << sl; } break; }
case OP_BARRIER: { int32_t bits; memcpy(&bits, a, 4); if (bits) return n; break; }
case OP_UPDATE_UB: if (!ubd_on()) return n; break;
case OP_DRAW: case OP_DRAW_IDX: case OP_VIEWPORTS: case OP_SCISSOR: case OP_DEPTH_RANGE: case OP_BIND_PROGRAM: case OP_BIND_STATE: case OP_BIND_VATTR: case OP_BIND_VSTREAM:
case OP_BIND_VBUF: case OP_BIND_UBO: case OP_BIND_SSBO: case OP_BIND_TEX: case OP_BIND_TEX1: case OP_BIND_SAMP: case OP_BIND_IMG: case OP_STENCIL: case OP_POLY_OFFSET:
case OP_BLEND_COLOR: case OP_DEPTH_CLAMP: case OP_VP_SWIZZLE: case OP_PRIM_RESTART: case OP_SET_TEX_POOL: case OP_SET_SAMP_POOL: case OP_COPY_STRIDE: break;
default: return n;
}
p += h.size;
}
return n;
}
// The draw about to open a pass: plan it as slice-merged when its target is one layer of an array or cube texture level and the next targets switch to other layers of that level
// (every layer between the lowest and the highest drawn into), and the draw's amplified pipeline is built (else it is queued, and this pass stays as it is)
static int draw_samples_rt(NvProg *pr);
static void sm_plan(NvProg *pr) {
g_sm_plan.on = 0;
if (!pr || !pr->sh[0] || pr->sh[0]->layer_out || R.clear_d || R.clear_s || draw_samples_rt(pr)) return;
for (int i = 0; i < 8; i++) if (R.clear_c[i]) return;
int cur; if (!sm_rt_slice(&R.rt, &R.rt, &cur) || cur >= 64) return;
id<MTLTexture> t = tex_mtl(R.rt.color[0]);
if (!t || t.sampleCount != 1 || mtl_fmt_block(t.pixelFormat) || mtl_fmt_depth(t.pixelFormat) || (t.textureType != MTLTextureType2DArray && t.textureType != MTLTextureTypeCube && t.textureType != MTLTextureTypeCubeArray)) return;
int lo, hi; uint64_t mask; if (!sm_scan(g_exec_next, &R.rt, cur, &lo, &hi, &mask)) return;
if (__builtin_popcountll(mask) != hi - lo + 1 || (NSUInteger)hi >= t.arrayLength * (t.textureType == MTLTextureType2DArray ? 1 : 6)) return;
__typeof__(g_pass) keep = g_pass; memset(g_pass.cfmt, 0, sizeof g_pass.cfmt); g_pass.cfmt[0] = (uint32_t)t.pixelFormat; g_pass.dfmt = g_pass.sfmt = 0; g_pass.samples = 1; // the key pso_get makes in the pass
g_pso_amp = 1; id ps = pso_get(pr, nil); g_pso_amp = 0; g_pass = keep;
if (!ps) { g_sm_nwait++; return; }
g_sm_plan.on = 1; g_sm_plan.lo = lo; g_sm_plan.hi = hi;
}
// Experimental RT range candidate: only ignore positive nlevels differences not used by the actual render attachments; other fields stay word-for-word identical.
#define NVN_RT_RANGE_IMPL 1
static int rt_range_shape(const p_set_rt *a, const p_set_rt *b) {
@@ -6460,17 +6608,30 @@ static int rt_range_same(const p_set_rt *a, const p_set_rt *b) {
depth.textureType == MTLTextureType2D && color.sampleCount == 1 && depth.sampleCount == 1 &&
g_pass.reg_gen == g_reg_gen && g_pass.ser_gen == g_ser_gen && g_pass.range_dirty == nvn_dirty_now();
}
static int tex_storage_overlap(const tex_t *a, const tex_t *b) {
if (!a || !b) return 0;
if (a->serial == b->serial) return 1;
if (!a->mem || !b->mem) return 0;
uint64_t x = (uint64_t)a->mem, y = (uint64_t)b->mem;
if (!a->L.size || !b->L.size || a->L.size > UINT64_MAX - x || b->L.size > UINT64_MAX - y) return 1;
return (x < y + b->L.size && y < x + a->L.size) || nvn_sparse_alias(x, x + a->L.size) || nvn_sparse_alias(y, y + b->L.size);
}
static int draw_samples_rt(NvProg *pr) { // Sampled attachments keep the original conservative ser-level guard, not split by level/layer
uint64_t rs[9]; int nr = 0;
for (int k = 0; k < R.rt.n && k < 8; k++) if (R.rt.color[k]) rs[nr++] = R.rt.color[k]->serial;
if (R.rt.depth) rs[nr++] = R.rt.depth->serial;
if (!pr) return 1;
for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh) continue;
if (sh->rf.bindless || sh->rf.dyn) return 1;
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; uint64_t h;
if (b->kind == NVN_RB_TEXU || b->kind == NVN_RB_TEXD || b->kind == NVN_RB_TEXA || b->kind == NVN_RB_IMGA) return 1;
if (b->kind == NVN_RB_IMG) h = b->reg < 32 && sh->stage < 6 ? S.img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = sh->stage < 6 ? tex_slot_h(S.tex[sh->stage], S.img[sh->stage], b->reg) : 0; else continue;
int tid = (int)(h & 0xfffff); if (!tid) continue;
os_unfair_lock_lock(&g_rmu); uint64_t ser = tid < g_ntreg && g_treg[tid].ok ? g_treg[tid].t.serial : 0; os_unfair_lock_unlock(&g_rmu);
for (int j = 0; j < nr; j++) if (ser && ser == rs[j]) return 1; } }
os_unfair_lock_lock(&g_rmu); tex_t t = tid < g_ntreg && g_treg[tid].ok ? g_treg[tid].t : (tex_t){0}; os_unfair_lock_unlock(&g_rmu);
if (!t.serial) return 1;
for (int j = 0; j < nr; j++) if (t.serial == rs[j]) return 1;
for (int j = 0; j < R.rt.n && j < 8; j++) if (tex_storage_overlap(&t, R.rt.color[j])) return 1;
if (tex_storage_overlap(&t, R.rt.depth)) return 1; } }
return 0;
}
// The whole render pass's need for depth / stencil (bits: 1 depth 2 stencil; 3 = both needed or cannot be proven). Starting from the draw that opens the render pass, scan in real replay order to the end of the render pass:
@@ -7983,6 +8144,115 @@ static int lc_keeps(uint32_t op) {
}
}
static void dispatch(const p_dispatch *dp);
// ---------- Compute dispatches moved ahead of the render pass they would split (DM_NVN_CS_HOIST) ----------
// Metal cannot run a compute dispatch inside a render pass, so a dispatch between two draws of one pass ends it, and on a tile-based GPU every attachment is stored and loaded again:
// Breath of the Wild runs one GPU particle dispatch per emitter between the G-buffer's effect draws, six to eight splits of a 1600x900 G-buffer a frame (about 350 MB of attachment
// traffic). When a draw opens a pass, the ops after it are scanned up to the first op that would end the pass anyway; each dispatch met on the way runs right away, before the pass
// opens, if moving it ahead of the draws before it in that stretch changes nothing they read or write: it writes no memory those draws read (vertex, index, indirect, uniform and
// storage buffers; storage buffers count as written by the dispatch), samples no colour attachment of the pass, writes no image, and the draws write no memory (no storage buffer,
// image or transform feedback). Replay then skips it, and the pass stays open across it. A dispatch that samples the pass's depth attachment sees it as the pass began, so it moves
// only while no draw before it in the pass has depth writes on (until then that depth is the same; Breath of the Wild's particle collision samples the G-buffer depth). On by default
// (0 off): Breath of the Wild at 900p on an M2 Max, 28 → 8 compute encoders and 149 → 129 render passes a frame, frame p50 10.2 → 8.8 ms
static int cshoist_on(void) { static int m = -1; if (m < 0) { const char *e = getenv("DM_NVN_CS_HOIST"); m = !(e && !strcmp(e, "0")); if (!m) MLOG("DM_NVN_CS_HOIST=0: compute dispatches between the draws of a pass end it"); } return m; }
typedef struct { uint64_t lo, hi; } csh_rng;
enum { CSH_RN = 1024, CSH_MAX = 64 };
static const uint8_t *g_csh_op[CSH_MAX]; static int g_csh_n; static uint64_t g_csh_moved, g_csh_depth, g_csh_stop[8]; // dispatches run ahead and not yet met by replay; moved, of them sampling the pass's depth; why scans stopped
static int csh_take(const uint8_t *p) { for (int i = 0; i < g_csh_n; i++) if (g_csh_op[i] == p) { g_csh_op[i] = g_csh_op[--g_csh_n]; return 1; } return 0; }
static int csh_add(csh_rng *r, int *n, uint64_t a, uint64_t sz) {
if (!a) return 1;
// Sparse virtual addresses can name the same storage through different
// pools. A guest-address comparison cannot establish independence there.
if (!sz || sz > UINT64_MAX - a || *n >= CSH_RN || nvn_sparse_alias(a, a + sz)) return 0;
r[(*n)++] = (csh_rng){a, a + sz}; return 1;
}
static int csh_hit(const csh_rng *a, int na, const csh_rng *b, int nb) { for (int i = 0; i < na; i++) for (int j = 0; j < nb; j++) if (a[i].lo < b[j].hi && b[j].lo < a[i].hi) return 1; return 0; }
static int csh_attachment_hit(uint64_t addr, uint64_t size) {
if (!addr) return 0;
csh_rng access[1]; int na = 0; if (!csh_add(access, &na, addr, size)) return 1;
for (int k = 0; k < 9; k++) {
const tex_t *t = k == 8 ? R.rt.depth : k < R.rt.n ? R.rt.color[k] : NULL;
if (!t || !t->mem) continue;
csh_rng storage[1]; int ns = 0;
if (!csh_add(storage, &ns, (uint64_t)t->mem, t->L.size) || csh_hit(access, na, storage, ns)) return 1;
}
return 0;
}
// The memory a draw reads besides textures; 0 when it cannot be listed or the draw may write memory
static int csh_draw_reads(NvProg *pr, const p_draw *dw, const p_draw_idx *di, csh_rng *r, int *n) {
if (!pr || pr->sh[2] || pr->sh[3] || pr->sh[4] || S.rast_discard || (dw && dw->mdi_cnt) || (di && (di->mdi_cnt || di->ind))) return 0;
for (int k = 0; k < 16; k++) if (!csh_add(r, n, S.vbuf[k].address, S.vbuf[k].size)) return 0;
if (di && !csh_add(r, n, di->ibuf, (uint64_t)(di->count > 0 ? di->count : 0) * 4 + 16)) return 0;
if (!csh_add(r, n, dw ? dw->ind : di ? di->ind : 0, 32)) return 0;
for (int st = 0; st < 2; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh) continue;
if (sh->rf.bindless || sh->rf.dyn) return 0;
for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i];
switch (b->kind) {
case NVN_RB_CBUF: case NVN_RB_UBOP: if (b->reg < 16 && !csh_add(r, n, S.ubo[st][b->reg].addr, S.ubo[st][b->reg].size)) return 0; break;
case NVN_RB_TEX: { uint64_t h = tex_slot_h(S.tex[st], S.img[st], b->reg); int tid = (int)(h & 0xfffff); if (!tid) break;
os_unfair_lock_lock(&g_rmu); tex_t t = tid < g_ntreg && g_treg[tid].ok ? g_treg[tid].t : (tex_t){0}; os_unfair_lock_unlock(&g_rmu);
// Linear and buffer textures can read directly from pool
// buffers. Include storage even when reflection calls it TEX.
if (!t.serial || !csh_add(r, n, (uint64_t)t.mem, t.L.size)) return 0;
break; }
case NVN_RB_TEXU: case NVN_RB_TEXD: case NVN_RB_TEXA: return 0;
case NVN_RB_SBUFP: case NVN_RB_SBUFU: case NVN_RB_IMG: case NVN_RB_IMGA: case NVN_RB_TBUF: return 0; // may write memory, or reads memory not listed here
default: break; } } }
return 1;
}
// A dispatch at its place in the stream (S as it is there): its storage buffers (counted as written), and whether it samples a colour attachment of the pass (1), only the depth (2)
static int csh_cs(NvShader *cs, const p_dispatch *dp, csh_rng *w, int *nw, int *depth) {
if (!cs || cs->builtin || dp->ind || cs->rf.bindless || cs->rf.dyn) return 0;
for (int i = 0; i < cs->rf.nb; i++) { const nvn_rbind *b = &cs->rf.b[i];
switch (b->kind) {
case NVN_RB_SBUFP: if (b->reg >= 16 || csh_attachment_hit(S.ssbo[5][b->reg].addr, S.ssbo[5][b->reg].size) || !csh_add(w, nw, S.ssbo[5][b->reg].addr, S.ssbo[5][b->reg].size)) return 0; break;
case NVN_RB_CBUF: case NVN_RB_UBOP: if (b->reg >= 16 || csh_attachment_hit(S.ubo[5][b->reg].addr, S.ubo[5][b->reg].size)) return 0; break;
case NVN_RB_SBUFU: case NVN_RB_IMG: case NVN_RB_IMGA: case NVN_RB_TEXU: case NVN_RB_TEXD: case NVN_RB_TEXA: return 0;
case NVN_RB_TEX: case NVN_RB_TBUF: { uint64_t h = tex_slot_h(S.tex[5], S.img[5], b->reg); int tid = (int)(h & 0xfffff); if (!tid) break;
os_unfair_lock_lock(&g_rmu); tex_t t = tid < g_ntreg && g_treg[tid].ok ? g_treg[tid].t : (tex_t){0}; os_unfair_lock_unlock(&g_rmu);
if (!t.serial) return 0;
for (int k = 0; k < R.rt.n && k < 8; k++) if (tex_storage_overlap(&t, R.rt.color[k])) return 0;
if (tex_storage_overlap(&t, R.rt.depth)) *depth = 1;
break; }
default: break; } }
return 1;
}
static void cs_hoist_try(NvProg *pr0, const p_draw *dw0, const p_draw_idx *di0) {
if (!g_exec_next || g_ret_n > 8 || g_fz_checking || g_defer_replaying || g_nlc || R.clear_d || R.clear_s) return;
for (int i = 0; i < 8; i++) if (R.clear_c[i]) return; // a clear this pass takes as its load action comes before anything a moved dispatch could read
static csh_rng rd[CSH_RN]; int nrd = 0;
if (!csh_draw_reads(pr0, dw0, di0, rd, &nrd)) { g_csh_stop[0]++; return; }
fz_cursor cur = {.p = g_exec_next}; for (int k = 0; k < g_ret_n; k++) cur.ret[cur.ret_n++] = g_ret[k];
static typeof(S) Ssave; memcpy(&Ssave, &S, sizeof S); typeof(R) Rsave = R; int pso_dirty = g_pso_dirty, dss_dirty = g_dss_dirty, moved = 0;
int dwrote = R.rt.depth && (S.ds.dw || S.ds.st); // either plane: a compute texture view may read stencil
for (int guard = 0; guard < 4096; guard++) {
const uint8_t *q; op_hdr hq; const void *aq;
if (!fz_next(&cur, &q, &hq, &aq)) break;
if (hq.op == OP_BARRIER) { uint32_t bits; memcpy(&bits, aq, sizeof bits); if (bits) { g_csh_stop[5]++; break; } continue; }
if (hq.op == OP_DRAW || hq.op == OP_DRAW_IDX) {
p_draw tdw; p_draw_idx tdi; const p_draw *d = hq.op == OP_DRAW ? draw_rec(aq, hq.size, &tdw) : NULL; const p_draw_idx *x = hq.op == OP_DRAW_IDX ? draw_idx_rec(aq, hq.size, &tdi) : NULL;
prog_resolve(); __unsafe_unretained NvProg *pr = S.prog ? prog_get(S.prog) : nil;
if (!csh_draw_reads(pr, d, x, rd, &nrd)) { g_csh_stop[1]++; break; }
if (R.rt.depth && (S.ds.dw || S.ds.st)) dwrote = 1;
continue; }
if (hq.op == OP_DISPATCH) {
const p_dispatch *dp = aq; __unsafe_unretained NvShader *cs = S.cprog && prog_get(S.cprog) ? prog_get(S.cprog)->sh[5] : nil;
static csh_rng wr[CSH_RN]; int nwr = 0, depth = 0;
if (!csh_cs(cs, dp, wr, &nwr, &depth)) { g_csh_stop[2]++; break; }
if (depth && dwrote) { g_csh_stop[7]++; break; } // it would miss depth the draws before it wrote
if (csh_hit(wr, nwr, rd, nrd)) { g_csh_stop[3]++; break; } // the draws before it read what it writes
if (g_csh_n >= CSH_MAX) { g_csh_stop[4]++; break; }
PD("H %llu dispatch %s moved ahead of the pass%s", g_stat_pass + 1, cs->key, depth ? " (samples the pass's depth, which no draw before it wrote)" : "");
dispatch(dp); g_csh_op[g_csh_n++] = q; g_csh_moved++; g_csh_depth += depth; moved++;
continue; }
if (state_op(hq.op, aq)) continue;
g_csh_stop[5]++; break; // anything else may end the pass, write memory or depend on the order (copies, updates, clears, targets, queries)
}
if (cur.bad) g_csh_stop[6]++;
memcpy(&S, &Ssave, sizeof S); args_dirty_all(); R = Rsave; g_pso_dirty = pso_dirty || moved; g_dss_dirty = dss_dirty || moved; prog_resolve();
{ static int once; if (moved && !once++) MLOG("compute dispatches moved ahead of the render pass they would split (DM_NVN_CS_HOIST): first case pass %llu, %d dispatches", g_stat_pass + 1, moved); }
}
void mtl_cshoist_stats(uint64_t *moved, uint64_t *depth, uint64_t *stops) { *moved = g_csh_moved; *depth = g_csh_depth; for (int i = 0; i < 8; i++) stops[i] = g_csh_stop[i]; }
static void draw(const p_draw *dw, const p_draw_idx *di) {
g_draw_binst = (uint32_t)(dw ? dw->base_instance : di ? di->base_instance : 0); // c0 + 0x644 of the stages that read the driver segment (encode_args)
if (!g_geo_program && S.prog) {
@@ -8054,13 +8324,16 @@ static void draw(const p_draw *dw, const p_draw_idx *di) {
// The deferral decision comes after hoist_try: hoisted pure-clear render passes are skipped in the lookahead, and only then can P's same-target continuation segment be recognized
if (!g_enc && tbdr_on() && !g_geo_program && !defer_has_in_p() && !g_defer_replaying && !g_fz.active && !g_fz_checking && g_exec_next) { defer_try(dw, di); if (g_defer_cur->active && g_defer_cur->in_p) return; }
if (!g_enc && tbdr_on() && !g_fz_checking && !g_geo_program) fuse_try(); // Readable tile render pass merge is on by default; when the structure / same-pixel proof does not hold it still falls back to a normal render pass, and it is not re-entered during the CHECK redraw
if (!g_enc && !tbdr_on() && !g_geo_program && cshoist_on()) cs_hoist_try(pr, dw, di); // see DM_NVN_CS_HOIST
int need = g_enc || !tbdr_on() ? 3 : pass_ds_need(), mode = defer_q_draw ? 0 : need == 0 ? 2 : need == 2 ? 3 : need == 1 ? 4 : 0; // 2 attach neither, 3 attach stencil only, 4 attach depth only
int skip_d = mode != 0;
if (!g_enc) { g_fd_dw = dw; g_fd_di = di; }
g_rp_layered = pr->sh[0] && pr->sh[0]->layer_out;
if (g_enc && g_pass.layered != g_rp_layered) rp_end(); // a layered pass only for layer-writing vertex shaders (see rp_begin_ex)
if (g_enc && g_sm_pass && draw_samples_rt(pr)) { PD("M %llu a draw in the slice-merged pass samples its attachment; split here", g_stat_pass); rp_end(); }
if (!g_enc && mode == 0 && !g_rp_layered && !defer_q_draw && g_exec_next && !tbdr_on() && !g_geo_program && sm_on()) sm_plan(pr);
id<MTLRenderCommandEncoder> en = rp_begin_ex(mode);
g_rp_layered = 0;
g_rp_layered = 0; g_sm_plan.on = 0;
g_fd_dw = NULL; g_fd_di = NULL;
if (!en) { PD("Z %llu draw skipped: the render pass did not begin vs=%.16s fs=%.16s", g_stat_pass, pr && pr->sh[0] ? pr->sh[0]->key : "-", pr && pr->sh[1] ? pr->sh[1]->key : "-"); return; } // Z lines: why a draw was dropped at these early returns
if (defer_q_draw) defer_replay();
@@ -8068,11 +8341,17 @@ static void draw(const p_draw *dw, const p_draw_idx *di) {
// The pipeline depends only on (program, render pass targets, blend / color / write mask / multisample state, vertex layout): if these did not change, reuse the previous one
static id<MTLRenderPipelineState> cur_ps; static NvProg *cur_pr; static uint64_t cur_ps_pass; static int cur_pull; // cur_pull: a hybrid (pulling) pipeline, asked for again at the next draw
if (g_fz.active && !g_defer_replaying) { g_pso_lmap = g_fz.phase == 1 ? g_fz.lmap1 : g_fz.lmap2; g_pso_fsfn = g_fz.phase == 1 ? nil : fz_fn; g_pso_fz = g_fz.phase == 1 ? (pso_fz_desc){0} : g_fz_desc; g_pso_dirty = 1; } // Readable tile merge: fetch the pipeline by the physical -> logical mapping and the specialization
if (g_pso_dirty || pr != cur_pr || cur_ps_pass != g_pass.serial) { cur_ps = pso_get(pr, nil); cur_pull = g_pso_pull; cur_pr = pr; cur_ps_pass = g_pass.serial; g_pso_dirty = !cur_ps || cur_pull; }
if (g_pso_dirty || pr != cur_pr || cur_ps_pass != g_pass.serial) { g_pso_amp = g_sm_pass; cur_ps = pso_get(pr, nil); g_pso_amp = 0; cur_pull = g_pso_pull; cur_pr = pr; cur_ps_pass = g_pass.serial; g_pso_dirty = !cur_ps || cur_pull; }
if (!cur_ps && g_sm_pass) { // a draw whose amplified pipeline is not built yet (sm_amp_get queued it): the slice-merged pass ends and the draw opens a pass of its own
g_sm_nwait++; PD("M %llu slice-merged pass: no amplified pipeline yet for vertex %s fragment %s; split here", g_stat_pass, pr->sh[0] ? pr->sh[0]->key : "-", pr->sh[1] ? pr->sh[1]->key : "-");
rp_end(); en = rp_begin_ex(mode); if (!en) return;
cur_ps = pso_get(pr, nil); cur_pull = g_pso_pull; cur_pr = pr; cur_ps_pass = g_pass.serial; g_pso_dirty = !cur_ps || cur_pull; }
if (g_pso_lmap) { g_pso_lmap = NULL; g_pso_fsfn = nil; memset(&g_pso_fz, 0, sizeof g_pso_fz); g_pso_dirty = 1; } // The next draw fetches again according to its own situation
__unsafe_unretained id<MTLRenderPipelineState> ps = cur_ps;
if (!ps) { PD("Z %llu draw skipped: no render pipeline (vertex %s fragment %s)", g_stat_pass, pr->sh[0] ? pr->sh[0]->key : "-", pr->sh[1] ? pr->sh[1]->key : "-"); return; }
if (ps != g_enc_pso) { enc_r_pso(en, ps); g_enc_pso = ps; }
if (g_sm_pass && g_sm_applied != g_sm_cur) { // the layer this draw goes to (see g_sm_plan)
MTLVertexAmplificationViewMapping vm = {0, (uint32_t)(g_sm_cur - g_sm_lo)}; [(id<MTL4RenderCommandEncoder>)en setVertexAmplificationCount:1 viewMappings:&vm]; g_sm_applied = g_sm_cur; }
static id<MTLDepthStencilState> cur_ds; static uint64_t cur_ds_pass;
// Diagnostic DM_NVN_DEPTHOVR=fragment shader key prefix,...: those draws compare depth with ALWAYS, which tells "the depth test rejects every fragment" from "nothing rasterizes" (with DM_NVN_HILITE); unset = unchanged
static char dko[8][24]; static int ndko = -1;
@@ -8733,6 +9012,7 @@ static void sub_cut(void) {
res_commit();
[g_cb endCommandBuffer];
rec_commit(g_cb, o);
if (rec_on()) rec_publish(); // the segment reaches the encode thread now, not when its block of records fills
g_cb = rec_cb_new();
[g_cb beginCommandBufferWithAllocator:g_alloc4[g_alloc_cur]]; // one allocator encodes several command buffers in sequence; it is reset only once its fence (the last segment) has passed, when a later submission starts
g_cut_draw0 = g_stat_draw;
@@ -8964,6 +9244,9 @@ void mtl_exec(const uint8_t *p) {
g_rt_merged = 1; if (!g_rt_merge_n++) MLOG("same-target continuation: first case pass %llu", g_stat_pass); PD("M %llu same-target continuation pass", g_stat_pass); break; }
// Do not continue a render pass opened only for a clear; one with fewer depth / stencil attachments can be continued (check the planes before drawing)
if (g_enc && g_enc_mode != 1 && !pend && R.has_rt && tbdr_on() && rt_same(&R.rt, r)) { g_rt_merged = 1; if (!g_rt_merge_n++) MLOG("same-target continuation: first case pass %llu", g_stat_pass); PD("M %llu same-target continuation pass", g_stat_pass); R.rt = *r; break; }
if (g_enc && g_sm_pass && !pend && !g_game_barrier) { int sl; // another layer of the slice-merged pass's level (sm_scan planned it)
if (sm_rt_slice(&R.rt, r, &sl) && sl >= g_sm_lo && sl <= g_sm_hi) { R.rt = *r; g_sm_cur = sl; g_sm_ncont++;
{ void td_note_layer(id<MTLTexture>, uint64_t, NSUInteger, NSUInteger); td_note_layer(tex_mtl(r->color[0]), r->color[0]->serial, r->cview[0].set_levels ? r->cview[0].base_level : 0, (NSUInteger)sl); } PD("M %llu slice-merged pass goes on in layer %d", g_stat_pass, sl); break; } }
if (pend && !g_enc && g_cb && lc_on()) lc_push(); else rp_flush(); // clears with no draw wait for the next pass that attaches their targets (lc_push)
sub_cut(); R.rt = *r; R.has_rt = 1; break;
}
@@ -8973,7 +9256,7 @@ void mtl_exec(const uint8_t *p) {
// A scissored clear to 0 outside a pass with all channels (SMBW clears pyramid levels as sub-rectangles of 480×272 and other targets): copy from the zero buffer into the scissor rectangle in the replay stream
if (!g_enc && c->mask == 0xf && !c->rgba[0] && !c->rgba[1] && !c->rgba[2] && !c->rgba[3] && R.has_sc && c->index >= 0 && c->index < R.rt.n && R.rt.color[c->index]
&& !scissor_full(R.rt.color[c->index], &R.rt.cview[c->index], R.rt.has_cview[c->index]) && clear_rect_zero(c->index)) break;
if (g_rt_merged) rp_end(); // A clear arrived in a continued render pass: end it here, and the clear is attached to the start of the next render pass as usual
if (g_rt_merged || g_sm_pass) rp_end(); // A clear arrived in a continued render pass: end it here, and the clear is attached to the start of the next render pass as usual
if (c->index < 0 || c->index >= R.rt.n || !R.rt.color[c->index]) { RED_ONCE(5, "color clear target %d invalid", c->index); break; }
if (g_enc || c->mask != 0xf || !scissor_full(R.rt.color[c->index], &R.rt.cview[c->index], R.rt.has_cview[c->index])) {
// Inside a pass, with a partial mask or a partial scissor: drawn as a triangle (clear_color_tri)
@@ -8987,7 +9270,7 @@ void mtl_exec(const uint8_t *p) {
case OP_CLEAR_DS: {
const p_clear_ds *c = a;
if (g_rc_skip) { PD("c %llu depth-stencil clear skipped by conditional rendering", g_stat_pass); break; }
if (g_rt_merged) rp_end();
if (g_rt_merged || g_sm_pass) rp_end();
// A load-action clear covers the whole attachment and every stencil bit. A clear whose scissor does not cover the depth target, or whose stencil mask is partial, is drawn as a triangle
// (clear_ds_tri), in or before a pass: before the first draw and for a partial stencil mask in a pass it used to clear the whole target (PR #108)
unsigned sm = (unsigned)c->stencil_mask & 0xff;
@@ -9019,7 +9302,8 @@ void mtl_exec(const uint8_t *p) {
case OP_COPY_T2B: copy_t2b(a); break;
case OP_COPY_T2T: copy_t2t(a); break;
case OP_CLEAR_TEX: clear_tex(a); break;
case OP_DISPATCH: dispatch(a); break;
case OP_DISPATCH: if (g_csh_n && csh_take(p)) { PD("H %llu dispatch already run ahead of the pass", g_stat_pass); break; }
dispatch(a); break;
case OP_BARRIER: { int32_t bits; memcpy(&bits, a, 4); // ordered at the next draw pass (barrier_game); inside a pass with shader writes, the pass ends first (swrite_note)
int split = g_enc && g_pass_swrite && (bits & (NVN_BARRIER_ORDER_FRAGMENTS_BIT | NVN_BARRIER_ORDER_FRAGMENTS_TILED_BIT));
PD("I %llu barrier bits %#x%s", g_stat_pass, bits, split ? ": ends the pass (shader writes)" : g_enc ? " (render pass open)" : "");
@@ -9168,14 +9452,23 @@ static void shot(id<MTLTexture> src, const char *path, int screen, const int cro
// ---------- DM_NVN_TEXDUMP=frame number:path (%d in the path is replaced with "index*100+mip level"): save each color render target used by that frame as an image ----------
// Diagnostic: before the first render pass of the next frame is opened, wait for the GPU to finish, then draw each one (the mip level used by that frame) by full-screen sampling into RGBA8 and save as PNG (float formats are clamped to [0,1])
static long g_td_frame = -2; static const char *g_td_path; static NSMutableArray *g_td_list; static int g_td_done;
// DM_NVN_TEXDUMP=layers=<n>:path: the first frame with at least n draws into single layers of small (at most 64x64) array or cube texture levels, counted per pass or slice-merged layer
// (Breath of the Wild's cube map filter frame, every tenth: about 150 against 20): its targets are kept frame by frame and dumped when such a frame ends, the same frame with or without DM_NVN_SLICE_MERGE
static int g_td_layers, g_td_lcount; static long g_td_lframe = -1;
static void td_layer_count(id<MTLTexture> t, NSUInteger lv) { if (t && (t.textureType == MTLTextureType2DArray || t.textureType == MTLTextureTypeCube) && MAX(t.width >> lv, 1ul) <= 64) g_td_lcount++; }
static void td_dump(void);
// DM_NVN_TEXDUMP=size=WxH:path instead picks the frame by content: the first frame that opens a render pass whose first color attachment level is WxH (for one-off work whose frame differs between runs)
static unsigned g_td_w, g_td_h;
void td_note(MTL4RenderPassDescriptor *pd, NSUInteger nc) {
if (g_td_frame == -2) { const char *e = getenv("DM_NVN_TEXDUMP"), *c = e ? strchr(e, ':') : NULL; g_td_frame = c ? atol(e) : -1; g_td_path = c ? c + 1 : NULL; g_td_list = [NSMutableArray array];
if (c && !strncmp(e, "size=", 5) && sscanf(e + 5, "%ux%u", &g_td_w, &g_td_h) == 2) g_td_frame = LONG_MAX; }
if (c && !strncmp(e, "size=", 5) && sscanf(e + 5, "%ux%u", &g_td_w, &g_td_h) == 2) g_td_frame = LONG_MAX;
if (c && !strncmp(e, "layers=", 7)) { g_td_layers = atoi(e + 7); g_td_frame = (long)atomic_load(&g_cur_frame); } }
if (g_td_frame < 0 || g_td_done) return;
long f = (long)atomic_load(&g_cur_frame);
if (g_td_layers) {
if (f != g_td_lframe) { if (g_td_lframe >= 0 && g_td_lcount >= g_td_layers) { MLOG("[texdump] frame %ld: %d draws into small array / cube layers", g_td_lframe, g_td_lcount); g_td_frame = g_td_lframe; td_dump(); return; }
g_td_lframe = f; g_td_frame = f; g_td_lcount = 0; [g_td_list removeAllObjects]; }
if (nc) td_layer_count(pd.colorAttachments[0].texture, pd.colorAttachments[0].level); }
if (g_td_frame == LONG_MAX) { id<MTLTexture> t0 = nc ? pd.colorAttachments[0].texture : nil; NSUInteger lv = pd.colorAttachments[0].level;
if (t0 && MAX(t0.width >> lv, 1ul) == g_td_w && MAX(t0.height >> lv, 1ul) == g_td_h) { g_td_frame = f; MLOG("[texdump] first %ux%u render target at frame %ld: dumping this frame", g_td_w, g_td_h, f); } }
if (f > g_td_frame) { td_dump(); return; }
@@ -9188,6 +9481,13 @@ void td_note(MTL4RenderPassDescriptor *pd, NSUInteger nc) {
if (!seen) [g_td_list addObject:@[t, @(ser), @(lv), @(sl)]];
}
}
void td_note_layer(id<MTLTexture> t, uint64_t ser, NSUInteger lv, NSUInteger sl) { // a slice-merged pass's other layers (see g_sm_plan)
if (g_td_frame < 0 || g_td_done || g_td_frame == LONG_MAX || (long)atomic_load(&g_cur_frame) != g_td_frame || !t) return;
if (g_td_layers) td_layer_count(t, lv);
if (t.textureType != MTLTextureType2DArray) sl = 0;
for (NSArray *e in g_td_list) if (e.count < 5 && [e[1] unsignedLongLongValue] == ser && [e[2] unsignedIntegerValue] == lv && [e[3] unsignedIntegerValue] == sl) return;
[g_td_list addObject:@[t, @(ser), @(lv), @(sl)]];
}
// The next frame's targets, on request (the Mac's DM_CTL "texdump <path>", hle_hid.c): the path is kept (strdup), the frame number is the one after the current
// The Mac's DM_CTL "texraw <serial> <path>" (hle_hid.c): level 0 / slice 0 of the host texture behind a serial, raw, after the GPU finishes (header line: width height Metal format bytes per row);
// for a texture that is no render target (a default texture, a lookup table) whose content a shader depends on
+3 -1
View File
@@ -63,7 +63,7 @@ enum {
RO_CB_NEW, RO_CB_BEGIN, RO_CB_END, RO_COMMIT, RO_COMMIT_REAL, RO_SIGNAL,
RO_RE_BEGIN, RO_CE_BEGIN, RO_END,
RO_BAR_QUEUE, RO_BAR_AFTER_STAGES, RO_BAR_ENCODER,
RO_R_TABLE, RO_R_PSO, RO_R_DSS, RO_R_VIEWPORT, RO_R_SCISSOR, RO_R_CULL, RO_R_WINDING, RO_R_FILL, RO_R_BIAS, RO_R_CLIP, RO_R_SREF, RO_R_BLEND, RO_R_VIS,
RO_R_TABLE, RO_R_PSO, RO_R_DSS, RO_R_VIEWPORT, RO_R_SCISSOR, RO_R_CULL, RO_R_WINDING, RO_R_FILL, RO_R_BIAS, RO_R_CLIP, RO_R_SREF, RO_R_BLEND, RO_R_VIS, RO_R_AMP,
RO_R_DRAW, RO_R_DRAW_INST, RO_R_DRAW_IND, RO_R_DRAW_IDX, RO_R_DRAW_IDX_IND, RO_R_STORE_C, RO_R_STORE_D, RO_R_STORE_S,
RO_C_TABLE, RO_C_PSO, RO_C_DISPATCH, RO_C_DISPATCH_IND, RO_C_COPY_BB, RO_C_COPY_BT, RO_C_COPY_TB, RO_C_COPY_TT, RO_C_FILL,
RO_T_ADDR, RO_T_TEX, RO_T_SAMP,
@@ -140,6 +140,7 @@ static void rec_exec(const rec_t *r) {
case RO_R_SREF: [(id<MTL4RenderCommandEncoder>)e setStencilReferenceValue:(uint32_t)a[0]]; break;
case RO_R_BLEND: [(id<MTL4RenderCommandEncoder>)e setBlendColorRed:rec_fv(a[0]) green:rec_fv(a[1]) blue:rec_fv(a[2]) alpha:rec_fv(a[3])]; break;
case RO_R_VIS: [(id<MTL4RenderCommandEncoder>)e setVisibilityResultMode:a[0] offset:a[1]]; break;
case RO_R_AMP: { MTLVertexAmplificationViewMapping m = {(uint32_t)a[1], (uint32_t)a[2]}; [(id<MTL4RenderCommandEncoder>)e setVertexAmplificationCount:a[0] viewMappings:a[0] ? &m : NULL]; break; } // count 0 or 1 (slice-merged passes)
case RO_R_DRAW: [(id<MTL4RenderCommandEncoder>)e drawPrimitives:a[0] vertexStart:a[1] vertexCount:a[2]]; break;
case RO_R_DRAW_INST: [(id<MTL4RenderCommandEncoder>)e drawPrimitives:a[0] vertexStart:a[1] vertexCount:a[2] instanceCount:a[3] baseInstance:a[4]]; break;
case RO_R_DRAW_IND: [(id<MTL4RenderCommandEncoder>)e drawPrimitives:a[0] indirectBuffer:a[1]]; break;
@@ -269,6 +270,7 @@ static inline id rec_table_real(id t) { return [t isKindOfClass:[DmRecTable clas
- (void)setStencilReferenceValue:(uint32_t)v { rec_put(RO_R_SREF, 0)->a[0] = v; }
- (void)setBlendColorRed:(float)r_ green:(float)g blue:(float)b alpha:(float)a { rec_t *r = rec_put(RO_R_BLEND, 0); r->a[0] = rec_f(r_); r->a[1] = rec_f(g); r->a[2] = rec_f(b); r->a[3] = rec_f(a); }
- (void)setVisibilityResultMode:(MTLVisibilityResultMode)m offset:(NSUInteger)o { rec_t *r = rec_put(RO_R_VIS, 0); r->a[0] = m; r->a[1] = o; }
- (void)setVertexAmplificationCount:(NSUInteger)n viewMappings:(const MTLVertexAmplificationViewMapping *)m { rec_t *r = rec_put(RO_R_AMP, 0); r->a[0] = n; r->a[1] = n && m ? m[0].viewportArrayIndexOffset : 0; r->a[2] = n && m ? m[0].renderTargetArrayIndexOffset : 0; }
- (void)drawPrimitives:(MTLPrimitiveType)p vertexStart:(NSUInteger)v vertexCount:(NSUInteger)n { rec_t *r = rec_put(RO_R_DRAW, 0); r->a[0] = p; r->a[1] = v; r->a[2] = n; }
- (void)drawPrimitives:(MTLPrimitiveType)p vertexStart:(NSUInteger)v vertexCount:(NSUInteger)n instanceCount:(NSUInteger)i baseInstance:(NSUInteger)b {
rec_t *r = rec_put(RO_R_DRAW_INST, 0); r->a[0] = p; r->a[1] = v; r->a[2] = n; r->a[3] = i; r->a[4] = b; }
+4
View File
@@ -24,6 +24,10 @@ t_hmap
t_cover
t_fuse_check
t_fuse_check.dSYM/
t_perf_order
t_perf_order.dSYM/
t_perf_order_tsan
t_perf_order_tsan.dSYM/
t_cube_placeholder
t_cube_placeholder.dSYM/
t_present_crop
+13
View File
@@ -73,6 +73,19 @@ thermal-tsan: t_thermal_tsan
run: check_t_thermal
t_fuse_check: t_fuse_check.m $(MTL_T_DEP)
clang $(CFLAGS) -fobjc-arc -o $@ t_fuse_check.m $(MTL_T_SRC) $(FW)
t_perf_order: t_perf_order.m $(MTL_T_DEP)
clang $(CFLAGS) -fobjc-arc -o $@ t_perf_order.m $(MTL_T_SRC) $(FW)
t_perf_order_tsan: t_perf_order.m $(MTL_T_DEP)
clang $(CFLAGS) -fobjc-arc -fsanitize=thread -o $@ t_perf_order.m $(MTL_T_SRC) $(FW)
.PHONY: perf-order-tsan
perf-order-tsan: t_perf_order_tsan
./t_perf_order_tsan --cpu
.PHONY: check_t_perf_order
check_t_perf_order: t_perf_order
MTL_DEBUG_LAYER=1 ./t_perf_order
DM_NVN_ENC_THREAD=1 MTL_DEBUG_LAYER=1 ./t_perf_order
DM_NVN_FAST_WAKE=0 DM_NVN_SLICE_MERGE=0 DM_NVN_CS_HOIST=0 MTL_DEBUG_LAYER=1 ./t_perf_order
run: check_t_perf_order
t_cube_placeholder: t_cube_placeholder.m $(MTL_T_DEP)
clang $(CFLAGS) -fobjc-arc -o $@ t_cube_placeholder.m $(MTL_T_SRC) $(FW)
.PHONY: check_t_cube_placeholder
+2
View File
@@ -28,6 +28,8 @@ static void check_color(int blue, const prog_t *pg, tex_t t[4]) {
mtl_sub_begin(); rp_begin_ex(1); rp_end();
uint64_t f = mtl_sub_commit();
EXPECT("GPU done", 1, mtl_wait(f, 5000000000ull));
// the comparison runs in the submission's commit feedback handler, which a SyncWait woken by the GPU's own signal (DM_NVN_FAST_WAKE) does not wait for
for (int i = 0; i < 2000 && g_fzc_same - same + g_fzc_diff - diff == 0; i++) usleep(1000);
EXPECT(blue ? "negative control: blue merged result vs red plain result, really compared as different attachments" : "physical c3 → logical c0, really compared as the same red attachment",
1, blue ? g_fzc_diff - diff : g_fzc_same - same);
EXPECT("exactly one comparison recorded for this case, none missed or duplicated", 1, g_fzc_same - same + g_fzc_diff - diff);
+200
View File
@@ -0,0 +1,200 @@
// Default performance paths: real replay and Metal pixels, plus a gated CPU
// completion owner. No game shaders/files or frame timing is used as an oracle.
#include "../nvn/nvn_mtl.m"
void *front_metal_layer(void) { return NULL; }
module_t *module_of(const void *p) { (void)p; return NULL; }
module_t g_mod[MAX_MOD]; int g_nmod;
void svc_thread_dump(void) {}
uint64_t now_ns(void) { return bp_now(); }
static int checks, failed;
#define CHECK(s,c) do { int ok=!!(c); checks++; failed+=!ok; printf("%s: %s\n",s,ok?"PASS":"FAIL"); } while(0)
typedef struct { uint8_t p[16384]; size_t n; } chain;
static void emit(chain *c, unsigned op, const void *a, size_t n) {
size_t sz=(8+n+7)&~7ull; if(c->n+sz>sizeof c->p) abort();
op_hdr h={op,(uint32_t)sz}; memcpy(c->p+c->n,&h,8);
if(n) memcpy(c->p+c->n+8,a,n); c->n+=sz;
}
static tex_t texture(int target, int format, int layers) {
static uint64_t serial=800000;
serial++;
tex_t t={.magic=MAGIC('T','E','X',0),.serial=serial,.off=(int64_t)serial<<20};
t.d=(nvn_texdesc){.target=target,.format=format,.width=8,.height=8,.depth=layers,.levels=1,.samples=1};
nvn_tex_layout(&t.d,&t.L); return t;
}
static void rt(chain *c, const tex_t *t, int layer, const tex_t *ds) {
p_set_rt r={.n=1,.depth=ds}; r.color[0]=t;
if(layer>=0) { r.has_cview[0]=1; r.cview[0]=(view_t){.set_layers=1,.min_layer=layer,.nlayers=1}; }
emit(c,OP_SET_RT,&r,sizeof r);
}
static void bind(chain *c, const prog_t *p, int stages) {
p_bind_program b={p,stages}; emit(c,OP_BIND_PROGRAM,&b,sizeof b);
}
static void triangle(chain *c) {
p_draw d={.prim=NVN_DRAW_PRIMITIVE_TRIANGLES,.count=3,.instances=1}; emit(c,OP_DRAW,&d,sizeof d);
}
static void reset(void) {
state_reset(); memset(&R,0,sizeof R); g_csh_n=0;
}
static uint64_t execute(chain *c) {
emit(c,OP_END,NULL,0); reset(); mtl_sub_begin(); mtl_exec(c->p);
uint64_t f=mtl_sub_commit(); CHECK("submission completes",mtl_wait(f,2000000000ull));
mtl_drain(); CHECK("no GPU error",!mtl_last_commit_error()); return f;
}
static NvProg *make_program(id<MTLLibrary> lib, prog_t *pg, const char *fragment) {
static uint64_t serial=900000; pg->serial=++serial;
NvProg *p=[NvProg new]; p->sh[0]=[NvShader new]; p->sh[1]=[NvShader new];
p->sh[0]->fn=[lib newFunctionWithName:@"v"]; p->sh[0]->stage=0;
p->sh[1]->fn=[lib newFunctionWithName:@(fragment)]; p->sh[1]->stage=1;
p->sh[1]->rf.color_known=1; p->sh[1]->rf.color_mask=1;
serial_put(pg->serial,p); return p;
}
static void slices(id<MTLLibrary> lib) {
tex_t t=texture(NVN_TEXTURE_TARGET_2D_ARRAY,NVN_FORMAT_RGBA8,6);
prog_t pg={0}; NvProg *p=make_program(lib,&pg,"red");
chain c={0}; bind(&c,&pg,3);
for(int sl=0;sl<6;sl++) { rt(&c,&t,sl,NULL); triangle(&c); }
// Warm the demanded amplified pipeline before checking the actual merge.
execute(&c);
for(int i=0;i<g_sm_amp_n;i++) {
for(int k=0;k<2000;k++) { os_unfair_lock_lock(&g_sm_amp_lock); int done=g_sm_amp[i].done; os_unfair_lock_unlock(&g_sm_amp_lock); if(done) break; usleep(1000); }
}
c.n-=8; uint64_t before=g_sm_ncont; execute(&c);
CHECK("six independent slices take five continuations",!sm_on() || g_sm_ncont-before==5);
id<MTLTexture> mt=tex_mtl(&t);
id<MTLBuffer> out=[g_dev newBufferWithLength:6*256 options:MTLResourceStorageModeShared];
id<MTLCommandBuffer> cb=[g_q commandBuffer]; id<MTLBlitCommandEncoder> e=[cb blitCommandEncoder];
for(int sl=0;sl<6;sl++) [e copyFromTexture:mt sourceSlice:sl sourceLevel:0 sourceOrigin:MTLOriginMake(0,0,0) sourceSize:MTLSizeMake(8,8,1) toBuffer:out destinationOffset:sl*256 destinationBytesPerRow:32 destinationBytesPerImage:256];
[e endEncoding]; [cb commit]; [cb waitUntilCompleted];
int bad=0; const uint8_t *bytes=out.contents;
for(int i=0;i<6*64;i++) bad+=memcmp(bytes+4*i,(uint8_t[]){255,0,0,255},4)!=0;
CHECK("every pixel of all six layers is red",!bad);
reset(); R.has_rt=1; R.rt.n=1; R.rt.color[0]=&t;
R.rt.has_cview[0]=1; R.rt.cview[0]=(view_t){.set_layers=1,.nlayers=1};
mtl_reg_tex(501,&t,NULL); S.tex[1][0]=501;
p->sh[1]->rf.nb=1; p->sh[1]->rf.b[0]=(nvn_rbind){.kind=NVN_RB_TEX,.reg=0};
chain next={0}; rt(&next,&t,1,NULL); triangle(&next); emit(&next,OP_END,NULL,0); g_exec_next=next.p;
sm_plan(p); CHECK("first draw sampling attachment cannot start slice merge",!g_sm_plan.on);
int kinds[]={NVN_RB_TEXU,NVN_RB_TEXD,NVN_RB_TEXA,NVN_RB_IMGA};
for(unsigned i=0;i<sizeof kinds/sizeof *kinds;i++) {
p->sh[1]->rf.b[0].kind=kinds[i];
CHECK("unknown/bindless attachment reads split merged pass",draw_samples_rt(p));
}
p->sh[1]->rf.nb=0; p->sh[1]->rf.bindless=1;
CHECK("bindless reflection splits merged pass",draw_samples_rt(p));
p->sh[1]->rf.bindless=0; p->sh[1]->rf.dyn=1;
CHECK("dynamic reflection splits merged pass",draw_samples_rt(p));
g_exec_next=NULL; g_sm_plan.on=0;
}
static void hoists(id<MTLLibrary> lib) {
prog_t pg={0}, cp={.serial=910000}; make_program(lib,&pg,"red");
NvProg *cs=[NvProg new]; cs->sh[5]=[NvShader new]; cs->sh[5]->stage=5;
cs->sh[5]->rf.tg[0]=cs->sh[5]->rf.tg[1]=cs->sh[5]->rf.tg[2]=1;
cs->sh[5]->fn=[lib newFunctionWithName:@"compute_one"];
cs->sh[5]->ae=[cs->sh[5]->fn newArgumentEncoderWithBufferIndex:0];
cs->sh[5]->rf.nb=1; cs->sh[5]->rf.b[0]=(nvn_rbind){.kind=NVN_RB_SBUFP,.rw=1,.reg=0,.aid=0};
serial_put(cp.serial,cs);
void *mem=NULL; if(posix_memalign(&mem,getpagesize(),getpagesize())) abort(); memset(mem,0,getpagesize()); mtl_pool_add(mem,getpagesize());
tex_t t=texture(NVN_TEXTURE_TARGET_2D,NVN_FORMAT_RGBA8,1);
for(int hazard=0;hazard<2;hazard++) {
chain c={0}; rt(&c,&t,-1,NULL); bind(&c,&pg,3); bind(&c,&cp,32);
p_bind_buf b={.stage=5,.index=0,.addr=(uint64_t)mem,.size=4}; emit(&c,OP_BIND_SSBO,&b,sizeof b);
triangle(&c);
if(hazard) { uint32_t bits=~0u; emit(&c,OP_BARRIER,&bits,sizeof bits); }
p_dispatch d={.x=1,.y=1,.z=1}; emit(&c,OP_DISPATCH,&d,sizeof d); triangle(&c);
uint64_t before=g_csh_moved; execute(&c);
CHECK(hazard?"explicit game barrier prevents hoist":"independent dispatch hoisted once",g_csh_moved-before==(cshoist_on() && !hazard));
CHECK("compute writes expected shared memory",*(uint32_t *)mem==73);
}
// Drive the production scanner with stencil writes and a stencil-only read.
tex_t ds=texture(NVN_TEXTURE_TARGET_2D,NVN_FORMAT_DEPTH32F_STENCIL8,1);
view_t view={.set_dsmode=1,.dsmode=NVN_TEXTURE_DEPTH_STENCIL_MODE_STENCIL}; mtl_reg_tex(502,&ds,&view);
cs->sh[5]->rf.nb=2; cs->sh[5]->rf.b[1]=(nvn_rbind){.kind=NVN_RB_TEX,.reg=0,.aid=1};
reset(); R.has_rt=1; R.rt.n=1; R.rt.color[0]=&t; R.rt.depth=&ds;
S.cprog=&cp; S.tex[5][0]=502; S.ssbo[5][0]=(typeof(S.ssbo[5][0])){.addr=(uint64_t)mem,.size=4};
S.ds.dw=0; S.ds.st=1; S.ds.zpass_f=NVN_STENCIL_OP_REPLACE; S.st_wmask[0]=255;
chain next={0}; p_dispatch d={.x=1,.y=1,.z=1}; emit(&next,OP_DISPATCH,&d,sizeof d); emit(&next,OP_END,NULL,0);
p_draw dw={.prim=NVN_DRAW_PRIMITIVE_TRIANGLES,.count=3};
uint64_t before=g_csh_moved; mtl_sub_begin(); g_exec_next=next.p; cs_hoist_try(prog_get(&pg),&dw,NULL); mtl_sub_commit(); mtl_drain();
CHECK("stencil read stays after stencil-writing draw",g_csh_moved==before);
g_exec_next=NULL;
reset(); csh_rng ranges[CSH_RN]; int nr=0;
p_draw_idx indirect={.prim=NVN_DRAW_PRIMITIVE_TRIANGLES,.ibuf=(uint64_t)mem,.ind=(uint64_t)mem+512};
CHECK("unknown indirect index range prevents hoist",!csh_draw_reads(prog_get(&pg),NULL,&indirect,ranges,&nr));
tex_t linear=t; linear.mem=mem; linear.d.flags=NVN_TEXTURE_FLAGS_LINEAR; linear.d.target=NVN_TEXTURE_TARGET_2D; linear.L.size=256;
mtl_reg_tex(503,&linear,NULL); S.tex[1][0]=503;
NvProg *drawpr=prog_get(&pg); drawpr->sh[1]->rf.nb=1; drawpr->sh[1]->rf.b[0]=(nvn_rbind){.kind=NVN_RB_TEX,.reg=0};
nr=0; int known=csh_draw_reads(drawpr,&dw,NULL,ranges,&nr);
csh_rng write={(uint64_t)mem+128,(uint64_t)mem+132};
CHECK("sampled linear storage conflicts with compute write",!known || csh_hit(ranges,nr,&write,1));
drawpr->sh[1]->rf.b[0].kind=NVN_RB_TEXU; nr=0;
CHECK("unresolved draw texture storage prevents hoist",!csh_draw_reads(drawpr,&dw,NULL,ranges,&nr));
R.has_rt=1; R.rt.n=1; R.rt.color[0]=&linear;
cs->sh[5]->rf.nb=1; cs->sh[5]->rf.b[0]=(nvn_rbind){.kind=NVN_RB_UBOP,.reg=0};
S.ubo[5][0]=(p_bind_buf){.addr=(uint64_t)mem,.size=4}; nr=0; int depth=0;
CHECK("compute uniform read cannot precede aliased attachment write",!csh_cs(cs->sh[5],&d,ranges,&nr,&depth));
cs->sh[5]->rf.b[0].kind=NVN_RB_SBUFP;
S.ssbo[5][0]=S.ubo[5][0]; nr=0;
CHECK("compute storage access cannot precede aliased attachment write",!csh_cs(cs->sh[5],&d,ranges,&nr,&depth));
tex_t overlapping=linear; overlapping.serial++; overlapping.mem=(uint8_t *)mem+64; overlapping.L.size=128;
mtl_reg_tex(504,&overlapping,NULL); S.tex[5][0]=504;
cs->sh[5]->rf.b[0].kind=NVN_RB_TEX; nr=0;
CHECK("offset compute texture read cannot precede overlapping target",!csh_cs(cs->sh[5],&d,ranges,&nr,&depth));
S.tex[1][0]=504; drawpr->sh[1]->rf.b[0].kind=NVN_RB_TEX;
CHECK("offset sampled storage splits merged pass",draw_samples_rt(drawpr));
mtl_pool_del(mem); free(mem);
}
static dispatch_semaphore_t waiter_started, waiter_done;
static void *early_waiter(void *arg) {
dispatch_semaphore_signal(waiter_started); post_early((uintptr_t)arg); dispatch_semaphore_signal(waiter_done); return NULL;
}
static void posts(void) {
// Pause the real completion owner between claim and its first event write.
uint64_t f=g_fence+1; post_rec *r=&g_postr[f%POST_RING]; volatile uint32_t event=0;
r->ops=calloc(1,sizeof *r->ops); r->ops[0]=(post_op){.kind=1,.val=&event,.v=47}; r->n=1; r->fence=f; r->claimed=0;
CHECK("completion owner claimed",post_claim(r,f));
waiter_started=dispatch_semaphore_create(0); waiter_done=dispatch_semaphore_create(0);
pthread_t th; pthread_create(&th,NULL,early_waiter,(void *)(uintptr_t)f);
dispatch_semaphore_wait(waiter_started,DISPATCH_TIME_FOREVER);
int returned=dispatch_semaphore_wait(waiter_done,dispatch_time(DISPATCH_TIME_NOW,50000000))==0;
CHECK("competing waiter waits for CPU effects",!returned);
CHECK("fence remains incomplete while owner is paused",g_done<f);
post_run(r,0,0); if(!returned) dispatch_semaphore_wait(waiter_done,DISPATCH_TIME_FOREVER); pthread_join(th,NULL);
CHECK("event visible before successful wait",event==47 && g_done>=f);
// A late callback must not touch the new generation of its old ring slot.
r->fence=f+POST_RING; r->claimed=0; r->n=1;
r->ops=calloc(1,sizeof *r->ops); r->ops[0]=(post_op){.kind=1,.val=&event,.mode=1,.v=3};
int stale=post_claim(r,f);
CHECK("old fence cannot claim recycled slot",!stale);
if(stale) post_run(r,0,0);
post_complete(f,0,0);
CHECK("old callback leaves future event untouched",event==47);
free(r->ops); r->ops=NULL; r->n=0; r->claimed=1;
post_rec *a=&g_postr[(f+1)%POST_RING], *b=&g_postr[(f+2)%POST_RING];
*a=(post_rec){.fence=f+1,.n=1,.ops=calloc(1,sizeof(post_op))};
*b=(post_rec){.fence=f+2,.n=1,.ops=calloc(1,sizeof(post_op))};
a->ops[0]=(post_op){.kind=1,.val=&event,.v=10};
b->ops[0]=(post_op){.kind=1,.val=&event,.mode=1,.v=5};
post_complete(f+2,0,0);
CHECK("later completion applies earlier events in order",event==15 && g_done==f+2);
post_complete(f+1,0,0); post_complete(f+2,0,0);
CHECK("callbacks never repeat an event",event==15);
}
int main(int argc, char **argv) { @autoreleasepool {
if(argc==2 && !strcmp(argv[1],"--cpu")) {
alarm(10); setbuf(stdout,NULL); posts();
printf("CPU completion: %d checks, %d failed\n",checks,failed); return failed!=0;
}
alarm(30); setbuf(stdout,NULL); mtl_init();
NSError *error=nil;
id<MTLLibrary> lib=[g_dev newLibraryWithSource:@"#include <metal_stdlib>\nusing namespace metal;\n"
"vertex float4 v(uint i [[vertex_id]]) { float2 p[3]={float2(-1,-1),float2(3,-1),float2(-1,3)}; return float4(p[i],0,1); }\n"
"fragment float4 red() { return float4(1,0,0,1); }\n"
"struct Args { device uint *p [[id(0)]]; };\n"
"kernel void compute_one(constant Args &a [[buffer(0)]]) { *a.p=73; }\n" options:nil error:&error];
if(!lib) { fprintf(stderr,"%s\n",error.localizedDescription.UTF8String); return 2; }
slices(lib); hoists(lib); posts();
printf("performance ordering: %d checks, %d failed\n",checks,failed); return failed!=0;
} }
+1 -1
View File
@@ -65,7 +65,7 @@ What earlier titles needed beyond sections 1–5 is filed by mechanism: textures
| `DM_NVN_SHOT=f1,f2:path_%d.png` | Window screenshots at those presented frames |
| `DM_NVN_PASSDUMP=f`, `f1-f2` or `every:n` (+ `DM_NVN_PASSDUMP_ALL=1`) | Every render pass of those frames (`every:n`: one frame in every n, halfway between two `DM_NVN_GPUPROF=every:n` frames): `P` attachments (with `mtl=` the Metal texture behind them), `C` clear colours, `K` shader keys, `B` each binding's actual source (also with `mtl=`), `d`/`D` draw state (`b0`–`b3` the blend states of targets 0–3; on `d`, `po=` the polygon offset enables / factor, units, clamp and `dc=` depth clamp), `Y` inline updates, `X` texture clears, `T` texture copies, `Z` the game's barriers (their bits, and whether one ended the pass), `N` passes skipped because no attachment could be attached, `C` buffer-to-texture copies (target, size, format, region, source address, row and image strides), `W` image bindings of compute dispatches; a buffer texture's `B` line gives its format, element count and storage address (for the Mac's `DM_CTL` `memdump`). Without `_ALL` the `K` / `B` / `R` lines cover only the first draw of each pass |
| `DM_NVN_TEXDUMP=f:path_%d.png` (+ `DM_NVN_TEXSNAP=i,…`) | Every colour render target of frame f, as PNG (`%d` = serial × 100 + layer × 10 + level; 2D arrays by the attached layer). `TEXSNAP` copies the attachments of in-frame pass i (2D, or slice 0 of a 2D array) in a separate command buffer, which does not run at the pass's position in the GPU timeline: a snapshot taken right after a clearing pass showed a complete image, so read it as "what this target holds around the frame", not "right after pass i" |
| `DM_NVN_TEXDUMP=size=WxH:path_%d.png` | The same dump for the first frame that opens a render pass whose first colour attachment level is W×H: for one-off work (a mask built when a prompt appears) whose frame differs between runs. With `DM_NVN_SNAPRAW=1` each image also gets a `.raw` file with the alpha channel |
| `DM_NVN_TEXDUMP=size=WxH:path_%d.png` | The same dump for the first frame that opens a render pass whose first colour attachment level is W×H: for one-off work (a mask built when a prompt appears) whose frame differs between runs. `layers=<n>` instead of `size=` takes the first frame with at least n draws into single layers of small (at most 64×64) array or cube levels (Breath of the Wild's cube map filter frame), the same frame with or without `DM_NVN_SLICE_MERGE`. With `DM_NVN_SNAPRAW=1` each image also gets a `.raw` file with the alpha channel |
| `DM_NVN_CBDUMP=dir` (with PASSDUMP) | Raw uniform buffer contents per binding, and the source bytes of every buffer-to-texture copy into a 3D texture (`F<frame>_c3d_<serial>.bin`: what the game uploaded, before any layout question) |
| `DM_NVN_HILITE=fskey,…` | Replace those fragment shaders with solid colours: tells "geometry is wrong" from "shading is wrong" |
| `DM_NVN_DEPTHOVR=fskey,…` | Those fragment shaders' draws compare depth with `ALWAYS`: with `HILITE`, tells "the depth test rejects every fragment" from "nothing rasterizes". With `PASSDUMP`, the `D` line shows the depth range (`dr=`) and logic op (`lo=`), and `c` lines each recorded clear (slot, serial, mask, scissor; depth / stencil values) |
+3
View File
@@ -251,6 +251,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
- `nvnCommandBufferFenceSync` unimplemented, every `SyncWait` timed out at once. → [Fence syncs recorded in command buffers](../reference/nvn-objects.md#fence-syncs-recorded-in-command-buffers)
- `FATAL submit re-entered` in the first frames. → [Submissions from two queues on two threads](../reference/nvn-objects.md#submissions-from-two-queues-on-two-threads)
- frames stop, no crash, the main thread sits in `nvnSyncWait(sync, -1)`, and the watchdog's wait-state line shows the main fence one short (`main fence submitted N / completed N-1`). → [A GPU command buffer that never completes](../reference/nvn-objects.md#a-gpu-command-buffer-that-never-completes)
- the frame's blocking SyncWait on its newest fence wakes about 0.26–0.29 ms after the GPU end (M2 Max; the wait ledger, `DM_NVN_WAITS=1`), on the critical path of every frame. → [The game's SyncWait wakes on the GPU's own signal of its fence](../reference/nvn-objects.md#the-games-syncwait-wakes-on-the-gpus-own-signal-of-its-fence)
- loading screens many times slower on the iPhone than on the Mac (Three Houses' first loading screen 78 s on the iPhone, 5 s on the Mac). → [`nvnQueueFinish` waits for the GPU, not the display](../reference/nvn-objects.md#nvnqueuefinish-waits-for-the-gpu-not-the-display)
- driver comparisons report the wrong block dimensions for packaged 2D / 3D textures, or too little storage for linear block-compressed textures (64x64 BC1 with stride 128: 2048 bytes instead of 8192). → [Packaged texture block dimensions and linear storage rows](../reference/nvn-objects.md#packaged-texture-block-dimensions-and-linear-storage-rows)
- menus and dialogue boxes draw but contain no text, while images and pictogram digits are fine; the sky is black (CPU-written linear colour ramps). → [Linear textures in CPU-visible pools](../reference/nvn-objects.md#linear-textures-in-cpu-visible-pools)
@@ -303,6 +304,8 @@ One line per symptom in the [subsystem references](../reference/): what you see
- a G-buffer target holds only scattered squares (a different order for each 32×32 tile). → [The same texture attached at two colour slots](../reference/rendering.md#the-same-texture-attached-at-two-colour-slots)
- `no render target could get an MTLTexture` (the pass skipped). → [Passes with no render target](../reference/rendering.md#passes-with-no-render-target)
- a black world: everything rendered through a 3D LUT built by layered draws comes out black (every plane went to plane 0); `DM_NVN_PASSDUMP` shows no `L` line. → [Layered passes for vertex shaders that write the layer](../reference/rendering.md#layered-passes-for-vertex-shaders-that-write-the-layer)
- a frame-time spike every few frames (the Metal HUD's frame-time line jumps by several ms at a regular interval) while the scene stands still; `DM_NVN_GPUPROF` over consecutive frames shows the slow frame with about 130 more render passes, each one draw into a 64x64 to 8x8 face of a cube or array level, and the next pass starting late. → [Passes into other layers of one texture level run as one layered pass](../reference/rendering.md#passes-into-other-layers-of-one-texture-level-run-as-one-layered-pass)
- a GPU-bound frame with many passes on the same targets, one after the other, each preceded by a compute encoder of one dispatch (`DM_NVN_GPUPROF`: `compute / copy after pass … dispatches 1` between render passes of the same size); about 4 ms of GPU idle a frame inside command buffers in Metal System Trace. → [Compute dispatches between the draws of a pass run ahead of it](../reference/rendering.md#compute-dispatches-between-the-draws-of-a-pass-run-ahead-of-it)
- a scene drawn only in a small square in the top-left corner of a black screen; speckled and blocky patches; a full-screen overlay showing one tile; the texture dump right after the scene pass shows the picture only in the corner, and `DM_NVN_HILITE` on the scene's shaders lights only the corner. → [Clears of attachments larger than the render pass](../reference/rendering.md#clears-of-attachments-larger-than-the-render-pass)
- `texture clear case not implemented … mask 0x1`. → [A channel mask that covers every channel is a whole clear](../reference/rendering.md#a-channel-mask-that-covers-every-channel-is-a-whole-clear)
- `texture clear case not implemented … mask 0x8` while entering a rendered scene. → [Texture clears with partial channel masks or regions](../reference/rendering.md#texture-clears-with-partial-channel-masks-or-regions)
+1
View File
@@ -139,6 +139,7 @@ Frame pacing: the host's frame step, engine frame limiters, present intervals an
- **Fix:** the replay commits the command buffer encoded so far every 64 draws, between two passes (`sub_cut` in `nvn_mtl.m`, called after `rp_flush` in `OP_SET_RT`; `DM_NVN_CUT=<draws>`, 0 = off). It cuts only with no open encoder and nothing deferred, merged or hoisted, commits the pending upload batch first, and continues on the same command allocator. The fence signal, the completion work (`post` ops, `g_done`) and the allocator's fence stay on the last segment: the queue runs command buffers in commit order, and every encoder starts with a queue barrier.
- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): a stack sample in the shrine had the main thread in `nvnSyncWait` (the fence of the frame just submitted) for 59% of the time and the render thread in `nvnQueueSubmitCommands` for 40%; the translation was about 3,600 draws and 347 passes, 7 ms, so per frame the GPU ran 13–14 ms and then sat idle 7.7 ms (`DM_NVN_TSV` rows `@gpu_beg_ms` / `@gpu_end_ms` against `@call_ms`). GPU idle per frame went from 7.7 to 3.8 ms; most of the rest is the game's own work between its fence wait and the next submit (about 2.5 ms). The other five games' frame times were unchanged; Metroid Dread's p95 dropped from 19.6 to 16.9 ms.
- **Note:** after this, BotW's docked frame p50 was 16.7 ms (was 20.8 ms before segmenting). Host translation is 8 ms per frame (22,000 instructions per draw), spread over Metal's encoder creation and draw calls, argument encoding, Objective-C retain / release and pool lookups. The per-pass GPU timer (`DM_NVN_PASSGPU`) is switched off under Metal 4 (`pg_render` returns at once), so GPU time cannot yet be attributed to passes. Render-target traffic per frame is in `DM_NVN_TSV`'s `@rt_load_mb` / `@rt_store_mb`.
- **Note:** with the encode thread (`DM_NVN_ENC_THREAD`) a segment's commit record is handed to it at once (`rec_publish` in `sub_cut`), not when its block of records fills. Breath of the Wild on an M2 Max (encode thread and snapshots, `DM_FPS_TARGET=120`): the frame's wait on its newest fence 12.5 ms with `DM_NVN_CUT=0`, 5.6 ms with 64, 5.35 ms with 16.
### Address to memory pool lookups by binary search
+10
View File
@@ -162,6 +162,16 @@ Games read this count as a position in their own command memory and size their h
- **Check:** GPU load from outside: `ioreg -r -d 1 -c IOAccelerator | grep 'Device Utilization'` at 100 % means a shader is looping forever. Find the shaders first used in the last frames (`DM_NVN_PASSDUMP`, compare the `K` keys) and look at their `.metal` for an empty `do {} while (true);` with no `return` (one such case: [Exact program start for the decoder](shaders.md#exact-program-start-for-the-decoder)).
- **Found on:** no game named in the source (general).
### The game's SyncWait wakes on the GPU's own signal of its fence
- **Symptom:** the frame's blocking SyncWait on its newest fence wakes about 0.26–0.29 ms after the GPU end (M2 Max; the wait ledger, `DM_NVN_WAITS=1`), on the critical path of every frame.
- **Cause:** the waiter slept until the commit feedback handler of the fence's submission had run, which comes that long after the GPU end.
- **Fix:** the game's waits (newest fence, older fence, acquire sync) wait on `g_ev`, which the queue signals after each submission, and then make the CPU-side writes of every submission up to the fence that no handler has made yet (`post_early`: events, samples-passed and counter reports; a ring of `POST_RING` records, each run once by whichever comes first). A counter report of the fence's last segment takes the wake as that segment's end. `DM_NVN_FAST_WAKE=0` turns it off.
- **Note:** the commit feedback handler's other blocks (diagnostics such as `DM_NVN_FUSE_CHECK`) still run when the handler does, after a fast-woken wait has returned.
- **Resolved in review of #158:** claiming a record is not completion. The original early path skipped records a handler or another waiter owned and advanced `g_done` before their writes finished; a delayed callback also retained a recyclable slot pointer. `post_claim` now holds `g_post_mu` through `post_run`, and both contenders call `post_complete` with their fence to apply earlier records in order. A slot is reused under that lock only after its writes finish; an old fence cannot claim its new generation. `g_done` and `g_fence` are atomic.
- **Check:** `make -C dm/gfx/tests GAME=010015100B514000 check_t_perf_order` covers a paused completion owner, a competing waiter, recycled-slot generations, event order and exactly-once writes, with direct/deferred encoding and fast wake disabled. `t_perf_order --cpu` and `perf-order-tsan` run the completion cases without a GPU in CI. The original harness reproduced premature completion; the corrected CPU cases pass ThreadSanitizer. Current gameplay coverage and timing limits are recorded in [BotW](../../dm/games/01007EF00011E000/README.md#final-synchronization-and-pass-validation-2026-10-26-current) and [Metroid Dread](../../dm/games/010093801237C000/README.md#final-gameplay-validation-2026-10-26-current); [13 Sentinels](../../dm/games/01003FC01670C000/README.md#rendering) has additional menu/report traffic coverage.
- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md): it reads 5 timestamp reports a frame right after the wait (its interval controller and dynamic resolution), which is why the waiter makes them itself. Wake after the GPU end 0.29 → 0.08 ms on an M2 Max, 0.147 ms on an iPhone 17 Pro Max.
### `nvnQueueFinish` waits for the GPU, not the display
- **Symptom:** loading screens many times slower on the iPhone than on the Mac (Three Houses' first loading screen 78 s on the iPhone, 5 s on the Mac).
+19
View File
@@ -204,6 +204,25 @@ The replay on Metal (`dm/gfx/nvn/nvn_mtl.m`) and the Vulkan and OpenGL layers. E
- **Fix:** a draw whose vertex shader writes the layer opens a layered pass (`g_rp_layered` in `draw`, `g_pass.layered` in `rp_begin_ex`): the pass gets every layer the attachments have from their attached one on (3D textures: depth planes), and it is split when the next draw's shader differs. `DM_NVN_PASSDUMP` prints `L … layered pass: n layers`.
- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): the 3D scene came out black once the opening reached Lumiose. The game builds its 32×32×32 colour grading LUT with `b29f08619f379a83`, one instance per depth plane, each writing its plane through `render_target_array_index`, and tone-maps the frame through that LUT.
### Passes into other layers of one texture level run as one layered pass
- **Symptom:** a frame-time spike every few frames (the Metal HUD's frame-time line jumps by several ms at a regular interval) while the scene stands still; `DM_NVN_GPUPROF` over consecutive frames shows the slow frame with about 130 more render passes, each one draw into a 64x64 to 8x8 face of a cube or array level, and the next pass starting late.
- **Cause:** on Apple GPUs a render pass costs about 35–40 µs whatever it draws (M2 Max, with or without barriers: a Metal 4 microbenchmark of 132 one-draw 64x64 cube-face passes took 5.4 ms, 4.2 ms without barriers). An environment cube map filtered face by face, one pass per face and level, pays that cost once per face.
- **Fix:** the draw that opens a pass looks ahead (`sm_scan`, `sm_plan` in `nvn_mtl.m`): when its only target is one layer of an array or cube texture level and the following targets switch to other layers of the same level with nothing but state, uniform updates and draws between them (no game barrier, clear, copy or dispatch), the pass is opened over those layers (`renderTargetArrayLength`) and `OP_SET_RT` switches layers inside it. Each draw reaches its layer through a vertex amplification view mapping (count 1, `renderTargetArrayIndexOffset`); a pipeline built with the default `maxVertexAmplificationCount` of 1 ignores the mapping (every draw landed in the first layer), so these draws use pipelines built with 2 (`sm_amp_get`, keyed `AMP_MARK`), compiled in the background on first use while the passes stay apart. A draw that samples the attachment or a clear ends the merged pass. `DM_NVN_SLICE_MERGE=0` turns it off; `DM_NVN_PASSDUMP` prints `L … slice-merged pass` and `M … goes on in layer n`, and the residency set line counts merged passes and layer switches.
- **Found on:** [Breath of the Wild](../../dm/games/01007EF00011E000/README.md): every tenth frame filters the environment cube map in 132 passes (22 levels of 6 faces); 284 → 173 render passes in that frame, frame p95 13.2 → 11.1 ms at 900p on an M2 Max.
- **Check:** `DM_NVN_TEXDUMP=layers=100:path` dumps the first such frame with and without the switch: the 173 targets matched (105 identical, the rest within 1% RMSE, the scene's rain).
- **Resolved in review of #213:** the first draw now gets the attachment-read check before planning a merged pass. Every later check refuses unknown bindless/dynamic reads (`TEXU`, `TEXD`, `TEXA`, image atomics and their reflection flags), and considers textures sharing the attachment's storage. Slot-only reads cannot prove those draws independent. `t_perf_order` checks all six array layers' pixels and five actual continuations after pipeline warm-up, plus the first-draw and unknown-read refusals. It runs with both encoding modes and merging disabled.
### Compute dispatches between the draws of a pass run ahead of it
- **Symptom:** a GPU-bound frame with many passes on the same targets, one after the other, each preceded by a compute encoder of one dispatch (`DM_NVN_GPUPROF`: `compute / copy after pass … dispatches 1` between render passes of the same size); about 4 ms of GPU idle a frame inside command buffers in Metal System Trace.
- **Cause:** Metal cannot dispatch compute inside a render pass, so every dispatch between two draws ends the pass; on Apple GPUs each extra pass and encoder costs its fixed 35–40 µs plus the store and reload of every attachment.
- **Fix:** `cs_hoist_try` (`nvn_mtl.m`, `DM_NVN_CS_HOIST`, on by default, `0` off): when a draw opens a pass, the ops up to the first one that would end the pass anyway are scanned, and each dispatch met on the way runs before the pass when moving it ahead of the draws before it changes nothing they read (no storage buffer they read, no colour attachment of the pass sampled, no image written, the draws write no memory). A dispatch that samples the pass's depth moves only while no draw before it in the pass writes depth, so it sees the same depth. `DM_NVN_PASSDUMP` prints `H … dispatch … moved ahead of the pass`; the residency set line counts moved dispatches and why scans stopped.
- **Found on:** [Breath of the Wild](../../dm/games/01007EF00011E000/README.md): the NintendoWare VFX GPU particles run one dispatch per emitter between the effect draws of the G-buffer and of the half-resolution passes; 28 → 8 compute encoders and 149 → 132 render passes a frame, frame p50 9.9 → 9.0 ms at 900p on an M2 Max. Its particle collision samples the G-buffer depth, and no draw before those dispatches writes depth (counted: 0 stops).
- **Check:** the cube-filter frame's 173 targets dumped with and without (`DM_NVN_TEXDUMP=layers=100`): every 3D target within 0.7% RMSE; only the 2D overlay of the final picture differed (a loading banner's animation, timed by the wall clock).
- **Resolved in review of #213:** a nonzero game barrier stops the scan. A sample of depth-stencil storage cannot move ahead of either a depth or stencil write. Unknown indexed-indirect ranges and bindless/dynamic draw texture reads refuse hoisting; fixed-slot sampled textures contribute their storage ranges (including linear and buffer-backed textures). Sparse virtual addresses, unknown sizes and overflowing ranges do not prove independence. Compute uniform/storage accesses overlapping attachment storage also refuse hoisting; sampled texture storage is compared by extent, including different offsets into one linear buffer. Texture registration uses its actual unfair lock. These are general dependency checks, without profile rules.
- **Regression:** `t_perf_order` drives the production replay with an independent dispatch (moved exactly once, GPU writes 73), an explicit barrier (not moved), and the scanner with a stencil-only view after a stencil-writing draw. It also checks indirect index ranges, sampled buffer storage, unknown texture reads and aliased attachments. The initial PR failed 13 harness assertions; restoring only its depth-only write guard makes the stencil case fail too. The fixed harness passes under Metal validation in both encoding modes and with optimizations disabled. [BotW gameplay](../../dm/games/01007EF00011E000/README.md#final-synchronization-and-pass-validation-2026-10-26-current) exercises both optimizations on/off; it retains pass savings but does not demonstrate a timing gain in the tested 60 fps workload. [Metroid Dread gameplay](../../dm/games/010093801237C000/README.md#final-gameplay-validation-2026-10-26-current) broadens coverage without activating either optimization.
### Clears of attachments larger than the render pass
- **Symptom:** a scene drawn only in a small square in the top-left corner of a black screen; speckled and blocky patches; a full-screen overlay showing one tile; the texture dump right after the scene pass shows the picture only in the corner, and `DM_NVN_HILITE` on the scene's shaders lights only the corner.