diff --git a/dm/games/010029B018432000/README.md b/dm/games/010029B018432000/README.md index afde7a9..6a690f7 100644 --- a/dm/games/010029B018432000/README.md +++ b/dm/games/010029B018432000/README.md @@ -20,7 +20,7 @@ No `fps_target`: the game runs at 60 on the Switch. No `pad_layout`: the virtual ## Files and shaders - One RE Engine PAK, `re_chunk_000.pak` (2.66 GB), and `natives/NSW/runtime.args`. `romfs_marker` is `re_chunk_000.pak`. -- **Current:** 7,744 shaders from the PAK's SDF definitions and `main` (3,731 pixel, 1,559 vertex, 2,401 compute, 20 geometry, 15 hull, 18 domain) and 8,530 pairs ([SDF definitions in RE Engine PAKs](../../../docs/reference/shaders.md#sdf-definitions-in-re-engine-paks)). 7,739 build; the other 5 are compute shaders that do atomics on a 3D image (`16a78d0fd09ccc18`, `33559ae5667779fe`, `7c65836255e05a87`, `a838b76ccc822be7`, `ecef686222a38548`), which SPIRV-Cross's MSL 2.3 emulation indexes with an `int3` (`array subscript is not an integer`), as Octopath Traveler 0's 16 ([Unreal Engine 5 shader code archives (Oodle)](../../../docs/reference/shaders.md#unreal-engine-5-shader-code-archives-oodle)). None was requested in the runs above (a missing one prints a `[RED] failed to load shader` line). +- **Current:** 7,744 shaders from the PAK's SDF definitions and `main` (3,731 pixel, 1,559 vertex, 2,401 compute, 20 geometry, 15 hull, 18 domain) and 8,530 pairs ([SDF definitions in RE Engine PAKs](../../../docs/reference/shaders.md#sdf-definitions-in-re-engine-paks)). 7,739 built on chain `dec5ee141a9f`; the other 5 are compute shaders that do atomics on a 3D image (`16a78d0fd09ccc18`, `33559ae5667779fe`, `7c65836255e05a87`, `a838b76ccc822be7`, `ecef686222a38548`), which SPIRV-Cross's MSL 2.3 emulation indexed with an `int3` (`array subscript is not an integer`), as Octopath Traveler 0's 16. With native image atomics all 5 build (a build of those keys on chain `0ce4b5e8aefb`, 2026-10-27; `33559ae5667779fe` is a compare-and-swap): [Image atomics](../../../docs/reference/shaders.md#image-atomics). The whole library was not rebuilt and none of the 5 was requested in the runs above (a missing one prints a `[RED] failed to load shader` line). - PR #138 (not merged) built them by flattening the coordinate in `dmsh_build.py` (`x + width × (y + height × z)` into the atomics pointer); that is a change of the chain, so every game's libraries would be rebuilt. It is left out until a run of this game requests one of them. ## Movies and audio diff --git a/dm/games/01005270232F2000/README.md b/dm/games/01005270232F2000/README.md index 7941888..6a3da72 100644 --- a/dm/games/01005270232F2000/README.md +++ b/dm/games/01005270232F2000/README.md @@ -39,9 +39,9 @@ No `pad_layout` key: the iPhone gamepad starts in the 2D layout ([README](../../ - Vertex and pixel shaders combine per draw from Unreal's pipeline state, so every vertex shader is paired with the single-stage cover and made to link with any pixel shader (`shader_split.py` `separable_pairs`, `dmsh_build.py` `vs_pad`). - Bindings: separate textures and samplers, as Engage's (`0x168` / `0x568`). - **Current: 61 vertex shaders pick a texture slot at run time** (particles reading their data from buffer textures). The shader reads one of several c2 texture-table words, chosen by branches, and ORs it with a sampler-table word; the translator cannot trace that to one slot and falls back to Ryujinx's bindless arrays (`array, 32> vp_tb_cb0_40000000`, `array vp_s_cb0_40000000`), which used not to load. Every integer read of c2 in those shaders is a table slot (texture slots 0-10, samplers 0-4), so `tex_dyn_slots` is right here. Host only: the chain version does not change. -- **Current: 2,084 shaders do atomics on buffer images** (Unreal's GPU particles; three ran in the first 100 seconds), now bound to the image's memory. +- **Current: 2,084 shaders do atomics on buffer images** (Unreal's GPU particles), native texture atomics on the buffer texture since chain `0ce4b5e8aefb` ([Image atomics](../../../docs/reference/shaders.md#image-atomics)). On the fresh-save route (2026-10-27, Mac) three compute shaders run them (`766bc02c3b4c76d3`, `65327534de18f480`, `df56bcd022bf1e73`, image slot 2, texture ID 2049: an 8 KB R32Uint buffer texture), with no red line to frame 14,400 under Metal API validation (no validation error). - **Current: a buffer texture at pool offset `0x168`**, read by the particle shaders, is created there (it used to be left a dummy, below Metal's reported 16-byte alignment). -- **Unbuilt:** 16 compute shaders do atomics on a 3D `r32ui` image, which SPIRV-Cross's MSL 2.3 image-atomic emulation indexes with an `int3` (`array subscript is not an integer`). All are Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run). +- **Resolved: 16 compute shaders with atomics on a 3D `r32ui` image did not build** (SPIRV-Cross's MSL 2.3 emulation indexed its pointer with the `int3` coordinate: `array subscript is not an integer`; GitHub issue #290 listed them beside 2,077 returning atomics, which build since `2e655b7`: `40b643b` builds 10,308 of 10,324). With native image atomics all 10,324 build (chain `0ce4b5e8aefb`, 2026-10-27). They are Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run). ## Files @@ -82,7 +82,7 @@ General host changes first needed here; each applies to every game (or every Unr - **Unreal's frame limits written in game memory** (`hle_fpslock.c`, `fps_target`), explained in [Frame rate](#frame-rate) above. - **Game arguments set before `nninitStartup`** (`main.c`, profile key `game_args`): [Unreal's engine address region and its options](../../../docs/reference/memory.md#unreals-engine-address-region-and-its-options). - **Unreal pak reading and Oodle-compressed shaders decoded by the game's own Oodle** (`uepak.py`, `DM_OODLE_JOB`, `decode_ue_shaders`): [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content), [Unreal Engine 5 shader code archives (Oodle)](../../../docs/reference/shaders.md#unreal-engine-5-shader-code-archives-oodle). -- **Texture slots picked at run time** (`tex_dyn_slots`, `NVN_RB_TEXD` / `NVN_RB_SAMPD`), **image atomics bound to the image's storage** (`NVN_RB_IMGA`, `atom_scratch`), **buffer textures below 16 bytes**: [Texture slots picked at run time](../../../docs/reference/shaders.md#texture-slots-picked-at-run-time), [Image atomics](../../../docs/reference/shaders.md#image-atomics), [Buffer textures at offsets below 16 bytes](../../../docs/reference/rendering.md#buffer-textures-at-offsets-below-16-bytes). +- **Texture slots picked at run time** (`tex_dyn_slots`, `NVN_RB_TEXD` / `NVN_RB_SAMPD`), **image atomics, now native texture atomics** (`dmsh_atomic.py`, `img_usage`, `dummy_img_atomic`), **buffer textures below 16 bytes**: [Texture slots picked at run time](../../../docs/reference/shaders.md#texture-slots-picked-at-run-time), [Image atomics](../../../docs/reference/shaders.md#image-atomics), [Buffer textures at offsets below 16 bytes](../../../docs/reference/rendering.md#buffer-textures-at-offsets-below-16-bytes). - **Rewriter:** byte tables far from their `br` (`jump_bytes_far`): [Byte tables whose add is far from the br](../../../docs/reference/rewriter.md#byte-tables-whose-add-is-far-from-the-br). Not first found here but met here: long functions analysed in pieces (`pieces`, [Long functions analysed in pieces](../../../docs/reference/rewriter.md#long-functions-analysed-in-pieces)) and spin waits that store (`patch_nso.py` "Spin waits", [development.md, Power](../../../docs/development.md#power)). - **Undefined weak imports bound to 0** (`loader.c`), **`vi::GetLatestFrameNumber`** (`hle_vi.c`), **`fs::GetSaveDataSizeMax`** (`hle_fs.c`), **NVN pool getters and `nvnCommandBufferFenceSync`** (`OP_FENCE_SYNC`), **the guest clock left running behind the software keyboard** (`hle_swkbd.c`, `front_ios.m`, `guest_clock_freeze_for`): [Undefined weak imports bind to 0](../../../docs/reference/services.md#undefined-weak-imports-bind-to-0), [Layer frame numbers](../../../docs/reference/services.md#layer-frame-numbers), [Save data size queries](../../../docs/reference/services.md#save-data-size-queries), [Pool size getters, texture address and command memory size](../../../docs/reference/nvn-objects.md#pool-size-getters-texture-address-and-command-memory-size), [Fence syncs recorded in command buffers](../../../docs/reference/nvn-objects.md#fence-syncs-recorded-in-command-buffers), [The software keyboard blocks only its caller](../../../docs/reference/services.md#the-software-keyboard-blocks-only-its-caller). - **`DM_ALIAS_WINDOWS`**, the alias-region window census: [05, Memory growth](../../../docs/agents/05-debugging.md#memory-growth). diff --git a/dm/games/010057D006492000/README.md b/dm/games/010057D006492000/README.md index 67a9520..346d9f2 100644 --- a/dm/games/010057D006492000/README.md +++ b/dm/games/010057D006492000/README.md @@ -27,7 +27,7 @@ Status: brought up on the Mac: the boot, the language notice, the title, a new g - **Resolved: the world map on the traveler selection a patchwork of rectangles:** [Sparse textures take the block height of their 64 KB tile](../../../docs/reference/rendering.md#sparse-textures-take-the-block-height-of-their-64-kb-tile). - **Resolved: streamed textures black or missing** (S'warkii's ground, plants and props) and `[RED] MapVirtual: non-contiguous mapping (true sparse) not implemented`: [Virtual pools of one texture each](../../../docs/reference/nvn-objects.md#virtual-pools-of-one-texture-each-mapped-before-and-after-the-texture). - **Current (checked, not confirmed on hardware): a dark red cast at the screen edges in S'warkii.** The tonemapper is right (checked by snapshots of its inputs, `DM_NVN_TEXSNAP`): its scene input is neutral there and its LUT turns dark greys red. The LUT is computed every frame from the area's grading parameters (pass with `172ec899e321a2a5`, no textures), which in S'warkii hold a red-weighted gain (1.5, 1.08, 1.0), an offset (1, 0, 0) with −0.116 and a white balance of 8000 K (6500 K on the title), so the strong vignette (intensity 1.17) darkens the edges into the red shadows. The grid there is the out-of-focus foreground foliage's pixel-art blocks. -- Two compute shaders of the global shader cache do not translate (`327fce53f1fce4c1`, `6d87d48e86bd7f3c`: image atomics whose result is used, which the translator refuses on MSL 2.3); their dispatch is skipped with a red line. Nothing visibly missing so far. +- Two compute shaders of the global shader cache did not translate (`327fce53f1fce4c1`, `6d87d48e86bd7f3c`: image atomics whose result is used, refused on MSL 2.3), and their dispatch was skipped with a red line. Both build on chain `0ce4b5e8aefb` (a build of those two keys, 2026-10-27; [Image atomics](../../../docs/reference/shaders.md#image-atomics)); not run since. - 5,434 shaders (4,517 pixel, 672 vertex, 232 compute, 13 geometry), all but those two built for the Mac; all 5,434 built for iOS. ## Files and shaders diff --git a/dm/games/0100606026F12000/README.md b/dm/games/0100606026F12000/README.md index 0338a2f..3067975 100644 --- a/dm/games/0100606026F12000/README.md +++ b/dm/games/0100606026F12000/README.md @@ -38,7 +38,7 @@ Status: brought up on the Mac: the first-boot brightness screen, the title, a ne - **Current: 2,077 particle vertex shaders** (Unreal's particle vertex factories) pick one of several c2 table slots at run time: texture table words 0x168-0x1c0 and sampler table words 0x568-0x590, and no other word of c2, so `tex_dyn_slots` is right here. Without the key the first one aborted the host in a double free (now fixed). - **Resolved: texture reads removed by the translator** (GitHub issue #168). 58,122 reads, 57,861 of them texture-buffer reads in Unreal's Niagara particle vertex factories, reached their handle through a phi that also carried non-handle values, and bindless elimination replaced them with 0: particle scale 0, so fire, smoke and muzzle-flash meshes collapsed to a point. 58,096 of them in 3,197 shaders are now resolved; 26 in 14 shaders, whose handle comes from no c2 word, are still removed. The SPIR-V of the 69,449 other shaders is unchanged. See [Bindless reads whose handle phi also carries other values](../../../docs/reference/shaders.md#bindless-reads-whose-handle-phi-also-carries-other-values). - **Current: particle data in buffer textures at offsets 4, 8 and 12 past a 16-byte boundary** (pool offset `0x2fd208`, for one), which an Apple GPU reads from the boundary below. They are read through an aligned copy ([Buffer textures at offsets below 16 bytes](../../../docs/reference/rendering.md#buffer-textures-at-offsets-below-16-bytes)). -- **Unbuilt:** 11 shaders, none met so far: 3 geometry shaders and 8 compute shaders with 3D image atomics. +- **Unbuilt:** 11 shaders, none met so far: 3 geometry shaders and 8 compute shaders with 3D image atomics. Image atomics are native texture atomics since chain `0ce4b5e8aefb`, which build 3D ones on Octopath Traveler 0 and Ghost Trick ([Image atomics](../../../docs/reference/shaders.md#image-atomics)); this game's library has not been rebuilt on it yet. ## Files diff --git a/dm/games/010069C01AB82000/README.md b/dm/games/010069C01AB82000/README.md index 17de717..f454fcf 100644 --- a/dm/games/010069C01AB82000/README.md +++ b/dm/games/010069C01AB82000/README.md @@ -38,7 +38,7 @@ Status (2026-10-26): opening gameplay tested on the Mac with the capped memory l ## Rendering -- **Resolved:** two global compute shaders (`2d2093c1f1d3a7e0`, `4fe56176fa5a5dc6`) dispatched at boot failed translation. They use `IGN` buffer-image atomics and consume the returned old value, which the Metal path previously rejected. The general translator now preserves that value, with a zero result and no memory access outside the image: [Bounded MSL 2.3 atomic emulation](../../../docs/reference/shaders.md#bounded-msl-23-atomic-emulation). Synthetic SPIR-V, Mac/iOS Metal and Mac GPU result/sentinel checks pass. All 19,746 shaders build for both Mac and iOS with chain `86c8282f6fb4`; one folded geometry stage needs no separate Metal library. +- **Resolved:** two global compute shaders (`2d2093c1f1d3a7e0`, `4fe56176fa5a5dc6`) dispatched at boot failed translation. They use `IGN` buffer-image atomics and consume the returned old value, which the Metal path previously rejected. The general translator now preserves that value, with a zero result and no memory access outside the image: [Bounds of image atomics](../../../docs/reference/shaders.md#bounds-of-image-atomics). Synthetic SPIR-V, Mac/iOS Metal and Mac GPU result/sentinel checks pass. All 19,746 shaders build for both Mac and iOS with chain `86c8282f6fb4`; one folded geometry stage needs no separate Metal library. - **Resolved:** the first dispatch then failed image admission because reflection classified `texture_buffer` as 2D. Preserving its actual Metal type binds the real four-byte counter storage: [Buffer image reflection keeps its texture type](../../../docs/reference/shaders.md#buffer-image-reflection-keeps-its-texture-type). The corrected Mac runs have no `[RED]` diagnostics through the tested opening. ## Mac runs @@ -53,7 +53,7 @@ The NSP script's build directory is `/010069C01AB82000_build`; analy ## Found on this game - **Provisional Unreal profiles leave room for the initial heap** (`profile_guess.py`): [Provisional Unreal layout leaves no heap](../../../docs/reference/memory.md#provisional-unreal-layout-leaves-no-heap). -- **Returning `IGN` image atomics** (the offline translator, `t_atomic_return.m`): [Bounded MSL 2.3 atomic emulation](../../../docs/reference/shaders.md#bounded-msl-23-atomic-emulation). +- **Returning `IGN` image atomics** (the offline translator, `t_atomic_return.m`): [Bounds of image atomics](../../../docs/reference/shaders.md#bounds-of-image-atomics). - **Buffer-image reflection preserves its Metal texture type** (`nvn_refl.m`, `t_refl.m`): [Buffer image reflection keeps its texture type](../../../docs/reference/shaders.md#buffer-image-reflection-keeps-its-texture-type). ## Open diff --git a/dm/games/0100F43008C44000/README.md b/dm/games/0100F43008C44000/README.md index 2e8ee54..9451990 100644 --- a/dm/games/0100F43008C44000/README.md +++ b/dm/games/0100F43008C44000/README.md @@ -54,7 +54,8 @@ Each item was found from a red line or a pass dump on the way through the langua - **Resolved: characters' legs folded back from the hips once they walked.** From the first step on, the player's legs stuck straight out behind her at hip height, and the rival's forward in a later cutscene; standing still looked right. The bone palette the game computes on the CPU was already bent. The cause was six occlusion queries a frame, one 36-vertex box each (`a793f436dcabd350`, base instance 61, 62, …), drawn against mip level 2 of the depth pyramid: every query came back with 0 samples, and the game took its characters for hidden and posed their legs that way. Forcing every result to visible fixed the legs, and depth testing with ALWAYS did not. The box shader picks its transform with (instance index − base instance) + `c0[0x644]`, where the NVN driver keeps the base instance; the host left `0x644` at 0, so every box was box 0. 793 of the game's vertex shaders read that word. How it was found: the raycasts were logged with scratch observation gates (all correct), then `DM_HLECOUNT` showed nothing walking-specific, then the counter reports. - **Resolved: a min-reduction sampler averaged.** The game builds that depth pyramid with `SamplerBuilderSetReductionFilter 1` (`151607c7a92386cc` samples the level above at an explicit LOD), which the host used to average. - **Resolved: image atomics past their buffer.** From about frame 5300 of the route (the scene after the story opening's first cutscene), the fragment shader `2bd5037003980716` does an atomic max into a 480x270 r32ui 2D array; sent to the 64 KB dummy buffer, the atomics ran about 1.4 MB past it. They wrote into zero-copy pools over the game's heap (pointers in the game's objects and the host's program objects got their upper 4 bytes replaced, and the game or the replay crashed) and into unmapped addresses (`kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers and the picture freezes). An iPhone 17 freezing that way was reported in GitHub issue #101. Under `MTL_SHADER_VALIDATION=1` the scene played. Three runs of the whole route then reached frame 12,300 with no GPU error. -- **Current: invalid coordinates can exceed even an image-sized scratch buffer.** [PR #178](https://github.com/seedds/nxnative/pull/178) reports 1,836 invalid stores into a 3 MiB buffer on Z-A 1.0.0 on M1 Max, including offsets around 16 GiB. Its development build reduced those reports to zero, and the tester reported no freeze without shader validation. The shared [bounds and binding fix](../../../docs/reference/shaders.md#bounded-msl-23-atomic-emulation) is applied here; that development result does not establish gameplay or device validation of this revision or of update 2.0.2. The earlier scratch-size fix above addresses a separate insufficient-capacity case. +- **Resolved: invalid coordinates could exceed even an image-sized scratch buffer.** [PR #178](https://github.com/seedds/nxnative/pull/178) reports 1,836 invalid stores into a 3 MiB buffer on Z-A 1.0.0 on M1 Max, including offsets around 16 GiB. Its development build reduced those reports to zero, and the tester reported no freeze without shader validation. The shared [bounds fix](../../../docs/reference/shaders.md#bounds-of-image-atomics) is applied here; that development result does not establish gameplay or device validation of this revision or of update 2.0.2. The earlier scratch-size fix above addresses a separate insufficient-capacity case. Both scratch cases are gone since the atomics are native (next entry). +- **Current: the atomic max runs on texture 302 itself** (native texture atomics, [Image atomics](../../../docs/reference/shaders.md#image-atomics)), so its results reach the passes that read the image; before, they went to a scratch buffer that nothing read. Chain `0ce4b5e8aefb` builds all 28,603 shaders. Update 2.0.2 from a fresh save on the docs' route, headless on the Mac under Metal API validation (2026-10-27): `image atomics on image slot 1, texture ID 302 (target 4, block-linear, 2211840 bytes)`, then on to frame 12,000 with no validation error, GPU fault or new red line. The picture of that scene was not compared, and the iPhone freeze of GitHub issue #149 is not re-checked. - **Current validation (2026-10-25):** with the bounds fix and strict descriptor-format/bindless-handle checks, chain `467efdfd0b62` rebuilt all 28,603 shaders for Mac and iPhone with zero failures and no undecoded bindings. After rebuilding the host, update 2.0.2 ran for 240.1 seconds / 6,987 frames around the station from an isolated copy of the save, with movement and camera changes, shader validation disabled, and no observed freeze, GPU page fault, missing shader or atomic-binding rejection. The log confirmed the new chain and the 480x270, three-layer image using 3 MiB scratch. This is a Mac smoke test, not confirmation of the reported iPhone/version cases or the station-to-world transition. Artifacts are in the NSP folder's `0100F43008C44000-work/pr178/` (shader libraries and `gameplay/`). ## Mac routes @@ -71,8 +72,8 @@ General host changes first needed here; each applies to every game: - **Trinity packs decoded with the game's own Oodle** (`trpfs_jobs` / `trinity_shaders`, `hle_oodle.c`, `DM_OODLE_HOST` / `DM_OODLE_MODULES`, set by the NSP script for any game with `ONEPACK` files): [BNSH in Trinity packs](../../../docs/reference/shaders.md#bnsh-in-trinity-packs), [01, Compressed RomFS content](../../../docs/agents/01-extract.md#compressed-romfs-content). - **SDK 19 calls:** csrng bytes, `GetAudioOutBufferCount` from the host queue, NVN state getters, `WaitForAddress` type 3, `OP_FENCE_SYNC`: [Cryptographically random bytes](../../../docs/reference/services.md#cryptographically-random-bytes), [AudioOut buffer count from the host queue](../../../docs/reference/audio.md#audioout-buffer-count-from-the-host-queue), [State read back through NVN getters](../../../docs/reference/nvn-objects.md#state-read-back-through-nvn-getters), [WaitForAddress type 3](../../../docs/reference/threads.md#waitforaddress-type-3-64-bit-wait-if-equal), [Fence syncs recorded in command buffers](../../../docs/reference/nvn-objects.md#fence-syncs-recorded-in-command-buffers). - **Transform feedback** (`tf_layout`, `vs_tf` / `main0_tf`, `TF_SLOT`, `OP_TF_BIND` / `OP_TF_STATE` / `OP_RAST_DISCARD`, `tf_draw`, `pend_add` kind 4), the **feedback encoder ending `g_blit`**, **layered pipelines** (`vs_layer`, `layer_out`), **function constants specialized** (`shader_load`), the **zero cube for a 2D texture** (`tex_fit`), **layered passes** (`g_rp_layered`, `g_pass.layered`), **stencil-mode views** (`tex_by_id`, `X32_Stencil8`; every depth-stencil texture with pixel-format-view usage, `tex_desc`, PR #106, `t_stencil_view`), **the base instance in the driver segment** (`encode_args`, `c0[0x644]`; `nvn_refl.m` treats any declared driver-segment binding as read), **reduction samplers** (`samp_by_id`) and **pool creation retries** (`mtl_pool_add`): [Transform feedback captured by a vertex entry point](../../../docs/reference/rendering.md#transform-feedback-captured-by-a-vertex-entry-point) and the other entries of [rendering.md](../../../docs/reference/rendering.md) that name this game. -- **Image atomics into a scratch buffer as large as the image** (`atom_scratch`): [Image atomics](../../../docs/reference/shaders.md#image-atomics). -- **Bounds checks before image-atomic pointer construction and matched image/buffer admission:** [Bounded MSL 2.3 atomic emulation](../../../docs/reference/shaders.md#bounded-msl-23-atomic-emulation). +- **Image atomics into a scratch buffer as large as the image** (`atom_scratch`, since replaced by native texture atomics on the image): [Image atomics](../../../docs/reference/shaders.md#image-atomics). +- **Bounds checks before image-atomic pointer construction and matched image/buffer admission:** [Bounds of image atomics](../../../docs/reference/shaders.md#bounds-of-image-atomics). ## Open diff --git a/dm/gfx/nvn/dmsh_chain.h b/dm/gfx/nvn/dmsh_chain.h index d699345..aec1f0e 100644 --- a/dm/gfx/nvn/dmsh_chain.h +++ b/dm/gfx/nvn/dmsh_chain.h @@ -1,3 +1,3 @@ // Generated file (dm/gfx/tools/dmsh_chain.py --header): the B shader library translation chain version this source tree expects (default options), one for every game; // the host loads only /b-/. Run dmsh_chain.py --header after changing the chain's tools and commit this file with the change -#define DMSH_CHAIN "7776d7a9719d" +#define DMSH_CHAIN "a4e2088c2e69" diff --git a/dm/gfx/nvn/nvn_imga_layout.h b/dm/gfx/nvn/nvn_imga_layout.h deleted file mode 100644 index da1729e..0000000 --- a/dm/gfx/nvn/nvn_imga_layout.h +++ /dev/null @@ -1,33 +0,0 @@ -// Admission for the packed 32-bit IMG/IMGA contract, independent of Metal. -#pragma once -#include - -static inline int nvn_imga_bytes(uint64_t w, uint64_t h, uint64_t layers, uint64_t *out) { - if (!out || !w || !h || !layers || w > UINT32_MAX || h > UINT32_MAX || layers > UINT32_MAX || - w > UINT64_MAX / h || w * h > UINT64_MAX / layers || w * h * layers > UINT64_MAX / 4) return 0; - *out = w * h * layers * 4; - return 1; -} -static inline int nvn_imga_capacity(uint64_t need, uint64_t length, uint64_t offset) { - return need && offset <= length && need <= length - offset; -} -static inline int nvn_imga_scratch_length(uint64_t need, uint64_t *out) { - const uint64_t mask = (1u << 20) - 1; - if (!out || !need || need > UINT64_MAX - mask) return 0; - *out = (need + mask) & ~mask; - return 1; -} -typedef struct { - uint64_t width, height, layers, row_bytes; - uint64_t buffer_length, texture_offset, pointer_offset, guest_bytes; - unsigned word_bytes; - int buffer_texture, base_view, same_buffer; -} nvn_imga_direct_layout; -static inline int nvn_imga_direct_ok(const nvn_imga_direct_layout *d) { - uint64_t need; - if (!d || !d->base_view || !d->same_buffer || d->word_bytes != 4 || d->layers != 1 || - !nvn_imga_bytes(d->width, d->height, d->layers, &need) || d->texture_offset != d->pointer_offset || - d->texture_offset % 4 || !nvn_imga_capacity(need, d->buffer_length, d->texture_offset) || need > d->guest_bytes) return 0; - // Buffer textures have no next row: Metal may pad their unused row length. - return d->buffer_texture ? d->height == 1 : d->row_bytes == d->width * 4; -} diff --git a/dm/gfx/nvn/nvn_mtl.m b/dm/gfx/nvn/nvn_mtl.m index 212da34..552f4e4 100644 --- a/dm/gfx/nvn/nvn_mtl.m +++ b/dm/gfx/nvn/nvn_mtl.m @@ -63,7 +63,6 @@ static uint64_t c2_handle(int stage, unsigned b); // the handle c2 holds at by #include #include #include -#include "nvn_imga_layout.h" #include #include // Function parameters in this file are never retained again in the callee (the caller / global tables already hold them, and lifetime is guaranteed by defer_obj_release): saves a retain / release pair per call on the hot path @@ -1690,6 +1689,16 @@ static int pkg_edges_clean(NvPkg *pk, const tex_t *t) { // the covered pages w pk->stamp = st; g_pkg_edge_skips++; return 1; } +// Usage an NVN image takes: shader writes, and for R32Uint / R32Sint the shader-atomic usage too, since Metal runs texture atomics only on such textures (dummy_img_atomic) and NVN does +// not say which images a shader will use atomically +static inline int pf_r32i(MTLPixelFormat pf) { return pf == MTLPixelFormatR32Uint || pf == MTLPixelFormatR32Sint; } +static MTLTextureUsage img_usage(const tex_t *t, MTLPixelFormat pf) { + if (!(t->d.flags & NVN_TEXTURE_FLAGS_IMAGE)) return 0; + return MTLTextureUsageShaderWrite | (pf_r32i(pf) ? MTLTextureUsageShaderAtomic : 0); +} +// Metal leaves MTLTextureUsageShaderAtomic out of a texture's usage property (0x3 for a texture made with 0x23), so it cannot be read back: every shader-writable R32Uint / R32Sint +// texture the host makes has it (img_usage, tex_upgrade, dummy_img_atomic), and that is what this tests +static int tex_atomic_capable(id base) { return pf_r32i(base.pixelFormat) && (base.usage & MTLTextureUsageShaderWrite); } static MTLTextureDescriptor *tex_desc(const tex_t *t, int packaged) { uint32_t pf = 0; uint8_t swz[4]; if (!nvn_fmt_to_mtl(t->d.format, &pf, swz) || !pf) { RED_ONCE(0, "texture format %#x has no Metal equivalent (first case)", t->d.format); return nil; } @@ -1740,7 +1749,7 @@ static MTLTextureDescriptor *tex_desc(const tex_t *t, int packaged) { } d.storageMode = MTLStorageModePrivate; // Render targets and assets both go in GPU private storage (lossless compression available) d.usage = MTLTextureUsageShaderRead | (fmt_compressed(t->d.format) || packaged ? 0 : MTLTextureUsageRenderTarget) - | (t->d.flags & NVN_TEXTURE_FLAGS_IMAGE ? MTLTextureUsageShaderWrite : 0); // Image usage: shader-writable (compute kernels generating the 3D color lookup table, etc. + | img_usage(t, d.pixelFormat); // Image usage: shader-writable (compute kernels generating the 3D color lookup table, etc. // A depth-stencil texture has the format-view usage from the start: its stencil is read through an X32_Stencil8 view (stencil-mode views and textures, the depth-stencil copy), which Metal's // contract allows only with that usage. Apple GPUs read the view correctly without it, but the API validation layer aborts the process at the first such view (PR #106, found on Luigi's // Mansion 3), and upgrading later would recreate a depth buffer in use. Unlike colour formats it costs nothing measured: on an M4 Pro a 2560x1440 one has the same allocated size and the same @@ -1756,7 +1765,7 @@ static id tex_upgrade(const tex_t *t, id base, MTLTextur d.textureType = base.textureType; d.pixelFormat = base.pixelFormat; d.width = base.width; d.height = base.height; d.depth = base.depth; // Keep the original format, only add the format-view usage d.arrayLength = base.arrayLength; d.mipmapLevelCount = base.mipmapLevelCount; d.sampleCount = base.sampleCount; d.storageMode = base.storageMode; d.cpuCacheMode = base.cpuCacheMode; d.hazardTrackingMode = base.hazardTrackingMode; - d.resourceOptions = base.resourceOptions; d.usage = base.usage | add; + d.resourceOptions = base.resourceOptions; d.usage = base.usage | add | (tex_atomic_capable(base) ? MTLTextureUsageShaderAtomic : 0); // the usage property leaves the atomic usage out // A linear texture aliases the guest pool. Adding usage must keep that backing: a new standalone texture copies the old pixels once, but later GPU writes never reach the CPU map. id backing = base.buffer; id upgraded = backing ? [backing newTextureWithDescriptor:d offset:base.bufferOffset bytesPerRow:base.bufferBytesPerRow] : [g_dev newTextureWithDescriptor:d]; @@ -2246,7 +2255,7 @@ static id tex_linear(const tex_t *t) { if (mo % al || row % al) { RED_ONCE(65, "linear texture offset %#llx / row pitch %zu not aligned to %lu bytes; created as a private texture (CPU writes invisible)", (unsigned long long)mo, row, (unsigned long)al); return nil; } MTLTextureDescriptor *d = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:(MTLPixelFormat)pf width:t->d.width ? t->d.width : 1 height:t->d.height ? t->d.height : 1 mipmapped:NO]; d.resourceOptions = b.resourceOptions; - d.usage = MTLTextureUsageShaderRead | (t->d.flags & NVN_TEXTURE_FLAGS_IMAGE ? MTLTextureUsageShaderWrite : 0) | (t->d.flags & NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET ? MTLTextureUsageRenderTarget : 0); + d.usage = MTLTextureUsageShaderRead | img_usage(t, (MTLPixelFormat)pf) | (t->d.flags & NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET ? MTLTextureUsageRenderTarget : 0); return [b newTextureWithDescriptor:d offset:mo bytesPerRow:row]; } // CPU-written block-linear textures (WriteTexels): the game gave no packaged layout and changes the content from the CPU each frame (Engage's movie frames: a Y plane R8 + a UV plane RG8, WriteTexels every frame). @@ -2377,7 +2386,7 @@ static id tex_mtl_(const tex_t *ct) { static int once; if (!once++) MLOG("buffer texture at pool offset %#llx, below Metal's %lu-byte alignment: created there, read through an aligned copy (tbuf_shadow)", (unsigned long long)mo, (unsigned long)al); } MTLTextureDescriptor *d = [MTLTextureDescriptor textureBufferDescriptorWithPixelFormat:(MTLPixelFormat)pf width:t->d.width resourceOptions:b.resourceOptions - usage:MTLTextureUsageShaderRead | (t->d.flags & NVN_TEXTURE_FLAGS_IMAGE ? MTLTextureUsageShaderWrite : 0)]; + usage:MTLTextureUsageShaderRead | img_usage(t, (MTLPixelFormat)pf)]; size_t row = ((size_t)t->d.width * bpb + al - 1) / al * al; id mt = [b newTextureWithDescriptor:d offset:mo bytesPerRow:row]; if (!mt) { RED_ONCE(39, "cannot create buffer texture: %u elements format %#x", t->d.width, t->d.format); return nil; } @@ -4181,7 +4190,7 @@ static void bare_dims(NSUInteger *w, NSUInteger *h) { static void swrite_note(NvProg *pr) { if (!pr || g_pass_swrite) return; for (int st = 0; st < 6; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh) continue; - for (int i = 0; i < sh->rf.nb; i++) if (sh->rf.b[i].rw && (sh->rf.b[i].kind == NVN_RB_IMG || sh->rf.b[i].kind == NVN_RB_IMGA || sh->rf.b[i].kind == NVN_RB_SBUFP)) { g_pass_swrite = 1; return; } } + for (int i = 0; i < sh->rf.nb; i++) if (sh->rf.b[i].rw && (sh->rf.b[i].kind == NVN_RB_IMG || sh->rf.b[i].kind == NVN_RB_SBUFP)) { g_pass_swrite = 1; return; } } } static void udc_note_draw(NvProg *pr) { if (!tbdr_on()) { // without the TBDR set only the duplicate-slot report below is live (the zero-source skip and the unwritten-attachment store are part of the set): skip the per-draw scans @@ -5847,28 +5856,7 @@ static id dummy_samp(void) { static uint64_t dummy_samp_rid(void) { static uint64_t rid; if (!rid) rid = dummy_samp().gpuResourceID._impl; return rid; } static id dummy_buf(void) { static id b; if (!b) { b = [g_dev newBufferWithLength:64 << 10 options:MTLResourceStorageModeShared]; res_add(b); } return b; } static uint64_t dummy_buf_gpu(void) { static uint64_t gpu; if (!gpu) gpu = dummy_buf().gpuAddress; return gpu; } -// Image atomics on block-linear images retain discard scratch at least as large as the matched IMG extent; linear images require proven direct backing below. SPIRV-Cross indexes -// the IMGA pointer using that image's dimensions (including the array layer), so a full image paired with a 64 KB dummy can run past its end into whatever GPU memory followed: -// Pokémon Legends: Z-A's atomic max into an r32ui 2D array (texture ID 302, fragment shader 2bd5037003980716) wrote into zero-copy pools over the game's heap, which broke pointers in the game's -// objects and in the host's program objects, and into unmapped addresses, a GPU page fault (GitHub issue #101). The results are discarded as before; the buffer only grows -static __strong id g_atom_scratch; -static id atom_scratch(const tex_t *t, uint64_t image_bytes) { - int bw, bh, bpb = nvn_fmt_bpb(t->d.format, &bw, &bh); - uint64_t layers = t->d.target == 8 ? 6 : t->d.depth > 1 ? t->d.depth : 1; - uint64_t need, len; - if (bpb != 4 || bw != 1 || bh != 1 || - !nvn_imga_bytes(MAX(t->d.width, 1u), MAX(t->d.height, 1u), layers, &need)) return nil; - if (need < t->L.size) need = t->L.size; - if (need < image_bytes) return nil; // Never invent a larger image/storage contract. - if (g_atom_scratch.length >= need) return g_atom_scratch; - if (!nvn_imga_scratch_length(need, &len)) return nil; - id b = [g_dev newBufferWithLength:len options:MTLResourceStorageModePrivate]; - if (!b) return nil; // Caller rejects the binding: a full image must never keep a short dummy pointer. - res_add(b); if (g_atom_scratch) defer_obj_release(g_atom_scratch); g_atom_scratch = b; - MLOG("image atomics that are not followed: scratch buffer of %llu MB (texture %ux%u, %llu layers)", (unsigned long long)(len >> 20), t->d.width, t->d.height, (unsigned long long)layers); - return b; -} -// Resolve IMG once for both image and atomic pointer. Caller holds g_rmu. +// The texture of an image slot (a view of the shader's class when the texture's differs, img_fit), else the write sink. Caller holds g_rmu static id img_binding(int tid, const nvn_rbind *b, uint64_t *rid, int *real) { void *tv = NULL; *rid = tid ? tex_fast_rid(tid, &tv) : 0; id t = (__bridge id)tv; @@ -5882,11 +5870,29 @@ static id img_binding(int tid, const nvn_rbind *b, uint64_t *rid, in if (!t) { t = dummy_img(b->dtt, b->dpf); *rid = g_dummy_img_rid[b->dtt & 7][b->dpf & 3]; } return t; } -static int imga_extent(id t, const nvn_rbind *b, uint64_t *bytes) { - if (!t || t.textureType != (MTLTextureType)b->dtt || t.sampleCount > 1) return 0; - if (t.textureType != MTLTextureTypeTextureBuffer && t.textureType != MTLTextureType2D && t.textureType != MTLTextureType2DArray) return 0; - return nvn_imga_bytes(t.width, t.textureType == MTLTextureTypeTextureBuffer ? 1 : t.height, - t.textureType == MTLTextureType2DArray ? t.arrayLength : 1, bytes); +// Image atomics run natively on the image's own texture (dmsh_atomic.py), so they and every other access of the image see the same texels: a block-linear image's Metal texture as +// much as a linear one's pool memory. They used to go through a raw pointer beside the texture, which for a block-linear image (a deswizzled copy) could only be a scratch buffer +// nothing read, and a 3D image's did not compile. Metal allows texture atomics only on an R32Uint / R32Sint texture made with the shader-atomic usage (img_usage); any other +// texture in an atomic slot is replaced by an atomic-capable write sink, logged red, rather than left to the GPU's undefined behaviour +static id dummy_img_atomic(int dtt, int dpf) { + static __strong id cache[8][2]; + int s = dpf == NVN_DPF_SINT; id t = cache[dtt & 7][s]; if (t) return t; + MTLPixelFormat pf = s ? MTLPixelFormatR32Sint : MTLPixelFormatR32Uint; + MTLTextureUsage u = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite | MTLTextureUsageShaderAtomic; + if ((MTLTextureType)dtt == MTLTextureTypeTextureBuffer) { + MTLTextureDescriptor *d = [MTLTextureDescriptor textureBufferDescriptorWithPixelFormat:pf width:16 resourceOptions:MTLResourceStorageModePrivate usage:u]; + id b = [g_dev newBufferWithLength:256 options:MTLResourceStorageModePrivate]; res_add(b); + t = [b newTextureWithDescriptor:d offset:0 bytesPerRow:256]; + } else { + MTLTextureDescriptor *d = [MTLTextureDescriptor new]; + d.textureType = (MTLTextureType)dtt; d.pixelFormat = pf; d.width = d.height = d.depth = 1; d.storageMode = MTLStorageModePrivate; d.usage = u; + t = [g_dev newTextureWithDescriptor:d]; + } + cache[dtt & 7][s] = t; res_add(t); return t; +} +static int img_atomic_ok(id t, const nvn_rbind *b) { + id base = t.parentTexture ? t.parentTexture : t; + return pf_r32i(t.pixelFormat) && tex_atomic_capable(base) && t.textureType == (MTLTextureType)b->dtt && t.sampleCount == 1; } // Snapshot substitution for same-pass feedback sampling (registered by fb_prepare, used by encode_args; see the notes before draw) static uint64_t g_draw_seq, g_fb_seq, g_fb_count; static int g_fb_n, g_fb_rebind; @@ -5960,21 +5966,6 @@ static int encode_args_(id en, NvShader *sh, int nvn_stage, i int bl = passdump() && (bl_all || g_kd_force || bl_pass[nvn_stage] != g_stat_pass); if (bl) { bl_pass[nvn_stage] = g_stat_pass; PD("K %llu st%d key=%s nb=%d", g_stat_pass, nvn_stage, sh->key[0] ? sh->key : sh->builtin ? sh->builtin : "?", sh->rf.nb); } os_unfair_lock_lock(&g_rmu); // The texture / sampler registration tables may be being grown or rewritten by the game thread - __unsafe_unretained id atomic_img[32] = {nil}; - uint64_t atomic_rid[32] = {0}; int atomic_real[32] = {0}; - const nvn_rbind *atomic_desc[32] = {0}; - for (int i = 0; i < sh->rf.nb; i++) if (sh->rf.b[i].kind == NVN_RB_IMGA) { - unsigned r = sh->rf.b[i].reg; const nvn_rbind *ib = NULL; - for (int j = 0; j < sh->rf.nb; j++) if (sh->rf.b[j].kind == NVN_RB_IMG && sh->rf.b[j].reg == r) { - if (ib) MDIE("[RED] image atomics: ambiguous IMG partner (shader %s stage %d slot %u)", sh->key, nvn_stage, r); - ib = &sh->rf.b[j]; - } - if (r >= 32 || !ib) MDIE("[RED] image atomics: missing IMG partner (shader %s stage %d slot %u)", sh->key, nvn_stage, r); - if (!atomic_desc[r]) { - int tid = (int)(S.img[nvn_stage][r] & 0xfffff); atomic_desc[r] = ib; - atomic_img[r] = img_binding(tid, ib, &atomic_rid[r], &atomic_real[r]); - } - } for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; int reg = b->reg, aid = b->aid; @@ -6052,50 +6043,18 @@ static int encode_args_(id en, NvShader *sh, int nvn_stage, i } else if (b->kind == NVN_RB_IMG) { // Writable texture: NVN image slot r (BindImages) ↔ D3D u_r int tid = reg < 32 ? (int)(S.img[nvn_stage][reg] & 0xfffff) : 0; uint64_t trid; int real; - id t = reg < 32 && atomic_desc[reg] ? atomic_img[reg] : img_binding(tid, b, &trid, &real); - if (reg < 32 && atomic_desc[reg]) { trid = atomic_rid[reg]; real = atomic_real[reg]; } + id t = img_binding(tid, b, &trid, &real); + if (b->atom && !img_atomic_ok(t, b)) { // see dummy_img_atomic; an unbound slot's write sink is not atomic-capable either + if (real) { id base = t.parentTexture ? t.parentTexture : t; + RED_ONCE(93, "image atomics on a texture Metal cannot run them on (shader %s stage %d slot %d texture ID %d: Metal format %lu of base format %lu, type %lu for %u, %s): they go to a write sink", + sh->key, nvn_stage, reg, tid, (unsigned long)t.pixelFormat, (unsigned long)base.pixelFormat, (unsigned long)t.textureType, b->dtt, base.usage & MTLTextureUsageShaderWrite ? "shader-writable" : "not shader-writable"); } + t = dummy_img_atomic(b->dtt, b->dpf); trid = t.gpuResourceID._impl; real = 0; + } else if (b->atom && !(sh->logged & 2)) { sh->logged |= 2; static int nlog; const tex_t *tt = tid < g_ntreg && g_treg[tid].ok ? &g_treg[tid].t : NULL; + if (tt && nlog++ < 8) MLOG("shader %s (stage %d): image atomics on image slot %d, texture ID %d (target %d, %s, %llu bytes)", sh->key, nvn_stage, reg, tid, (int)tt->d.target, + tt->d.target == NVN_TEXTURE_TARGET_BUFFER || (tt->d.flags & (NVN_TEXTURE_FLAGS_LINEAR | NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET)) ? "linear" : "block-linear", (unsigned long long)tt->L.size); } if (passdump()) PD("W %llu st%d reg%d tid=%d %s ser=%llu", g_stat_pass, nvn_stage, reg, tid, real ? "real" : "dummy", real && tid < g_ntreg ? g_treg[tid].t.serial : 0ull); slots[aid] = trid; USE(t, MTLResourceUsageRead | MTLResourceUsageWrite); - } else if (b->kind == NVN_RB_IMGA) { - int tid = (int)(S.img[nvn_stage][reg] & 0xfffff); - const tex_t *tt = tid && tid < g_ntreg && g_treg[tid].ok ? &g_treg[tid].t : NULL; - id image = atomic_img[reg]; uint64_t need; - if (!imga_extent(image, atomic_desc[reg], &need)) - MDIE("[RED] image atomics: unsupported or overflowing IMG extent (shader %s stage %d slot %d texture %d)", sh->key, nvn_stage, reg, tid); - if (tt && (!atomic_real[reg] || (image.pixelFormat != MTLPixelFormatR32Uint && image.pixelFormat != MTLPixelFormatR32Sint))) - MDIE("[RED] image atomics: real image is not a resolved R32 integer IMG (shader %s stage %d slot %d texture %d)", sh->key, nvn_stage, reg, tid); - int linear = tt && (tt->d.target == NVN_TEXTURE_TARGET_BUFFER || (tt->d.flags & (NVN_TEXTURE_FLAGS_LINEAR | NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET))); - id storage = nil; uint64_t gpu = 0; - if (linear) { - void *mb = NULL; uint64_t mo = 0; - int found = tt->mem && mtl_gpu_addr((uint64_t)tt->mem, need, &mb, &mo, &gpu); - storage = (__bridge id)mb; - int bw, bh, bpb = nvn_fmt_bpb(tt->d.format, &bw, &bh); - // Only actual base buffer textures / packed base 2D images have a proven pointer layout. - nvn_imga_direct_layout d = { .width = image.width, .height = image.height, .layers = image.arrayLength, - .row_bytes = image.buffer ? image.bufferBytesPerRow : 0, .buffer_length = storage.length, - .texture_offset = image.buffer ? image.bufferOffset : 0, .pointer_offset = mo, .guest_bytes = tt->L.size, - .word_bytes = bw == 1 && bh == 1 ? (unsigned)bpb : 0, .buffer_texture = image.textureType == MTLTextureTypeTextureBuffer, - .base_view = !image.parentTexture && image.mipmapLevelCount == 1 && tt->d.levels <= 1 && tt->d.samples <= 1 && - image.width == tt->d.width && (image.textureType == MTLTextureTypeTextureBuffer ? tt->d.target == NVN_TEXTURE_TARGET_BUFFER : - image.textureType == MTLTextureType2D && tt->d.target == NVN_TEXTURE_TARGET_2D && image.height == tt->d.height), - .same_buffer = found && image.buffer == storage }; - if (!gpu || !nvn_imga_direct_ok(&d)) - MDIE("[RED] image atomics: unsupported direct IMG/IMGA layout or capacity (shader %s stage %d slot %d texture %d)", sh->key, nvn_stage, reg, tid); - static int nlog; if (nlog < 8 && !(sh->logged & 2)) { sh->logged |= 2; nlog++; MLOG("shader %s (stage %d): image atomics on image slot %d, texture ID %d (target %d, %llu bytes)", sh->key, nvn_stage, reg, tid, (int)tt->d.target, (unsigned long long)tt->L.size); } - } else if (tt) { - RED_ONCE(93, "image atomics on a block-linear texture (texture ID %d target %d): not followed, the atomics go to a scratch buffer", tid, (int)tt->d.target); - storage = atom_scratch(tt, need); gpu = storage.gpuAddress; - if (!storage || !gpu || !nvn_imga_capacity(need, storage.length, 0)) - MDIE("[RED] image atomics: scratch allocation or capacity failed; refusing short dummy (shader %s stage %d slot %d texture %d need %llu)", sh->key, nvn_stage, reg, tid, (unsigned long long)need); - } else { - storage = dummy_buf(); gpu = storage.gpuAddress; - if (atomic_real[reg] || !storage || !gpu || !nvn_imga_capacity(need, storage.length, 0)) - MDIE("[RED] image atomics: unmatched unbound IMG/IMGA pair (shader %s stage %d slot %d)", sh->key, nvn_stage, reg); - } - slots[aid] = gpu; USE(storage, MTLResourceUsageRead | MTLResourceUsageWrite); - if (bl) PD("B %llu st%d imga reg%d tid=%d %s", g_stat_pass, nvn_stage, reg, tid, linear ? "direct" : tt ? "scratch" : "dummy"); } else if (b->kind == NVN_RB_TBUF) { // Typed buffer: the buffer texture (target BUFFER) in texture slot r int tid = (int)(tex_slot_h(S.tex[nvn_stage], S.img[nvn_stage], reg) & 0xfffff); void *tv = NULL; uint64_t trid = tid ? tex_fast_rid(tid, &tv) : 0; __unsafe_unretained id t = (__bridge id)tv; @@ -6791,7 +6750,7 @@ static int draw_samples_rt(NvProg *pr) { // Sampled attachments keep the origi for (int st = 0; st < 5; st++) { __unsafe_unretained NvShader *sh = pr->sh[st]; if (!sh) continue; if (sh->rf.bindless || sh->rf.dyn) return 1; for (int i = 0; i < sh->rf.nb; i++) { const nvn_rbind *b = &sh->rf.b[i]; uint64_t h; - if (b->kind == NVN_RB_TEXU || b->kind == NVN_RB_TEXD || b->kind == NVN_RB_TEXA || b->kind == NVN_RB_IMGA) return 1; + if (b->kind == NVN_RB_TEXU || b->kind == NVN_RB_TEXD || b->kind == NVN_RB_TEXA) return 1; if (b->kind == NVN_RB_IMG) h = b->reg < 32 && sh->stage < 6 ? S.img[sh->stage][b->reg] : 0; else if (b->kind == NVN_RB_TEX || b->kind == NVN_RB_TBUF) h = sh->stage < 6 ? tex_slot_h(S.tex[sh->stage], S.img[sh->stage], b->reg) : 0; else continue; int tid = (int)(h & 0xfffff); if (!tid) continue; os_unfair_lock_lock(&g_rmu); tex_t t = tid < g_ntreg && g_treg[tid].ok ? g_treg[tid].t : (tex_t){0}; os_unfair_lock_unlock(&g_rmu); @@ -8363,7 +8322,7 @@ static int csh_draw_reads(NvProg *pr, const p_draw *dw, const p_draw_idx *di, cs if (!t.serial || !csh_add(r, n, (uint64_t)t.mem, t.L.size)) return 0; break; } case NVN_RB_TEXU: case NVN_RB_TEXD: case NVN_RB_TEXA: return 0; - case NVN_RB_SBUFP: case NVN_RB_SBUFU: case NVN_RB_IMG: case NVN_RB_IMGA: case NVN_RB_TBUF: return 0; // may write memory, or reads memory not listed here + case NVN_RB_SBUFP: case NVN_RB_SBUFU: case NVN_RB_IMG: case NVN_RB_TBUF: return 0; // may write memory, or reads memory not listed here default: break; } } } return 1; } @@ -8374,7 +8333,7 @@ static int csh_cs(NvShader *cs, const p_dispatch *dp, csh_rng *w, int *nw, int * switch (b->kind) { case NVN_RB_SBUFP: if (b->reg >= 16 || csh_attachment_hit(S.ssbo[5][b->reg].addr, S.ssbo[5][b->reg].size) || !csh_add(w, nw, S.ssbo[5][b->reg].addr, S.ssbo[5][b->reg].size)) return 0; break; case NVN_RB_CBUF: case NVN_RB_UBOP: if (b->reg >= 16 || csh_attachment_hit(S.ubo[5][b->reg].addr, S.ubo[5][b->reg].size)) return 0; break; - case NVN_RB_SBUFU: case NVN_RB_IMG: case NVN_RB_IMGA: case NVN_RB_TEXU: case NVN_RB_TEXD: case NVN_RB_TEXA: return 0; + case NVN_RB_SBUFU: case NVN_RB_IMG: case NVN_RB_TEXU: case NVN_RB_TEXD: case NVN_RB_TEXA: return 0; case NVN_RB_TEX: case NVN_RB_TBUF: { uint64_t h = tex_slot_h(S.tex[5], S.img[5], b->reg); int tid = (int)(h & 0xfffff); if (!tid) break; os_unfair_lock_lock(&g_rmu); tex_t t = tid < g_ntreg && g_treg[tid].ok ? g_treg[tid].t : (tex_t){0}; os_unfair_lock_unlock(&g_rmu); if (!t.serial) return 0; diff --git a/dm/gfx/nvn/nvn_refl.h b/dm/gfx/nvn/nvn_refl.h index a38de40..6a58a3b 100644 --- a/dm/gfx/nvn/nvn_refl.h +++ b/dm/gfx/nvn/nvn_refl.h @@ -7,10 +7,9 @@ #include enum { NVN_RB_CBUF = 1, NVN_RB_SAMP, NVN_RB_TEX, NVN_RB_IMG, NVN_RB_TBUF, - NVN_RB_SBUFP, NVN_RB_SUPPORT, NVN_RB_C1, NVN_RB_DRV0, NVN_RB_DRV2, NVN_RB_SAMPT, NVN_RB_UBOP, NVN_RB_TEXU, NVN_RB_SAMPU, NVN_RB_SAMPI, NVN_RB_SBUFU, NVN_RB_TEXD, NVN_RB_SAMPD, NVN_RB_IMGA, NVN_RB_TEXA, NVN_RB_SAMPA, NVN_RB_GEOMETRY }; + NVN_RB_SBUFP, NVN_RB_SUPPORT, NVN_RB_C1, NVN_RB_DRV0, NVN_RB_DRV2, NVN_RB_SAMPT, NVN_RB_UBOP, NVN_RB_TEXU, NVN_RB_SAMPU, NVN_RB_SAMPI, NVN_RB_SBUFU, NVN_RB_TEXD, NVN_RB_SAMPD, NVN_RB_TEXA, NVN_RB_SAMPA, NVN_RB_GEOMETRY }; // TEXA / SAMPA: an array of n handles in the driver table c2 that the shader indexes at run time (dmsh_build.py c2_array): element k = the texture / sampler of the slot whose c2 word lies // at byte off + 8k (combined, else separate); words outside both tables give the dummy. Luigi's Mansion 3's shadow cascades -// IMGA: SPIRV-Cross's MSL 2.3 image atomics: a device atomic_uint / atomic_int pointer (_atomic) beside the image of NVN image slot r; the replay binds the image's storage address // TEXD / SAMPD: Ryujinx's fallback for a texture handle it cannot trace to one slot (handle type 4, __cb0_40000000): an array of n textures / samplers indexed by // the handle's texture / sampler ID (capped at n - 1); the replay binds slot k at element k and puts slot indices in c2 (profile key tex_dyn_slots, Octopath Traveler 0) // SAMPI: the sampler ID is a constant in the shader (reg = the ID; Ryujinx's SeparateConstantSamplerHandle: the handle ORs a constant sampler into a bound texture, Monster Hunter Rise) @@ -31,6 +30,7 @@ typedef struct { uint16_t off; // TEXU / SAMPU: byte offset of the handle within the uniform buffer; SBUFU: of the 64-bit storage buffer address uint8_t n; // TEXD / SAMPD, and TEXU / SAMPU arrays (element k = the handle at byte 8k of the uniform buffer): array length (argument buffer entries aid .. aid + n - 1) uint8_t f1d; // TEX: read by 1D integer fetches as 2D rows of 16384 texels (dmsh_build.py "fetch1d"): a buffer texture here is bound as such rows (nvn_mtl.m tbuf_as_2d) + uint8_t atom; // IMG: the shader runs image atomics on it (dmsh_build.py "atomic"): the texture must allow Metal's texture atomics (nvn_mtl.m dummy_img_atomic) } nvn_rbind; typedef struct { int nb, abi, nv; // Binding count, argument buffer slot, vertex input count diff --git a/dm/gfx/nvn/nvn_refl.m b/dm/gfx/nvn/nvn_refl.m index 96407a1..c429758 100644 --- a/dm/gfx/nvn/nvn_refl.m +++ b/dm/gfx/nvn/nvn_refl.m @@ -55,11 +55,6 @@ int nvn_refl_from_json(NSDictionary *j, nvn_refl *o, char *err, int errlen) { else FAIL("B binding %s: dynamic handle array of %s", nm.UTF8String, el.UTF8String); r->n = (uint8_t)n; o->dyn = 1; o->nb++; continue; } - // SPIRV-Cross's MSL 2.3 image atomics: the image (texture_buffer / texture2d… or ) and, for the atomics, a device pointer to the same memory, _atomic, which the build - // reads as a second image of the same slot. Octopath Traveler 0: 2,084 shaders (Unreal's buffer images: cp_i_cb2_4C_r32ui_atomic) - if ([k isEqualToString:@"img"] && reg >= 0 && [nm isKindOfClass:[NSString class]] && [nm hasSuffix:@"_atomic"] && [ty isKindOfClass:[NSString class]] && [ty hasPrefix:@"device atomic_"]) { - r->kind = NVN_RB_IMGA; r->rw = 1; o->nb++; continue; - } if ([k isEqualToString:@"cbuf"]) r->kind = NVN_RB_CBUF; else if ([k isEqualToString:@"samp"]) r->kind = NVN_RB_SAMP; else if ([k isEqualToString:@"samp_tex"]) r->kind = NVN_RB_SAMPT; @@ -77,7 +72,7 @@ int nvn_refl_from_json(NSDictionary *j, nvn_refl *o, char *err, int errlen) { if (r->kind == NVN_RB_TEXA) dummy_kind(el, &r->dtt, &r->dpf); } else if ([k isEqualToString:@"img"]) { r->kind = NVN_RB_IMG; dummy_kind(b[@"msl_type"], &r->dtt, &r->dpf); // rw: the shader writes it (access::write / read_write; a load-only view of the slot is not) - r->rw = [ty isKindOfClass:[NSString class]] && ([ty containsString:@"access::write"] || [ty containsString:@"access::read_write"]); } + r->rw = [ty isKindOfClass:[NSString class]] && ([ty containsString:@"access::write"] || [ty containsString:@"access::read_write"]); r->atom = [b[@"atomic"] boolValue]; } else if ([k isEqualToString:@"tbuf"]) { r->kind = NVN_RB_TBUF; uint8_t tt; dummy_kind(b[@"msl_type"], &tt, &r->dpf); } else if ([k isEqualToString:@"sbuf"]) { r->kind = NVN_RB_SBUFP; r->rw = sbuf_rw(b); } else if ([k isEqualToString:@"ubo_ptr"]) r->kind = NVN_RB_UBOP; diff --git a/dm/gfx/tests/Makefile b/dm/gfx/tests/Makefile index 37c3a1b..bf6c941 100644 --- a/dm/gfx/tests/Makefile +++ b/dm/gfx/tests/Makefile @@ -9,17 +9,6 @@ endif GAME_DIR = ../../games/$(GAME) GAME_H = ../../host/gen/$(GAME)/game_profile.h -# Portable admission contract only; native Metal binding remains an Apple test. -.PHONY: imga-layout imga-layout-sanitize -imga-layout: t_imga_layout - ./t_imga_layout -t_imga_layout: t_imga_layout.c ../nvn/nvn_imga_layout.h - $(CC) -std=c11 -O2 -g -Wall -Wextra -Werror -o $@ $< -imga-layout-sanitize: t_imga_layout_sanitize - ./t_imga_layout_sanitize -t_imga_layout_sanitize: t_imga_layout.c ../nvn/nvn_imga_layout.h - $(CC) -std=c11 -O1 -g -Wall -Wextra -Werror -fsanitize=address,undefined -fno-omit-frame-pointer -o $@ $< - CFLAGS=-O1 -g -Wall -Wno-unused-function -I../nvn/include -I$(dir $(GAME_H)) GFX=$(filter-out ../nvn/hle_nvn.c,$(sort $(wildcard ../nvn/*.c ../nvn/*.m ../nvn/*.S ../tex/*.c ../tex/*.m))) GFX_H=$(wildcard ../nvn/*.h ../nvn/*.inc ../nvn/include/*.h ../tex/*.h) $(GAME_H) diff --git a/dm/gfx/tests/t_atomic_return.m b/dm/gfx/tests/t_atomic_return.m index 5350472..b27b478 100644 --- a/dm/gfx/tests/t_atomic_return.m +++ b/dm/gfx/tests/t_atomic_return.m @@ -1,11 +1,13 @@ -// Execute a generated IGN buffer-image atomic fixture. First max(value, 1), then -// add its returned old value to texel zero; invalid first coordinates add zero. +// Execute a generated IGN image atomic fixture on a native atomic texture (dmsh_atomic.py). First max(value, 1) at the fixture's +// coordinate, then add its returned old value to texel zero; invalid first coordinates add zero. Every other texel must stay 3. +// Usage: t_atomic_return ; the image is 8 wide, 4 high, 3 layers or slices deep #import #import #include +#include int main(int argc, const char **argv) { @autoreleasepool { - assert(argc == 3); + assert(argc == 4); id dev = MTLCreateSystemDefaultDevice(); assert(dev); NSError *error = nil; id lib = [dev newLibraryWithURL:[NSURL fileURLWithPath:@(argv[1])] error:&error]; @@ -13,29 +15,49 @@ int main(int argc, const char **argv) { @autoreleasepool { id fn = [lib newFunctionWithName:@"main0"]; assert(fn); id pso = [dev newComputePipelineStateWithFunction:fn error:&error]; assert(pso && !error); - id data = [dev newBufferWithLength:4096 options:MTLResourceStorageModeShared]; - uint32_t *words = data.contents; - for (int i = 0; i < 1024; i++) words[i] = 3; - MTLTextureDescriptor *desc = [MTLTextureDescriptor textureBufferDescriptorWithPixelFormat:MTLPixelFormatR32Uint - width:8 resourceOptions:data.resourceOptions usage:MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite]; - id tex = [data newTextureWithDescriptor:desc offset:0 bytesPerRow:32]; assert(tex); + const char *shape = argv[3]; + MTLTextureUsage usage = MTLTextureUsageShaderRead | MTLTextureUsageShaderWrite | MTLTextureUsageShaderAtomic; + enum { W = 8, H = 4, L = 3 }; + uint32_t init[W * H * L]; for (int i = 0; i < W * H * L; i++) init[i] = 3; + id data = nil; id tex; NSUInteger n; + if (!strcmp(shape, "buffer")) { // a view of a buffer, as the host makes for buffer images; texels outside it stay 3 too + data = [dev newBufferWithLength:4096 options:MTLResourceStorageModeShared]; + uint32_t *words = data.contents; for (int i = 0; i < 1024; i++) words[i] = 3; + MTLTextureDescriptor *desc = [MTLTextureDescriptor textureBufferDescriptorWithPixelFormat:MTLPixelFormatR32Uint width:W resourceOptions:data.resourceOptions usage:usage]; + tex = [data newTextureWithDescriptor:desc offset:0 bytesPerRow:32]; n = 1024; + } else { + MTLTextureDescriptor *desc = [MTLTextureDescriptor new]; + desc.textureType = !strcmp(shape, "2d") ? MTLTextureType2D : !strcmp(shape, "2darray") ? MTLTextureType2DArray : MTLTextureType3D; + assert(desc.textureType != MTLTextureType3D || !strcmp(shape, "3d")); + desc.pixelFormat = MTLPixelFormatR32Uint; desc.width = W; desc.height = H; + desc.depth = desc.textureType == MTLTextureType3D ? L : 1; desc.arrayLength = desc.textureType == MTLTextureType2DArray ? L : 1; + desc.storageMode = MTLStorageModeShared; desc.usage = usage; + tex = [dev newTextureWithDescriptor:desc]; assert(tex); + n = desc.textureType == MTLTextureType2D ? W * H : W * H * L; + for (NSUInteger s = 0; s < desc.arrayLength; s++) + [tex replaceRegion:MTLRegionMake3D(0, 0, 0, W, H, desc.depth) mipmapLevel:0 slice:s withBytes:init + s * W * H bytesPerRow:W * 4 bytesPerImage:W * H * 4]; + } + assert(tex); id args = [fn newArgumentEncoderWithBufferIndex:0]; id ab = [dev newBufferWithLength:args.encodedLength options:MTLResourceStorageModeShared]; [args setArgumentBuffer:ab offset:0]; [args setTexture:tex atIndex:0]; - [args setBuffer:data offset:0 atIndex:1]; id queue = [dev newCommandQueue]; id command = [queue commandBuffer]; id enc = [command computeCommandEncoder]; [enc setComputePipelineState:pso]; [enc setBuffer:ab offset:0 atIndex:0]; - [enc useResource:data usage:MTLResourceUsageRead | MTLResourceUsageWrite]; [enc useResource:tex usage:MTLResourceUsageRead | MTLResourceUsageWrite]; [enc dispatchThreads:MTLSizeMake(1, 1, 1) threadsPerThreadgroup:MTLSizeMake(1, 1, 1)]; [enc endEncoding]; [command commit]; [command waitUntilCompleted]; assert(command.status == MTLCommandBufferStatusCompleted); - assert(words[0] == (uint32_t)atoi(argv[2])); - for (int i = 1; i < 1024; i++) assert(words[i] == 3); - puts("returning IGN atomic: result and sentinel checks passed"); + uint32_t out[1024]; + if (data) memcpy(out, data.contents, sizeof out); + else if (tex.textureType == MTLTextureType2DArray) + for (NSUInteger s = 0; s < L; s++) [tex getBytes:out + s * W * H bytesPerRow:W * 4 bytesPerImage:W * H * 4 fromRegion:MTLRegionMake2D(0, 0, W, H) mipmapLevel:0 slice:s]; + else [tex getBytes:out bytesPerRow:W * 4 bytesPerImage:W * H * 4 fromRegion:MTLRegionMake3D(0, 0, 0, W, H, tex.depth) mipmapLevel:0 slice:0]; + assert(out[0] == (uint32_t)atoi(argv[2])); + for (NSUInteger i = 1; i < n; i++) assert(out[i] == 3); + printf("returning IGN atomic on a native %s texture: result and sentinel checks passed\n", shape); return 0; }} diff --git a/dm/gfx/tests/t_imga_layout.c b/dm/gfx/tests/t_imga_layout.c deleted file mode 100644 index 13c034a..0000000 --- a/dm/gfx/tests/t_imga_layout.c +++ /dev/null @@ -1,47 +0,0 @@ -#include -#include -#include "../nvn/nvn_imga_layout.h" - -static nvn_imga_direct_layout packed(void) { - return (nvn_imga_direct_layout){ .width=512, .height=256, .layers=1, .row_bytes=2048, - .buffer_length=1048576, .texture_offset=4096, .pointer_offset=4096, .guest_bytes=524288, - .word_bytes=4, .base_view=1, .same_buffer=1 }; -} -int main(void) { - uint64_t n=77, rounded=88; - assert(nvn_imga_bytes(512,512,3,&n) && n==3u*1024*1024); - assert(!nvn_imga_capacity(n,64u*1024,0)); // The former allocation-failure fallback is unsafe. - assert(nvn_imga_capacity(n,n,0) && !nvn_imga_capacity(n,n-1,0)); - assert(nvn_imga_scratch_length(n,&rounded) && rounded==n); - assert(nvn_imga_scratch_length(n+1,&rounded) && rounded==4u*1024*1024); - assert(nvn_imga_bytes(480,270,1,&n) && n==518400); - n=77; rounded=88; - assert(!nvn_imga_bytes(0,1,1,&n) && n==77); - assert(!nvn_imga_bytes(1,0,1,&n) && n==77); - assert(!nvn_imga_bytes(1,1,0,&n) && n==77); - assert(!nvn_imga_bytes(UINT32_MAX,UINT32_MAX,2,&n) && n==77); - assert(!nvn_imga_bytes(UINT32_MAX,UINT32_MAX,1,&n) && n==77); - assert(!nvn_imga_bytes(UINT64_MAX,1,1,&n) && n==77); - assert(!nvn_imga_bytes(1,1,1,NULL)); - assert(!nvn_imga_scratch_length(UINT64_MAX,&rounded) && rounded==88); - assert(!nvn_imga_scratch_length(0,&rounded) && rounded==88); - assert(!nvn_imga_capacity(4,UINT64_MAX,UINT64_MAX-2)); - assert(!nvn_imga_capacity(1,0,UINT64_MAX)); - assert(!nvn_imga_capacity(0,64u*1024,0)); - nvn_imga_direct_layout d=packed(); assert(nvn_imga_direct_ok(&d)); -#define REJECT(field, value) do { d=packed(); d.field=(value); assert(!nvn_imga_direct_ok(&d)); } while (0) - REJECT(base_view,0); REJECT(same_buffer,0); REJECT(word_bytes,2); REJECT(word_bytes,8); - REJECT(layers,2); REJECT(width,0); REJECT(height,0); REJECT(width,UINT64_MAX); - REJECT(row_bytes,2048+256); REJECT(row_bytes,2048-4); // Padding and truncated/explicit stride. - REJECT(pointer_offset,8192); REJECT(texture_offset,4097); - REJECT(buffer_length,4096+524287); REJECT(guest_bytes,524287); - REJECT(buffer_texture,1); // Multi-row direct buffer images are not a packed buffer view. -#undef REJECT - d=packed(); d.buffer_length=d.texture_offset+d.guest_bytes; assert(nvn_imga_direct_ok(&d)); - d=packed(); d.texture_offset=d.pointer_offset=UINT64_MAX-3; assert(!nvn_imga_direct_ok(&d)); - d=packed(); d.buffer_texture=1; d.height=1; d.row_bytes=4096; d.guest_bytes=2048; - assert(nvn_imga_direct_ok(&d)); // The unused next row of a texture buffer may be padded. - d.guest_bytes--; assert(!nvn_imga_direct_ok(&d)); - puts("image atomic layout/capacity admission: PASS"); - return 0; -} diff --git a/dm/gfx/tests/t_perf_order.m b/dm/gfx/tests/t_perf_order.m index e9d5029..5081912 100644 --- a/dm/gfx/tests/t_perf_order.m +++ b/dm/gfx/tests/t_perf_order.m @@ -121,7 +121,7 @@ static void slices(id lib) { p->sh[1]->rf.nb=1; p->sh[1]->rf.b[0]=(nvn_rbind){.kind=NVN_RB_TEX,.reg=0}; chain next={0}; rt(&next,&t,1,NULL); triangle(&next); emit(&next,OP_END,NULL,0); g_exec_next=next.p; sm_plan(p); CHECK("first draw sampling attachment cannot start slice merge",!g_sm_plan.on); - int kinds[]={NVN_RB_TEXU,NVN_RB_TEXD,NVN_RB_TEXA,NVN_RB_IMGA}; + int kinds[]={NVN_RB_TEXU,NVN_RB_TEXD,NVN_RB_TEXA}; for(unsigned i=0;ish[1]->rf.b[0].kind=kinds[i]; CHECK("unknown/bindless attachment reads split merged pass",draw_samples_rt(p)); diff --git a/dm/gfx/tests/t_refl.m b/dm/gfx/tests/t_refl.m index de6fbd1..726ca07 100644 --- a/dm/gfx/tests/t_refl.m +++ b/dm/gfx/tests/t_refl.m @@ -23,15 +23,18 @@ static NSDictionary *load(const char *key) { int main(void) { @autoreleasepool { nvn_refl r; char err[160] = ""; - { NSDictionary *j = @{@"route": @"B", @"bindings": @[ - @{@"kind": @"img", @"reg": @2, @"arg_id": @0, @"msl_name": @"cp_i_cb2_4C_r32ui", @"msl_type": @"texture_buffer"}, - @{@"kind": @"img", @"reg": @2, @"arg_id": @1, @"msl_name": @"cp_i_cb2_4C_r32ui_atomic", @"msl_type": @"device atomic_uint*"}]}; - check("buffer image: conversion succeeds", 0, nvn_refl_from_json(j, &r, err, sizeof err)); - check("buffer image: IMG kind", NVN_RB_IMG, r.b[0].kind); - check("buffer image: texture type", MTLTextureTypeTextureBuffer, r.b[0].dtt); - check("buffer image: unsigned format", NVN_DPF_UINT, r.b[0].dpf); - check("buffer image: paired atomic pointer", NVN_RB_IMGA, r.b[1].kind); - check("buffer image: same image slot", r.b[0].reg, r.b[1].reg); + { // native image atomics (dmsh_atomic.py): one read_write image, marked atomic by the build; an image the shader only stores to is not + NSDictionary *j = @{@"route": @"B", @"bindings": @[ + @{@"kind": @"img", @"reg": @2, @"arg_id": @0, @"msl_name": @"cp_i_cb2_4C_r32ui", @"msl_type": @"texture_buffer", @"atomic": @YES}, + @{@"kind": @"img", @"reg": @3, @"arg_id": @1, @"msl_name": @"cp_i_cb2_4E_r32ui", @"msl_type": @"texture3d"}]}; + check("atomic image: conversion succeeds", 0, nvn_refl_from_json(j, &r, err, sizeof err)); + check("atomic image: IMG kind", NVN_RB_IMG, r.b[0].kind); + check("atomic image: texture type", MTLTextureTypeTextureBuffer, r.b[0].dtt); + check("atomic image: unsigned format", NVN_DPF_UINT, r.b[0].dpf); + check("atomic image: written", 1, r.b[0].rw); + check("atomic image: atomic flag", 1, r.b[0].atom); + check("stored image: 3D texture type", MTLTextureType3D, r.b[1].dtt); + check("stored image: no atomic flag", 0, r.b[1].atom); } { int f0 = fail; // Route A is retired: PC-chain reflection (no route = B) must be rejected NSDictionary *pc = @{@"arg_buffer_index": @0, @"bindings": @[@{@"type": @"cbuffer", @"reg": @10, @"arg_id": @0, @"access": @"constant", @"msl_type": @"constant cb1_struct*"}], @"vertex_inputs": @[]}; diff --git a/dm/gfx/tests/test_imga_binding_source.py b/dm/gfx/tests/test_imga_binding_source.py deleted file mode 100644 index 71e2d90..0000000 --- a/dm/gfx/tests/test_imga_binding_source.py +++ /dev/null @@ -1,35 +0,0 @@ -"""Narrow production wiring checks, not a native Metal execution fixture.""" -from pathlib import Path -import unittest - -SOURCE = (Path(__file__).resolve().parents[1] / "nvn/nvn_mtl.m").read_text() - -class BindingWiring(unittest.TestCase): - def test_failed_scratch_never_falls_back_to_dummy(self): - branch = SOURCE.split('storage = atom_scratch(tt, need);', 1)[1].split('} else {', 1)[0] - self.assertIn('!storage || !gpu || !nvn_imga_capacity', branch) - self.assertIn('MDIE(', branch) - self.assertIn('refusing short dummy', branch) - self.assertNotIn('dummy_buf', branch) - - def test_actual_image_is_shared_by_pair(self): - self.assertIn('atomic_img[r] = img_binding(tid, ib,', SOURCE) - self.assertIn('atomic_desc[reg] ? atomic_img[reg]', SOURCE) - self.assertIn('id image = atomic_img[reg]', SOURCE) - self.assertIn('missing IMG partner', SOURCE) - self.assertIn('ambiguous IMG partner', SOURCE) - - def test_direct_path_checks_actual_backing(self): - branch = SOURCE.split('if (linear) {', 1)[1].split('} else if (tt) {', 1)[0] - for token in ('image.buffer == storage', 'image.bufferOffset', 'image.bufferBytesPerRow', - '!image.parentTexture', 'nvn_imga_direct_ok(&d)', 'MDIE('): - self.assertIn(token, branch) - self.assertNotIn('atom_scratch', branch) - self.assertNotIn('dummy_buf', branch) - - def test_rejection_precedes_argument_publication(self): - body = SOURCE.split('static int encode_args_(id en, NvShader *sh, int nvn_stage, int metal_vertex) {', 1)[1] - self.assertLess(body.index('refusing short dummy'), body.index('memcpy((uint8_t *)ab.contents')) - -if __name__ == '__main__': - unittest.main() diff --git a/dm/gfx/tools/dmsh_atomic.py b/dm/gfx/tools/dmsh_atomic.py index b1545cd..b3dddc8 100644 --- a/dm/gfx/tools/dmsh_atomic.py +++ b/dm/gfx/tools/dmsh_atomic.py @@ -1,57 +1,59 @@ -"""Source-owned MSL 2.3 atomic coordinate helper, after exact ABI validation. +"""Image atomics run natively on the Metal texture (MSL 3.1 texture atomics). -IR guards each original coordinate before ImageTexelPointer and remains valid -SPIR-V. Only the known standalone compiler helper is replaced: shader calls, -control flow and expressions are never matched or rewritten. Unknown backends -fail closed rather than receiving a heuristic source transformation. +SPIRV-Cross below MSL 3.1 emulates an image atomic with a raw device pointer +beside the texture (_atomic), indexed by the bound texture's width: a +block-linear image is a deswizzled Metal copy whose pool memory has another +layout, so those atomics went to a scratch buffer that nothing read, and a 3D +image did not compile at all (an int3 subscript). Native texture atomics act on +the texture every other access of the image uses, for buffer, 2D, 2D-array and +3D images. The translator still guards each coordinate (IGN bounds) before the +atomic. + +Only shaders with image atomics (OpImageTexelPointer) are written at MSL 3.1: +every other shader keeps its MSL 2.3 source byte for byte, so its library comes +from the MSL-keyed compile cache. The emulation must never come back silently: +check_msl fails the shader when SPIRV-Cross writes the pointer form anyway. """ import re +import struct -# The recognized upstream macro is the SPIRV-Cross emulation ABI (KhronosGroup, -# Apache-2.0); the typed, layer-aware implementation below is source-owned. - -_XY_HELPER = '''#define spvImage2DAtomicCoord(tc, tex) (((((tex).get_width() + spvLinearTextureAlignment / 4 - 1) & ~(spvLinearTextureAlignment / 4 - 1)) * (tc).y) + (tc).x)''' -_DEFAULT_ALIGNMENT = '''constant uint spvLinearTextureAlignment = is_function_constant_defined(spvLinearTextureAlignmentOverride) ? spvLinearTextureAlignmentOverride : 4;''' -# Explicit overloads preserve the original image-coordinate domain. uint casts -# are safe only because the IR guard dominates the atomic; use ulong for the -# address arithmetic so separate array layers cannot wrap at 32-bit offsets. -HELPERS = '''template -inline ulong spvImage2DAtomicCoord(int2 tc, texture2d tex) -{ - ulong alignment = ulong(spvLinearTextureAlignment) / 4; - ulong stride = (ulong(tex.get_width()) + alignment - 1) & ~(alignment - 1); - return stride * ulong(uint(tc.y)) + ulong(uint(tc.x)); -} -template -inline ulong spvImage2DAtomicCoord(int3 tc, texture2d_array tex) -{ - ulong alignment = ulong(spvLinearTextureAlignment) / 4; - ulong stride = (ulong(tex.get_width()) + alignment - 1) & ~(alignment - 1); - ulong row = ulong(uint(tc.z)) * ulong(tex.get_height()) + ulong(uint(tc.y)); - return stride * row + ulong(uint(tc.x)); -}''' +MSL_VERSION = '20300' +MSL_VERSION_ATOMICS = '30100' +OP_IMAGE_TEXEL_POINTER = 60 -def _helper_line(line): - return re.match(r'^\s*#\s*define\s+spvImage2DAtomicCoord\b', line) is not None +def uses_image_atomics(spv): + """Whether the SPIR-V module (path) has an image texel pointer, the operand of every image atomic""" + with open(spv, 'rb') as f: + data = f.read() + words = struct.unpack('<%dI' % (len(data) // 4), data[:len(data) // 4 * 4]) + if len(words) < 5 or words[0] != 0x07230203: + raise ValueError('not a SPIR-V module: ' + spv) + i = 5 + while i < len(words): + n, op = words[i] >> 16, words[i] & 0xffff + if op == OP_IMAGE_TEXEL_POINTER: + return True + if n == 0: + raise ValueError('malformed SPIR-V instruction in ' + spv) + i += n + return False -def validate_msl(source): - """Validate upstream emulation ABI without modifying it.""" - if 'spvImage2DAtomicCoord' not in source: - return source # buffer-image atomics have no 2D helper - compact = lambda s: re.sub(r'\s+', '', s) - helpers = [line for line in source.splitlines() if _helper_line(line)] - if len(helpers) != 1 or compact(helpers[0]) != compact(_XY_HELPER): - raise ValueError('image atomic emulation: unsupported SPIRV-Cross coordinate helper; rebuild with the reviewed MSL 2.3 compiler') - if compact(_DEFAULT_ALIGNMENT) not in compact(source): - raise ValueError('image atomic emulation: expected four-byte default linear-texture alignment') +def cross_args(xc, spv): + """spirv-cross arguments for this module: xc with its --msl-version raised to 3.1 when the module has image atomics""" + if not uses_image_atomics(spv): + return list(xc) + args = list(xc) + args[args.index('--msl-version') + 1] = MSL_VERSION_ATOMICS + return args + + +_EMULATED = re.compile(r'\bspvImage2DAtomicCoord\b|device\s+atomic_u?int\s*\*\s*\w+_atomic\b') + + +def check_msl(source): + """Fail closed on SPIRV-Cross's pointer emulation of image atomics; return the source unchanged""" + if _EMULATED.search(source): + raise ValueError('image atomics emulated through a raw pointer; they must be native texture atomics (MSL 3.1)') return source - - -def lower_msl(source): - validate_msl(source) - # The exact definition was checked above. No shader-specific source enters - # this module and no expression/call-site rewrite is permitted here. - return '\n'.join(HELPERS if _helper_line(line) else line - for line in source.split('\n')) diff --git a/dm/gfx/tools/dmsh_build.py b/dm/gfx/tools/dmsh_build.py index ace9332..2b1feb4 100644 --- a/dm/gfx/tools/dmsh_build.py +++ b/dm/gfx/tools/dmsh_build.py @@ -518,6 +518,8 @@ def reflect(key, stage, msl, res): d = decode_tex(nm) if not d: e.update(kind='?') else: e.update(kind='img' if re.match(r'[a-z]+_i', nm) else ('tbuf' if 'texture_buffer' in ty else 'tex'), reg=d[0]) + # native texture atomics (dmsh_atomic.py): Metal allows them only on an R32Uint / R32Sint texture made with the shader-atomic usage, which the host checks for this slot + if e.get('kind') == 'img' and re.search(r'\b%s\.atomic_' % re.escape(nm), msl): e['atomic'] = True # a 1D integer fetch the translator read as 2D rows of 16384 texels (Program.cs RecordTexelFetch1D): the host binds a texture buffer in this slot as such rows (nvn_mtl.m tbuf_as_2d) if e.get('kind') == 'tex' and e.get('reg') in res.get('fetch1d', ()): e['fetch1d'] = True @@ -809,9 +811,9 @@ def geometry_finish(key, w, j, c1): layout = dmsh_geometry.output_layout(code, t) k = job['key']; spv = os.path.join(w, k + '.spv') dmsh_geometry.flatten_sets(spv) - r = subprocess.run(['spirv-cross', *XC, spv], capture_output=True, text=True) + r = subprocess.run(['spirv-cross', *dmsh_atomic.cross_args(XC, spv), spv], capture_output=True, text=True) if r.returncode: return 'geometry MSL failed', r.stderr.strip()[:160], None - src = dmsh_geometry.capture_input(dmsh_geometry.topology_pointer(precise(dmsh_atomic.lower_msl(r.stdout)))) + src = dmsh_geometry.capture_input(dmsh_geometry.topology_pointer(precise(dmsh_atomic.check_msl(r.stdout)))) res = json.load(open(os.path.join(w, k + '.res.json'))) refl = {'key': key + '_bg' + str(t), 'route': 'B', 'chain': CHAIN, 'dm_ctl': key, 'source': f'{loc[key][0]}#{loc[key][1]}', 'pc_paired': pc_of[key], 'stage': 'gs', @@ -831,9 +833,9 @@ def geometry_finish(key, w, j, c1): base = {**refl, 'key': key + '_b'} save('_b', src, base) spv = os.path.join(w, k + '.reader.spv'); dmsh_geometry.flatten_sets(spv) - r = subprocess.run(['spirv-cross', *XC, spv], capture_output=True, text=True) + r = subprocess.run(['spirv-cross', *dmsh_atomic.cross_args(XC, spv), spv], capture_output=True, text=True) if r.returncode: return 'geometry reader failed', r.stderr.strip()[:160], None - reader = vs_layer(vs_init(dmsh_geometry.reader(precise(dmsh_atomic.lower_msl(r.stdout))))) + reader = vs_layer(vs_init(dmsh_geometry.reader(precise(dmsh_atomic.check_msl(r.stdout))))) analysis, full = vs_depth(reader) meta = {'key': key + '_bgv', 'route': 'B', 'chain': CHAIN, 'dm_ctl': key, 'stage': 'vs', 'entry': 'main0', 'entry_z': 'main0_z', 'msl_entry_kind': 'vertex', @@ -856,7 +858,7 @@ def finish(key, w, j, c1): try: return geometry_finish(key, w, j, c1) except (ValueError, subprocess.CalledProcessError) as e: return 'geometry failed', str(e)[:160], None if j['stage'] == 'vs': spv_single_lane(os.path.join(w, key + '.spv')) # one-lane subgroups (see spv_single_lane) - r = subprocess.run(['spirv-cross', *XC, os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True) + r = subprocess.run(['spirv-cross', *dmsh_atomic.cross_args(XC, os.path.join(w, key + '.spv')), os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True) if r.returncode: return 'MSL failed', r.stderr.strip()[:160], None src = open(msl).read() if j['stage'] == 'vs': # second pass: re-translate with inputs that are only used as integers reported as uint (see int_inputs) @@ -866,10 +868,10 @@ def finish(key, w, j, c1): rx = subprocess.run([XLATE, jp, w], capture_output=True, text=True) if rx.returncode or not os.path.exists(os.path.join(w, key + '.spv')): return 'translation failed', ('integer-input retranslation: ' + rx.stderr.strip())[:160], None spv_single_lane(os.path.join(w, key + '.spv')) - r = subprocess.run(['spirv-cross', *XC, os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True) + r = subprocess.run(['spirv-cross', *dmsh_atomic.cross_args(XC, os.path.join(w, key + '.spv')), os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True) if r.returncode: return 'MSL failed', r.stderr.strip()[:160], None src = open(msl).read() - try: src = dmsh_atomic.lower_msl(src) + try: src = dmsh_atomic.check_msl(src) except ValueError as e: return 'image atomic MSL failed', str(e), None src = precise(src); open(msl, 'w').write(src) # min / max / clamp with Maxwell's NaN results (see precise) pad = [] diff --git a/dm/gfx/tools/dmsh_chain.lock b/dm/gfx/tools/dmsh_chain.lock index d50b6b5..109498d 100644 --- a/dm/gfx/tools/dmsh_chain.lock +++ b/dm/gfx/tools/dmsh_chain.lock @@ -1,8 +1,8 @@ # Translation chain tool fingerprints (written by dmsh_chain.py --update-lock; regenerate after changing tools and commit it) -dmsh_xlate.dll 45e24808a2a9b98a -Ryujinx.Graphics.Shader.dll 24190e274674f0c9 -Ryujinx.Common.dll 0fd1116a59e8ab97 -xlate_source 7c5bb25142d9eb33 +dmsh_xlate.dll 00fd93fb700c00bd +Ryujinx.Graphics.Shader.dll 7f672712dc65473e +Ryujinx.Common.dll c0ea16a8c189cd95 +xlate_source f21af1efa737a270 spirv-cross d9dc197f061b42a5 metal_macosx Apple metal version 32023.921 (metalfe-32023.921.6) metal_iphoneos Apple metal version 32023.921 (metalfe-32023.921.6) diff --git a/dm/gfx/tools/dmsh_xlate/Program.cs b/dm/gfx/tools/dmsh_xlate/Program.cs index 72c3cb0..f53c2ef 100644 --- a/dm/gfx/tools/dmsh_xlate/Program.cs +++ b/dm/gfx/tools/dmsh_xlate/Program.cs @@ -161,7 +161,7 @@ static class SwshXlate byte[] code = File.ReadAllBytes((string)job["code"]); var acc = new Accessor(code, job); // Job debug = false: turn off DebugMode (DebugMode skips the AST optimizer and the output is full of temporaries; semantics unchanged) - var flags = TranslationFlags.Msl23ImageAtomics | (job["debug"] is JsonNode dbg && !(bool)dbg ? 0 : TranslationFlags.DebugMode) | ((string)job["stage"] == "cs" ? TranslationFlags.Compute : 0); + var flags = TranslationFlags.MetalImageAtomics | (job["debug"] is JsonNode dbg && !(bool)dbg ? 0 : TranslationFlags.DebugMode) | ((string)job["stage"] == "cs" ? TranslationFlags.Compute : 0); var ctx = Translator.CreateContext(0, acc, new TranslationOptions(TargetLanguage.Spirv, TargetApi.Vulkan, flags)); bool geometry = job["geometry_topology"] != null; ShaderProgram p = ctx.Translate(geometry); diff --git a/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Common.dll b/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Common.dll index a04a844..4847689 100644 Binary files a/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Common.dll and b/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Common.dll differ diff --git a/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Graphics.Shader.dll b/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Graphics.Shader.dll index a784f87..1df7c1f 100644 Binary files a/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Graphics.Shader.dll and b/dm/gfx/tools/dmsh_xlate_bin/Ryujinx.Graphics.Shader.dll differ diff --git a/dm/gfx/tools/dmsh_xlate_bin/Spv.Generator.dll b/dm/gfx/tools/dmsh_xlate_bin/Spv.Generator.dll index ee9f4ff..7bc4dde 100644 Binary files a/dm/gfx/tools/dmsh_xlate_bin/Spv.Generator.dll and b/dm/gfx/tools/dmsh_xlate_bin/Spv.Generator.dll differ diff --git a/dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate.dll b/dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate.dll index 91bce36..e143939 100644 Binary files a/dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate.dll and b/dm/gfx/tools/dmsh_xlate_bin/dmsh_xlate.dll differ diff --git a/dm/third_party/ryujinx/README.md b/dm/third_party/ryujinx/README.md index 00a2fac..a85f6b2 100644 --- a/dm/third_party/ryujinx/README.md +++ b/dm/third_party/ryujinx/README.md @@ -12,7 +12,7 @@ Three projects from Ryujinx 1.3.2: `Ryujinx.Graphics.Shader`, `Ryujinx.Common` a - `Ryujinx.Graphics.Shader/Translation/EmitterContext.cs` and `Translation/ShaderDefinitions.cs`: fixed-input compute lowering explicitly forwards passthrough positions and all captured generic attributes not replaced by the geometry shader, and inherits its output primitive from input topology. Independent offline stages have no next-stage usage mask, and Metal cannot provide the native NV passthrough raster stage. See [passthrough compute emission](../../../docs/reference/shaders.md#passthrough-geometry-kernels-lose-raster-vertices) and [output bounds](../../../docs/reference/shaders.md#passthrough-geometry-output-bounds-follow-input-topology). -- `Ryujinx.Graphics.Shader/Instructions/InstEmitSurface.cs`, `IntermediateRepresentation/TextureFlags.cs` and `CodeGen/Spirv/Instructions.cs`: the MSL 2.3 path bounds image atomics before forming the texel pointer. Returning `IGN` atomics preserve the old texel on the valid edge and return zero outside the image; CAS, predicate results and non-IGN modes remain rejected. See [bounded MSL 2.3 atomic emulation](../../../docs/reference/shaders.md#bounded-msl-23-atomic-emulation), found on SHIN MEGAMI TENSEI V: Vengeance's particle counters. +- `Ryujinx.Graphics.Shader/Instructions/InstEmitSurface.cs`, `IntermediateRepresentation/TextureFlags.cs` and `CodeGen/Spirv/Instructions.cs`: the Metal path (`TranslationFlags.MetalImageAtomics`) bounds image atomics before forming the texel pointer, which SPIRV-Cross writes as a native texture atomic (MSL 3.1) on buffer, 2D, 2D-array and 3D images. Returning `IGN` atomics preserve the old texel on the valid edge and return zero outside the image; CAS is a native texture compare-exchange behind the same guard; predicate results and non-IGN modes remain rejected. See [bounds of image atomics](../../../docs/reference/shaders.md#bounds-of-image-atomics) and [image atomics](../../../docs/reference/shaders.md#image-atomics), found on SHIN MEGAMI TENSEI V: Vengeance's particle counters. The shader translation tool in `dm/gfx/tools/dmsh_xlate` references them to translate the original Switch shaders to SPIR-V. License: MIT, see `LICENSE.txt`. Upstream: . diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/CodeGen/Spirv/Instructions.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/CodeGen/Spirv/Instructions.cs index 06724bd..ca609bc 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/CodeGen/Spirv/Instructions.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/CodeGen/Spirv/Instructions.cs @@ -622,19 +622,22 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv ImageDeclaration declaration = context.Images[texOp.GetTextureSetAndBinding()]; SpvInstruction image = declaration.Image; - bool bounded = context.HostCapabilities.Msl23ImageAtomics; + bool bounded = context.HostCapabilities.MetalImageAtomics; + // CAS is a native texture compare-exchange on Metal (SPIRV-Cross loops + // atomic_compare_exchange_weak into a strong one), so it takes the same guard. if (bounded && ((texOp.Flags & TextureFlags.AtomicBoundsMask) != 0 || - texOp.Flags.HasFlag(TextureFlags.AtomicPredicateResult) || - (texOp.Flags & TextureFlags.AtomicMask) == TextureFlags.CAS)) + texOp.Flags.HasFlag(TextureFlags.AtomicPredicateResult))) { - throw new NotSupportedException("MSL 2.3 image atomics require IGN without CAS or a predicate result; TRAP semantics are not implemented."); + throw new NotSupportedException("Metal image atomics require IGN without a predicate result; TRAP semantics are not implemented."); } + // Metal's texture atomics (MSL 3.1) cover buffer, 2D, 2D-array and 3D textures of + // R32Uint / R32Sint; source 1D surfaces arrive here already normalized to 2D. if (bounded && (declaration.IsIndexed || (texOp.Type != SamplerType.TextureBuffer && texOp.Type != SamplerType.Texture2D && - texOp.Type != (SamplerType.Texture2D | SamplerType.Array)) || + texOp.Type != (SamplerType.Texture2D | SamplerType.Array) && texOp.Type != SamplerType.Texture3D) || (texOp.Format != TextureFormat.R32Uint && texOp.Format != TextureFormat.R32Sint))) { - throw new NotSupportedException("MSL 2.3 bounded image atomics support non-indexed 32-bit buffer/2D/2D-array images only."); + throw new NotSupportedException("Metal image atomics support non-indexed 32-bit buffer/2D/2D-array/3D images only."); } SpvInstruction resultType = context.GetType(componentType); diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Instructions/InstEmitSurface.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Instructions/InstEmitSurface.cs index b8ef502..317a0dd 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Instructions/InstEmitSurface.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Instructions/InstEmitSurface.cs @@ -201,11 +201,11 @@ namespace Ryujinx.Graphics.Shader.Instructions { // Preserve source width before descriptor-sized formats are resolved. // Other backends retain their existing surface-instruction handling. - bool msl23 = context.TranslatorContext.Options.Flags.HasFlag(TranslationFlags.Msl23ImageAtomics); - if (msl23 && + bool metalAtomics = context.TranslatorContext.Options.Flags.HasFlag(TranslationFlags.MetalImageAtomics); + if (metalAtomics && size is not (SuatomSize.U32 or SuatomSize.S32 or SuatomSize.Sd32)) { - throw new NotSupportedException("MSL 2.3 image atomics require a 32-bit integer source size."); + throw new NotSupportedException("Metal image atomics require a 32-bit integer source size."); } SamplerType type = ConvertSamplerType(dimensions); @@ -276,7 +276,7 @@ namespace Ryujinx.Graphics.Shader.Instructions // TODO: FP and 64-bit formats. TextureFormat format = size is SuatomSize.Sd32 or SuatomSize.Sd64 - ? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm, strict: msl23)) + ? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm, strict: metalAtomics)) : GetTextureFormat(size); if (compareAndSwap) @@ -289,9 +289,9 @@ namespace Ryujinx.Graphics.Shader.Instructions Operand[] sources = sourcesList.ToArray(); TextureFlags flags = (compareAndSwap ? TextureFlags.CAS : GetAtomicOpFlags(atomicOp)) | (TextureFlags)((int)clamp << 20); - if (msl23) + if (metalAtomics) { - flags |= TextureFlags.AtomicMsl23; + flags |= TextureFlags.AtomicMetal; } if (dest == RegisterConsts.RegisterZeroIndex && !compareAndSwap && !hasPredicateResult) { @@ -504,11 +504,11 @@ namespace Ryujinx.Graphics.Shader.Instructions { // Preserve source width before descriptor-sized formats are resolved. // Other backends retain their existing surface-instruction handling. - bool msl23 = context.TranslatorContext.Options.Flags.HasFlag(TranslationFlags.Msl23ImageAtomics); - if (msl23 && + bool metalAtomics = context.TranslatorContext.Options.Flags.HasFlag(TranslationFlags.MetalImageAtomics); + if (metalAtomics && size is not (SuatomSize.U32 or SuatomSize.S32 or SuatomSize.Sd32)) { - throw new NotSupportedException("MSL 2.3 image atomics require a 32-bit integer source size."); + throw new NotSupportedException("Metal image atomics require a 32-bit integer source size."); } SamplerType type = ConvertSamplerType(dimensions); @@ -577,7 +577,7 @@ namespace Ryujinx.Graphics.Shader.Instructions // TODO: FP and 64-bit formats. TextureFormat format = size is SuatomSize.Sd32 or SuatomSize.Sd64 - ? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm, strict: msl23)) + ? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm, strict: metalAtomics)) : GetTextureFormat(size); sourcesList.Add(Rb()); @@ -586,9 +586,9 @@ namespace Ryujinx.Graphics.Shader.Instructions TextureFlags flags = GetAtomicOpFlags((SuatomOp)atomicOp) | (TextureFlags)((int)clamp << 20) | TextureFlags.AtomicDiscardResult; - if (msl23) + if (metalAtomics) { - flags |= TextureFlags.AtomicMsl23; + flags |= TextureFlags.AtomicMetal; } if (isBindless) diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/IntermediateRepresentation/TextureFlags.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/IntermediateRepresentation/TextureFlags.cs index cc78174..934ea5b 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/IntermediateRepresentation/TextureFlags.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/IntermediateRepresentation/TextureFlags.cs @@ -20,7 +20,7 @@ namespace Ryujinx.Graphics.Shader.IntermediateRepresentation // the operation; TRAP/unknown must not silently become IGN on Metal. AtomicBoundsMask = 3 << 20, AtomicDiscardResult = 1 << 22, - AtomicMsl23 = 1 << 23, + AtomicMetal = 1 << 23, AtomicPredicateResult = 1 << 24, AtomicMask = 15 << 16, diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/StructuredIr/ShaderProperties.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/StructuredIr/ShaderProperties.cs index d1962c5..6726672 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/StructuredIr/ShaderProperties.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/StructuredIr/ShaderProperties.cs @@ -102,7 +102,7 @@ namespace Ryujinx.Graphics.Shader.StructuredIr { if (strict) { - throw new NotSupportedException($"MSL 2.3 image atomics require an R32 integer descriptor format, got {format}."); + throw new NotSupportedException($"Metal image atomics require an R32 integer descriptor format, got {format}."); } gpuAccessor.Log($"Unsupported format for texture {handle}: {format}."); diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/HostCapabilities.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/HostCapabilities.cs index be90357..bab2050 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/HostCapabilities.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/HostCapabilities.cs @@ -2,7 +2,7 @@ namespace Ryujinx.Graphics.Shader.Translation { class HostCapabilities { - public readonly bool Msl23ImageAtomics; + public readonly bool MetalImageAtomics; public readonly bool ReducedPrecision; public readonly bool SupportsFragmentShaderInterlock; public readonly bool SupportsFragmentShaderOrderingIntel; @@ -25,9 +25,9 @@ namespace Ryujinx.Graphics.Shader.Translation bool supportsShaderNonUniformIndexing, bool supportsTextureShadowLod, bool supportsViewportMask, - bool msl23ImageAtomics) + bool metalImageAtomics) { - Msl23ImageAtomics = msl23ImageAtomics; + MetalImageAtomics = metalImageAtomics; ReducedPrecision = reducedPrecision; SupportsFragmentShaderInterlock = supportsFragmentShaderInterlock; SupportsFragmentShaderOrderingIntel = supportsFragmentShaderOrderingIntel; diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/Optimizations/BindlessElimination.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/Optimizations/BindlessElimination.cs index ec8ab43..0250c0f 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/Optimizations/BindlessElimination.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/Optimizations/BindlessElimination.cs @@ -48,9 +48,9 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations } bool converted = table == TableHandle.Set && TryConvertBindless(block, resourceManager, gpuAccessor, texOp); - if (!converted && texOp.Inst == Instruction.ImageAtomic && texOp.Flags.HasFlag(TextureFlags.AtomicMsl23)) + if (!converted && texOp.Inst == Instruction.ImageAtomic && texOp.Flags.HasFlag(TextureFlags.AtomicMetal)) { - throw new NotSupportedException("MSL 2.3 image atomics require a statically resolved image handle."); + throw new NotSupportedException("Metal image atomics require a statically resolved image handle."); } if (!converted && @@ -573,7 +573,7 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations if (texOp.Inst == Instruction.ImageAtomic) { texOp.Format = ShaderProperties.GetTextureFormatAtomic(gpuAccessor, cbufOffset, cbufSlot, - strict: texOp.Flags.HasFlag(TextureFlags.AtomicMsl23)); + strict: texOp.Flags.HasFlag(TextureFlags.AtomicMetal)); } else { diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslationFlags.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslationFlags.cs index 253d38b..2307ee6 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslationFlags.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslationFlags.cs @@ -10,7 +10,7 @@ namespace Ryujinx.Graphics.Shader.Translation VertexA = 1 << 0, Compute = 1 << 1, DebugMode = 1 << 2, - // Offline MSL 2.3: SPIRV-Cross uses a raw pointer for image atomics. - Msl23ImageAtomics = 1 << 3, + // Offline Metal: image atomics run natively on the texture (MSL 3.1), guarded for IGN bounds. + MetalImageAtomics = 1 << 3, } } diff --git a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslatorContext.cs b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslatorContext.cs index 085b980..67f07c0 100644 --- a/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslatorContext.cs +++ b/dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader/Translation/TranslatorContext.cs @@ -367,7 +367,7 @@ namespace Ryujinx.Graphics.Shader.Translation GpuAccessor.QueryHostSupportsShaderNonUniformIndexing(), GpuAccessor.QueryHostSupportsTextureShadowLod(), GpuAccessor.QueryHostSupportsViewportMask(), - Options.Flags.HasFlag(TranslationFlags.Msl23ImageAtomics)); + Options.Flags.HasFlag(TranslationFlags.MetalImageAtomics)); CodeGenParameters parameters = new CodeGenParameters(attributeUsage, definitions, resourceManager.Properties, hostCapabilities, GpuAccessor, Options.TargetApi); diff --git a/dm/tools/tests/test_image_atomic_bounds.py b/dm/tools/tests/test_image_atomic_bounds.py index 88065da..5aa40ce 100644 --- a/dm/tools/tests/test_image_atomic_bounds.py +++ b/dm/tools/tests/test_image_atomic_bounds.py @@ -1,4 +1,5 @@ -"""MSL-2.3 image atomic contract, using synthetic shaders only. +"""Native Metal image atomic contract (MSL 3.1 texture atomics behind the IGN bounds guard), +using synthetic shaders only. Portable tests require Python. To exercise the actual rebuilt offline compiler: DMSH_TEST_XLATE=/absolute/path/to/dmsh_xlate python3 -m unittest discover \ @@ -7,7 +8,8 @@ Requires spirv-cross and spirv-val on PATH; optionally DMSH_TEST_METAL=1 on Mac also compiles and links each generated MSL file. DMSH_TEST_METAL_SDKS defaults to macosx; set macosx,iphoneos to check both SDKs with the production flags. Set DMSH_TEST_GPU=1 with DMSH_TEST_METAL=1 to execute the returning buffer -fixtures on a Mac GPU and check returned values and surrounding sentinels. +fixtures (buffer, 2D, 2D-array and 3D textures) on a Mac GPU and check returned +values and surrounding sentinels. No game inputs are used. """ import ast @@ -26,6 +28,15 @@ ROOT = Path(__file__).resolve().parents[3] RYU = ROOT / 'dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader' spec = importlib.util.spec_from_file_location('dmsh_atomic', ROOT / 'dm/gfx/tools/dmsh_atomic.py') atomic = importlib.util.module_from_spec(spec); spec.loader.exec_module(atomic) +XC = ['--msl', '--msl-version', '20300', '--msl-argument-buffers', '--msl-texture-buffer-native'] # dmsh_build.XC without fragment padding + + +def cross(spv, msl): + """spirv-cross as dmsh_build runs it: MSL 3.1 for a module with image atomics, which must come out as native texture atomics""" + subprocess.run(['spirv-cross', *atomic.cross_args(XC, str(spv)), str(spv), '--output', str(msl)], check=True) + source = atomic.check_msl(msl.read_text()) + assert '.atomic_fetch_' in source and 'access::read_write' in source, msl + return source def offset(coord, extent): @@ -35,16 +46,22 @@ def offset(coord, extent): return coord[0] + extent[0] * (coord[1] + extent[1] * coord[2]) -def synthetic_shader(x, y, layer=0, *, array=True, mode=0, discard=True, reduction=False, size=0, dim=None, bindless=None, consume_result=False): +def synthetic_shader(x, y, layer=0, *, array=True, mode=0, discard=True, reduction=False, size=0, dim=None, bindless=None, consume_result=False, cas=None): table = (RYU / 'Decoders/InstTable.cs').read_text() def base(name): pattern = re.search(r'Add\("([01x]{64})", InstName\.' + name + r'\s*,', table)[1] return int(pattern.replace('x', '0'), 2) def mov(reg, value): return base('Mov32i') | ((value & 0xffffffff) << 20) | (7 << 16) | (15 << 12) | reg - instructions = [mov(0, x), mov(1, y), mov(2, layer), mov(3, 1)] + instructions = [mov(0, x), mov(1, y), mov(2, layer), mov(3, cas[0] if cas else 1)] dim = (4 if array else 3) if dim is None else dim - if reduction: + if cas: + # SUATOM.CAS R4, [R0..R2], R3, R4: compare with R3, swap in R4, the old texel into R4. + assert not reduction and not discard and bindless is None + instructions.append(mov(4, cas[1])) + # Bit 54 set, as in the real encoding (0xEAC...): clear, the word also matches SUATOM's pattern. + op = base('SuatomCas') | (1 << 54) | (mode << 49) | (0x48 << 36) | (dim << 33) | (3 << 20) | (7 << 16) | 4 + elif reduction: op = base('Sured') | (mode << 49) | (0x48 << 36) | (dim << 33) | (2 << 24) | (7 << 16) | 3 else: op = base('Suatom') | (mode << 49) | (0x48 << 36) | (dim << 33) | (2 << 29) | (3 << 20) | (7 << 16) | (255 if discard else 4) @@ -137,44 +154,30 @@ class Portable(unittest.TestCase): self.assertEqual(offset((7, 3, 2), (8, 4, 3)), 95) self.assertEqual([offset((0, 0, z), (8, 4, 3)) for z in range(3)], [0, 32, 64]) - def test_compiler_helper_contract(self): - source = atomic._XY_HELPER + '\n' + atomic._DEFAULT_ALIGNMENT - self.assertEqual(atomic.validate_msl(source), source) - self.assertIn(atomic.HELPERS, atomic.lower_msl(source)) - self.assertNotIn('#define spvImage2DAtomicCoord', atomic.lower_msl(source)) - self.assertIn(atomic.HELPERS, atomic.lower_msl(source.replace(' ', ' '))) - self.assertEqual(atomic.validate_msl('fragment void main0() {}'), 'fragment void main0() {}') - for changed in [source.replace('(tc).x)', '(tc).z)'), source.replace(': 4;', ': 256;'), source.replace('#define', 'inline'), source + '\n' + atomic._XY_HELPER]: - with self.assertRaises(ValueError): atomic.validate_msl(changed) + def test_msl_version_follows_image_atomics(self): + with tempfile.TemporaryDirectory(prefix='dm-atomic-spv-') as temp: + def module(name, ops): + path = Path(temp) / name + path.write_bytes(struct.pack('<5I', 0x07230203, 0x10000, 0, 100, 0) + b''.join(struct.pack('<%dI' % len(o), *o) for o in ops)) + return str(path) + capability = [(2 << 16) | 17, 1] # OpCapability Shader + plain = module('plain.spv', [capability]) + texel = module('texel.spv', [capability, [(6 << 16) | 60, 1, 2, 3, 4, 5]]) # OpImageTexelPointer + self.assertFalse(atomic.uses_image_atomics(plain)) + self.assertTrue(atomic.uses_image_atomics(texel)) + self.assertEqual(atomic.cross_args(XC, plain), XC) + self.assertEqual(atomic.cross_args(XC, texel), XC[:2] + ['30100'] + XC[3:]) + self.assertEqual(XC[2], '20300') # not changed in place + with self.assertRaises(ValueError): atomic.uses_image_atomics(module('zero.spv', [[0, 0]])) + not_spirv = Path(temp) / 'not.spv'; not_spirv.write_bytes(b'\0' * 20) + with self.assertRaises(ValueError): atomic.uses_image_atomics(str(not_spirv)) - @unittest.skipUnless(shutil.which('c++'), 'portable C++ compiler unavailable') - def test_exact_helper_body(self): - # Compile and execute the exact source-owned helper, with only the - # texture/type API mocked. This is not a Metal compiler/GPU test. - source = '''#include -#include -using uint = uint32_t; using ulong = uint64_t; -enum access { sample }; struct int2 { int x,y; }; struct int3 { int x,y,z; }; -uint spvLinearTextureAlignment = 4; -template struct texture2d { uint w,h; uint get_width() const {return w;} uint get_height() const {return h;} }; -template struct texture2d_array { uint w,h; uint get_width() const {return w;} uint get_height() const {return h;} }; -''' + atomic.HELPERS + ''' -int main() { - texture2d two{8,4}; texture2d_array array{8,4}; - assert(spvImage2DAtomicCoord(int2{7,3},two)==31); - assert(spvImage2DAtomicCoord(int3{7,3,0},array)==31); - assert(spvImage2DAtomicCoord(int3{0,0,1},array)==32); - assert(spvImage2DAtomicCoord(int3{7,3,2},array)==95); - texture2d_array large{16384,16384}; - assert(spvImage2DAtomicCoord(int3{7,3,65},large)==ulong(65)*16384*16384+3*16384+7); - spvLinearTextureAlignment=256; - texture2d_array padded{17,3}; - assert(spvImage2DAtomicCoord(int3{16,2,1},padded)==64*(3+2)+16); -}''' - with tempfile.TemporaryDirectory(prefix='dm-atomic-helper-') as temp: - path=Path(temp); (path/'test.cpp').write_text(source) - subprocess.run(['c++','-std=c++17','-Wall','-Wextra','-Werror',str(path/'test.cpp'),'-o',str(path/'test')],check=True) - subprocess.run([str(path/'test')],check=True) + def test_pointer_emulation_rejected(self): + native = 'uint _1 = spvDescriptorSet0.img.atomic_fetch_add(uint3(c), 1u).x;' + self.assertEqual(atomic.check_msl(native), native) + for emulated in (' device atomic_uint* img_atomic [[id(2)]];', 'device atomic_int* cp_i_cb2_4C_r32i_atomic [[id(1)]];', + '#define spvImage2DAtomicCoord(tc, tex) ((tc).x)'): + with self.assertRaises(ValueError): atomic.check_msl(emulated) def test_mode_and_result_are_retained(self): src = (RYU / 'Instructions/InstEmitSurface.cs').read_text() @@ -188,7 +191,8 @@ int main() { self.assertIn('context.ULessThan(compareType, coords, extent)', part) self.assertNotIn('context.IMul(uintType, height, layer)', part) self.assertEqual(part.count('pCoords ='), 2) # scalar/vector construction only; never folded - self.assertIn('context.HostCapabilities.Msl23ImageAtomics', part) + self.assertIn('context.HostCapabilities.MetalImageAtomics', part) + self.assertIn('texOp.Type != SamplerType.Texture3D', part) self.assertIn('texOp.Format != TextureFormat.R32Uint', part) self.assertIn('texOp.Format != TextureFormat.R32Sint', part) self.assertEqual(src.count('size is not (SuatomSize.U32 or SuatomSize.S32 or SuatomSize.Sd32)'), 2) @@ -213,7 +217,7 @@ class Compiler(unittest.TestCase): def test_returning_ign_atomic(self): with tempfile.TemporaryDirectory(prefix='dm-image-atomic-return-') as temp: work = Path(temp) - for dim, image in ((1, 'buf'), (3, '2d'), (4, '2darray')): + for dim, image in ((1, 'buf'), (3, '2d'), (4, '2darray'), (5, '3d')): for x in (-1, 0, 8): key = 'return_%d_%d' % (dim, x) code = work / (key + '.bin') @@ -249,10 +253,32 @@ class Compiler(unittest.TestCase): return any(reaches(source, target, seen) for source in dependencies.get(value, ())) self.assertTrue(any(op == 234 and reaches(args[-1], phi[1]) for op, args in ops)) msl = work / (key + '.metal') - subprocess.run(['spirv-cross', '--msl', '--msl-version', '20300', '--msl-argument-buffers', - '--msl-texture-buffer-native', str(spv), '--output', str(msl)], check=True) - msl.write_text(atomic.lower_msl(msl.read_text())) - self.compile_metal(msl, work, key) + cross(spv, msl) + self.compile_metal(msl, work, key, gpu=('buffer' if image == 'buf' else image, 6 if x == 0 else 3)) + + def test_compare_and_swap(self): + # Texel 0 holds 3. A CAS at x swaps in 7 when it compares equal and returns the old texel, + # which a second atomic adds to texel 0: 7 + 3 = 10 (swapped), 3 + 3 = 6 (compared unequal), 3 (outside). + with tempfile.TemporaryDirectory(prefix='dm-image-atomic-cas-') as temp: + work = Path(temp) + for dim, image in ((1, 'buf'), (3, '2d'), (4, '2darray'), (5, '3d')): + for x in (-1, 0, 8): + for compare in (3, 2): + key = 'cas_%d_%d_%d' % (dim, x, compare) + code = work / (key + '.bin') + code.write_bytes(synthetic_shader(x, 0, dim=dim, discard=False, consume_result=True, cas=(compare, 7))) + jobs = work / 'jobs.json' + jobs.write_text(json.dumps([dict(key=key, stage='cs', code=str(code), + img={'0': dict(type=image, fmt='R32Uint')}, local=[1, 1, 1], debug=True)])) + result = subprocess.run([os.environ['DMSH_TEST_XLATE'], str(jobs), str(work)], capture_output=True, text=True) + self.assertEqual(result.returncode, 0, result.stderr) + spv = work / (key + '.spv') + subprocess.run(['spirv-val', '--target-env', 'vulkan1.1', str(spv)], check=True) + self.assertTrue(any(op == 230 for op, _ in instructions(spv))) # OpAtomicCompareExchange + msl = work / (key + '.metal') + self.assertIn('atomic_compare_exchange_weak', cross(spv, msl)) + expected = (10 if compare == 3 else 6) if x == 0 else 3 + self.compile_metal(msl, work, key, gpu=('buffer' if image == 'buf' else image, expected)) def test_descriptor_sized_formats(self): # Sd32 takes its format from the descriptor. Validate that original @@ -290,7 +316,7 @@ class Compiler(unittest.TestCase): self.assertIn('require a statically resolved image handle', result.stderr) self.assertFalse((work / 'unresolved.spv').exists()) - def compile_metal(self, msl, work, key): + def compile_metal(self, msl, work, key, gpu=None): if os.environ.get('DMSH_TEST_METAL') != '1': return metal_sdks = os.environ.get('DMSH_TEST_METAL_SDKS', 'macosx').split(',') @@ -308,20 +334,21 @@ class Compiler(unittest.TestCase): self.assertGreater(air.stat().st_size, 0) self.assertGreater(library.stat().st_size, 0) print('synthetic Metal compile/link passed:', key, sdk, library.stat().st_size, 'bytes') - if sdk == 'macosx' and key.startswith('return_1_') and os.environ.get('DMSH_TEST_GPU') == '1': + if sdk == 'macosx' and gpu and os.environ.get('DMSH_TEST_GPU') == '1': executable = work / 'atomic-return' if not executable.exists(): subprocess.run(['clang', '-fobjc-arc', '-O1', '-Wall', '-Werror', str(ROOT / 'dm/gfx/tests/t_atomic_return.m'), '-framework', 'Foundation', '-framework', 'Metal', '-o', str(executable)], check=True) - expected = 6 if int(key.removeprefix('return_1_')) == 0 else 3 - subprocess.run([str(executable), str(library), str(expected)], check=True) + shape, expected = gpu + subprocess.run([str(executable), str(library), str(expected), shape], check=True, + env=dict(os.environ, MTL_DEBUG_LAYER='1')) def test_dimension_routes(self): # 1D routes retain the existing y=0 normalization; buffer images stay - # scalar and must not acquire the source-owned 2D helper. + # scalar; 3D images keep all three coordinates. shapes = [('1d', (1, 0)), ('buffer', (1,)), ('1darray', (1, 0, 2)), - ('2d', (1, 2)), ('2darray', (1, 2, 3)), ('3d', None)] + ('2d', (1, 2)), ('2darray', (1, 2, 3)), ('3d', (1, 2, 3))] with tempfile.TemporaryDirectory(prefix='dm-image-atomic-dim-') as temp: work = Path(temp) for reduction in (False, True): @@ -334,24 +361,13 @@ class Compiler(unittest.TestCase): img={'0': dict(type=shape, fmt='R32Uint')}, local=[1, 1, 1], debug=True)])) result = subprocess.run([os.environ['DMSH_TEST_XLATE'], str(jp), str(work)], capture_output=True, text=True) spv = work / (key + '.spv') - if expected is None: - self.assertNotEqual(result.returncode, 0) - self.assertIn('support non-indexed 32-bit buffer/2D/2D-array images only', result.stderr) - self.assertFalse(spv.exists()) - continue self.assertEqual(result.returncode, 0, result.stderr) subprocess.run(['spirv-val', '--target-env', 'vulkan1.1', str(spv)], check=True) check_coordinate_guard(instructions(spv), expected) msl = work / (key + '.metal') - subprocess.run(['spirv-cross', '--msl', '--msl-version', '20300', '--msl-argument-buffers', - '--msl-texture-buffer-native', str(spv), '--output', str(msl)], check=True) - original = msl.read_text(); source = atomic.lower_msl(original) - if shape == 'buffer': - self.assertEqual(source, original) - self.assertNotIn('spvImage2DAtomicCoord', source) - else: - self.assertIn(atomic.HELPERS, source) - msl.write_text(source) + source = cross(spv, msl) + self.assertIn({'buffer': 'texture_buffer<', '3d': 'texture3d<'}.get(shape, 'texture2d_array<' if 'array' in shape else 'texture2d<'), source) + if shape == '3d': self.assertIn('.get_depth()', source) self.compile_metal(msl, work, key) def test_source_size_gate(self): @@ -371,9 +387,7 @@ class Compiler(unittest.TestCase): self.assertEqual(result.returncode, 0, result.stderr) subprocess.run(['spirv-val', '--target-env', 'vulkan1.1', str(spv)], check=True) msl = work / (key + '.metal') - subprocess.run(['spirv-cross', '--msl', '--msl-version', '20300', '--msl-argument-buffers', - '--msl-texture-buffer-native', str(spv), '--output', str(msl)], check=True) - msl.write_text(atomic.lower_msl(msl.read_text())) + cross(spv, msl) self.compile_metal(msl, work, key) else: # U64, S64, descriptor-sized64, float and half2. self.assertNotEqual(result.returncode, 0, key) @@ -414,11 +428,7 @@ class Compiler(unittest.TestCase): self.assertTrue(any(op == 176 for op, _ in ops)) # OpULessThan self.assertTrue(any(op == 239 for op, _ in ops)) # OpAtomicUMax msl = work / (job['key'] + '.metal') - subprocess.run(['spirv-cross','--msl','--msl-version','20300','--msl-argument-buffers','--msl-texture-buffer-native',str(spv),'--output',str(msl)], check=True) - original = msl.read_text(); source = atomic.lower_msl(original) - helper_line = next(line for line in original.splitlines() if atomic._helper_line(line)) - self.assertEqual(source, original.replace(helper_line, atomic.HELPERS, 1)) - msl.write_text(source) + source = cross(spv, msl) self.assertIn('.get_height()', source) if array: self.assertIn('.get_array_size()', source) self.compile_metal(msl, work, job['key']) @@ -428,7 +438,7 @@ class Compiler(unittest.TestCase): jp.write_text(json.dumps([dict(key=key,stage='cs',code=str(code),img={'0':dict(type='2darray',fmt='R32Uint')},local=[1,1,1],debug=True)])) result=subprocess.run([os.environ['DMSH_TEST_XLATE'],str(jp),str(work)],capture_output=True,text=True) self.assertNotEqual(result.returncode,0) - self.assertIn('require IGN without CAS or a predicate result',result.stderr) + self.assertIn('require IGN without a predicate result',result.stderr) self.assertFalse((work/(key+'.spv')).exists()) diff --git a/docs/agents/symptoms.md b/docs/agents/symptoms.md index 95f82a1..17561a2 100644 --- a/docs/agents/symptoms.md +++ b/docs/agents/symptoms.md @@ -450,9 +450,9 @@ One line per symptom in the [subsystem references](../reference/): what you see - `shaders with a binding whose slot was not decoded 588`; shadow cascades read through `array, 32> fp_t2das_cb2_0`. → [An array of handles from the driver table c2](../reference/shaders.md#an-array-of-handles-from-the-driver-table-c2) - characters and props glowing white, about 10⁴ too bright, in DRAGON QUEST VII Reimagined's opening cutscene only, while walls and gameplay look right. → [A texture slot that shares its c2 word with an image slot](../reference/shaders.md#a-texture-slot-that-shares-its-c2-word-with-an-image-slot) - `[RED] shader … reflection conversion failed: 66 bindings, over the limit 64`, then `failed to load`, and the draws with it are dropped (a black map). → [More than 64 bindings in one shader](../reference/shaders.md#more-than-64-bindings-in-one-shader) -- the replay put a texture resource ID where the shader wanted an address; with the old 64 KB dummy buffer, pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced and the game or the replay crashed, or `kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers (GitHub issue #101). → [Image atomics](../reference/shaders.md#image-atomics) +- the replay put a texture resource ID where the shader wanted an address; with the old 64 KB dummy buffer, pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced and the game or the replay crashed, or `kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers (GitHub issue #101). Later, `[RED] image atomics on a block-linear texture … not followed, the atomics go to a scratch buffer`, and compute shaders with atomics on a 3D image did not build (`array subscript is not an integer`). → [Image atomics](../reference/shaders.md#image-atomics) - `image atomics: unsupported or overflowing IMG extent` at the first dispatch, despite a valid four-byte counter buffer. → [Buffer image reflection keeps its texture type](../reference/shaders.md#buffer-image-reflection-keeps-its-texture-type) -- Metal shader validation reports invalid image-atomic stores even with an image-sized scratch buffer; without validation, GPU page faults can freeze rendering. → [Bounded MSL 2.3 atomic emulation](../reference/shaders.md#bounded-msl-23-atomic-emulation) +- Metal shader validation reports invalid image-atomic stores even with an image-sized scratch buffer; without validation, GPU page faults can freeze rendering. → [Bounds of image atomics](../reference/shaders.md#bounds-of-image-atomics) - shaders missing from the map that no game file holds. → [Built-in shaders compiled with the game's own GLSLC](../reference/shaders.md#built-in-shaders-compiled-with-the-games-own-glslc) - a null read at `main+0x3316dc` in `nnMain`, right after the window setup (the program object's interface pointers at +0xd0 / +0xd8 both null). → [GLSLC GPU code version against the driver's range](../reference/shaders.md#glslc-gpu-code-version-against-the-drivers-range) - a self-contained Mac app (`--platform mac-app`) logs `pipeline prebuild: no .../Resources/dm/shader_pairs.tsv, nothing to build` although every shader library is there; it draws no hybrid stand-ins, and each pipeline seen for the first time compiles on the submit thread (frames of 100–550 ms). → [Shader-pair tables in a Mac app](../reference/shaders.md#shader-pair-tables-in-a-mac-app) diff --git a/docs/image-atomic-binding-contract.md b/docs/image-atomic-binding-contract.md deleted file mode 100644 index e26abf3..0000000 --- a/docs/image-atomic-binding-contract.md +++ /dev/null @@ -1,83 +0,0 @@ -# Packed image-atomic binding admission - -MSL 2.3 represents an image atomic with two arguments: the IMG texture supplies -its dimensions, while IMGA is an atomic-word buffer pointer. Capacity and layout -must agree **before** either is used by a draw or dispatch. This host check is -separate from the shader translator's per-coordinate bounds checks: neither -replaces the other. Packed addressing assumes the translator's reviewed default -linear alignment of four bytes per 32-bit word. - -`encode_args_` now resolves each IMGA's unique, same-register IMG once under the -existing registry lock, then uses that exact texture for both arguments. It does -not change argument-cache reuse, queue submission, resource residency, or deferred -resource lifetime. - -## Admitted paths - -- Real images must resolve to a scalar R32 integer texture of the shader's - texture type: buffer, 2D, or 2D array, without multisampling. -- Direct guest-pool writes require an actual non-view buffer texture or base 2D - texture, the matching guest dimensions/format size, one mip, the same Metal - backing buffer and base offset, and enough bytes in both guest storage and the - buffer. For 2D the actual buffer row pitch must equal width times four. A buffer - texture's unused next row may be padded. -- The existing **discard scratch** path for non-linear images is retained, not - made coherent with the image. Its previous source-sized request, 1 MiB rounding, - reuse and deferred replacement policy remain for admitted images. Checked - arithmetic and actual IMG dimensions ensure the existing request covers the - matched image; no request is enlarged to conceal a mismatch. -- An unbound IMG may retain the existing dummy pointer only if its actual dummy - extent fits and neither side resolves to a real image. - -## Fail-closed limits - -Direct padded-row textures, array images, original 1D descriptors (even when -normalized to a one-row Metal 2D texture), mip/layer/format views, private fallback -textures without the expected buffer, unsupported formats/shapes, failed scratch -allocation, and capacity/arithmetic failures stop with a diagnostic before the -argument buffer is bound and before the offending draw/dispatch. This is a -conservative trial compatibility restriction; a safe but unproved direct view is -also rejected. Real linear writes are **never redirected to discard scratch**. -The translator's existing 1D/1D-array normalization is preserved, but that alone -does not prove direct native storage. Non-linear images still follow the -separate matched-extent discard-scratch admission above. Buffer shader routes -remain scalar and are not rewritten by the 2D MSL helper; direct buffer host -admission still requires the actual backing buffer and offset checks. - -The decisive diagnostic suffixes are: - -- `ambiguous IMG partner` / `missing IMG partner` -- `unsupported or overflowing IMG extent` -- `real image is not a resolved R32 integer IMG` -- `unsupported direct IMG/IMGA layout or capacity` -- `scratch allocation or capacity failed; refusing short dummy` -- `unmatched unbound IMG/IMGA pair` - -Each is prefixed by `image atomics:` and includes shader/stage/slot context. -These are fatal errors, not warning suppression or silent skipped draws. The -former scratch-allocation failure branch could return a short 64 KiB dummy while -IMG retained its original extent; it can no longer do so. This source-level -failure path has not been observed in the supplied validation run, which had a -real scratch buffer. This patch does not establish that failure path, direct-pool -layout mismatch, or residency caused an earlier GPU fault. - -## Verification - -``` -make -C dm/gfx/tests GAME=0100F43008C44000 imga-layout -ASAN_OPTIONS=detect_leaks=0 make -C dm/gfx/tests GAME=0100F43008C44000 imga-layout-sanitize -python3 dm/gfx/tests/test_imga_binding_source.py -``` - -The portable fixture executes production checked arithmetic/admission helpers: -3 MiB versus 64 KiB capacity, valid packed layouts, overflow, zero dimensions, -short storage, wrong buffer/offset, view rejection, padding and direct arrays. -The source checks establish bounded wiring invariants, not Metal execution. -The fixture does not execute Metal binding. Native Objective-C compilation, -actual Metal texture-property behavior and game compatibility require separate -Apple validation. No allocator, texture readout, -shader algorithm, or game eligibility changes are included. - -The buffer metadata comes from Apple's read-only [buffer offset](https://developer.apple.com/documentation/metal/mtltexture/bufferoffset?language=objc) -and [row pitch](https://developer.apple.com/documentation/metal/mtltexture/bufferbytesperrow) -properties; row/offset are queried only when a source buffer is present. diff --git a/docs/reference/shaders.md b/docs/reference/shaders.md index bc3114e..9e0fb42 100644 --- a/docs/reference/shaders.md +++ b/docs/reference/shaders.md @@ -244,7 +244,7 @@ Finding, keying and translating shaders (`dm/gfx/tools/shader_split.py`, `dmsh_b - **Record:** a decoded shader is an `LSLG` record: `LSLG`, a stage tag (`SV`, `SP`, `SC`), a header with the source name (`/Engine/Private/…usf:MainPS`), then `u32` code length, the code area, `u32` control length (0x880), the control section and a 24- or 29-byte tail, all at unaligned offsets (`uepak.lslg`). - **Check:** the host keys came out exactly: 37 of 37 shaders a first run reported missing were in the two archives. - **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): 10,324 unique shaders (1,457 vertex, 6,346 pixel, 2,521 compute); vertex and pixel shaders combine per draw, so every vertex shader is paired with the single-stage cover. Bindings: separate textures and samplers, as Engage's (`0x168` / `0x568`); 61 vertex shaders pick a texture slot at run time (profile key `tex_dyn_slots`, [Texture slots picked at run time](#texture-slots-picked-at-run-time)), 2,084 shaders do atomics on buffer images ([Image atomics](#image-atomics)). -- **Note:** 16 compute shaders do atomics on a 3D `r32ui` image, which SPIRV-Cross's MSL 2.3 image-atomic emulation indexes with an `int3` (`array subscript is not an integer`): all are Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run), so they stay unbuilt. +- **Note:** 16 compute shaders do atomics on a 3D `r32ui` image, all Unreal's paged global distance field (`GlobalDistanceFieldCompositeObjects.usf:CompositeObjectsIntoPagesCS`), which the mobile renderer does not dispatch (none in any run). They build since image atomics are native ([Image atomics](#image-atomics)). ### Unreal Engine 4.27 shader code archives (LZ4) @@ -645,84 +645,26 @@ Finding, keying and translating shaders (`dm/gfx/tools/shader_split.py`, `dmsh_b ### Image atomics -- **Symptom:** the replay put a texture resource ID where the shader wanted an address; with the old 64 KB dummy buffer, pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced and the game or the replay crashed, or `kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers (GitHub issue #101). -- **Cause:** SPIRV-Cross's MSL 2.3 emulation declares the image twice: the texture (`texture_buffer cp_i_cb2_4C_r32ui`) and a `device atomic_uint* cp_i_cb2_4C_r32ui_atomic` pointer to the same memory, which the build read as a second image of the same slot. -- **Fix:** the pointer is now its own kind (`NVN_RB_IMGA`), bound to the storage of the image in that slot: a buffer texture is a view of its memory pool, so the atomics and the texture reads see the same words. A block-linear image is a deswizzled Metal copy whose pool memory has another layout, so its atomics go to a scratch buffer (logged red) until a game needs them followed; a 3D image does not compile at all ([Unreal Engine 5 shader code archives (Oodle)](#unreal-engine-5-shader-code-archives-oodle)). The scratch buffer is at least as large as the image (`atom_scratch`): SPIRV-Cross indexes the atomics by the image bound in the same slot (`spvImage2DAtomicCoord`: x + y × its width). -- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): 2,084 such shaders (Unreal's GPU particles); three ran in the first 100 seconds. [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): in the 64 KB dummy buffer used before, its atomic max into a 480x270 r32ui 2D array (fragment shader `2bd5037003980716`, the scene after the story opening's first cutscene) ran about 1.4 MB past its end, writing into zero-copy pools over the game's heap and into unmapped addresses (GitHub issue #101). -- **Note:** Metal shader validation (`MTL_SHADER_VALIDATION=1`) stops out-of-bounds accesses, so under it the scene played. +- **Symptom:** the replay put a texture resource ID where the shader wanted an address; with the old 64 KB dummy buffer, pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced and the game or the replay crashed, or `kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers (GitHub issue #101). Later, `[RED] image atomics on a block-linear texture … not followed, the atomics go to a scratch buffer`, and compute shaders with atomics on a 3D image did not build (`array subscript is not an integer`). +- **Cause:** below MSL 3.1, SPIRV-Cross emulates an image atomic with a raw device pointer beside the texture (`texture_buffer cp_i_cb2_4C_r32ui` and `device atomic_uint* cp_i_cb2_4C_r32ui_atomic`), indexed by the bound image's width (`spvImage2DAtomicCoord`), so the host had to bind memory in that packed layout. A buffer image's pool memory has it. A block-linear image's Metal texture is a deswizzled copy whose pool memory has another layout, so its atomics went to a scratch buffer that nothing read, and a pointer shorter than the image (or an invalid coordinate) wrote over other GPU memory. For a 3D image SPIRV-Cross indexes the pointer with the `int3` coordinate itself. +- **Fix:** image atomics are Metal's native texture atomics (MSL 3.1: `texture3d::atomic_fetch_max`). `dmsh_atomic.py` has SPIRV-Cross write shaders with image atomics (an `OpImageTexelPointer`) at MSL 3.1, and every other shader at MSL 2.3 as before, so their MSL is unchanged and their libraries come from the compile cache; a shader whose MSL still has the pointer form fails the build. The atomics act on the texture every other access of the image uses (a buffer image's view of its pool, a linear image's pool memory, a block-linear image's Metal texture), for buffer, 2D, 2D-array and 3D images. Metal runs texture atomics only on an R32Uint / R32Sint texture made with `MTLTextureUsageShaderAtomic`: NVN images of those formats take that usage when they are made (`nvn_mtl.m` `img_usage`; a view keeps working through its base), the build marks the slots a shader uses atomically (`"atomic"` in the reflection, `nvn_rbind.atom`), and the host binds an atomic-capable write sink, logged red, in place of any other texture there (`dummy_img_atomic`). The translator's coordinate guard stays ([Bounds of image atomics](#bounds-of-image-atomics)). +- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): 2,084 shaders with atomics on buffer images (Unreal's GPU particles; three ran in the first 100 seconds) and 16 on a 3D image. [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): an atomic max into a 480x270 r32ui block-linear 2D array (fragment shader `2bd5037003980716`, the scene after the story opening's first cutscene), which ran about 1.4 MB past the 64 KB dummy buffer into zero-copy pools over the game's heap and into unmapped addresses (GitHub issue #101), then went to a scratch buffer. [FINAL FANTASY RESONANCE](../../dm/games/0100606026F12000/README.md) (8 shaders) and [Ghost Trick](../../dm/games/010029B018432000/README.md) (5, one of them a compare-and-swap): atomics on 3D images. +- **Check:** `dm/tools/tests/test_image_atomic_bounds.py` with the rebuilt translator, `DMSH_TEST_METAL=1 DMSH_TEST_GPU=1`: synthetic Maxwell atomics on buffer, 2D, 2D-array and 3D images, compare-and-swaps among them, run on native atomic textures of a Mac GPU under the Metal validation layer (`t_atomic_return.m`). `t_refl.m` checks the atomic flag of a reflected image. ### Buffer image reflection keeps its texture type - **Symptom:** `image atomics: unsupported or overflowing IMG extent` at the first dispatch, despite a valid four-byte counter buffer. -- **Cause:** `nvn_refl.m` classified `texture_buffer` as the default 2D texture. The atomic admission check correctly rejected the actual Metal buffer texture against that 2D descriptor. -- **Fix:** the shared reflection type decoder recognizes `texture_buffer` as `MTLTextureTypeTextureBuffer`; the matched IMG and IMGA retain their buffer-image shape. `t_refl.m` checks the actual reflection converter with a paired buffer image and atomic pointer. +- **Cause:** `nvn_refl.m` classified `texture_buffer` as the default 2D texture. The atomic admission check of the pointer emulation then rejected the actual Metal buffer texture against that 2D descriptor. +- **Fix:** the shared reflection type decoder recognizes `texture_buffer` as `MTLTextureTypeTextureBuffer`, so a buffer image keeps its buffer-image shape. `t_refl.m` checks the reflection converter with a buffer image. - **Found on:** [SHIN MEGAMI TENSEI V: Vengeance](../../dm/games/010069C01AB82000/README.md), compute shader `2d2093c1f1d3a7e0`, image slot 2 (one R32Uint texel). -### Bounded MSL 2.3 atomic emulation +### Bounds of image atomics - **Symptom:** Metal shader validation reports invalid image-atomic stores even with an image-sized scratch buffer; without validation, GPU page faults can freeze rendering. -- **Cause:** the translator lost the surface instruction's `IGN` bounds mode before lowering an image atomic to a raw buffer pointer. Scratch capacity cannot protect against invalid coordinates; the upstream 2D coordinate helper also omits array layers. -- **Fix:** preserve the supported bounds mode, guard every coordinate before pointer construction, use layer-aware 64-bit addressing, and validate the matching image and buffer's layout and capacity before binding. The supported contract and tests are described below. -- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md#rendering), with the development-build report and remaining gameplay validation recorded there. - -Sizing the backing buffer does not make an out-of-range image coordinate safe. -The offline translator now preserves the Maxwell surface instruction's bounds -mode and whether the result is discarded. Its Metal-only lowering accepts `IGN` -reductions and non-CAS atomics, including returned old-texel values. Predicate-returning, -CAS, `TRAP`, and unknown modes fail translation. This path supports non-indexed 32-bit buffer, 2D and -2D-array images. Other translator callers do not enable it. Source 1D and -1D-array instructions retain the existing one-row normalization to 2D `(x, 0)` -and 2D-array `(x, 0, layer)`; buffer coordinates stay scalar, and buffer MSL is -not rewritten by the 2D helper. Source 64-bit, floating-point and half2 atomics -are rejected before descriptor-format resolution can erase their original size. -Descriptor-sized 32-bit atomics also reject non-R32 integer descriptors before -the legacy format fallback, both for bound images and constant-buffer-resolved -bindless handles. An unresolved bindless atomic fails translation instead of -being removed by bindless elimination. These checks apply only to the Metal -atomic path; other translator backends retain their existing behavior. -Indexed, 3D, multisample and other unsupported effective image shapes fail -closed on this opt-in path; this is not a native-image-atomic implementation. - -Returning `IGN` atomics join the valid operation's old-texel result with zero -from the invalid-coordinate branch using an SPIR-V phi. The pointer and memory -update exist only in the valid branch. This implements image-atomic bounds -semantics ([ARB_shader_image_load_store, issue 30](https://registry.khronos.org/OpenGL/extensions/ARB/ARB_shader_image_load_store.txt)): -invalid coordinates leave memory unchanged and return zero. Found on -[SHIN MEGAMI TENSEI V: Vengeance](../../dm/games/010069C01AB82000/README.md), whose -two buffer-image counter shaders were dispatched at boot but rejected because -they consume their returned values. `test_image_atomic_bounds.py` keeps that -value live through a second atomic, validates the SPIR-V dataflow and both Metal -SDKs, and optionally executes the buffer fixtures on a Mac GPU -(`DMSH_TEST_GPU=1`, `t_atomic_return.m`): valid coordinates return the old value, -negative and edge coordinates return zero, and surrounding sentinels stay intact. - -The SPIR-V queries the bound image's dimensions and tests every original -coordinate (including the layer) before forming the texel pointer or executing -the atomic. Negative coordinates fail an unsigned comparison. Coordinates are -not clamped, flattened for the bounds check, or changed in SPIR-V. -`dmsh_atomic.py` recognizes the exact standalone SPIRV-Cross MSL 2.3 coordinate -helper and replaces only that definition with typed, layer-aware helpers using -64-bit address arithmetic; shader statements and call sites are untouched. -An unrecognized helper or default alignment fails the build. This is a packed -emulation layout, not proof that an arbitrary native pool's row pitch, layer -stride or texture-view offset matches it. The host checks the matched IMG/IMGA -[layout and capacity contract](../image-atomic-binding-contract.md) before -binding; block-linear scratch still discards results and does not implement -image read/write coherence. - -Changing this path invalidates the chain and shader cache. Rebuild the translator -and record its actual fingerprints before preparing shaders again; a host-only -rebuild is insufficient. The synthetic regression is -`dm/tools/tests/test_image_atomic_bounds.py`: portable helper/algebra checks plus -an opt-in rebuilt-translator → SPIR-V validation → MSL 2.3 check. Set -`DMSH_TEST_XLATE` to the rebuilt translator, with `spirv-val` and `spirv-cross` on -`PATH`; `DMSH_TEST_METAL=1` also compiles and links with Apple's Metal compiler. -Set `DMSH_TEST_METAL_SDKS=macosx,iphoneos` to check both SDKs. -The dimension fixture covers both instruction families for source 1D, buffer, -1D array, 2D, 2D array and rejected 3D; accepted forms pass SPIR-V validation, -Cross/helper lowering and the optional Metal checks. Shader acceptance does not -imply admission of every native host layout. None of these tests executes a -shader on a GPU. +- **Cause:** the translator lost the surface instruction's `IGN` bounds mode before lowering an image atomic, so a coordinate outside the image reached memory. Neither a pointer nor a native texture atomic outside its image is defined on Metal. +- **Fix:** the translator (`TranslationFlags.MetalImageAtomics`, Metal only; other backends keep their behaviour) keeps the bounds mode and whether the result is used, queries the bound image's size and tests every original coordinate, the layer included, before the atomic. Negative coordinates fail an unsigned comparison; coordinates are not clamped or flattened. Invalid coordinates leave memory unchanged and return zero ([ARB_shader_image_load_store, issue 30](https://registry.khronos.org/OpenGL/extensions/ARB/ARB_shader_image_load_store.txt)): a returning `IGN` atomic joins the old texel from the valid branch and zero from the other with an SPIR-V phi. A compare-and-swap (`SUATOM.CAS`: compare with Rb, swap in Rb+1) is Metal's texture compare-exchange behind the same guard; SPIRV-Cross loops `atomic_compare_exchange_weak` until it succeeds or the texel differs. Predicate-returning, `TRAP` and unknown modes fail translation, as do 64-bit, floating-point and half2 sources (checked before the descriptor format can erase their size), descriptors that are not R32 integers, bindless handles that do not resolve statically, and indexed or multisample images. Source 1D and 1D-array surfaces keep the translator's one-row normalization to 2D. +- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md#rendering) ([PR #178](https://github.com/seedds/nxnative/pull/178)); [SHIN MEGAMI TENSEI V: Vengeance](../../dm/games/010069C01AB82000/README.md), whose two buffer-image counter shaders dispatched at boot consume their returned values. +- **Check:** `dm/tools/tests/test_image_atomic_bounds.py`. Portable tests check the MSL version choice and the rejection of the pointer form. With `DMSH_TEST_XLATE` set to a rebuilt translator (and `spirv-val`, `spirv-cross` on `PATH`) the synthetic shaders are translated, the guard's dataflow is checked in the SPIR-V (the size query and comparison dominate the atomic, the phi carries zero on the invalid edge), and the MSL is produced; `DMSH_TEST_METAL=1` compiles and links it (`DMSH_TEST_METAL_SDKS=macosx,iphoneos` for both SDKs) and `DMSH_TEST_GPU=1` runs the returning fixtures (above). Changing this path changes the chain version: rebuild the translator and update the lock before building shaders. ## Shaders compiled at run time