Guard Metal image atomics and reject unsupported bindings
Apply the generalized image-atomic bounds and backing checks from PR #178. Also reject unsupported descriptor formats before integer fallback and fail unresolved bindless atomics instead of silently deleting them. Rebuild the translator and record chain 467efdfd0b62, with regression tests and findings. Validation: compiler/helper and sanitizer tests, 96 Metal compile/link checks, ten GPU probes, all 28,603 Z-A shaders for Mac and iPhone, and a 240-second Mac station-area run reaching 6,987 frames without a freeze or GPU fault. Reported iPhone cases and the station-to-world transition remain unverified. Co-authored-by: b4f8abbb24dbf56a7d0280f296e16eeb <51352400+b4f8abbb24dbf56a7d0280f296e16eeb@users.noreply.github.com>
This commit is contained in:
1 parent
4f29560fac
commit
c3a0880304
28 files changed
+939
-48
No files matched your search
@@ -53,6 +53,8 @@ Each item was found from a red line or a pass dump on the way through the langua
|
||||
- **Resolved: characters' legs folded back from the hips once they walked.** From the first step on, the player's legs stuck straight out behind her at hip height, and the rival's forward in a later cutscene; standing still looked right. The bone palette the game computes on the CPU was already bent. The cause was six occlusion queries a frame, one 36-vertex box each (`a793f436dcabd350`, base instance 61, 62, …), drawn against mip level 2 of the depth pyramid: every query came back with 0 samples, and the game took its characters for hidden and posed their legs that way. Forcing every result to visible fixed the legs, and depth testing with ALWAYS did not. The box shader picks its transform with (instance index − base instance) + `c0[0x644]`, where the NVN driver keeps the base instance; the host left `0x644` at 0, so every box was box 0. 793 of the game's vertex shaders read that word. How it was found: the raycasts were logged with scratch observation gates (all correct), then `DM_HLECOUNT` showed nothing walking-specific, then the counter reports.
|
||||
- **Resolved: a min-reduction sampler averaged.** The game builds that depth pyramid with `SamplerBuilderSetReductionFilter 1` (`151607c7a92386cc` samples the level above at an explicit LOD), which the host used to average.
|
||||
- **Resolved: image atomics past their buffer.** From about frame 5300 of the route (the scene after the story opening's first cutscene), the fragment shader `2bd5037003980716` does an atomic max into a 480x270 r32ui 2D array; sent to the 64 KB dummy buffer, the atomics ran about 1.4 MB past it. They wrote into zero-copy pools over the game's heap (pointers in the game's objects and the host's program objects got their upper 4 bytes replaced, and the game or the replay crashed) and into unmapped addresses (`kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers and the picture freezes). An iPhone 17 freezing that way was reported in GitHub issue #101. Under `MTL_SHADER_VALIDATION=1` the scene played. Three runs of the whole route then reached frame 12,300 with no GPU error.
|
||||
- **Current: invalid coordinates can exceed even an image-sized scratch buffer.** [PR #178](https://github.com/seedds/nxnative/pull/178) reports 1,836 invalid stores into a 3 MiB buffer on Z-A 1.0.0 on M1 Max, including offsets around 16 GiB. Its development build reduced those reports to zero, and the tester reported no freeze without shader validation. The shared [bounds and binding fix](../../../docs/reference/shaders.md#bounded-msl-23-atomic-emulation) is applied here; that development result does not establish gameplay or device validation of this revision or of update 2.0.2. The earlier scratch-size fix above addresses a separate insufficient-capacity case.
|
||||
- **Current validation (2026-10-25):** with the bounds fix and strict descriptor-format/bindless-handle checks, chain `467efdfd0b62` rebuilt all 28,603 shaders for Mac and iPhone with zero failures and no undecoded bindings. After rebuilding the host, update 2.0.2 ran for 240.1 seconds / 6,987 frames around the station from an isolated copy of the save, with movement and camera changes, shader validation disabled, and no observed freeze, GPU page fault, missing shader or atomic-binding rejection. The log confirmed the new chain and the 480x270, three-layer image using 3 MiB scratch. This is a Mac smoke test, not confirmation of the reported iPhone/version cases or the station-to-world transition. Artifacts are in the NSP folder's `0100F43008C44000-work/pr178/` (shader libraries and `gameplay/`).
|
||||
|
||||
## Mac routes
|
||||
|
||||
@@ -69,6 +71,7 @@ General host changes first needed here; each applies to every game:
|
||||
- **SDK 19 calls:** csrng bytes, `GetAudioOutBufferCount` from the host queue, NVN state getters, `WaitForAddress` type 3, `OP_FENCE_SYNC`: [Cryptographically random bytes](../../../docs/reference/services.md#cryptographically-random-bytes), [AudioOut buffer count from the host queue](../../../docs/reference/audio.md#audioout-buffer-count-from-the-host-queue), [State read back through NVN getters](../../../docs/reference/nvn-objects.md#state-read-back-through-nvn-getters), [WaitForAddress type 3](../../../docs/reference/threads.md#waitforaddress-type-3-64-bit-wait-if-equal), [Fence syncs recorded in command buffers](../../../docs/reference/nvn-objects.md#fence-syncs-recorded-in-command-buffers).
|
||||
- **Transform feedback** (`tf_layout`, `vs_tf` / `main0_tf`, `TF_SLOT`, `OP_TF_BIND` / `OP_TF_STATE` / `OP_RAST_DISCARD`, `tf_draw`, `pend_add` kind 4), the **feedback encoder ending `g_blit`**, **layered pipelines** (`vs_layer`, `layer_out`), **function constants specialized** (`shader_load`), the **zero cube for a 2D texture** (`tex_fit`), **layered passes** (`g_rp_layered`, `g_pass.layered`), **stencil-mode views** (`tex_by_id`, `X32_Stencil8`; every depth-stencil texture with pixel-format-view usage, `tex_desc`, PR #106, `t_stencil_view`), **the base instance in the driver segment** (`encode_args`, `c0[0x644]`; `nvn_refl.m` treats any declared driver-segment binding as read), **reduction samplers** (`samp_by_id`) and **pool creation retries** (`mtl_pool_add`): [Transform feedback captured by a vertex entry point](../../../docs/reference/rendering.md#transform-feedback-captured-by-a-vertex-entry-point) and the other entries of [rendering.md](../../../docs/reference/rendering.md) that name this game.
|
||||
- **Image atomics into a scratch buffer as large as the image** (`atom_scratch`): [Image atomics](../../../docs/reference/shaders.md#image-atomics).
|
||||
- **Bounds checks before image-atomic pointer construction and matched image/buffer admission:** [Bounded MSL 2.3 atomic emulation](../../../docs/reference/shaders.md#bounded-msl-23-atomic-emulation).
|
||||
|
||||
## Open
|
||||
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
// Generated file (dm/gfx/tools/dmsh_chain.py --header): the B shader library translation chain version this source tree expects (default options), one for every game;
|
||||
// the host loads only <library dir>/b-<V>/. Run dmsh_chain.py --header after changing the chain's tools and commit this file with the change
|
||||
#define DMSH_CHAIN "dec5ee141a9f"
|
||||
#define DMSH_CHAIN "467efdfd0b62"
|
||||
@@ -0,0 +1,33 @@
|
||||
// Admission for the packed 32-bit IMG/IMGA contract, independent of Metal.
|
||||
#pragma once
|
||||
#include <stdint.h>
|
||||
|
||||
static inline int nvn_imga_bytes(uint64_t w, uint64_t h, uint64_t layers, uint64_t *out) {
|
||||
if (!out || !w || !h || !layers || w > UINT32_MAX || h > UINT32_MAX || layers > UINT32_MAX ||
|
||||
w > UINT64_MAX / h || w * h > UINT64_MAX / layers || w * h * layers > UINT64_MAX / 4) return 0;
|
||||
*out = w * h * layers * 4;
|
||||
return 1;
|
||||
}
|
||||
static inline int nvn_imga_capacity(uint64_t need, uint64_t length, uint64_t offset) {
|
||||
return need && offset <= length && need <= length - offset;
|
||||
}
|
||||
static inline int nvn_imga_scratch_length(uint64_t need, uint64_t *out) {
|
||||
const uint64_t mask = (1u << 20) - 1;
|
||||
if (!out || !need || need > UINT64_MAX - mask) return 0;
|
||||
*out = (need + mask) & ~mask;
|
||||
return 1;
|
||||
}
|
||||
typedef struct {
|
||||
uint64_t width, height, layers, row_bytes;
|
||||
uint64_t buffer_length, texture_offset, pointer_offset, guest_bytes;
|
||||
unsigned word_bytes;
|
||||
int buffer_texture, base_view, same_buffer;
|
||||
} nvn_imga_direct_layout;
|
||||
static inline int nvn_imga_direct_ok(const nvn_imga_direct_layout *d) {
|
||||
uint64_t need;
|
||||
if (!d || !d->base_view || !d->same_buffer || d->word_bytes != 4 || d->layers != 1 ||
|
||||
!nvn_imga_bytes(d->width, d->height, d->layers, &need) || d->texture_offset != d->pointer_offset ||
|
||||
d->texture_offset % 4 || !nvn_imga_capacity(need, d->buffer_length, d->texture_offset) || need > d->guest_bytes) return 0;
|
||||
// Buffer textures have no next row: Metal may pad their unused row length.
|
||||
return d->buffer_texture ? d->height == 1 : d->row_bytes == d->width * 4;
|
||||
}
|
||||
+87
-25
@@ -55,6 +55,7 @@ static uint64_t c2_handle(int stage, unsigned b); // the handle c2 holds at by
|
||||
#include <unistd.h>
|
||||
#include <stdatomic.h>
|
||||
#include <errno.h>
|
||||
#include "nvn_imga_layout.h"
|
||||
#include <execinfo.h>
|
||||
#include <os/lock.h>
|
||||
// Function parameters in this file are never retained again in the callee (the caller / global tables already hold them, and lifetime is guaranteed by defer_obj_release): saves a retain / release pair per call on the hot path
|
||||
@@ -4849,24 +4850,47 @@ static id<MTLSamplerState> dummy_samp(void) {
|
||||
static uint64_t dummy_samp_rid(void) { static uint64_t rid; if (!rid) rid = dummy_samp().gpuResourceID._impl; return rid; }
|
||||
static id<MTLBuffer> dummy_buf(void) { static id<MTLBuffer> b; if (!b) { b = [g_dev newBufferWithLength:64 << 10 options:MTLResourceStorageModeShared]; res_add(b); } return b; }
|
||||
static uint64_t dummy_buf_gpu(void) { static uint64_t gpu; if (!gpu) gpu = dummy_buf().gpuAddress; return gpu; }
|
||||
// Image atomics that are not followed into the image's storage (a block-linear image, or storage in no memory pool) go to a scratch buffer at least as large as the image. SPIRV-Cross indexes
|
||||
// them by the image bound in the same slot (spvImage2DAtomicCoord: x + y × its width), so in the 64 KB dummy buffer they ran up to the image's size past its end, into whatever GPU memory followed:
|
||||
// Image atomics on block-linear images retain discard scratch at least as large as the matched IMG extent; linear images require proven direct backing below. SPIRV-Cross indexes
|
||||
// the IMGA pointer using that image's dimensions (including the array layer), so a full image paired with a 64 KB dummy can run past its end into whatever GPU memory followed:
|
||||
// Pokémon Legends: Z-A's atomic max into an r32ui 2D array (texture ID 302, fragment shader 2bd5037003980716) wrote into zero-copy pools over the game's heap, which broke pointers in the game's
|
||||
// objects and in the host's program objects, and into unmapped addresses, a GPU page fault (GitHub issue #101). The results are discarded as before; the buffer only grows
|
||||
static __strong id<MTLBuffer> g_atom_scratch;
|
||||
static id<MTLBuffer> atom_scratch(const tex_t *t) {
|
||||
static id<MTLBuffer> atom_scratch(const tex_t *t, uint64_t image_bytes) {
|
||||
int bw, bh, bpb = nvn_fmt_bpb(t->d.format, &bw, &bh);
|
||||
uint64_t layers = t->d.target == 8 ? 6 : t->d.depth > 1 ? t->d.depth : 1; // 3D, 2D array and cube array: depth (a cube array's counts layer-faces, tex_desc)
|
||||
uint64_t need = (uint64_t)MAX(t->d.width, 1u) * MAX(t->d.height, 1u) * layers * (uint64_t)MAX(bpb, 4);
|
||||
uint64_t layers = t->d.target == 8 ? 6 : t->d.depth > 1 ? t->d.depth : 1;
|
||||
uint64_t need, len;
|
||||
if (bpb != 4 || bw != 1 || bh != 1 ||
|
||||
!nvn_imga_bytes(MAX(t->d.width, 1u), MAX(t->d.height, 1u), layers, &need)) return nil;
|
||||
if (need < t->L.size) need = t->L.size;
|
||||
if (need < image_bytes) return nil; // Never invent a larger image/storage contract.
|
||||
if (g_atom_scratch.length >= need) return g_atom_scratch;
|
||||
uint64_t len = (need + (1u << 20) - 1) & ~(uint64_t)((1u << 20) - 1);
|
||||
if (!nvn_imga_scratch_length(need, &len)) return nil;
|
||||
id<MTLBuffer> b = [g_dev newBufferWithLength:len options:MTLResourceStorageModePrivate];
|
||||
if (!b) { static int once; if (!once++) MLOG("[RED] image atomics: cannot create a %llu MB scratch buffer, they go to the 64 KB dummy buffer", (unsigned long long)(len >> 20)); return nil; }
|
||||
if (!b) return nil; // Caller rejects the binding: a full image must never keep a short dummy pointer.
|
||||
res_add(b); if (g_atom_scratch) defer_obj_release(g_atom_scratch); g_atom_scratch = b;
|
||||
MLOG("image atomics that are not followed: scratch buffer of %llu MB (texture %ux%u, %llu layers)", (unsigned long long)(len >> 20), t->d.width, t->d.height, (unsigned long long)layers);
|
||||
return b;
|
||||
}
|
||||
// Resolve IMG once for both image and atomic pointer. Caller holds g_rmu.
|
||||
static id<MTLTexture> img_binding(int tid, const nvn_rbind *b, uint64_t *rid, int *real) {
|
||||
void *tv = NULL; *rid = tid ? tex_fast_rid(tid, &tv) : 0;
|
||||
id<MTLTexture> t = (__bridge id<MTLTexture>)tv;
|
||||
int swz_only = t && !(t.usage & MTLTextureUsageShaderWrite) && (t.parentTexture.usage & MTLTextureUsageShaderWrite);
|
||||
if (t && !(t.usage & MTLTextureUsageShaderWrite) && !swz_only) { RED_ONCE(32, "texture bound to an image slot lacks the shader-write usage (texture ID %d)", tid); t = nil; }
|
||||
if (t && (swz_only || (b->dpf != NVN_DPF_FLOAT && mtl_pf_dpf(t.pixelFormat) != b->dpf))) {
|
||||
uint64_t r2 = 0; id<MTLTexture> v = img_fit(tid, t, b->dpf, &r2);
|
||||
if (v) { t = v; *rid = r2; } else if (swz_only) t = nil;
|
||||
}
|
||||
*real = t != nil;
|
||||
if (!t) { t = dummy_img(b->dtt, b->dpf); *rid = g_dummy_img_rid[b->dtt & 7][b->dpf & 3]; }
|
||||
return t;
|
||||
}
|
||||
static int imga_extent(id<MTLTexture> t, const nvn_rbind *b, uint64_t *bytes) {
|
||||
if (!t || t.textureType != (MTLTextureType)b->dtt || t.sampleCount > 1) return 0;
|
||||
if (t.textureType != MTLTextureTypeTextureBuffer && t.textureType != MTLTextureType2D && t.textureType != MTLTextureType2DArray) return 0;
|
||||
return nvn_imga_bytes(t.width, t.textureType == MTLTextureTypeTextureBuffer ? 1 : t.height,
|
||||
t.textureType == MTLTextureType2DArray ? t.arrayLength : 1, bytes);
|
||||
}
|
||||
// Snapshot substitution for same-pass feedback sampling (registered by fb_prepare, used by encode_args; see the notes before draw)
|
||||
static uint64_t g_draw_seq, g_fb_seq, g_fb_count; static int g_fb_n, g_fb_rebind;
|
||||
static __unsafe_unretained id<MTLTexture> g_fb_src[4]; static __strong id<MTLTexture> g_fb_use[4]; static uint64_t g_fb_rid[4];
|
||||
@@ -4939,6 +4963,21 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
int bl = passdump() && (bl_all || g_kd_force || bl_pass[nvn_stage] != g_stat_pass);
|
||||
if (bl) { bl_pass[nvn_stage] = g_stat_pass; PD("K %llu st%d key=%s nb=%d", g_stat_pass, nvn_stage, sh->key[0] ? sh->key : sh->builtin ? sh->builtin : "?", sh->rf.nb); }
|
||||
pthread_mutex_lock(&g_rmu); // The texture / sampler registration tables may be being grown or rewritten by the game thread
|
||||
__unsafe_unretained id<MTLTexture> atomic_img[32] = {nil};
|
||||
uint64_t atomic_rid[32] = {0}; int atomic_real[32] = {0};
|
||||
const nvn_rbind *atomic_desc[32] = {0};
|
||||
for (int i = 0; i < sh->rf.nb; i++) if (sh->rf.b[i].kind == NVN_RB_IMGA) {
|
||||
unsigned r = sh->rf.b[i].reg; const nvn_rbind *ib = NULL;
|
||||
for (int j = 0; j < sh->rf.nb; j++) if (sh->rf.b[j].kind == NVN_RB_IMG && sh->rf.b[j].reg == r) {
|
||||
if (ib) MDIE("[RED] image atomics: ambiguous IMG partner (shader %s stage %d slot %u)", sh->key, nvn_stage, r);
|
||||
ib = &sh->rf.b[j];
|
||||
}
|
||||
if (r >= 32 || !ib) MDIE("[RED] image atomics: missing IMG partner (shader %s stage %d slot %u)", sh->key, nvn_stage, r);
|
||||
if (!atomic_desc[r]) {
|
||||
int tid = (int)(S.img[nvn_stage][r] & 0xfffff); atomic_desc[r] = ib;
|
||||
atomic_img[r] = img_binding(tid, ib, &atomic_rid[r], &atomic_real[r]);
|
||||
}
|
||||
}
|
||||
for (int i = 0; i < sh->rf.nb; i++) {
|
||||
const nvn_rbind *b = &sh->rf.b[i];
|
||||
int reg = b->reg, aid = b->aid;
|
||||
@@ -5014,28 +5053,51 @@ static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, i
|
||||
USE(t, MTLResourceUsageRead);
|
||||
} else if (b->kind == NVN_RB_IMG) { // Writable texture: NVN image slot r (BindImages) ↔ D3D u_r
|
||||
int tid = reg < 32 ? (int)(S.img[nvn_stage][reg] & 0xfffff) : 0;
|
||||
void *tv = NULL; uint64_t trid = tid ? tex_fast_rid(tid, &tv) : 0; __unsafe_unretained id<MTLTexture> t = (__bridge id<MTLTexture>)tv;
|
||||
if (passdump()) PD("W %llu st%d reg%d tid=%d %s ser=%llu", g_stat_pass, nvn_stage, reg, tid, t ? "real" : "dummy", t && tid < g_ntreg ? g_treg[tid].t.serial : 0ull);
|
||||
int swz_only = t && !(t.usage & MTLTextureUsageShaderWrite) && (t.parentTexture.usage & MTLTextureUsageShaderWrite); // a swizzled view of an image texture (img_fit)
|
||||
if (t && !(t.usage & MTLTextureUsageShaderWrite) && !swz_only) { RED_ONCE(32, "texture bound to an image slot lacks the shader-write usage (texture ID %d)", tid); t = nil; }
|
||||
if (t && (swz_only || (b->dpf != NVN_DPF_FLOAT && mtl_pf_dpf(t.pixelFormat) != b->dpf))) { uint64_t r2 = 0; id<MTLTexture> v = img_fit(tid, t, b->dpf, &r2); if (v) { t = v; trid = r2; } else if (swz_only) t = nil; } // see img_fit
|
||||
if (!t) { t = dummy_img(b->dtt, b->dpf); trid = g_dummy_img_rid[b->dtt & 7][b->dpf & 3]; } // Unbound: D3D semantics discard the write - write into a dedicated dummy texture so the set that reads 0 is not polluted
|
||||
uint64_t trid; int real;
|
||||
id<MTLTexture> t = reg < 32 && atomic_desc[reg] ? atomic_img[reg] : img_binding(tid, b, &trid, &real);
|
||||
if (reg < 32 && atomic_desc[reg]) { trid = atomic_rid[reg]; real = atomic_real[reg]; }
|
||||
if (passdump()) PD("W %llu st%d reg%d tid=%d %s ser=%llu", g_stat_pass, nvn_stage, reg, tid, real ? "real" : "dummy", real && tid < g_ntreg ? g_treg[tid].t.serial : 0ull);
|
||||
slots[aid] = trid;
|
||||
USE(t, MTLResourceUsageRead | MTLResourceUsageWrite);
|
||||
} else if (b->kind == NVN_RB_IMGA) { // image atomics: the storage of the image in slot r. A buffer texture is a view of its memory pool (tex_mtl), so the texture reads and the atomics see the same words
|
||||
int tid = reg < 32 ? (int)(S.img[nvn_stage][reg] & 0xfffff) : 0;
|
||||
} else if (b->kind == NVN_RB_IMGA) {
|
||||
int tid = (int)(S.img[nvn_stage][reg] & 0xfffff);
|
||||
const tex_t *tt = tid && tid < g_ntreg && g_treg[tid].ok ? &g_treg[tid].t : NULL;
|
||||
void *mb; uint64_t mo, gpu;
|
||||
// A block-linear image is a deswizzled Metal copy whose pool memory has another layout (and is re-uploaded from): its atomics go to a scratch buffer instead (logged, atom_scratch)
|
||||
int linear = tt && (tt->d.target == 10 || (tt->d.flags & (NVN_TEXTURE_FLAGS_LINEAR | NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET)));
|
||||
if (tt && !linear) RED_ONCE(93, "image atomics on a block-linear texture (texture ID %d target %d): not followed, the atomics go to a scratch buffer", tid, (int)tt->d.target);
|
||||
id<MTLBuffer> sb = nil;
|
||||
if (linear && tt->mem && mtl_gpu_addr((uint64_t)tt->mem, tt->L.size ? tt->L.size : 4, &mb, &mo, &gpu)) {
|
||||
slots[aid] = gpu; USE((__bridge id<MTLBuffer>)mb, MTLResourceUsageRead | MTLResourceUsageWrite);
|
||||
id<MTLTexture> image = atomic_img[reg]; uint64_t need;
|
||||
if (!imga_extent(image, atomic_desc[reg], &need))
|
||||
MDIE("[RED] image atomics: unsupported or overflowing IMG extent (shader %s stage %d slot %d texture %d)", sh->key, nvn_stage, reg, tid);
|
||||
if (tt && (!atomic_real[reg] || (image.pixelFormat != MTLPixelFormatR32Uint && image.pixelFormat != MTLPixelFormatR32Sint)))
|
||||
MDIE("[RED] image atomics: real image is not a resolved R32 integer IMG (shader %s stage %d slot %d texture %d)", sh->key, nvn_stage, reg, tid);
|
||||
int linear = tt && (tt->d.target == NVN_TEXTURE_TARGET_BUFFER || (tt->d.flags & (NVN_TEXTURE_FLAGS_LINEAR | NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET)));
|
||||
id<MTLBuffer> storage = nil; uint64_t gpu = 0;
|
||||
if (linear) {
|
||||
void *mb = NULL; uint64_t mo = 0;
|
||||
int found = tt->mem && mtl_gpu_addr((uint64_t)tt->mem, need, &mb, &mo, &gpu);
|
||||
storage = (__bridge id<MTLBuffer>)mb;
|
||||
int bw, bh, bpb = nvn_fmt_bpb(tt->d.format, &bw, &bh);
|
||||
// Only actual base buffer textures / packed base 2D images have a proven pointer layout.
|
||||
nvn_imga_direct_layout d = { .width = image.width, .height = image.height, .layers = image.arrayLength,
|
||||
.row_bytes = image.buffer ? image.bufferBytesPerRow : 0, .buffer_length = storage.length,
|
||||
.texture_offset = image.buffer ? image.bufferOffset : 0, .pointer_offset = mo, .guest_bytes = tt->L.size,
|
||||
.word_bytes = bw == 1 && bh == 1 ? (unsigned)bpb : 0, .buffer_texture = image.textureType == MTLTextureTypeTextureBuffer,
|
||||
.base_view = !image.parentTexture && image.mipmapLevelCount == 1 && tt->d.levels <= 1 && tt->d.samples <= 1 &&
|
||||
image.width == tt->d.width && (image.textureType == MTLTextureTypeTextureBuffer ? tt->d.target == NVN_TEXTURE_TARGET_BUFFER :
|
||||
image.textureType == MTLTextureType2D && tt->d.target == NVN_TEXTURE_TARGET_2D && image.height == tt->d.height),
|
||||
.same_buffer = found && image.buffer == storage };
|
||||
if (!gpu || !nvn_imga_direct_ok(&d))
|
||||
MDIE("[RED] image atomics: unsupported direct IMG/IMGA layout or capacity (shader %s stage %d slot %d texture %d)", sh->key, nvn_stage, reg, tid);
|
||||
static int nlog; if (nlog < 8 && !(sh->logged & 2)) { sh->logged |= 2; nlog++; MLOG("shader %s (stage %d): image atomics on image slot %d, texture ID %d (target %d, %llu bytes)", sh->key, nvn_stage, reg, tid, (int)tt->d.target, (unsigned long long)tt->L.size); }
|
||||
} else if (tt && (sb = atom_scratch(tt))) { slots[aid] = sb.gpuAddress; USE(sb, MTLResourceUsageRead | MTLResourceUsageWrite); }
|
||||
else { slots[aid] = dummy_buf_gpu(); USE(dummy_buf(), MTLResourceUsageRead | MTLResourceUsageWrite); } // unbound: the image slot has the dummy image, so the writes stay in the 64 KB dummy buffer
|
||||
if (bl) PD("B %llu st%d imga reg%d tid=%d %s", g_stat_pass, nvn_stage, reg, tid, tt && tt->mem ? "real" : "dummy");
|
||||
} else if (tt) {
|
||||
RED_ONCE(93, "image atomics on a block-linear texture (texture ID %d target %d): not followed, the atomics go to a scratch buffer", tid, (int)tt->d.target);
|
||||
storage = atom_scratch(tt, need); gpu = storage.gpuAddress;
|
||||
if (!storage || !gpu || !nvn_imga_capacity(need, storage.length, 0))
|
||||
MDIE("[RED] image atomics: scratch allocation or capacity failed; refusing short dummy (shader %s stage %d slot %d texture %d need %llu)", sh->key, nvn_stage, reg, tid, (unsigned long long)need);
|
||||
} else {
|
||||
storage = dummy_buf(); gpu = storage.gpuAddress;
|
||||
if (atomic_real[reg] || !storage || !gpu || !nvn_imga_capacity(need, storage.length, 0))
|
||||
MDIE("[RED] image atomics: unmatched unbound IMG/IMGA pair (shader %s stage %d slot %d)", sh->key, nvn_stage, reg);
|
||||
}
|
||||
slots[aid] = gpu; USE(storage, MTLResourceUsageRead | MTLResourceUsageWrite);
|
||||
if (bl) PD("B %llu st%d imga reg%d tid=%d %s", g_stat_pass, nvn_stage, reg, tid, linear ? "direct" : tt ? "scratch" : "dummy");
|
||||
} else if (b->kind == NVN_RB_TBUF) { // Typed buffer: the buffer texture (target BUFFER) in texture slot r
|
||||
int tid = (int)(tex_slot_h(S.tex[nvn_stage], S.img[nvn_stage], reg) & 0xfffff);
|
||||
void *tv = NULL; uint64_t trid = tid ? tex_fast_rid(tid, &tv) : 0; __unsafe_unretained id<MTLTexture> t = (__bridge id<MTLTexture>)tv;
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
t_imga_layout
|
||||
t_imga_layout_sanitize
|
||||
t_imga_layout.dSYM/
|
||||
t_imga_layout_sanitize.dSYM/
|
||||
t_nvn
|
||||
t_nvn.dSYM/
|
||||
t_builtin
|
||||
|
||||
@@ -8,6 +8,18 @@ endif
|
||||
endif
|
||||
GAME_DIR = ../../games/$(GAME)
|
||||
GAME_H = ../../host/gen/$(GAME)/game_profile.h
|
||||
|
||||
# Portable admission contract only; native Metal binding remains an Apple test.
|
||||
.PHONY: imga-layout imga-layout-sanitize
|
||||
imga-layout: t_imga_layout
|
||||
./t_imga_layout
|
||||
t_imga_layout: t_imga_layout.c ../nvn/nvn_imga_layout.h
|
||||
$(CC) -std=c11 -O2 -g -Wall -Wextra -Werror -o $@ $<
|
||||
imga-layout-sanitize: t_imga_layout_sanitize
|
||||
./t_imga_layout_sanitize
|
||||
t_imga_layout_sanitize: t_imga_layout.c ../nvn/nvn_imga_layout.h
|
||||
$(CC) -std=c11 -O1 -g -Wall -Wextra -Werror -fsanitize=address,undefined -fno-omit-frame-pointer -o $@ $<
|
||||
|
||||
CFLAGS=-O1 -g -Wall -Wno-unused-function -I../nvn/include -I$(dir $(GAME_H))
|
||||
GFX=$(filter-out ../nvn/hle_nvn.c,$(sort $(wildcard ../nvn/*.c ../nvn/*.m ../nvn/*.S ../tex/*.c ../tex/*.m)))
|
||||
GFX_H=$(wildcard ../nvn/*.h ../nvn/*.inc ../nvn/include/*.h ../tex/*.h) $(GAME_H)
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
#include <assert.h>
|
||||
#include <stdio.h>
|
||||
#include "../nvn/nvn_imga_layout.h"
|
||||
|
||||
static nvn_imga_direct_layout packed(void) {
|
||||
return (nvn_imga_direct_layout){ .width=512, .height=256, .layers=1, .row_bytes=2048,
|
||||
.buffer_length=1048576, .texture_offset=4096, .pointer_offset=4096, .guest_bytes=524288,
|
||||
.word_bytes=4, .base_view=1, .same_buffer=1 };
|
||||
}
|
||||
int main(void) {
|
||||
uint64_t n=77, rounded=88;
|
||||
assert(nvn_imga_bytes(512,512,3,&n) && n==3u*1024*1024);
|
||||
assert(!nvn_imga_capacity(n,64u*1024,0)); // The former allocation-failure fallback is unsafe.
|
||||
assert(nvn_imga_capacity(n,n,0) && !nvn_imga_capacity(n,n-1,0));
|
||||
assert(nvn_imga_scratch_length(n,&rounded) && rounded==n);
|
||||
assert(nvn_imga_scratch_length(n+1,&rounded) && rounded==4u*1024*1024);
|
||||
assert(nvn_imga_bytes(480,270,1,&n) && n==518400);
|
||||
n=77; rounded=88;
|
||||
assert(!nvn_imga_bytes(0,1,1,&n) && n==77);
|
||||
assert(!nvn_imga_bytes(1,0,1,&n) && n==77);
|
||||
assert(!nvn_imga_bytes(1,1,0,&n) && n==77);
|
||||
assert(!nvn_imga_bytes(UINT32_MAX,UINT32_MAX,2,&n) && n==77);
|
||||
assert(!nvn_imga_bytes(UINT32_MAX,UINT32_MAX,1,&n) && n==77);
|
||||
assert(!nvn_imga_bytes(UINT64_MAX,1,1,&n) && n==77);
|
||||
assert(!nvn_imga_bytes(1,1,1,NULL));
|
||||
assert(!nvn_imga_scratch_length(UINT64_MAX,&rounded) && rounded==88);
|
||||
assert(!nvn_imga_scratch_length(0,&rounded) && rounded==88);
|
||||
assert(!nvn_imga_capacity(4,UINT64_MAX,UINT64_MAX-2));
|
||||
assert(!nvn_imga_capacity(1,0,UINT64_MAX));
|
||||
assert(!nvn_imga_capacity(0,64u*1024,0));
|
||||
nvn_imga_direct_layout d=packed(); assert(nvn_imga_direct_ok(&d));
|
||||
#define REJECT(field, value) do { d=packed(); d.field=(value); assert(!nvn_imga_direct_ok(&d)); } while (0)
|
||||
REJECT(base_view,0); REJECT(same_buffer,0); REJECT(word_bytes,2); REJECT(word_bytes,8);
|
||||
REJECT(layers,2); REJECT(width,0); REJECT(height,0); REJECT(width,UINT64_MAX);
|
||||
REJECT(row_bytes,2048+256); REJECT(row_bytes,2048-4); // Padding and truncated/explicit stride.
|
||||
REJECT(pointer_offset,8192); REJECT(texture_offset,4097);
|
||||
REJECT(buffer_length,4096+524287); REJECT(guest_bytes,524287);
|
||||
REJECT(buffer_texture,1); // Multi-row direct buffer images are not a packed buffer view.
|
||||
#undef REJECT
|
||||
d=packed(); d.buffer_length=d.texture_offset+d.guest_bytes; assert(nvn_imga_direct_ok(&d));
|
||||
d=packed(); d.texture_offset=d.pointer_offset=UINT64_MAX-3; assert(!nvn_imga_direct_ok(&d));
|
||||
d=packed(); d.buffer_texture=1; d.height=1; d.row_bytes=4096; d.guest_bytes=2048;
|
||||
assert(nvn_imga_direct_ok(&d)); // The unused next row of a texture buffer may be padded.
|
||||
d.guest_bytes--; assert(!nvn_imga_direct_ok(&d));
|
||||
puts("image atomic layout/capacity admission: PASS");
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
"""Narrow production wiring checks, not a native Metal execution fixture."""
|
||||
from pathlib import Path
|
||||
import unittest
|
||||
|
||||
SOURCE = (Path(__file__).resolve().parents[1] / "nvn/nvn_mtl.m").read_text()
|
||||
|
||||
class BindingWiring(unittest.TestCase):
|
||||
def test_failed_scratch_never_falls_back_to_dummy(self):
|
||||
branch = SOURCE.split('storage = atom_scratch(tt, need);', 1)[1].split('} else {', 1)[0]
|
||||
self.assertIn('!storage || !gpu || !nvn_imga_capacity', branch)
|
||||
self.assertIn('MDIE(', branch)
|
||||
self.assertIn('refusing short dummy', branch)
|
||||
self.assertNotIn('dummy_buf', branch)
|
||||
|
||||
def test_actual_image_is_shared_by_pair(self):
|
||||
self.assertIn('atomic_img[r] = img_binding(tid, ib,', SOURCE)
|
||||
self.assertIn('atomic_desc[reg] ? atomic_img[reg]', SOURCE)
|
||||
self.assertIn('id<MTLTexture> image = atomic_img[reg]', SOURCE)
|
||||
self.assertIn('missing IMG partner', SOURCE)
|
||||
self.assertIn('ambiguous IMG partner', SOURCE)
|
||||
|
||||
def test_direct_path_checks_actual_backing(self):
|
||||
branch = SOURCE.split('if (linear) {', 1)[1].split('} else if (tt) {', 1)[0]
|
||||
for token in ('image.buffer == storage', 'image.bufferOffset', 'image.bufferBytesPerRow',
|
||||
'!image.parentTexture', 'nvn_imga_direct_ok(&d)', 'MDIE('):
|
||||
self.assertIn(token, branch)
|
||||
self.assertNotIn('atom_scratch', branch)
|
||||
self.assertNotIn('dummy_buf', branch)
|
||||
|
||||
def test_rejection_precedes_argument_publication(self):
|
||||
body = SOURCE.split('static int encode_args_(id<MTLCommandEncoder> en, NvShader *sh, int nvn_stage, int metal_vertex) {', 1)[1]
|
||||
self.assertLess(body.index('refusing short dummy'), body.index('memcpy((uint8_t *)ab.contents'))
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,57 @@
|
||||
"""Source-owned MSL 2.3 atomic coordinate helper, after exact ABI validation.
|
||||
|
||||
IR guards each original coordinate before ImageTexelPointer and remains valid
|
||||
SPIR-V. Only the known standalone compiler helper is replaced: shader calls,
|
||||
control flow and expressions are never matched or rewritten. Unknown backends
|
||||
fail closed rather than receiving a heuristic source transformation.
|
||||
"""
|
||||
import re
|
||||
|
||||
# The recognized upstream macro is the SPIRV-Cross emulation ABI (KhronosGroup,
|
||||
# Apache-2.0); the typed, layer-aware implementation below is source-owned.
|
||||
|
||||
_XY_HELPER = '''#define spvImage2DAtomicCoord(tc, tex) (((((tex).get_width() + spvLinearTextureAlignment / 4 - 1) & ~(spvLinearTextureAlignment / 4 - 1)) * (tc).y) + (tc).x)'''
|
||||
_DEFAULT_ALIGNMENT = '''constant uint spvLinearTextureAlignment = is_function_constant_defined(spvLinearTextureAlignmentOverride) ? spvLinearTextureAlignmentOverride : 4;'''
|
||||
# Explicit overloads preserve the original image-coordinate domain. uint casts
|
||||
# are safe only because the IR guard dominates the atomic; use ulong for the
|
||||
# address arithmetic so separate array layers cannot wrap at 32-bit offsets.
|
||||
HELPERS = '''template<typename T, access A>
|
||||
inline ulong spvImage2DAtomicCoord(int2 tc, texture2d<T, A> tex)
|
||||
{
|
||||
ulong alignment = ulong(spvLinearTextureAlignment) / 4;
|
||||
ulong stride = (ulong(tex.get_width()) + alignment - 1) & ~(alignment - 1);
|
||||
return stride * ulong(uint(tc.y)) + ulong(uint(tc.x));
|
||||
}
|
||||
template<typename T, access A>
|
||||
inline ulong spvImage2DAtomicCoord(int3 tc, texture2d_array<T, A> tex)
|
||||
{
|
||||
ulong alignment = ulong(spvLinearTextureAlignment) / 4;
|
||||
ulong stride = (ulong(tex.get_width()) + alignment - 1) & ~(alignment - 1);
|
||||
ulong row = ulong(uint(tc.z)) * ulong(tex.get_height()) + ulong(uint(tc.y));
|
||||
return stride * row + ulong(uint(tc.x));
|
||||
}'''
|
||||
|
||||
|
||||
def _helper_line(line):
|
||||
return re.match(r'^\s*#\s*define\s+spvImage2DAtomicCoord\b', line) is not None
|
||||
|
||||
|
||||
def validate_msl(source):
|
||||
"""Validate upstream emulation ABI without modifying it."""
|
||||
if 'spvImage2DAtomicCoord' not in source:
|
||||
return source # buffer-image atomics have no 2D helper
|
||||
compact = lambda s: re.sub(r'\s+', '', s)
|
||||
helpers = [line for line in source.splitlines() if _helper_line(line)]
|
||||
if len(helpers) != 1 or compact(helpers[0]) != compact(_XY_HELPER):
|
||||
raise ValueError('image atomic emulation: unsupported SPIRV-Cross coordinate helper; rebuild with the reviewed MSL 2.3 compiler')
|
||||
if compact(_DEFAULT_ALIGNMENT) not in compact(source):
|
||||
raise ValueError('image atomic emulation: expected four-byte default linear-texture alignment')
|
||||
return source
|
||||
|
||||
|
||||
def lower_msl(source):
|
||||
validate_msl(source)
|
||||
# The exact definition was checked above. No shader-specific source enters
|
||||
# this module and no expression/call-site rewrite is permitted here.
|
||||
return '\n'.join(HELPERS if _helper_line(line) else line
|
||||
for line in source.split('\n'))
|
||||
@@ -9,7 +9,7 @@
|
||||
# Environment IOSOUT=dir: the same MSL also goes through dm/ios/tools/ios_metal.compile to produce an iphoneos metallib + reflection (chain_metal records the iOS compile convention)
|
||||
# Final summary: success / failure counts and the distribution of reasons; distribution of driver segment read offsets (those outside the range the replay layer can fill are listed separately)
|
||||
import ast, inspect, os, re, sys, json, struct, hashlib, shutil, subprocess, tempfile, collections, threading, concurrent.futures as cf
|
||||
import dmsh_geometry
|
||||
import dmsh_geometry, dmsh_atomic
|
||||
DM_ROOT = os.environ.get('DM_ROOT') or os.path.expanduser('~/dm-work') # Workspace root (game files, build outputs and run artifacts are all under it)
|
||||
HOME = os.path.expanduser('~')
|
||||
dm_dir, out = sys.argv[1:3]; want = set(sys.argv[3:]); cache_root = out
|
||||
@@ -525,7 +525,7 @@ def unfillable(dr):
|
||||
|
||||
RULES = [] # MSL rewrite rules attached by shader name (function, applicability test); the former dark-range remap toe has been removed (back to the original Switch brightness)
|
||||
# The first-stage code fingerprint does not include dmsh_fetch.py; readable-tile variants get a separate second-stage cache keyed on the generated MSL.
|
||||
CODE_FP = hashlib.sha256(source_without_rule_bodies().encode() + b''.join(b'\0' + open(m.__file__, 'rb').read() for m in (dmsh_chain, dmsh_geometry) + ((ios_metal,) if IOSOUT else ()))).hexdigest()
|
||||
CODE_FP = hashlib.sha256(source_without_rule_bodies().encode() + b''.join(b'\0' + open(m.__file__, 'rb').read() for m in (dmsh_chain, dmsh_geometry, dmsh_atomic) + ((ios_metal,) if IOSOUT else ()))).hexdigest()
|
||||
TOOL_FP = tool_fingerprint().encode()
|
||||
|
||||
_CAST = ('as_type<int>(', 'as_type<uint>(')
|
||||
@@ -764,7 +764,7 @@ def geometry_finish(key, w, j, c1):
|
||||
dmsh_geometry.flatten_sets(spv)
|
||||
r = subprocess.run(['spirv-cross', *XC, spv], capture_output=True, text=True)
|
||||
if r.returncode: return 'geometry MSL failed', r.stderr.strip()[:160], None
|
||||
src = dmsh_geometry.topology_pointer(precise(r.stdout))
|
||||
src = dmsh_geometry.topology_pointer(precise(dmsh_atomic.lower_msl(r.stdout)))
|
||||
res = json.load(open(os.path.join(w, k + '.res.json')))
|
||||
refl = {'key': key + '_bg' + str(t), 'route': 'B', 'chain': CHAIN, 'dm_ctl': key,
|
||||
'source': f'{loc[key][0]}#{loc[key][1]}', 'pc_paired': pc_of[key], 'stage': 'gs',
|
||||
@@ -787,7 +787,7 @@ def geometry_finish(key, w, j, c1):
|
||||
spv = os.path.join(w, k + '.reader.spv'); dmsh_geometry.flatten_sets(spv)
|
||||
r = subprocess.run(['spirv-cross', *XC, spv], capture_output=True, text=True)
|
||||
if r.returncode: return 'geometry reader failed', r.stderr.strip()[:160], None
|
||||
reader = vs_layer(vs_init(precise(r.stdout)))
|
||||
reader = vs_layer(vs_init(precise(dmsh_atomic.lower_msl(r.stdout))))
|
||||
analysis, full = vs_depth(reader)
|
||||
meta = {'key': key + '_bgv', 'route': 'B', 'chain': CHAIN, 'dm_ctl': key,
|
||||
'stage': 'vs', 'entry': 'main0', 'entry_z': 'main0_z', 'msl_entry_kind': 'vertex',
|
||||
@@ -823,6 +823,8 @@ def finish(key, w, j, c1):
|
||||
r = subprocess.run(['spirv-cross', *XC, os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True)
|
||||
if r.returncode: return 'MSL failed', r.stderr.strip()[:160], None
|
||||
src = open(msl).read()
|
||||
try: src = dmsh_atomic.lower_msl(src)
|
||||
except ValueError as e: return 'image atomic MSL failed', str(e), None
|
||||
src = precise(src); open(msl, 'w').write(src) # min / max / clamp with Maxwell's NaN results (see precise)
|
||||
pad = []
|
||||
if j['stage'] == 'vs': # outputs start as (0,0,0,1) (see vs_init); no point size beside a layer output (see vs_layer)
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# Translation chain tool fingerprints (written by dmsh_chain.py --update-lock; regenerate after changing tools and commit it)
|
||||
dmsh_xlate.dll 08d31903d666ea0c
|
||||
Ryujinx.Graphics.Shader.dll eb315f0da558ca5a
|
||||
dmsh_xlate.dll 0d9b8700c85c7194
|
||||
Ryujinx.Graphics.Shader.dll eff19af935aefcdb
|
||||
Ryujinx.Common.dll b99db57e72c1fa16
|
||||
xlate_source 16bbef6c762573ee
|
||||
xlate_source 729b1339ce845abb
|
||||
spirv-cross d9dc197f061b42a5
|
||||
metal_macosx Apple metal version 32023.921 (metalfe-32023.921.6)
|
||||
metal_iphoneos Apple metal version 32023.921 (metalfe-32023.921.6)
|
||||
@@ -15,7 +15,14 @@
|
||||
import os, sys, hashlib, subprocess
|
||||
DM_ROOT = os.environ.get('DM_ROOT') or os.path.expanduser('~/dm-work') # Workspace root (game files, build outputs and run artifacts are all under it)
|
||||
HERE = os.path.dirname(os.path.abspath(__file__)); GFX = os.path.dirname(HERE); HOME = os.path.expanduser('~')
|
||||
FILES = ['tools/dmsh_chain.py', 'tools/dmsh_build.py', 'tools/dmsh_fetch.py', 'tools/dmsh_vin.py', 'tools/dmsh_geometry.py', 'tools/dmsh_facts.py', 'tools/dmsh_xlate/Program.cs', 'tools/dmsh_xlate/dmsh_xlate.csproj', '../ios/tools/ios_metal.py', 'tools/dmsh_chain.lock']
|
||||
FILES = ['tools/dmsh_chain.py', 'tools/dmsh_build.py', 'tools/dmsh_atomic.py', 'tools/dmsh_fetch.py', 'tools/dmsh_vin.py', 'tools/dmsh_geometry.py', 'tools/dmsh_facts.py', 'tools/dmsh_xlate/Program.cs', 'tools/dmsh_xlate/dmsh_xlate.csproj', '../ios/tools/ios_metal.py', 'tools/dmsh_chain.lock']
|
||||
# This target-specific lowering lives in the vendored translator. Include its
|
||||
# sources as well as the rebuilt DLL fingerprint; no old shader cache can bypass it.
|
||||
FILES += ['../third_party/ryujinx/src/Ryujinx.Graphics.Shader/' + f for f in (
|
||||
'Instructions/InstEmitSurface.cs', 'IntermediateRepresentation/TextureFlags.cs',
|
||||
'Translation/TranslationFlags.cs', 'Translation/HostCapabilities.cs',
|
||||
'Translation/TranslatorContext.cs', 'CodeGen/Spirv/Instructions.cs',
|
||||
'StructuredIr/ShaderProperties.cs', 'Translation/Optimizations/BindlessElimination.cs')]
|
||||
INPUT_TABLES = ['shader_facts.tsv', 'shader_map.tsv', 'shader_map.tsv.conflicts.tsv', 'shader_pairs.tsv', 'bf_want.tsv'] # in the game profile folder
|
||||
LOCK = os.path.join(HERE, 'dmsh_chain.lock'); HEADER = os.path.join(GFX, 'nvn/dmsh_chain.h')
|
||||
STAMP = 'inputs.tsv' # in the library directory b-<V>/
|
||||
|
||||
@@ -156,7 +156,7 @@ static class SwshXlate
|
||||
byte[] code = File.ReadAllBytes((string)job["code"]);
|
||||
var acc = new Accessor(code, job);
|
||||
// Job debug = false: turn off DebugMode (DebugMode skips the AST optimizer and the output is full of temporaries; semantics unchanged)
|
||||
var flags = (job["debug"] is JsonNode dbg && !(bool)dbg ? 0 : TranslationFlags.DebugMode) | ((string)job["stage"] == "cs" ? TranslationFlags.Compute : 0);
|
||||
var flags = TranslationFlags.Msl23ImageAtomics | (job["debug"] is JsonNode dbg && !(bool)dbg ? 0 : TranslationFlags.DebugMode) | ((string)job["stage"] == "cs" ? TranslationFlags.Compute : 0);
|
||||
var ctx = Translator.CreateContext(0, acc, new TranslationOptions(TargetLanguage.Spirv, TargetApi.Vulkan, flags));
|
||||
bool geometry = job["geometry_topology"] != null;
|
||||
ShaderProgram p = ctx.Translate(geometry);
|
||||
|
||||
Binary file not shown.
Binary file not shown.
+50
@@ -622,6 +622,20 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
|
||||
ImageDeclaration declaration = context.Images[texOp.GetTextureSetAndBinding()];
|
||||
SpvInstruction image = declaration.Image;
|
||||
|
||||
bool bounded = context.HostCapabilities.Msl23ImageAtomics;
|
||||
if (bounded && ((texOp.Flags & TextureFlags.AtomicBoundsMask) != 0 ||
|
||||
!texOp.Flags.HasFlag(TextureFlags.AtomicDiscardResult)))
|
||||
{
|
||||
throw new NotSupportedException("MSL 2.3 image atomics require IGN with a discarded result; TRAP/returning semantics are not implemented.");
|
||||
}
|
||||
if (bounded && (declaration.IsIndexed ||
|
||||
(texOp.Type != SamplerType.TextureBuffer && texOp.Type != SamplerType.Texture2D &&
|
||||
texOp.Type != (SamplerType.Texture2D | SamplerType.Array)) ||
|
||||
(texOp.Format != TextureFormat.R32Uint && texOp.Format != TextureFormat.R32Sint)))
|
||||
{
|
||||
throw new NotSupportedException("MSL 2.3 bounded image atomics support non-indexed 32-bit buffer/2D/2D-array images only.");
|
||||
}
|
||||
|
||||
SpvInstruction resultType = context.GetType(componentType);
|
||||
SpvInstruction imagePointerType = context.TypePointer(StorageClass.Image, resultType);
|
||||
|
||||
@@ -658,6 +672,34 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
|
||||
|
||||
SpvInstruction value = Src(componentType);
|
||||
|
||||
SpvInstruction merge = null;
|
||||
if (bounded)
|
||||
{
|
||||
// Test each original coordinate, including the array layer, before
|
||||
// constructing a raw pointer. Unsigned comparison also rejects
|
||||
// negative signed coordinates; a final linear-index check would not.
|
||||
SpvInstruction uintType = context.TypeU32();
|
||||
SpvInstruction coordType = pCount == 1 ? uintType : context.TypeVector(uintType, pCount);
|
||||
SpvInstruction boolType = context.TypeBool();
|
||||
SpvInstruction compareType = pCount == 1 ? boolType : context.TypeVector(boolType, pCount);
|
||||
SpvInstruction coords = context.Bitcast(coordType, pCoords);
|
||||
SpvInstruction loaded = context.Load(declaration.ImageType, image);
|
||||
SpvInstruction extent = context.ImageQuerySize(coordType, loaded);
|
||||
SpvInstruction inside = context.ULessThan(compareType, coords, extent);
|
||||
if (pCount != 1)
|
||||
{
|
||||
inside = context.All(boolType, inside);
|
||||
}
|
||||
|
||||
SpvInstruction valid = context.Label();
|
||||
merge = context.Label();
|
||||
context.SelectionMerge(merge, SelectionControlMask.MaskNone);
|
||||
context.BranchConditional(inside, valid, merge);
|
||||
context.AddLabel(valid);
|
||||
|
||||
|
||||
}
|
||||
|
||||
SpvInstruction pointer = context.ImageTexelPointer(imagePointerType, image, pCoords, context.Constant(context.TypeU32(), 0));
|
||||
SpvInstruction one = context.Constant(context.TypeU32(), 1);
|
||||
SpvInstruction zero = context.Constant(context.TypeU32(), 0);
|
||||
@@ -681,6 +723,14 @@ namespace Ryujinx.Graphics.Shader.CodeGen.Spirv
|
||||
_ => context.AtomicIAdd(resultType, pointer, one, zero, value),
|
||||
};
|
||||
|
||||
if (bounded)
|
||||
{
|
||||
context.Branch(merge);
|
||||
context.AddLabel(merge);
|
||||
// The source instruction discards this result. This constant is
|
||||
// IR plumbing, not an invented OOB return value for SUATOM.
|
||||
result = context.Constant(resultType, 0);
|
||||
}
|
||||
return new OperationResult(componentType, result);
|
||||
}
|
||||
|
||||
|
||||
+48
-8
@@ -19,6 +19,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
EmitSuatom(
|
||||
context,
|
||||
op.Dim,
|
||||
op.Clamp,
|
||||
op.Op,
|
||||
op.Size,
|
||||
0,
|
||||
@@ -38,6 +39,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
EmitSuatom(
|
||||
context,
|
||||
op.Dim,
|
||||
op.Clamp,
|
||||
op.Op,
|
||||
op.Size,
|
||||
op.TidB,
|
||||
@@ -57,6 +59,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
EmitSuatom(
|
||||
context,
|
||||
op.Dim,
|
||||
op.Clamp,
|
||||
op.Op,
|
||||
op.Size,
|
||||
0,
|
||||
@@ -66,7 +69,8 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
op.Dest,
|
||||
op.Ba,
|
||||
isBindless: true,
|
||||
compareAndSwap: false);
|
||||
compareAndSwap: false,
|
||||
hasPredicateResult: op.DestPred != RegisterConsts.PredicateTrueIndex);
|
||||
}
|
||||
|
||||
public static void SuatomCasB(EmitterContext context)
|
||||
@@ -76,6 +80,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
EmitSuatom(
|
||||
context,
|
||||
op.Dim,
|
||||
op.Clamp,
|
||||
0,
|
||||
op.Size,
|
||||
0,
|
||||
@@ -95,6 +100,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
EmitSuatom(
|
||||
context,
|
||||
op.Dim,
|
||||
op.Clamp,
|
||||
0,
|
||||
op.Size,
|
||||
op.TidB,
|
||||
@@ -139,14 +145,14 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
{
|
||||
InstSuredB op = context.GetOp<InstSuredB>();
|
||||
|
||||
EmitSured(context, op.Dim, op.Op, op.Size, 0, op.SrcA, op.Dest, op.SrcC, op.Ba, isBindless: true);
|
||||
EmitSured(context, op.Dim, op.Clamp, op.Op, op.Size, 0, op.SrcA, op.Dest, op.SrcC, op.Ba, isBindless: true);
|
||||
}
|
||||
|
||||
public static void Sured(EmitterContext context)
|
||||
{
|
||||
InstSured op = context.GetOp<InstSured>();
|
||||
|
||||
EmitSured(context, op.Dim, op.Op, op.Size, op.TidB, op.SrcA, op.Dest, 0, op.Ba, isBindless: false);
|
||||
EmitSured(context, op.Dim, op.Clamp, op.Op, op.Size, op.TidB, op.SrcA, op.Dest, 0, op.Ba, isBindless: false);
|
||||
}
|
||||
|
||||
public static void SustDB(EmitterContext context)
|
||||
@@ -180,6 +186,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
private static void EmitSuatom(
|
||||
EmitterContext context,
|
||||
SuDim dimensions,
|
||||
Clamp clamp,
|
||||
SuatomOp atomicOp,
|
||||
SuatomSize size,
|
||||
int imm,
|
||||
@@ -189,8 +196,18 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
int dest,
|
||||
bool byteAddress,
|
||||
bool isBindless,
|
||||
bool compareAndSwap)
|
||||
bool compareAndSwap,
|
||||
bool hasPredicateResult = false)
|
||||
{
|
||||
// Preserve source width before descriptor-sized formats are resolved.
|
||||
// Other backends retain their existing surface-instruction handling.
|
||||
bool msl23 = context.TranslatorContext.Options.Flags.HasFlag(TranslationFlags.Msl23ImageAtomics);
|
||||
if (msl23 &&
|
||||
size is not (SuatomSize.U32 or SuatomSize.S32 or SuatomSize.Sd32))
|
||||
{
|
||||
throw new NotSupportedException("MSL 2.3 image atomics require a 32-bit integer source size.");
|
||||
}
|
||||
|
||||
SamplerType type = ConvertSamplerType(dimensions);
|
||||
|
||||
if (type == SamplerType.None)
|
||||
@@ -259,7 +276,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
|
||||
// TODO: FP and 64-bit formats.
|
||||
TextureFormat format = size is SuatomSize.Sd32 or SuatomSize.Sd64
|
||||
? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm))
|
||||
? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm, strict: msl23))
|
||||
: GetTextureFormat(size);
|
||||
|
||||
if (compareAndSwap)
|
||||
@@ -271,7 +288,15 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
|
||||
Operand[] sources = sourcesList.ToArray();
|
||||
|
||||
TextureFlags flags = compareAndSwap ? TextureFlags.CAS : GetAtomicOpFlags(atomicOp);
|
||||
TextureFlags flags = (compareAndSwap ? TextureFlags.CAS : GetAtomicOpFlags(atomicOp)) | (TextureFlags)((int)clamp << 20);
|
||||
if (msl23)
|
||||
{
|
||||
flags |= TextureFlags.AtomicMsl23;
|
||||
}
|
||||
if (dest == RegisterConsts.RegisterZeroIndex && !compareAndSwap && !hasPredicateResult)
|
||||
{
|
||||
flags |= TextureFlags.AtomicDiscardResult;
|
||||
}
|
||||
|
||||
if (isBindless)
|
||||
{
|
||||
@@ -463,6 +488,7 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
private static void EmitSured(
|
||||
EmitterContext context,
|
||||
SuDim dimensions,
|
||||
Clamp clamp,
|
||||
RedOp atomicOp,
|
||||
SuatomSize size,
|
||||
int imm,
|
||||
@@ -472,6 +498,15 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
bool byteAddress,
|
||||
bool isBindless)
|
||||
{
|
||||
// Preserve source width before descriptor-sized formats are resolved.
|
||||
// Other backends retain their existing surface-instruction handling.
|
||||
bool msl23 = context.TranslatorContext.Options.Flags.HasFlag(TranslationFlags.Msl23ImageAtomics);
|
||||
if (msl23 &&
|
||||
size is not (SuatomSize.U32 or SuatomSize.S32 or SuatomSize.Sd32))
|
||||
{
|
||||
throw new NotSupportedException("MSL 2.3 image atomics require a 32-bit integer source size.");
|
||||
}
|
||||
|
||||
SamplerType type = ConvertSamplerType(dimensions);
|
||||
|
||||
if (type == SamplerType.None)
|
||||
@@ -538,14 +573,19 @@ namespace Ryujinx.Graphics.Shader.Instructions
|
||||
|
||||
// TODO: FP and 64-bit formats.
|
||||
TextureFormat format = size is SuatomSize.Sd32 or SuatomSize.Sd64
|
||||
? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm))
|
||||
? (isBindless ? TextureFormat.Unknown : ShaderProperties.GetTextureFormatAtomic(context.TranslatorContext.GpuAccessor, imm, strict: msl23))
|
||||
: GetTextureFormat(size);
|
||||
|
||||
sourcesList.Add(Rb());
|
||||
|
||||
Operand[] sources = sourcesList.ToArray();
|
||||
|
||||
TextureFlags flags = GetAtomicOpFlags((SuatomOp)atomicOp);
|
||||
TextureFlags flags = GetAtomicOpFlags((SuatomOp)atomicOp) |
|
||||
(TextureFlags)((int)clamp << 20) | TextureFlags.AtomicDiscardResult;
|
||||
if (msl23)
|
||||
{
|
||||
flags |= TextureFlags.AtomicMsl23;
|
||||
}
|
||||
|
||||
if (isBindless)
|
||||
{
|
||||
|
||||
Vendored
+6
@@ -16,6 +16,12 @@ namespace Ryujinx.Graphics.Shader.IntermediateRepresentation
|
||||
Offsets = 1 << 7,
|
||||
Coherent = 1 << 8,
|
||||
|
||||
// Preserve the surface instruction's OOB policy through IR. IGN drops
|
||||
// the operation; TRAP/unknown must not silently become IGN on Metal.
|
||||
AtomicBoundsMask = 3 << 20,
|
||||
AtomicDiscardResult = 1 << 22,
|
||||
AtomicMsl23 = 1 << 23,
|
||||
|
||||
AtomicMask = 15 << 16,
|
||||
|
||||
Add = 0 << 16,
|
||||
|
||||
+7
-1
@@ -1,3 +1,4 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
|
||||
namespace Ryujinx.Graphics.Shader.StructuredIr
|
||||
@@ -90,7 +91,7 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
|
||||
return format is TextureFormat.R32Sint or TextureFormat.R32Uint;
|
||||
}
|
||||
|
||||
public static TextureFormat GetTextureFormatAtomic(IGpuAccessor gpuAccessor, int handle, int cbufSlot = -1)
|
||||
public static TextureFormat GetTextureFormatAtomic(IGpuAccessor gpuAccessor, int handle, int cbufSlot = -1, bool strict = false)
|
||||
{
|
||||
// Atomic image instructions do not support GL_EXT_shader_image_load_formatted,
|
||||
// and must have a type specified. Default to R32Sint if not available.
|
||||
@@ -99,6 +100,11 @@ namespace Ryujinx.Graphics.Shader.StructuredIr
|
||||
|
||||
if (!FormatSupportsAtomic(format))
|
||||
{
|
||||
if (strict)
|
||||
{
|
||||
throw new NotSupportedException($"MSL 2.3 image atomics require an R32 integer descriptor format, got {format}.");
|
||||
}
|
||||
|
||||
gpuAccessor.Log($"Unsupported format for texture {handle}: {format}.");
|
||||
|
||||
format = TextureFormat.R32Sint;
|
||||
|
||||
+4
-1
@@ -2,6 +2,7 @@ namespace Ryujinx.Graphics.Shader.Translation
|
||||
{
|
||||
class HostCapabilities
|
||||
{
|
||||
public readonly bool Msl23ImageAtomics;
|
||||
public readonly bool ReducedPrecision;
|
||||
public readonly bool SupportsFragmentShaderInterlock;
|
||||
public readonly bool SupportsFragmentShaderOrderingIntel;
|
||||
@@ -23,8 +24,10 @@ namespace Ryujinx.Graphics.Shader.Translation
|
||||
bool supportsShaderFloat64,
|
||||
bool supportsShaderNonUniformIndexing,
|
||||
bool supportsTextureShadowLod,
|
||||
bool supportsViewportMask)
|
||||
bool supportsViewportMask,
|
||||
bool msl23ImageAtomics)
|
||||
{
|
||||
Msl23ImageAtomics = msl23ImageAtomics;
|
||||
ReducedPrecision = reducedPrecision;
|
||||
SupportsFragmentShaderInterlock = supportsFragmentShaderInterlock;
|
||||
SupportsFragmentShaderOrderingIntel = supportsFragmentShaderOrderingIntel;
|
||||
|
||||
Vendored
+9
-2
@@ -32,7 +32,13 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!TryConvertBindless(block, resourceManager, gpuAccessor, texOp) &&
|
||||
bool converted = TryConvertBindless(block, resourceManager, gpuAccessor, texOp);
|
||||
if (!converted && texOp.Inst == Instruction.ImageAtomic && texOp.Flags.HasFlag(TextureFlags.AtomicMsl23))
|
||||
{
|
||||
throw new NotSupportedException("MSL 2.3 image atomics require a statically resolved image handle.");
|
||||
}
|
||||
|
||||
if (!converted &&
|
||||
!GenerateBindlessAccess(block, resourceManager, gpuAccessor, texOp, node))
|
||||
{
|
||||
// If we can't do bindless elimination, remove the texture operation.
|
||||
@@ -334,7 +340,8 @@ namespace Ryujinx.Graphics.Shader.Translation.Optimizations
|
||||
{
|
||||
if (texOp.Inst == Instruction.ImageAtomic)
|
||||
{
|
||||
texOp.Format = ShaderProperties.GetTextureFormatAtomic(gpuAccessor, cbufOffset, cbufSlot);
|
||||
texOp.Format = ShaderProperties.GetTextureFormatAtomic(gpuAccessor, cbufOffset, cbufSlot,
|
||||
strict: texOp.Flags.HasFlag(TextureFlags.AtomicMsl23));
|
||||
}
|
||||
else
|
||||
{
|
||||
|
||||
+2
@@ -10,5 +10,7 @@ namespace Ryujinx.Graphics.Shader.Translation
|
||||
VertexA = 1 << 0,
|
||||
Compute = 1 << 1,
|
||||
DebugMode = 1 << 2,
|
||||
// Offline MSL 2.3: SPIRV-Cross uses a raw pointer for image atomics.
|
||||
Msl23ImageAtomics = 1 << 3,
|
||||
}
|
||||
}
|
||||
+2
-1
@@ -366,7 +366,8 @@ namespace Ryujinx.Graphics.Shader.Translation
|
||||
GpuAccessor.QueryHostSupportsShaderFloat64(),
|
||||
GpuAccessor.QueryHostSupportsShaderNonUniformIndexing(),
|
||||
GpuAccessor.QueryHostSupportsTextureShadowLod(),
|
||||
GpuAccessor.QueryHostSupportsViewportMask());
|
||||
GpuAccessor.QueryHostSupportsViewportMask(),
|
||||
Options.Flags.HasFlag(TranslationFlags.Msl23ImageAtomics));
|
||||
|
||||
CodeGenParameters parameters = new CodeGenParameters(attributeUsage, definitions, resourceManager.Properties, hostCapabilities, GpuAccessor, Options.TargetApi);
|
||||
|
||||
|
||||
@@ -0,0 +1,375 @@
|
||||
"""MSL-2.3 image atomic contract, using synthetic shaders only.
|
||||
|
||||
Portable tests require Python. To exercise the actual rebuilt offline compiler:
|
||||
DMSH_TEST_XLATE=/absolute/path/to/dmsh_xlate python3 -m unittest discover \
|
||||
-s dm/tools/tests -p test_image_atomic_bounds.py -v
|
||||
Requires spirv-cross and spirv-val on PATH; optionally DMSH_TEST_METAL=1 on Mac
|
||||
also compiles and links each generated MSL file. DMSH_TEST_METAL_SDKS defaults
|
||||
to macosx; set macosx,iphoneos to check both SDKs with the production flags.
|
||||
No shader is submitted to a GPU and no game inputs are used.
|
||||
"""
|
||||
import ast
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[3]
|
||||
RYU = ROOT / 'dm/third_party/ryujinx/src/Ryujinx.Graphics.Shader'
|
||||
spec = importlib.util.spec_from_file_location('dmsh_atomic', ROOT / 'dm/gfx/tools/dmsh_atomic.py')
|
||||
atomic = importlib.util.module_from_spec(spec); spec.loader.exec_module(atomic)
|
||||
|
||||
|
||||
def offset(coord, extent):
|
||||
"""Reference algebra, not a substitute for executing the compiler fixture."""
|
||||
if not all((x & 0xffffffff) < n for x, n in zip(coord, extent)):
|
||||
return None
|
||||
return coord[0] + extent[0] * (coord[1] + extent[1] * coord[2])
|
||||
|
||||
|
||||
def synthetic_shader(x, y, layer=0, *, array=True, mode=0, discard=True, reduction=False, size=0, dim=None, bindless=None):
|
||||
table = (RYU / 'Decoders/InstTable.cs').read_text()
|
||||
def base(name):
|
||||
pattern = re.search(r'Add\("([01x]{64})", InstName\.' + name + r'\s*,', table)[1]
|
||||
return int(pattern.replace('x', '0'), 2)
|
||||
def mov(reg, value):
|
||||
return base('Mov32i') | ((value & 0xffffffff) << 20) | (7 << 16) | (15 << 12) | reg
|
||||
instructions = [mov(0, x), mov(1, y), mov(2, layer), mov(3, 1)]
|
||||
dim = (4 if array else 3) if dim is None else dim
|
||||
if reduction:
|
||||
op = base('Sured') | (mode << 49) | (0x48 << 36) | (dim << 33) | (2 << 24) | (7 << 16) | 3
|
||||
else:
|
||||
op = base('Suatom') | (mode << 49) | (0x48 << 36) | (dim << 33) | (2 << 29) | (3 << 20) | (7 << 16) | (255 if discard else 4)
|
||||
op |= size << (20 if reduction else 51)
|
||||
if bindless is not None:
|
||||
assert reduction # This fixture uses SURED.B's non-returning form.
|
||||
if bindless == 'cbuf':
|
||||
# LDC.32 R4, c[2][0x120], RZ: the descriptor for image slot zero.
|
||||
instructions.append(base('Ldc') | (4 << 48) | (2 << 36) | (0x120 << 20) | (7 << 16) | (255 << 8) | 4)
|
||||
handle = 4
|
||||
else:
|
||||
assert bindless == 'unresolved'
|
||||
handle = 3 # The constant in R3 cannot resolve to a descriptor.
|
||||
op = base('SuredB') | (handle << 39) | (mode << 49) | (dim << 33) | (2 << 24) | (size << 20) | (7 << 16) | 3
|
||||
assert ((op >> 49) & 3) == mode
|
||||
instructions += [op, base('Exit') | (7 << 16) | 15]
|
||||
words = []
|
||||
for i, instruction in enumerate(instructions):
|
||||
if i % 3 == 0: words.append(0) # Maxwell scheduling word
|
||||
words.append(instruction)
|
||||
return struct.pack('<%dQ' % len(words), *words)
|
||||
|
||||
|
||||
def instructions(path):
|
||||
data = path.read_bytes(); words = struct.unpack('<%dI' % (len(data) // 4), data)
|
||||
assert words[0] == 0x07230203
|
||||
result = []; i = 5
|
||||
while i < len(words):
|
||||
n, op = words[i] >> 16, words[i] & 0xffff
|
||||
assert n > 0 and i + n <= len(words)
|
||||
result.append((op, list(words[i + 1:i + n]))); i += n
|
||||
return result
|
||||
|
||||
|
||||
def check_coordinate_guard(ops, expected):
|
||||
"""Verify actual emitted dataflow, not merely a guard-opcode census."""
|
||||
defs, blocks, predecessors, branches, pointers = {}, {}, {}, [], []
|
||||
current = None
|
||||
for op, args in ops:
|
||||
if op in (43, 80, 124, 61, 104, 176, 155, 60): defs[args[1]] = (op, args)
|
||||
if op == 248:
|
||||
current = args[0]; blocks[current] = []; predecessors.setdefault(current, set())
|
||||
if current is not None: blocks[current].append((op, args))
|
||||
if op == 60: pointers.append((current, args))
|
||||
if op == 249: predecessors.setdefault(args[0], set()).add(current)
|
||||
if op == 250:
|
||||
branches.append((current, args))
|
||||
for target in args[1:3]: predecessors.setdefault(target, set()).add(current)
|
||||
assert len(pointers) == 1
|
||||
pointer_block, pointer = pointers[0]
|
||||
coord_id = pointer[3]
|
||||
op, coord = defs[coord_id]
|
||||
if len(expected) == 1: # Native texture-buffer indexing keeps a scalar coordinate.
|
||||
assert op == 43 and coord[2] == (expected[0] & 0xffffffff)
|
||||
else:
|
||||
assert op == 80
|
||||
assert tuple(defs[c][1][2] for c in coord[2:]) == tuple(v & 0xffffffff for v in expected)
|
||||
assert all(defs[c][0] == 43 for c in coord[2:])
|
||||
guard_block, branch = next((b, a) for b, a in branches if a[1] == pointer_block and a[2] != pointer_block)
|
||||
assert predecessors[pointer_block] == {guard_block}
|
||||
op, comparison = defs[branch[0]]
|
||||
if len(expected) != 1:
|
||||
assert op == 155 # OpAll
|
||||
op, comparison = defs[comparison[2]]
|
||||
assert op == 176
|
||||
op, cast = defs[comparison[2]]; assert op == 124 and cast[2] == coord_id
|
||||
op, query = defs[comparison[3]]; assert op == 104
|
||||
op, load = defs[query[2]]; assert op == 61 and load[2] == pointer[2]
|
||||
entry = next(iter(blocks))
|
||||
dominators = {b: ({b} if b == entry else set(blocks)) for b in blocks}
|
||||
while True:
|
||||
updated = {b: ({b} if b == entry else {b} | set.intersection(*(dominators[p] for p in predecessors[b]))) for b in blocks}
|
||||
if updated == dominators: break
|
||||
dominators = updated
|
||||
assert guard_block in dominators[pointer_block]
|
||||
assert [b for b, body in blocks.items() if any(op == 239 for op, _ in body)] == [pointer_block]
|
||||
|
||||
|
||||
class Portable(unittest.TestCase):
|
||||
def test_axis_bounds_not_flattened_bounds(self):
|
||||
for coord in [(-1, 1, 0), (8, 0, 0), (0, -1, 0), (0, 4, 0), (0, 0, -1), (0, 0, 3), (-2**31, 0, 0)]:
|
||||
self.assertIsNone(offset(coord, (8, 4, 3)), coord)
|
||||
self.assertEqual(offset((0, 0, 0), (8, 4, 3)), 0)
|
||||
self.assertEqual(offset((7, 3, 2), (8, 4, 3)), 95)
|
||||
self.assertEqual([offset((0, 0, z), (8, 4, 3)) for z in range(3)], [0, 32, 64])
|
||||
|
||||
def test_compiler_helper_contract(self):
|
||||
source = atomic._XY_HELPER + '\n' + atomic._DEFAULT_ALIGNMENT
|
||||
self.assertEqual(atomic.validate_msl(source), source)
|
||||
self.assertIn(atomic.HELPERS, atomic.lower_msl(source))
|
||||
self.assertNotIn('#define spvImage2DAtomicCoord', atomic.lower_msl(source))
|
||||
self.assertIn(atomic.HELPERS, atomic.lower_msl(source.replace(' ', ' ')))
|
||||
self.assertEqual(atomic.validate_msl('fragment void main0() {}'), 'fragment void main0() {}')
|
||||
for changed in [source.replace('(tc).x)', '(tc).z)'), source.replace(': 4;', ': 256;'), source.replace('#define', 'inline'), source + '\n' + atomic._XY_HELPER]:
|
||||
with self.assertRaises(ValueError): atomic.validate_msl(changed)
|
||||
|
||||
@unittest.skipUnless(shutil.which('c++'), 'portable C++ compiler unavailable')
|
||||
def test_exact_helper_body(self):
|
||||
# Compile and execute the exact source-owned helper, with only the
|
||||
# texture/type API mocked. This is not a Metal compiler/GPU test.
|
||||
source = '''#include <cassert>
|
||||
#include <cstdint>
|
||||
using uint = uint32_t; using ulong = uint64_t;
|
||||
enum access { sample }; struct int2 { int x,y; }; struct int3 { int x,y,z; };
|
||||
uint spvLinearTextureAlignment = 4;
|
||||
template<typename T, access A> struct texture2d { uint w,h; uint get_width() const {return w;} uint get_height() const {return h;} };
|
||||
template<typename T, access A> struct texture2d_array { uint w,h; uint get_width() const {return w;} uint get_height() const {return h;} };
|
||||
''' + atomic.HELPERS + '''
|
||||
int main() {
|
||||
texture2d<uint, sample> two{8,4}; texture2d_array<uint,sample> array{8,4};
|
||||
assert(spvImage2DAtomicCoord(int2{7,3},two)==31);
|
||||
assert(spvImage2DAtomicCoord(int3{7,3,0},array)==31);
|
||||
assert(spvImage2DAtomicCoord(int3{0,0,1},array)==32);
|
||||
assert(spvImage2DAtomicCoord(int3{7,3,2},array)==95);
|
||||
texture2d_array<uint,sample> large{16384,16384};
|
||||
assert(spvImage2DAtomicCoord(int3{7,3,65},large)==ulong(65)*16384*16384+3*16384+7);
|
||||
spvLinearTextureAlignment=256;
|
||||
texture2d_array<uint,sample> padded{17,3};
|
||||
assert(spvImage2DAtomicCoord(int3{16,2,1},padded)==64*(3+2)+16);
|
||||
}'''
|
||||
with tempfile.TemporaryDirectory(prefix='dm-atomic-helper-') as temp:
|
||||
path=Path(temp); (path/'test.cpp').write_text(source)
|
||||
subprocess.run(['c++','-std=c++17','-Wall','-Wextra','-Werror',str(path/'test.cpp'),'-o',str(path/'test')],check=True)
|
||||
subprocess.run([str(path/'test')],check=True)
|
||||
|
||||
def test_mode_and_result_are_retained(self):
|
||||
src = (RYU / 'Instructions/InstEmitSurface.cs').read_text()
|
||||
self.assertEqual(src.count(' op.Clamp,'), 5)
|
||||
self.assertEqual(src.count('EmitSured(context, op.Dim, op.Clamp,'), 2)
|
||||
self.assertIn('dest == RegisterConsts.RegisterZeroIndex && !compareAndSwap', src)
|
||||
self.assertEqual(src.count('(TextureFlags)((int)clamp << 20)'), 2)
|
||||
instructions_source = (RYU / 'CodeGen/Spirv/Instructions.cs').read_text()
|
||||
part = instructions_source.split('private static OperationResult GenerateImageAtomic', 1)[1].split('private static OperationResult GenerateImageLoad', 1)[0]
|
||||
self.assertLess(part.index('context.BranchConditional(inside'), part.index('context.ImageTexelPointer'))
|
||||
self.assertIn('context.ULessThan(compareType, coords, extent)', part)
|
||||
self.assertNotIn('context.IMul(uintType, height, layer)', part)
|
||||
self.assertEqual(part.count('pCoords ='), 2) # scalar/vector construction only; never folded
|
||||
self.assertIn('context.HostCapabilities.Msl23ImageAtomics', part)
|
||||
self.assertIn('texOp.Format != TextureFormat.R32Uint', part)
|
||||
self.assertIn('texOp.Format != TextureFormat.R32Sint', part)
|
||||
self.assertEqual(src.count('size is not (SuatomSize.U32 or SuatomSize.S32 or SuatomSize.Sd32)'), 2)
|
||||
|
||||
def test_helper_participates_in_cache_digest(self):
|
||||
source = (ROOT / 'dm/gfx/tools/dmsh_build.py').read_text()
|
||||
assignment = next(n for n in ast.parse(source).body if isinstance(n, ast.Assign) and any(isinstance(t, ast.Name) and t.id == 'CODE_FP' for t in n.targets))
|
||||
self.assertIn('dmsh_atomic', {n.id for n in ast.walk(assignment) if isinstance(n, ast.Name)})
|
||||
self.assertIn("'tools/dmsh_atomic.py'", (ROOT / 'dm/gfx/tools/dmsh_chain.py').read_text())
|
||||
|
||||
def test_synthetic_inputs(self):
|
||||
for mode in range(4):
|
||||
data = synthetic_shader(0, -1, mode=mode)
|
||||
self.assertEqual(len(data), 64)
|
||||
op = struct.unpack_from('<Q', data, 48)[0]
|
||||
self.assertEqual((op >> 49) & 3, mode)
|
||||
self.assertEqual(op & 255, 255)
|
||||
|
||||
|
||||
@unittest.skipUnless(os.environ.get('DMSH_TEST_XLATE'), 'rebuilt .NET translator not configured; portable tests only')
|
||||
class Compiler(unittest.TestCase):
|
||||
def test_descriptor_sized_formats(self):
|
||||
# Sd32 takes its format from the descriptor. Validate that original
|
||||
# format before the legacy translator can default it to R32Sint.
|
||||
with tempfile.TemporaryDirectory(prefix='dm-image-atomic-format-') as temp:
|
||||
work = Path(temp)
|
||||
for reduction, bindless in ((False, None), (True, None), (True, 'cbuf')):
|
||||
for fmt in ('R32Uint', 'R32Sint', 'R16Float', 'R32Float', 'R8G8B8A8Unorm', 'Unknown'):
|
||||
key = ('sured' if reduction else 'suatom') + '_' + str(bindless) + '_' + fmt
|
||||
code = work / (key + '.bin')
|
||||
code.write_bytes(synthetic_shader(0, 0, reduction=reduction, size=6, bindless=bindless))
|
||||
jobs = work / 'jobs.json'
|
||||
jobs.write_text(json.dumps([dict(key=key, stage='cs', code=str(code),
|
||||
img={'0': dict(type='2darray', fmt=fmt)}, local=[1, 1, 1], debug=True)]))
|
||||
result = subprocess.run([os.environ['DMSH_TEST_XLATE'], str(jobs), str(work)], capture_output=True, text=True)
|
||||
spv = work / (key + '.spv')
|
||||
if fmt in ('R32Uint', 'R32Sint'):
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
subprocess.run(['spirv-val', '--target-env', 'vulkan1.1', str(spv)], check=True)
|
||||
else:
|
||||
self.assertNotEqual(result.returncode, 0, fmt)
|
||||
self.assertIn('require an R32 integer descriptor format', result.stderr)
|
||||
self.assertFalse(spv.exists())
|
||||
|
||||
def test_unresolved_bindless_atomic_rejected(self):
|
||||
with tempfile.TemporaryDirectory(prefix='dm-image-atomic-bindless-') as temp:
|
||||
work = Path(temp)
|
||||
code = work / 'unresolved.bin'
|
||||
code.write_bytes(synthetic_shader(0, 0, reduction=True, bindless='unresolved'))
|
||||
jobs = work / 'jobs.json'
|
||||
jobs.write_text(json.dumps([dict(key='unresolved', stage='cs', code=str(code),
|
||||
img={'0': dict(type='2darray', fmt='R32Uint')}, local=[1, 1, 1], debug=True)]))
|
||||
result = subprocess.run([os.environ['DMSH_TEST_XLATE'], str(jobs), str(work)], capture_output=True, text=True)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('require a statically resolved image handle', result.stderr)
|
||||
self.assertFalse((work / 'unresolved.spv').exists())
|
||||
|
||||
def compile_metal(self, msl, work, key):
|
||||
if os.environ.get('DMSH_TEST_METAL') != '1':
|
||||
return
|
||||
metal_sdks = os.environ.get('DMSH_TEST_METAL_SDKS', 'macosx').split(',')
|
||||
self.assertTrue(metal_sdks and all(sdk in ('macosx', 'iphoneos') for sdk in metal_sdks),
|
||||
'DMSH_TEST_METAL_SDKS must be macosx, iphoneos or macosx,iphoneos')
|
||||
for sdk in metal_sdks:
|
||||
# Match dmsh_build.MAC_METAL and ios_metal.METAL;
|
||||
# the latter needs relaxed NaN-aware arithmetic.
|
||||
flags = ['-fpreserve-invariance'] if sdk == 'macosx' else [
|
||||
'-mios-version-min=18.0', '-fmetal-math-mode=relaxed', '-fpreserve-invariance']
|
||||
air = work / (key + '.' + sdk + '.air')
|
||||
library = work / (key + '.' + sdk + '.metallib')
|
||||
subprocess.run(['xcrun','-sdk',sdk,'metal','-c',*flags,str(msl),'-o',str(air)],check=True)
|
||||
subprocess.run(['xcrun','-sdk',sdk,'metallib',str(air),'-o',str(library)],check=True)
|
||||
self.assertGreater(air.stat().st_size, 0)
|
||||
self.assertGreater(library.stat().st_size, 0)
|
||||
print('synthetic Metal compile/link passed:', key, sdk, library.stat().st_size, 'bytes')
|
||||
|
||||
def test_dimension_routes(self):
|
||||
# 1D routes retain the existing y=0 normalization; buffer images stay
|
||||
# scalar and must not acquire the source-owned 2D helper.
|
||||
shapes = [('1d', (1, 0)), ('buffer', (1,)), ('1darray', (1, 0, 2)),
|
||||
('2d', (1, 2)), ('2darray', (1, 2, 3)), ('3d', None)]
|
||||
with tempfile.TemporaryDirectory(prefix='dm-image-atomic-dim-') as temp:
|
||||
work = Path(temp)
|
||||
for reduction in (False, True):
|
||||
for dim, (shape, expected) in enumerate(shapes):
|
||||
key = ('sured' if reduction else 'suatom') + '_dim_' + shape
|
||||
code = work / (key + '.bin')
|
||||
code.write_bytes(synthetic_shader(1, 2, 3, reduction=reduction, dim=dim))
|
||||
jp = work / 'jobs.json'
|
||||
jp.write_text(json.dumps([dict(key=key, stage='cs', code=str(code),
|
||||
img={'0': dict(type=shape, fmt='R32Uint')}, local=[1, 1, 1], debug=True)]))
|
||||
result = subprocess.run([os.environ['DMSH_TEST_XLATE'], str(jp), str(work)], capture_output=True, text=True)
|
||||
spv = work / (key + '.spv')
|
||||
if expected is None:
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertIn('support non-indexed 32-bit buffer/2D/2D-array images only', result.stderr)
|
||||
self.assertFalse(spv.exists())
|
||||
continue
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
subprocess.run(['spirv-val', '--target-env', 'vulkan1.1', str(spv)], check=True)
|
||||
check_coordinate_guard(instructions(spv), expected)
|
||||
msl = work / (key + '.metal')
|
||||
subprocess.run(['spirv-cross', '--msl', '--msl-version', '20300', '--msl-argument-buffers',
|
||||
'--msl-texture-buffer-native', str(spv), '--output', str(msl)], check=True)
|
||||
original = msl.read_text(); source = atomic.lower_msl(original)
|
||||
if shape == 'buffer':
|
||||
self.assertEqual(source, original)
|
||||
self.assertNotIn('spvImage2DAtomicCoord', source)
|
||||
else:
|
||||
self.assertIn(atomic.HELPERS, source)
|
||||
msl.write_text(source)
|
||||
self.compile_metal(msl, work, key)
|
||||
|
||||
def test_source_size_gate(self):
|
||||
with tempfile.TemporaryDirectory(prefix='dm-image-atomic-width-') as temp:
|
||||
work = Path(temp)
|
||||
for reduction in (False, True):
|
||||
for size in range(8):
|
||||
key = ('sured' if reduction else 'suatom') + '_size_' + str(size)
|
||||
code = work / (key + '.bin')
|
||||
code.write_bytes(synthetic_shader(0, 0, reduction=reduction, size=size))
|
||||
jp = work / 'jobs.json'
|
||||
jp.write_text(json.dumps([dict(key=key, stage='cs', code=str(code),
|
||||
img={'0': dict(type='2darray', fmt='R32Uint')}, local=[1, 1, 1], debug=True)]))
|
||||
result = subprocess.run([os.environ['DMSH_TEST_XLATE'], str(jp), str(work)], capture_output=True, text=True)
|
||||
spv = work / (key + '.spv')
|
||||
if size in (0, 1, 6): # U32, S32, descriptor-sized32.
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
subprocess.run(['spirv-val', '--target-env', 'vulkan1.1', str(spv)], check=True)
|
||||
msl = work / (key + '.metal')
|
||||
subprocess.run(['spirv-cross', '--msl', '--msl-version', '20300', '--msl-argument-buffers',
|
||||
'--msl-texture-buffer-native', str(spv), '--output', str(msl)], check=True)
|
||||
msl.write_text(atomic.lower_msl(msl.read_text()))
|
||||
self.compile_metal(msl, work, key)
|
||||
else: # U64, S64, descriptor-sized64, float and half2.
|
||||
self.assertNotEqual(result.returncode, 0, key)
|
||||
self.assertIn('require a 32-bit integer source size', result.stderr)
|
||||
self.assertFalse(spv.exists())
|
||||
|
||||
def test_real_translation_and_cross(self):
|
||||
for tool in ('spirv-cross', 'spirv-val'):
|
||||
self.assertIsNotNone(shutil.which(tool), tool + ' required for compiler acceptance')
|
||||
with tempfile.TemporaryDirectory(prefix='dm-image-atomic-') as temp:
|
||||
work = Path(temp)
|
||||
cases = [('negative_x', -1, 1, 0), ('negative_y', 0, -1, 0), ('int_min', -2**31, 0, 0),
|
||||
('width_edge', 8, 0, 0), ('height_edge', 0, 4, 0), ('layer_edge', 0, 0, 3),
|
||||
('valid_origin', 0, 0, 0), ('valid_last', 7, 3, 2)]
|
||||
for array in (False, True):
|
||||
for reduction in (False, True):
|
||||
jobs = []; expected = {}
|
||||
for name, x, y, z in cases:
|
||||
key = name + ('_array' if array else '_2d') + ('_sured' if reduction else '_suatom')
|
||||
code = work / (key + '.bin'); code.write_bytes(synthetic_shader(x, y, z, array=array, reduction=reduction))
|
||||
expected[key] = (x, y, z) if array else (x, y)
|
||||
jobs.append(dict(key=key, stage='cs', code=str(code), img={'0':dict(type='2darray' if array else '2d',fmt='R32Uint')},local=[1,1,1],debug=True))
|
||||
jp = work / 'jobs.json'; jp.write_text(json.dumps(jobs))
|
||||
subprocess.run([os.environ['DMSH_TEST_XLATE'], str(jp), str(work)], check=True)
|
||||
for job in jobs:
|
||||
spv = work / (job['key'] + '.spv')
|
||||
subprocess.run(['spirv-val', '--target-env', 'vulkan1.1', str(spv)], check=True)
|
||||
ops = instructions(spv)
|
||||
check_coordinate_guard(ops, expected[job['key']])
|
||||
current = None; pointer_blocks = []; branches = []
|
||||
for op, args in ops:
|
||||
if op == 248: current = args[0]
|
||||
if op == 60: pointer_blocks.append(current)
|
||||
if op == 250: branches.append(args)
|
||||
self.assertEqual(len(pointer_blocks), 1)
|
||||
self.assertTrue(any(b[1] == pointer_blocks[0] and b[2] != pointer_blocks[0] for b in branches))
|
||||
self.assertTrue(any(op == 104 for op, _ in ops)) # OpImageQuerySize
|
||||
self.assertTrue(any(op == 176 for op, _ in ops)) # OpULessThan
|
||||
self.assertTrue(any(op == 239 for op, _ in ops)) # OpAtomicUMax
|
||||
msl = work / (job['key'] + '.metal')
|
||||
subprocess.run(['spirv-cross','--msl','--msl-version','20300','--msl-argument-buffers','--msl-texture-buffer-native',str(spv),'--output',str(msl)], check=True)
|
||||
original = msl.read_text(); source = atomic.lower_msl(original)
|
||||
helper_line = next(line for line in original.splitlines() if atomic._helper_line(line))
|
||||
self.assertEqual(source, original.replace(helper_line, atomic.HELPERS, 1))
|
||||
msl.write_text(source)
|
||||
self.assertIn('.get_height()', source)
|
||||
if array: self.assertIn('.get_array_size()', source)
|
||||
self.compile_metal(msl, work, job['key'])
|
||||
for mode, discard in [(1, True), (2, True), (3, True), (0, False)]:
|
||||
key = 'reject_%d_%d' % (mode, discard)
|
||||
code=work/(key+'.bin'); code.write_bytes(synthetic_shader(0,0,mode=mode,discard=discard))
|
||||
jp.write_text(json.dumps([dict(key=key,stage='cs',code=str(code),img={'0':dict(type='2darray',fmt='R32Uint')},local=[1,1,1],debug=True)]))
|
||||
result=subprocess.run([os.environ['DMSH_TEST_XLATE'],str(jp),str(work)],capture_output=True,text=True)
|
||||
self.assertNotEqual(result.returncode,0)
|
||||
self.assertIn('require IGN with a discarded result',result.stderr)
|
||||
self.assertFalse((work/(key+'.spv')).exists())
|
||||
|
||||
|
||||
if __name__ == '__main__': unittest.main()
|
||||
@@ -33,6 +33,7 @@ class Header(Base):
|
||||
self.repo = self.tmp / 'repo'
|
||||
(self.repo / 'dm/gfx/nvn').mkdir(parents=True)
|
||||
(self.repo / 'dm/gfx/tools').symlink_to(REPO / 'dm/gfx/tools'); (self.repo / 'dm/ios').symlink_to(REPO / 'dm/ios')
|
||||
(self.repo / 'dm/third_party').symlink_to(REPO / 'dm/third_party')
|
||||
self.h = self.repo / nsp.CHAIN_H
|
||||
shutil.copy(REPO / nsp.CHAIN_H, self.h)
|
||||
self.env = {k: v for k, v in os.environ.items() if k != 'DM_GAME'} # the version needs no game
|
||||
|
||||
@@ -380,6 +380,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
|
||||
- characters and props glowing white, about 10⁴ too bright, in DRAGON QUEST VII Reimagined's opening cutscene only, while walls and gameplay look right. → [A texture slot that shares its c2 word with an image slot](../reference/shaders.md#a-texture-slot-that-shares-its-c2-word-with-an-image-slot)
|
||||
- `[RED] shader … reflection conversion failed: 66 bindings, over the limit 64`, then `failed to load`, and the draws with it are dropped (a black map). → [More than 64 bindings in one shader](../reference/shaders.md#more-than-64-bindings-in-one-shader)
|
||||
- the replay put a texture resource ID where the shader wanted an address; with the old 64 KB dummy buffer, pointers in the game's objects and in the host's program objects got their upper 4 bytes replaced and the game or the replay crashed, or `kIOGPUCommandBufferCallbackErrorPageFault`, after which macOS ignores the process's command buffers (GitHub issue #101). → [Image atomics](../reference/shaders.md#image-atomics)
|
||||
- Metal shader validation reports invalid image-atomic stores even with an image-sized scratch buffer; without validation, GPU page faults can freeze rendering. → [Bounded MSL 2.3 atomic emulation](../reference/shaders.md#bounded-msl-23-atomic-emulation)
|
||||
- shaders missing from the map that no game file holds. → [Built-in shaders compiled with the game's own GLSLC](../reference/shaders.md#built-in-shaders-compiled-with-the-games-own-glslc)
|
||||
- a null read at `main+0x3316dc` in `nnMain`, right after the window setup (the program object's interface pointers at +0xd0 / +0xd8 both null). → [GLSLC GPU code version against the driver's range](../reference/shaders.md#glslc-gpu-code-version-against-the-drivers-range)
|
||||
- a vertex shader whose pairs changed was restored unpadded from `cas/`. → [Shader cache key covering the pairs](../reference/shaders.md#shader-cache-key-covering-the-pairs)
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
# Packed image-atomic binding admission
|
||||
|
||||
MSL 2.3 represents an image atomic with two arguments: the IMG texture supplies
|
||||
its dimensions, while IMGA is an atomic-word buffer pointer. Capacity and layout
|
||||
must agree **before** either is used by a draw or dispatch. This host check is
|
||||
separate from the shader translator's per-coordinate bounds checks: neither
|
||||
replaces the other. Packed addressing assumes the translator's reviewed default
|
||||
linear alignment of four bytes per 32-bit word.
|
||||
|
||||
`encode_args_` now resolves each IMGA's unique, same-register IMG once under the
|
||||
existing registry lock, then uses that exact texture for both arguments. It does
|
||||
not change argument-cache reuse, queue submission, resource residency, or deferred
|
||||
resource lifetime.
|
||||
|
||||
## Admitted paths
|
||||
|
||||
- Real images must resolve to a scalar R32 integer texture of the shader's
|
||||
texture type: buffer, 2D, or 2D array, without multisampling.
|
||||
- Direct guest-pool writes require an actual non-view buffer texture or base 2D
|
||||
texture, the matching guest dimensions/format size, one mip, the same Metal
|
||||
backing buffer and base offset, and enough bytes in both guest storage and the
|
||||
buffer. For 2D the actual buffer row pitch must equal width times four. A buffer
|
||||
texture's unused next row may be padded.
|
||||
- The existing **discard scratch** path for non-linear images is retained, not
|
||||
made coherent with the image. Its previous source-sized request, 1 MiB rounding,
|
||||
reuse and deferred replacement policy remain for admitted images. Checked
|
||||
arithmetic and actual IMG dimensions ensure the existing request covers the
|
||||
matched image; no request is enlarged to conceal a mismatch.
|
||||
- An unbound IMG may retain the existing dummy pointer only if its actual dummy
|
||||
extent fits and neither side resolves to a real image.
|
||||
|
||||
## Fail-closed limits
|
||||
|
||||
Direct padded-row textures, array images, original 1D descriptors (even when
|
||||
normalized to a one-row Metal 2D texture), mip/layer/format views, private fallback
|
||||
textures without the expected buffer, unsupported formats/shapes, failed scratch
|
||||
allocation, and capacity/arithmetic failures stop with a diagnostic before the
|
||||
argument buffer is bound and before the offending draw/dispatch. This is a
|
||||
conservative trial compatibility restriction; a safe but unproved direct view is
|
||||
also rejected. Real linear writes are **never redirected to discard scratch**.
|
||||
The translator's existing 1D/1D-array normalization is preserved, but that alone
|
||||
does not prove direct native storage. Non-linear images still follow the
|
||||
separate matched-extent discard-scratch admission above. Buffer shader routes
|
||||
remain scalar and are not rewritten by the 2D MSL helper; direct buffer host
|
||||
admission still requires the actual backing buffer and offset checks.
|
||||
|
||||
The decisive diagnostic suffixes are:
|
||||
|
||||
- `ambiguous IMG partner` / `missing IMG partner`
|
||||
- `unsupported or overflowing IMG extent`
|
||||
- `real image is not a resolved R32 integer IMG`
|
||||
- `unsupported direct IMG/IMGA layout or capacity`
|
||||
- `scratch allocation or capacity failed; refusing short dummy`
|
||||
- `unmatched unbound IMG/IMGA pair`
|
||||
|
||||
Each is prefixed by `image atomics:` and includes shader/stage/slot context.
|
||||
These are fatal errors, not warning suppression or silent skipped draws. The
|
||||
former scratch-allocation failure branch could return a short 64 KiB dummy while
|
||||
IMG retained its original extent; it can no longer do so. This source-level
|
||||
failure path has not been observed in the supplied validation run, which had a
|
||||
real scratch buffer. This patch does not establish that failure path, direct-pool
|
||||
layout mismatch, or residency caused an earlier GPU fault.
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
make -C dm/gfx/tests GAME=0100F43008C44000 imga-layout
|
||||
ASAN_OPTIONS=detect_leaks=0 make -C dm/gfx/tests GAME=0100F43008C44000 imga-layout-sanitize
|
||||
python3 dm/gfx/tests/test_imga_binding_source.py
|
||||
```
|
||||
|
||||
The portable fixture executes production checked arithmetic/admission helpers:
|
||||
3 MiB versus 64 KiB capacity, valid packed layouts, overflow, zero dimensions,
|
||||
short storage, wrong buffer/offset, view rejection, padding and direct arrays.
|
||||
The source checks establish bounded wiring invariants, not Metal execution.
|
||||
The fixture does not execute Metal binding. Native Objective-C compilation,
|
||||
actual Metal texture-property behavior and game compatibility require separate
|
||||
Apple validation. No allocator, texture readout,
|
||||
shader algorithm, or game eligibility changes are included.
|
||||
|
||||
The buffer metadata comes from Apple's read-only [buffer offset](https://developer.apple.com/documentation/metal/mtltexture/bufferoffset?language=objc)
|
||||
and [row pitch](https://developer.apple.com/documentation/metal/mtltexture/bufferbytesperrow)
|
||||
properties; row/offset are queried only when a source buffer is present.
|
||||
@@ -542,6 +542,60 @@ Finding, keying and translating shaders (`dm/gfx/tools/shader_split.py`, `dmsh_b
|
||||
- **Found on:** [Octopath Traveler 0](../../dm/games/01005270232F2000/README.md): 2,084 such shaders (Unreal's GPU particles); three ran in the first 100 seconds. [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md): in the 64 KB dummy buffer used before, its atomic max into a 480x270 r32ui 2D array (fragment shader `2bd5037003980716`, the scene after the story opening's first cutscene) ran about 1.4 MB past its end, writing into zero-copy pools over the game's heap and into unmapped addresses (GitHub issue #101).
|
||||
- **Note:** Metal shader validation (`MTL_SHADER_VALIDATION=1`) stops out-of-bounds accesses, so under it the scene played.
|
||||
|
||||
### Bounded MSL 2.3 atomic emulation
|
||||
|
||||
- **Symptom:** Metal shader validation reports invalid image-atomic stores even with an image-sized scratch buffer; without validation, GPU page faults can freeze rendering.
|
||||
- **Cause:** the translator lost the surface instruction's `IGN` bounds mode before lowering an image atomic to a raw buffer pointer. Scratch capacity cannot protect against invalid coordinates; the upstream 2D coordinate helper also omits array layers.
|
||||
- **Fix:** preserve the supported bounds mode, guard every coordinate before pointer construction, use layer-aware 64-bit addressing, and validate the matching image and buffer's layout and capacity before binding. The supported contract and tests are described below.
|
||||
- **Found on:** [Pokémon Legends: Z-A](../../dm/games/0100F43008C44000/README.md#rendering), with the development-build report and remaining gameplay validation recorded there.
|
||||
|
||||
Sizing the backing buffer does not make an out-of-range image coordinate safe.
|
||||
The offline translator now preserves the Maxwell surface instruction's bounds
|
||||
mode and explicitly discarded result. Its Metal-only lowering accepts `IGN`
|
||||
reductions and non-CAS atomics writing `RZ`; returning, predicate-returning,
|
||||
`TRAP`, and unknown modes fail translation rather than acquiring invented
|
||||
out-of-bounds semantics. This path supports non-indexed 32-bit buffer, 2D and
|
||||
2D-array images. Other translator callers do not enable it. Source 1D and
|
||||
1D-array instructions retain the existing one-row normalization to 2D `(x, 0)`
|
||||
and 2D-array `(x, 0, layer)`; buffer coordinates stay scalar, and buffer MSL is
|
||||
not rewritten by the 2D helper. Source 64-bit, floating-point and half2 atomics
|
||||
are rejected before descriptor-format resolution can erase their original size.
|
||||
Descriptor-sized 32-bit atomics also reject non-R32 integer descriptors before
|
||||
the legacy format fallback, both for bound images and constant-buffer-resolved
|
||||
bindless handles. An unresolved bindless atomic fails translation instead of
|
||||
being removed by bindless elimination. These checks apply only to the Metal
|
||||
atomic path; other translator backends retain their existing behavior.
|
||||
Indexed, 3D, multisample and other unsupported effective image shapes fail
|
||||
closed on this opt-in path; this is not a native-image-atomic implementation.
|
||||
|
||||
The SPIR-V queries the bound image's dimensions and tests every original
|
||||
coordinate (including the layer) before forming the texel pointer or executing
|
||||
the atomic. Negative coordinates fail an unsigned comparison. Coordinates are
|
||||
not clamped, flattened for the bounds check, or changed in SPIR-V.
|
||||
`dmsh_atomic.py` recognizes the exact standalone SPIRV-Cross MSL 2.3 coordinate
|
||||
helper and replaces only that definition with typed, layer-aware helpers using
|
||||
64-bit address arithmetic; shader statements and call sites are untouched.
|
||||
An unrecognized helper or default alignment fails the build. This is a packed
|
||||
emulation layout, not proof that an arbitrary native pool's row pitch, layer
|
||||
stride or texture-view offset matches it. The host checks the matched IMG/IMGA
|
||||
[layout and capacity contract](../image-atomic-binding-contract.md) before
|
||||
binding; block-linear scratch still discards results and does not implement
|
||||
image read/write coherence.
|
||||
|
||||
Changing this path invalidates the chain and shader cache. Rebuild the translator
|
||||
and record its actual fingerprints before preparing shaders again; a host-only
|
||||
rebuild is insufficient. The synthetic regression is
|
||||
`dm/tools/tests/test_image_atomic_bounds.py`: portable helper/algebra checks plus
|
||||
an opt-in rebuilt-translator → SPIR-V validation → MSL 2.3 check. Set
|
||||
`DMSH_TEST_XLATE` to the rebuilt translator, with `spirv-val` and `spirv-cross` on
|
||||
`PATH`; `DMSH_TEST_METAL=1` also compiles and links with Apple's Metal compiler.
|
||||
Set `DMSH_TEST_METAL_SDKS=macosx,iphoneos` to check both SDKs.
|
||||
The dimension fixture covers both instruction families for source 1D, buffer,
|
||||
1D array, 2D, 2D array and rejected 3D; accepted forms pass SPIR-V validation,
|
||||
Cross/helper lowering and the optional Metal checks. Shader acceptance does not
|
||||
imply admission of every native host layout. None of these tests executes a
|
||||
shader on a GPU.
|
||||
|
||||
## Shaders compiled at run time
|
||||
|
||||
### Built-in shaders compiled with the game's own GLSLC
|
||||
|
||||
Reference in new issue
Block a user