diff --git a/dm/games/README.md b/dm/games/README.md index d999839..c203257 100644 --- a/dm/games/README.md +++ b/dm/games/README.md @@ -56,7 +56,7 @@ None of them has a default game: without one they stop and list the profiles. To | `pad_layout` | The iPhone virtual gamepad's layout on first launch: `simple` (the default) = 2D (left stick, A/B/X/Y, shoulders, − / +), `full` = 3D (adds a D-pad, a right stick and L3 / R3; the right stick is not drawn: a touch landing in the right third of the screen on no control holds it, centred where it lands). The layout button at the top centre switches between them in game and the app remembers the player's choice (`DmGamepad.m`) | | `feedback_snap` | Fragment shader keys (prefixes) whose draws sample a colour attachment of their own render pass and must read a snapshot instead (`fb_prepare` in `nvn_mtl.m`). Writing and sampling one attachment in a render pass is undefined in Metal: the Mac usually happens to read the old content, the iPhone sometimes reads whole 16×16 blocks as magenta. Each such draw costs a pass split and a copy, so only the shaders seen to break are listed (Super Mario 3D World + Bowser's Fury: its final composite; Captain Toad: Treasure Tracker: the same agl program, whose blocks showed on the Mac too). `DM_NVN_FB_SNAP=0` / `all` / `` overrides, also from `Documents/env.txt` | | `alpha_from_red` | Fragment shader keys (prefixes, up to 48) that read a single-channel render target whose channel mapping is exactly (R, 0, 0, 1) as (R, 0, 0, R) (`tex_alpha_red` in `nvn_mtl.m`). It was a workaround for NintendoWare layout captures: the game rebuilds each capture with its own mapping ((1, 1, 1, R) for one tagged `A8`), and the textures kept (R, 0, 0, 1) only while the host's `nvnTextureCompare` ignored the mapping (fixed, see [04](../../docs/agents/04-graphics.md)). Breath of the Wild no longer lists it; Super Mario Odyssey still lists its 26 layout shaders. `DM_NVN_ALPHA_RED=0` / `` overrides, also from `Documents/env.txt` | -| `tex_flush_upload` | `on` / `off` (the default): a texture the game fills from the CPU in place (no packaged layout, no `WriteTexels`) and publishes with `nvnMemoryPoolFlushMappedRange` is uploaded from guest memory like a packaged one, and re-uploaded when its pages change (`ft_add` in `hle_nvn.c`). Tears of the Kingdom streams all its textures this way; without it the 3D scene is black. Off by default because a flushed range in another game could cover an image a shader writes, whose GPU content the upload would replace | +| `tex_flush_upload` | `on` / `off` (the default): a texture the game fills from the CPU in place (no packaged layout, no `WriteTexels`) and publishes with `nvnMemoryPoolFlushMappedRange` is uploaded from guest memory like a packaged one, and re-uploaded when its pages change (`ft_add` in `hle_nvn.c`). Tears of the Kingdom streams all its textures this way; without it the 3D scene is black. Off by default because a flushed range in another game could cover an image a shader writes, whose GPU content the upload would replace. Without the key, every game still gets a texture's storage once when the game flushed it before the texture's first use (`nvn_flushed` in `hle_nvn.c`: Breath of the Wild's sky tables); the key is for textures the CPU writes again after that | | `render_cond` | `on` / `off` (the default): `SetRenderEnableConditional` / `SetRenderEnable` are recorded and evaluated at replay, and a timestamp report's counter field is 0 as on the GPU (`OP_RENDER_COND` in `nvn_mtl.m`). Tears of the Kingdom wraps its clears in conditions that skip clearing a target nothing has drawn into since its last clear; without the key every clear runs and its 3D scene is black. Off by default: other games draw everything, as before | | `tex_dirty_upload` | `on` / `off` (the default), needs `tex_flush_upload`: a registered texture whose pages the CPU writes after `nvnTextureInitialize` without any NVN flush is uploaded as well (its pages are write-protected at registration, checked on first use; `nvn_ft_written` in `hle_nvn.c`). Tears of the Kingdom fills its ASTC terrain material arrays this way; without it the cave is magenta. FINAL FANTASY TACTICS - The Ivalice Chronicles loads its UI textures straight into CPU-uncached pools, which need no flush, after creating them; without both keys its screen stays black. The Great Ace Attorney Chronicles copies every texture into a CPU-cached pool after creating it and never flushes (its `main` has no cache maintenance at all): black too | | `tex_dyn_slots` | `on` / `off` (the default): a shader whose texture handle the translator could not trace to one slot gets Ryujinx's fallback arrays (`__cb0_40000000`: 32 textures and samplers indexed by the handle's texture / sampler ID). With the key the replay puts slot indices in that shader's c2 texture and sampler tables (texture slot r → r, sampler slot r → r << 20) and binds slot k at element k, so the shader's own arithmetic picks the binding (`NVN_RB_TEXD` / `NVN_RB_SAMPD` in `nvn_mtl.m`). Right only where the handle comes from c2's tables: Octopath Traveler 0's 61 such vertex shaders read separate texture slots 0-10 and samplers 0-4 there and nothing else; Paper Mario: The Thousand-Year Door's 2,386 pixel shaders read combined texture slots 0, 7, 8, 13 and 28 (slot r → r \| r << 20). Off, such shaders do not load. Not part of the shader chain version | diff --git a/dm/gfx/nvn/hle_nvn.c b/dm/gfx/nvn/hle_nvn.c index 2b9a400..8ca709b 100644 --- a/dm/gfx/nvn/hle_nvn.c +++ b/dm/gfx/nvn/hle_nvn.c @@ -200,11 +200,17 @@ NVN(nvnMemoryPoolInitialize, NVNboolean, NVNmemoryPool *p, const NVNmemoryPoolBu OLOG("MemoryPool %p mem=%p size=%#llx flags=%#x", (void *)p, (void *)o->mem, o->size, o->flags); return 1; } +static void fl_add(const uint8_t *a, uint64_t n); static void fl_drop(const uint8_t *a, uint64_t n); // flushed ranges, see nvn_flushed NVN(nvnMemoryPoolFinalize, void, NVNmemoryPool *p) { if (((pool_t *)p)->flags & 0x200) vpool_release((pool_t *)p); if (((pool_t *)p)->mem && !(((pool_t *)p)->flags & 0x200)) { // A virtual pool's storage is borrowed from a physical pool; do not unregister it - nvn_dirty_remove_range(((pool_t *)p)->mem); mtl_pool_del(((pool_t *)p)->mem); } mtl_alias_drop_pool(p); ((pool_t *)p)->magic = 0; } + nvn_dirty_remove_range(((pool_t *)p)->mem); mtl_pool_del(((pool_t *)p)->mem); fl_drop(((pool_t *)p)->mem, ((pool_t *)p)->size); } mtl_alias_drop_pool(p); ((pool_t *)p)->magic = 0; } static void ft_flush(const uint8_t *a, uint64_t n); -NVN(nvnMemoryPoolFlushMappedRange, void, const NVNmemoryPool *p, ptrdiff_t off, size_t n) { const pool_t *o = (const pool_t *)p; if (o->mem) ft_flush(o->mem + off, n); } // see ft_add +NVN(nvnMemoryPoolFlushMappedRange, void, const NVNmemoryPool *p, ptrdiff_t off, size_t n) { // see ft_add and nvn_flushed + const pool_t *o = (const pool_t *)p; + if (!o->mem) return; + if (!(o->flags & 0x200)) fl_add(o->mem + off, n); + ft_flush(o->mem + off, n); +} NVN(nvnMemoryPoolGetBufferAddress, NVNbufferAddress, const NVNmemoryPool *p) { return (NVNbufferAddress)((const pool_t *)p)->mem; } // ---------- Descriptor pools ---------- @@ -371,6 +377,55 @@ static void ft_flush(const uint8_t *a, uint64_t n) { if (g_ft[i].mem < a + n && a < g_ft[i].mem + g_ft[i].size) { mtl_tex_cpu_written(&g_ft[i].t); g_ft[i] = g_ft[--g_nft]; } else i++; pthread_mutex_unlock(&g_ft_mu); } +// Flushed ranges (every game): what the CPU published with nvnMemoryPoolFlushMappedRange, as disjoint [lo, hi) spans sorted by start (touching spans merge), dropped with their pool. +// Until the GPU first uses a texture it has not written it, so storage the game flushed before that first use is the texture's content: tex_init_fill (nvn_mtl.m) fills the new GPU +// texture from it once, as it does for a pool the CPU cannot access. Breath of the Wild copies its precomputed sky scattering tables (System/KSys/sky.skybin: a 256x32x16 RGBA16F 3D +// texture and two 2D ones) into CPU_CACHED pools of their own with memcpy and flushes them (main+0x1257d50); as GPU textures of zeros the sky had no scattered light and came out +// brownish grey instead of blue (GitHub issue #52). Textures the CPU keeps writing after their first use still need the profile's tex_flush_upload / tex_dirty_upload +typedef struct { const uint8_t *lo, *hi; } fl_span; +static fl_span *g_fl; static size_t g_nfl, g_cfl; static pthread_mutex_t g_fl_mu = PTHREAD_MUTEX_INITIALIZER; +static size_t fl_find(const uint8_t *a, int touch) { // the first span that ends after a (touch: or at a); g_fl_mu held + size_t l = 0, h = g_nfl; + while (l < h) { size_t m = (l + h) / 2; if (g_fl[m].hi < a || (!touch && g_fl[m].hi == a)) l = m + 1; else h = m; } + return l; +} +static void fl_add(const uint8_t *a, uint64_t n) { + if (!a || !n) return; + const uint8_t *b = a + n; + pthread_mutex_lock(&g_fl_mu); + size_t i = fl_find(a, 1), j = i; + if (i < g_nfl && g_fl[i].lo <= a && b <= g_fl[i].hi) { pthread_mutex_unlock(&g_fl_mu); return; } // already covered (a buffer flushed every frame) + while (j < g_nfl && g_fl[j].lo <= b) { if (g_fl[j].lo < a) a = g_fl[j].lo; if (g_fl[j].hi > b) b = g_fl[j].hi; j++; } // spans i .. j-1 overlap or touch [a, b): merged into one + if (i == j) { + if (g_nfl == g_cfl && !(g_fl = realloc(g_fl, (g_cfl = g_cfl ? 2 * g_cfl : 256) * sizeof *g_fl))) die("flushed range allocation failed"); + memmove(g_fl + i + 1, g_fl + i, (g_nfl - i) * sizeof *g_fl); g_nfl++; + } else if (j > i + 1) { memmove(g_fl + i + 1, g_fl + j, (g_nfl - j) * sizeof *g_fl); g_nfl -= j - i - 1; } + g_fl[i] = (fl_span){a, b}; + pthread_mutex_unlock(&g_fl_mu); +} +static void fl_drop(const uint8_t *a, uint64_t n) { // a pool goes: its part of every span (spans of neighbouring pools may have merged with it) + const uint8_t *b = a + n; + pthread_mutex_lock(&g_fl_mu); + for (size_t i = fl_find(a, 0); i < g_nfl && g_fl[i].lo < b;) { + fl_span s = g_fl[i]; + if (s.lo < a && s.hi > b) { // the pool lies inside the span: split it + if (g_nfl == g_cfl && !(g_fl = realloc(g_fl, (g_cfl = 2 * g_cfl) * sizeof *g_fl))) die("flushed range allocation failed"); + memmove(g_fl + i + 2, g_fl + i + 1, (g_nfl - i - 1) * sizeof *g_fl); g_nfl++; + g_fl[i].hi = a; g_fl[i + 1] = (fl_span){b, s.hi}; break; + } + if (s.lo < a) { g_fl[i].hi = a; i++; } + else if (s.hi > b) { g_fl[i].lo = b; break; } + else { memmove(g_fl + i, g_fl + i + 1, (g_nfl - i - 1) * sizeof *g_fl); g_nfl--; } + } + pthread_mutex_unlock(&g_fl_mu); +} +int nvn_flushed(const void *p, uint64_t n) { // whether the CPU flushed any of [p, p + n) + const uint8_t *a = p; + pthread_mutex_lock(&g_fl_mu); + size_t i = fl_find(a, 0); int r = i < g_nfl && g_fl[i].lo < a + n; + pthread_mutex_unlock(&g_fl_mu); + return r; +} NVN(nvnTextureInitialize, NVNboolean, NVNtexture *t, const NVNtextureBuilder *b) { const tex_builder_t *s = (const tex_builder_t *)b; tex_t *o = TX(t); // Re-initializing a texture object that is still live (without a Finalize first): release the host texture of the old serial first. SMBW re-initializes its transient render targets on the same object every frame and never Finalizes them; diff --git a/dm/gfx/nvn/nvn_mtl.m b/dm/gfx/nvn/nvn_mtl.m index c86b01b..79b3a41 100644 --- a/dm/gfx/nvn/nvn_mtl.m +++ b/dm/gfx/nvn/nvn_mtl.m @@ -1715,7 +1715,11 @@ static void tex_zero_fill(id mt) { // game never writes it again. Rhythm Heaven Groove reads each texture file into memory and makes a CPU_NO_ACCESS | GPU_CACHED pool on it (245 by its first screens), with BC7 textures // that have no packaged layout; as GPU textures of zeros its sprite shader discarded every fragment (alpha 0) and the screen stayed black. A new texture in such a pool whose storage // holds data is filled from it once, as a GPU texture that can still be rendered into (no re-upload on page writes, unlike packaged / CPU-written ones). Left out: storage of zeros -// (the zero fill does the same), COMPRESSIBLE textures (their memory is meaningless without the compression tags), multisample, depth / stencil and sparse ones +// (the zero fill does the same), COMPRESSIBLE textures (their memory is meaningless without the compression tags), multisample, depth / stencil and sparse ones. +// In a pool the CPU can write, the same holds for storage the game flushed (nvnMemoryPoolFlushMappedRange) before the texture's first use, when the GPU has not written it yet +// (nvn_flushed in hle_nvn.c: Breath of the Wild's sky scattering tables, copied into their pools and flushed; without them the sky was brownish grey). DM_NVN_FLUSH_FILL=0 leaves +// those out, as before +int nvn_flushed(const void *p, uint64_t n); // hle_nvn.c static int mem_any(const uint8_t *p, uint64_t n) { // whether any byte is non-zero (stops at the first) uint64_t i = 0, v; for (; i + 8 <= n; i += 8) { memcpy(&v, p + i, 8); if (v) return 1; } @@ -1723,11 +1727,15 @@ static int mem_any(const uint8_t *p, uint64_t n) { // whether any byte is non- return 0; } static int tex_init_fill(const tex_t *t, id mt) { - if (!t->mem || !t->pool || !(t->pool->flags & NVN_MEMORY_POOL_FLAGS_CPU_NO_ACCESS) || (t->pool->flags & NVN_MEMORY_POOL_FLAGS_VIRTUAL) || t->has_layout) return 0; + if (!t->mem || !t->pool || (t->pool->flags & NVN_MEMORY_POOL_FLAGS_VIRTUAL) || t->has_layout) return 0; if (t->d.samples > 1 || (t->d.flags & NVN_TEXTURE_FLAGS_COMPRESSIBLE) || ((int)mt.pixelFormat >= 250 && (int)mt.pixelFormat <= 262)) return 0; // 250..262: Depth16Unorm … X24_Stencil8 (some names are macOS-only) + int no_cpu = (t->pool->flags & NVN_MEMORY_POOL_FLAGS_CPU_NO_ACCESS) != 0; + static int flush_fill = -1; if (flush_fill < 0) { const char *e = getenv("DM_NVN_FLUSH_FILL"); flush_fill = !e || atoi(e) != 0; } + if (!no_cpu && (!flush_fill || !nvn_flushed(t->mem, t->L.size))) return 0; if (!atomic_load(&g_sub_thread_set) || !on_sub_thread() || !mem_any(t->mem, t->L.size)) return 0; // the upload batch belongs to the submit thread if (pkg_upload(t, NULL, mt) != mt) return 0; - static uint64_t n; if (n++ < 4) MLOG("initial content: ser=%llu %ux%u format %#x filled from its CPU_NO_ACCESS pool's storage (%llu so far)", (unsigned long long)t->serial, t->d.width, t->d.height, t->d.format, (unsigned long long)n); + static uint64_t n[2]; if (n[no_cpu]++ < 4) MLOG("initial content: ser=%llu %ux%ux%u format %#x filled from its storage, %s (%llu so far)", (unsigned long long)t->serial, t->d.width, t->d.height, t->d.depth, t->d.format, + no_cpu ? "a CPU_NO_ACCESS pool's" : "flushed by the CPU before its first use", (unsigned long long)n[no_cpu]); return 1; } diff --git a/dm/gfx/tests/t_nvn.c b/dm/gfx/tests/t_nvn.c index d8c6b0c..747468a 100644 --- a/dm/gfx/tests/t_nvn.c +++ b/dm/gfx/tests/t_nvn.c @@ -124,6 +124,39 @@ int main(void) { P(nvnTextureFinalize)(&original); P(nvnTextureFinalize)(&replacement); } + // Flushed ranges (nvn_flushed): a texture's storage flushed before its first use is filled from memory (tex_init_fill). Spans merge when they overlap or touch, and go with their pool + { + P(nvnMemoryPoolFlushMappedRange)(&pool, 0x1000, 0x100); + EXPECT("flushed: the flushed range", 1, nvn_flushed(poolmem + 0x1000, 0x100)); + EXPECT("flushed: its last byte", 1, nvn_flushed(poolmem + 0x10ff, 1)); + EXPECT("flushed: a range ending at its start", 0, nvn_flushed(poolmem + 0xf00, 0x100)); + EXPECT("flushed: a range starting at its end", 0, nvn_flushed(poolmem + 0x1100, 0x100)); + P(nvnMemoryPoolFlushMappedRange)(&pool, 0x1100, 0x100); + EXPECT("flushed: a touching flush merges (1 span)", 1, (long long)g_nfl); + P(nvnMemoryPoolFlushMappedRange)(&pool, 0x3000, 0x10); + EXPECT("flushed: a separate flush (2 spans)", 2, (long long)g_nfl); + EXPECT("flushed: the gap between them", 0, nvn_flushed(poolmem + 0x1200, 0x1e00)); + P(nvnMemoryPoolFlushMappedRange)(&pool, 0x1080, 0x40); + EXPECT("flushed: a flush inside a span (still 2)", 2, (long long)g_nfl); + P(nvnMemoryPoolFlushMappedRange)(&pool, 0, 0x4000); + EXPECT("flushed: a flush over both (1 span)", 1, (long long)g_nfl); + fl_drop(poolmem + 0x1000, 0x1000); // a pool inside the span: split + EXPECT("flushed: a pool dropped from inside a span (2 spans)", 2, (long long)g_nfl); + EXPECT("flushed: the dropped part", 0, nvn_flushed(poolmem + 0x1000, 0x1000)); + EXPECT("flushed: before it", 1, nvn_flushed(poolmem + 0xfff, 1)); + EXPECT("flushed: after it", 1, nvn_flushed(poolmem + 0x2000, 1)); + static uint8_t smem[0x2000] __attribute__((aligned(4096))); NVNmemoryPool sp; + P(nvnMemoryPoolBuilderSetStorage)(&pb, smem, sizeof smem); + P(nvnMemoryPoolInitialize)(&sp, &pb); + P(nvnMemoryPoolFlushMappedRange)(&sp, 0x800, 0x800); + EXPECT("flushed: a second pool's flush", 1, nvn_flushed(smem + 0x800, 1)); + P(nvnMemoryPoolFinalize)(&sp); + EXPECT("flushed: gone with its pool", 0, nvn_flushed(smem, sizeof smem)); + fl_drop(poolmem, sizeof poolmem); + EXPECT("flushed: the first pool's spans dropped", 0, (long long)g_nfl); + P(nvnMemoryPoolBuilderSetStorage)(&pb, poolmem, sizeof poolmem); + } + // 5. Command buffer: record uniform updates / events / counters / buffer copies / nested calls, with memory in 256 B segments to force multi-segment linking NVNcommandBuffer cb; P(nvnCommandBufferInitialize)(&cb, &dev); NVNmemoryPoolBuilder cpb; P(nvnMemoryPoolBuilderSetDefaults)(&cpb); P(nvnMemoryPoolBuilderSetStorage)(&cpb, g_cmdmem, sizeof g_cmdmem); diff --git a/docs/agents/04-graphics.md b/docs/agents/04-graphics.md index 5216421..690ae65 100644 --- a/docs/agents/04-graphics.md +++ b/docs/agents/04-graphics.md @@ -164,6 +164,7 @@ So the translator leaves z alone (`QueryGraphicsState` depth mode false in `Prog - **Frame rate in the Shrine of Resurrection on the Mac (M4 Pro)**, after the above: docked (1600×900) GPU about 13 ms, frame p50 16.7 ms (60 fps; p90 17.6 ms), was 20.8 ms before segmenting; handheld (`DM_OPMODE=0`, 1280×720) GPU p50 9.3 ms and a steady 60. Host translation is 8 ms per frame (22,000 instructions per draw), spread over Metal's encoder creation and draw calls, argument encoding, Objective-C retain / release and pool lookups. The per-pass GPU timer (`DM_NVN_PASSGPU`) is switched off under Metal 4 (`pg_render` returns at once), so GPU time cannot yet be attributed to passes. The frame loads 376 MB and stores 432 MB of render-target data (`@rt_load_mb` / `@rt_store_mb`; 325 of 347 passes load and store everything), which is the next thing to look at. - **Fixed: dark boxes and cream cloud rectangles behind HUD prompts, a black stamina wheel and a black System menu background (layout captures; GitHub PR #6).** Panes tagged `CaptureOn` (about 140 layouts, e.g. `PaGuideKeySmoke_00`; user data `A8`, `CaptureScale` 0.5, `MultiFilter`) are drawn into a target, blurred at half size and copied into a capture texture that the layout's materials sample in place of the placeholder `FBLayout_00`. The game creates that texture with its per-format default mapping (one call site, `main+0xecb0bc`: R8 → (R, 0, 0, 1), RG8 → (R, G, 0, 1), and so on), then rebuilds it on the same storage with the capture's own mapping: `main+0xf597dc` picks (1, 1, 1, R) for an `A8` capture (the material colour kept, the mask as alpha) and alpha 1 when the pane says `CaptureOutputAlpha` 255 (read at `main+0xf58e98`; an RGB10A2 capture then reads (R, G, B, 1)); `main+0xecacf0` stores the mapping and rebuilds the texture (`main+0xecabe8`), and `main+0xec33f0` replaces the registered descriptor only if `nvnTextureCompare` reports the two textures different. The host compared the description and the storage but not the mapping, so every rebuild was dropped and the captures kept (R, 0, 0, 1), whose alpha is always 1: hard-edged dark boxes behind every prompt (the shadow pane `e6a88e12dbae9743`: black, alpha = capture alpha × vertex alpha 180/255) and rectangles filled with cream cloud (the glow `aa209f57b133e862`: alpha = min(cloud alpha, capture alpha)). Before the cause was known, the profile key `alpha_from_red` listed all 26 pixel-shader variants of the layout library (`main`'s BNSH at 0x1d83000) and bound such render targets through a view mapped (R, 0, 0, R) (`tex_alpha_red` in `nvn_mtl.m`). That fixed the alpha but zeroed green and blue, so tinted panes such as the stamina wheel came out black, and the System menu background (a capture with alpha forced to 1) stayed black. `nvnTextureCompare` now compares the mapping too, an unset mapping counting as (R, G, B, A), and BotW's profile no longer lists `alpha_from_red`. Checked on the Mac by the PR's author (M4 Pro, unchanged shader libraries): green stamina, the purple noise meter and the game's own blurred System menu background. `t_nvn` covers the comparison. - **BC textures are never attached or viewed raw (GitHub issue #4, reported from an iPhone 14 Pro / A16).** The game encodes one 32×32 BC4 texture on the GPU, drawing its blocks through an RGBA16UI view (one texel = one 8-byte block). Metal can neither attach a compressed texture nor make an uncompressed view of one, so a pass that draws into it uses an uncompressed shadow (above), and `attach`, the pipeline prediction and the sampling views go by the NVN format. The A16 has no BC formats; the host used to store BC textures decompressed on such GPUs (about 3.9 GB instead of 2.8 GB for this game, over iOS's limit on a 6 GB iPhone), and that path has since been removed: only GPUs with BC are supported. +- **Fixed: the sky was brownish grey, the distant haze brown (GitHub issue #52, Mac and iPhone).** The sky is precomputed atmospheric scattering: each frame a pass (`08fe25061d583637`, `uking_pass_shader.sharcb`) turns a 256×32×16 RGBA16F 3D table of in-scattered light into a 256×256 sky map, which the sky draw (`8110c74078548172`) and the materials' aerial perspective sample. The table and two 2D ones (transmittance 256×64, irradiance 64×64) come from `System/KSys/sky.skybin` in `Pack/Bootup_Graphics.pack` (1,212,416 bytes, the three textures' storage back to back). The game makes a `CPU_CACHED | GPU_CACHED` pool for each texture, creates the textures, then copies the file's data into the pools with `memcpy` and flushes them (`main+0x1257d50`, once, at the sky's setup; the flush goes through `main+0xebf4c4`, which flushes only CPU-cached pools). The host created the 3D table as a GPU texture of zeros at its first use, so the sky had no scattered light at all: only the clouds and the sun's colour were left. `DM_NVN_TEXRAW=212,214` (all zero) and `DM_NVN_TEXINFO=256x32` (`storage 1048576 bytes non-zero 781567` at first use, `swizzled=0`) showed it. Every game now fills a new GPU texture from its storage once when the game flushed that storage before the texture's first use, as it already did for a pool the CPU cannot access (`nvn_flushed` in `hle_nvn.c`, `tex_init_fill` in `nvn_mtl.m`): the GPU has not written such a texture yet, so its content is what the CPU put there. Unlike the profile's `tex_flush_upload` it never re-uploads, so a texture the game renders into afterwards stays a GPU texture. In the Great Plateau save it fills that one texture; the flushes (about 15 a frame) merge into about 530 ranges, and frame times did not change. DRAGON QUEST XI S (title, free play, menus; 300 s) filled none. `DM_NVN_FLUSH_FILL=0` turns it off. ### Seen in The Legend of Zelda: Link's Awakening @@ -254,6 +255,7 @@ Game Freak's Trinity engine on SDK 19 (NVN 55); every shader comes from the Oodl | `DM_NVN_UB_PARTIAL=0` | A uniform block the deferred updates cover only in part ends the pass, as before. By default its shadow starts from memory and takes every inline update queued in the submission, newest last, when nothing else can have written the range by the time the pass runs: no copy, fill or transform-feedback write queued in the submission, no storage buffer a shader may write (every one but the slots `cb_extent.tsv` proves unwritten) and no texture→buffer copy over it in the submission, and no GPU write or counter report of an earlier submission still running (by 4 KB page). A counter report of the same submission does not count: the host writes reports on the CPU after the whole submission completes. The `residency set` line splits the refusals by reason | | `DM_NVN_UBP_CHECK=1` | Check every uniform shadow read from memory: the pass ends, the updates land, the replay waits for the GPU and compares (`uniform shadow check: N equal, M different`; a difference is a red line). Breath of the Wild outdoors: 388,000 equal, 0 different | | `DM_NVN_LAZY_CLEAR=0` | Clears with no draw after them open a pass of their own when the game sets other targets, as before. By default they wait: the next pass that attaches the same texture level and layer through the same view clears it as its load action, and the rest land in passes of their own before anything else could see their textures (a draw sampling one, a pass not attaching it, any operation other than targets, clears, state, uniform updates and buffer copies or fills, the end of the command list). Off in TBDR mode. The `residency set` line counts clears taken by a later pass and those landed alone | +| `DM_NVN_FLUSH_FILL=0` | A new GPU texture in a pool the CPU can write starts as zeros, as before. By default one whose storage the game flushed (`nvnMemoryPoolFlushMappedRange`) before its first use is filled from that storage once (`initial content: … flushed by the CPU before its first use` lines; Breath of the Wild's sky tables) | | `DM_NVN_BARRIER=game` / `full` | How draw passes are ordered. `full` (default): every pass waits for all earlier GPU work. `game`: a pass waits only for the previous pass's fragment stage, so its vertex work overlaps the previous pass's fragments, unless the game called `nvnCommandBufferBarrier` in between or a compute / copy / upload ran (those keep full barriers). Only vertex-stage reads of the previous pass's output without a game barrier change order | | `DM_NVN_GPU_TS=segment` / `submission` | What timestamp reports record, instead of the profile's `gpu_timestamps`: the GPU start / end of the segment holding each report, or one time for the whole submission | | `DM_NVN_DISPATCH_OWN=1` | Every compute dispatch in a compute encoder of its own with full barriers, as before (by default dispatches share the copy encoder, `dispatch_enc`) | diff --git a/docs/development.md b/docs/development.md index 56e2397..a35e2a7 100644 --- a/docs/development.md +++ b/docs/development.md @@ -195,6 +195,7 @@ The game's own work and its frame rate stay as they are; the host avoids work of - **The world-load crash (about 1 run in 3 on the Mac) came from the host's command record sizes.** About 3 s after Start, a task worker's `operator new` (`main+0x155a79c`) got null and the game wrote through it (`main+0x155a7a4`). Each model file gets a heap of its resource size table entry + 0x1008; the file goes in 4 KB aligned, and the model's setup (`main+0x155a2ac`) takes the rest as a frame heap. Part of that setup is GPU memory for the model's prerecorded commands, which the game sizes by recording each mesh's draw and each shape's state for each render pass into a scratch command buffer and adding up `nvnCommandBufferGetCommandMemoryUsed`, so the host's own record sizes count against the table. Draw records had grown to carry the indirect and multi-draw fields (0x38 → 0x50 per recorded draw on 2026-10-18 and 10-20), and every recording began with an 8-byte BEGIN record. `Obj_TreeConiferous`, `Obj_TreeBroadleaf_B` and `FldObj_HyliaBridge_A-02` then needed one 4 KB page more than their entries allow, which fit only when the file's alignment pad happened to be small (about 15% of loads each failed). Plain draws now leave those fields out and recordings have no BEGIN (0x30 per recorded draw, `nvn_obj.h` `draw_rec`): all 510 models of the world load fit whatever the pad. How it was measured: [03](agents/03-bringup.md). - **Memory:** about 2.8 GB in the first shrine in handheld mode, idle or walking (Mac measurement), against the 3.4 GB an 8 GB iPhone gives the app. - The game waits for each frame's GPU work before submitting the next, so the host hands its translated commands to the GPU in segments ([04](agents/04-graphics.md)). +- **Sky (GitHub issue #52):** outdoors the sky was brownish grey and the distant haze brown, on the Mac and the iPhone. The game copies its precomputed sky scattering tables (`System/KSys/sky.skybin`) into memory pools with the CPU and flushes them; the host made the 3D table a GPU texture of zeros. Every game now fills a texture from its storage at first use when the game flushed that storage before then ([04](agents/04-graphics.md)); the sky is blue again. - **HUD prompt shadows and glows, stamina wheel, System menu background:** the game rebuilds each layout capture (a blurred mask of a prompt bar, the menu's background) on the same storage with its own channel mapping and keeps the new texture only if `nvnTextureCompare` calls the two different; the host's comparison ignored the mapping, so captures read alpha 1 (hard-edged dark boxes, cream cloud rectangles, a black menu background). It compares the mapping now ([04](agents/04-graphics.md)). **The Legend of Zelda: Link's Awakening (`01006BB00C6F0000`)**