nxnative-easy/docs/agents/04-graphics.md

04 — Graphics: NVN on Metal for a new title

The graphics layer records NVN commands into its own command format (dm/gfx/nvn/hle_nvn.c) and replays them as Metal 4 at submit time (nvn_mtl.m). Shaders are never translated at run time: the original Maxwell shader binaries are translated offline (Ryujinx's translator → SPIR-V → spirv-cross → metallib) and looked up by key. See dm/gfx/README.md for the full pipeline; this page lists what changes per title.

1. Find every shader container

Do not stop at the obvious archive. In Mario the main bfsha held 4,888 of the 12,811 shaders the game actually uses.

dm/gfx/tools/shader_split.py <RomFS> <ExeFS main> <pkg dir> <map out> <facts out> [<pairs out>] scans every RomFS file (zstd-decompressed, SARC expanded recursively) and main, and recognizes the containers listed by engine in reference/shaders.md. The pairs table lists the (vertex key, pixel key) of every BNSH variation or Unity program. Unity shaders need a second pass: files whose type trees are missing are scanned again once the bundles have supplied the Shader tree (or with the one unityfs.py has written out for that hash, when no bundle has it). For a base game plus an update, give RomFS and main as comma-separated lists, base first: keys are content hashes, so the output is the union of both versions and one shader build serves both (Mario 1.2.1 recompiled 3,451 of its pixel shaders; v0 rows keep their package indices).

Validate a new title's coverage at run time: DM_NVN_SHDUMP=<dir> makes the host dump the control section and code of every program it can't find in the map; search those bytes in the game files to find the container that was missed.

2. Key shaders by control section + code

The host key is sha256(control[0:0x880] + code)[:16] (prog_get in nvn_mtl.m, shader_split.py). D2R keyed by the control section alone; in Mario, 6,367 shader pairs share a control section but differ in code. A different code length or layout in a new title must be reflected in both places.

3. Find the binding layout by census

The NVN driver puts its binding tables in driver constant banks. Every game so far, NVN 53 and 55 drivers from SDK 6 to 21, has the one layout below, which dmsh_build.py decodes and nvn_mtl.m fills at run time. Check a new game against it with a census of what its translated shaders reference (the translator alone takes seconds) rather than assuming it.

Binding	Where	How it was measured
Combined texture handles (nvnCommandBufferBindTexture)	c2[0x20 + 8 × slot], 32 slots; a higher slot's word lies in the image table	Mario: immediate handles of all TEX instructions, 0x20…0xF0 in steps of 8. Hades II binds slot 34 (0x130) and no image
Image handles	c2[0x120 + 8 × slot], 8 slots	agl compute shaders: 0x120 / 0x128 / 0x130
Separate textures (BindSeparateTexture(s))	c2[0x168 + 8 × slot], 128 slots: the host's texture slots 128 + n (NVN_SEPTEX_SLOT0)	Engage (Unity binds only separate textures and samplers): the translator's texture names (fp_t_cb<slots>_<handle>, Ryujinx TextureHandle packing) give word 0x5A = 0x168 for the texture and 0x15A = 0x568 for the sampler. D2R (NVN 53) had the same
Separate samplers	c2[0x568 + 8 × slot], 128 slots	as above
Uniform buffers	c[3 + slot]
Storage buffers	c0[base + 16 × slot], base vs 0x110 / cs 0x310 / ps 0x510
Largest point size	c0 + 0x610	Super Mario Odyssey (SDK 7): GLSLC clamps a shader-written gl_PointSize to it (alPointSpriteShader: min(size, c0[0x610])). It is not the SetPointSize value, which the game leaves at 1. The host writes 2047 there for the graphics stages

Ace Attorney Investigations Collection (Unity 2022.3, SDK 17.5.3) binds separate textures and samplers like Engage: all 4,950 translatable shaders decode every binding (shaders with a binding whose slot was not decoded 0), and 190 read c2 rows as data.

Metroid Dread (SDK 11.4.3) uses Mario's texture handle layout, 0x20 + 8 × slot: the TEX / TEXS handle immediates in its shader files' own SASS text are words 0x8–0x22 in steps of 2. Its full build leaves no unresolved bindings and no unfillable driver-bank reads; the other rows are not measured separately.

Hades II (SDK 19.3.9) binds combined textures at fixed, sparse slots up to 34 (fp_t_tcb_4C: c2 word 0x4C = 0x20 + 8 × 34, in SceneFade.frag). With a 32-slot table, dmsh_build.py reported that binding as not decoded ("shaders with a binding whose slot was not decoded 1", which prints a red line when the shader loads). The combined table now runs up to the separate textures at 0x168 (41 slots); past slot 31 it shares its words with the image table, and the replay writes a bound texture there when the image slot is unbound. The census of fp_t_* names in the translated libraries gives the slots in use.

Monster Hunter Rise (RE Engine, SDK 11.4.3) uses both tables at once: combined handles from nvnCommandBufferBindTexture at 0x20 + 8 × slot and separate textures from BindSeparateTexture(s) at 0x168 + 8 × slot, with separate samplers at 0x568, and binds slot 0 of both in the same frames. A census of 23,478 translated shaders' texture names: 33,813 combined (fp_t_tcb_1E), 29,342 separate textures with a constant sampler ID (handle type 3, fp_t_cb65538_3041005C: texture at c2 word 0x5C, sampler 0x104 ORed in), 8,265 separate texture + separate sampler (type 1, fp_t_cb196610_1017815A), 5,227 separate textures whose own handle carries the sampler (fp_t_tcb_5A), about 1,500 uniform-buffer handles with a constant sampler (fp_t_cb65541_30410028), and 121 fully dynamic handles (type 4, fp_t2d_cb0_40000000, not supported). Separate binds go to slots 128 + n of the host's table and their handles decode to 128 + n, as for every game; a constant sampler is the binding kind samp_id (NVN_RB_SAMPI, reg = the sampler ID).

dmsh_build.py decodes the bindings with this layout (decode_tex), Program.cs numbers slots the same way when it looks up per-slot texture / image types in the facts table, and the replay layer fills driver banks with it: the combined, image, separate texture and separate sampler tables of c2 (NVN_RB_DRV2 in encode_args). Some shaders read c2 rows as data at run time (Engage: 243, the texture and sampler handles of slots 0–11); the build lists reads outside those tables as cannot fill. After a full build, the summary must say shaders with a binding whose slot was not decoded 0.

4. Bindless textures

Mario's shaders read 38 % of their textures through handles stored in the game's own uniform buffers (fp_t_cb7_2A = handle at word 0x2A of c[7] = uniform slot 4). These became binding kinds tex_ubo / samp_ubo (NVN_RB_TEXU / NVN_RB_SAMPU): at replay the host reads the 32-bit word from the bound uniform buffer (low 20 bits texture ID, high 12 bits sampler ID).

Consequence: any optimization that infers "which textures a draw reads" from bound slots is wrong for such shaders. The TBDR optimizations (store / load elision, pass merging) are therefore off by default in this repository (tbdr_on(), DM_NVN_TBDR=1 to enable).

5. Depth convention

nvnDeviceSetDepthMode near = −w (GL, z in [−w, w], NVN's default) versus near = 0 (D3D / Metal, z in [0, w]). Metal clips anything with z < 0, so a GL-mode game's vertex shaders must end with z' = z/2 + w/2. The mode is runtime state, and games differ: Mario, Metroid Dread, Super Mario 3D World and Breath of the Wild keep GL; Link's Awakening, 13 Sentinels and Engage set near = 0.

So the translator leaves z alone (QueryGraphicsState depth mode false in Program.cs), and dmsh_build.py (vs_depth) writes two entry points into every vertex library: main0 with the GL rewrite (the same arithmetic Ryujinx emitted) and main0_z without it (reflection entry_z). The host picks one when it loads a vertex shader, from the mode the game set (mtl_set_depth_zero, shader_load in nvn_mtl.m). A new game needs no setting; if a game changed its mode after vertex shaders were loaded, a red line says so.

• Rewriting a near = 0 game for GL still draws correctly (depth tests keep their order), but every depth lands in 0.5..1, so effects that turn depth back into distance are wrong: Link's Awakening's outdoor depth of field blurred the whole screen.
• Not rewriting a GL game: the full-screen copy into the window is clipped away, so the window stays black even with a solid-colour fragment shader.
• Metal function constants could pick the variant too, but any function with a function constant has to be specialized before a pipeline can use it (measured: about 2.4 ms per vertex shader on an M4 Pro); the second entry point only costs build time and about 30% more library size.

6. Where the per-title findings are

What earlier titles needed beyond sections 1–5 is filed by mechanism: textures, passes, clears, vertex input, uniforms, conditional rendering and the Vulkan and OpenGL layers in reference/rendering.md; containers by engine, control sections, translation, bindings and run-time shaders in reference/shaders.md; memory pools, command memory, queues, syncs and CPU-written textures in reference/nvn-objects.md; present intervals and display rates in reference/frame-rate.md. Each entry names the games it was found on, and each game's own findings (its passes, shader keys, measurements, profile keys) are in its note, dm/games/<id>/README.md (games table). Search symptoms.md for what you see first.

7. Diagnostic switches (host)

Switch	Output
DM_NVN_SHOT=f1,f2:path_%d.png	Window screenshots at those presented frames
DM_NVN_PASSDUMP=f or f1-f2 (+ DM_NVN_PASSDUMP_ALL=1)	Every render pass of those frames: P attachments (with mtl= the Metal texture behind them), C clear colours, K shader keys, B each binding's actual source (also with mtl=), d/D draw state (b0–b3 the blend states of targets 0–3; on d, po= the polygon offset enables / factor, units, clamp and dc= depth clamp), Y inline updates, X texture clears, T texture copies, Z the game's barriers (their bits, and whether one ended the pass), N passes skipped because no attachment could be attached, C buffer-to-texture copies (target, size, format, region, source address, row and image strides), W image bindings of compute dispatches; a buffer texture's B line gives its format, element count and storage address (for the Mac's DM_CTL memdump). Without _ALL the K / B / R lines cover only the first draw of each pass
DM_NVN_TEXDUMP=f:path_%d.png (+ DM_NVN_TEXSNAP=i,…)	Every colour render target of frame f, as PNG (%d = serial × 100 + layer × 10 + level; 2D arrays by the attached layer). TEXSNAP copies the attachments of in-frame pass i (2D, or slice 0 of a 2D array) in a separate command buffer, which does not run at the pass's position in the GPU timeline: a snapshot taken right after a clearing pass showed a complete image, so read it as "what this target holds around the frame", not "right after pass i"
DM_NVN_TEXDUMP=size=WxH:path_%d.png	The same dump for the first frame that opens a render pass whose first colour attachment level is W×H: for one-off work (a mask built when a prompt appears) whose frame differs between runs. With DM_NVN_SNAPRAW=1 each image also gets a .raw file with the alpha channel
DM_NVN_CBDUMP=dir (with PASSDUMP)	Raw uniform buffer contents per binding, and the source bytes of every buffer-to-texture copy into a 3D texture (F<frame>_c3d_<serial>.bin: what the game uploaded, before any layout question)
DM_NVN_HILITE=fskey,…	Replace those fragment shaders with solid colours: tells "geometry is wrong" from "shading is wrong"
DM_NVN_DEPTHOVR=fskey,…	Those fragment shaders' draws compare depth with ALWAYS: with HILITE, tells "the depth test rejects every fragment" from "nothing rasterizes". With PASSDUMP, the D line shows the depth range (dr=) and logic op (lo=), and c lines each recorded clear (slot, serial, mask, scissor; depth / stencil values)
DM_NVN_SHDUMP=dir	Control + code of programs missing from the shader map
DM_NVN_ALLOCPROF=1	Metal allocation growth per replayed op type and per function, command allocator sizes, new textures
DM_NVN_CBDUMP_KEY=vskey (with CBDUMP and PASSDUMP_ALL)	Only that vertex shader's draws: uniform buffers plus vertex streams and index buffer (F<frame>_p<pass>_vb<stream>_<addr>.bin, _ib<type>_<addr>.bin)
DM_NVN_TEXINFO=WxH	Storage and description (pool, offset, flags, packaged or not, storage size and how many of its bytes in guest memory are non-zero, i.e. written by the CPU) of every texture of that size on first use, each pool registration of one (TEXINFO register: builder channel mapping, view and view mapping), and each one as it is initialized (TEXINFO init, render targets included, the first 200: serial, pool and offset, target, format, flags, storage size)
DM_NVN_ZBLOCK=1 or =<folder>	Each ASTC texture uploaded on the CPU whose bound levels hold all-zero blocks (ASTC's error colour, magenta): [zb] lines with each level's zero blocks, its total and whether its pages are mapped (sparse pools). With a folder, also the raw storage of the first one of each format and size (zb_<serial>.bin; 2 MB and a byte-per-4 KB-page map zbm_<serial>.bin for sparse textures)
DM_NVN_TEXRAW=info (with TEXDUMP)	Raw bytes of the non-render-target textures TEXINFO picked out, at the TEXDUMP frame
DM_NVN_TEXRAW=ser,… (with TEXDUMP)	Raw bytes of those textures (by serial, as PASSDUMP prints them) at the TEXDUMP frame; a Depth32Float_Stencil8 texture gives its 32-bit float depth plane (a shadow map)
DM_NVN_T2T_TRACE=WxH:file (+ DM_NVN_T2T_FROM=f)	After each texture copy into an RGBA16F texture of that size, a GPU-timeline copy of the result into a trace buffer; 240 entries written to file (header line, 240 frame numbers, then the texels)
DM_TEST_PRESENT_LAG=n,ms	Sleep ms before every nth present: a slow frame on demand (Mario then switches to present interval 2)
DM_NVN_BLIT_ORDER=0	No barriers between dependent operations in the copy encoder (reproduces the fountain bug)
DM_NVN_KEYDUMP=<fs key>[:n[:every]]	For up to n draws of that fragment shader (default 6; at most one every every frames), the PASSDUMP binding lines plus [kd] lines: vertex attribute formats, each vertex stream's data (all instances of per-instance streams) and every value of both stages' uniform buffers. Meant for devices: no frame number to guess; compare with the same switch on the Mac
DM_NVN_MAGSCAN=<fs key>[:every[:phase]]	Device self-check for solid colour blocks. Every every frames (default 30) a compute kernel reads each render pass's colour attachments as the pass ends and logs [mag] lines for 16×16 blocks that hold one magenta value throughout, or non-finite pixels, naming the pass relative to the one holding that shader. Every phase frames (default 600) it rotates: 0 draws as usual, 1 drops that shader's draws, 2 / 3 keep its geometry with a constant grey / bright magenta fragment output. Each phase ends with a summary line, including the magenta pixel count in that shader's own pass
DM_NVN_PREDICT=0	No parallel precompile prediction: a pipeline miss compiles only the pipeline it needs. For telling a fault of the look-ahead from one of the pipelines (reference)
DM_NVN_FB_SNAP=0 / all / <keys>	Which fragment shaders get the snapshot for same-pass feedback sampling (a draw sampling a colour attachment of its own render pass reads a copy taken just before it): none, every shader (slow, see BotW above), or a comma-separated key list instead of the profile's feedback_snap
DM_NVN_ALPHA_RED=0 / <keys>	Which fragment shaders read single-channel render targets mapped (R, 0, 0, 1) with alpha = red: none, or a comma-separated key list instead of the profile's alpha_from_red (Odyssey's layout shaders; BotW no longer needs it since the nvnTextureCompare fix, see above)
DM_NVN_CUT=<draws>	Commit the replay's command buffer every that many draws, between passes (default 64; 0 = one command buffer per submission, except the cut at the first pass boundary after a buffer → texture copy, which always happens: Seen in Catherine: Full Body). Growing segments (64, 128, 256 … draws) were tried for fewer commits: Breath of the Wild fell from 60 to 30 fps in the same scene (GPU idle while the long last segments were encoded), and the commits cost little once the residency set stopped churning
DM_NVN_UB_DEFER=0	End the render pass at every inline uniform update (nvnCommandBufferUpdateUniformBuffer), as before. By default an update inside an open pass is deferred to the pass end; the draws after it read a shadow made of the deferred updates when they cover all the shader can read of the binding (cb_extent.tsv), or of memory and the updates queued in the submission when they cover part of it (DM_NVN_UB_PARTIAL), and the pass still ends first for a draw that needs more (the data read as vertices, indices, indirect arguments or a storage buffer, or a partly covered block memory cannot give). The residency set line counts deferred updates, shadows read from memory and early ends
DM_NVN_UB_PARTIAL=0	A uniform block the deferred updates cover only in part ends the pass, as before. By default its shadow starts from memory and takes every inline update queued in the submission, newest last, when nothing else can have written the range by the time the pass runs: no copy, fill or transform-feedback write queued in the submission, no storage buffer a shader may write (every one but the slots cb_extent.tsv proves unwritten) and no texture→buffer copy over it in the submission, and no GPU write or counter report of an earlier submission still running (by 4 KB page). A counter report of the same submission does not count: the host writes reports on the CPU after the whole submission completes. The residency set line splits the refusals by reason
DM_NVN_UBP_CHECK=1	Check every uniform shadow read from memory: the pass ends, the updates land, the replay waits for the GPU and compares (uniform shadow check: N equal, M different; a difference is a red line). Breath of the Wild outdoors: 388,000 equal, 0 different
DM_NVN_LAZY_CLEAR=0	Clears with no draw after them open a pass of their own when the game sets other targets, as before. By default they wait: the next pass that attaches the same texture level and layer through the same view clears it as its load action, and the rest land in passes of their own before anything else could see their textures (a draw sampling one, a pass not attaching it, any operation other than targets, clears, state, uniform updates and buffer copies or fills, the end of the command list). Off in TBDR mode. The residency set line counts clears taken by a later pass and those landed alone
DM_NVN_FLUSH_FILL=0	A new GPU texture in a pool the CPU can write starts as zeros, as before. By default one whose storage the game flushed (nvnMemoryPoolFlushMappedRange) before its first use is filled from that storage once (initial content: … flushed by the CPU before its first use lines; Breath of the Wild's sky tables)
DM_NVN_BARRIER=game / full	How draw passes are ordered. full (default): every pass waits for all earlier GPU work. game: a pass waits only for the previous pass's fragment stage, so its vertex work overlaps the previous pass's fragments, unless the game called nvnCommandBufferBarrier in between or a compute / copy / upload ran (those keep full barriers). Only vertex-stage reads of the previous pass's output without a game barrier change order
DM_NVN_GPU_TS=segment / submission	What timestamp reports record, instead of the profile's gpu_timestamps: the GPU start / end of the segment holding each report, or one time for the whole submission
DM_NVN_DISPATCH_OWN=1	Every compute dispatch in a compute encoder of its own with full barriers, as before (by default dispatches share the copy encoder, dispatch_enc)
DM_NVN_VCONV_CPU=1	Integer vertex inputs fed formats of another class are converted on the CPU, as before (by default the shader's <key>_bv library takes them through shadow inputs, vin_on)
DM_NVN_GPUPROF=f or f1-f2	GPU start and duration of every encoder of those frames, from Metal 4 counter heap timestamps ([gpuprof] lines: kind, size, attachments, draws, command-buffer segment; a frame line with render and compute / copy time and the span from first start to last end)
DM_NVN_ENC_THREAD=1	Metal encoding on a thread of its own: the submit thread records the encoder calls and a worker replays them (nvn_rec.h); on by default on iOS (DM_NVN_ENC_THREAD=0 turns it off), off on macOS
DM_NVN_UB_LAZY=n	The most deferred inline uniform updates kept across a render pass end (default 160): they land before the next copy, a draw or dispatch that reads their memory other than through uniform bindings the shadows cover (ubd_conflict, ubd_conflict_cs), a transform feedback draw, or the end of the submission, instead of at every pass end; 0 lands them at every pass end, as before
DM_NVN_UB_COPY_THREAD=0	Uniform shadows of 512 bytes or more are copied on the submit thread, as before (by default a worker copies them)
MTL_DEBUG_LAYER=1 MTL_DEBUG_LAYER_ERROR_MODE=nslog	Metal validation without aborting

A first-frame bring-up that worked for Mario: screenshot is black → TEXDUMP shows which targets have content → PASSDUMP the pass that writes the window → HILITE its fragment shader → CBDUMP its uniforms → read the translated .metal of its vertex shader.