main
29
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 seeddsandv0lumehi
|
92fc2f2711 |
Build and ship offline Metal pipeline archives (#226)
Compile eligible shader-pair bases on the Mac with metal-tt, package one archive per selected iPhone GPU, and use a matching bundle before falling back to on-device preparation. Carry the profile depth mode through packing and include the pulling libraries required by the bundled bases. Integrate PR #226 with four confirmed fixes: validate archive container bounds before invoking Metal; require at least one actual successful bundle probe; hash consumed library bytes into offline cache keys; and emit device remainder lists for constant-only plans instead of aborting packaging. Keep game measurements in the game note and document the general findings. Validation on current main: 52 real Metal bundle checks, 57 archive checks, 79 prewarm checks on each pipeline path, 73 Python regressions covering offline tools, packaging, documentation and chain checks, and compilation of all 133 iOS source files. No new iPhone gameplay validation. The archive guard validates container structure, not private Metal payloads. Co-authored-by: v0lumehi <aljic.zijad@gmail.com> |
||
|
seedds
|
02e40ba753 | Record threading audit and retain verified game workarounds | ||
|
seedds
|
3a993fa3fe |
Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in <game>" subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games/<id>/README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv)
|
||
|
seedds
|
846c54c63c |
Per-game notes beside each profile: dm/games/<id>/README.md for all 58 games, and the games table in dm/games/README.md. Each note says what the profile needs and why (a table of its game.tsv keys, patches.py cuts and frame-step stubs, sdk_patches.py cuts for its sdk build, manual shader rows), the game's own findings by topic (frame rate, memory, rendering, audio, movies, services, files and shaders, input, Mac routes), each marked current or resolved, a "Found on this game" list of the general host changes first needed there (one line each, linked to where they are explained), and what is still open, including stale profile comments and figures the sources disagree on. The notes gather what was spread over AGENTS.md, docs/development.md (Game notes and Power), docs/agents/01-05 and the profiles' own comments. The Game notes section of docs/development.md (523 lines, 53 games) is now a pointer to the table; development.md goes from 33,300 to 8,600 words. README.md, AGENTS.md and the Breath of the Wild link in 04-graphics.md point to the notes. A README.md in a profile folder changes no shader cache key (dmsh_chain.inputs hashes only the named tables and keys) and no profile listing (game.py lists folders with a game.tsv). Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (five file patterns now appear as the full names they stood for), and every relative link and heading anchor in the repository's Markdown resolves
|
||
|
seedds
|
86d9ba1239 | One translation chain version for every game: DMSH_CHAIN in dm/gfx/nvn/dmsh_chain.h replaces the 47 dm/games/<id>/dmsh_chain.h, which every chain change rewrote and some were left behind (GitHub issue #78). The version hashed each profile's shader tables and nvn_* / shader_* keys too, once the only thing that kept the host off a library built from older tables; each shader's cache key (dmsh_build.py shader_key) has covered them since, so the version now covers only the chain's tools (dmsh_chain.py, dmsh_build.py, dmsh_fetch.py, dmsh_vin.py, dmsh_geometry.py, dmsh_facts.py, the translator's source, ios_metal.py, the lock) and the options, and needs no DM_GAME. What the per-game part still did is done by the library itself: a full dmsh_build.py run stamps b-<V>/inputs.tsv (Mac and iOS) with the game and the digest of the profile's inputs (dmsh_chain.py inputs / --inputs: the four shader tables, bf_want.tsv, which the variants use but the version never counted, and the keys), starts from no shader outputs (a partial one from none of the keys it builds), so a key that left the map, a shader that no longer builds or a variant no longer made keeps no old library, refuses a directory stamped for another game, and writes the profile's c1_full.tsv and cb_extent.tsv from its Mac outputs; a partial build keeps the stamp only while it names the current inputs. c1_guard.py / cb_extent.py head their table with the library's chain version and stamp (table_head), and the host takes a table only beside that stamp, logs the stamp and reports one of another game; build_ipa.sh reads the one header and stops when the iOS libraries' stamp is not this profile's current inputs or c1_full.tsv names another. NSP script: chain_header writes the one header when stale, before any host is built; provisional and derived profiles no longer write one; chain_tables makes a table again when it does not name the reused library's stamp (False without one). Makefiles take the header beside nvn_mtl.m; Xcode projects prepared earlier point at a profile header that is gone and are told to prepare again. Tests: test_nsp_chain.py (the committed header is current, stale and missing ones are written, tables against the stamp). Checked on The Great Ace Attorney Chronicles: a full build gave 460 of 460 shaders and both tables (0 and 409 entries, as before), the Mac host booted with the stamp and both tables past frame 900; one added facts row rebuilt one shader under the same chain version with a new digest and new tables, and the host refused the old tables beside the new stamp; a full build removed planted outputs, another game's build into that directory stopped, a partial build after an input change dropped the stamp; build_ipa.sh stopped on a stale stamp, a missing one and a stale or old-form c1_full.tsv; three NSP packs built (no table lines: dmsh_build made them), reused everything, and remade deleted tables from the reused build. Graphics unit tests give the same results as before (on this Mac t_refl needs ~/dm-work's libraries and several GPU tests fail on the previous commit too). AGENTS.md's chain procedure (header once after a tool change; a full shader build makes the tables), dm/games/README.md, dm/gfx/README.md and docs/development.md say so | ||
|
seedds
|
f4f0a85d6e | patch_nso.py split into the general rewriter and per-build patch files (GitHub issue #63): every game's bring-up edited the one file, so PRs for unrelated games conflicted. dm/tools/patch_nso.py (1,711 → about 1,100 lines) keeps the general passes and the driver; the instruction encoders move unchanged to dm/tools/nso_asm.py; a game's own patches (CUTS, OBS and the frame-step stubs, now FRAME_STEP = {build ID: stub(ctx)}) move verbatim with their comments to dm/games/<id>/patches.py (16 profiles: Tears of the Kingdom, Breath of the Wild, Age of Calamity and Paper Mario: The Thousand-Year Door with their stubs), and the cuts of Nintendo's sdk module builds, 6 of the 15 shipped by 2 to 4 games, to dm/tools/sdk_patches.py. The rewriter loads every patch file, whatever the game, and applies those of a module's build ID at the points it did before (cuts before the scan, gates after the rewrite, stubs after the gates); a build ID claimed twice stops it, and its command line needs no DM_GAME. build_src.sha256 lists the rewriter's files and every patch file (the same for every module). NSP script: the rewrite step's reuse key counts nso_asm.py and only the patch files that name one of the game's builds (patch_files), profile_files leaves patches.py out (only the rewrite step reads it), and a derived profile gets none. dm/tools/tests/test_patches.py: no build claimed twice, the entries' forms, a game's file names only its own profile's non-sdk builds, sdk_patches.py only sdk builds some profile ships, and the reuse key's patch files. Checked byte-identical: every extracted module on the game drive (131 builds) rewritten with the previous and the new rewriter gives the same outputs but build_src.sha256, 28 of the 30 patched builds among them (all four frame-step mains), and 7 fail alike (rtld, and New Super Mario Bros. U Deluxe's, which no profile rewrites); the other two (Super Mario Bros. Wonder's mains and Absolum's, no game files here) by their CUTS / OBS data, equal to the previous tables. AGENTS.md, docs, dm/games/README.md, the profiles' comments and the host comments that named a game's cut in patch_nso.py now name its patches.py or sdk_patches.py | ||
|
seedds
|
4c7adf4619 |
c1_full.tsv and cb_extent.tsv are no longer committed: each checkout makes them from its own shader build's Mac outputs (git ignores them in dm/games/*/), as the NSP script does after its iOS shader build. They follow from the MSL this Mac built, every chain change rewrote about 90 of them, and 56704a0 could only carry most over unchecked; 81 files, 18.7 MB. The NSP script makes a missing one as a stale one (chain_tables now says whether both name the chain: with no Mac outputs a reuse builds again and a build stops), and no step's reuse key counts them (profile_files), so the run after they were made does not split again. build_ipa.sh stops for an NVN game whose c1_full.tsv is missing or names another chain, with the c1_guard.py command, before generating anything (the host would give every c1 its own 64 KiB). Checked on The Great Ace Attorney Chronicles from a checkout without them: the first pack made both (0 and 409 entries, as committed), the second reused the split and the iOS shaders with no table lines, a third with them deleted made them from the reused build's Mac outputs; build_ipa.sh stops on a missing and a stale c1_full.tsv. AGENTS.md's chain procedure, dm/games/README.md, dm/gfx/README.md and docs/development.md say so
|
||
|
seedds
|
56704a0be1 |
Super Mario Galaxy 2's half-white Mario on the iPhone (GitHub issue 72): min / max / clamp with a NaN operand give the other operand on every GPU, in the Vulkan, GL and NVN shader chains
- Cause: Galaxy 2's GX vertex lighting (Hagi's SPIR-V) normalizes the light directions the game leaves at (0, 0, 0) for its characters' key and fill lights and takes fast::max(0.0, dot(ldir, NaN)) as the spot term. fast:: leaves a NaN operand's result to the GPU: the Switch's and the M4 Pro's return 0, so the attenuation is 1; the A20 Pro's did not, and the half of Mario that the fill light faces came out white (the fill light sits at the camera's left in view space, so its terminator runs down the middle of a character facing the camera). Found on the Mac by dumping the light uniforms of the character draws (every one has both directions at zero) and a Metal test of the expression on the M4 Pro. - vk_shaders.py, gl_shaders.py and dmsh_build.py (precise): spirv-cross's fast::min / max / clamp are written as precise:: (IEEE fmin / fmax, which give Maxwell's FMNMX and saturation results); the iOS Vulkan and GL libraries are compiled with -fmetal-math-mode=relaxed, as the NVN chain's are. Plain or fast:: min / max / clamp compile to the fast_ intrinsics in every Metal math mode, so relaxed alone does not do it. The NVN chain had the same calls, unguarded by its isnan selects (Paper Mario: The Thousand-Year Door: 124,000 fast::max, 98,000 fast::clamp, 41,000 fast::min). - dmsh_geometry.py capture: clip distances as spirv-cross writes them (gl_ClipDistance [[clip_distance]] [8], and gl_ClipDistance_N [[user(clipN)]] for the pixel stage); since |
||
|
seedds
|
626e6a6188 |
Luigi's Mansion 3 plays on the Mac at 60 fps: profile 0100DCA0064A6000 (update 1.4.0, SDK 7.3.2)
- Files and shaders: Next Level Games' engine keeps every RomFS file but the Wwise banks in archives of its own, <name>.dict indexing zlib chunks in <name>.data; update 1.4.0 ships <name>.patch beside the unchanged base files, a variant bitmask and per-section records that rebuild each section from base and patch bytes (the game's algorithm, main+0x1c8d0 / 0x1cbd0). shader_split.py nlg_shaders reads both: 5,317 shaders (5,149 in the base game), all in init, control sections back to back in one section and their code areas in the same order in another, one NVN program per stage (SINGLE_STAGE '#NLG'); 46 of the first 103 the title uses exist only in the update (docs/agents 01)
- Shader chain, for every game: a texture array over the driver table c2 (Ryujinx's texture array from a buffer, fp_t2das_cb2_0: 586 shadow-cascade pixel shaders indexed by (0x80 + cascade offset) / 8) is named tex_c2a / samp_c2a with its c2 offset (dmsh_build.py c2_array), and the replay fills element k from the slot whose c2 word lies at that offset + 8k (nvn_refl NVN_RB_TEXA / NVN_RB_SAMPA, nvn_mtl.m c2_handle). A pixel shader that writes an output from integer bits, or nothing but constants, gets a second entry point main0_u whose colour outputs are the unsigned vectors of the same bits (dmsh_build.py fs_uint); a pipeline whose written colour attachments are all unsigned integer formats takes it (pso_all_uint). The G-buffer is R32Uint + RGBA8Uint: about 1,400 pipelines failed to build and every scene draw was skipped. Every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (the change touches no c1 or uniform read); Luigi's Mansion 3 built for Mac and iOS
- Host, for every game: the game patches texture handles into its recorded command buffers in place, at the command start + 0x10 it took from GetCommandMemoryUsed before nvnCommandBufferBindTexture (main+0x7c8964; the store at main+0xb7d0d4), so a single texture bind is recorded in the hardware's layout, {stage, slot, handle at +0x10, texture / sampler pool bases} (OP_BIND_TEX1, nvn_obj.h bind_rec decodes it at every replay site); the host's own layout put the patched handle over the bind count, the post passes read stale handles (a stretched icon over the title, a white screen in a new game) and a corrupted record now gets a red line. t_nvn: the handle patched at the command start + 0x10 is the one replayed (102 checks)
- Host, for every game: a depth texture drawn through a colour view of the same pixel size (a D16 1024x1024 texture through an R16 view; Metal has no colour view of a depth texture, and the boot aborted in pipeline validation) draws into a colour shadow, copied in before and back after the pass as a compressed target is (nvn_mtl.m shrt_block, D16 and D32F); nvnCommandBufferSetDebugLabel; nvnBlendStateSetAdvancedMode (red line for a mode other than none); a draw whose program has a geometry shader with no library gets a red line naming it (it is drawn without that stage, with the vertex shader's positions as clip coordinates)
- Frame rate: 60 fps (30 on the Switch). The frame limiter (main+0x7a3e70, a frame callback called with the milliseconds since the frame began, main+0xb7c308) runs jobs or sleeps until 33.333 ms have passed, a float in read-only data (main+0x13c94dc), and the game steps by the time that really passed: the same 2 s walk and 1.5 s turn from the same save end at the same spot at 30 and 60 fps, and the title's animation matches at time scale 1.00 (DM_NVN_REGLUM). hle_fpslock.c gets rows with no pointer (the field at main + offset; the loader leaves read-only data writable, the image's data block on iOS too) and a frame-time kind: 1000 / fps_target, at 60 less 0.5 ms, since the work after the limiter made an exact 16.667 ms 59.6 fps with a frame shown twice every 2.5 s; slightly shorter, the presents set the pace (lobby 60.0 fps, display interval p99 18.69 → 16.67 ms). Profile fps_target 60 with present_interval display (the game also asks for interval 2); DM_FPS_TARGET=30 keeps the game's 33.333 ms
- Not done: the tessellation stages (15 hull, 18 domain shaders: two programs draw twice a frame for about 30 frames, in the title scene behind the file select as it slides out before a new game; skipped, since Metal 4 has no tessellation stage) and 10 geometry shaders (none drawn from the title to the lobby). Vertex shaders that feed them can be found from the shader headers (docs/agents 04)
- Mac: the boot notices, the title, a new game, the opening cutscene (the bus ride, the arrival at the hotel) and the lobby in play at 60 fps, about 3.7 GB phys_footprint (3.0 GB of Metal allocations); red lines at run time: the geometry shader that does not build, the conditional render the host draws anyway, primitive restart 0xff (no 8-bit strip seen). Diagnostics: DM_NVN_TREG_ID=n logs that texture ID's registrations; the pass dump prints the handles of texture binds
|
||
|
seedds
|
e72a06510e |
DRAGON QUEST VII Reimagined plays on the Mac: profile 0100A9D01C446000 (update 1.2.0, SDK 19.3.1)
- Files and shaders: Unreal Engine 4.27 with IoStore containers beside one pak (version 11, its footer magic 0xcc bytes from the end, uepak.py magic_back); the 62,901 shaders of the pak's two shader code archives are LZ4 blocks, which uepak.py decodes itself (lz4_shaders), so the NSP script runs no Oodle host when no record is Oodle. One global shader writes integers into an R8Uint target (shader_facts_manual.tsv) - Shader chain, for every game: a texture query on an image handle (c2 0x120 + 8 r: Unreal's GetDimensions on the RWTexture3D a compute shader writes) had no type in a facts table built without PC shaders, and the 2D default took a 3D image's depth from its width; the translator now translates such a shader again with the image types of its first translation (dmsh_xlate ImageTypesOf; dmsh_xlate.dll rebuilt, dmsh_chain.lock updated). Host, for every game: a texture binding of slot 32-39 reads the image bound in the image slot that shares its c2 word (nvn_mtl.m tex_slot_h; Hades II's texture slot 34 without an image still reads the texture). Unreal's volumetric lightmap indirection patch (7b958a7b21e72801) read the empty texture slot 32's 1x1 size and wrote only cell (0, 0, 0), so in the opening cutscene every character and prop sampled the atlas origin, a daytime level's bricks baked 2^15 brighter (white figures about 10^4 too bright). Monster Hunter Rise has four such queries. Every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (the change touches no c1 or uniform read); DRAGON QUEST VII Reimagined rebuilt for Mac and iOS - Host, for every game: stencil-mode texture builders (nvnTextureBuilderSetDepthStencilMode STENCIL) read the stencil of their depth-stencil storage (the hero was drawn as a dithered silhouette); the audio renderer at 32 kHz on the CPU path (the SDK makes its upsampler only for DSP rendering: UpsamplerManager::Allocate returns null and the host resamples 3:2); GetAudioDeviceOutputVolume; the shop and Nintendo Account calls of the title (ec Finalize / Cancel / GetErrorCode / ShowShopSubscriptionDetails, GetAuthorizationCode / GetState); kpr::GetKeyCodeMap; the movie output format's nv12-width / nv12-height beside nv12-uv-stride (Sofdec's VP9 movies were grey with bands) - Memory: MallocBinned3 reserves 37 GB of pools in the engine region; its creation is cut and an observation gate hands the engine the host's malloc zone (patch_nso.py cut + gate OBS_UE_MALLOC, hle_uemalloc.c layout UE4_27; patch_nso.py places a gate where the scratch registers it uses are dead, scratch_dead); the region is 2 GB (game_args) with the SDK's node-area cut on this sdk build, in a 3.75 GB guest block; about 2.6 GB resident on the Mac - Frame rate: this build's t.MaxFPS and rhi.SyncInterval in hle_fpslock.c; the profile keeps the game's 30 fps - Diagnostics: DM_CTL (Mac) drives a running game line by line: presses, shots, pass and texture dumps, texlist (textures of a target with storage and block-linear layout), memdump, peek, Unreal console variables (cvarf / cvari), env; the pass dump prints buffer-to-texture copies and buffer textures' format, size and storage, and DM_NVN_CBDUMP saves the source bytes of 3D uploads (docs/agents 04, 05) - Mac: the logos and opening movie, the title, a new adventure log, the opening cutscene (the well, the night scene with Maribel by the boat) into play at Pilchard Bay, and a saved log continued, at 30 fps with saves; no red line at run time apart from 62 of the 63 geometry shaders (one-pass cube shadow maps), which do not build. Unit tests: the graphics tests that build pass (t_refl's two sample libraries are not on this machine); 13 that compile nvn_mtl.m with hle_nvn.c already failed to build on main (g_ft defined in both, hle_nvn.c symbols missing from their stubs) |
||
|
seedds
|
2bdad7d19c |
Catherine: Full Body plays on the Mac: profile 0100BF00112C0000 (update 1.0.1a, SDK 8.2.1)
- Atlus's GFD engine with CRI middleware (CPK archives, ADX2 sound, Sofdec2 movies through movie::Decoder); 3,128 GLSLC shaders: 3,050 loose files in data/shaders_binary, 76 in main and 2 the game compiles at boot (captured on the Mac) - Shader chain, for every game: this GLSLC (GPU code 1.15) writes each vertex shader as a VertexA part (shader type 0, the position) and a VertexB part (the other outputs) in one code area; shader_split.py reads type 0 as a vertex shader (sph_stage) and dmsh_build.py merges the two parts as Ryujinx does (vertex_ab). Without it every varying was lost and the screen was black. Every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (the merge touches only type-0 vertex shaders, which no other game has); The Great Ace Attorney Chronicles checked on the Mac - Host, for every game: nvnDeviceGetInteger reports at least the GPU code version of the game's own GLSLC (the game refused to compile its built-in shaders against SDK 7's 1.5–1.14); nn::settings::GetAvailableLanguageCodeCount; nv12-uv-stride in the movie decoder's output format (CRI's player reads it unchecked and crashed on the first picture); a segment commit at the first pass boundary after a buffer → texture copy, since the game reuses the copy's staging memory a frame later (the subtitle band came out as another texture's blocks) - Memory: mem_used 96 MB (the rewritten modules take 92 MB, so SetHeapSize failed and the game ran on a 32 MB heap) - Prewarm list recorded from boot through the intro into the first Nightmare stage |
||
|
seedds
|
df0820eb68 |
DRAGON QUEST XI S plays on the Mac: profile 01006C300E9F0000 (update 1.0.4, SDK 7.3.2)
New profile for Square Enix's Unreal Engine 4.18 game: from the opening movie and the title through the prologue into free play at Cobblestone Tor, 30 fps with sound and saves, no red line. An IPA builds from the NSP folder; it has not run on an iPhone yet. - Files and shaders: Unreal 4.18 legacy (version 4) paks with LZ5 blocks, decoded on the Mac with the game's own LZ5_decompress_safe (uepak.py, hle_oodle.c serving after nninitStartup); 38,892 shaders inline in the cooked packages as zlib-compressed LSLG records (shader_split.py ue_legacy_jobs, ue_inline_shaders; the NSP script and profile_guess know both pak kinds). - Memory: Unreal's address region capped at 4064 MB (game_args, MB up to 4095); SDK 7.3.2 cuts in patch_nso.py (node area 62 MB, 32 MB alignment for large regions; the build Odyssey shares, which boots unchanged) for Engage's 5 GB block. Rewriter: a jump table whose bounds check sits right before the load (check_before_copy). - Host, for every game: audio catch-up frames wait for queue space (Square Enix's sound driver never waits on the renderer event: 11,000 updates a second before); movie decoder events read at signal time as the SDK does, audio decoders keep 32 outputs (the player plays only once every track holds one: the opening movie never started in half the boots), and no input after an end-of-stream packet until Flush / Stop (a skipped movie mapped 2.25 GB of end-of-stream samples); ec shop accessors, friends, auto-sleep, album, file data cache and movie hooks. - Graphics, for every game: a geometry shader that only picks the layer (Unreal's FWriteToSliceGS, the colour-grading LUT) folded into the vertex shader's layer entry points (dmsh_build.py gs_slice / vs_layer_entry, nvn_mtl.m prog_fold_gs); cube-map array depth counted in layer-faces in the Metal texture, the upload and the guest layout (t_nvn checks it); nvnTextureGetTextureAddress and the command memory size / free getters. - Shader chain changed (dmsh_build.py): DQ11 rebuilt for Mac and iOS; every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (no c1 or uniform read touched). Mac checks on this host: Big Brain Academy (movie loops, 23 catch-up renders as before), Monster Hunter Rise (movies), Super Mario Odyssey (boot on the cut sdk). |
||
|
seedds
|
87e647f5bb |
Age of Calamity's last red lines, fixed for every game: buffer textures read by 1D integer fetches are bound as 2D rows, and shadow vertex inputs replace the CPU vertex conversion
- Row wrap: Maxwell reads a texture buffer and a 1D texture with the same instructions (TLD with integer coordinates only, TLDS Texture1DLodZero) and the bound descriptor decides. Without a type answer the translator read such a slot as 2D at (x, 0) and the host bound a dummy for a buffer texture, so it read 0. The vendored translator (InstEmitTexture.cs, marked nxnative:) now addresses those fetches as (x & 0x3FFF, x >> 14) and reports the slot (fetch1d); the host binds a buffer texture there as a linear 2D texture of 16,384-texel rows over the same memory (tbuf_as_2d, nvn_tbuf_rows), cached by texture serial. KTGL's light and decal lists: Age of Calamity 16,438 shaders, Three Houses 321 (torches now light its map, blood decals are lit); FFT 5, Monster Hunter Rise 3 - Shadow vertex inputs: for each vertex shader with integer inputs the build writes <key>_bv (dmsh_vin.py), which declares every integer input again as float4 and as the other-sign integer at attribute indices 16-30 and ORs them into the input; the host sends each attribute to the index of its format's class and the zero buffer to the others (vin_ct, vin_on). Age of Calamity's Mac battle: 0 vertices converted on the CPU instead of 545,000 per submission, submit + present 10.0 -> 5.7 ms a frame, CPU 3.97 -> 2.44 W, the same picture. DM_NVN_VCONV_CPU=1 gives the CPU conversion back - A texture type mismatch no view bridges is logged once per shader and slot with its texture ID, NVN target and formats (tex_mis_note); it showed Age of Calamity's remaining one: 3D slots 8 and 9 of three vertex shaders bound with 2D textures (bound as is, as before) - Chain procedure for every profile: the 18 with game files on this drive rebuilt for Mac and iOS (c1_full.tsv / cb_extent.tsv: header lines only; new cb_extent.tsv for Hades II, Cuphead and Monster Hunter Rise), the others header-only with their lists carried over. Mac smoke runs of 13 games: no new red lines - Tests: t_tbuf2d (row aliases against texture_buffer views), t_vin (shadow inputs against the CPU conversion, every format class), test_dmsh_vin.py, nvn_tbuf_rows in t_nvn |
||
|
seedds
|
f5646efc47 |
Uniform updates that cover part of what a draw reads no longer end the render pass; clears with no draw ride on the next pass
Outdoors in Breath of the Wild (the user's Great Plateau save, Mac M4 Pro) a frame had about 386 render passes and loaded and stored about 900 MB of attachments, which an iPhone at 60 fps can hardly move (about 55 GB/s). 220 of those passes ended only because the game updates the first 64 bytes of a 64 KB vertex uniform block before each draw group (nvnCommandBufferUpdateUniformBuffer inside the pass) while its shaders read up to 544 bytes of it. A deferred update that covers only part of what a draw reads used to end the pass. - Partly covered blocks (nvn_mtl.m ubp_get): the shadow starts from memory and takes every inline update queued in the submission, newest last, incrementally per range. It is refused, and the pass ends as before, when something else may have written the range by the time the pass runs: a copy, fill or feedback write queued in the submission, a storage buffer a shader may write or a texture->buffer copy over it (sbw, by 4 KB page), or a GPU write or counter report of an earlier submission still running (gw). A counter report of the same submission does not count: the host writes reports on the CPU after the whole submission completes (Fire Emblem Engage writes its reports into a uniform block's page). DM_NVN_UB_PARTIAL=0 turns it off; DM_NVN_UBP_CHECK=1 lands the updates, waits for the GPU and compares every shadow with memory. - cb_extent.tsv (new tool dm/gfx/tools/cb_extent.py, c1_guard.py's interval arithmetic over the translated MSL): per shader, how far into each uniform slot it can read, and the storage slots it never writes. Like c1_full.tsv, the host loads it only for its chain version; build_ipa.sh copies it, the NSP script writes it for provisional profiles, and the chain procedure regenerates it. Without it bindings are copied whole (up to 64 KB). Ten profiles have one. - Route B's reflection never says whether a storage buffer is written, so every one read as read-only and its writes went unnoticed: a storage buffer now counts as written unless SPIRV-Cross typed it const (nvn_refl.m sbuf_rw) or cb_extent.tsv proves the slot unwritten. - Lazy clears (lc_*): a set of targets that gets only clears before the game sets other targets no longer opens a pass just to store them; the next pass that attaches the same texture level and layer through the same view clears it as its load action, and the rest land as before before anything else could see them. Breath of the Wild's four clear-only passes before the G-buffer fold into its first pass. Off in TBDR mode; DM_NVN_LAZY_CLEAR=0 turns it off. - The residency set line counts shadows from memory, refusals by reason, and lazy clears. On the Mac, old (DM_NVN_UB_PARTIAL=0 DM_NVN_LAZY_CLEAR=0) and new side by side on each game's route; attachment loads + stores per frame: - Fire Emblem Engage battles: about 1,850 -> 360 MB, 88 -> 38 passes. - Tears of the Kingdom: about 1,100 -> 700 MB, 124 -> 116 passes. - Breath of the Wild outdoors: 386 -> about 158 passes, GPU p50 about 9.3 -> 6.5 ms, submit 5.7 -> 4.2-4.9 ms; where the old host fell to 30 or 20 fps under load, the new held 60. - Link's Awakening, 13 Sentinels: about 15% less. Super Mario 3D World: 5% less (lazy clears). - Odyssey, Three Houses: 2-8% less. Mario Wonder, Metroid Dread: unchanged. No new red lines in any of them. The shadow check found every shadow equal to the GPU's memory (BotW 346,000, Engage 284,000, TOTK 86,000), and t_nvn passes 68/68. Screenshots differ only by timing. Tried and dropped: DM_NVN_TBDR=1 (it turns the deferred updates off), DM_NVN_BARRIER=game (no gain), and same-target continuation outside TBDR mode (BotW's decals end the pass anyway). What still splits BotW's big passes: the GPU particles' compute dispatches (about 190 MB a frame) and decals that sample the G-buffer. Mac shader libraries were built for the current chain of TOTK, Engage, Wonder, Dread, Odyssey, Link's Awakening, Three Houses and 13 Sentinels (c1_full.tsv headers already matched). The IPA ZeldaBreathOfTheWild-2026-10-21-passes.ipa (an earlier state of these changes plus the uncommitted frame-step probes) awaits an iPhone test. |
||
|
seedds
|
56b806155f |
One NVN driver layout for every game; shader sources read from the game files
The NVN driver's constant segments have the same layout in every game, so the nvn_* profile keys are gone. Combined textures are at c2 0x20 + 8 r (a slot past 31 shares the image table; Hades II binds 34), images at 0x120, separate textures at 0x168 (now always the host's slots 128 + n, as Monster Hunter Rise had them), samplers at 0x568, and the storage buffer tables at c0 0x110 / 0x310 / 0x510. dmsh_build.py decodes with that layout, the replay fills both texture tables, and the translator numbers slots the same way (rebuilt; its SPIR-V and reflection are byte-identical on 1,491 sample shaders, lock updated). The shader_* switches and graphics are gone too; the build reads them from the files (profile_guess.py shader_sources): - Vulkan when main imports Vulkan and not NVN; its capture when the RomFS has no SPIR-V. - A GLSLC capture when main holds GLSL sources and imports glslcCompile. - An Oodle decode for Unreal paks, and the tool-mode decoder for Trinity packs. - GLSLC outputs from every main. - A cover pairing for each single-stage vertex shader (GLSLC, Unreal). build_ipa.sh takes GFXAPI from the NSP script. Also removed: the last D2R pieces (the embedded-shader path in dmsh_build.py, dmsh_facts.py --freeze / --check, maxwell_refs.py, ref/hle_hid_d2r.c). Chain: every profile's dmsh_chain.h is regenerated. The c1_full lists are carried over: the change touches no c1 read, and the rebuilt Octopath Traveler 0, Hades II, Unicorn Overlord and Rhythm Heaven Groove gave the same lists, identical tables and the same reflection apart from separate texture slots + 128. Still to do: - Run the chain procedure for the other NVN profiles. - Engage's and Three Houses' shader tables need a new split: Engage gains Unity's 11 GLSLC shaders in main, and Three Houses' 17 BNSH vertex shaders lose their cover pairs. - Ace Attorney's new tables are in, but its libraries are not rebuilt. |
||
|
seedds
|
d34e0446aa |
Pokémon Legends: Z-A plays on the Mac: Trinity packs through the game's Oodle, transform feedback, stencil views, the base instance in the driver segment
- Profile 0100F43008C44000 (2.0.2, SDK 19.3.11): 5 GB layout with two cuts (StandardAllocator 12 → 4 GB, node area), mem_map_limit (the pool enforced as on Horizon), shader_oodle. - Shaders: Trinity packs (arc/data.trpfs, FlatBuffers, Oodle Mermaid) decoded with the game's own OodleLZ_Decompress in the Mac host; 28,603 shaders. - SDK 19: GetAudioOutBufferCount, crypto random bytes, ToPosixTime with a rule, NVN state getters, CommandBufferFenceSync, the 64-bit WaitForAddress type. - Graphics: transform feedback (vertex capture entry point, rasterizer discard), layered passes (the colour grading LUT), point size dropped beside a layer output, function constants specialized, a stencil-only view for STENCIL-mode texture views (striped lighting), sampler min / max reduction (Metal 26), the draw's base instance at c0 + 0x644 (occlusion boxes; characters' legs folded back from the hips), the zero cube for a 2D texture in a cube slot. - Shader chain changed (dmsh_build.py): every profile's dmsh_chain.h rewritten; the other profiles' c1_full lists carried over (that change touches no c1 read). |
||
|
seedds
|
9923c5a032 |
NVN API version and shared-font layout follow the SDK; Oodle found by its name string
- The loader reads the SDK version from the sdk module's NintendoSDK_nnSdk string (dm_sdk_major). SDK 10 and older report NVN API 53.313 and point the shared-font address at the TTF itself; later SDKs 55.15 and the BFTTF header. That is what every profile set, so device_api and shared_font leave the five that had them (Odyssey, 3D World, Link's Awakening, Age of Calamity, Three Houses) and profile_guess.py no longer guesses them; a profile can still override. - DM_OODLE_JOB finds OodleLZ_Decompress as the one function in main that takes the address of the library's own name string, with its start from .eh_frame_hdr, instead of an address per main build: Octopath Traveler 0 main+0x4f77e10 and Pokemon Legends: Z-A main+0x1a9c9e0. Octopath's 10,596 shaders decode byte-identical to before. - Rows that only restated defaults are gone (pad_layout simple, graphics nvn, present_interval game), and the NSP script writes no default rows into provisional profiles. No chain version changes. All 19 profiles build, the host tests pass; Mac boots of 3D World (SDK 10.4.0: API 53.313) and Engage (SDK 12.3.7) read the version. |
||
|
seedds
|
8d7f957fa2 |
Monster Hunter Rise: plays on the Mac into the opening; translator phi fix, all chains rebuilt
- Profile 0100B04011742000 (update 16.0.2, RE Engine): KPKA PAK shaders (shader_split.py kpka_jobs), XCI base game in nx_extract and the NSP script, BIG_TEXT patch area in patch_nso.py. - Sparse virtual pool: a 4 KB page table (vpt_*) that the replay reads textures through, dirty tracking per mapped page, texture views from the first mapped level. - MultiDraw*IndirectCount with a CPU wait for the count; pending GPU writes indexed by page; bindless handles read from queued inline updates. - sbuf_ubo binding kind, integer vertex formats into float inputs, clears on textures that are not render targets. - Inline keyboard text, state and finished callback; ngc::ProfanityFilter host vtable. - Ryujinx BindlessElimination follows phi webs (218 MHR pixel shaders); dmsh_xlate_bin and dmsh_chain.lock updated; dmsh_chain.h and c1_full.tsv rebuilt for every Maxwell profile. - README: Monster Hunter Rise under being ported, gallery image, XCI base games, Ryujinx local fixes. |
||
|
seedds
|
51e4e1d25e |
Chain rebuild after the English-only change: c1_full.tsv restamped for 7 more profiles; the NVN stub summary line no longer contains [RED]
Every rebuilt profile matches the previous chain: same shader counts, same Metal failures (Engage 1, 3D World 1, Three Houses 2, Odyssey 3), same iOS library counts. Metroid Dread on the Mac renders the same frames as before the change. |
||
|
seedds
|
c5955d42a6 |
English only: no Chinese left in code, scripts, logs, tables or docs
- Log messages, script output, test labels, Makefile errors and comments are English. Red log lines start with "[RED] " (was 【红】). - Shader map stage column: vertex / pixel / compute / geometry / hull / domain (generators, dmsh_build / dmsh_facts and nvn_mtl.m stage_of follow). - The NSP script's regexes match the new shader_split.py / dmsh_build.py summary lines; patch_nso cut scopes are all / host-only. - Host UI text table (dm_i18n_tab.h) is English only; dm_lang_zh() removed. - Text that matches game data (t_swkbd.c, t_textbox.c) is written as \x escapes. - Shader chain: the chain files lose their bilingual comments, so every profile has a new chain version. dmsh_xlate.dll rebuilt from the translated Program.cs (SPIR-V byte-identical on 4,500 shaders of Wonder, TOTK and Age of Calamity), lock and dmsh_chain.h headers regenerated. c1_full.tsv restamped for Dread, 13 Sentinels and Wonder; the other profiles follow when their rebuild finishes. - AGENTS.md and docs/agents: the language rule now covers logs; quoted log lines updated. |
||
|
seedds
|
a492473eb4 | c1_full.tsv for the last six profiles after the chain rebuild (Link's Awakening, Engage, 3D World, BotW, Odyssey, Age of Calamity; only the chain line changes) | ||
|
seedds
|
b3f25a09bb |
Shader chain: translator fixes for 39 shaders; compiled shaders cached by their Metal source
- Ryujinx (vendored, Instructions.cs GenerateTextureSample): texel offsets read from a register are built with OpCompositeConstruct + Offset, and a four-offset gather becomes four one-offset gathers. Upstream wrote an OpConstantComposite, invalid SPIR-V that SPIRV-Cross refused (2 shaders each in Wonder, Link's Awakening, TOTK, Age of Calamity). - dmsh_xlate: a shader whose SPIR-V declares a 64-bit float is translated again with Ryujinx's DoubleToFloat pass (30 TOTK main-material vertex shaders); TOTK's visibility-counter vertex shader gets MacNativeGames' subgroup lowering (MIT). Every other shader's SPIR-V is byte-identical. - New dm/gfx/tools/dmsh_mtlcache.py: dmsh_build.py compiles each MSL through a cache keyed by the source, the compile flags and the compiler (<cache>/cas/metallib/). Before a build it is filled from the earlier b-*/ folders beside the output and IOSOUT, and from DMSH_PREFILL, once a fresh compile of a few of a folder's shaders reproduces their libraries byte for byte. A chain change that leaves a shader's MSL unchanged no longer recompiles it. - dmsh_build.py clones files with the clonefile call instead of one cp per file: each fork of the build process cost about 6 ms and serialized the finishing phase. - TOTK's rebuild after the translator fix: 49 min -> 9 min, libraries byte-identical to the previous build apart from the 39 fixed shaders. All 12 chain versions are new; Dread, 13 Sentinels, Three Houses, TOTK and Wonder are rebuilt (Mac + iOS + c1_full.tsv), the other six follow. |
||
|
seedds
|
5f1a4b97c2 |
c1_full.tsv for five rebuilt shader chains
These five games' shader libraries were rebuilt after f1cc6be compiled every shader with -fpreserve-invariance, which changed every chain version: - Super Mario Odyssey (123e7650b527) - Link's Awakening (7363976f0e53) - Fire Emblem Engage (32031379f5b9) - 13 Sentinels (1c784ff32922) - Super Mario Bros. Wonder (8c8ebaa0059a) Their c1_full.tsv files now name the new chains, so hosts built from main use the lists instead of falling back to exclusive 64 KiB c1 segments. Breath of the Wild, Super Mario 3D World and Metroid Dread are still rebuilding and follow in a later commit. |
||
|
seedds
|
9b1d22d81f |
Super Mario Odyssey playable on the Mac: H.264 movies, sticks, HUD; all shaders compiled with preserved invariance
- Movies: H.264 decoded with VideoToolbox (dm/host/hle_movie_avc.c). This SDK's MP4 demuxer passes the parameter sets as the "avcc" record; pictures leave in presentation order through a 4-deep queue (41 of Odyssey's 44 movies have B-frames), and an end of stream drains the queue. The hint videos in Odyssey's tutorial boxes now play (they were black). VP9 unchanged (Engage: 2,816 packets in, 2,816 frames out). - Controller input: GetNpadStates fills every requested history entry with the current state under consecutive sampling numbers. Odyssey reads all 16 entries whatever count comes back and polls every slot each frame; with only the newest written, neither stick did anything in play while buttons and menus worked (its first hint, "Look Around", waited forever). DM_KEYS takes RLeft / RUp / RRight / RDown. - Odyssey's HUD counters were solid white cards: the profile lists its 26 layout shaders (main's BNSH at 0x132c000) under alpha_from_red, as for Breath of the Wild. - -fpreserve-invariance on every Metal compile (dmsh_build.py MAC_METAL, ios_metal.py): Odyssey draws a depth pre-pass with alRenderDepthOnly, then the G-buffer with other vertex shaders and depth test less-or-equal; Metal gave the identical position math slightly different depths and those pixels kept stale content (Cap Kingdom's ground in 32x32 blocks on every fourth frame). Checked by recompiling the frame's 78 vertex shaders with the flag: four consecutive frames identical, no blocks. All eight chain versions change; Mac libraries and c1_full.tsv are being rebuilt and follow. - DM_NVN_TEXSNAP works again (its entries lacked the layer field the dump loop reads) and names its files <path>_s<pass index>_<serial>_<level>.png. - Smoke runs of the seven playable games on the Mac after these changes: only their known red lines. |
||
|
seedds
|
65deb2f03d |
Vertex outputs start at the hardware's (0, 0, 0, 1): Link's Awakening's outdoor sunlight fixed; IPA version is the game's display version
- dmsh_build.py vs_init: every float4 vertex output is set to (0, 0, 0, 1) before the translated body, as unwritten components read on the hardware. The translator's MSL started from all zeros; Link's Awakening's surface shaders divide a world position by its w, which the vertex shader never writes, so every sun-shadow compare value was infinite and the whole outdoor scene was lit as if in shadow (dim, no shadows from fences, house or trees). All seven chain versions change. - Mac libraries and c1_full.tsv rebuilt for Link's Awakening, Engage, 13 Sentinels and Mario Wonder (Breath of the Wild, Super Mario 3D World and Metroid Dread follow). - DM_NVN_TEXRAW dumps the depth plane of a Depth32Float_Stencil8 texture (shadow maps). - The IPA's version (CFBundleShortVersionString / CFBundleVersion) is the packed game's display version: the update's when an update is packed, else the base game's (nacp.py --version, DM_APP_VERSION in build_ipa.sh, project.yml; the NSP script prints it). - README: Link's Awakening screenshot outdoors in Mabe Village. |
||
|
seedds
|
d035e37c37 |
Depth mode per game at run time: Link's Awakening's outdoor blur fixed; README screenshots of Engage, 13 Sentinels and Link's Awakening
- NVN's depth mode is runtime state: Mario, Metroid Dread, Super Mario 3D World and Breath of the Wild keep near = -w (GL); Link's Awakening, 13 Sentinels and Engage set near = 0. The translator rewrote z for GL in every vertex shader, so near = 0 games had all depths in 0.5..1 and depth-to-distance effects were wrong: Link's Awakening's depth of field blurred the whole screen outdoors. - The translator no longer rewrites z; dmsh_build.py (vs_depth) writes two entry points into every vertex library, main0 with the GL rewrite (the same arithmetic as before) and main0_z without; the host picks one from the mode the game sets (mtl_set_depth_zero, shader_load). No per-game setting; a red line if the mode changes after vertex shaders loaded. Metal function constants were measured at about 2.4 ms of specialization per vertex shader, so two entry points instead. - New translator dll (Ryujinx dlls unchanged), lock and chain versions for every profile; Mac libraries and c1_full.tsv rebuilt for Link's Awakening, Engage, 13 Sentinels and Mario Wonder (Breath of the Wild, Super Mario 3D World and Metroid Dread follow). - Checked on the Mac: Link's Awakening outdoors (from a save in Mabe Village) has a sharp middle band again; Mario Wonder's title renders the same as before (differences only in animation). - README gallery: Engage, 13 Sentinels and Link's Awakening recaptured with this build. |
||
|
seedds
|
7fdf6dccbe |
README screenshots of every game; NSP script packs unknown games with a provisional profile; Link's Awakening stencil copy and outdoor shaders
- README: a gallery with one picture per game (docs/images): iPhone shots with the virtual gamepad for Mario Wonder, 3D World, BotW and Link's Awakening; Mac shots (captioned) for Engage, Metroid Dread and 13 Sentinels. Link's Awakening is now listed as in testing on an iPhone. - scripts/build_ipa_from_nsp.py: a game with no profile is surveyed, then gets a provisional profile guessed from the survey in <NSP folder>/<id>_build/profile/<id>/ (outside the repo, reused on the next run) and a best-effort IPA; --survey-only keeps the old behaviour. game.py searches DM_GAMES_DIR after dm/games; the host Makefile reaches such a profile through a symlink; build_ipa.sh finds it through game.py. zip_store also stores files whose headers say they are compressed. - copy_t2t_draw: depth-stencil copies now carry the stencil ([[stencil]] export from an X32_Stencil8 view). Link's Awakening's occlusion pass tests stencil bit 1 of its quarter-size depth copy, whose stencil was the new texture's undefined content. - Link's Awakening: four outdoor shaders with the foliage shader's input loop failed on the iPhone (Half2 / normalized formats); listed in shader_facts_manual.tsv. Chain c4995bb3161c. |
||
|
seedds
|
ab5c8ff1f4 |
Link's Awakening: game sound, doorway lighting tiles and missing foliage fixed
- Sound: SDK 8 numbers the DSP commands differently and its DeviceSink is 0x10, not 0x13, so every rendered frame played silence (only the movies had sound). capture() now takes the one 0x148-byte command with 1-6 inputs as the sink when the expected ID is missing. - Lighting: the light-culling compute shader reads its workgroup count from NVN's driver segment c0 (+0x200). dmsh_build's drv_reads scan only matches fragment-shader names, so compute and vertex shaders got an all-zero c0: every tile row wrote row 0, and the doorway step flickered with white 64-pixel tiles. A declared driver-segment binding now counts as read, and compute c0 carries the dispatch size (0x200) and the uniform buffer table (0x210 + 16 r). BotW and SM3DW render as before. - Foliage: the shadow / depth pre-pass pipeline 147c829e79fb322f failed (uint4 inputs fed 16-bit normalized formats), so with depth test EQUAL the palm tree and flowers never drew. New profile file shader_facts_manual.tsv (merged by shader_split.py) declares those inputs float; Link's Awakening's chain is now ccb18ba743bf. |
||
|
seedds
|
e734ced7d9 |
Zelda Link's Awakening runs on the Mac (profile 01006BB00C6F0000, 1.1.0, SDK 8); BotW HUD prompt boxes fixed
BotW: the button-prompt shadow samples an R8 blur whose swizzle (R,0,0,1) made alpha 1, so every prompt got a solid box. New profile key alpha_from_red takes alpha from red for the named shader. Link's Awakening: - NVN lower-left window origin (negative viewport height, winding flip, flipped present and raw screenshots) - texture copies with format change, scaling, mirroring and depth (copy_t2t_draw); SetCopyImageStride; nvnWindowGetPresentInterval - oe::PopNotificationMessage, vi SetLayerCrop / DestroyLayer / CloseDisplay / Finalize, audren P_SIZE 0x40 for SDK 8, hid style-set update event signalled once on bind - Vorbis movie audio through stb_vorbis (dm/third_party/stb_vorbis) - patch_nso jump tables: other index registers, mov copies, masked copies |