- Engine: Unreal Engine 5.6.1 (Square Enix, the desktop renderer), the newest Unreal so far; everything in one IoStore container beside one pak; the sdk and subsdk0 builds are STEINS;GATE RE:BOOT's
- Rewriter, for every module: code before the first unwind entry is one function (each site there was a function of one instruction, so a byte jump table at main+0x219234 went unseen), and a function longer than 256 KB is analysed in pieces cut after a ret / br / b that no conditional branch or b within 64 KB crosses, with far branches and jump table targets checked against windows and retry blocks afterwards (this main's 82 MB without unwind entries took 100 GB of memory as one function; now 2.7 GB resident). Checked: DRAGON QUEST VII Reimagined's main byte-identical to its rewrite by the previous patch_nso.py; FINAL FANTASY RESONANCE's, Octopath Traveler 0's and DRAGON QUEST XI S's byte-identical between two different cut placements; only DQ VII's main among the 120 modules of 33 games on hand has other sites before its first unwind entry (none in a window or jump table)
- Shaders: no archive in the pak; the two libraries are IoStore shader library chunks whose tables and shader groups are Oodle-compressed, read through the game's own Oodle in the Mac host's tool mode with no boot (uepak.py Toc / io_archive / OodleHost; library names from CityHash64 of the manifest's archive names) and written as decoded version 2 archives; the NSP script uses them for any game with IoStore shader libraries. The pak's single-block files store block size 0 (Pak.entry). 73,496 unique shaders; 11 do not build (3 geometry, 8 compute with 3D image atomics), none met so far. 2,077 particle vertex shaders pick their texture slot at run time (tex_dyn_slots); CopyStencilToLightingChannelsPS writes integers to an R8Uint target (shader_facts_manual.tsv; without it the sprites were dark silhouettes)
- Host, for every game: a shader with run-time texture handles in a game without tex_dyn_slots freed its reflection twice and aborted the host (nvn_mtl.m); oe::MuteRecording / UnmuteRecording (no-ops)
- Memory: the FMallocBinned3 constructor call is cut and the host's allocator runs Unreal's heap (hle_uemalloc.c layout UE5_6: UE 5.5+ FMalloc has 32 slots, with MallocZeroed and two more TLS cache calls); the SDK's node area is cut to 62 MB. The engine's own page allocator only grows its use of the engine region (2.03 GB of address space over seven minutes, about 1 GB mapped), and a 2 GB region aborted in the first battle, so the region is 4064 MB in Engage's 5 GB block (this sdk build aligns regions to 2 MB, no alignment cut needed)
- Frame rate: 60 fps (30 on the Switch): the host writes rhi.SyncInterval 1 (hle_fpslock.c rows for this build; profile fps_target 60); the game steps by real time (DM_NVN_REGLUM traces at 30 and 60 fps match from the load through a walk)
- Mac: first-boot settings, the title, a new game, the opening cutscene, the first field and its autosave, Continue, and the first two battles at 59.7 fps, 2.3-2.6 GB resident, flat over seven minutes, no red line apart from a few texture type mismatches at one moment of the new game's load in some runs. Docs: docs/agents 01, 03, 04 and development.md
Each game gets its own project in <NSP folder>/<title ID>_build/xcode, registered in a shared NxNative.xcworkspace; a target phase copies the RomFS, shader libraries and add-on content into the app before Xcode signs it, and preparing again keeps the bundle ID and team chosen in Xcode. The unsigned IPA build is unchanged.
On top of the PR: the resource phase stops when the profile's shader chain is no longer the prepared shaders' (the host loads only its own chain), prepared projects ask for the Increased Memory Limit capability (DM_ENTITLEMENTS overrides), the source links are made again on every preparation, and scheme names keep non-ASCII letters. The PR's TOML launcher is left out.
Co-authored-by: Animax Deng <Animax.deng@gmail.com>
- Files and shaders: Unreal Engine 4.27 with IoStore containers beside one pak (version 11, its footer magic 0xcc bytes from the end, uepak.py magic_back); the 62,901 shaders of the pak's two shader code archives are LZ4 blocks, which uepak.py decodes itself (lz4_shaders), so the NSP script runs no Oodle host when no record is Oodle. One global shader writes integers into an R8Uint target (shader_facts_manual.tsv)
- Shader chain, for every game: a texture query on an image handle (c2 0x120 + 8 r: Unreal's GetDimensions on the RWTexture3D a compute shader writes) had no type in a facts table built without PC shaders, and the 2D default took a 3D image's depth from its width; the translator now translates such a shader again with the image types of its first translation (dmsh_xlate ImageTypesOf; dmsh_xlate.dll rebuilt, dmsh_chain.lock updated). Host, for every game: a texture binding of slot 32-39 reads the image bound in the image slot that shares its c2 word (nvn_mtl.m tex_slot_h; Hades II's texture slot 34 without an image still reads the texture). Unreal's volumetric lightmap indirection patch (7b958a7b21e72801) read the empty texture slot 32's 1x1 size and wrote only cell (0, 0, 0), so in the opening cutscene every character and prop sampled the atlas origin, a daytime level's bricks baked 2^15 brighter (white figures about 10^4 too bright). Monster Hunter Rise has four such queries. Every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (the change touches no c1 or uniform read); DRAGON QUEST VII Reimagined rebuilt for Mac and iOS
- Host, for every game: stencil-mode texture builders (nvnTextureBuilderSetDepthStencilMode STENCIL) read the stencil of their depth-stencil storage (the hero was drawn as a dithered silhouette); the audio renderer at 32 kHz on the CPU path (the SDK makes its upsampler only for DSP rendering: UpsamplerManager::Allocate returns null and the host resamples 3:2); GetAudioDeviceOutputVolume; the shop and Nintendo Account calls of the title (ec Finalize / Cancel / GetErrorCode / ShowShopSubscriptionDetails, GetAuthorizationCode / GetState); kpr::GetKeyCodeMap; the movie output format's nv12-width / nv12-height beside nv12-uv-stride (Sofdec's VP9 movies were grey with bands)
- Memory: MallocBinned3 reserves 37 GB of pools in the engine region; its creation is cut and an observation gate hands the engine the host's malloc zone (patch_nso.py cut + gate OBS_UE_MALLOC, hle_uemalloc.c layout UE4_27; patch_nso.py places a gate where the scratch registers it uses are dead, scratch_dead); the region is 2 GB (game_args) with the SDK's node-area cut on this sdk build, in a 3.75 GB guest block; about 2.6 GB resident on the Mac
- Frame rate: this build's t.MaxFPS and rhi.SyncInterval in hle_fpslock.c; the profile keeps the game's 30 fps
- Diagnostics: DM_CTL (Mac) drives a running game line by line: presses, shots, pass and texture dumps, texlist (textures of a target with storage and block-linear layout), memdump, peek, Unreal console variables (cvarf / cvari), env; the pass dump prints buffer-to-texture copies and buffer textures' format, size and storage, and DM_NVN_CBDUMP saves the source bytes of 3D uploads (docs/agents 04, 05)
- Mac: the logos and opening movie, the title, a new adventure log, the opening cutscene (the well, the night scene with Maribel by the boat) into play at Pilchard Bay, and a saved log continued, at 30 fps with saves; no red line at run time apart from 62 of the 63 geometry shaders (one-pass cube shadow maps), which do not build. Unit tests: the graphics tests that build pass (t_refl's two sample libraries are not on this machine); 13 that compile nvn_mtl.m with hle_nvn.c already failed to build on main (g_ft defined in both, hle_nvn.c symbols missing from their stubs)
- The newest version in the folder is packed (-p / PACKAGE picks another). A registered version with every shader_versions version present builds from the profile as before
- Any other version (the base game alone, another update, other build IDs) builds from a profile derived from it in <id>_build/profile: the profile's keys, that version's build row, and shader tables made from its own files; a red line names the modules whose per-build fixes do not apply
- Host prewarm skips shader keys the shader map does not have (a list of another game version), counted in the "prewarm done" line
- Tested with Hollow Knight: Silksong: the base game alone (1.0.27776) packs with its own 2,244 shaders and plays the opening on the Mac; base + 1.0.30000 builds as before
- README: the update is optional; Silksong's row no longer names it
- Unity 6000.0.50f1 IL2CPP with FMOD; the bundles carry their type trees, so unityfs.py reads Unity 6's Shader objects with no layout written out (2,450 shaders, the one geometry shader does not build)
- Host: nn::hid::GetDebugPadStates, which Unity 6 polls every frame and the SDK's own aborts on (Result 0x2aca)
- Rewriter: the address-region node area cut on this sdk build (alias high-water mark +3.00 GB → +2.26 GB in the first room)
- profile_guess.py: every game with an NPDM system resource gets the 5 GB block; heapless ones (Silksong, Absolum, Pokémon Legends: Z-A) used to get a 256 MB alias region
- Prewarm list recorded from boot to the first two rooms of Moss Grotto
stdout carries one JSON object per line: stage, step, progress (done, total,
unit, est, note, frac, eta, elapsed; null while unknown), stage_end and say,
which carries every message, errors and a failed step's log tail included.
A count change goes out at most every 0.5 s, any other change at once; the
ticker now runs in json mode too and sends a count that was held back, so a
stage whose count moves rarely (the rewrite) no longer shows a stale count
until it ends. json mode never prompts; close() stops the ticker by an event
of its own and keeps the mode, so a second build in one Spyder session gets
its redraws again.
dm/tools/tests/test_nsp_progress.py covers the events, the held-back count,
the failed step's tail, the CLI error and the bar / lines / off output,
with no game files, keys or Xcode.
Co-authored-by: nocmt <nocmtone@gmail.com>
Unity 2018.3.10f1 IL2CPP with FMOD; its sdk and movie module are DRAGON QUEST XI S's builds, so
that build's rewriter cuts apply as they are, and no host change was needed. Brought up on the Mac
through the opening movie, the title, options, the tutorial and the prologue (choices, menu,
scenario chart, log) at 60 fps as on the Switch, with sound and saves; no red line at run time,
0.6 GB resident at the title and 0.77 GB after ten minutes of story.
- unityfs.py: Unity 2018.3's Shader layout (SHADER_2018_3, type hash 8db76007...). The game has no
bundles and its serialized files carry no type trees, so the shader split stopped. Written from
AssetRipper's TypeTreeDumps for 2018.3.10f1 and checked node for node against the dump (1,325
nodes with the alignment flags); all 48 Shader objects read to their exact end (450 subprograms,
452 shaders with main's GLSLC outputs). Other profiles' hashes are untouched.
- Memory: the hardware pool (3.2 GB) in Engage's 5 GB block. Unity 2018.3 reserves no address
regions of its own, so every mapped page lies in StandardAllocator's region (377 MB at the peak).
- Input: the game reads no touch, so it stays docked with the 2D pad; the left stick moves the menu
cursor.
Development notes, the graphics catalogue and the survey's list of Unity versions read.
A rerun after an interrupted IPA pack did a lot of the build again:
- The iOS shaders were keyed on the shader split's key, and that covers every dm/gfx/tools/*.py.
A pull that changed only the Unity readers (shader_split.py, unityfs.py) made Monster Hunter
Rise split again and then restore all 63,111 shaders from the cache (9 minutes). The iOS shaders
are now keyed on the shader package's content and on what dmsh_build.py loads (the repository
modules it imports, at any depth, and the chain lock). A split that runs again and writes the
same package leaves them reused; hashing Rise's 510 MB package takes 0.3 s.
- The DLC was extracted on every run (Rise: 252 DLC, 16-30 s). It is now reused while the DLC NSPs
(names, sizes, times) and nx_extract's sources are unchanged (aoc.key).
- Xcode's custom build location setting overrode -derivedDataPath, so every game's app was built
in one shared ~/Library/Developer/Xcode/DerivedData/Build: a build after another game's compiled
all 129 files, and two games built at once wrote the same app. build_ipa.sh sets SYMROOT and
OBJROOT under the game's work folder; a repeat build there compiled nothing (1 s).
Checked: Celeste with its split forced to run again (same package, iOS shaders reused), Rise's DLC
twice (15.9 s, then 0.1 s), and an Xcode build into a path with spaces and an apostrophe.
Each game's next build runs its iOS shader step once more, to write the new key.
New profile for Square Enix's Unreal Engine 4.18 game: from the opening movie and the title
through the prologue into free play at Cobblestone Tor, 30 fps with sound and saves, no red
line. An IPA builds from the NSP folder; it has not run on an iPhone yet.
- Files and shaders: Unreal 4.18 legacy (version 4) paks with LZ5 blocks, decoded on the Mac
with the game's own LZ5_decompress_safe (uepak.py, hle_oodle.c serving after nninitStartup);
38,892 shaders inline in the cooked packages as zlib-compressed LSLG records (shader_split.py
ue_legacy_jobs, ue_inline_shaders; the NSP script and profile_guess know both pak kinds).
- Memory: Unreal's address region capped at 4064 MB (game_args, MB up to 4095); SDK 7.3.2 cuts
in patch_nso.py (node area 62 MB, 32 MB alignment for large regions; the build Odyssey shares,
which boots unchanged) for Engage's 5 GB block. Rewriter: a jump table whose bounds check sits
right before the load (check_before_copy).
- Host, for every game: audio catch-up frames wait for queue space (Square Enix's sound driver
never waits on the renderer event: 11,000 updates a second before); movie decoder events read
at signal time as the SDK does, audio decoders keep 32 outputs (the player plays only once
every track holds one: the opening movie never started in half the boots), and no input after
an end-of-stream packet until Flush / Stop (a skipped movie mapped 2.25 GB of end-of-stream
samples); ec shop accessors, friends, auto-sleep, album, file data cache and movie hooks.
- Graphics, for every game: a geometry shader that only picks the layer (Unreal's
FWriteToSliceGS, the colour-grading LUT) folded into the vertex shader's layer entry points
(dmsh_build.py gs_slice / vs_layer_entry, nvn_mtl.m prog_fold_gs); cube-map array depth counted
in layer-faces in the Metal texture, the upload and the guest layout (t_nvn checks it);
nvnTextureGetTextureAddress and the command memory size / free getters.
- Shader chain changed (dmsh_build.py): DQ11 rebuilt for Mac and iOS; every other profile's
dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (no c1 or uniform read
touched). Mac checks on this host: Big Brain Academy (movie loops, 23 catch-up renders as
before), Monster Hunter Rise (movies), Super Mario Odyssey (boot on the cut sdk).
- MonoGame for the Switch, its C# and the game's compiled to C++ by Sickhead's BRUTE into main; FMOD Studio through AudioOut; only main and sdk. No host change: the logos, the title, a new file and the prologue at 60 fps with music and sound, no red line, about 1.65 GB resident, flat over five minutes. Not yet tested on an iPhone
- The shaders are MonoGame effects (MGFX version 8): 13 in Content/Effects/*.xnb and SpriteBatch's built-in one in main. shader_split.py reads them (mgfx_effect), each effect parsed to its end; a shader's bytecode is u32 control length, the control section, u32 code length, the code area. 26 shaders (20 pixel, 6 vertex), every binding decodes. A pass may name no vertex shader and keep SpriteBatch's (10 of the 13 effects), so they pair as single-stage programs. Cuphead's and 13 Sentinels' tables split again unchanged
- NSP script: a provisional profile never got the sys_resource the survey lists, so the host answered 32 MB of system resource for every new game; write_profile writes it now (Celeste: 0, as its NPDM declares)
- Survey: XNB files and the MonoGame and FMOD markers (profile_guess.py), and a note on where a MonoGame game's shaders are
- Prewarm list recorded through boot, the title, a new file and the prologue
- NintendoWare Bezel Engine 1.56: nearly all of RomFS is in Bezel Engine archives (.bea, magic SCNE, every entry a zstd frame). shader_split.py reads them (bea_members): 2,125 shaders, most of them the engine's own in System.nx.bea. The 12 geometry shaders of Bezel's line renderer do not build (the main menu chart's outlines are missing); ten pixel shaders that choose their texture handle between two of the game's uniform buffers at run time do not load. The survey names Bezel Engine archives (profile_guess.py, the NSP script's notes)
- nninitStartup picks virtual address memory (a 60 GB StandardAllocator region) or a heap by IsVirtualAddressMemoryEnabled, which the host answered yes for every game. The new profile key sys_resource reports the NPDM's system resource (0 here); the survey writes it for new games, profiles without it keep the 32 MB answer
- SDK 11 calls that reach a service: GetHardwareOpusDecoderWorkBufferSize(int, int), the IR camera (every camera unconnected), LoadCachedNintendoAccountInfo, Enable / DisableIndividualFileDataCache, ldn::GetNetworkConfig and the socket overloads with plain sockaddr
- Movie decoder: the activity previews loop with Flush and Start from four threads. An input index the game held across a flush went back to the free list and was handed out twice; the fifth pending packet overwrote the decoder's mutex and the process died in 2 runs of 5. A send now claims such an index, held indices are withheld for 250 ms after a flush, pend[] cannot overflow, a flush keeps each slot's last picture and an output with no picture leaves the client texture as it was (the black flash at every loop). DM_MOVIE_TRACE=1 logs every decoder call
- The crash handler takes SIGABRT; a command buffer with no GPU start time no longer counts the whole clock as GPU time
- Handheld with Boost (op_mode, perf_mode): handheld, the game reads the touch screen; Boost gives 1920x1080
- 60 fps, sound, previews and saves; the title, enrollment, the menus and activities of every category. About 1.5 GB resident at the first activity, 1.86 GB after 15 minutes of activities (the game's pools as each activity loads). Prewarm list recorded through enrollment and a full activity. Not yet tested on an iPhone
- Capcom's MT Framework: ARC archives (version 7, zlib entries) under nativeNXx64. Its 460 shaders (230 vertex, 230 pixel) are in one shader package (SPK, sc\NX_64\root in sa/NX_64/root.arc), each a code area followed by its control section; shader_split.py reads it (mtarc_members, mtspk_shaders) and pairs the vertex shaders with the single-stage cover. All build, every binding decodes (Mario's combined layout)
- SDK 10 calls that reach a service, from the import list against the hooks: oe::DisableAlbumImageTakenNotification (the first boot's crash), oe::IsInStoreDemoModeEnabled, vi::ListDisplays, fs::RenameDirectory, account save thumbnails, the ec::ShowShop overloads without an ApplicationId, friends::Start*FriendRequest, and ngc::ProfanityFilter's UTF-8 masking, SetMaskMode and Finalize called by name. nvnDeviceSetIntermediateShaderCache and nvnCommandBufferDiscardColor / DiscardDepthStencil are no-ops
- Black screen: the engine copies its textures into a CPU-cached pool with the CPU after creating them and never flushes (main has no cache maintenance at all): tex_flush_upload and tex_dirty_upload, as FINAL FANTASY TACTICS
- 30 fps (the game's own pacing), sound, H.264 movies, saves, the DLC reported in Special Contents; about 1.1 GB resident, flat over 14 minutes from the title to the first trial
- Prewarm list recorded through the first-boot notices, the title, the opening movie and the first episode's opening scenes
- The survey names MT Framework ARC archives (profile_guess.py, the NSP script's notes)
- FNA on .NET NativeAOT with SDL3 and FNA3D's NVN driver; 162 GLSLC outputs in shaders.nx.bin
- Rewriter: data blocks in the text segment are left alone (dm/tools/nso_code.py; main starts with 1.7 MB of data that read as x18 sites); the census applies the same rule. The 52 modules of the 19 other games on hand rewrite byte-identically
- Upside-down picture: nvnCommandBufferSetViewportSwizzles is recorded and a NEGATIVE_Y swizzle flips the viewport
- Logo movie: an H.264 decoder made at Configure (parameter sets in csd-0 / csd-1) now gives RGBA in the native texture mode, not NV12
- SDK calls: vi::CreateLayer with a size; AudioIn (no microphone); the EOS NRO stays unloaded and the game plays offline
- Memory for a 5 GB block: getenv answers the new profile key game_env (DOTNET_GCRegionRange: the GC range 3.75 → 1.5 GB); build cuts for malloc's address space (40 → 2.5 GB), the SDK node area and the SDK's 1 GB alignment of large regions
- Modules: NVIDIA's OpenGL and Vulkan drivers are never loaded (profile_guess.py); the survey's hook list reads dm/gfx too
- dm/host/Makefile: a build for another game always relinks (two builds in the same second kept the first game's host)
- Shaders: 257 GLSLC outputs in its technique files (resources_nx/.../shader/*.tec); shader_split.py reads GLSLC outputs in any loose file no other reader takes (no other profile's tables change)
- Boot hang in 4 of 6 runs: two job-pool workers share one auto-clear event and a wake reached the wrong one; a cut on this build makes the woken worker pass the wake on (patch_nso.py)
- UI textures written by the CPU into uncached pools: tex_flush_upload + tex_dirty_upload
- Translucent UI yellow: an image the shader reads as uint / int gets a view of that class (img_fit; the texture is upgraded to format views before the next pass)
- Dialog boxes with holes: a game barrier with ORDER_FRAGMENTS ends a draw pass that has shader writes (Metal orders nothing between draws of one pass); reflection marks written images rw
- New NVN entry points: CommandBufferBindTextures, SamplerPoolRegisterSamplerBuilder, TextureWriteTexelsStrided, device item 0x67; a clear mask covering all of a format's channels is whole
- NSP script: the title ID comes from the ticket first (this base NSP's file name is tagged ...0001); thread dumps print each condition waiter's arrival order
kpka_data (Monster Hunter Rise's PAKs) imported compression.zstd with no fallback; one helper, zstd_mod, now serves it and the Tears of the Kingdom dictionary paths. The NSP script checks for either module up front, and the README's pip line installs backports.zstd only before Python 3.14.
Outdoors in Breath of the Wild (the user's Great Plateau save, Mac M4 Pro) a frame had about 386
render passes and loaded and stored about 900 MB of attachments, which an iPhone at 60 fps can
hardly move (about 55 GB/s). 220 of those passes ended only because the game updates the first
64 bytes of a 64 KB vertex uniform block before each draw group (nvnCommandBufferUpdateUniformBuffer
inside the pass) while its shaders read up to 544 bytes of it. A deferred update that covers only
part of what a draw reads used to end the pass.
- Partly covered blocks (nvn_mtl.m ubp_get): the shadow starts from memory and takes every inline
update queued in the submission, newest last, incrementally per range. It is refused, and the
pass ends as before, when something else may have written the range by the time the pass runs:
a copy, fill or feedback write queued in the submission, a storage buffer a shader may write or
a texture->buffer copy over it (sbw, by 4 KB page), or a GPU write or counter report of an
earlier submission still running (gw). A counter report of the same submission does not count:
the host writes reports on the CPU after the whole submission completes (Fire Emblem Engage
writes its reports into a uniform block's page). DM_NVN_UB_PARTIAL=0 turns it off;
DM_NVN_UBP_CHECK=1 lands the updates, waits for the GPU and compares every shadow with memory.
- cb_extent.tsv (new tool dm/gfx/tools/cb_extent.py, c1_guard.py's interval arithmetic over the
translated MSL): per shader, how far into each uniform slot it can read, and the storage slots
it never writes. Like c1_full.tsv, the host loads it only for its chain version; build_ipa.sh
copies it, the NSP script writes it for provisional profiles, and the chain procedure
regenerates it. Without it bindings are copied whole (up to 64 KB). Ten profiles have one.
- Route B's reflection never says whether a storage buffer is written, so every one read as
read-only and its writes went unnoticed: a storage buffer now counts as written unless
SPIRV-Cross typed it const (nvn_refl.m sbuf_rw) or cb_extent.tsv proves the slot unwritten.
- Lazy clears (lc_*): a set of targets that gets only clears before the game sets other targets
no longer opens a pass just to store them; the next pass that attaches the same texture level
and layer through the same view clears it as its load action, and the rest land as before
before anything else could see them. Breath of the Wild's four clear-only passes before the
G-buffer fold into its first pass. Off in TBDR mode; DM_NVN_LAZY_CLEAR=0 turns it off.
- The residency set line counts shadows from memory, refusals by reason, and lazy clears.
On the Mac, old (DM_NVN_UB_PARTIAL=0 DM_NVN_LAZY_CLEAR=0) and new side by side on each game's
route; attachment loads + stores per frame:
- Fire Emblem Engage battles: about 1,850 -> 360 MB, 88 -> 38 passes.
- Tears of the Kingdom: about 1,100 -> 700 MB, 124 -> 116 passes.
- Breath of the Wild outdoors: 386 -> about 158 passes, GPU p50 about 9.3 -> 6.5 ms, submit
5.7 -> 4.2-4.9 ms; where the old host fell to 30 or 20 fps under load, the new held 60.
- Link's Awakening, 13 Sentinels: about 15% less. Super Mario 3D World: 5% less (lazy clears).
- Odyssey, Three Houses: 2-8% less. Mario Wonder, Metroid Dread: unchanged.
No new red lines in any of them. The shadow check found every shadow equal to the GPU's memory
(BotW 346,000, Engage 284,000, TOTK 86,000), and t_nvn passes 68/68. Screenshots differ only by
timing. Tried and dropped: DM_NVN_TBDR=1 (it turns the deferred updates off), DM_NVN_BARRIER=game
(no gain), and same-target continuation outside TBDR mode (BotW's decals end the pass anyway).
What still splits BotW's big passes: the GPU particles' compute dispatches (about 190 MB a frame)
and decals that sample the G-buffer. Mac shader libraries were built for the current chain of
TOTK, Engage, Wonder, Dread, Odyssey, Link's Awakening, Three Houses and 13 Sentinels (c1_full.tsv
headers already matched). The IPA ZeldaBreathOfTheWild-2026-10-21-passes.ipa (an earlier state of
these changes plus the uncommitted frame-step probes) awaits an iPhone test.
The app shows the frames put on screen each second in the top row (front_ios.m, from mtl_present_count, so NVN and Vulkan games alike).
build_ipa.sh's DM_SHOW_FPS=1 sets DMShowFPS in Info.plist; DM_SHOW_FPS=0 / 1 in Documents/env.txt overrides it without a rebuild.
The NVN driver's constant segments have the same layout in every game, so the
nvn_* profile keys are gone. Combined textures are at c2 0x20 + 8 r (a slot past
31 shares the image table; Hades II binds 34), images at 0x120, separate textures
at 0x168 (now always the host's slots 128 + n, as Monster Hunter Rise had them),
samplers at 0x568, and the storage buffer tables at c0 0x110 / 0x310 / 0x510.
dmsh_build.py decodes with that layout, the replay fills both texture tables,
and the translator numbers slots the same way (rebuilt; its SPIR-V and
reflection are byte-identical on 1,491 sample shaders, lock updated).
The shader_* switches and graphics are gone too; the build reads them from the
files (profile_guess.py shader_sources):
- Vulkan when main imports Vulkan and not NVN; its capture when the RomFS has
no SPIR-V.
- A GLSLC capture when main holds GLSL sources and imports glslcCompile.
- An Oodle decode for Unreal paks, and the tool-mode decoder for Trinity packs.
- GLSLC outputs from every main.
- A cover pairing for each single-stage vertex shader (GLSLC, Unreal).
build_ipa.sh takes GFXAPI from the NSP script.
Also removed: the last D2R pieces (the embedded-shader path in dmsh_build.py,
dmsh_facts.py --freeze / --check, maxwell_refs.py, ref/hle_hid_d2r.c).
Chain: every profile's dmsh_chain.h is regenerated. The c1_full lists are
carried over: the change touches no c1 read, and the rebuilt Octopath
Traveler 0, Hades II, Unicorn Overlord and Rhythm Heaven Groove gave the same
lists, identical tables and the same reflection apart from separate texture
slots + 128.
Still to do:
- Run the chain procedure for the other NVN profiles.
- Engage's and Three Houses' shader tables need a new split: Engage gains
Unity's 11 GLSLC shaders in main, and Three Houses' 17 BNSH vertex shaders
lose their cover pairs.
- Ace Attorney's new tables are in, but its libraries are not rebuilt.
- Profile 0100F43008C44000 (2.0.2, SDK 19.3.11): 5 GB layout with two cuts (StandardAllocator 12 → 4 GB, node area), mem_map_limit (the pool enforced as on Horizon), shader_oodle.
- Shaders: Trinity packs (arc/data.trpfs, FlatBuffers, Oodle Mermaid) decoded with the game's own OodleLZ_Decompress in the Mac host; 28,603 shaders.
- SDK 19: GetAudioOutBufferCount, crypto random bytes, ToPosixTime with a rule, NVN state getters, CommandBufferFenceSync, the 64-bit WaitForAddress type.
- Graphics: transform feedback (vertex capture entry point, rasterizer discard), layered passes (the colour grading LUT), point size dropped beside a layer output,
function constants specialized, a stencil-only view for STENCIL-mode texture views (striped lighting), sampler min / max reduction (Metal 26),
the draw's base instance at c0 + 0x644 (occlusion boxes; characters' legs folded back from the hips), the zero cube for a 2D texture in a cube slot.
- Shader chain changed (dmsh_build.py): every profile's dmsh_chain.h rewritten; the other profiles' c1_full lists carried over (that change touches no c1 read).
- The loader reads the SDK version from the sdk module's NintendoSDK_nnSdk string
(dm_sdk_major). SDK 10 and older report NVN API 53.313 and point the shared-font
address at the TTF itself; later SDKs 55.15 and the BFTTF header. That is what
every profile set, so device_api and shared_font leave the five that had them
(Odyssey, 3D World, Link's Awakening, Age of Calamity, Three Houses) and
profile_guess.py no longer guesses them; a profile can still override.
- DM_OODLE_JOB finds OodleLZ_Decompress as the one function in main that takes
the address of the library's own name string, with its start from
.eh_frame_hdr, instead of an address per main build: Octopath Traveler 0
main+0x4f77e10 and Pokemon Legends: Z-A main+0x1a9c9e0. Octopath's 10,596
shaders decode byte-identical to before.
- Rows that only restated defaults are gone (pad_layout simple, graphics nvn,
present_interval game), and the NSP script writes no default rows into
provisional profiles.
No chain version changes. All 19 profiles build, the host tests pass; Mac boots
of 3D World (SDK 10.4.0: API 53.313) and Engage (SDK 12.3.7) read the version.
D2R (Diablo II Resurrected) has no profile and its code could not run here: the
host-call svcs its cuts emitted only died, the touch-run-v4 rule set needed a
missing dm/PRODUCT, and the host files its hooks called were never in this
repository. Removed with it:
- rewriter: D2R cuts, observation gates and parameter slots, the touch-run-v4
rule set and private gates, the boot list, the cut scope column and the
KRES / KASPECT key slots (fixups.tsv keeps the four thread slots; no
params.tsv any more); dm/tools/product.sh
- host: D2R save edits, add-on content, render-scale controller, time()
takeover and Terror Zone clock, crash decoding at D2R offsets, host-call
registry and stubs, i18n, UI-screen, shell-app loader, zero-stripped
archives, boot profiling hooks with no implementation; D2R headers, tests
and scripts; dm/ios/Sources/dm_touch_controls; dm/third_party/libdeflate
- graphics: runtime extrapolation, world freeze, iPad UI shift, shop drag
image, brightness LUT, visual ruler, bloom / FXAA shader hashes, DM_GFXUSE,
the eco-fps setting, the loading-screen gate; shader_map.py,
nometal_run.py, nvn_slots.py, nvn_slots.tsv and the PC-texture samples
The legacy frame step (frame_step legacy, DM_FRAME_STEP) is retired: the
Breath of the Wild and Tears of the Kingdom stubs lose their mode test.
Breath of the Wild's interval-controller cuts stay: the framework keeps the
controller's interval while the game's VFR override holds.
patch_nso.py output is byte-identical for all 15 games on this machine except
the two frame-step stubs; all 18 profiles build; the host tests pass (their
Makefile now builds the general tests only).
New profile 0100D9F01D474000 (base game 1.0.0, SDK 21.4.0, Nintendo's own 2D engine; one BNSH of 35 shaders,
Mario's binding layout). On the Mac it plays from a first boot through the setup, the main menu and Play Solo into
Hoop Trundling at 60 fps with sound and no red line. Not yet run on an iPhone.
- SDK 21 calls: nn::pl::GetSharedFont (every shared font at once; the SDK opened the pl service and aborted in
sm::Initialize) returns the stand-in TTFs past the BFTTF header, as the service's offsets point there and the game
hands them straight to ScalableFontEngine::LoadFont; vi::GetDefaultDisplayPanelIdentifier (SIGSEGV through the vi
service object) answers an all-zero panel.
- Audio: SDK 21's DSP command header has no magic (+0 enabled, +1 command ID, +2 size) and DeviceSink is 0x140
bytes with its fields 8 bytes earlier. capture() recognizes that layout on the first non-empty command list (the
walk bounded by the generated list's length); the game was silent.
- Pool initial content (all games): a new texture in a CPU_NO_ACCESS pool whose storage holds data is filled from it
once, as NVN keeps a pool's storage content at nvnMemoryPoolInitialize (tex_init_fill); it stays a GPU texture
that can be rendered into. Linear textures are made on the pool buffer in any pool. The game reads each texture
file into memory and makes a CPU_NO_ACCESS pool on it; every texture read 0 and the screen stayed black.
COMPRESSIBLE, multisample, depth / stencil and sparse textures and storage of zeros are left out.
- NVN block height (all games): the automatic block height truncates height in blocks x 1.5 / 8 (MacNativeGames'
finding, confirmed on this game's 40-block-row BC7 textures and by Touchable on 16 of its files); rounding up
scrambled the calibration screen's beat numbers. Only data a game lays out itself without a packaged layout
changes (Tears of the Kingdom's flushed and Monster Hunter Rise's sparse textures were not re-checked).
- Profile key perf_mode (follow by default): boost reports Boost whatever the operation mode. This game plays
handheld with Boost: docked means a TV to it, which asks for a latency calibration, and Boost alone picks its
1920x1080 draw size.
- The NSP script's survey no longer calls nn::ro missing (hle_ro.c answers it; it now counts the .nro files);
sdis.py --callers takes imported names and scans to the end of the PLT.
The dm/gfx unit tests give the same results as before; Unicorn Overlord renders the same with the old and new host
(no texture takes the initial-content path there).
Octopath Traveler 0 1.0.5 (Unreal Engine 4.26 / 4.27 mobile renderer, SDK 19.3.13) plays from boot through character
creation into the story on the Mac (15 minutes with the iPhone memory layout, no crash); not yet run on an iPhone.
- Shaders: the 10,324 are in the two shader code archives of the game's pak, each one Oodle-compressed. uepak.py reads
the version 11 pak (AES-256 index; the key is the window of main that decrypts it, never printed or written; AES
through CommonCrypto) and the archives; the Mac host decodes them with the game's own OodleLZ_Decompress
(DM_OODLE_JOB, hle_oodle.c, build-gated) and shader_split.py reads the decoded LSLG records (profile key
shader_ue_decode; the NSP script's decode_ue_shaders stage). Bindings measured: Engage's separate texture layout.
- Memory: Unreal's nninitStartup reserves a 62 GB address region and its FMallocBinned3 37 GB of it. A cut skips the
allocator's constructor and the host puts its own FMalloc vtable on a malloc zone in the object (hle_uemalloc.c);
the region is capped at 2 GB with the game's own -SwitchVirtualAddressSpaceSize (new profile key game_args, also
for the iOS app) and SDK 19's node area is cut to 62 MB as Engage's, for a 3.75 GB guest block.
- Host: game arguments are set before nninitStartup, as on hardware; an undefined weak import nothing defines binds
to 0, as rtld binds it (Tears of the Kingdom boots the same with the old and new hosts: its weak glslc* imports
now resolve to 0); nvnCommandBufferFenceSync (OP_FENCE_SYNC), the texture / sampler pool getters,
vi::GetLatestFrameNumber / ListFrameHistory; DM_ALIAS_WINDOWS for sizing fixed-window allocators.
- Rewriter: a byte jump table whose add sits 17 instructions before its br (jump_bytes_far, tried last).
- build_ipa.sh quotes FONT_PY (a python path with spaces failed the font step).
Open: 61 vertex shaders pick a texture slot at run time (dynamic handle), 16 compute shaders with 3D image atomics
do not build. The profile (dm/games/01005270232F2000) holds the shader tables, translation chain b-ba76ba138e78.
New profile 010005501E68C000 (update 1.0.0.2, Unity 2022.3.17f1 IL2CPP, SDK 17.5.3; the base game an XCI):
- Rewriter: two IL2CPP thunks end in an indirect tail call, ldr x18, [x0, #0x18]; ...; br x18 (main+0x1d271bc,
+0x204ebec), which asserted as an unknown jump table. When neither table form is found, the last write of x18 is a
plain load and the br ends the function, it becomes x16 = guest x18; br x16 (tail_call / site_br_call). Tried only
where the rewriter used to stop, so modules it accepted before come out byte-identical.
- sdk cut: the AddressRegionManager node area 1 GB -> 62 MB on this build (sdk+0x139468 / +0x1394fc), as for Engage;
StandardAllocator::Initialize aborted in nninitStartup without it.
- Shaders (unityfs.py, shader_split.py): Unity 2022 keeps each program's parameters as blob entries of their own
(m_ParameterBlobIndices; the programs are m_PlayerSubPrograms[tier][].m_BlobIndex), and subprogram version
202012090 (from 2021.2) has no local keyword list. The player's files borrow the unitybuiltinshaders bundle's Shader
type tree. 4,951 shaders, 4,950 build (one geometry shader); Engage's layout of driver segments decodes every binding.
UnityPy's type trees for 2020.3.18 and 2021.3 have neither field, so Engage reads every entry as before.
- hle_fs.c: nnmusl (open / stat / opendir) checks every path with nn::fs::IsPathMounted, GetPathLengthMax,
GetUtf16PathLengthMax, GetEntryNameLengthMax and GetUtf16EntryNameLengthMax, which the SDK answered from its empty
mount table: File.Exists failed with no nn::fs call, Addressables never found settings.json and the game crashed on
a null asset (main+0x31100ec). Answered from the host's mount table with Horizon's 0x300-byte limits.
- hle_time.c: clock_gettime(CLOCK_REALTIME) reads the source nntimeStartup selects (sdk+0x8e414),
nn::time::GetStandardUserSystemClockCurrentTimespec, which asked the time service; system_clock::now() then called
std::terminate after the first dialog. It and the adjustable clock's getter are the host clock now.
- Memory: Engage's 5 GB block with a 1 GB application pool. StandardAllocator commits 0.19 GB; with a 2.5 GB pool
its reservation and Unity's own pools reached the end of the 4.375 GB alias region after 140 s (probes on
VammManager::AllocateAddressRegion / FreeAddressRegion). 25 minutes now peak at +2.81 GB.
On the Mac: first-boot settings, title, Investigations 1's Episode 1 into the first investigation, 60 fps, FMOD sound,
25 minutes with no red line, 1.55 GB resident and 1.25 GB of Metal allocations, flat.
IPA AceAttorneyInvestigations.ipa (7.0 GB) built from HEAD + these changes; awaits an iPhone test.
Also: UNIMPL stubs log once (the seen counter was a byte and wrapped every 256 calls); build_ipa.sh quotes FONT_PY
(a Python path with spaces broke the font step); profile_guess.py marks IL2CPP by global-metadata.dat (Unity 2022's
main has no il2cpp_ names). Not needed in the end: pull requests #15 (this main's string table is intact) and #16
(nninitStartup sizes no heap with virtual address memory).
Much of the bring-up was found first by armyja (GitHub issues #17, #18, #19, pull requests #15, #16).
Co-authored-by: armyja <546323405@qq.com>
- dm/ios/tools/pack_ipa.py replaces the Payload/ copy and Info-ZIP: it packs the app, the shader
libraries, the add-on content and the RomFS straight from their folders, reading and deflating on
all cores (zip64, modes, symlinks, the profile's zip_store as before). Metroid Dread 156 s -> 12 s,
Breath of the Wild 171 s -> 57 s (the T7 writes at about 310 MB/s).
- The shader libraries no longer go through Xcode (Gen/msl left the project): no 1.5 GB detour
through the home disk.
- NSP script: the rewrite, the shader split and the iOS / Vulkan shaders are reused when a sha256
of their inputs matches the one stored beside their output; the modules are rewritten in
parallel; J = the core count; the extraction is kept by default (--no-keep-extracted).
A Metroid Dread repeat build: about 8 min -> 19 s.
- Profile 0100B04011742000 (update 16.0.2, RE Engine): KPKA PAK shaders (shader_split.py kpka_jobs), XCI base game in nx_extract and the NSP script, BIG_TEXT patch area in patch_nso.py.
- Sparse virtual pool: a 4 KB page table (vpt_*) that the replay reads textures through, dirty tracking per mapped page, texture views from the first mapped level.
- MultiDraw*IndirectCount with a CPU wait for the count; pending GPU writes indexed by page; bindless handles read from queued inline updates.
- sbuf_ubo binding kind, integer vertex formats into float inputs, clears on textures that are not render targets.
- Inline keyboard text, state and finished callback; ngc::ProfanityFilter host vtable.
- Ryujinx BindlessElimination follows phi webs (218 MHR pixel shaders); dmsh_xlate_bin and dmsh_chain.lock updated; dmsh_chain.h and c1_full.tsv rebuilt for every Maxwell profile.
- README: Monster Hunter Rise under being ported, gallery image, XCI base games, Ryujinx local fixes.
dm/games/profile_guess.py decides the profile keys a game's files show, each written with its reason:
graphics vulkan and the modules to load (a subsdk only when main takes something from it the host does
not supply), shader_vk_capture / shader_main_glslc / shader_separable / shader_glslc_capture, the memory
partition (giant bss, no heap, virtual address memory), the SDK 8 / 10 font and NVN API, Hagi's handheld
pointer. --check compares the guess with every profile in dm/games: all 13 agree on these keys.
After the rewrite the script sets mem_used from the rewritten modules (Age of Calamity's 72 MB), writes
the empty shader tables and chain header a provisional Vulkan game lacked, and boots a game with virtual
address memory that sets a heap until its heap request to size the heap region (about 7 s; Tears of the
Kingdom gets its 0x98000000 alias region). A minute's boot could not tell which games need render_cond,
tex_flush_upload or core_preempt, so they stay with the developer.
--provisional builds a game that has a profile from a guessed one (DM_GAMES_FIRST in game.py and the host
Makefile, whose profile header is now rewritten only when it changes); --no-heap-check skips the check.
A capture run whose host stops early now says so. hle_svc.c logs a heap request the pool check refuses.
- Log messages, script output, test labels, Makefile errors and comments are English. Red log lines start with "[RED] " (was 【红】).
- Shader map stage column: vertex / pixel / compute / geometry / hull / domain (generators, dmsh_build / dmsh_facts and nvn_mtl.m stage_of follow).
- The NSP script's regexes match the new shader_split.py / dmsh_build.py summary lines; patch_nso cut scopes are all / host-only.
- Host UI text table (dm_i18n_tab.h) is English only; dm_lang_zh() removed.
- Text that matches game data (t_swkbd.c, t_textbox.c) is written as \x escapes.
- Shader chain: the chain files lose their bilingual comments, so every profile has a new chain version. dmsh_xlate.dll rebuilt from the translated Program.cs (SPIR-V byte-identical on 4,500 shaders of Wonder, TOTK and Age of Calamity), lock and dmsh_chain.h headers regenerated. c1_full.tsv restamped for Dread, 13 Sentinels and Wonder; the other profiles follow when their rebuild finishes.
- AGENTS.md and docs/agents: the language rule now covers logs; quoted log lines updated.
The Wii game on NERD's Hagi layer (PowerPC code recompiled into main), rendering
with Vulkan through Hades' layer. Title screen, file creation and the storybook
prologue into the Star Festival at about 58 fps on the Mac; the IPA awaits an
iPhone test.
- Vulkan layer (hle_vk.m): push constants, descriptor arrays, dual-source
blending variants, copies inside a render pass (cut and resume with deferred
store actions), linear images over their memory, timestamp and occlusion
queries, pipeline caches, vkCmdDraw, image copies and read-backs, USCALED /
SSCALED and three-component vertex formats, viewport and scissor kept across
passes, the INHERIT swapchain transform, the vi layer crop at present.
- vk_shaders.py: ArrayStride on struct members, push constants, array counts,
`_d` dual-source variants, extra SPIR-V input dirs; non-vert/frag stages
skipped with a red line.
- Run-time SPIR-V capture: DM_VK_CAPTURE in the layer, profile key
shader_vk_capture and vk_capture.tsv (3,089 keys), and the NSP script's
Vulkan shader capture stage.
- Handheld pointer: profile key op_mode, touch hooks fill touches only in
handheld mode, InitializeTouchScreen(int,int), DM_KEYS taps, iOS gamepad
forwards untaken touches to the game.
- SleepThread under 20 us takes the yield back-off (Hagi's 100 ns poll kept a
core at 100%).
- nx_extract / NSP script: --allow-damaged for NSPs whose files fail the NCA
hash check.
Hades, Hyrule Warriors: Age of Calamity and Fire Emblem: Three Houses move into the Games table and
the screenshot gallery (Mac captures); Tears of the Kingdom is listed as being ported. The NSP script
now handles a graphics vulkan profile: vk_shaders.py translates the SPIR-V shaders for the Mac and iOS
in place of the NVN shader split and chain build (the change from an uncommitted worktree, ported to
the current script; Hades packed with it: 74 shaders, 3.6 GB IPA). AGENTS.md and the development
notes follow.
Tears of the Kingdom (two profile keys, off for every other game):
- render_cond: NVN conditional rendering is evaluated at replay from the
timestamp reports' counter 0; before, every conditional clear ran and the
lighting passes drew nothing.
- tex_dirty_upload: textures the game fills from the CPU without an NVN flush
(its 121-layer ASTC terrain arrays) are found through write-protected pages
and uploaded; before, they stayed empty (magenta in ASTC).
Breath of the Wild, issue #9 (iPhone 16 Pro Max killed for memory in the first
shrine; an 8 GB iPhone gives the app 3,375 MB):
- Finalized textures' serials no longer stay in the serial table for good:
live serials are marked at nvnTextureInitialize, dead ones resolve through a
512-entry table (about 10 a frame in BotW before, 276k after 25k frames).
- The memory-pool tracking table grows past 4,096 entries (BotW reached 4,006).
- Most of the growth was the defer_drain leak fixed in 0fc0ce4: the shrine now
stays at 2.8-2.9 GB resident on the Mac, about 500 MB under the limit.
- Falls back to 30 fps at the right speed when the GPU can't hold 60: two cuts
make the frame-interval controller always use its GPU-time path, and the
profile key gpu_timestamps segment gives each timestamp report its
segment's GPU start / end (the default, submission, is unchanged). Under
load, interval 2 held at 0.93-0.98 of real time (0.78-0.95 before). The
check that it still holds 60 on a quiet machine is pending.
NVN: clears inside a pass, with a partial channel mask or a partial scissor
are drawn as a triangle in the open pass instead of dropped, and 8-bit indices
written by the GPU earlier in the submission are widened on the GPU (TOTK red
lines 6 / 7 / 48). Adapted from MacNativeGames (MIT, dm/third_party/macnativegames).
NVN vertex fetch: 8 / 16-bit integer attributes fed to an input of the other
sign are widened by their own sign (CPU conversion, red 96 once); before, the
signedness swap widened them the other way and Three Houses' terrain heights
below 0 left the screen (no ground under close-up cameras). The pass dump's d
line shows polygon offset and depth clamp.
NSP script: a GLSLC capture stage for profiles with shader_glslc_capture
(Three Houses' 9 run-time shaders), so only the NSPs and prod.keys are needed.
AGENTS.md: only the coordinating session runs the multi-game smoke test.
Each part was checked on its own game on the Mac. The multi-game smoke test
runs after this push (Wonder and Engage already passed on the same changes
minus the vertex-fetch fix). The translator fix and its chain rebuild are not
in this commit.
scripts/build_ipa_from_nsp.py shows [k/n] stage · step. Where the work can be counted, the line has a bar, done/total, percent, elapsed time and ETA: bytes extracted, modules rewritten, shaders (cache check, translation, compile, read from the files dmsh_build.py writes), files compiled by Xcode, and bytes copied and zipped into the IPA. Elsewhere it shows a spinner. In a terminal or Spyder the line is redrawn in place; piped output gets a plain line every BEAT (30) seconds. PROGRESS / --progress auto|bar|lines|off picks the mode. Tool output now goes only to <id>_build/logs/; a failed step prints its log's last 40 lines and the log's path. No chain-hashed file and no build_ipa.sh change.
- dmsh_build.py vs_init: every float4 vertex output is set to (0, 0, 0, 1) before the translated body, as unwritten components read on the hardware. The translator's MSL started from all zeros; Link's Awakening's surface shaders divide a world position by its w, which the vertex shader never writes, so every sun-shadow compare value was infinite and the whole outdoor scene was lit as if in shadow (dim, no shadows from fences, house or trees). All seven chain versions change.
- Mac libraries and c1_full.tsv rebuilt for Link's Awakening, Engage, 13 Sentinels and Mario Wonder (Breath of the Wild, Super Mario 3D World and Metroid Dread follow).
- DM_NVN_TEXRAW dumps the depth plane of a Depth32Float_Stencil8 texture (shadow maps).
- The IPA's version (CFBundleShortVersionString / CFBundleVersion) is the packed game's display version: the update's when an update is packed, else the base game's (nacp.py --version, DM_APP_VERSION in build_ipa.sh, project.yml; the NSP script prints it).
- README: Link's Awakening screenshot outdoors in Mabe Village.
- README: a gallery with one picture per game (docs/images): iPhone shots
with the virtual gamepad for Mario Wonder, 3D World, BotW and Link's
Awakening; Mac shots (captioned) for Engage, Metroid Dread and 13
Sentinels. Link's Awakening is now listed as in testing on an iPhone.
- scripts/build_ipa_from_nsp.py: a game with no profile is surveyed, then
gets a provisional profile guessed from the survey in
<NSP folder>/<id>_build/profile/<id>/ (outside the repo, reused on the
next run) and a best-effort IPA; --survey-only keeps the old behaviour.
game.py searches DM_GAMES_DIR after dm/games; the host Makefile reaches
such a profile through a symlink; build_ipa.sh finds it through game.py.
zip_store also stores files whose headers say they are compressed.
- copy_t2t_draw: depth-stencil copies now carry the stencil ([[stencil]]
export from an X32_Stencil8 view). Link's Awakening's occlusion pass
tests stencil bit 1 of its quarter-size depth copy, whose stencil was
the new texture's undefined content.
- Link's Awakening: four outdoor shaders with the foliage shader's input
loop failed on the iPhone (Half2 / normalized formats); listed in
shader_facts_manual.tsv. Chain c4995bb3161c.
- Magenta 16x16 blocks in Super Mario 3D World 1-1 (iPhone only): the composite shader 66a71a3081b91b9a
renders into the RGB10A2 image while sampling it in the same pass. That is undefined in Metal: the
M4 Pro reads the old content, the A20 Pro sometimes returns whole 16x16 blocks as (1,0,1). Found with
the new device self-check DM_NVN_MAGSCAN (blocks first appear on that attachment, and still appear
with the suspected particle dropped). fb_prepare ends the pass, copies the attachment level into a
snapshot and binds the snapshot for that draw, only for shaders in the new profile key feedback_snap
(doing it for every shader cost BotW 16.6 -> 36.6 ms). DM_NVN_FB_SNAP=0 / all / <keys> overrides.
- The game language now defaults to English (nn::oe::GetDesiredLanguage returns en-US; DM_LANG=zh-Hans
in Documents/env.txt for Chinese).
- Code comments, docstrings and docs are English only; the zh-CN doc twins are removed. README.md has the
English text followed by the same text in Chinese. Files the shader chain version hashes keep their
bilingual comments (editing them changes every game's chain version).
- Profile 01007EF00011E000 (SDK 20.5.22, NVN 55, main + subsdk0 + sdk): 19,592 shaders, all translated (chain bd3979ea9b3d).
- Loader applies DT_RELR relative relocations (SDK 20; main has 223,994); nvn_slots_got.py and DM_LOADHASH read them too.
- 60 fps: five rewriter cuts lower the GameFrameworkNx dynamic-interval floor from 2 to 1; ksys::VFR scales the
logic time step by the interval (ratio 0.5 at 60 fps, falls back to 30 with ratio 1 when frames run long).
- Black 3D scene: nvnCommandBufferBindTexturesDeferred reads the handles at execution (bh_fresh); the engine
records once and fills the handles later, so the lighting composite sampled texture 0.
- A texture attached at two colour slots is attached once (TBDR tile stores raced, albedo kept scattered tiles).
- Cube textures are 6-layer 2D arrays so tex_fit can bind cube-array views (1,512 shaders sample cube arrays).
- NVN functions other than command recording run on the per-thread host stack (Metal compiled on a small guest stack).
- Address-to-pool lookups use the last hit plus a sorted index (1,760 pools): submit time 21 ms -> 7 ms per frame.
- SDK 20: ARUID hook, audio renderer layout (Span GenerateCommand, new offsets, DeviceSink ID 0x14), nfp events,
VR mode, seven-six-axis sensor, steady clock, launch parameter, stereo vision permission.
- Compressed render targets (BC4 via an RGBA16UI view) are skipped with a red line instead of a Metal abort.
- DM_KEYS takes frame numbers (f2500:A); TEXDUMP saves 2D-array layers; PASSDUMP prints skipped passes.
- NSP script skips empty NSP files and keeps one copy per DLC.
- Profile 010028600EBDA000 (SDK 10.4.0, NVN API 53): 11,080 shaders from Yaz0 SARCs (BNSH, BAHS v9, effects, main).
- Shader key hashes min(0x880, control +0x20) bytes (0x850 / 0x848 controls); other games' tables unchanged.
- Profile keys device_api (NVN API version the game checks) and shared_font (ttf / bfttf at the shared-font address).
- File system hooks run on a per-thread host stack (HLE_FORCE_BIG, hle_bigstack.c): the save thread has 8 KB.
- Washed-out scenes: a one-layer 2D array sampled as texture2d read one value on Metal; tex_fit binds 2D / array / cube views.
- Unwritten colour outputs get a zero write mask; NVN 1D textures are one-row 2D.
- Hooks: InlineKeyboard, settings LanguageCode, ListAllUsers, oe perf / album, nifm SDK 10, prepo session, vi indirect image size;
nvnQueueAcquireTexture, SamplerCompare, debug groups, TextureGetSwizzle / GetStorageClass.
- NSP script builds nx_extract in a temp folder (an apostrophe in the path broke MSBuild silently); DM_KEYS left stick.
- New profile 01003FC01670C000: sdk and subsdk0 are byte-identical to Metroid Dread's; data is six CRI CPKs; the 751 shaders are GLSLC outputs compiled into main (shader_split.py: shader_main_glslc), built as single-stage programs and paired per draw (shader_separable: every vertex shader is paired with a cover of the pixel shader inputs, so vs_pad lets any pair link)
- Host hooks for SDK 11 calls that reach a service: hid::GetMouseState, fs::CleanDirectoryRecursively, fs::EnableGlobalFileDataCache, oe::SetUserInactivityDetectionTimeExtendedUnsafe, time::ToPosixTime, audio::GetAudioOutName
- StandardAllocator::Allocate(size): on this build only, blocks of 64 KB or more are 64 KB aligned; the engine cuts three ring heaps out of a GPU pool at 64 KB alignment relative to its GPU address, and with the host's 4 KB-aligned address the third heap kept base 0 (null write when the prologue enters its first battle)
- NVN: triangle fans and indexed quads, vertex streams 14 and 15 on Metal slots 14 / 15 (Metal assertion buffer index 31), TextureBuilderSetPackagedTextureData copied into the storage at TextureInitialize, InsertDebugMarker as a no-op
- Mario 1.2.1 sets present interval 2 for single frames after a slow one and advances its time step by the interval; frames were shown for one tick and the game jumped ahead on the iPhone. The display link now keeps a frame for its whole period, and the 30 fps target waits until interval 2 has held for a second (Engage, which asks for 1 with its 60 fps cut, is unchanged). Test switch DM_TEST_PRESENT_LAG
- Survey notes: extra modules are supported, GLSLC outputs in main are counted
- Regression: Mario 1.2.1, Engage and Metroid Dread runs show only their known red lines; t_nvn 63/63; t_present_crop / t_mip_extent build again (t_mip_extent's one failure is the same at a9c56fd)
- App name = the game name in the base NSP's control data (control.nacp, saved by nx_extract), read by the new dm/tools/nacp.py that the NSP script and build_ipa.sh share; Engage → "Fire Emblem Engage", Mario → "Super Mario Bros. Wonder"
- Bundle ID = org.nxnative.<base game title ID in lowercase>, e.g. org.nxnative.0100a6301214e000: unique per game and independent of language or version. A new bundle ID installs as a separate app, so existing saves (Documents/save) have to be copied over once
- Overrides: the NSP script's APP_NAME / BUNDLE_ID settings (--app-name / --bundle-id), build_ipa.sh's DM_DISPLAY_NAME / DM_BUNDLE_ID. With no readable name the NSP script stops and says to re-extract
- Profiles drop app_name / bundle_id; a plain name key is only a label for game.py list, error messages and the Mac window title
- Verified: the built IPA's Info.plist has CFBundleDisplayName "Fire Emblem Engage" and CFBundleIdentifier org.nxnative.0100a6301214e000
- Engage's two DLC NSPs (…F001, …F002, 121 KB each) hold one empty marker file each (e0, h0); the content itself is already in update 2.0.0 (Fell Xenologue maps fld_e001–e006, the DLC Emblem room, Patch0–3). The host used to report no DLC and had no MountAddOnContent
- Host: aoc::CountAddOnContent / ListAddOnContent report the numbered folders under DM_AOC_DIR (index = title ID − base − 0x1000; hle_misc.c); fs::MountAddOnContent / QueryMountAddOnContentCacheSize mount DM_AOC_DIR/<index> read-only, a missing index answers fs 2-1002 (hle_fs.c). D2R keeps its fixed {1} without DM_AOC_DIR; DM_AOC_OWNED=0 still reports none
- Packing: nx_extract --aoc <dlc.nsp> <dir> writes a DLC's data NCA RomFS; build_ipa_from_nsp.py extracts every DLC NSP to <id>_build/aoc/<index>/ instead of skipping them; build_ipa.sh copies AOC (default $W/data/aoc) to <App>/aoc; the iOS app sets DM_AOC_DIR when it exists
- Verified on the Mac: Engage mounts e0 / h0 and DLCManager.RefreshHasList marks both contents owned (probe main+0x29f3c88: 2 of 2 with the folders, 0 without); nx_extract --aoc matches hactool; the script step tested on the user's NSP folder
- README games row and a bring-up catalogue entry
- The check that the split shader tables match the game profile compared whole files, so a header comment written by an older shader_split.py stopped the build
- Fire Emblem Engage hit this: shader_map.tsv differs only in its first comment line (checked against a fresh split of the extracted 2.0.0 files: 9,270 data rows identical); rewriting the header instead would change the chain version and force a shader-library rebuild
- nx_extract saves the game icon from the control data as icon.jpg
next to control.nacp (256x256 JPEG; American English first, then
British English, then the first other language).
- build_ipa.sh turns icon.jpg from beside the RomFS (or DM_ICON) into a
1024x1024 single-size app icon in Gen/Assets.xcassets and sets
ASSETCATALOG_COMPILER_APPICON_NAME; project.yml lists the catalog as
optional. Without an icon the build is as before (blank icon).
- build_ipa_from_nsp.py passes the extracted icon (packed version
first, then the base game) and says so when an older extraction has
none.
- Checked: nx_extract on the Engage NSPs saves the English icon; both
IPAs built with icons (Assets.car, AppIcon60x60@2x.png,
AppIcon76x76@2x~ipad.png, CFBundleIcons in Info.plist). README notes
it.
- With OUT_IPA / -o empty, the IPA is written to the NSP folder
(<the profile's ipa>, e.g. FEEngage.ipa), next to the NSPs and
<id>_build/, instead of ~/Desktop.
- The script prints the output IPA and its work folder at the start;
the extracted game files are in <id>_build/extract/ in the NSP
folder (deleted after the IPA is built unless --keep-extracted).
- README and the script header say so. Checked with a run on the
Engage NSP folder (default and -o) up to the free-space check.
- Scripts the user runs now live in scripts/ at the repository root;
build_ipa_from_nsp.py moves there from dm/ios/tools (history kept).
AGENTS.md records the convention.
- It still runs in Spyder (F5, settings block, prompts for an empty
NSP folder or prod.keys). From a terminal it now takes options that
override the settings: <NSP folder>, -k prod.keys, -p version,
-o out.ipa, --app-name, --keep-extracted, --env-file (-h for help).
Without keyboard input a missing value is an error instead of a hung
prompt. Exit codes 0 / 1 / 130 in a terminal; inside IPython (Spyder)
it does not call sys.exit, so the console stays clean.
- The repository root is found by walking up to dm/tools/nx_extract
rather than a fixed depth.
- Checked: -h from another directory; error paths (missing folder, no
terminal input, empty folder, unregistered version) exit 1 with a
clear message; a CLI run on the Engage NSP folder identifies the game,
skips DLC, picks base + update, passes the tool check and stops at
the free-space check; IPython %run under the Spyder interpreter
(prompts, -h, an error) keeps the console alive.
- README, dm/games READMEs and docs/agents point at the new path.
seedds
