nxnative-easy/AGENTS.md

Notes for agents

• This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it. Forty-two games are playable on the iPhone: Super Mario Bros. Wonder (update 1.2.1, profile 010015100B514000), Fire Emblem Engage (update 2.0.0, 0100A6301214E000), Metroid Dread (update 2.1.0, 010093801237C000), 13 Sentinels: Aegis Rim (update 1.10, 01003FC01670C000), Super Mario 3D World + Bowser's Fury (update 1.2.2, 010028600EBDA000), The Legend of Zelda: Breath of the Wild (update 1.9.0, SDK 20, 01007EF00011E000), The Legend of Zelda: Link's Awakening (update 1.1.0, SDK 8, 01006BB00C6F0000) Super Mario Odyssey (update 1.4.1, SDK 7, 0100000000010000), Hades (update 1.0.38233, SDK 9, 0100535012974000), Hyrule Warriors: Age of Calamity (update 1.3.0, SDK 9.6, 01002B00111A2000), Fire Emblem: Three Houses (update 1.2.0, SDK 6.5, 010055D009F78000), Super Mario Galaxy (update 1.3.1, SDK 18, 010099C022B96000) The Legend of Zelda: Tears of the Kingdom (update 1.4.3, SDK 20, 0100F2C0115B6000) Monster Hunter Rise (update 16.0.2, SDK 11.4.3, 0100B04011742000), Ace Attorney Investigations Collection (update 1.0.0.2, SDK 17.5.3, 010005501E68C000), Hades II (base game, SDK 19.3.9, 0100A00019DE0000), Unicorn Overlord (update 1.05, SDK 16.2.4, 010069401ADB8000), Rhythm Heaven Groove (base game 1.0.0, SDK 21.4.0, 0100D9F01D474000), Octopath Traveler 0 (update 1.0.5, SDK 19.3.13, 01005270232F2000), Pokémon Legends: Z-A (update 2.0.2, SDK 19.3.11, 0100F43008C44000), Cuphead (update 1.3.7, SDK 5.5.0, 0100A5C00D162000), Super Mario Galaxy 2 (update 1.4.0, SDK 18.3.5, 0100FD8022DAA000), FINAL FANTASY TACTICS - The Ivalice Chronicles (update 1.4.0, SDK 19.3.9, 010038B015560000), Absolum (update 1.1.1, SDK 18.3.2, 0100A83022A22000), Captain Toad: Treasure Tracker (update 1.4.2, SDK 8.2.0, 01009BF0072D4000) and Brain Memory Training Academy (base game 1.0.0, SDK 16.2.4, 0100F5D01DA3A000), The Great Ace Attorney Chronicles (base game 1.0.0, SDK 10.4.1, 010036E00FB20000), STEINS;GATE RE:BOOT (update 1.0.3, SDK 20.5.17, 01008BC024FDE000), Big Brain Academy: Brain vs. Brain (update 1.2.0, SDK 11.4.3, 0100620012D6E000), Patrick's Parabox (update 1.0.1, SDK 13.3.7, 0100E44016C22000), Celeste (update 1.4.1.1, SDK 15.3.2, 01002B30028F6000), DRAGON QUEST XI S: Echoes of an Elusive Age – Definitive Edition (update 1.0.4, SDK 7.3.2, 01006C300E9F0000), Monster Train 2 (update 1.1.1, SDK 18.3.5, 010051701FB46000), Raging Loop (base game 1.0.0, SDK 7.3.2, 010020400E1C2000), DARK SOULS: REMASTERED (update 1.0.3, SDK 4.5.0, 01004AB00A260000), Paper Mario: The Thousand-Year Door (update 1.0.1, SDK 15.3.2, 0100ECD018EBE000), Hollow Knight: Silksong (update 1.0.30000, SDK 20.5.8, 010013C00E930000), Catherine: Full Body (update 1.0.1a, SDK 8.2.1, 0100BF00112C0000), Digimon Story Cyber Sleuth: Complete Edition (update 1.0.3, SDK 6.4.0, 010014E00DB56000), Animal Crossing: New Horizons (update 3.0.3, SDK 20.5.4, 01006F8002326000), DRAGON QUEST VII Reimagined (update 1.2.0, SDK 19.3.1, 0100A9D01C446000) and FINAL FANTASY RESONANCE (update 1.0.1, SDK 20.5.17, 0100606026F12000). Hades renders with Vulkan, which the host implements on Metal (dm/gfx/vk/hle_vk.m; main imports Vulkan, not NVN). Age of Calamity is Koei Tecmo's KTGL engine: files in RDB archives, shaders as GLSLC outputs (shader_split.py idrk_shaders). Three Houses is KTGL too; 9 of its shaders exist only as compiles of the game's own GLSLC at run time, captured once on the Mac (its main holds GLSL sources and imports glslcCompile). Breath of the Wild and Tears of the Kingdom run at 60 fps through a frame step the host decides in whole vsyncs (patch_nso.py FRAME_STEP, nvn_fstep.h, profile keys fps_target and frame_step); Tears of the Kingdom has a 5 GB guest layout. Monster Hunter Rise is Capcom's RE Engine: files in KPKA PAK archives (shader_split.py kpka_jobs, shaders in SDF files), the base game an XCI, its GPU heap a 64 GB sparse virtual pool (hle_nvn.c vpool_*, a 4 KB page table the replay reads textures through); it runs at 60 fps (30 on the Switch). Super Mario Galaxy is the Wii game on NERD's Hagi layer (PowerPC code recompiled into main), renders with Vulkan, and generates its shaders at run time, which the NSP script captures on the Mac (no SPIR-V file in its RomFS); its pointer is the touch screen in handheld mode (profile key op_mode). Ace Attorney Investigations Collection is Unity 2022 IL2CPP: its shaders keep their parameters as blob entries of their own (shader_split.py unity_code_blobs), and its C# file API goes through the SDK's C library, whose path checks the host answers (hle_fs.c IsPathMounted and the length limits). Hades II, unlike Hades, renders with NVN; its shaders bind textures at fixed slots up to 34 (slot 34's handle lies in the driver's image table, dmsh_build.py), and the two long lines through Melinoë while the staff's Omega attack charges are the game's own aim lines, not a rendering bug. Unicorn Overlord is Vanillaware's engine like 13 Sentinels (shaders compiled into main); both need the profile key alloc_big_align, which aligns large allocations to 64 KB so the ring heaps the engine cuts from a memory pool fit. Rhythm Heaven Groove is Nintendo's own 2D engine on SDK 21, the newest SDK so far (its DSP command list has no magic, hle_audren.c); it plays handheld with Boost (profile keys op_mode handheld, perf_mode boost), which skips its TV latency calibration and keeps its 1920x1080 draw size. Octopath Traveler 0 is Unreal Engine 5: everything in one pak with an AES-encrypted index whose key is found in main (dm/gfx/tools/uepak.py), its shaders Oodle-compressed in the pak's shader code archives and decoded once on the Mac by the game's own Oodle (any game with Unreal paks, DM_OODLE_JOB), its allocator replaced by the host's because Unreal's reserves 37 GB of address space (patch_nso.py cut, hle_uemalloc.c), its address region capped by the game's own argument (profile key game_args), and the particle shaders that pick a texture slot at run time bound by slot (profile key tex_dyn_slots); it runs at 60 fps (30 on the Switch) because the host writes Unreal's t.MaxFPS and rhi.SyncInterval in game memory (hle_fpslock.c, profile key fps_target). Pokémon Legends: Z-A is Game Freak's Trinity engine: everything in the packs of arc/data.trpfs (FlatBuffers, every file Oodle-compressed), which the shader split decodes with the game's own decoder in the Mac host's tool mode (shader_split.py trinity_shaders, hle_oodle.c DM_OODLE_SERVE); its 12 GB StandardAllocator is cut to 4 GB to fit a 5 GB guest layout, and the host refuses maps past the pool as Horizon does (profile key mem_map_limit); its characters are skinned with transform feedback (dmsh_build.py vs_tf, nvn_mtl.m tf_draw), and its occlusion queries, which decide how characters are animated, need the draw's base instance in the driver segment (c0 + 0x644, encode_args). Cuphead is Unity 2017.4 IL2CPP on SDK 5.5.0: Unity 2017's serialized and Shader formats are read by unityfs.py / shader_split.py, its rewritten modules (80 MB) need the profile key mem_used above the default 64 MB, or its SetHeapSize fails, and its NVN pool allocator breaks on a zero-size block, so the host never reports a ZCull storage size of 0 (hle_nvn.c zcull_size). Super Mario Galaxy 2 is Galaxy's Hagi layer with Galaxy's sdk build; its movie player reserves 384 MB of virtual address memory, which fits only because a rewriter cut on that sdk build shrinks the SDK's address-region node area to 62 MB (patch_nso.py; the profile's alias region is 512 MB). FINAL FANTASY TACTICS - The Ivalice Chronicles is Square Enix's engine on NVN: its shaders are GLSLC outputs in loose technique files, which shader_split.py reads in any RomFS file no other reader takes; its UI textures are written by the CPU into uncached pools (profile keys tex_flush_upload, tex_dirty_upload); and two job-pool workers share one auto-clear event, so a build cut makes a worker woken in another's place pass the wake on (patch_nso.py). Absolum is FNA (XNA 4 in C#) on .NET NativeAOT with SDL3 and FNA3D's NVN driver: its main starts its text with 1.7 MB of data the rewriter leaves alone (nso_code.py), its shaders are GLSLC outputs in shaders.nx.bin, its picture is flipped by a viewport swizzle the host applies, and everything lives in virtual address memory: malloc's 40 GB StandardAllocator is cut to 2.5 GB and the SDK's node area and large-region alignment are cut (patch_nso.py), and the .NET GC's region range is 1.5 GB (profile key game_env), to fit a 5 GB guest layout. Captain Toad: Treasure Tracker is Nintendo EPD Tokyo's engine (the al library over sead / agl, as 3D World and Odyssey) and needed no host change: it plays handheld because its pointer is the touch screen there (the Joy-Con gyro docked; profile key op_mode), and agl's post-effect program that samples its own attachment reads a snapshot (profile key feedback_snap). Brain Memory Training Academy is Unity 2022.3 IL2CPP with no bundles: its serialized files carry no type trees and nothing lends it a Shader tree, so unityfs.py has Unity 2022.3's Shader layout written out (KNOWN_TREES), and like Ace Attorney Investigations Collection it needs the SDK's address-region node area cut on its sdk build (patch_nso.py). The Great Ace Attorney Chronicles is Capcom's MT Framework: files in ARC archives of zlib entries, its 460 shaders in one shader package inside one of them (shader_split.py mtspk_shaders); it copies its textures into a CPU-cached pool after creating them and never flushes (its main has no cache maintenance at all), so it needs tex_flush_upload and tex_dirty_upload like FINAL FANTASY TACTICS. STEINS;GATE RE:BOOT is M2's ADV engine (E-mote characters, PSB archives, H.264 movies through the SDK's movie library): it needed nn::audio::SetAudioOutVolume (hle_audio.c), and because it decodes every picture into one reused region of a pool that lives for good, the storage-sharing texture table now drops an entry whose storage another texture takes over (nvn_mtl.m alias_take_over); it draws 1280x720 in both modes by its own choice. Big Brain Academy: Brain vs. Brain is NintendoWare Bezel Engine 1.56: nearly all of RomFS is in Bezel Engine archives of zstd entries, its shaders in their .bfsha entries (shader_split.py bea_members); its nninitStartup sets a heap or a 60 GB virtual address region by IsVirtualAddressMemoryEnabled, so the host reports the NPDM's system resource (profile key sys_resource, 0 here); its activity previews loop the movie decoder with Flush and Start from four threads (hle_movie.c); and it plays handheld with Boost (profile keys op_mode handheld, perf_mode boost), because only handheld does it read the touch screen. Patrick's Parabox is Unity 2020.3 IL2CPP with no bundles: unityfs.py has Unity 2020.3's Shader layout written out (KNOWN_TREES), it needs the SDK's address-region node area cut on its sdk build like Brain Memory Training Academy (patch_nso.py), and Unity's player hands the shared fonts straight to FreeType, so a game that imports no ScalableFontEngineHelper::Decode gets the TTF itself (hle_sdk16.c font_skip). Celeste is MonoGame for the Switch, its C# compiled to C++ by Sickhead's BRUTE: its shaders are MonoGame effects (MGFX) in its XNB files and in main (shader_split.py mgfx_effect), and it needed no host change. DRAGON QUEST XI S is Unreal Engine 4.18: legacy paks with LZ5 blocks, decoded on the Mac by the game's own LZ5 (uepak.py, hle_oodle.c), its shaders inline in the cooked packages (shader_split.py ue_legacy_jobs), and the slice-writing geometry shader of its colour-grading LUT folded into the vertex shader (dmsh_build.py gs_slice, nvn_mtl.m prog_fold_gs); Square Enix's sound driver never waits on the renderer event, so audio catch-up frames wait for queue space (hle_audren.c); its movie player plays only once every track holds a decoded output, so audio movie decoders keep 32 (hle_movie.c); and its Unreal region is capped at 4064 MB (profile key game_args, MB up to 4095) with the node area and large-region alignment cut on its SDK 7.3.2 build (patch_nso.py) for a 5 GB guest layout; it runs at 60 fps (30 on the Switch) because the host writes Unreal's t.MaxFPS and nvn.SyncInterval in game memory (hle_fpslock.c, profile key fps_target). Monster Train 2 is Unity 2022.3 IL2CPP on Super Mario Galaxy's SDK 18.3.5 sdk build: Unity creates a thread only when nn::os::GetFreeThreadCount (svcGetInfo type 24, hle_svc.c) is at least 1; its MP4 intro needs a movie audio decoder with more output slots than a video one, because Unity's player holds about half a second of audio before it plays (hle_movie.c MD_NOUT); and its post-processing compute shaders (the colour grading LUT among them) are Unity ComputeShader objects, which shader_split.py reads (unity_compute) with Unity 2022.3's layout written out in unityfs.py. It reads no touch, so it stays docked with the 3D pad layout, and it runs at 60 fps (30 on the Switch) with the profile key present_interval display, as its animations follow the time that passed. Raging Loop is Unity 2018.3 IL2CPP with no bundles on DRAGON QUEST XI S's sdk and subsdk0 builds: unityfs.py has Unity 2018.3's Shader layout written out (KNOWN_TREES, an older form than the 2020 and 2022 layouts, with the parameters in each subprogram), and it needed no host change; it reads no touch, so it stays docked with the 2D pad layout. DARK SOULS: REMASTERED is FromSoftware's engine on SDK 4.5.0, the oldest SDK so far: its shaders are GLSLC outputs in BND3 archives inside the dvdbnd archives (shader_split.py bdt_shaders), and SDK 4's GLSLC outputs are read through their section headers (glslc_section); it keeps 12 sampler pools with different samplers under the same IDs, so handles a shader reads from a uniform buffer take the base of the pool bound at replay (nvn_mtl.m ubo_tex_id / ubo_samp_id); and it runs at 60 fps (30 on the Switch) with two rewriter cuts, its frame limiter's 30 and its frame update's fixed 1/30 s step (patch_nso.py). Paper Mario: The Thousand-Year Door is Intelligent Systems' engine on NintendoWare, on Celeste's SDK 15.3.2 sdk build: nearly every RomFS file is zstd (.zst, which shader_split.py decompresses by magic); its AngelScript call bridges leave sp misaligned, which a general rewriter rule rounds up (site_sp_align); it runs at 60 fps (30 on the Switch) by the host's frame step (FRAME_STEP, with the floor of its vsync step cut from 2 to 1); and its three job workers poll the job list with no SVC, which held it at 24 fps on an iPhone until the general spin wait rule (patch_nso.py "Spin waits", hle_thread.c dm_spin_wait). Hollow Knight: Silksong is Unity 6 (6000.0.50f1) IL2CPP, the newest Unity so far: its bundles carry their own type trees, so unityfs.py needed no layout; Unity 6 polls nn::hid::GetDebugPadStates every frame (hle_hid.c), and it takes all its memory itself through nn::os::AllocateAddressRegion / AllocateMemoryPages with no heap, so it runs in the 5 GB guest layout that profile_guess.py now gives every game with an NPDM system resource, with the SDK's address-region node area cut on its sdk build (patch_nso.py). Catherine: Full Body is Atlus's GFD engine with CRI middleware (CPK archives, ADX2 sound, Sofdec2 movies through movie::Decoder, whose player reads nv12-uv-stride, hle_movie.c): its 3,128 shaders are GLSLC outputs in data/shaders_binary, main and two compiles of its own GLSLC at boot, which the host lets run by reporting at least that GLSLC's GPU code version (hle_glslc.c); that GLSLC writes every vertex shader as a VertexA part (shader type 0, the position) and a VertexB part, which dmsh_build.py merges as Ryujinx does (vertex_ab; the screen was black without it), and because the game reuses a copy's staging memory a frame after recording it, the host commits a segment at the first pass boundary after a buffer → texture copy (nvn_mtl.m g_cut_copy). Digimon Story Cyber Sleuth: Complete Edition is Media.Vision's MVGL engine and renders with NVIDIA's desktop OpenGL 4.5 through EGL, not NVN, which the host implements on Metal (dm/gfx/gl/hle_gl.m: the 13 EGL imports and the 79 GL functions its code calls of the 2,956 its glad loader fetches, counted with dm/tools/loader_census.py; profile_guess.py gives graphics gl when main imports eglCreateWindowSurface and names glShaderSource); its shaders are GLSL ES 3.20 source in its MVGL archives (dm/gfx/tools/mvgl.py: MDB1 with doboz entries) and in main, which gl_shaders.py translates offline with glslang and SPIRV-Cross (packing needs brew install glslang) and the host finds by the hash of its text. Animal Crossing: New Horizons is Nintendo EPD's engine (sead / agl / gsys) on SDK 20.5.4: its main links its own HIPC client for olsc:u (cloud backup settings), which the host answers with sessions of its own (hle_sf.c), its island lives in device save data (MountDeviceSaveData, hle_fs.c), it skips clears with conditional rendering (profile key render_cond), its 63 MB main needed halfword jump tables padded only to their largest entry (patch_nso.py half_count), and its SaveProc builds a new island in grid walks that yield once per cell, so a yield whose loop moved its callee-saved registers since the last one now returns at once instead of backing off, for every game (hle_thread.c thr_yield; the island showed after 77 s of black screen, now about 10 s). DRAGON QUEST VII Reimagined is Unreal Engine 4.27: IoStore containers beside one pak, its shaders LZ4 blocks in the pak's shader code archives, which uepak.py decodes itself (lz4_shaders), and its allocator replaced as Octopath Traveler 0's (patch_nso.py cut and observation gate, hle_uemalloc.c layout UE4_27) in a 2 GB engine region (profile key game_args); its audio renderer runs at 32 kHz, resampled by the host on the CPU path (hle_audren.c); and Unreal's volumetric lightmap merge sizes its indirection texture with a query on the image it writes, so a texture slot that shares its c2 word with an image slot reads the bound image (nvn_mtl.m tex_slot_h) and the translator types such a query from the image (dmsh_xlate ImageTypesOf): without them every character in the opening cutscene glowed white; it runs at 60 fps (30 on the Switch) because the host writes Unreal's t.MaxFPS and rhi.SyncInterval in game memory (hle_fpslock.c, profile key fps_target), the same two limits a 60 fps mod for the base game lifts with code patches (UEngine::SetMaxFPS returning at once, the name r.VSync broken). FINAL FANTASY RESONANCE is Unreal Engine 5.6.1, the newest Unreal so far: its shader libraries are IoStore chunks in its container, not in its pak, two layers of Oodle that uepak.py reads through the game's own decoder in the Mac host with no boot (Toc, io_archive, OodleHost); its 94 MB main is 95% without unwind entries, so the rewriter analyses long functions in pieces (patch_nso.py pieces; one 82 MB function took 100 GB of memory) and treats code before the first unwind entry as a function; its allocator is replaced as Octopath Traveler 0's (UE 5.6's 32-slot FMalloc, hle_uemalloc.c layout UE5_6), and its engine region is 4064 MB (profile key game_args), because the engine's own page allocator never gives address space back (2 GB aborted in the first battle); 2,077 particle shaders need tex_dyn_slots; it runs at 60 fps (30 on the Switch) by the host's rhi.SyncInterval write (hle_fpslock.c, profile key fps_target). README.md is the player-facing page (games, packing, playing); docs/development.md has the internals, the script's options, building by hand, prewarm lists and per-game notes.

• Each game's constants and shader tables form a game profile in dm/games/<id>/, where <id> is the base game's title ID: game.tsv plus the tables and dmsh_chain.h. Mario's profile is 010015100B514000. Tools pick the profile with DM_GAME; make -C dm/host and build_ipa.sh pick it with GAME. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code (dm/games/README.md).

• Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it.

• Per-build knowledge is keyed by the NSO build ID: rewriter cuts in dm/tools/patch_nso.py, the profile's build rows (the builds the game was brought up on, and their display versions; the NSP script packs any other version from a derived profile, without those builds' fixes), and host behaviour that uses fixed module addresses checks module_t.build_id (from fixups.tsv; dm_main_build()). When a new game build appears, carry addresses over with dm/tools/nso_fnmap.py and check them by disassembly; never let fixed addresses run on a build they were not found in.

• To analyse or port another Switch title, first run scripts/build_ipa_from_nsp.py on its NSP folder. With no matching profile, it writes a survey to <NSP folder>/<title ID>_survey/survey.md, then a provisional profile guessed from the game files (dm/games/profile_guess.py; <NSP folder>/<title ID>_build/profile/<title ID>/, found by the tools through DM_GAMES_DIR), with a short heap check on the Mac when the game has virtual address memory, and a best-effort IPA that may not boot. Then follow docs/agents (extraction, binary survey, host bring-up failure catalogue, graphics specifics, debugging recipes) and add a profile.

• Scripts the user runs live in scripts/ at the repository root (now build_ipa_from_nsp.py). Each one runs both in Spyder (F5, configured by the settings block at its top) and from a terminal (command-line options override those settings; non-zero exit on error). Tools that only agents or other scripts call stay under dm/.

• Game files, keys and anything extracted from them live in the game's NSP folder, never in this repository and never in the home folder (it is nearly full): the work folder <NSP folder>/<id>-work for extraction, runs and analysis, beside the NSP script's <id>_build/. Tool venvs and other large scratch data go on that drive too. Do not print key values.

• Language: everything is in English: README.md, docs, code comments, log messages, script output and other files. Red log lines (something unimplemented or wrong) start with [RED] . Text that must match a game's own Chinese data is written as escapes ("\xE8\xB5\xB7"), never as characters.

• After changing anything on the shader translation chain, run the chain procedure for each affected profile. The chain covers dm/gfx/tools/dmsh_*, Program.cs, the profile's shader_map.tsv / shader_facts.tsv / shader_pairs.tsv, and its shader_* keys. The procedure:

  1. Run dmsh_chain.py --header, which writes the profile's dmsh_chain.h.
  2. Rebuild the shaders (Mac and, with IOSOUT, iOS).
  3. Regenerate the profile's c1_full.tsv (c1_guard.py) and cb_extent.tsv (cb_extent.py).
  4. Rebuild the host.

  The host only loads libraries of its own chain version. The translator is dm/gfx/tools/dmsh_xlate_bin, which matches dmsh_chain.lock; a fresh dotnet publish does not.

• iOS builds are unsigned IPAs that the user sideloads and tests; diagnostics come back as the app's Documents/log.txt. Do not sign or install builds on the user's devices.

• Commit only when asked, with git -c user.name=seedds -c user.email=seedds@gmail.com commit.