0 Fork 0

Files

T

d979c346b9 Integrate demand-first PSOs, far-base tracing and NRO loading

Integrate PRs #75, #90 and #91 with focused correctness fixes:
- Prioritize demanded render pipelines and bound speculative compilation.
- Follow copied jump-table bases and reject W/X register clobbers.
- Collect and rewrite registered NROs, load them on the Mac, and preserve
  runtime activation order through unload/reload and symbol rebinding.
- Preserve refused-only manifests and invalidate collection caches when
  archive readers or symbol parsing change.

Add pipeline scheduling, register tracing, NRO packaging and loader
lifecycle regression coverage. Document that executable NRO embedding
on iOS remains outside this integration.

Validation: 41 Python tests; 48 pipeline scheduling, 102 NVN and 95
geometry checks; three pipeline-key serialization cases; full host test
suite; Mac host build; iOS loader syntax checks; git diff --check.
Gameplay and controlled performance A/B measurements were not run.

Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>

2026-10-24 11:38:12 +08:00

01-extract.md

ROMANCE OF THE THREE KINGDOMS 8 REMAKE (0100C2A01A730000, update 1.1.3, Koei Tecmo's KTGL on SDK 17.5.3): game profile plus three general changes. Plays on the iPhone with a controller (IPA 16.8 GB); on the Mac from the logo movie, the intro and the title through a new game (the recommended setup, the opening story), the council, espionage, a month's missions, the 3D city view, a visit, a practice duel in 3D and a save and load, at 30 fps as on the Switch with sound, about 2 GB resident in the council and 2.5 GB in the duel, flat on a screen left idle. nx_extract: the update is a multi-program patch, the game's patch (…0000) beside a whole second program (…0001, 34 MB, its own Control NCA) that main starts with nn::oe::ExecuteProgram(1); nx_extract took the first Program NCA, the second program, and stopped with "NCA section 1 does not exist" layering it over the base game. It now takes the base NSP's lowest title ID (program index 0) and the update's Program and Control NCAs of that title ID (Balatro and Mario Kart 8 Deluxe extract as before; the host runs program 0 only). shader_split.py: a newer RDB than Age of Calamity's, 296 .fdata files (PDRK0000 and u64 0x10, then IDRK entries) beside data/*.file; flag 0x400000 marks chunks of a new form (u16 stored length, 8 bytes, the stream; a zero length ends; 16 KB inflated at most), LZ4 blocks with 0x100000 too, zlib otherwise; every other flag value is read as before. All 51,655 entries (49,620 LZ4, 2,026 zlib, 9 stored) inflate to the length in their header. The shaders are G2S (_S2G5000 / 6000 / 7000) and _DOK0000 containers again: 59,214 GLSLC outputs there and 149 in main (149 before the change), all built for the Mac and iOS. hle_audren.c: SDK 17.5.3's OpenAudioRenderer with an event calls the form without one through the PLT (sdk+0x1e489c), which the loader binds to the host's hook, so ar_open ran twice for one renderer: the inner call started the render thread before the event existed and the outer one logged a second renderer; the inner call (t_in_open) now goes straight to the SDK. Profile: pad_layout full (the dialogue boxes' auto and log buttons are the right stick and R3; the game reads a USB mouse, not the touch screen), mem_used 80 MB, a prewarm list recorded on the Mac over the route above (451 entries). Left as is: three shaders get textures of another type on some slots (texture type mismatch, once each), with nothing wrong on screen. Checked: the NSP script packs it from the NSPs alone (Mac and iOS shaders, 1 h 19 min and 1 h 29 min on a 12-core M6), the four tool test files pass (34 tests), profile_guess.py gives the profile's memory keys. README (games table, a Mac picture of the duel), AGENTS.md (48 → 49), docs/development.md and docs/agents say so

2026-10-24 08:29:17 +08:00

02-survey.md

Balatro (0100CD801CE5E000, update 1.1.3, LÖVE 11.4 "love-nx" with LuaJIT and SDL on SDK 18.3.2): game profile and the GL layer LÖVE needs, plus four general changes. Plays on the iPhone by touch; on the Mac from the intro and the title into a run and its tutorial, at 60 fps with sound, about 0.65 GB resident and flat over three minutes. The game is Lua source in the RomFS and renders with NVIDIA's desktop OpenGL through EGL, as Digimon does. GL layer (dm/gfx/gl/hle_gl.m): LÖVE reads GL_CONTEXT_PROFILE_MASK and takes its core-profile paths only when it says core (with a compatibility profile it streams vertices from client memory, which Metal cannot draw from), so the layer answers core and implements what those paths use, about 150 functions in all against Digimon's 79: persistently mapped buffers (glBufferStorage, glMapBufferRange: the shared MTLBuffer's own memory), glBufferSubData in command order, fences (a fence is the serial of the command buffer being recorded; a wait on one still recording commits it), glBindAttribLocation (the game's attribute index of each Metal input, set at the link) and current attribute values (glVertexAttrib4f: LÖVE's VertexColor and ConstantColor are disabled arrays), glGetActiveUniform and the uniform read-backs, every glUniform* form, glTexStorage2D / 3D with their level counts, 3D and 2D array textures (LÖVE makes a default of each type on 96 units), GL_TEXTURE_SWIZZLE_* (its font glyphs are RG8 read as R, R, R, G), glGenerateMipmap as a blit (the game's images are mipmapped), instanced and base-vertex draws, separate read and draw framebuffers, glBlitFramebuffer (colour, same size), glReadPixels, glClearBufferfv, glFrontFace and the EGL calls SDL makes; stack arguments of the functions with more than eight are 64-bit, as AAPCS64 lays them out. DM_GL_STUBS=1 gives every GL name the layer lacks a stub of its own that reports its first call, instead of a null pointer. Shaders: LÖVE puts its 35 GLSL sources together at run time in Lua, so the NSP script captures them on the Mac as it does Super Mario Galaxy's SPIR-V (capture_shaders now serves both, profile gl_capture.tsv; profile_guess.py's gl_capture rule: an OpenGL game with no GLSL source in its RomFS or main), and gl_shaders.py keeps a bool uniform's declared type (a uint in SPIR-V; LÖVE's Shader:send takes only a boolean for a bool) and takes from main only strings that start with '#version <number>' (Balatro's main has glslang's messages). General: profile_guess.py leaves vkGetInstanceProcAddr, SDL's Vulkan loader, out of its Vulkan count (Balatro's provisional profile said Vulkan; Hades and both Galaxies import 99 or 100 Vulkan functions); OpenAudioRenderer without an event now starts the host's render thread too (hle_audren.c ar_open: only the form with an event did, and such a game was silent); loader_census.py follows registers in a straight line (adrp, add, loads through relocated GOT entries, mov, csel), for LÖVE's glad, whose pointers are globals its code keeps the addresses of in registers; and the iPhone gamepad has a third layout, Tap (profile pad_layout touch, which needs op_mode handheld; DM_GAME_PAD_LAYOUT replaces DM_GAME_PAD_FULL): no control but the layout button, every touch to the game, offered in the layout cycle only in handheld mode. Profile: op_mode handheld (the Switch build plays by touch there), pad_layout touch, gl_capture.tsv. A Lua error makes Balatro quit (its Switch error handler), and love-nx's video worker is then deleted twice (main+0x339c00): the bring-up notes say how the errors were found. Checked: the NSP script packs it from the NSPs alone (the capture in under a second, 35 shaders for the Mac and iOS, an 82 MB IPA), tool tests pass, profile_guess.py --check changes no other profile; Digimon was not rerun (its files are not on hand). README (games table, a Mac picture of the title), AGENTS.md (47 → 48), docs/development.md, docs/agents and the READMEs of dm/games and dm/gfx say so

2026-10-24 01:50:19 +08:00

03-bringup.md

Integrate demand-first PSOs, far-base tracing and NRO loading

2026-10-24 11:38:12 +08:00

04-graphics.md

Present: commit the residency set after resolving the presented texture, for every game. Super Smash Bros. Ultimate 13.0.5 presents its window texture in its first frames before drawing into it; the texture was made on that first resolution in the present path and joined the residency set (res_add), which only a submission commits, so the display command buffer sampled it outside the committed set: kIOGPUCommandBufferCallbackErrorPageFault at the first presents, after which macOS ignored every command buffer of the process (194 command buffer and 174 upload errors in the user's run) and the window stayed pink. Headless runs never present, which is why they looked right. res_commit is free when nothing changed. Smash windowed now: the opening movie, the DLC first-boot screens, the title, the main menu and a first match render right, no GPU error. Breath of the Wild windowed with only this change on the sync-lag branch its libraries match: its title, 60 fps, no GPU error. docs/agents/04-graphics.md says so

2026-10-24 09:12:22 +08:00

05-debugging.md

Integrate demand-first PSOs, far-base tracing and NRO loading

2026-10-24 11:38:12 +08:00

README.md

Remove the D2R code and the legacy frame step

2026-10-21 02:09:49 +08:00

README.md

Porting a Switch title: agent guide

These pages are for an agent asked to analyse a Switch NSP and port it with the native-execution approach of this repository and d2r-mobile: the game's own aarch64 code runs natively on Apple hardware, a rewriter patches only the instructions Apple's kernel cannot run, the host replaces Horizon system services, and NVN is reimplemented on Metal.

Everything here was learned on Diablo II: Resurrected (SDK ~12, NVN 53, the original target of d2r-mobile) and Super Mario Bros. Wonder (SDK 16.2.0, NVN 55, this repository; its update 1.2.1 uses SDK 16.2.12), then extended by Fire Emblem Engage (Unity, SDK 12.3.7), Metroid Dread (MercurySteam's engine, SDK 11.4.3) 13 Sentinels: Aegis Rim (Vanillaware's engine, SDK 11.4.3, the same sdk and movie module as Dread) Super Mario 3D World + Bowser's Fury (Nintendo EPD's sead / agl engine, SDK 10.4.0, NVN API 53) The Legend of Zelda: Breath of the Wild (EPD's sead / ksys engine, SDK 20.5.22, DT_RELR relocations, update 1.9.0), The Legend of Zelda: Link's Awakening (Grezzo's engine on NintendoWare, SDK 8.2.0, lower-left window origin) Hyrule Warriors: Age of Calamity (Koei Tecmo's KTGL, SDK 9.6.0, RDB archives, shaders as GLSLC outputs) Super Mario Galaxy (a Wii game on NERD's Hagi layer: recompiled PowerPC code, SDK 18.3.5, Vulkan with SPIR-V generated at run time, a 2.9 GB bss), Ace Attorney Investigations Collection (Unity 2022.3 IL2CPP, SDK 17.5.3: the C# file API through the SDK's C library), and the bring-up of Octopath Traveler 0 (Unreal Engine 5, SDK 19.3.13: an AES-encrypted pak, Oodle-compressed shaders that the game's own code decodes on the Mac, its allocator replaced by the host's). Where titles differed, each is given; the next title will differ again, so treat every number below as "measure it, don't assume it".

Page	Covers
01-extract.md	Keys, NSP → NCA → ExeFS / RomFS, update NSPs, the workspace layout
02-survey.md	Static survey of a new title before touching the host: SDK version, imports vs. host hooks, SVCs, instructions to rewrite, NVN surface, data containers, Ghidra / REA setup
03-bringup.md	Getting the host to boot the game: the run loop, how to read the log, and a catalogue of every failure pattern met so far with its root cause and fix
04-graphics.md	NVN on Metal for a new title: object sizes, shader containers and keys, binding layouts, bindless textures, depth mode, index sizes, swizzle, transient render targets, the diagnostic switches
05-debugging.md	Recipes: crash reports, lldb scripts, Ghidra headless decompilation, objdump, memory growth, render-pass and texture dumps, on-device diagnostics, offline pipeline link checks

Ground rules

Bring your own files. Keys, NSPs and anything extracted from them stay in the game's work folder ($W = <NSP folder>/<title>-work, next to the NSPs on the user's external drive; the home folder is nearly full), never in the repository, never in a commit, never printed. Print key names and lengths only.
Measure, then change. Each fix in this repository started from a concrete observation (a crash address and its decompilation, a census over all shaders, a memory profile). Write the evidence into the code comment next to the fix, the way the existing code does; the next title will need it.
One repository, one profile per title. nxnative (first called smbw-mobile) started as a copy of d2r-mobile's generic parts. Title- or SDK-specific constants live in the game profile dm/games/<id>/ (dm/games/README.md). That covers program ID, known builds, modules, address-space partition, NVN driver tables, the shader tables and the app identity. The code reads the profile through game_profile.h (host) or dm/games/game.py (tools). Behaviour that fits only one title's code is gated by that title's build ID (dm_main_build()), never left on for every title. Some things are still built into the translator: the texture / sampler handle bases and the depth mode (see 04). dmsh_build.py refuses a profile whose bases differ from the translator's.
Write code comments in English, and start log lines that mark something unimplemented or wrong with [RED] .
Test where it runs. The Mac host (dm/host/dmhost) is the development loop; the iOS app reuses the same host and graphics code. Never claim something works on iOS without running it there.

Checklist for a new title

Run scripts/build_ipa_from_nsp.py on the title's NSP folder. With no profile for its title ID, it writes <NSP folder>/<title ID>_survey/survey.md:
- modules and build IDs, SDK version and components, engine and graphics API markers;
- RomFS containers;
- the rewrite-site census and imports vs. host hooks.
It extracts only the code modules plus a RomFS list for the survey. Unless given --survey-only, it then writes a provisional profile into <NSP folder>/<title ID>_build/profile/<title ID>/, extracts the game, boots it on the Mac until its heap request when it has virtual address memory (the heap check) and packs a best-effort IPA. That profile is the starting point for step 3: the static keys follow rules checked against every profile, the rest are guesses or defaults (dm/games/README.md); the run-time keys (render_cond, tex_flush_upload, core_preempt, …) are the bring-up's to find. For Mac runs, export DM_GAMES_DIR=<NSP folder>/<title ID>_build/profile until the profile moves into dm/games/.
Extract base (and update, if in scope) — 01. Finish the survey — 02. Write down: SDK version, NVN API version, module list, imports the host lacks, SVCs, instruction census, NVN loader pattern, shader containers.
Add a game profile dm/games/<title ID>/game.tsv, named after the base game's title ID (start from 010015100B514000/game.tsv: identity, build rows, modules; keep the memory / NVN values until they are measured) and build the host for it with make -C dm/host GAME=<id>.
Run the rewriter and check its counts against insn_census.py.
Boot on the Mac and iterate on the failure catalogue in 03 until the game reaches its frame loop.
Extract and translate the shaders, find the binding layout by census, and get the first frame — 04.
Check memory over a minute of running (footprint, the host's memory report line); fix growth before moving on.
Input, audio, then the iOS bundle (d2r-mobile's dm/ios, RomFS inside the app).
Test on a phone: an unsigned IPA the user sideloads, switches through env.txt, results through Documents/log.txt (05). Several bugs only showed there (the command-buffer byte and the audio buffer size in 03, pipeline link failures in later levels in 04).
For a game update: extract it over the base game, diff the imports, carry the per-build addresses over, and rebuild the shader tables for both versions (01).