Files
seedds 3a993fa3fe Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in <game>" subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games/<id>/README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv)
2026-10-25 12:31:58 +08:00
..
patch_nso.py split into the general rewriter and per-build patch files (GitHub issue #63): every game's bring-up edited the one file, so PRs for unrelated games conflicted. dm/tools/patch_nso.py (1,711 → about 1,100 lines) keeps the general passes and the driver; the instruction encoders move unchanged to dm/tools/nso_asm.py; a game's own patches (CUTS, OBS and the frame-step stubs, now FRAME_STEP = {build ID: stub(ctx)}) move verbatim with their comments to dm/games/<id>/patches.py (16 profiles: Tears of the Kingdom, Breath of the Wild, Age of Calamity and Paper Mario: The Thousand-Year Door with their stubs), and the cuts of Nintendo's sdk module builds, 6 of the 15 shipped by 2 to 4 games, to dm/tools/sdk_patches.py. The rewriter loads every patch file, whatever the game, and applies those of a module's build ID at the points it did before (cuts before the scan, gates after the rewrite, stubs after the gates); a build ID claimed twice stops it, and its command line needs no DM_GAME. build_src.sha256 lists the rewriter's files and every patch file (the same for every module). NSP script: the rewrite step's reuse key counts nso_asm.py and only the patch files that name one of the game's builds (patch_files), profile_files leaves patches.py out (only the rewrite step reads it), and a derived profile gets none. dm/tools/tests/test_patches.py: no build claimed twice, the entries' forms, a game's file names only its own profile's non-sdk builds, sdk_patches.py only sdk builds some profile ships, and the reuse key's patch files. Checked byte-identical: every extracted module on the game drive (131 builds) rewritten with the previous and the new rewriter gives the same outputs but build_src.sha256, 28 of the 30 patched builds among them (all four frame-step mains), and 7 fail alike (rtld, and New Super Mario Bros. U Deluxe's, which no profile rewrites); the other two (Super Mario Bros. Wonder's mains and Absolum's, no game files here) by their CUTS / OBS data, equal to the previous tables. AGENTS.md, docs, dm/games/README.md, the profiles' comments and the host comments that named a game's cut in patch_nso.py now name its patches.py or sdk_patches.py
2026-10-23 23:36:21 +08:00
patch_nso.py split into the general rewriter and per-build patch files (GitHub issue #63): every game's bring-up edited the one file, so PRs for unrelated games conflicted. dm/tools/patch_nso.py (1,711 → about 1,100 lines) keeps the general passes and the driver; the instruction encoders move unchanged to dm/tools/nso_asm.py; a game's own patches (CUTS, OBS and the frame-step stubs, now FRAME_STEP = {build ID: stub(ctx)}) move verbatim with their comments to dm/games/<id>/patches.py (16 profiles: Tears of the Kingdom, Breath of the Wild, Age of Calamity and Paper Mario: The Thousand-Year Door with their stubs), and the cuts of Nintendo's sdk module builds, 6 of the 15 shipped by 2 to 4 games, to dm/tools/sdk_patches.py. The rewriter loads every patch file, whatever the game, and applies those of a module's build ID at the points it did before (cuts before the scan, gates after the rewrite, stubs after the gates); a build ID claimed twice stops it, and its command line needs no DM_GAME. build_src.sha256 lists the rewriter's files and every patch file (the same for every module). NSP script: the rewrite step's reuse key counts nso_asm.py and only the patch files that name one of the game's builds (patch_files), profile_files leaves patches.py out (only the rewrite step reads it), and a derived profile gets none. dm/tools/tests/test_patches.py: no build claimed twice, the entries' forms, a game's file names only its own profile's non-sdk builds, sdk_patches.py only sdk builds some profile ships, and the reuse key's patch files. Checked byte-identical: every extracted module on the game drive (131 builds) rewritten with the previous and the new rewriter gives the same outputs but build_src.sha256, 28 of the 30 patched builds among them (all four frame-step mains), and 7 fail alike (rtld, and New Super Mario Bros. U Deluxe's, which no profile rewrites); the other two (Super Mario Bros. Wonder's mains and Absolum's, no game files here) by their CUTS / OBS data, equal to the previous tables. AGENTS.md, docs, dm/games/README.md, the profiles' comments and the host comments that named a game's cut in patch_nso.py now name its patches.py or sdk_patches.py
2026-10-23 23:36:21 +08:00
Subsystem references and a symptom index: the failure catalogue of docs/agents/03-bringup.md and the per-title material of docs/agents/04-graphics.md (section 1's container list, section 6 and its "Seen in <game>" subsections) move into docs/reference/, one heading per mechanism: rewriter.md, memory.md, threads.md, services.md, movies.md, audio.md, frame-rate.md, nvn-objects.md, rendering.md (with the Vulkan and OpenGL layers), shaders.md and game-state.md, indexed by docs/reference/README.md. Each entry gives the symptom, the cause, the fix (file and function), the games it was found on (linked to their notes in dm/games/<id>/README.md) and any check, caveat or later change; one mechanism met on several games is one entry. docs/agents/symptoms.md lists every entry's symptom (an assertion, a [RED] line, an abort or crash site, a visible result) with a link to it, 350 lines, generated from the Symptom bullets. 03 keeps the loop and reading the log, and its Failure catalogue section now points to the references, the index and where a new finding goes; 04 keeps sections 1-5 and 7 (the container list is shaders.md's Containers by engine), and section 6 says where each kind of finding is. Facts in the old sections about one game only were checked against that game's note, where they already were. Statements the code no longer matches are marked as history beside the current value (movie output slots: MD_NOUT 64, MD_NOUT_AAC 32, MD_NOUT_VIDEO 16; APP_POOL is the profile key app_pool; Engage's layout is 5 / 4.375 GB; inline uniform updates are deferred by default; scaled copies are drawn), and Tears of the Kingdom joins the node-area entry's build list (issue #70). The game notes, docs/agents/README.md, 02 and 05 link to the exact entries, and so do the comments in nvn_mtl.m (nvnTextureCompare) and hle_audio.c (the RemoteIO buffer) and the provisional-profile note in profile_guess.py. 03 goes from 29,600 to 470 words, 04 from 34,400 to 3,500. Checked: every backticked span, module address, issue / PR number and 16-digit hash of AGENTS.md, development.md, dm/games/README.md and docs/agents/01-05 at 6a06877 (4,600 items) still appears in the repository's Markdown (the same five file patterns as before appear as the full names they stood for), every relative link and heading anchor in the repository's Markdown resolves, and the tool tests pass (the four that need capstone or lz4 in the T7 venv)
2026-10-25 12:31:58 +08:00

DRAGON QUEST VII Reimagined (0100A9D01C446000)

Update 1.2.0 (v131072) over the base game, SDK 19.3.1. Square Enix / HEXA DRIVE, Unreal Engine 4.27 (PhysX 3.4, so UE4; the source paths say UEngine/4.27-source): IoStore containers beside one pak, the desktop renderer with forward shading, NVN, CRIWARE sound (a 6-channel AudioOut) and Sofdec movies (VP9 through the SDK's movie decoder, mode 1, NV12). Modules main, subsdk0 (the movie module, which Sofdec feeds its VP9 video to) and sdk. Plays on the iPhone at 30 fps; 60 fps (30 on the Switch) on the Mac by the host's writes of Unreal's frame limits.

Status: brought up on the Mac: the logos and opening movie with sound, the title, a new adventure log (name entry, settings), the opening cutscene (the well, the night scene with Maribel by the boat) into play at Pilchard Bay with the controls tutorial, and a saved log continued, at 30 fps with saves; the opening cutscene into the first free play (the hero's house) again at 60 fps (fps_target 60). No red line at run time apart from the geometry shaders below; about 2.6 GB resident (1.1 GB of Metal allocations) in the cutscene. Playable on an iPhone at 30 fps; 60 fps on an iPhone is not tested yet.

What it needs

Setting Where Why
fps_target 60 game.tsv, hle_fpslock.c the host writes 60 into t.MaxFPS and 1 into rhi.SyncInterval; see Frame rate
FMallocBinned3 constructor cut: main+0x32758dc bl 0x291714 → nop patches.py, main f7ec03e4… its pools take 37 GB of the engine region; see Memory
Observation gate OBS_UE_MALLOC at main+0x32758e4 (orig mov x0, x19) patches.py puts the host allocator's vtable into the new object in x19 (hle_uemalloc.c layout UE4_27)
game_args -SwitchVirtualAddressSpaceSize=2048 game.tsv a 2 GB engine region instead of 62 GB
Node area cut 1 GB → 62 MB (sdk+0x1423fc / +0x142468) dm/tools/sdk_patches.py, sdk build 145dde94… the same AddressRegionManager constructor as Octopath Traveler 0's SDK 19.3.13 build, at the same offsets
3.75 GB guest block: mem_reserve 0xF0000000, mem_alias 0xC8000000, mem_stack 0x10000000, app_pool 0xCD500000 game.tsv alias region = node area + up to 1 GB of alignment (regions of 1 GB and more are 1 GB aligned) + the 2 GB region; 256 MB stacks, 384 MB heap region
sys_resource 0x5000000 game.tsv the NPDM declares 80 MB of system resource
Integer output row f555b53f62ec5196 (fout 0 uint) shader_facts_manual.tsv ShaderArchive-Global-SF_GLSL_SWITCH shader 688 (pixel) writes integers it computes per pixel, bit-cast into its output, to an R8Uint target (seen at the camera settings' preview in the new-game menus); Unreal's GLSL_SWITCH shaders carry no output types, and Metal refuses a float output on an integer attachment
pad_layout full game.tsv a 3D game with a free camera on the right stick. It reads the controller only, so it stays docked (1920x1080)

Frame rate

  • Current: 60 fps (30 on the Switch). The game's config caps Unreal at 30 twice: FrameRateLimit=30 in DefaultGameUserSettings.ini and SwitchGameUserSettings.ini, which UEngine::SetMaxFPS (main+0x134d2c0) puts in t.MaxFPS, and rhi.syncinterval=2 in SwitchDeviceProfiles.ini, the present interval 2 it asks for while r.VSync=1 (BaseSwitchEngine.ini) locks presents to vsync. As for Octopath Traveler 0 (01005270232F2000), the host writes 60 and 1 into the two console variables (hle_fpslock.c, profile key fps_target 60) whenever the game sets them back, and the game then asks for present interval 1 itself.
  • StevensND's 60 fps mod for the base game's main (build ab0602…; GameBanana) lifts the same two limits with patches in code: it returns at once from UEngine::SetMaxFPS, so t.MaxFPS stays 0 (no limit), and it overwrites the first two characters of the UTF-16 name r.VSync, so the config's r.VSync=1 finds no variable and presents are no longer locked to rhi.SyncInterval. Its "Uncap" variant also patches the sdk for present interval 0. Neither patch touches how the game steps time.
  • Unreal steps the game by the time that really passed. On the Mac, DM_NVN_REGLUM traces of run/to_maribel.sh from the same save at 30 and 60 fps (frame time p50 33.4 → 16.7 ms) match at time scale 1.00 with a constant offset (−1.75 ± 0.2 s over the last 90 s, correlation 0.998–1.000), the night scene comes out at the same resolution, and no new red line appears.
  • DM_FPS_TARGET=30 (Documents/env.txt on the iPhone) keeps the game's own 30 and saves power.

Memory

  • nninitStartup (main+0xe89fa4) needs virtual address memory, sets a 6 MB heap for the C library's malloc and reserves the same 62 GB region as Octopath Traveler 0's (main+0xe8a2c0) unless -SwitchVirtualAddressSpaceSize= caps it. Its FMallocBinned3 cuts 37 GB of pools out of that region (the mapped high-water mark reached +38 GB of a 64 GB alias region at boot), so a capped region would starve it.
  • Current: the allocator is the host's. Here the allocator is not in the bss: GCreateMalloc (main+0x32758b0) mallocs 0x1c38 bytes on the first allocation, during the static constructors, constructs them (main+0x32758dc → main+0x291714) and stores GMalloc (main+0x77767e0). The constructor call is cut, and the observation gate right after it (OBS_UE_MALLOC, x19 = the new object) puts the host allocator's vtable in (hle_uemalloc.c). No code outside the allocator reads its pool base (main+0x7744770, read only there) or calls its methods directly: every FMemory call goes through the vtable. UE 4.27's FMalloc has one FExec slot, not four, so the slot numbers differ from UE5's (Malloc 3 … GetAllocationSize 9; Free is main+0x10, right after the 16-byte module header).
  • How the allocator was found: from the commit calls (probe nn::os::AllocateMemoryPages past its adrp, sdk+0x140f84: every backtrace runs through the allocator's methods around main+0x3270000), the relative relocations that point at them (the vtable main+0x5e4c0e8; this build has no RTTI) and the one store of that vtable outside the destructors.
  • Current: a 2 GB region (game_args -SwitchVirtualAddressSpaceSize=2048) with Octopath Traveler 0's node-area cut on this SDK build, in a 3.75 GB guest block. The Unreal heap lives on a malloc zone outside the guest block.
  • On the Mac: up to +0.75 GB of the alias region with 0.5 GB mapped through the opening and the new-game menus, beside an Unreal heap of 1 GB and 0.9 GB of Metal allocations (about 2.3 GB resident; game.tsv); about 2.6 GB resident in play (the Unreal heap about 1.25 GB, 0.9 GB of Metal allocations).

Rendering

The desktop renderer with forward shading: one RG11B10 colour pass draws every object, lit per pixel from the light grid, the volumetric lightmap and the screen-space shadow mask, with HEXA DRIVE's toon lighting on top.

  • Resolved: the hero drawn as a dithered silhouette inside a dithered square. Unreal's stencil shader resources are second textures on the depth buffer's storage built with nvnTextureBuilderSetDepthStencilMode(STENCIL), which the host only logged, so the character occlusion mask (edbfc7a088be545e) read depth where it tests stencil bits. The builder's mode now goes to the texture (tex_t.stencil), whose sampling view is the X32_Stencil8 one a stencil-mode texture view already got.
  • Resolved: characters and props glowing white in the opening cutscene only, about 10⁴ too bright, while walls and gameplay looked right. The per-pixel volumetric lightmap read brick (0, 0, 0) of the brick atlas for almost every cell. Unreal merges each streamed level's lightmap into one atlas on the GPU: brick copies (0a7ff4171bd1a82a), then a patch of the shared 16³ RGBA8_UINT indirection texture (7b958a7b21e72801) whose bounds come from GetDimensions on the very RWTexture3D it writes. That size query is a TXQ on the image's handle, c2 0x120 (image slot 0): the build numbered it combined texture slot 32, the slot that shares that word, and with no type in the facts table the translator read it as 2D. The host bound the empty texture slot 32, the 1×1 dummy's size passed only thread (0, 0, 0), so only that cell was written, every other cell kept size 0, and the shader fell back to the atlas origin: the first daytime level's bricks, baked about 2¹⁵ brighter (the same bricks in gameplay hold 0.5–1.4). Two general fixes: a texture binding of slot 32–39 reads the image bound in slot 0–7 when there is one, as c2 holds it (nvn_mtl.m tex_slot_h), and a query on an image handle the facts do not type is translated again with the type of that image from the first translation (dmsh_xlate ImageTypesOf).
  • How the glow was found: the raw scene colour (texraw) put the characters at 1,000–2,000 and the walls at 0.05; a debug build of one character shader that wrote its terms in alternating 8-pixel columns isolated the volumetric lightmap ambient term; dumps of the atlas (texlist / texraw), of the uploaded level bricks (the 3D upload dump) and of the indirection texture, with the cell computed from the primitive's origin and View VolumetricLightmapWorldToUVScale / Add (View 220 / 221), gave the rest.
  • Integer textures and images read and written through float declarations (the indirection RGBA8_UINT texture read by texture3d<float> and bit-cast, R16_UINT / R32_UINT light grid buffers through texture_buffer<float>, integer stores into a float-declared texture3d image): checked on an Apple GPU, all keep the integer bits, so they need no facts.
  • Unbuilt: 62 of the 63 geometry shaders (one-pass cube shadow maps, six layers from one draw) do not build; their draws are skipped (point light shadows).

Files and shaders

  • Cooked packages in IoStore containers (DOLL/Content/Paks/pakchunk0-Switch.utoc / .ucas, 2.8 GB, and smaller chunks beside it; .ucas is the profile's romfs_marker); everything else in pakchunk0-Switch.pak (1.5 GB, 3,972 files: configs, the asset registry, fonts, CRIWARE .awb banks, the two shader code archives). The other pakchunk*-Switch.pak files are 339-byte empty paks.
  • The pak is version 11 with the same 0xdd-byte footer as Octopath Traveler 0's, but the magic sits 0xcc from the end, not 0xc8 (five whole 32-byte compression method names, Zlib and Oodle); uepak.py takes either. The index is not encrypted. The update (v131072) is a delta over the base RomFS, so the merged extraction holds the whole game.
  • Shaders: ShaderArchive-Global-SF_GLSL_SWITCH.ushaderbytecode (1,764 shaders) and ShaderArchive-DOLL_Chunk0-SF_GLSL_SWITCH.ushaderbytecode (63,185), stored in the pak: the same version 2 archives and LSLG records as Octopath Traveler 0's, but every shader is an LZ4 block (the first bytes are LZ4 tokens followed by LSLG; LZ4 is Unreal's default shader compression since 4.26), which uepak.py decodes itself (lz4_shaders: every compressed shader must decode to its raw size as an LSLG record, else the archive goes to the Oodle job as before). The NSP script runs no Oodle host for them.
  • 62,901 unique shaders (10,114 vertex, 50,756 pixel, 1,968 compute, 63 geometry) of 64,949 in the archives. The game has no PC version, so the facts table holds only compute group sizes; every vertex shader is made to link with any pixel shader (shader_split.py separable_pairs).

Audio and movies

  • Resolved: SIGSEGV at address 0x18 in UpsamplerManager::Allocate on the render thread. The audio renderer runs at 32 kHz (rate 32000, 160 samples per frame, GenerateSinkCommands sdk+0x225a80). A DeviceSink below 48 kHz is upsampled by the SDK's upsampler, and AudioRenderSystem::SetUpRenderResourceSet allocates the UpsamplerManager only for the DSP (sdk+0x21eb6c: rate 0x7d00 and rendering device +0x48 == 0); with the host's CPU rendering the manager pointer (AudioRenderSystem+0x150, the render context's +0x20) is null. The host answers Allocate on a null manager with no upsampler, so the generator skips the upsample command (sdk+0x225b40), and resamples the DeviceSink's stereo downmix 3:2 itself (a 16-tap windowed sinc in three phases, up_3_2 in hle_audren.c), 240 samples per 5 ms frame for the 48 kHz output.
  • Resolved: a movie in grey with vertical bands of colour, its luma sharp. CRIWARE's Sofdec hands its VP9 video to movie::Decoder in mode 1, NV12 into its own buffers, and draws the planes with its shaders; the bands were the top row's chroma, every chroma row read from the first one. Sofdec reads nv12-uv-stride from the output format, which the host never set. The host now sets everything the SDK's DecoderImpl::GetOutputBufferFormatFromDecoder (SDK 19 subsdk0+0x121f50) does: also nv12-uv-stride (the Y stride), nv12-width and nv12-height (md_format in hle_movie.c).

Services

  • Resolved: an abort in sm::Initialize under audio::GetAudioDeviceOutputVolume (the game thread, main+0xee8edc): the volume SetAudioDeviceOutputVolume sets is now kept and read back (hle_audren.c).
  • The import list against the hooks then gave the rest before they were met: ec::FinalizeForShopServiceAccessors, the shop response's Cancel and GetErrorCode, ec::ShowShopSubscriptionDetails, the Nintendo Account request's GetAuthorizationCode and GetState (both fail, as GetIdToken), and kpr::GetKeyCodeMap, which asks the settings service (settings::detail::GetKeyCodeMap) for the keyboard layout's map; the game zeroes a 0x1000-byte map and fetches it only once a keyboard reports keys (main+0x1f0eb4), and the host leaves it zero. The rest fill the game's own structures or sit behind a socket that fails offline.

Rewriter

  • Resolved: observation gate neither at a function entry nor after a bl for the gate on mov x0, x19 at main+0x32758e4, after a cut bl and an adrp. A site is now also accepted when the straight-line code from it to the next bl reads none of x9–x17 before writing it (scratch_dead). The gate table takes the original instruction as a word value (aa1303e0), the cut table as bytes (e00313aa).
  • main has two 23 MB stretches without unwind entries; they got through on a quieter Mac before functions were analysed in pieces, and the rewrite by the pieces rule is byte-identical to the previous patch_nso.py's. It is the only main of 120 modules of 33 games with other x18 sites before its first FDE (110 before +0xb310, none in a window or a br x18), and its rewrite stayed byte-identical when that text became one function.

Mac routes

  • The work folder's run/ctl.sh drives a run started with DM_CTL (wait S / snap NAME), and run/to_maribel.sh <run> goes from the title through a new adventure log into the opening cutscene's night scene by the boat (a save with an adventure log in the first slot): 05, Scripted input.

Found on this game

General host changes first needed here; each applies to every game:

Open

  • 60 fps on the iPhone: held for 3 minutes in an iPhone log of 2026-10-23 (GPU 13–15 ms mid-session, compile hitches up to 0.66 s). The same log had red lines for 7 point-light geometry shaders (missing shadows) and 1 texture type mismatch.
  • 62 of the 63 geometry shaders do not build, so point light shadows are not drawn.