Movie decoder (hle_movie.c, every game), both found on the opening movie (Movie/Cutscene01.mp4, H.264 and AAC through movie::Extractor):
- AAC decoders offer at most 32 outputs (MD_NOUT_AAC; Vorbis keeps 64, video 16). The game's movie player keeps the outputs it holds in a ring of 32 entries per track (main+0xe93d4c: 0x200 bytes) and pushes with no bounds check (main+0xe94140), so with 64 the 33rd AAC output wrote its index (0x3f) and time over the next heap block, and the game crashed in sead::ExpHeap's free a second into the movie, at the same address every boot. The console's AAC decoder cannot offer more than 32, or the game would overflow there too; Monster Train 2 peaked at 30 held.
- An AAC decoder's GetOutputBufferSize is one frame (1024 samples per channel, 2048 with SBR), what each output holds. The player sizes its own AudioOut buffers by it, decodes into them and appends each whole, advancing its audio clock by size / 4 samples (main+0xe92a74); with the 128 KB Vorbis bound every 4 KB AAC frame played as 0.68 s, the clock crawled and the movie froze on its first second. Vorbis keeps the bound (its packets vary in length).
Advanced blend modes (hle_nvn.c, nvn_obj.h, nvn_mtl.m, every game): the blend state keeps the mode, the premultiplied-source flag and the overlap (with their getters), and the pipeline blends the Porter-Duff modes (ZERO, SRC, DST, SRC/DST_OVER, _IN, _OUT, _ATOP, XOR, the PLUS modes) with the fixed-function factors they amount to under uncorrelated overlap (adv_blend); the others keep the state's functions with a red line naming the mode. The game's levels set SRC_OVER on one blend state, which the host used to ignore.
DM_PROBE_POLL takes a value filter (=<value>): every change is logged, the threads are snapshotted and dumped only at changes to that value. That is how the ring overflow was caught: DM_PROBE_WATCH on the heap link made the movie's file reads into the same page fail.
Profile: the provisional profile's guesses held (no system resource, a 3.14 GB heap in the 4 GB layout, modules main subsdk0 sdk), plus render_cond on (the game skips clears under render conditions; without it every level was black behind its HUD). 12,691 shaders (BNSH / BAHS in the zstd RomFS and main), all translated. Pad only (main imports no touch screen), the default 2D layout. A prewarm list recorded on the Mac from a fresh save over the route above (550 lines; with it nothing was built on demand). Checked: the NSP script packs it from the NSPs with the profile in dm/games; the graphics run list and the 49 tool tests pass; after rebasing on 6d34984 (spin waits) the game boots into level 1-1 as before. README (gallery, a Mac picture of level 1-1; 2D layout), AGENTS.md (57 -> 58) and docs/development.md, docs/agents 03 / 04 / 05 say so
On the Mac from update 13.0.5 (v2031616): the opening movie, the title, the first launch's DLC unlocks and Challenges board, the main menu, rulesets, stage and fighter select, 2-player battles (Frigate Orpheon, Big Blue, Great Bay, Find Mii, Prism Tower, Kongo Jungle 64) at 60 fps with about 2.7 ms of GPU per frame and 2 GB resident (1.2 GB of Metal allocations), quitting to the results and a second battle whose fighters' NROs the game loaded again; no red line in battle.
- Profile: modules main subsdk0 sdk (Catherine: Full Body's sdk and movie builds), the 5 GB guest layout, shader groups for data.arc's shader databases, effects and UI layouts, 35,012 shaders.
- patches.py: the engine's allocator (main+0x37c9a20, a static constructor) reserves 62 GB of address space with AllocateAddressRegion and aborted at main+0x37c9bd0. Booted once with a 64 GB alias region it commits from the region's start up (1.73 GB at the title, up to 2.5 GB in a battle) and keeps its page tables in the top 14 MB; all 11 instructions derived from the size are cut to a 4 GB region.
- sdk_patches.py: this SDK 8.2.1 sdk build (5fd4f531, Catherine's too) has the 1 GB AddressRegionManager node area and the 1 GB alignment of large regions (DRAGON QUEST XI S's SDK 7.3.2 layout at sdk+0x336b40 / +0x336d5c); both cut, so the region starts 64 MB into the 4.375 GB alias region.
- NVN: nvnSetExternalDebuggerEnabledNVX, fetched with nvnDeviceGetProcAddress(NULL, …) before the device and called unchecked (pc=0 from main+0x37fe92c), is a no-op in names.txt / hle_nvn.c, as docs/agents/03 already described.
- iOS NRO embedding (the part d979c34 left out): gen_embed.py packs the NROs nro.tsv lists into three sections (__TEXT,__dm_nro_x0, __DM_NRO_W,__img, __TEXT,__dm_nro_x1) with a table of their pieces (__DATA,__dm_nro_tab), since a Mach-O file has at most 255 sections and three per NRO would take 294; loader.c map_image maps an NRO from there (nro_pieces); build_ipa.sh puts nro.tsv beside the rewriter tables. The 98 NROs add 157 MB of code and 51 MB of data to the executable.
- shader_split.py reads data.arc through smash_arc.py (smash_arc_jobs / smash_arc_shaders: each current file scanned, labelled /data.arc/<format>#<hash40>); smash_arc.files(regions=True) gives every region's data of a regional file, without which 128 effect shaders were missing (34,884 of 35,012).
- Docs: README (games table, a Mac picture of Great Bay), AGENTS.md (50 → 51), docs/development.md (game notes, the NRO paragraph, nro_collect.py), docs/agents/01 (data.arc) and 03 (the 4 GB cut and its measurements, iOS NRO embedding).
Checked: the NSP script packs it from the NSPs alone, for the Mac and for iOS; tool tests (41), host unit tests and the NRO loader test pass. Played on an iPhone with this port's build from before it moved onto d979c34's NRO loader; the merged build was checked on the Mac (battles, an NRO loaded again) and builds for iOS.
On the Mac, the fresh-save route of the docs crashed at about frame 3390 and from about frame 5300 page-faulted or crashed on corrupted pointers, with the encoding thread on and off; issue #101 reports an iPhone 17 freezing on kIOGPUCommandBufferCallbackErrorPageFault.
- tf_draw opened its render encoder while the copy encoder (g_blit) was open: ubd_land's inline updates leave it open, as does a copy or a dispatch just before. Metal validation: "Command buffer already has an open command encoder"; the driver then crashed in -[AGXG18GFamilyComputeContext_mtlnext performEndEncoding] (or its dealloc on the encoding thread). Every other helper pass ends g_blit before it opens its own encoder; tf_draw now does too. A host built from before PR #62 crashed the same way.
- A block-linear image's atomics went to the 64 KB dummy buffer, but SPIRV-Cross indexes them by the image bound in the same slot (spvImage2DAtomicCoord: x + y × its width). Z-A's atomic max into a 480x270 r32ui 2D array (texture ID 302, fragment shader 2bd5037003980716) ran about 1.4 MB past the dummy: into zero-copy pools over the game's heap (pointers in the game's objects and the host's program objects had their upper 4 bytes replaced by packed values, rewritten every frame) and into unmapped addresses (the page fault, after which macOS ignores the process's command buffers). MTL_SHADER_VALIDATION=1, which stops out-of-bounds accesses, let the scene play. Such atomics now go to a scratch buffer at least as large as the image (atom_scratch: width × height × layers × texel size, at least the NVN storage size; 3 MB here), grown when a larger image needs it; the results are discarded as before. An unbound image keeps the 64 KB dummy, since its slot holds the 1x1 dummy image. The bring-up host (d34e044) with today's shaders page-faulted the same way.
Three headless Mac runs of the whole route (encoding thread on, off, on) reached frame 12,300 with no GPU error and no crash; before, every run crashed or page-faulted between frames 5230 and 5480. t_nvn 102/102, t_ub 47/47, t_geometry 95/95, t_vbuf 43/43. docs/agents/04-graphics.md and Z-A's notes in docs/development.md say so.
Integrate PRs #75, #90 and #91 with focused correctness fixes:
- Prioritize demanded render pipelines and bound speculative compilation.
- Follow copied jump-table bases and reject W/X register clobbers.
- Collect and rewrite registered NROs, load them on the Mac, and preserve
runtime activation order through unload/reload and symbol rebinding.
- Preserve refused-only manifests and invalidate collection caches when
archive readers or symbol parsing change.
Add pipeline scheduling, register tracing, NRO packaging and loader
lifecycle regression coverage. Document that executable NRO embedding
on iOS remains outside this integration.
Validation: 41 Python tests; 48 pipeline scheduling, 102 NVN and 95
geometry checks; three pipeline-key serialization cases; full host test
suite; Mac host build; iOS loader syntax checks; git diff --check.
Gameplay and controlled performance A/B measurements were not run.
Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>
Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Preserve per-player motor state and SDK vibration handles, apply batches atomically, and route Core Haptics output through existing controller assignments. Add independent launcher controls and lifecycle tests.
Supersedes #86. Physical feel and controller compatibility still require device testing.
Every GameController fed the same input path (front_mac.m), so the last one moved won, and hle_hid.c reported
exactly one npad whatever the game supports. Now the first controller is player 1, with the keyboard, the next
player 2 and so on up to 8 (pad_slots.h; a removed controller frees its player and gets it back when a
controller of the same name reconnects). Player 1 stays on No1 docked and Handheld in handheld mode, player n is
on No n (No1 stays empty in handheld mode: games take Handheld and No1 both for player 1). The game's supported
npad ids decide how many play: a one-player game ({No1, Handheld}) gets every controller as player 1, a
controller past a game's last player is not used, as on the Switch. A connect or disconnect signals that npad's
style-set event through hle_signal_bound_system_event (hle_thread.c), which signals only an event still bound,
from the macOS main thread where the notifications arrive; player LEDs follow the Switch's patterns; the
controller-support applet reports the connected players, capped at the game's maximum; only player 1 shakes.
DM_PAD_MERGE=1 merges every controller into player 1 as before; DM_KBD_PLAYER=own makes the keyboard player 1 on
its own. One controller, or the keyboard alone, behaves as before. iOS is unchanged: DmGamepad.m still feeds
player 1 and never reports further players.
Rebased onto a678703 (front_mac.m's includes, the Input row of docs/development.md). Checked: host unit tests
pass with the new t_hid (44 checks: slots and reconnects, npads docked and handheld, one- and four-player games,
events, LEDs, the applet); the Mac host builds for Breath of the Wild and Mario Kart 8 Deluxe with no new warning;
hle_hid.c and hle_thread.c pass an iOS SDK syntax check; Hades in a window with the keyboard and DM_KEYS reads
the supported ids (0x10f), and its scripted presses take it past the title. Two controllers were not run here;
the PR's author played Mario Kart 8 Deluxe in 2-player split screen with an Xbox Wireless Controller and a
DUALSHOCK 4, each its own kart, and a controller disconnected and reconnected during play got its slot back.
Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
A development option; the Mac is the development host, not a platform for players. The same steps up to the
shaders (extraction, DLC, rewrite, run-time captures and decodes, heap check, shader split), then only the
Mac's: dmsh_build.py without IOSOUT (an inherited one is dropped), a Vulkan or GL game's library without --ios,
the Mac host for the profile into <id>_build/tools/dmhost (build_mac_host, through DM_GAMES_DIR /
DM_GAMES_FIRST for a provisional or derived profile) and the button-icon font into <id>_build/font. No iOS
shaders, Xcode project or IPA; the extracted files are kept whatever KEEP_EXTRACTED says, and the last line is
the command that runs the game (its save in <id>_build/run/save, DM_NVN_SHMAP for an NVN game, DM_AOC_DIR
with DLC, the --env-file switches). It refuses --prepare-xcode and an Intel Mac.
Reuse: the Mac libraries have a key of their own (shader/mac-b-<chain>.key beside the cache folder,
shader/vk.key / gl.key), which an iOS build writes too, so a Mac-only build after an IPA reuses its shaders;
either key only while the Mac libraries carry a full build's stamp (inputs.tsv). An iOS build after a Mac-only
one translates again (dmsh_build.py's per-shader keys include the iOS compile convention) and takes the Mac
libraries from the Metal compile cache. build_mac_host now has a reuse key too (the compile line, the sources and
headers of the folders it compiles, the generated profile header, libvpx, the third-party headers, clang and the
SDK), which also saves the rebuild for the Oodle / LZ5 decodes and the run-time captures. build_vk_shaders and
build_gl_shaders share layer_shaders. The default (ios) path builds as before.
Checked: dm/tools/tests 34 of 34 (11 in the new test_nsp_mac.py); Hades in Mac mode ran the Mac stages only,
printed the run command and exited 0, and a second run reused every stage, the host included (2 s); the host it
built boots Hades from that command. Breath of the Wild (NVN, with DLC) in Mac mode restored its 19,592 shaders
from the cache, printed a command with DM_AOC_DIR and DM_NVN_SHMAP, and the host booted it from that command into
play. Not run end to end here: a GL, provisional or derived game in Mac mode, and an IPA after a Mac-only build
(unit tests with the tools stubbed).
Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
60 fps on a 144 Hz screen cannot be paced evenly: the display link counts a tick on the refresh nearest each 1/60 s
slot (link_range's virtual ticks), so frames stay up two and three refreshes in turn (13.9 / 20.8 ms) and Apple's HUD
shows the frame interval jumping between them. link_sync now tells the front end the link's rate (60, 30, or 120
with DM_PRESENT_LINK=2; front_screen_match), and in fullscreen front_mac.m switches a screen whose mode is not a
multiple of it to the mode of the same size with the highest rate that is, with kCGConfigureForAppOnly, so macOS
switches it back when the host quits; leaving fullscreen or that screen switches it back at once. A window keeps
the screen's rate and the virtual ticks: the switch changes the whole screen for every app and most external
screens go black for a second or more. DM_SCREEN_MATCH=0 keeps the rate in fullscreen too.
Changes to the PR: the switch only in fullscreen, with the way back on leaving it (the PR switched in a window
too); no switch without a window (the windowless host's main thread runs the guest); the rate to match is
atomic; the link takes a rate change only once its object exists, so one made while its thread starts is not lost,
and sets the range directly instead of logging a pacing mode -1; the screen this host switched takes its rate
from its mode, which maximumFramesPerSecond may not show yet. The link now also sets its range again when the
window moves to another screen, which kept the first screen's range before.
Checked on this Mac's built-in 120 Hz screen (a multiple of every link rate) with a harness that drives
front_mac.m asked for 50 fps: no switch in the window, 120 → 50 Hz in fullscreen, 120 Hz again on leaving it, and
120 Hz after quitting in fullscreen; at 60 fps and with DM_SCREEN_MATCH=0 nothing switched. Hades windowed and
fullscreen (DM_FULLSCREEN=1) paced at 60 as before, the unchanged host alike. nvn_mtl.m passes an iOS SDK
syntax check. The link's new range after a switch has not run here (it needs a 144 or 165 Hz screen); the PR's
author measured the windowed switch on a 144 Hz XB273U at 2560x1440: 120 Hz, the HUD's frame interval steady at
16.7 ms, 144 Hz again after quitting.
Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
Capture vertex outputs, run geometry shaders as compute, and rasterize compacted primitives through a general NVN pipeline. Rebuild the pinned translator and AI shader chain, add geometry regression coverage, and document supported behavior and remaining limits.
The launcher from v0lumehi's PR #62 (Launcher.swift, SwiftUI: the game's icon, name and version, Play, and the settings), built as their issue #64 proposes: dm/games/settings.tsv declares each player-facing setting once (the environment variable it sets, its kind: choice, range or toggle, its default and the condition under which a game offers it). build_ipa.sh runs dm/games/settings.py, which keeps the rows that hold for the game and fills in its defaults: the host's own from game_profile.h (DM_GAME_FPS_TARGET, DM_GAME_OPMODE), the build's (DM_LANG, DM_SHOW_FPS), the game's languages from its control data, and the render scale only for a build with the rewriter's render-scale stub, which patch_nso.py now records in main's table (RENDER_SCALE: Breath of the Wild 1.9.0, Tears of the Kingdom 1.4.3). The result goes into the app as dm/launcher.json, and the launcher draws one row per setting from it, so a new setting is one table row. Diagnostic switches stay environment-only. Breath of the Wild offers frame rate, render scale, console mode, 15 languages, the frame-rate counter and Apple's performance HUD.
Differences from the PR's launcher:
- The host's loading still starts at app launch. Only the loader read the frame rate and the render scale that early; it now just maps the frame step state page, and dm_fps_state_apply (main.c, when the game starts) writes both, so a boot is no slower with the launcher (the PR started all loading when Play was tapped). The game view, the pad and the frame-rate counter are made after Play, since they read the launcher's switches.
- The language choice applies: main.m sets the packed language as a default before the launcher runs, which the PR then took for a setting from env.txt and kept. main.m now lists the variables an env file set (DM_ENV_FILE_KEYS); only those are kept, and the launcher shows them greyed out.
- No new profile key for render scale (the rewriter's table tells), and the scale stops at 100 % (the above-1080p part is not in main). No Mac launcher.
The choices are kept in UserDefaults; DM_LAUNCHER=0 in Documents/env.txt skips the screen, as does an app without launcher.json. README: the settings screen in the Play section; dm/games/README.md and docs/development.md describe the table.
Checked: Launcher.swift type-checks against the iOS SDK and the Breath of the Wild IPA builds with it (launcher.json and RENDER_SCALE inside); the Mac host applies DM_FPS_TARGET=30 from the first present and DM_RENDER_SCALE=0.75 when the game starts. Not yet run on an iPhone.
Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
Taken from v0lumehi's PR #62, the parts that hold for every game (none is gated by a title, build or profile; all but the Mac front end reach only the NVN games):
- nvn_rec.h: the replay's Metal calls (encoders, draws, state, argument table writes, commits, event signals) go to a worker thread as records, in the same order. The submit thread waits for it at the end of each submission and before anything that waits for the GPU or uses another queue. On by default on iOS, DM_NVN_ENC_THREAD=1 on the Mac.
- Uniform shadows of 512 bytes or more (ubp_get) are copied by a worker; a commit waits for the copies queued before it, a CPU read of a shadow for its own (DM_NVN_UB_COPY_THREAD=0 copies on the submit thread as before).
- Deferred inline uniform updates are kept across render pass ends (DM_NVN_UB_LAZY, at most 160) and land before the next copy, a draw that reads their memory other than through uniform bindings the shadows cover, or the end of the submission, instead of in a copy encoder after every pass.
- A two-level address → pool cache (by 4 KB page, then the last 8 pools found), and the pass dump check answered first when it is off.
- DM_NVN_GPUPROF=f or f1-f2: every encoder's GPU start and duration, from Metal 4 counter heap timestamps.
- Mac: the drawable follows the window in pixels (in fullscreen it had stayed at 1280x720), the process declares latency-critical activity, and DM_METAL_HUD=1 shows Apple's performance HUD (on the iPhone too, from Documents/env.txt).
On top of the PR:
- A dispatch lands the kept updates first when it reads or writes their memory through a storage buffer or its indirect arguments, or reads a part of a uniform binding the shadows cannot give (ubd_conflict_cs, with the per-stage check it shares with draws, ubd_conflict_sh). Dispatches open the copy encoder through blit_open, which never landed them, so they ran before the updates. Transform feedback draws land them first, as every pass end did before.
- A deferred update that a newer one covers whole is dropped from the landing list. Breath of the Wild rewrites the same 64 bytes before every draw group, so the list held up to 160 repeats that every binding lookup scanned: the port cost 16,800 instructions a draw against main's 13,200, and costs 13,500 with the drop.
- The encoding thread records encoder labels (DM_NVN_ENC_LABEL) and counter heap timestamps (DM_NVN_GPUPROF): without them either diagnostic stopped the host on iOS.
- DM_NVN_UBP_CHECK waits for the copy worker before it reads a shadow. It compared unfilled shadows (293,447 "different"); now 232,000 equal and 0 different with the encoding thread on, 234,000 and 0 with it off.
Left out: Breath of the Wild above 60 fps and above 1080p (rewriter cuts for its build and a profile key), the Mac GPU filler and screen mode switch, and the iOS launcher.
Breath of the Wild on the Mac (the Great Plateau save, 60 fps cap, beside main under the same load): submit thread 4.30 against 4.48 ms a frame, GPU p95 6.2 against 7.0-7.3 ms, 1.000 of real time, no red lines; with the encoding thread the submit thread takes 3.16 ms (9,700 instructions a draw). t_nvn 100/100 with the encoding thread off and on; the hosts for TOTK and Hades build.
Co-authored-by: v0lumehi <34629096+v0lumehi@users.noreply.github.com>
- Files and shaders: Next Level Games' engine keeps every RomFS file but the Wwise banks in archives of its own, <name>.dict indexing zlib chunks in <name>.data; update 1.4.0 ships <name>.patch beside the unchanged base files, a variant bitmask and per-section records that rebuild each section from base and patch bytes (the game's algorithm, main+0x1c8d0 / 0x1cbd0). shader_split.py nlg_shaders reads both: 5,317 shaders (5,149 in the base game), all in init, control sections back to back in one section and their code areas in the same order in another, one NVN program per stage (SINGLE_STAGE '#NLG'); 46 of the first 103 the title uses exist only in the update (docs/agents 01)
- Shader chain, for every game: a texture array over the driver table c2 (Ryujinx's texture array from a buffer, fp_t2das_cb2_0: 586 shadow-cascade pixel shaders indexed by (0x80 + cascade offset) / 8) is named tex_c2a / samp_c2a with its c2 offset (dmsh_build.py c2_array), and the replay fills element k from the slot whose c2 word lies at that offset + 8k (nvn_refl NVN_RB_TEXA / NVN_RB_SAMPA, nvn_mtl.m c2_handle). A pixel shader that writes an output from integer bits, or nothing but constants, gets a second entry point main0_u whose colour outputs are the unsigned vectors of the same bits (dmsh_build.py fs_uint); a pipeline whose written colour attachments are all unsigned integer formats takes it (pso_all_uint). The G-buffer is R32Uint + RGBA8Uint: about 1,400 pipelines failed to build and every scene draw was skipped. Every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (the change touches no c1 or uniform read); Luigi's Mansion 3 built for Mac and iOS
- Host, for every game: the game patches texture handles into its recorded command buffers in place, at the command start + 0x10 it took from GetCommandMemoryUsed before nvnCommandBufferBindTexture (main+0x7c8964; the store at main+0xb7d0d4), so a single texture bind is recorded in the hardware's layout, {stage, slot, handle at +0x10, texture / sampler pool bases} (OP_BIND_TEX1, nvn_obj.h bind_rec decodes it at every replay site); the host's own layout put the patched handle over the bind count, the post passes read stale handles (a stretched icon over the title, a white screen in a new game) and a corrupted record now gets a red line. t_nvn: the handle patched at the command start + 0x10 is the one replayed (102 checks)
- Host, for every game: a depth texture drawn through a colour view of the same pixel size (a D16 1024x1024 texture through an R16 view; Metal has no colour view of a depth texture, and the boot aborted in pipeline validation) draws into a colour shadow, copied in before and back after the pass as a compressed target is (nvn_mtl.m shrt_block, D16 and D32F); nvnCommandBufferSetDebugLabel; nvnBlendStateSetAdvancedMode (red line for a mode other than none); a draw whose program has a geometry shader with no library gets a red line naming it (it is drawn without that stage, with the vertex shader's positions as clip coordinates)
- Frame rate: 60 fps (30 on the Switch). The frame limiter (main+0x7a3e70, a frame callback called with the milliseconds since the frame began, main+0xb7c308) runs jobs or sleeps until 33.333 ms have passed, a float in read-only data (main+0x13c94dc), and the game steps by the time that really passed: the same 2 s walk and 1.5 s turn from the same save end at the same spot at 30 and 60 fps, and the title's animation matches at time scale 1.00 (DM_NVN_REGLUM). hle_fpslock.c gets rows with no pointer (the field at main + offset; the loader leaves read-only data writable, the image's data block on iOS too) and a frame-time kind: 1000 / fps_target, at 60 less 0.5 ms, since the work after the limiter made an exact 16.667 ms 59.6 fps with a frame shown twice every 2.5 s; slightly shorter, the presents set the pace (lobby 60.0 fps, display interval p99 18.69 → 16.67 ms). Profile fps_target 60 with present_interval display (the game also asks for interval 2); DM_FPS_TARGET=30 keeps the game's 33.333 ms
- Not done: the tessellation stages (15 hull, 18 domain shaders: two programs draw twice a frame for about 30 frames, in the title scene behind the file select as it slides out before a new game; skipped, since Metal 4 has no tessellation stage) and 10 geometry shaders (none drawn from the title to the lobby). Vertex shaders that feed them can be found from the shader headers (docs/agents 04)
- Mac: the boot notices, the title, a new game, the opening cutscene (the bus ride, the arrival at the hotel) and the lobby in play at 60 fps, about 3.7 GB phys_footprint (3.0 GB of Metal allocations); red lines at run time: the geometry shader that does not build, the conditional render the host draws anyway, primitive restart 0xff (no 8-bit strip seen). Diagnostics: DM_NVN_TREG_ID=n logs that texture ID's registrations; the pass dump prints the handles of texture binds
- The user confirmed the build on the iPhone
- README: games table row and picture (the first battle: the Archwitch's attack on the soldiers); AGENTS.md: the list (forty-two games) and a note on the port; development.md: status
- Engine: Unreal Engine 5.6.1 (Square Enix, the desktop renderer), the newest Unreal so far; everything in one IoStore container beside one pak; the sdk and subsdk0 builds are STEINS;GATE RE:BOOT's
- Rewriter, for every module: code before the first unwind entry is one function (each site there was a function of one instruction, so a byte jump table at main+0x219234 went unseen), and a function longer than 256 KB is analysed in pieces cut after a ret / br / b that no conditional branch or b within 64 KB crosses, with far branches and jump table targets checked against windows and retry blocks afterwards (this main's 82 MB without unwind entries took 100 GB of memory as one function; now 2.7 GB resident). Checked: DRAGON QUEST VII Reimagined's main byte-identical to its rewrite by the previous patch_nso.py; FINAL FANTASY RESONANCE's, Octopath Traveler 0's and DRAGON QUEST XI S's byte-identical between two different cut placements; only DQ VII's main among the 120 modules of 33 games on hand has other sites before its first unwind entry (none in a window or jump table)
- Shaders: no archive in the pak; the two libraries are IoStore shader library chunks whose tables and shader groups are Oodle-compressed, read through the game's own Oodle in the Mac host's tool mode with no boot (uepak.py Toc / io_archive / OodleHost; library names from CityHash64 of the manifest's archive names) and written as decoded version 2 archives; the NSP script uses them for any game with IoStore shader libraries. The pak's single-block files store block size 0 (Pak.entry). 73,496 unique shaders; 11 do not build (3 geometry, 8 compute with 3D image atomics), none met so far. 2,077 particle vertex shaders pick their texture slot at run time (tex_dyn_slots); CopyStencilToLightingChannelsPS writes integers to an R8Uint target (shader_facts_manual.tsv; without it the sprites were dark silhouettes)
- Host, for every game: a shader with run-time texture handles in a game without tex_dyn_slots freed its reflection twice and aborted the host (nvn_mtl.m); oe::MuteRecording / UnmuteRecording (no-ops)
- Memory: the FMallocBinned3 constructor call is cut and the host's allocator runs Unreal's heap (hle_uemalloc.c layout UE5_6: UE 5.5+ FMalloc has 32 slots, with MallocZeroed and two more TLS cache calls); the SDK's node area is cut to 62 MB. The engine's own page allocator only grows its use of the engine region (2.03 GB of address space over seven minutes, about 1 GB mapped), and a 2 GB region aborted in the first battle, so the region is 4064 MB in Engage's 5 GB block (this sdk build aligns regions to 2 MB, no alignment cut needed)
- Frame rate: 60 fps (30 on the Switch): the host writes rhi.SyncInterval 1 (hle_fpslock.c rows for this build; profile fps_target 60); the game steps by real time (DM_NVN_REGLUM traces at 30 and 60 fps match from the load through a walk)
- Mac: first-boot settings, the title, a new game, the opening cutscene, the first field and its autosave, Continue, and the first two battles at 59.7 fps, 2.3-2.6 GB resident, flat over seven minutes, no red line apart from a few texture type mismatches at one moment of the new game's load in some runs. Docs: docs/agents 01, 03, 04 and development.md
The host writes Unreal's t.MaxFPS 60 and rhi.SyncInterval 1 into the game's console variables (hle_fpslock.c rows for main f7ec03e4); these are the two limits StevensND's 60 fps mod for the base main lifts with code patches (UEngine::SetMaxFPS returning at once, the name r.VSync broken). Unreal steps the game by real time: on the Mac 59.8 fps, the same route at 30 and 60 matches at time scale 1.00, same resolution, no new red line.
stb_vorbis cut a type-2 residue's interleaved vector at n * 2 values even when its submap holds more channels, so the rest of the packet was read as the wrong data: the 5.1 tracks libvorbis encodes came out distorted and gurgling, with pops. The bound is now n * ch, as in the Vorbis spec and libvorbis's res2_inverse, and the temporary-memory estimate for a caller-owned buffer counts every channel of the stream for type-2 residues (with the decode fix alone, a 6-channel stream overran the advertised buffer). Upstream stb still has the bug (nothings/stb #879, #1594).
Link's Awakening's opening and title movies were the report; DRAGON QUEST XI S's movies open only 6-channel Vorbis tracks too. Its 23 6-channel cutscene and title tracks, decoded as the host does, now match FFmpeg to within 0.02 PCM16 units, where main's decoder was off by an RMS of 1,700 to 70,000 units.
dm/host/tests/t_vorbis builds tiny Vorbis streams in memory (types 0/1/2, mono to 8 channels, two submaps) and checks heap and fixed-buffer decoding through the pull and push APIs, with and without STB_VORBIS_DIVIDES_IN_RESIDUE; it needs no game profile. dmhost rebuilds when stb_vorbis.c changes.
On top of the PR: the test rules sit with the others, and the stb_vorbis README and the per-game notes say what the fix repairs.
Co-authored-by: ohxiaoguang <44696970+ohxiaoguang@users.noreply.github.com>
Each game gets its own project in <NSP folder>/<title ID>_build/xcode, registered in a shared NxNative.xcworkspace; a target phase copies the RomFS, shader libraries and add-on content into the app before Xcode signs it, and preparing again keeps the bundle ID and team chosen in Xcode. The unsigned IPA build is unchanged.
On top of the PR: the resource phase stops when the profile's shader chain is no longer the prepared shaders' (the host loads only its own chain), prepared projects ask for the Increased Memory Limit capability (DM_ENTITLEMENTS overrides), the source links are made again on every preparation, and scheme names keep non-ASCII letters. The PR's TOML launcher is left out.
Co-authored-by: Animax Deng <Animax.deng@gmail.com>
- The user gave the go-ahead after the iPhone build
- README: games table row and picture (Maribel and the hero by the boat at night, the scene the image-handle size query fix brought back); AGENTS.md: the list (forty-one games) and a note on the port; development.md: status
- Files and shaders: Unreal Engine 4.27 with IoStore containers beside one pak (version 11, its footer magic 0xcc bytes from the end, uepak.py magic_back); the 62,901 shaders of the pak's two shader code archives are LZ4 blocks, which uepak.py decodes itself (lz4_shaders), so the NSP script runs no Oodle host when no record is Oodle. One global shader writes integers into an R8Uint target (shader_facts_manual.tsv)
- Shader chain, for every game: a texture query on an image handle (c2 0x120 + 8 r: Unreal's GetDimensions on the RWTexture3D a compute shader writes) had no type in a facts table built without PC shaders, and the 2D default took a 3D image's depth from its width; the translator now translates such a shader again with the image types of its first translation (dmsh_xlate ImageTypesOf; dmsh_xlate.dll rebuilt, dmsh_chain.lock updated). Host, for every game: a texture binding of slot 32-39 reads the image bound in the image slot that shares its c2 word (nvn_mtl.m tex_slot_h; Hades II's texture slot 34 without an image still reads the texture). Unreal's volumetric lightmap indirection patch (7b958a7b21e72801) read the empty texture slot 32's 1x1 size and wrote only cell (0, 0, 0), so in the opening cutscene every character and prop sampled the atlas origin, a daytime level's bricks baked 2^15 brighter (white figures about 10^4 too bright). Monster Hunter Rise has four such queries. Every other profile's dmsh_chain.h rewritten with its c1_full / cb_extent lists carried over (the change touches no c1 or uniform read); DRAGON QUEST VII Reimagined rebuilt for Mac and iOS
- Host, for every game: stencil-mode texture builders (nvnTextureBuilderSetDepthStencilMode STENCIL) read the stencil of their depth-stencil storage (the hero was drawn as a dithered silhouette); the audio renderer at 32 kHz on the CPU path (the SDK makes its upsampler only for DSP rendering: UpsamplerManager::Allocate returns null and the host resamples 3:2); GetAudioDeviceOutputVolume; the shop and Nintendo Account calls of the title (ec Finalize / Cancel / GetErrorCode / ShowShopSubscriptionDetails, GetAuthorizationCode / GetState); kpr::GetKeyCodeMap; the movie output format's nv12-width / nv12-height beside nv12-uv-stride (Sofdec's VP9 movies were grey with bands)
- Memory: MallocBinned3 reserves 37 GB of pools in the engine region; its creation is cut and an observation gate hands the engine the host's malloc zone (patch_nso.py cut + gate OBS_UE_MALLOC, hle_uemalloc.c layout UE4_27; patch_nso.py places a gate where the scratch registers it uses are dead, scratch_dead); the region is 2 GB (game_args) with the SDK's node-area cut on this sdk build, in a 3.75 GB guest block; about 2.6 GB resident on the Mac
- Frame rate: this build's t.MaxFPS and rhi.SyncInterval in hle_fpslock.c; the profile keeps the game's 30 fps
- Diagnostics: DM_CTL (Mac) drives a running game line by line: presses, shots, pass and texture dumps, texlist (textures of a target with storage and block-linear layout), memdump, peek, Unreal console variables (cvarf / cvari), env; the pass dump prints buffer-to-texture copies and buffer textures' format, size and storage, and DM_NVN_CBDUMP saves the source bytes of 3D uploads (docs/agents 04, 05)
- Mac: the logos and opening movie, the title, a new adventure log, the opening cutscene (the well, the night scene with Maribel by the boat) into play at Pilchard Bay, and a saved log continued, at 30 fps with saves; no red line at run time apart from 62 of the 63 geometry shaders (one-pass cube shadow maps), which do not build. Unit tests: the graphics tests that build pass (t_refl's two sample libraries are not on this machine); 13 that compile nvn_mtl.m with hle_nvn.c already failed to build on main (g_ft defined in both, hle_nvn.c symbols missing from their stubs)
- The user confirmed the build on the iPhone
- README: games table row and picture (the arrival on the island's dock); AGENTS.md: the list (forty games) and a note on the port; development.md: status
- Nintendo EPD's engine (code name Park; sead / agl / gsys, NintendoWare, Havok, Pia and NEX) on NVN; 9,766 BNSH shaders in zstd and Yaz0 SARC files and main, the in-flight movie VP9 + Vorbis through movie::Extractor, the Happy Home Paradise add-on mounted. A new game runs through Tom Nook's counter, the movie and the island's creation to the player on the dock at 30 fps; not yet seen: a save, which the game first makes at the end of day 1
- Yield back-off, for every game: a yield whose caller's x19–x28 moved since the last one is progress (the SDK's YieldThread reaches the SVC through tail branches, so they are the loop's own): it returns at once with no host call and the run starts again; polls keep those registers and back off as before. The game's SaveProc builds a new island in grid walks that yield once per cell: 77 s of black screen after the movie, now about 10 s. Breath of the Wild's PlacementMgr and RadarMgr yield per batch item too: outdoors on the Mac CPU energy 0.35–0.74 → 0.21–0.34 W, CPU 76–83% → 96–106% of a core, frame interval p99 19 → 20–21 ms
- Rewriter, for every game: halfword jump tables get landing pads up to their largest entry (half_count), not all 65,536: this main's patch area went from 42 MB, out of a branch's reach, to 7.9 MB; the other games' modules change only in such tables
- Host: main's own HIPC client for olsc:u (cloud backup settings) is answered by host sessions (hle_sf.c); device save data is the host save directory, and a second mount name over one save shares its transaction; Commit with a CommitOption, GetSaveDataRestoreFlag; the adjustable user clock, GetStandardSteadyClockCurrentTimePoint, oe::GetRegionCode, SDK 20's album share calls; video movie decoders keep 16 output slots (the game's player holds every one it is offered)
- Profile: render_cond on (the game skips clears with conditional rendering; without it every frame was black), pad_layout full, mem_used 122 MB, the prewarm list recorded from boot to the dock, the chain header and tables for the current chain version
The game copies its precomputed sky scattering tables (System/KSys/sky.skybin) into CPU-cached pools and flushes them; the host made the 256x32x16 3D table a GPU texture of zeros, so the sky came out brownish grey. nvnMemoryPoolFlushMappedRange now records the flushed ranges (nvn_flushed in hle_nvn.c), and tex_init_fill fills a new GPU texture from such storage once, as it did for CPU_NO_ACCESS pools. DM_NVN_FLUSH_FILL=0 turns it off; t_nvn covers the range bookkeeping.
- The user confirmed the build on the iPhone
- README: games table row and picture (Vincent climbing the first Nightmare stage); AGENTS.md: the list (thirty-eight games) and a note on the port; development.md: status
seedds