Cuphead (Unity 2017.4) main+0x1bb872c loads the table entry into x17 and only
the add writes x18: cmp w17,#7; ldrsw x17,[x15,x17,lsl #2]; add x18,x17,x15;
br x18. jump_offsets now takes the loaded register from the add; the walk
from the index back to the bounds check is unchanged.
Ported from GitHub pull request #1 onto the current jump_offsets (the PR's
other two shapes already pass since Link's Awakening's subsdk work). All 35
rewritable modules of the 9 extracted titles give byte-identical output.
Co-authored-by: Jeffrey Wang <11900676+armyja@users.noreply.github.com>
- Movies: H.264 decoded with VideoToolbox (dm/host/hle_movie_avc.c). This SDK's MP4 demuxer passes the parameter sets as the "avcc" record; pictures leave in presentation order through a 4-deep queue (41 of Odyssey's 44 movies have B-frames), and an end of stream drains the queue. The hint videos in Odyssey's tutorial boxes now play (they were black). VP9 unchanged (Engage: 2,816 packets in, 2,816 frames out).
- Controller input: GetNpadStates fills every requested history entry with the current state under consecutive sampling numbers. Odyssey reads all 16 entries whatever count comes back and polls every slot each frame; with only the newest written, neither stick did anything in play while buttons and menus worked (its first hint, "Look Around", waited forever). DM_KEYS takes RLeft / RUp / RRight / RDown.
- Odyssey's HUD counters were solid white cards: the profile lists its 26 layout shaders (main's BNSH at 0x132c000) under alpha_from_red, as for Breath of the Wild.
- -fpreserve-invariance on every Metal compile (dmsh_build.py MAC_METAL, ios_metal.py): Odyssey draws a depth pre-pass with alRenderDepthOnly, then the G-buffer with other vertex shaders and depth test less-or-equal; Metal gave the identical position math slightly different depths and those pixels kept stale content (Cap Kingdom's ground in 32x32 blocks on every fourth frame). Checked by recompiling the frame's 78 vertex shaders with the flag: four consecutive frames identical, no blocks. All eight chain versions change; Mac libraries and c1_full.tsv are being rebuilt and follow.
- DM_NVN_TEXSNAP works again (its entries lacked the layer field the dump loop reads) and names its files <path>_s<pass index>_<serial>_<level>.png.
- Smoke runs of the seven playable games on the Mac after these changes: only their known red lines.
The game encodes one 32x32 BC4 texture on the GPU, drawing its blocks through an RGBA16UI view. On a GPU with BC that attachment was already left out (Metal cannot attach a compressed texture or make an uncompressed view of one). Without BC the texture is stored decompressed (R8), the check on the Metal format missed it, and the host asked Metal for an R8Unorm -> RGBA16Uint view, which aborts the process (iPhone 14 Pro / A16; DM_TEST_NO_BC=1 reproduces it on the Mac).
attach, the pipeline prediction and the sampling views now decide by the NVN format: a BC texture is never a render target and its raw blocks are not sampled through an uncompressed view, on any GPU (red line once each). Checked on the Mac with DM_TEST_NO_BC=1: Breath of the Wild runs with the same two red lines as with BC; smoke runs of the other games unchanged.
Comments and dm/gfx/tex/README.md no longer point at d2r-mobile's offline ASTC pack (dm/tools/astc_pack.py), which is not in this repository; the memory cost of no BC (about 3.9 GB instead of 2.8 GB in Breath of the Wild) is noted in 04-graphics.md.
- dmsh_build.py vs_init: every float4 vertex output is set to (0, 0, 0, 1) before the translated body, as unwritten components read on the hardware. The translator's MSL started from all zeros; Link's Awakening's surface shaders divide a world position by its w, which the vertex shader never writes, so every sun-shadow compare value was infinite and the whole outdoor scene was lit as if in shadow (dim, no shadows from fences, house or trees). All seven chain versions change.
- Mac libraries and c1_full.tsv rebuilt for Link's Awakening, Engage, 13 Sentinels and Mario Wonder (Breath of the Wild, Super Mario 3D World and Metroid Dread follow).
- DM_NVN_TEXRAW dumps the depth plane of a Depth32Float_Stencil8 texture (shadow maps).
- The IPA's version (CFBundleShortVersionString / CFBundleVersion) is the packed game's display version: the update's when an update is packed, else the base game's (nacp.py --version, DM_APP_VERSION in build_ipa.sh, project.yml; the NSP script prints it).
- README: Link's Awakening screenshot outdoors in Mabe Village.
- NVN's depth mode is runtime state: Mario, Metroid Dread, Super Mario 3D
World and Breath of the Wild keep near = -w (GL); Link's Awakening,
13 Sentinels and Engage set near = 0. The translator rewrote z for GL
in every vertex shader, so near = 0 games had all depths in 0.5..1 and
depth-to-distance effects were wrong: Link's Awakening's depth of field
blurred the whole screen outdoors.
- The translator no longer rewrites z; dmsh_build.py (vs_depth) writes two
entry points into every vertex library, main0 with the GL rewrite (the
same arithmetic as before) and main0_z without; the host picks one from
the mode the game sets (mtl_set_depth_zero, shader_load). No per-game
setting; a red line if the mode changes after vertex shaders loaded.
Metal function constants were measured at about 2.4 ms of
specialization per vertex shader, so two entry points instead.
- New translator dll (Ryujinx dlls unchanged), lock and chain versions for
every profile; Mac libraries and c1_full.tsv rebuilt for Link's
Awakening, Engage, 13 Sentinels and Mario Wonder (Breath of the Wild,
Super Mario 3D World and Metroid Dread follow).
- Checked on the Mac: Link's Awakening outdoors (from a save in Mabe
Village) has a sharp middle band again; Mario Wonder's title renders
the same as before (differences only in animation).
- README gallery: Engage, 13 Sentinels and Link's Awakening recaptured
with this build.
- Link's Awakening 1.1.0 main: the per-world PhysX completion task destroys
itself in its release (vtable back to the base, mTm = 0). At 30 fps the
game steps physics twice per frame; a worker descheduled between the run's
unlock and the release let the main thread rebuild the task first, and
the late destructor wiped the new one, so the next completion crashed in
removeReference (main+0x10ccf28). A per-build cut makes that release a
ret. Unfixed, every Mac run with the CPU loaded crashed; fixed, runs with
138-257 switches to 30 fps and walks into and out of the house are clean.
- Crash reports print what each register points at ([dm] *xN), so an
object's vtable names its class.
- RED_ONCE's flag array was one slot short.
- README gallery: Super Mario Bros. Wonder, Super Mario 3D World, Breath of
the Wild and Metroid Dread recaptured with the current build; no "(Mac)"
captions.
- docs: the crash in the bring-up catalogue; debugging recipe for racy
crashes (CPU load, trace gates, a ring printed at the crash).
- README: a gallery with one picture per game (docs/images): iPhone shots
with the virtual gamepad for Mario Wonder, 3D World, BotW and Link's
Awakening; Mac shots (captioned) for Engage, Metroid Dread and 13
Sentinels. Link's Awakening is now listed as in testing on an iPhone.
- scripts/build_ipa_from_nsp.py: a game with no profile is surveyed, then
gets a provisional profile guessed from the survey in
<NSP folder>/<id>_build/profile/<id>/ (outside the repo, reused on the
next run) and a best-effort IPA; --survey-only keeps the old behaviour.
game.py searches DM_GAMES_DIR after dm/games; the host Makefile reaches
such a profile through a symlink; build_ipa.sh finds it through game.py.
zip_store also stores files whose headers say they are compressed.
- copy_t2t_draw: depth-stencil copies now carry the stencil ([[stencil]]
export from an X32_Stencil8 view). Link's Awakening's occlusion pass
tests stencil bit 1 of its quarter-size depth copy, whose stencil was
the new texture's undefined content.
- Link's Awakening: four outdoor shaders with the foliage shader's input
loop failed on the iPhone (Half2 / normalized formats); listed in
shader_facts_manual.tsv. Chain c4995bb3161c.
- Sound: SDK 8 numbers the DSP commands differently and its DeviceSink is
0x10, not 0x13, so every rendered frame played silence (only the movies
had sound). capture() now takes the one 0x148-byte command with 1-6
inputs as the sink when the expected ID is missing.
- Lighting: the light-culling compute shader reads its workgroup count
from NVN's driver segment c0 (+0x200). dmsh_build's drv_reads scan only
matches fragment-shader names, so compute and vertex shaders got an
all-zero c0: every tile row wrote row 0, and the doorway step flickered
with white 64-pixel tiles. A declared driver-segment binding now counts
as read, and compute c0 carries the dispatch size (0x200) and the
uniform buffer table (0x210 + 16 r). BotW and SM3DW render as before.
- Foliage: the shadow / depth pre-pass pipeline 147c829e79fb322f failed
(uint4 inputs fed 16-bit normalized formats), so with depth test EQUAL
the palm tree and flowers never drew. New profile file
shader_facts_manual.tsv (merged by shader_split.py) declares those
inputs float; Link's Awakening's chain is now ccb18ba743bf.
The prompt mask is a layout capture (panes tagged CaptureOn / A8): an R8
render target mapped (R, 0, 0, 1) holding the blurred alpha in red. Besides
the black shadow (e6a88e12), each prompt bar draws a cream glow (aa209f57)
whose alpha is min(cloud alpha, capture alpha); with only the shadow shader
listed, the glow filled each key's rectangle with the Kumo64 cloud.
alpha_from_red now lists every pixel-shader variant of the layout library
(main's BNSH at 0x1d83000) and tex_alpha_red only converts render targets.
Game work folders now live in <NSP folder>/<id>-work instead of the home
folder: AGENTS.md and the docs say so, and build_ipa.sh requires W and
writes the IPA next to the work folder by default.
BotW: the button-prompt shadow samples an R8 blur whose swizzle (R,0,0,1)
made alpha 1, so every prompt got a solid box. New profile key
alpha_from_red takes alpha from red for the named shader.
Link's Awakening:
- NVN lower-left window origin (negative viewport height, winding flip,
flipped present and raw screenshots)
- texture copies with format change, scaling, mirroring and depth
(copy_t2t_draw); SetCopyImageStride; nvnWindowGetPresentInterval
- oe::PopNotificationMessage, vi SetLayerCrop / DestroyLayer /
CloseDisplay / Finalize, audren P_SIZE 0x40 for SDK 8, hid style-set
update event signalled once on bind
- Vorbis movie audio through stb_vorbis (dm/third_party/stb_vorbis)
- patch_nso jump tables: other index registers, mov copies, masked copies
- Magenta 16x16 blocks in Super Mario 3D World 1-1 (iPhone only): the composite shader 66a71a3081b91b9a
renders into the RGB10A2 image while sampling it in the same pass. That is undefined in Metal: the
M4 Pro reads the old content, the A20 Pro sometimes returns whole 16x16 blocks as (1,0,1). Found with
the new device self-check DM_NVN_MAGSCAN (blocks first appear on that attachment, and still appear
with the suspected particle dropped). fb_prepare ends the pass, copies the attachment level into a
snapshot and binds the snapshot for that draw, only for shaders in the new profile key feedback_snap
(doing it for every shader cost BotW 16.6 -> 36.6 ms). DM_NVN_FB_SNAP=0 / all / <keys> overrides.
- The game language now defaults to English (nn::oe::GetDesiredLanguage returns en-US; DM_LANG=zh-Hans
in Documents/env.txt for Chinese).
- Code comments, docstrings and docs are English only; the zh-CN doc twins are removed. README.md has the
English text followed by the same text in Chinese. Files the shader chain version hashes keep their
bilingual comments (editing them changes every game's chain version).
- Profile 01007EF00011E000 (SDK 20.5.22, NVN 55, main + subsdk0 + sdk): 19,592 shaders, all translated (chain bd3979ea9b3d).
- Loader applies DT_RELR relative relocations (SDK 20; main has 223,994); nvn_slots_got.py and DM_LOADHASH read them too.
- 60 fps: five rewriter cuts lower the GameFrameworkNx dynamic-interval floor from 2 to 1; ksys::VFR scales the
logic time step by the interval (ratio 0.5 at 60 fps, falls back to 30 with ratio 1 when frames run long).
- Black 3D scene: nvnCommandBufferBindTexturesDeferred reads the handles at execution (bh_fresh); the engine
records once and fills the handles later, so the lighting composite sampled texture 0.
- A texture attached at two colour slots is attached once (TBDR tile stores raced, albedo kept scattered tiles).
- Cube textures are 6-layer 2D arrays so tex_fit can bind cube-array views (1,512 shaders sample cube arrays).
- NVN functions other than command recording run on the per-thread host stack (Metal compiled on a small guest stack).
- Address-to-pool lookups use the last hit plus a sorted index (1,760 pools): submit time 21 ms -> 7 ms per frame.
- SDK 20: ARUID hook, audio renderer layout (Span GenerateCommand, new offsets, DeviceSink ID 0x14), nfp events,
VR mode, seven-six-axis sensor, steady clock, launch parameter, stereo vision permission.
- Compressed render targets (BC4 via an RGBA16UI view) are skipped with a red line instead of a Metal abort.
- DM_KEYS takes frame numbers (f2500:A); TEXDUMP saves 2D-array layers; PASSDUMP prints skipped passes.
- NSP script skips empty NSP files and keeps one copy per DLC.
- Profile 010028600EBDA000 (SDK 10.4.0, NVN API 53): 11,080 shaders from Yaz0 SARCs (BNSH, BAHS v9, effects, main).
- Shader key hashes min(0x880, control +0x20) bytes (0x850 / 0x848 controls); other games' tables unchanged.
- Profile keys device_api (NVN API version the game checks) and shared_font (ttf / bfttf at the shared-font address).
- File system hooks run on a per-thread host stack (HLE_FORCE_BIG, hle_bigstack.c): the save thread has 8 KB.
- Washed-out scenes: a one-layer 2D array sampled as texture2d read one value on Metal; tex_fit binds 2D / array / cube views.
- Unwritten colour outputs get a zero write mask; NVN 1D textures are one-row 2D.
- Hooks: InlineKeyboard, settings LanguageCode, ListAllUsers, oe perf / album, nifm SDK 10, prepo session, vi indirect image size;
nvnQueueAcquireTexture, SamplerCompare, debug groups, TextureGetSwizzle / GetStorageClass.
- NSP script builds nx_extract in a temp folder (an apostrophe in the path broke MSBuild silently); DM_KEYS left stick.
- New profile 01003FC01670C000: sdk and subsdk0 are byte-identical to Metroid Dread's; data is six CRI CPKs; the 751 shaders are GLSLC outputs compiled into main (shader_split.py: shader_main_glslc), built as single-stage programs and paired per draw (shader_separable: every vertex shader is paired with a cover of the pixel shader inputs, so vs_pad lets any pair link)
- Host hooks for SDK 11 calls that reach a service: hid::GetMouseState, fs::CleanDirectoryRecursively, fs::EnableGlobalFileDataCache, oe::SetUserInactivityDetectionTimeExtendedUnsafe, time::ToPosixTime, audio::GetAudioOutName
- StandardAllocator::Allocate(size): on this build only, blocks of 64 KB or more are 64 KB aligned; the engine cuts three ring heaps out of a GPU pool at 64 KB alignment relative to its GPU address, and with the host's 4 KB-aligned address the third heap kept base 0 (null write when the prologue enters its first battle)
- NVN: triangle fans and indexed quads, vertex streams 14 and 15 on Metal slots 14 / 15 (Metal assertion buffer index 31), TextureBuilderSetPackagedTextureData copied into the storage at TextureInitialize, InsertDebugMarker as a no-op
- Mario 1.2.1 sets present interval 2 for single frames after a slow one and advances its time step by the interval; frames were shown for one tick and the game jumped ahead on the iPhone. The display link now keeps a frame for its whole period, and the 30 fps target waits until interval 2 has held for a second (Engage, which asks for 1 with its 60 fps cut, is unchanged). Test switch DM_TEST_PRESENT_LAG
- Survey notes: extra modules are supported, GLSLC outputs in main are counted
- Regression: Mario 1.2.1, Engage and Metroid Dread runs show only their known red lines; t_nvn 63/63; t_present_crop / t_mip_extent build again (t_mip_extent's one failure is the same at a9c56fd)
New profile 010093801237C000 (MercurySteam engine, SDK 11.4.3). A new game
runs through the title, the story recap and the opening cutscene into ZDR,
with movies and music. An unsigned IPA builds from the NSP folder; it has
not been tested on a device yet.
Rewriter and loader
- main has a 1.8 GB bss (the engine's static heap): when an appended patch
area would start beyond 112 MB, it goes 32 MB before the module base
(negative PATCH_OFF, IMAGE_SIZE / FILE_IMAGE_LEN). Mac loader, iOS
embedding (no bss in the executable) and QueryMemory handle both layouts;
Mario's and Engage's output is byte-identical.
- Observation gates may sit right after a bl as well as at function entries.
- Used memory counts that bss, so SetHeapSize asks for 1.38 GB, not the pool.
Host
- Hooks: oe::FinishStartupLogo, fs::CommitSaveData / DeleteDirectory (SDK 11
names), hid::detail::SendVibrationValues (vibration graph flush).
- Guest paths resolve "." and ".." (Atk stream paths use ../../).
- Movies: the SDK's own demuxer runs natively; the decoder supports the
native-texture mode (RGBA into the client's NVNtexture, sync signalled),
the D1 destructor, copied client events, MediaCodec-like Flush / Stop, and
an asynchronous input signal after Start.
- Same-core preemption approximation (profile key core_preempt, Dread only)
plus gates that wait for the movie player's request ring: the game frees
objects its higher-priority threads still use, relying on them preempting
the main thread. 12 of 12 runs clean, before about half crashed.
- SleepThread is precise (macOS timer slack made 33 ms sleeps 36.4 ms): the
game's 30 Hz sound thread drives cutscene frames, now 33.3 ms. Mario's
frame-time p95 19.7 -> 18.4 ms.
- DM_KEYS takes <key>*<count>/<interval>.
Graphics
- BindProgram binds per stage: Dread's depth prepass unbinds only the
fragment stage, which used to drop every prepass draw and leave scenes
black. Composite programs give a vertex-only pipeline.
- WriteTexels writes regions in place (the map writes one texel per call);
t_nvn checks 15 regions byte for byte.
- Pass dumps print Z lines for dropped draws.
- shader_split.py reads MSHD containers (1,010 shaders, 659 pairs).
Regression: Mario reaches frame 6900 with only the known red line; Engage
plays its opening movie and battle cutscene.
- Engage lists all of RomFS (216,554 entries) before its first frame. ReadDirectory called stat, getxattr (guest name) and a failing fopen("<file>.holes") for every entry, about 5 s on the Mac and 10 s on an iPhone (device log)
- guest_name reads the xattr only for .dm-case-* physical names: set_guest_name writes it only when the physical name differs from the guest name, which only those collision names do
- Directories take their type from d_type; files use fstatat and the .holes probe uses openat, both relative to the directory fd instead of resolving the full path again
- First frame 5.4 → 1.8 s on the Mac (three runs: 2.1 / 1.8 / 1.8 s under another app's load), opening movie at 7.4 s; Engage save → battle run clean, DLC still mounted; t_fs 68/68, t_fs_case 51/51; Mario regression t83 clean
- The repository is now nxnative (first called smbw-mobile): a general NSP → native app converter, not a Mario port. README title and history lines updated; each game keeps its own app name and bundle ID (Mario's org.smbwmobile.app stays, so installed apps keep their saves)
- No default game: game.load() without DM_GAME, make -C dm/host / dm/gfx/tests without GAME, and build_ipa.sh without GAME stop and list the profiles (title ID and app name). The host Makefile still lets clean and a bare make (libvpx only) through; host unit tests only need GAME when they use the profile
- The NSP script is unchanged in use: it picks the profile from the NSPs' title ID
- Docs (README, AGENTS.md, profile and gfx READMEs, docs/agents) describe the required GAME / DM_GAME. dmsh_chain.py / dmsh_build.py comments still mention the old default: they are part of the chain hash
- dm/games/smbw → dm/games/010015100B514000 (Mario, still the default everywhere), dm/games/fe_engage → dm/games/0100A6301214E000; GAME= / DM_GAME= take the title ID, and game.py lists the profiles (ID and app name) when given an unknown name
- Defaults updated in game.py, the host / host tests / gfx tests Makefiles and build_ipa.sh; workspaces follow as ~/<title ID>-work and NSP builds as <NSP folder>/<title ID>_build/
- Both chain versions are unchanged (Mario 271056163ea2, Engage a295a8035b7b: the hash covers table contents and keys, not the profile name). dmsh_chain.py and dmsh_build.py keep "default smbw" in their comments on purpose, since their source is part of the chain hash and editing them would force a full shader rebuild
- dmsh_chain.h headers regenerated (comment only); README, AGENTS.md, profile READMEs, gfx READMEs and docs/agents use the IDs
- Mario regression t82 clean; Engage boots and mounts its DLC under the new name; t_fs 68/68, t_fs_case 51/51, t_oe 47/47, t_nvn 61/61; iOS builds
- Engage's two DLC NSPs (…F001, …F002, 121 KB each) hold one empty marker file each (e0, h0); the content itself is already in update 2.0.0 (Fell Xenologue maps fld_e001–e006, the DLC Emblem room, Patch0–3). The host used to report no DLC and had no MountAddOnContent
- Host: aoc::CountAddOnContent / ListAddOnContent report the numbered folders under DM_AOC_DIR (index = title ID − base − 0x1000; hle_misc.c); fs::MountAddOnContent / QueryMountAddOnContentCacheSize mount DM_AOC_DIR/<index> read-only, a missing index answers fs 2-1002 (hle_fs.c). D2R keeps its fixed {1} without DM_AOC_DIR; DM_AOC_OWNED=0 still reports none
- Packing: nx_extract --aoc <dlc.nsp> <dir> writes a DLC's data NCA RomFS; build_ipa_from_nsp.py extracts every DLC NSP to <id>_build/aoc/<index>/ instead of skipping them; build_ipa.sh copies AOC (default $W/data/aoc) to <App>/aoc; the iOS app sets DM_AOC_DIR when it exists
- Verified on the Mac: Engage mounts e0 / h0 and DLCManager.RefreshHasList marks both contents owned (probe main+0x29f3c88: 2 of 2 with the folders, 0 without); nx_extract --aoc matches hactool; the script step tested on the user's NSP folder
- README games row and a bring-up catalogue entry
- path_component matched each path component by scanning the whole parent directory and calling getxattr on every entry (case-sensitive guest names on case-insensitive APFS). Engage's RomFS has 216,554 entries, two Wwise bank directories of 38,000; walking RomFS at boot gave 17 s of black screen on the Mac and 32 s on an iPhone, and every audio file opened in play cost tens of milliseconds under the global path lock
- Read-only mounts (RomFS and overlays, a save that fell back to read-only) now build a sorted guest → physical name index per directory at its first lookup (dir_index_get, about 8 MB for all of Engage); writable mounts keep the exact scan
- Engage on the Mac: first frame 21.5 → 5.7 s, opening movie starts at 11 s instead of 49 s; prologue save → battle → combat run clean
- t_fs 68/68, t_fs_case 51/51 (its Makefile rule lacked $(GINC) and the test lacked g_mod / g_nmod, so it no longer built); Mario regression t81 clean; iOS builds
- docs/agents/03-bringup.md: catalogue entry for a silent black screen at boot
- Title menu → 新增内容 crashed on the iPhone: nn::ec::ShowShopAddOnContentList(ApplicationId, UserHandle) starts the shop applet through applet::CreateLibraryApplet, the host has no applet proxy (oe::Initialize is a no-op), and the SDK read through a null pointer at sdk+0x38660. Only D2R's no-argument overload was hooked
- All 18 nn::ec::ShowShop* overloads Engage imports return void, as if the player closed the shop at once (hle_misc.c); the title menu stays and 继续游戏 still works (Mac)
- Also hooked the other applet launchers Engage imports: friends::ShowFriendList / ShowBlockedUserList / ShowReceivedFriendRequestList / ShowMethodsOfSendingFriendRequest and web::ShowLobbyPage return failure, aoc::ShowAddOnContentLostError and err::ShowUnacceptable{AddOnContent,Application}VersionError are logged
- docs/agents/03-bringup.md: catalogue entry on finding unhooked Show… overloads
- Acknowledgements first: d2r-mobile, Ryujinx, Atmosphere, yuzu, LibHac / hactool, SPIRV-Cross, deko3d and the vendored libraries
- Pack an IPA: requirements, NSP folder layout, terminal examples for Mario and Fire Emblem Engage (version, output, app name, keep-extracted, diagnostic build), Spyder use, what the script does
- Games table lists the two games that pack today and what each NSP folder needs
- Install and play, how it works, porting another game, building by hand, prewarm lists, game notes, tools, license
- AGENTS.md mentions Engage; docs/agents links d2r-mobile on GitHub
- Each AudioOut is logged as audio #N: the open with its guest call site, Start / Stop / Close (first 32 per output), and the 2000-callback stats line per output. An Engage phone log had two outputs after a background round trip, one fed and one starved; the combined stats read as "every other callback underruns"
- DM_OE_SCRIPT gains bg=1 / bg=0, the same sequence as the iOS lifecycle callbacks (focus, audio take-over, guest clock freeze, graphics suspend); DM_AUDIO_MUTE=1 silences the real Core Audio path for such tests on the Mac
- The silent sink no longer makes up time the process was suspended (it pulled seconds' worth at once and logged hundreds of underruns)
- Host tests t_audio, t_audio_spsc and t_oe build again (game profile include, stubs) and pass
- docs/agents/03-bringup.md: several outputs, background round trip on the Mac
- Scripts the user runs now live in scripts/ at the repository root;
build_ipa_from_nsp.py moves there from dm/ios/tools (history kept).
AGENTS.md records the convention.
- It still runs in Spyder (F5, settings block, prompts for an empty
NSP folder or prod.keys). From a terminal it now takes options that
override the settings: <NSP folder>, -k prod.keys, -p version,
-o out.ipa, --app-name, --keep-extracted, --env-file (-h for help).
Without keyboard input a missing value is an error instead of a hung
prompt. Exit codes 0 / 1 / 130 in a terminal; inside IPython (Spyder)
it does not call sys.exit, so the console stays clean.
- The repository root is found by walking up to dm/tools/nx_extract
rather than a fixed depth.
- Checked: -h from another directory; error paths (missing folder, no
terminal input, empty folder, unregistered version) exit 1 with a
clear message; a CLI run on the Engage NSP folder identifies the game,
skips DLC, picks base + update, passes the tool check and stops at
the free-space check; IPython %run under the Spyder interpreter
(prompts, -h, an error) keeps the console alive.
- README, dm/games READMEs and docs/agents point at the new path.
Ported from d2r-mobile (b94670c79, 2eb575877, 704147218), adapted:
- Save transaction: if the work directory cannot be set up at mount
(leftovers not removable, clone fails, e.g. a full disk) the host
now mounts the committed saves read-only and tells the front end
instead of exiting; a reclone after a refused commit clones beside
the work directory and swaps it in atomically, and on failure keeps
it tainted for the next try; a failed rename rollback refuses the
transaction instead of exiting. Creating an existing file or folder
on a read-only mount answers "already exists", and file / directory
calls ignore the null handle a failed open leaves. Test switch
DM_TEST_FS_FAILAT; tests/t_fs.c section 6 (68 of 68 pass). An Engage
run with the mount clone failing loads its autosave read-only and
plays on.
- iOS front end: front_save_failed shows a bilingual notice (it only
logged before).
- thread.c: each guest thread's TLS block, thread-local region and
thread object (9.5 KB) are freed at thread exit through a key of
their own (the guest overwrites the TLS / TLR key slots).
tests/t_thread_leak.c: 600 blocks leaked before, none now.
- Memory line on iOS: os_proc_available_memory, how far the app is
from being killed for memory.
- The host test Makefile gets the game profile include path; t_fs
stubs the module table. Docs: 03-bringup memory line fields,
05-debugging save failures. Mario regression t78 and the iOS compile
are clean.
- Ported from d2r-mobile ef9b1ff13, adapted to our shared-page ranges:
setting a page read-only marked it PROT before mprotect landed, so a
concurrent unprotect-before-write could set it back to NONE and then
the mprotect made it read-only anyway. A game thread writing it then
spun forever in the fault handler, and a host system call into it
failed with EFAULT. Now NONE -> BUSY -> mprotect -> PROT (and back),
waiters re-check after BUSY clears, and the wait is capped at 2 s
with a red line. Engage's movie decoder writes frames from a host
thread while replay protects pages; this is a likely cause of the
old early stalls.
- tests/t_dirty_race.c (ported) reproduces it deterministically: on
the old tracker read() into the page fails with EFAULT; it passes
now, and t_dirty_slots still passes.
- The graphics unit tests had stopped linking (prog_key moved to
hle_nvn.c) and some crashed (hand-built textures lacked the texture
magic replay now checks; t_nvn used a function that has since been
implemented as its unimplemented example and hard-coded 523 names).
Fixed: t_nvn, t_fuse_check, t_zskip, t_xv, t_ub, t_pfv, t_present_crop,
t_rt_depth_ro, t_rt_range_shape, the dirty tests and others pass.
Still failing, the same before this change: t_reorder_chain (30 of
155), t_defer_chain (1 of 59), t_mip_extent (1 of 303), t_rt_range,
t_rt_range_ro; t_refl needs D2R shader samples.
- Mario regression t77, an Engage movie / title / demo run (0 missed
dirty pages) and the iOS compile are clean.
- The storage-sharing texture table (g_alias, keyed by pool object,
offset and description) never evicted. Engage frees memory pools on
scene changes and builds new ones, so the old pools' render targets
and other private textures stayed for good, and linear textures built
on a pool buffer kept the whole buffer alive. A reused pool object
address could also hand an old texture to a new pool. Finalizing a
pool now evicts its entries (mtl_alias_drop_pool) through deferred
release.
- With 21b3bc3, Engage's 850 s idle demo stays at 1.2 to 1.4 GB of GPU
memory (5.4 GB before both fixes) and ends at 2.5 GB resident instead
of 3.6 GB; in the prologue battle GPU memory is 1.07 GB at frame
11100 (2.38 GB before).
- The memory report line adds the host objects held by serial, the
finalized pool buffers still alive, and the sharing table size with
its evictions.
- Mario regression t72 (it does finalize pools: 32 evictions; same
scene, 60 fps, GPU memory unchanged), Engage demo scenes and combat
are clean; the iOS project compiles. Docs: 04-graphics entry.
- Measured over a 12-minute run: StandardAllocator commits only 0.58 GB
of the 3.14 GB it reserves, Unity's own regions 0.97 GB. The profile
now reports a 2.5 GB pool (StandardAllocator about 2.44 GB) and uses
a 5 GB block with a 4.375 GB alias region, the total proven on a
phone. The old 6 GB block was iOS's single-reservation limit and left
Metal no address space. Reserved address space peaks at 3.47 of
4.31 GB; a 670 s battle-script run is clean.
- The early stalls (one run in four) have not come back in 12 runs
since the real thread pause for IL2CPP's GC; the docs mark them
probably fixed and keep the sampling recipe.
- Docs: sizing entry with the commit split and how to measure it,
profile notes, README Engage row (it reaches the battle map; the
scripted presses end on the save screen, not in combat).
- The prologue battle crash is marked fixed, with a pointer to the
address-space entry in 03-bringup (missed in fe9bab9).
- The early stalls get their own open entry, with the sampling recipe
and the new stderr-lock line from the crash handler.
- crash.c: before printing a crash or time-limit report, the handler
tries the stderr lock for 1 s. If another thread holds it, it writes
one line (signal, pc / lr as module offsets) with write() and exits,
instead of hanging with the process and leaving an empty log. This
is for the intermittent early stalls, where even the SIGALRM report
never appeared.
- Engage's idle attract-demo crash (about 220 s) was the same alias
address-space shortage as the battle crash: the particle index
buffers got no mapping past 2 MB. With the node-area cut (fe9bab9) an
850 s idle run is clean, with no corrupted texture objects; with the
old SDK the same run dies at 159 s in Unity's heap allocator.
- Docs: the 04-graphics entry is marked fixed with the cause; 03-bringup
and the README Engage row updated. Mario regression t65 clean; the
iOS project compiles.
- The crash about 6 minutes into the prologue battle (null write at
main+0x9a4664) was the alias region running out of address space, not
memory: Unity's low-level virtual allocator got no region from
AllocateAddressRegion and returned 0. The SDK's AddressRegionManager
keeps the first 1 GB of the alias region as its node area and aligns
each region by size; after StandardAllocator's 3.14 GB block only about
1.2 GB was left for Unity, and it was full and fragmented.
- patch_nso.py: a host-only cut for Engage's sdk build (56ec2474…)
shrinks the node area from 0x3fe00000 to 0x3e00000 (62 MB, about 1.35
million nodes). Both constructor movs change together.
- A 12-minute scripted run now plays through the battle without a crash;
reserved address space levels off at 4.17 of 5.31 GB.
- hle_svc.c: the alias-mapped total counts pages with a bitmap, so maps
of already-mapped pages and oversized unmaps no longer skew it (it went
negative). Engage really maps about 1.5 GB, not 0.5 GB.
- Docs: bring-up catalogue entry with the measuring recipe, profile
memory notes, README Engage row. Mario regression t64 clean; the iOS
project compiles.
- SetThreadActivity and GetThreadContext3 were placeholders that only
reported success. IL2CPP's garbage collector pauses each thread and
reads its registers for roots, so threads kept running and roots held
in registers were missed. A thread inside an SVC now counts as paused
at once with the gate frame as its context (thr_svc_enter / leave
around every gate SVC; it waits at the SVC exit while paused). A
thread in guest code is stopped with SIGUSR2 and parks in the handler.
The context is Horizon's 0x320-byte ThreadContext.
- nn::pctl::IsFreeCommunicationAvailable answers "not available" like
the other free-communication calls; the SDK aborted in sm when Engage
entered its first battle map.
- Memory log lines: heap growth, alias-region high-water mark and the
total mapped in the alias region (one line per 256 MB).
- Engage profile: a 6 GB reservation (5.375 GB alias region, 128 MB
heap region), iOS's single-reservation limit without the extended
virtual addressing entitlement. Engage never calls SetHeapSize; its
mappings reach +5.26 GB.
- Engage on the Mac: with scripted input and DM_KBD answering the name
keyboard, a new game reaches the prologue cutscene and first battle
map. Open: a heap-allocator crash a few minutes into that battle, and
occasional early stalls (docs/agents/04-graphics.md).
- Docs: catalogue entries, README rows. Mario 1.2.1 regression (120 s
scripted) clean; the iOS project compiles.
- crash.c: only the first crashing thread prints its state; later ones
wait in pause(), and a 15 s alarm ends the process if the dump stalls.
Unity's job threads crash together, and the dumps used to interleave
and leave the process hung.
- NVN render target, texture copy and clear records now carry a copy of
each texture object (REC_SNAP / tex_snap) instead of the game's
pointer, as NVN commands do not depend on the API object once
recorded. Replay rejects objects without the texture magic and sizes
beyond Metal's limits with a red line (a 1869367126-high texture used
to abort in Metal's validation).
- SignalToAddress type 2 with count <= 0 and waiters present now
subtracts 2 (mesosphere, Ryujinx and yuzu agree); it subtracted 1.
- Engage on the Mac: scripted input reaches the main menu and a new
game's appearance choice. The idle attract demo still crashes after
about 220 s (Unity's particle geometry job overruns its index buffer);
findings are in docs/agents/04-graphics.md.
- Docs: catalogue entries, README row. Mario 1.2.1 regression (120 s
scripted) clean: only the known SetRenderEnableConditional red line.
- hle_movie.c implements movie::Decoder (the SDK's MediaCodec wrapper)
through hooks on every vtable method: DecoderEvents signalled with
the SDK's SignalEvent, input / output index queues, Buffer and
MediaData filled with the SDK's own setters, one NV12 frame per
packet into the buffer Unity allocates (w x h x 1.5); packets wait
while every output slot is taken. Frames are black for now (no VP9
decoding yet); the size comes from Configure or the VP9 header.
- Engage on the Mac: loading screen, the 94 s opening movie, then the
title screen and, left idle, the 3D attract demo. Not yet: input,
a crash about 221 s into the demo, an unmapped texture format, iOS.
- Docs: catalogue entries (movie protocol, shader keys, CPU-written
textures, NVNprogram size), Engage graphics notes, README row.
- New dm/gfx/tools/unityfs.py: UnityFS bundles (LZ4 / LZMA blocks),
serialized files (formats 19..22), objects read by their type trees
(Unity's common string table comes from the game's own main), and a
Shader object's per-platform compressed subprograms. The player's
own files have no type trees and borrow the bundles' Shader tree of
the same type hash.
- shader_split.py reads Unity Shader objects in a second pass: each
Switch subprogram is one NVN program (0x60-byte header of per-stage
offset / control size / code size, then control + code per stage);
a program's vertex / pixel stages form a pair. Engage: 9,269
shaders, 4,634 pairs. Mario's tables and package are byte-identical.
- Engage profile: shader groups engine / assets, measured NVN layout
(texture handles 0x168 + 8 r, 128 slots; separate samplers 0x568),
regenerated tables, chain a295a8035b7b. 9,268 of 9,269 compile (one
geometry shader; Metal has none).
- dmsh_build.py: the translator's built-in bases only matter with
per-slot facts, so the guard checks that; c2 reads count the sampler
table and 16-byte rows overlapping a table as fillable.
- game.py: DM_GAME_NVN_SAMP_BASE. NSP script survey note updated.
- Mario chain is now 271056163ea2 (Mac and iOS libraries rebuilt,
16,266 of 16,268 as before; c1_full regenerated).
- patch_nso.py: three more x18 forms from IL2CPP code: an offset jump
table (targets read at rewrite time), blr x18 (original return
address kept for C++ unwinding) and an sp write from x18 (alloca).
Mario's rewrite output stays byte-identical.
- loader.c: symbols resolve globally in load order, also for a
module's references to its own exports (local / non-default
visibility excepted). Engage's main overrides operator new / delete;
sdk bound its own copies and freed Unity's blocks into the SDK heap.
- hle_svc.c: QueryMemory reports each module's Horizon layout (code,
read-only, data, patch area) and other host memory as free, which
the SDK's C++ unwinder (nn::ro::detail::GetExceptionInfo) needs.
- New takeovers: nn::fs::IsMounted, nn::ro (hle_ro.c), prepo
Initialize, SDK 12 audio device names and OpenDefaultAudioOut, an
SDK 12 hid block (single GetNpadState, modes, touch, gestures,
six-axis details, colors, applets), movie::Decoder with no impl
(hle_movie.c) and failing nvdrv opens (hle_nvdrv.c), and NVN getters
/ no-ops Engage calls. nvn.h: TextureGetMemoryPool returns a pointer.
- Makefile: a game stamp, so changing GAME rebuilds the host.
- docs/agents/03-bringup.md: catalogue entries for all of the above.
- Mario 1.2.1 scripted run still reaches the opening course with the
same single red line.
- attach(): a render-target view that changes the format is attached
as a Metal view of that format (kept until the submission completes),
so pipeline formats follow it. Mario's last pass writes the RGBA8
swapchain texture through an RGBA8_SRGB view; with the base texture
attached, linear values were stored unencoded and the whole picture
was darker: on the Mac and the iPhone every channel equalled the
Switch value decoded from sRGB once more (0.55 -> 0.26). After the
fix the Mac title frame matches a Switch screenshot (mean RGB
0.556 / 0.621 / 0.633 vs 0.557 / 0.621 / 0.633).
- present_raw_src(): an sRGB-typed swapchain texture is sampled through
a non-sRGB view, so the stored bytes reach the screen as on the
Switch's scan-out (no change for Mario, whose swapchain is RGBA8).
t_present_crop gets an sRGB case (34/34; fails without the fix).
- docs/agents/04-graphics.md and the mapping docs: the rule and how it
was measured.
- hle_nvn.c: drop the command memory callback / grant / caller
logging, the overflow details, the +0x10 write log, the one-off
NVN table dump, and the DM_DBG_GRANTS, DM_DBG_CALLSITES,
DM_DBG_CMDMEM_SHRINK and DM_DBG_OLD_USED switches.
- nvn_mtl.m: drop the per-draw NaN probe (DM_DBG_NANPROBE), the
colour target statistics (DM_DBG_TEXSTAT) and the PASSDUMP
completion timing.
- Kept as regular diagnostics: the audio callback line every 2000
callbacks (frames per callback, queued blocks, underruns) and
DM_DBG_AU_FRAMES (docs/agents/03-bringup.md, crackling on iOS);
DM_NVN_SHOT up to 4096 frames (the 05-debugging burst recipe).
- Mac 1.2.1 scripted run reaches the opening course with the same
single red line as before.
- README: presented as Switch games running natively on Apple
platforms, with Mario as the one supported game. A games table
(Mario playable, Fire Emblem Engage surveyed), component status,
game profiles and how to add a game, the NSP script, generic
step-by-step build commands (profile, workspace and version as
variables), and a Mario section for the per-game notes (1.2.1).
- AGENTS.md: profiles, DM_GAME / GAME, workspaces, build-ID gating,
the chain rule per profile.
- docs/agents: one repository with one profile per title; survey
mode in 02-survey, including the compressed NACP title area.
- dm/gfx README (+ zh-CN): table paths in the profile.
- README: status for v0 / 1.2.1 on the iPhone, update 1.2.1 build
steps, diagnostic env files, finishing the IPA by hand when Xcode
uses a custom build location, app data folders after a reinstall,
prewarm lists (pso_user.tsv / pso_manifest.tsv).
- AGENTS.md: per-build knowledge keyed by build ID, shader_pairs.tsv on
the chain, unsigned IPAs only.
- docs/agents: update extraction and offset porting (01), the
command-buffer byte, command memory, iOS audio, update imports and
build-gated D2R code (03), vertex output padding and iOS math mode
(04), on-device diagnostics and the offline link check (05).
- dm/gfx README / nvn_metal_mapping (en + zh-CN): pairs table, several
game versions, prewarm lists, DM_PSO_USER, command buffer notes;
corrected shader key, build script and 8-bit index lines.
- dm/host/README(.zh-CN).md removed: an index of d2r-mobile's D2R
features; most files it listed no longer exist.
- Audio: run the sdk's own audio renderer on the CPU in manual mode (hle_audren.c),
feed its DeviceSink output to the AudioOut sink; catch-up render on back-to-back
updates fixes the ApplyLightLimiterEffect crash at the title screen
- Button icons: make_ext_font.py generates a stand-in Nintendo extension font
(A/B/X/Y, L/R/ZL/ZR/SL/SR, +/-, D-pad, stick arrows); served as shared font type 5;
the iOS bundle now carries both shared fonts
- Fountain panels: Metal 4 does not order copies inside one encoder; add barriers
when ranges overlap, so the GPU wave simulation no longer reads a stale uniform
- Zero-fill new colour textures; scissored zero clears; uniform-update snapshots
- Docs: graphics ordering, audio, diagnostics, intermittent-glitch method
- dm/ios: lean app shell (main.m boot/env/log, front_ios.m front.h + 16:9 Metal view,
DmGamepad.m touch gamepad in Nintendo positions + GameController), xcodegen project,
tools/build_ipa.sh (embedded game code, iOS shader libraries, RomFS in the bundle)
- Guest address space back to 5 GB (heap 1.625 GB; the game asks for 1.47 GB)
- README / agent guide: iOS build steps and address-space sizing
- dmsh_build.py: vertex shaders whose inputs are only used as integer bit casts are
re-translated with uint inputs (529 shaders); bone indices were converted to floats and
every character was stretched off screen
- Host: Metal vertex formats follow the shader input type (signedness swap, 32-bit float
reinterpretation), mismatches reported
- LINEAR_RENDER_TARGET textures in CPU-visible pools are zero-copy too (sky colour ramps)
- Pass dump lists all vertex attributes; docs and README updated
- patch_nso.py: Mario main cut gives every resource heap +16 KB; Model/NpcCommonerA.bfres
came up 0x70 bytes short of its size-table budget while loading Course900
- dmsh_build.py: compute instructions start at code area +0x100, not +0x30; the misaligned
decode turned 12 particle / post-effect compute shaders into endless loops (GPU hang)
- Linear textures in CPU-visible pools are built on the pool buffer, so the CPU-rasterized
glyph cache reaches the GPU (menu and dialogue text)
- Host command-memory segments grow on demand; RGB10A2SN vertex format; instanced DrawArrays
- DM_PROBE: thread IDs, register-relative dumps, hit filters, ExpHeap walker
- dm/tools/ghidra/Decompile.java; agent guides and README updated
- DM_KEYS scripted controller input ("seconds:button[+button][~hold]")
- NVN virtual memory pools aliased onto their physical pool when every
MapVirtual request shares one base (Mario's 128 MB pool); previously
MapVirtual failed and the game treated the pool as unusable
- DM_PROBE ported from d2r-mobile (host/probe.c): brk probes with
register / backtrace output, plus DM_PROBE_X2 object dumps; used
because lldb launches now wait on a macOS authorization prompt
- diagnostics: DM_NVN_VLOG (pools, MapVirtual), DM_NVN_CBLOG (command
memory grants and callbacks), DM_NVN_ALLOCPROF
- docs: new failure patterns and the probe recipe
Open: starting a new game crashes while loading the opening course
(model resource heap overflow for NpcCommonerA.bfres).
Shaders
- shader_split.py (new): scan RomFS and main for every BNSH and agl BAHS
container (bfsha, particle effects, layouts, agl/gsys packs, main):
12,811 shaders, map + compute facts from the control section
- key = sha256(control + code): many shaders share a control section
- NVN 55 binding layout: bound textures at 0x20 + 8*slot, images 0x120
- bindless textures read from uniform buffers (tex_ubo / samp_ubo)
- GL depth mode: translator remaps z' = z/2 + w/2 in vertex shaders
- compute code areas carry a 0x30 header (magic 0x12345679)
Replay
- 8-bit index buffers widened to 16-bit
- texture swizzle through Metal swizzled views
- non-packaged textures shared by storage (fixes a ~13 MB/frame leak
from transient render targets rebuilt every frame)
- view_t packed to 16 bytes (SDK 16 NVNtextureView; fixes a stack smash)
- TBDR optimizations off by default until they know bindless readers
Host
- alias region 2.875 GB for the 1.75 GB VirtualAddressMemory reservation
- ideal core -2 resolves to the process default core
- shared font served with a BFTTF "decoded" header
- crash reports scan the stack and dump the popped frame
Docs: docs/agents (guides for porting other titles), AGENTS.md
