Profile 0100C1F0051B6000 (SDK 4.4.0, Retro Studios' engine, NVN): plays on the Mac from boot through the menus and the intro cinematic into the first level at 60 fps. General changes: - svc SetHeapSize: a request beyond the heap region returns ResultOutOfMemory, as Horizon (the game binary-searches its heap size between 3 and 6 GB) - shader_split.py: SNVN program records in Retro's RFRM forms (MaterialArchive.arc, the packs' MTRL resources) and the LZSS-compressed startup material archive in main - nvn_mtl.m: a texture in a CPU_UNCACHED pool starts with its storage at first use (title logo, menu panels); transform feedback captures draws that rasterize too - dmsh_build.py: transform feedback buffers holding several outputs (DK's fur simulation); chain header regenerated - hle_audren.c: the renderer event is the kernel event the audio service gives, not the SDK's local one (the main thread was starved at the first level's load) - hle_oe.c: oe::GetUserInactivityDetectionTimeExtensionMode
9.4 KiB
Notes for agents
-
This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it (
dm/tools/patch_nso.py), on a host that answers Horizon's SVCs and the SDK's services (dm/host) and implements NVN, Vulkan and NVIDIA's desktop OpenGL on Metal (dm/gfx/nvn,dm/gfx/vk,dm/gfx/gl). Seventy-two games are playable, each tested on the Mac or the iPhone (a Mac gameplay test is enough to count); the games table lists each with its build, SDK, engine and graphics API, and links its note,dm/games/<id>/README.md: what its profile needs and why, what was found on it, how to drive it on the Mac, what is open. README.md is the player-facing page (games, packing, playing); docs/development.md has the internals, the script's options, building by hand, prewarm lists and power. -
Where to start:
Task Read first A crash, abort, hang, [RED]line or wrong picturedocs/agents/symptoms.md: search it for the text in the log or what you see; it leads to the entry in docs/reference and the games it was found on. Recipes (crash reports, lldb, probes, dumps) are in 05-debugging.md A game that does not reach its frame loop 03-bringup.md (the run loop, reading the log), then the symptom index Rendering or shaders 04-graphics.md, then rendering, shaders and NVN objects Work on one game, or a change to its profile its note, dm/games/<id>/README.md, then dm/games/README.md for the keysAnother Switch title the NSP script (below), then docs/agents Packing, building, prewarm lists, power docs/development.md -
Where a new finding goes, once: a fact about one game (its passes, shader keys, measurements, why its profile needs a key or a cut, its status) in that game's note, in the section it belongs to and marked current or resolved; a general mechanism (a host or tool change that holds for other games) as an entry in its
docs/reference/<subsystem>.mdsection (Symptom, Cause, Fix, Found on with the game's note linked), thenpython3 dm/tools/docs_symptoms.pyto update the symptom index, and a line under the game note's "Found on this game". Link to a finding instead of repeating it. Adding a game takes its note and its row in the games table. Once Mac gameplay testing passes, add it to README's regular games gallery with a screenshot and IPA size; Mac testing is sufficient for inclusion. Add caption notes only for noteworthy features or limitations (such as a 60 fps upgrade or controller requirement); otherwise show only the size. Keep testing milestones, platform and remaining validation in the game note. This file changes only when a rule does (and the count above).dm/tools/tests/test_docs.pychecks the links and anchors, the notes and their table rows, the symptom index, and the length of this file (1,500 words) and of each guide indocs/agents(6,000). -
Each game's constants and shader tables form a game profile in
dm/games/<id>/, where<id>is the base game's title ID:game.tsvplus the tables (andpatches.pywhen the rewriter has patches for its builds). Mario's profile is010015100B514000. Tools pick the profile withDM_GAME;make -C dm/hostandbuild_ipa.shpick it withGAME. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code (dm/games/README.md). -
Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it.
-
Fix a bug at its root cause, not around it. Before changing code, find why it happens: the faulting instruction and its decompilation, the value that went wrong and who wrote it, the Horizon / NVN / Vulkan / Metal semantics the host or the translator gets wrong. Then fix that place. A change that only hides the symptom (a stub, a skipped call, a clamp, a retry, a sleep or a longer timeout, a cache flush, a profile key that turns a feature off) is a workaround: use one only when the cause is known and lies where it cannot be fixed (the game's own code, which a rewriter cut changes deliberately, or a platform limit), or when the user asks for a stopgap. Then say so in the code comment and in the game note or reference entry: the cause, or what is still unknown, marked open.
-
Per-build knowledge is keyed by the NSO build ID: rewriter cuts, observation gates and frame-step stubs in the profile's
patches.py(a game's own modules) ordm/tools/sdk_patches.py(Nintendo'ssdkmodules, which several games ship), whichdm/tools/patch_nso.py(the general rewriter) loads, the profile'sbuildrows (the builds the game was brought up on, and their display versions; the NSP script packs any other version from a derived profile, without those builds' fixes), and host behaviour that uses fixed module addresses checksmodule_t.build_id(fromfixups.tsv;dm_main_build()). When a new game build appears, carry addresses over withdm/tools/nso_fnmap.pyand check them by disassembly; never let fixed addresses run on a build they were not found in. For automatic cuts across updates, opt individual patches intoMATCH_CUTSonly after verifying their function fingerprints and assumptions; follow Function-matched cuts. The shared matcher derives addresses from code; fixed addresses remain build-gated. -
To analyse or port another Switch title, first run
scripts/build_ipa_from_nsp.pyon its NSP folder. With no matching profile, it writes a survey to<NSP folder>/<title ID>_survey/survey.md, then a provisional profile guessed from the game files (dm/games/profile_guess.py;<NSP folder>/<title ID>_build/profile/<title ID>/, found by the tools throughDM_GAMES_DIR), with a short heap check on the Mac when the game has virtual address memory, and a best-effort IPA that may not boot. Then follow docs/agents (extraction, binary survey, host bring-up, graphics specifics, debugging recipes; the symptom index for each failure) and add a profile with its note. -
Scripts the user runs live in
scripts/at the repository root. Each one runs from a terminal (non-zero exit on error).build_ipa_from_nsp.pyalso runs in Spyder (F5, configured by the settings block at its top; command-line options override those settings); the others need not. Tools that only agents or other scripts call stay underdm/. -
Game files, keys and anything extracted from them live in the game's NSP folder, never in this repository and never in the home folder (it is nearly full): the work folder
<NSP folder>/<id>-workfor extraction, runs and analysis, beside the NSP script's<id>_build/. Tool venvs and other large scratch data go on that drive too. Do not print key values. -
Language: everything is in English:
README.md, docs, code comments, log messages, script output and other files. Red log lines (something unimplemented or wrong) start with[RED]. Text that must match a game's own Chinese data is written as escapes ("\xE8\xB5\xB7"), never as characters. -
The shader translation chain has one version for every game,
DMSH_CHAINindm/gfx/nvn/dmsh_chain.h, which the host is compiled with and loads only libraries of. It covers the chain's tools:dm/gfx/tools/dmsh_*(dmsh_build.py,dmsh_chain.py,dmsh_fetch.py,dmsh_vin.py,dmsh_pull.pywith its decoderdmsh_pull.metal,dmsh_geometry.py,dmsh_facts.py),Program.cs,ios_metal.pyanddmsh_chain.lock. A profile's own inputs (shader_map.tsv,shader_facts.tsv,shader_pairs.tsv,bf_want.tsv, itsshader_*/nvn_*keys) are not part of it: they are in each shader's cache key, and their digest stamps the libraries (b-<V>/inputs.tsv). The procedure:- After changing a chain tool, run
dmsh_chain.py --headeronce and commitdm/gfx/nvn/dmsh_chain.hwith the change (dm/tools/tests/test_nsp_chain.pyfails on a stale one). - After that, or after changing a profile's inputs, rebuild that profile's shaders: a full
dmsh_build.pyrun (Mac and, withIOSOUT, iOS) stamps its libraries and writes the profile'sc1_full.tsvandcb_extent.tsvfrom its Mac outputs (git ignores them; the host takes them only beside the stamp they name). Give each game its own output directory. - Rebuild the host.
The translator is
dm/gfx/tools/dmsh_xlate_bin, which matchesdmsh_chain.lock; a freshdotnet publishdoes not. - After changing a chain tool, run
-
iOS builds are unsigned IPAs that the user sideloads and tests; diagnostics come back as the app's
Documents/log.txt. Do not sign or install builds on the user's devices. -
Commit only when asked, with
git -c user.name=seedds -c user.email=seedds@gmail.com commit.