37 lines
10 KiB
Markdown
37 lines
10 KiB
Markdown
# Notes for agents
|
|
|
|
- nxnative runs rewritten ARM64 game code on Apple hardware. The rewriter is `dm/tools/patch_nso.py`; `dm/host` implements Horizon services; `dm/gfx` maps NVN, Vulkan and OpenGL to Metal. The [games table](dm/games/README.md#games) records profiles, builds, engines and per-game notes. [README.md](README.md) covers players; [docs/development.md](docs/development.md) covers internals and builds. A Mac gameplay test is sufficient for gallery inclusion; retain tested platforms and limitations in the game note.
|
|
- For every upstream update, follow [07-upstream-maintenance.md](docs/agents/07-upstream-maintenance.md). Preserve our frontend, controls, saves and identities. Synchronize every accepted profile, artwork mapping and both README galleries; run the catalog gate before merging to main and publishing.
|
|
- Where to start:
|
|
|
|
| Task | Read first |
|
|
|---|---|
|
|
| Desktop UI | [Design system](docs/design-system.md) |
|
|
| A crash, abort, hang, `[RED]` line or wrong picture | [docs/agents/symptoms.md](docs/agents/symptoms.md): search it for the text in the log or what you see; it leads to the entry in [docs/reference](docs/reference/README.md) and the games it was found on. Recipes (crash reports, lldb, probes, dumps) are in [05-debugging.md](docs/agents/05-debugging.md) |
|
|
| A game that does not reach its frame loop | [03-bringup.md](docs/agents/03-bringup.md) (the run loop, reading the log), then the symptom index |
|
|
| Rendering or shaders | [04-graphics.md](docs/agents/04-graphics.md), then [rendering](docs/reference/rendering.md), [shaders](docs/reference/shaders.md) and [NVN objects](docs/reference/nvn-objects.md) |
|
|
| Work on one game, or a change to its profile | its note, `dm/games/<id>/README.md`, then [dm/games/README.md](dm/games/README.md) for the keys |
|
|
| Another Switch title | the NSP script (below), then [docs/agents](docs/agents/README.md) |
|
|
| Packing, building, prewarm lists, power | [docs/development.md](docs/development.md) |
|
|
| Checking a change before a pull request | [06-dev-loop.md](docs/agents/06-dev-loop.md) |
|
|
- Where a new finding goes, once: a fact about one game (its passes, shader keys, measurements, why its profile needs a key or a cut, its status) in that game's note, in the section it belongs to and marked current or resolved; a general mechanism (a host or tool change that holds for other games) as an entry in its `docs/reference/<subsystem>.md` section (Symptom, Cause, Fix, Found on with the game's note linked), then `python3 dm/tools/docs_symptoms.py` to update the symptom index, and a line under the game note's "Found on this game". Link to a finding instead of repeating it. Adding a game takes its note and its row in the games table. Once Mac gameplay testing passes, add it to README's regular games gallery with a screenshot and IPA size; Mac testing is sufficient for inclusion. Add caption notes only for noteworthy features or limitations (such as a 60 fps upgrade or controller requirement); otherwise show only the size. Keep testing milestones, platform and remaining validation in the game note. This file changes only when a rule does (and the count above). `dm/tools/tests/test_docs.py` checks the links and anchors, the notes and their table rows, the symptom index, and the length of this file (1,500 words) and of each guide in `docs/agents` (6,000).
|
|
- Each game's constants and shader tables form a game profile in `dm/games/<id>/`, where `<id>` is the base game's title ID: `game.tsv` plus the tables (and `patches.py` when the rewriter has patches for its builds). Mario's profile is `010015100B514000`. Tools pick the profile with `DM_GAME`; `make -C dm/host` and `build_ipa.sh` pick it with `GAME`. There is no default game: set it every time (the tools stop and list the profiles otherwise). Put per-game values in the profile, not in code ([dm/games/README.md](dm/games/README.md)).
|
|
- Prefer general optimizations and fixes to game-specific rules. Performance work (frame pacing, power, caches, load times) counts here as much as correctness. First look for a change that holds for every game: Horizon / NVN / Vulkan semantics, or what the binary, its SDK version, its imports or its files show. Use a game-specific rule (a profile key, a rewriter cut, a build-ID gate, a per-game branch in the host) only when it is absolutely necessary, that is, when no general rule can do it.
|
|
- Fix a bug at its root cause, not around it. Before changing code, find why it happens: the faulting instruction and its decompilation, the value that went wrong and who wrote it, the Horizon / NVN / Vulkan / Metal semantics the host or the translator gets wrong. Then fix that place. A change that only hides the symptom (a stub, a skipped call, a clamp, a retry, a sleep or a longer timeout, a cache flush, a profile key that turns a feature off) is a workaround: use one only when the cause is known and lies where it cannot be fixed (the game's own code, which a rewriter cut changes deliberately, or a platform limit), or when the user asks for a stopgap. Then say so in the code comment and in the game note or reference entry: the cause, or what is still unknown, marked open.
|
|
- Per-build knowledge is keyed by the NSO build ID: rewriter cuts, observation gates and frame-step stubs in the profile's `patches.py` (a game's own modules) or `dm/tools/sdk_patches.py` (Nintendo's `sdk` modules, which several games ship), which `dm/tools/patch_nso.py` (the general rewriter) loads, the profile's `build` rows (the builds the game was brought up on, and their display versions; the NSP script packs any other version from a derived profile, without those builds' fixes), and host behaviour that uses fixed module addresses checks `module_t.build_id` (from `fixups.tsv`; `dm_main_build()`). When a new game build appears, carry addresses over with `dm/tools/nso_fnmap.py` and check them by disassembly; never let fixed addresses run on a build they were not found in. For automatic cuts across updates, opt individual patches into `MATCH_CUTS` only after verifying their function fingerprints and assumptions; follow [Function-matched cuts](docs/reference/rewriter.md#function-matched-cuts-for-unregistered-builds). The shared matcher derives addresses from code; fixed addresses remain build-gated.
|
|
- To analyse or port **another** Switch title, first run `scripts/build_ipa_from_nsp.py` on its NSP folder. With no matching profile, it writes a survey to `<NSP folder>/<title ID>_survey/survey.md`, then a provisional profile guessed from the game files (`dm/games/profile_guess.py`; `<NSP folder>/<title ID>_build/profile/<title ID>/`, found by the tools through `DM_GAMES_DIR`), with a short heap check on the Mac when the game has virtual address memory, and a best-effort IPA that may not boot. Then follow [docs/agents](docs/agents/README.md) (extraction, binary survey, host bring-up, graphics specifics, debugging recipes; the [symptom index](docs/agents/symptoms.md) for each failure) and add a profile with its note.
|
|
- Scripts the user runs live in `scripts/` at the repository root. Each one runs from a terminal (non-zero exit on error). `build_ipa_from_nsp.py` also runs in Spyder (F5, configured by the settings block at its top; command-line options override those settings); the others need not. Tools that only agents or other scripts call stay under `dm/`.
|
|
- Game files, keys and anything extracted from them live in the game's NSP folder, never in this repository and never in the home folder (it is nearly full): the work folder `<NSP folder>/<id>-work` for extraction, runs and analysis, beside the NSP script's `<id>_build/`. Tool venvs and other large scratch data go on that drive too. Do not print key values.
|
|
- Language: the main README and desktop player UI default to English, with a Spanish README and an English/Spanish desktop selector as requested by the repository owner; technical documentation and code remain in English: `README.md`, docs, code comments, log messages, script output and other files. Red log lines (something unimplemented or wrong) start with `[RED] `. Text that must match a game's own Chinese data is written as escapes (`"\xE8\xB5\xB7"`), never as characters.
|
|
- The shader translation chain has one version for every game, `DMSH_CHAIN` in `dm/gfx/nvn/dmsh_chain.h`, which the host is compiled with and loads only libraries of. It covers the chain's tools: `dm/gfx/tools/dmsh_*` (`dmsh_build.py`, `dmsh_chain.py`, `dmsh_fetch.py`, `dmsh_vin.py`, `dmsh_pull.py` with its decoder `dmsh_pull.metal`, `dmsh_geometry.py`, `dmsh_facts.py`), `Program.cs`, `ios_metal.py` and `dmsh_chain.lock`. A profile's own inputs (`shader_map.tsv`, `shader_facts.tsv`, `shader_pairs.tsv`, `bf_want.tsv`, its `shader_*` / `nvn_*` keys) are not part of it: they are in each shader's cache key, and their digest stamps the libraries (`b-<V>/inputs.tsv`). The procedure:
|
|
1. After changing a chain tool, run `dmsh_chain.py --header` once and commit `dm/gfx/nvn/dmsh_chain.h` with the change (`dm/tools/tests/test_nsp_chain.py` fails on a stale one).
|
|
2. After that, or after changing a profile's inputs, rebuild that profile's shaders: a full `dmsh_build.py` run (Mac and, with `IOSOUT`, iOS) stamps its libraries and writes the profile's `c1_full.tsv` and `cb_extent.tsv` from its Mac outputs (git ignores them; the host takes them only beside the stamp they name). Give each game its own output directory.
|
|
3. Rebuild the host.
|
|
|
|
The translator is `dm/gfx/tools/dmsh_xlate_bin`, which matches `dmsh_chain.lock`; a fresh `dotnet publish` does not.
|
|
- A change to the chain, host or tools runs `scripts/regress.py` over the local games, and its report goes into the PR ([06-dev-loop.md](docs/agents/06-dev-loop.md)).
|
|
- Build outputs remain unsigned IPAs. The user may sign/install through our native helper or Sideloadly. Agent tests never sign/install on user devices; diagnostics: the app's `Documents/log.txt`.
|
|
- Commit only when requested or as part of an authorized repository upload. Personal commits use the repository owner: `Erick Sanchez <7268597+nalancer08@users.noreply.github.com>`. Never attribute personal changes to seedds; retain original upstream authors.
|
|
|
|
- Desktop functionality releases must increment `desktop/release.json` version/build. The updater offers newer builds only. Preserve existing checkouts and games; test updates with an external cache.
|