Compact NVN buffer bindings and remove Wonder heap workaround

This commit is contained in:
seedds committed 2026-10-26 01:21:10 +08:00
1 parent 7dc250e2f1
commit caee62a853
12 files changed
+172 -80

No files matched your search

+5 -13
View File
@@ -232,21 +232,13 @@ The guest address space, heaps, pools, virtual address memory and the allocators
- **Note:** read the probe log carefully: `tryAlloc` probes show *requests*, not successes, so a running total that already exceeds the heap means an earlier request failed.
- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md), [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md) (the two entries below).
### Per-resource heaps sized from Nintendo's measured peaks
- **Symptom:** a null-pool buffer for `Model/NpcCommonerA.bfres` in the opening course.
- **Cause:** each resource gets its own `sead::ExpHeap` sized *resource-size-table value + 0x1008* (`main+0x1e603c` → `ExpHeap::create` `main+0x4aac38`). The table values are Nintendo's measured peaks with almost no margin (Yoshi's animation file uses 99 % of its budget). `Model/NpcCommonerA.bfres` came up about 0x70 bytes short: its last GPU chunk (0x9000 bytes, 4 KB aligned) fit, but the chunk's 0x598-byte descriptor did not.
- **Fix:** a registered code cut gives every resource heap +16 KB (`CUTS` in the profile's `patches.py`, keyed by the module build ID, original bytes verified). Per-title code patches go there, with the reason; those of Nintendo's `sdk` modules in `dm/tools/sdk_patches.py`.
- **Note:** Breath of the Wild's identical failure came from sizes the game takes from the host (next entry); whether Mario's chunk is sized the same way is not checked yet.
- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md).
### Model heaps sized by the host's command record sizes
- **Symptom:** `main+0x155a7a4` writes through a null `operator new` at the world load, about 1 run in 3.
- **Cause:** each model's heap is its table entry + 0x1008, the file goes in 4 KB aligned, and the setup (`main+0x155a2ac`) takes the rest as a frame heap (vtable `main+0x1d4f950`, head `+0xe0`, tail `+0xe8`). Its use is the same in every run; only the file's alignment pad varies (0 to 0xff8), and the entries leave room for any pad, about 0xd00 to spare on most models. Three models were a 4 KB page short, so the loads with a large pad failed. The game sizes the model's prerecorded commands by recording each mesh's draw and each shape's per-pass state into a scratch command buffer and summing `GetCommandMemoryUsed` (`main+0xf9aeb4`, `main+0xf9bb54`), and our records were larger than the driver's (draws carried the indirect and multi-draw fields, every recording began with a BEGIN record).
- **Fix:** plain draws now leave those fields out and recordings start with their first command (`hle_nvn.c`; [What `GetCommandMemoryUsed` counts](nvn-objects.md#what-getcommandmemoryused-counts)).
- **Check:** to check a whole scene, probe where the setup heap is cut (`main+0x155a520` with `!x21`, the resource heap) and the end of the setup (`main+0x155a5f4` with `!x2`, the frame heap), with `DM_PROBE_HEAP=28,38,40,e0,f8 DM_PROBE_HEAPFRAME=1d4f950 DM_PROBE_HEAPWORDS=1` ([05](../agents/05-debugging.md#probes-without-a-debugger-dm_probe)). A model fits any placement when entry + 0x1008 covers the resource heap's header and object (0x310), the file, the frame heap's block header and its peak use, plus 0xff8 of pad.
- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md).
- **Symptom:** a model load exhausts its resource heap: a null `operator new` at the world load, or `BufferInitialize: pool has no storage` followed by a null-pool crash.
- **Cause:** games size model allocations by measuring scratch command recordings; oversized host records can exceed the budgets in their resource-size tables. In Breath of the Wild, each model's heap is its table entry + 0x1008, the file goes in 4 KB aligned, and the setup (`main+0x155a2ac`) takes the rest as a frame heap (vtable `main+0x1d4f950`, head `+0xe0`, tail `+0xe8`). Its use is the same in every run; only the file's alignment pad varies (0 to 0xff8), and the entries leave room for any pad, about 0xd00 to spare on most models. Three models were a 4 KB page short, so the loads with a large pad failed. The game sizes the model's prerecorded commands by recording each mesh's draw and each shape's per-pass state into a scratch command buffer and summing `GetCommandMemoryUsed` (`main+0xf9aeb4`, `main+0xf9bb54`), and our records were larger than the driver's (draws carried the indirect and multi-draw fields, every recording began with a BEGIN record).
- **Fix:** plain draws leave unused indirect fields out and recordings start with their first command. Uniform and single vertex-buffer bindings also use compact 24-byte records instead of 32, preserving full GPU addresses and retaining full-width fallback records for ranges above `UINT32_MAX`; multi-vertex ranges retain their array representation. Replay expands the compact forms into the same canonical binding state. The native inline uniform command takes 20 bytes and the native vertex-buffer command 24, so these small differences can accumulate into an extra 4 KB GPU-pool chunk. Report the bytes actually recorded; do not underreport usage or add a game-specific heap allowance. See [What `GetCommandMemoryUsed` counts](nvn-objects.md#what-getcommandmemoryused-counts).
- **Check:** for Breath of the Wild's whole scene, probe where the setup heap is cut (`main+0x155a520` with `!x21`, the resource heap) and the end of the setup (`main+0x155a5f4` with `!x2`, the frame heap), with `DM_PROBE_HEAP=28,38,40,e0,f8 DM_PROBE_HEAPFRAME=1d4f950 DM_PROBE_HEAPWORDS=1` ([05](../agents/05-debugging.md#probes-without-a-debugger-dm_probe)). A model fits any placement when entry + 0x1008 covers the resource heap's header and object (0x310), the file, the frame heap's block header and its peak use, plus 0xff8 of pad.
- **Found on:** [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md), [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md#memory) (buffer bindings; the game note records the failed allocation, measured sizes and validation).
### Unity's GPU pool block allocation spins at size 0
+2 -2
View File
@@ -11,13 +11,13 @@ The NSO rewriter (`dm/tools/patch_nso.py`) and its per-build tables (`dm/tools/s
- **Fix:** a profile can opt individual cuts into `MATCH_CUTS` in its `patches.py`. The shared matcher, `dm/tools/nso_match.py`, selects a unique complete unwind function by length and SHA-256 fingerprint. It retains constants, registers and internal branch destinations; normalizes relocation addresses; and checks the normalized bodies of direct callees or their imported symbol names. Original bytes are checked again before a cut is applied. Missing boundaries, unknown callees, changed code and ambiguous matches produce a red diagnostic and no cut. Overlapping rules stop the rewrite. Every accepted cut is recorded in `<module>.cuts.tsv` with `[function match]` in its reason.
- **Scope:** the repository profile selected by `DM_GAME`, keyed by module name. A derived profile still uses that repository profile's rules. Other games' rules never participate. A build with registered `CUTS` uses those instead, including an explicitly empty list. SDK patches, observation gates, frame-step stubs and fixed host addresses retain their existing build-ID gates. The rewrite cache includes the selected profile's patch file even for unknown builds, the selected game ID and the matching tool.
- **Limits:** this is an opt-in code-identity check, not automatic compatibility for every game update. Direct callee bodies are checked one level deep; their callees, indirect-call targets and external data layouts are not proven unchanged. Enroll only a cut whose assumptions remain valid under those changes. Prefer fixing the general host behavior when possible. A replacement must contain no PC-relative instruction; relocated branch replacements need a different mechanism. Failure leaves the game potentially without its workaround, and stays visible in the rewrite log.
- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md#memory), whose resource-heap allowance was missing on 1.0.1.
- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md#memory), whose former resource-heap allowance was missing on 1.0.1. The shared matcher remains, but Wonder's allowance and matching rule were later removed after the general command-record fix described in its game note.
To enroll a cut for any game's own module:
1. Start from a verified fixed cut and its original NSO. Run `python3 dm/tools/nso_match.py <module.nso> <offset> <old hex> <new hex> <reason>` with the tools' Python environment. It emits a rule containing `name`, `size`, `fingerprints` and function-relative `cuts`, and rejects an ambiguous reference. Keep game files and scratch outputs in the game's NSP work folder.
2. Check the function and callees in another known build using `nso_fnmap.py` and disassembly. Check that the cut still has the same effect. `Image.match(rule)` must resolve the reviewed location. If a reviewed implementation differs, generate its fingerprint separately; combine fingerprints only when function size and relative cuts agree. Retain exact `CUTS` for known builds.
3. Add the rule under `MATCH_CUTS = {'main': [rule]}` (or the appropriate non-SDK module) in that game's `patches.py`. Record the evidence and remaining runtime validation in the game's note. Do not copy Wonder's rule or allowance into unrelated games.
3. Add the rule under `MATCH_CUTS = {'main': [rule]}` (or the appropriate non-SDK module) in that game's `patches.py`. Record the evidence and remaining runtime validation in the game's note. Do not transplant a workaround into unrelated games.
4. Verify an unregistered build-ID copy through the rewriter, comparing its output with the known fixed cut, and run `test_nso_match.py`, `test_patches.py` and `test_nro_pack.py`. The matching tests use synthetic modules from two profiles to check relocation, rejection and isolation without game files.
## Jump tables