Ghost Trick: Phantom Detective (010029B018432000, base game 1.0.0 = v0, Capcom's RE Engine on SDK 13.3.0, with the Bonus Content add-on): game profile, and finalized pool buffers no longer kept by a guest thread's autorelease pool (every game). Plays on the iPhone (IPA 3.0 GB, user)
The profile is the NSP script's provisional one with tex_flush_upload and tex_dirty_upload: RE Engine writes its UI textures (ASTC) into CPU-visible pools in place, as Monster Hunter Rise does, and without both keys the language screen's background and buttons were magenta (with tex_flush_upload alone they still were). 7,744 shaders from the PAK's SDF definitions and main; 7,739 build. The other 5 are compute shaders with atomics on a 3D image, which SPIRV-Cross indexes with an int3, as Octopath Traveler 0's 16; none was requested on the routes below. PR #138 (Nishikori-Yui) built them by flattening the coordinate in dmsh_build.py, a chain change that would rebuild every game's libraries, so it is left out until a run requests one (the game note). PR #138's profile is the same as this one (the same keys and shader tables). Host fix: the memory line's "freed but alive" count only grew in most games (Animal Crossing: New Horizons 3,113 after 27,900 frames, Pokémon Sword 2,050, Super Smash Bros. Ultimate 1,279; Ghost Trick 125 to 146 buffers, 6.6 to 8.7 GB of zero-copy buffers, so GPU allocated read 9.9 to 11.7 GB against 2.2 GB resident). A temporary census of every host table that can hold a texture over a pool buffer found no holder: mtl_pool_del runs on the guest thread that finalizes the pool, whose implicit autorelease pool never drains, and adding the buffer to the weak table g_dead_pools left it autoreleased there. containsObject: / removeObject: on a weak NSHashTable do the same (a standalone test), and res_add / res_remove call them on guest threads too. mtl_pool_del, res_add_at and res_remove now have their own @autoreleasepool. Ghost Trick: 0 freed but alive, GPU allocated 1.7 GB instead of 8.3 GB 300 frames into the boot and 2.9 GB in Chapter 1, resident memory, frame times and pictures unchanged; Pokémon Sword: 8 to 0 at the title after 90 s, the same frame pacing. On the Mac: the opening movies, the language screen, the first-boot notices, save creation, the Bonus Content notices and Chapter 1's opening in the junkyard (the Ghost World, the first trick), and a second boot that loads the save, at 60 fps (frame interval p99 about 17 to 19 ms), about 2.3 GB resident, no red line; the Wwise sound plays (a muted run with DM_AUDIO_DUMP). Checked: the tool tests (57, the docs check among them), t_nvn (124 checks), a boot of the rebased host. README (games gallery, a Mac picture of the junkyard), AGENTS.md's count (sixty-two: the gallery's cells), the game note, the games table, docs/reference/nvn-objects.md and the symptom index say so
This commit is contained in:
seedds committed
1 parent
16f4889192
commit
7d368c4249
15 files changed
+18819
-14
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Notes for agents
|
||||
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it (`dm/tools/patch_nso.py`), on a host that answers Horizon's SVCs and the SDK's services (`dm/host`) and implements NVN, Vulkan and NVIDIA's desktop OpenGL on Metal (`dm/gfx/nvn`, `dm/gfx/vk`, `dm/gfx/gl`). Sixty games are playable on the iPhone; the [games table](dm/games/README.md#games) lists each with its build, SDK, engine and graphics API, and links its note, `dm/games/<id>/README.md`: what its profile needs and why, what was found on it, how to drive it on the Mac, what is open. [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and power.
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it (`dm/tools/patch_nso.py`), on a host that answers Horizon's SVCs and the SDK's services (`dm/host`) and implements NVN, Vulkan and NVIDIA's desktop OpenGL on Metal (`dm/gfx/nvn`, `dm/gfx/vk`, `dm/gfx/gl`). Sixty-two games are playable on the iPhone; the [games table](dm/games/README.md#games) lists each with its build, SDK, engine and graphics API, and links its note, `dm/games/<id>/README.md`: what its profile needs and why, what was found on it, how to drive it on the Mac, what is open. [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and power.
|
||||
- Where to start:
|
||||
|
||||
| Task | Read first |
|
||||
|
||||
@@ -90,44 +90,45 @@ Under each game: the IPA's size, and "60 fps with 1.2.0" where the game runs at
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/fire-emblem-engage.jpg" width="256" alt="Fire Emblem Engage"><br>Fire Emblem Engage<br><sub>~15 GB · 60 fps with 2.0.0</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/fire-emblem-three-houses.jpg" width="256" alt="Fire Emblem: Three Houses"><br>Fire Emblem: Three Houses<br><sub>~13 GB · 60 fps with 1.2.0</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/the-great-ace-attorney-chronicles.jpg" width="256" alt="The Great Ace Attorney Chronicles"><br>The Great Ace Attorney Chronicles<br><sub>~4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/ghost-trick-phantom-detective.jpg" width="256" alt="Ghost Trick: Phantom Detective"><br>Ghost Trick: Phantom Detective<br><sub>~3 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/the-great-ace-attorney-chronicles.jpg" width="256" alt="The Great Ace Attorney Chronicles"><br>The Great Ace Attorney Chronicles<br><sub>~4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/hades.jpg" width="256" alt="Hades"><br>Hades<br><sub>~4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/hades-ii.jpg" width="256" alt="Hades II"><br>Hades II<br><sub>~4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/hollow-knight-silksong.jpg" width="256" alt="Hollow Knight: Silksong"><br>Hollow Knight: Silksong<br><sub>~6.4 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/hollow-knight-silksong.jpg" width="256" alt="Hollow Knight: Silksong"><br>Hollow Knight: Silksong<br><sub>~6.4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/hyrule-warriors-age-of-calamity.jpg" width="256" alt="Hyrule Warriors: Age of Calamity"><br>Hyrule Warriors: Age of Calamity<br><sub>~15 GB · 60 fps with 1.3.0</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/luigis-mansion-3.jpg" width="256" alt="Luigi's Mansion 3"><br>Luigi's Mansion 3<br><sub>~7.5 GB · 60 fps with 1.4.0</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/metroid-dread.jpg" width="256" alt="Metroid Dread"><br>Metroid Dread<br><sub>~5 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/metroid-dread.jpg" width="256" alt="Metroid Dread"><br>Metroid Dread<br><sub>~5 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/monster-hunter-rise.jpg" width="256" alt="Monster Hunter Rise"><br>Monster Hunter Rise<br><sub>~22 GB · 60 fps with 16.0.2</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/monster-train-2.jpg" width="256" alt="Monster Train 2"><br>Monster Train 2<br><sub>~1.6 GB · 60 fps with 1.1.1</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/octopath-traveler-0.jpg" width="256" alt="Octopath Traveler 0"><br>Octopath Traveler 0<br><sub>~4 GB · 60 fps with 1.0.5</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/octopath-traveler-0.jpg" width="256" alt="Octopath Traveler 0"><br>Octopath Traveler 0<br><sub>~4 GB · 60 fps with 1.0.5</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/octopath-traveler-ii.jpg" width="256" alt="Octopath Traveler II"><br>Octopath Traveler II<br><sub>~5.9 GB · 60 fps with 1.1.1</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/patricks-parabox.jpg" width="256" alt="Patrick's Parabox"><br>Patrick's Parabox<br><sub>~0.2 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pikmin-4.jpg" width="256" alt="Pikmin 4"><br>Pikmin 4<br><sub>~10.8 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pikmin-4.jpg" width="256" alt="Pikmin 4"><br>Pikmin 4<br><sub>~10.8 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-legends-z-a.jpg" width="256" alt="Pokémon Legends: Z-A"><br>Pokémon Legends: Z-A<br><sub>~6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-sword.jpg" width="256" alt="Pokémon Sword"><br>Pokémon Sword<br><sub>~12.4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/raging-loop.jpg" width="256" alt="Raging Loop"><br>Raging Loop<br><sub>~0.9 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/raging-loop.jpg" width="256" alt="Raging Loop"><br>Raging Loop<br><sub>~0.9 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/rhythm-heaven-groove.jpg" width="256" alt="Rhythm Heaven Groove"><br>Rhythm Heaven Groove<br><sub>~3 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/romance-of-the-three-kingdoms-8-remake.jpg" width="256" alt="ROMANCE OF THE THREE KINGDOMS 8 REMAKE"><br>ROMANCE OF THE THREE KINGDOMS 8 REMAKE<br><sub>~16.8 GB · controller only</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/stardew-valley.jpg" width="256" alt="Stardew Valley"><br>Stardew Valley<br><sub>~0.6 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/stardew-valley.jpg" width="256" alt="Stardew Valley"><br>Stardew Valley<br><sub>~0.6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/steins-gate-reboot.jpg" width="256" alt="STEINS;GATE RE:BOOT"><br>STEINS;GATE RE:BOOT<br><sub>~6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/super-smash-bros-ultimate.jpg" width="256" alt="Super Smash Bros. Ultimate"><br>Super Smash Bros. Ultimate<br><sub>~17.5 GB · offline</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/taiko-risshiden-v-dx.jpg" width="256" alt="Taiko Risshiden V DX"><br>Taiko Risshiden V DX<br><sub>~3.3 GB · Japanese · played by touch</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/taiko-risshiden-v-dx.jpg" width="256" alt="Taiko Risshiden V DX"><br>Taiko Risshiden V DX<br><sub>~3.3 GB · Japanese · played by touch</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/unicorn-overlord.jpg" width="256" alt="Unicorn Overlord"><br>Unicorn Overlord<br><sub>~4 GB</sub></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
# Ghost Trick: Phantom Detective (`010029B018432000`)
|
||||
|
||||
Base game 1.0.0 (v0), SDK 13.3.0, with the Bonus Content add-on (add-on index 1: songs and backgrounds, unlocked at the first boot). Capcom's RE Engine on NVN, Wwise audio, Havok. Modules `main`, `subsdk0` (the SDK's movie library, NintendoSDK_movie 13.3.0) and `sdk`. Plays on the iPhone (IPA 3.0 GB).
|
||||
|
||||
On the Mac, from a fresh save: the opening movies, the language screen, the first-boot notices, save creation, the Bonus Content notices, Chapter 1's opening in the junkyard (the Ghost World, moving between cores, the first trick on the crossing gate, the portraits and dialogue). A second boot loads that save and reaches the title menu. 60 fps (frame interval p99 about 17–19 ms), about 2.3 GB resident and flat after the first scenes, CPU about a quarter of a core (0.3–0.6 W); no red line on that route.
|
||||
|
||||
## What it needs
|
||||
|
||||
The provisional profile's guesses held; the two texture keys were added after the first boot.
|
||||
|
||||
| Setting | Where | Why |
|
||||
|---|---|---|
|
||||
| `tex_flush_upload on`, `tex_dirty_upload on` | [`game.tsv`](game.tsv) | RE Engine writes its UI textures (ASTC) into CPU-visible pools in place, as Monster Hunter Rise ([0100B04011742000](../0100B04011742000/README.md)) does; without the keys the language screen's background and buttons were magenta (all-zero ASTC blocks), and with `tex_flush_upload` alone they still were ([CPU writes with no flush, caught by page protection](../../../docs/reference/nvn-objects.md#cpu-writes-with-no-flush-caught-by-page-protection-tex_dirty_upload)) |
|
||||
| `sys_resource 0x0` | `game.tsv` | the NPDM declares no system resource, so there is no virtual address memory: the game sets a heap of the pool − used memory |
|
||||
| 4 GB layout: `mem_reserve 0x100000000`, `mem_alias 0x10000000`, `mem_stack 0x10000000`, `app_pool 0xCD500000` | `game.tsv` | 256 MB alias, 256 MB stacks, 3.25 GB heap region, as Monster Hunter Rise |
|
||||
| `mem_used 0x87bc000` (135 MB) | `game.tsv` | the rewritten modules are larger than the default 64 MB of used memory ([Heap sized from the application pool less used memory](../../../docs/reference/memory.md#heap-sized-from-the-application-pool-less-used-memory)) |
|
||||
|
||||
No `fps_target`: the game runs at 60 on the Switch. No `pad_layout`: the virtual gamepad starts in the default 2D layout.
|
||||
|
||||
## Files and shaders
|
||||
|
||||
- One RE Engine PAK, `re_chunk_000.pak` (2.66 GB), and `natives/NSW/runtime.args`. `romfs_marker` is `re_chunk_000.pak`.
|
||||
- **Current:** 7,744 shaders from the PAK's SDF definitions and `main` (3,731 pixel, 1,559 vertex, 2,401 compute, 20 geometry, 15 hull, 18 domain) and 8,530 pairs ([SDF definitions in RE Engine PAKs](../../../docs/reference/shaders.md#sdf-definitions-in-re-engine-paks)). 7,739 build; the other 5 are compute shaders that do atomics on a 3D image (`16a78d0fd09ccc18`, `33559ae5667779fe`, `7c65836255e05a87`, `a838b76ccc822be7`, `ecef686222a38548`), which SPIRV-Cross's MSL 2.3 emulation indexes with an `int3` (`array subscript is not an integer`), as Octopath Traveler 0's 16 ([Unreal Engine 5 shader code archives (Oodle)](../../../docs/reference/shaders.md#unreal-engine-5-shader-code-archives-oodle)). None was requested in the runs above (a missing one prints a `[RED] failed to load shader` line).
|
||||
- PR #138 (not merged) built them by flattening the coordinate in `dmsh_build.py` (`x + width × (y + height × z)` into the atomics pointer); that is a change of the chain, so every game's libraries would be rebuilt. It is left out until a run of this game requests one of them.
|
||||
|
||||
## Movies and audio
|
||||
|
||||
- The opening logos are H.264 (1280×720) into an NVN texture with PCM sound, through `subsdk0` ([H.264 into an RGBA texture and AAC through AudioToolbox](../../../docs/reference/movies.md#h264-into-an-rgba-texture-and-aac-through-audiotoolbox)).
|
||||
- Wwise opens one 6-channel AudioOut. A muted run with `DM_AUDIO_DUMP` had sound under the logos; the language screen is silent.
|
||||
|
||||
## Input
|
||||
|
||||
- The pad. In the Ghost World the D-pad (or the left stick) stretches the soul toward a core while held; `Right+Up~3` from Sissel's body reaches the crossing gate in the first scene. A tricks.
|
||||
|
||||
## Mac routes
|
||||
|
||||
- The work folder's `run/boot.sh <log> [secs] VAR=…` runs `build/ghosthost` (a renamed copy of `dmhost`) with the DLC folder; `run/ctl.sh`, `run/step.sh` and `run/see.sh` (a step plus a 640-wide copy of the shot) drive a run started with `DM_CTL`. `run/save_ref` is the save right after the first-boot notices (language English, Bonus Content unlocked, no chapter saved).
|
||||
- From a fresh save the language screen is up about 35 s after the boot: A (English), A (Next), `LRight~0.2` and A (Next on the privacy notice), A (create save data), A (autosave notice), then about nine A for the Bonus Content notices, which start Chapter 1.
|
||||
- In Chapter 1: about 63 A through the opening dialogue, `Right~0.3` and `Right+Up~3` to the crossing gate's core, then A through the dialogue and A for the trick.
|
||||
|
||||
## iPhone
|
||||
|
||||
- Plays (user, IPA 3.0 GB with the autorelease fix). The first IPA (A20 Pro, before that fix) logged the boot, the opening movies and save creation at 60 fps (frame interval p99 about 18 ms), no red line, 4.2 GB of headroom to the system memory limit at frame 1200, and the same freed-but-alive pools as the Mac.
|
||||
|
||||
## Memory
|
||||
|
||||
- **Resolved: 125 to 146 finalized memory pools stayed alive** (`freed but alive` in the memory line: 6.6 to 8.7 GB of zero-copy buffers, so "GPU allocated" read 9.9 to 11.7 GB against 2.2 GB resident). The game frees large pools while it loads its first scenes, and the host's weak-table bookkeeping of freed pools left each buffer in the finalizing thread's autorelease pool, which never drains ([Finalized pool buffers kept by a guest thread's autorelease pool](../../../docs/reference/nvn-objects.md#finalized-pool-buffers-kept-by-a-guest-threads-autorelease-pool)). Now 0, and GPU allocated 2.9 GB in Chapter 1; resident memory unchanged.
|
||||
|
||||
## Found on this game
|
||||
|
||||
General host changes first needed here; each applies to every game:
|
||||
|
||||
- **Autorelease pools around the pool and residency bookkeeping** (`mtl_pool_del`, `res_add_at`, `res_remove`): [Finalized pool buffers kept by a guest thread's autorelease pool](../../../docs/reference/nvn-objects.md#finalized-pool-buffers-kept-by-a-guest-threads-autorelease-pool).
|
||||
|
||||
## Open
|
||||
|
||||
- The chapter autosave was not reached in the runs above (the save files kept their first-boot contents through the junkyard tutorial).
|
||||
@@ -0,0 +1 @@
|
||||
# generated
|
||||
|
@@ -0,0 +1,47 @@
|
||||
# Game profile of Ghost Trick: Phantom Detective (Switch, base game 1.0.0 = v0, with the Bonus Content add-on)
|
||||
# Format: see dm/games/README.md
|
||||
# Capcom's RE Engine on SDK 13.3.0, NVN, Wwise audio; subsdk0 = NintendoSDK_movie 13.3.0 (the opening logos, H.264 into an NVN texture). Everything is in one RE Engine PAK (re_chunk_000.pak);
|
||||
# the shaders are its SDF definitions and GLSLC outputs in main. The NSP script's provisional profile with the two texture keys below. Plays on the iPhone and the Mac (60 fps)
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 010029B018432000
|
||||
program_id 010029B018432000
|
||||
|
||||
# ---- App ----
|
||||
name Ghost Trick: Phantom Detective
|
||||
ipa GhostTrickPhantomDetective.ipa
|
||||
app_category public.app-category.games
|
||||
|
||||
# ---- Versions ----
|
||||
build v0 1.0.0 main=9f52de8549d91cfe840e6d1094f5e3b4 subsdk0=7e9f31f59e001cf5d9b546ffb233f480 sdk=2bf3519216f5b9d3e8ce26863c2e98b6
|
||||
package v0
|
||||
shader_versions v0
|
||||
|
||||
# ---- Files ----
|
||||
# every module main imports from
|
||||
modules main subsdk0 sdk
|
||||
romfs_marker re_chunk_000.pak
|
||||
zip_store .pak:.metallib
|
||||
|
||||
# ---- Memory ----
|
||||
# main sets a heap of at most the pool − 64 MB − 2 MB of bss (counted as used memory): a 3328 MB heap region
|
||||
mem_reserve 0x100000000
|
||||
mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
# the NPDM declares 0 MB of system resource: IsVirtualAddressMemoryEnabled is false, as on Horizon (Big Brain Academy: Brain vs. Brain sets a heap only then)
|
||||
sys_resource 0x0
|
||||
|
||||
# ---- Graphics ----
|
||||
# RE Engine writes its UI textures (ASTC) into CPU-visible pools in place, as Monster Hunter Rise does: without both keys the language screen's background and buttons were magenta
|
||||
# (all-zero ASTC blocks); with tex_flush_upload alone they still were
|
||||
tex_flush_upload on
|
||||
tex_dirty_upload on
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_groups /natives/=natives
|
||||
shader_note Ghost Trick: Phantom Detective shaders
|
||||
|
||||
# ---- After the rewrite ----
|
||||
# the rewritten modules are larger than the default 64 MB of used memory (Age of Calamity: 66.6 MB → 72 MB)
|
||||
mem_used 0x87bc000
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# Prewarm seed list (shipped in the app, next to shader_map.tsv). Rows: S<TAB>shader key<TAB>stage / R<TAB>render pipeline key (pso_skey hex) / C<TAB>compute shader key.
|
||||
# Provisional profile: empty until recorded on the Mac (DM_PSO_MANIFEST=off DM_PSO_RECORD).
|
||||
|
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
# kind: evidence control-section key conflict rulings: empty
|
||||
dm_ctl_sha16 chosen_pc class candidates records
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -65,6 +65,7 @@ One row per profile; the notes are each profile's `README.md`.
|
||||
| [FINAL FANTASY TACTICS - The Ivalice Chronicles](010038B015560000/README.md) | `010038B015560000` | 1.4.0 | 19.3.9 | Square Enix's engine | NVN |
|
||||
| [Fire Emblem Engage](0100A6301214E000/README.md) | `0100A6301214E000` | 2.0.0 | 12.3.7 | Unity 2020.3.18f1 IL2CPP, Wwise | NVN |
|
||||
| [Fire Emblem: Three Houses](010055D009F78000/README.md) | `010055D009F78000` | 1.2.0 | 6.5.0 | Koei Tecmo's KTGL | NVN |
|
||||
| [Ghost Trick: Phantom Detective](010029B018432000/README.md) | `010029B018432000` | 1.0.0 (base) | 13.3.0 | Capcom's RE Engine, Wwise | NVN |
|
||||
| [Hades](0100535012974000/README.md) | `0100535012974000` | 1.0.38233 | 9.3.0 | Supergiant Games' engine (The Forge) | Vulkan |
|
||||
| [Hades II](0100A00019DE0000/README.md) | `0100A00019DE0000` | base game | 19.3.9 | Supergiant Games' engine | NVN |
|
||||
| [Hollow Knight: Silksong](010013C00E930000/README.md) | `010013C00E930000` | 1.0.30000 | 20.5.8 | Unity 6000.0.50f1 IL2CPP | NVN |
|
||||
|
||||
+11
-5
@@ -119,7 +119,9 @@ static NSHashTable *g_res_objects;
|
||||
// Residency churn by call site (the power line every 300 frames, mtl_res_stats): each commit with changes is a kernel call whose cost grows with what was added or removed
|
||||
enum { RS_SITES = 16 }; static struct { int line; uint64_t n, bytes; } g_rs_site[RS_SITES]; static uint64_t g_rs_rm, g_rs_commits, g_rs_commit_ns;
|
||||
#define res_add(r) res_add_at((r), __LINE__)
|
||||
static void res_add_at(id<MTLResource> r, int line) {
|
||||
// Both run on guest threads too (pool_res from mtl_addr, uploads), whose implicit autorelease pools never drain; containsObject: / removeObject: on a weak NSHashTable leave the object
|
||||
// autoreleased in the calling thread's pool (measured in a standalone test: the object outlived its last strong reference until the thread ended), so each has a pool of its own
|
||||
static void res_add_at(id<MTLResource> r, int line) { @autoreleasepool {
|
||||
if (!r) return;
|
||||
pthread_mutex_lock(&g_res_mu);
|
||||
if (!g_res_objects) g_res_objects = [NSHashTable weakObjectsHashTable];
|
||||
@@ -129,8 +131,8 @@ static void res_add_at(id<MTLResource> r, int line) {
|
||||
[g_res_objects addObject:r];
|
||||
g_res_dirty = 1;
|
||||
pthread_mutex_unlock(&g_res_mu);
|
||||
}
|
||||
static void res_remove(id<MTLResource> r) {
|
||||
} }
|
||||
static void res_remove(id<MTLResource> r) { @autoreleasepool {
|
||||
if (!r) return;
|
||||
pthread_mutex_lock(&g_res_mu);
|
||||
if (![g_res_objects containsObject:r]) { pthread_mutex_unlock(&g_res_mu); return; }
|
||||
@@ -138,7 +140,7 @@ static void res_remove(id<MTLResource> r) {
|
||||
[g_res_objects removeObject:r];
|
||||
g_res_dirty = 1; g_rs_rm++;
|
||||
pthread_mutex_unlock(&g_res_mu);
|
||||
}
|
||||
} }
|
||||
// DM_RES_CHECK=1 (diagnostic): if textures / buffers the shader accesses through the argument block are not in the residency set, print the first 40 (textures are attributed to the actual allocation via parent texture / underlying buffer)
|
||||
static int g_res_check_on;
|
||||
static void res_check(id r, const char *what, int reg) {
|
||||
@@ -446,11 +448,15 @@ void mtl_pool_del(const void *mem) {
|
||||
g_reg_gen++;
|
||||
pthread_mutex_lock(&g_mu);
|
||||
// The no-copy buffer goes through deferred release (out of the residency set and unreferenced after submitted commands complete). It used to be removed only from g_keep while the residency set still held a strong reference: after the pool was freed the buffer stayed alive wrapping that guest memory,
|
||||
// and a new pool at the same address wrapped it again. Engage's title demo cycles scenes; GPU allocated grew to 5.4 GB and resident memory to 3.6 GB in 850 s
|
||||
// and a new pool at the same address wrapped it again. Engage's title demo cycles scenes; GPU allocated grew to 5.4 GB and resident memory to 3.6 GB in 850 s.
|
||||
// Its own autorelease pool: this runs on the guest thread that finalizes the pool, whose implicit pool never drains, and adding to the weak table g_dead_pools left the buffer
|
||||
// autoreleased there, so every finalized pool buffer stayed alive for good (Ghost Trick: Phantom Detective, 125 buffers and 6.6 GB of GPU allocations 300 frames into the boot, 0 with the pool)
|
||||
@autoreleasepool {
|
||||
for (int i = 0; i < g_npool; i++)
|
||||
if (g_pools[i].key == mem) { id<MTLBuffer> b = g_pools[i].buf; [g_keep removeObject:b]; defer_obj_release(b); if (g_pools[i].res) g_pool_res_n--;
|
||||
if (!g_dead_pools) g_dead_pools = [NSHashTable weakObjectsHashTable];
|
||||
[g_dead_pools addObject:b]; g_pools[i] = g_pools[--g_npool]; g_pidx_ok = 0; g_plast = -1; atomic_fetch_add(&g_pool_gen, 1); break; }
|
||||
}
|
||||
pthread_mutex_unlock(&g_mu);
|
||||
}
|
||||
|
||||
|
||||
@@ -223,6 +223,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
|
||||
- `[RED] MapVirtual: non-contiguous mapping (true sparse) not implemented`. → [Sparse virtual pools built from separate physical pools](../reference/nvn-objects.md#sparse-virtual-pools-built-from-separate-physical-pools)
|
||||
- `CRASH sig=10` in `memmove` from `pkg_upload` → `nvn_deswizzle`, at an address 32 GB into a sparse virtual pool; textures with unaliased tails came out magenta. → [Textures read through a 4 KB page table of a sparse pool](../reference/nvn-objects.md#textures-read-through-a-4-kb-page-table-of-a-sparse-pool)
|
||||
- GPU memory grows with every scene change (Engage's idle title demo reached 5.4 GB of GPU memory and 3.6 GB resident in 850 s). → [Finalized pools kept alive by residency and the sharing table](../reference/nvn-objects.md#finalized-pools-kept-alive-by-residency-and-the-sharing-table)
|
||||
- the memory line's `freed but alive` count only grows, while no host table holds those buffers; "GPU allocated" is far above resident memory (Ghost Trick: Phantom Detective, 125 to 146 buffers and 6.6 to 8.7 GB of GPU allocations against 2.2 GB resident). → [Finalized pool buffers kept by a guest thread's autorelease pool](../reference/nvn-objects.md#finalized-pool-buffers-kept-by-a-guest-threads-autorelease-pool)
|
||||
- the serial table grows by about 10 entries a frame (276,000 after 25,000 frames). → [Dead texture serials kept in a ghost table](../reference/nvn-objects.md#dead-texture-serials-kept-in-a-ghost-table)
|
||||
- `FATAL NVN command memory: the game's pool is empty…`. → [Host-lent command memory segments](../reference/nvn-objects.md#host-lent-command-memory-segments)
|
||||
- frames stop after about 15 s; a game thread spins in its command-memory callback (a ring allocator at `main+0x9cc120` waiting for fences that have all signalled). → [What `GetCommandMemoryUsed` counts](../reference/nvn-objects.md#what-getcommandmemoryused-counts)
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 153 KiB |
@@ -81,6 +81,13 @@ The game may also write into an NVN object directly, without an NVN call, throug
|
||||
- **Found on:** [Fire Emblem Engage](../../dm/games/0100A6301214E000/README.md): it now stays at 1.2 to 1.4 GB.
|
||||
- **Note:** the `memory up to frame` line also reports `serial objects` (host objects by serial), `freed but alive` (finalized pool buffers still alive) and `shared textures` (sharing table size and evictions); growth in GPU memory with those counts flat points to an allocation outside them.
|
||||
|
||||
### Finalized pool buffers kept by a guest thread's autorelease pool
|
||||
|
||||
- **Symptom:** the memory line's `freed but alive` count only grows, while no host table holds those buffers; "GPU allocated" is far above resident memory (Ghost Trick: Phantom Detective, 125 to 146 buffers and 6.6 to 8.7 GB of GPU allocations against 2.2 GB resident).
|
||||
- **Cause:** `mtl_pool_del` runs on the guest thread that finalizes the pool, whose implicit autorelease pool never drains, and adding the buffer to the weak table `g_dead_pools` (the table behind `freed but alive`) left it autoreleased there: every finalized pool's zero-copy buffer stayed alive for good, wrapping freed guest memory. A temporary census of every host table that can hold a texture over a pool buffer (by serial, finalized serials, the sharing table, texture ID views, view caches, the deferred queue, the residency set) found no holder at all. `containsObject:` and `removeObject:` on a weak `NSHashTable` do the same (a standalone test: the object outlived its last strong reference until its thread ended), and `res_add` / `res_remove` call them on guest threads too.
|
||||
- **Fix:** `mtl_pool_del`, `res_add_at` and `res_remove` (`nvn_mtl.m`) each have their own `@autoreleasepool`, as `defer_drain` and the memory line's count already did ([Deferred-release drain without an autorelease pool](memory.md#deferred-release-drain-without-an-autorelease-pool)).
|
||||
- **Found on:** [Ghost Trick: Phantom Detective](../../dm/games/010029B018432000/README.md): 0 buffers freed but alive with the fix; GPU allocated 1.7 GB instead of 8.3 GB 300 frames into the boot and 2.9 GB instead of 9.9 to 11.7 GB in Chapter 1, resident memory, frame times and pictures unchanged (zero-copy buffers add no memory of their own). [Pokémon Sword](../../dm/games/0100ABF008968000/README.md): 8 to 0 at the title after 90 s. Earlier Mac logs show the count in most games, growing with play: Animal Crossing: New Horizons 3,113 after 27,900 frames, Pokémon Sword 2,050, Super Smash Bros. Ultimate 1,279, Pokémon Legends: Z-A 932 (small pools there: 100 to 300 MB).
|
||||
|
||||
### Memory-pool tracking table grows in blocks
|
||||
|
||||
- **Cause:** the memory-pool tracking table had a fixed 4,096 slots; a pool past the end would have its textures re-uploaded at every use.
|
||||
|
||||
Reference in new issue
Block a user