Port Kirby and the Forgotten Land 1.1.0
HAL Laboratory's engine on NintendoWare, SDK 11.4.0, NVN. Plays on the Mac at 30 fps, as on the Switch: online and difficulty prompts, the 1080p opening movie, Point of Arrival with its guide movies, the city reveal, the Sword ability and fights, title and Continue; a 15-minute soak with no crash. No game code is cut; the profile is the memory layout and the shader groups. All 27,297 shaders build for Mac and iOS; the IPA (6.4 GB) is untested on a device. - Threads (general): a guest thread waiting for a host lock that HLE code holds while it runs guest code is blocked, not runnable (dm_host_lock, a native HLE wait). The movie decoder held its decode lock across SignalEvent; the signal's same-core handoff waited for the movie input thread, which waited for that lock, and the game froze at its first guide movie. t_preempt has the case. Docs: the game note, games table, README gallery, threads.md, shaders.md, symptom index; AGENTS count 73 -> 74. Checked: host tests, preempt, tool tests (253, 15 skips), docs tests; a Mac run on chain 5cddc47253ab through the first guide movie.
This commit is contained in:
seedds authored and MacMini committed
1 parent
ff44c59e12
commit
ed3007f1ef
19 files changed
+48949
-24
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Notes for agents
|
||||
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it (`dm/tools/patch_nso.py`), on a host that answers Horizon's SVCs and the SDK's services (`dm/host`) and implements NVN, Vulkan and NVIDIA's desktop OpenGL on Metal (`dm/gfx/nvn`, `dm/gfx/vk`, `dm/gfx/gl`). Seventy-three games are playable, each tested on the Mac or the iPhone (a Mac gameplay test is enough to count); the [games table](dm/games/README.md#games) lists each with its build, SDK, engine and graphics API, and links its note, `dm/games/<id>/README.md`: what its profile needs and why, what was found on it, how to drive it on the Mac, what is open. [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and power.
|
||||
- This repository runs Nintendo Switch games natively on Apple platforms: the game's own code runs, rewritten only where Apple's kernel requires it (`dm/tools/patch_nso.py`), on a host that answers Horizon's SVCs and the SDK's services (`dm/host`) and implements NVN, Vulkan and NVIDIA's desktop OpenGL on Metal (`dm/gfx/nvn`, `dm/gfx/vk`, `dm/gfx/gl`). Seventy-four games are playable, each tested on the Mac or the iPhone (a Mac gameplay test is enough to count); the [games table](dm/games/README.md#games) lists each with its build, SDK, engine and graphics API, and links its note, `dm/games/<id>/README.md`: what its profile needs and why, what was found on it, how to drive it on the Mac, what is open. [README.md](README.md) is the player-facing page (games, packing, playing); [docs/development.md](docs/development.md) has the internals, the script's options, building by hand, prewarm lists and power.
|
||||
- Where to start:
|
||||
|
||||
| Task | Read first |
|
||||
|
||||
@@ -104,50 +104,51 @@ Under each game: the IPA's size, and "60 fps with 1.2.0" where the game runs at
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/hyrule-warriors-age-of-calamity.jpg" width="256" alt="Hyrule Warriors: Age of Calamity"><br>Hyrule Warriors: Age of Calamity<br><sub>~15 GB · 60 fps with 1.3.0</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/kirby-and-the-forgotten-land.jpg" width="256" alt="Kirby and the Forgotten Land"><br>Kirby and the Forgotten Land<br><sub>~6.4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/luigis-mansion-3.jpg" width="256" alt="Luigi's Mansion 3"><br>Luigi's Mansion 3<br><sub>~7.5 GB · 60 fps with 1.4.0</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/metroid-dread.jpg" width="256" alt="Metroid Dread"><br>Metroid Dread<br><sub>~5 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/metroid-dread.jpg" width="256" alt="Metroid Dread"><br>Metroid Dread<br><sub>~5 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/monster-hunter-rise.jpg" width="256" alt="Monster Hunter Rise"><br>Monster Hunter Rise<br><sub>~22 GB · 60 fps with 16.0.2</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/monster-train-2.jpg" width="256" alt="Monster Train 2"><br>Monster Train 2<br><sub>~1.6 GB · 60 fps with 1.1.1</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/octopath-traveler.jpg" width="256" alt="OCTOPATH TRAVELER"><br>OCTOPATH TRAVELER<br><sub>~3.6 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/octopath-traveler.jpg" width="256" alt="OCTOPATH TRAVELER"><br>OCTOPATH TRAVELER<br><sub>~3.6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/octopath-traveler-0.jpg" width="256" alt="Octopath Traveler 0"><br>Octopath Traveler 0<br><sub>~4 GB · 60 fps with 1.0.5</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/octopath-traveler-ii.jpg" width="256" alt="Octopath Traveler II"><br>Octopath Traveler II<br><sub>~5.9 GB · 60 fps with 1.1.1</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/patricks-parabox.jpg" width="256" alt="Patrick's Parabox"><br>Patrick's Parabox<br><sub>~0.2 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/patricks-parabox.jpg" width="256" alt="Patrick's Parabox"><br>Patrick's Parabox<br><sub>~0.2 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/persona-5-royal.jpg" width="256" alt="Persona 5 Royal"><br>Persona 5 Royal<br><sub>~14.9 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pikmin-4.jpg" width="256" alt="Pikmin 4"><br>Pikmin 4<br><sub>~10.8 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-brilliant-diamond.jpg" width="256" alt="Pokémon Brilliant Diamond"><br>Pokémon Brilliant Diamond<br><sub>~4.6 GB · 60 fps with 1.3.0</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-brilliant-diamond.jpg" width="256" alt="Pokémon Brilliant Diamond"><br>Pokémon Brilliant Diamond<br><sub>~4.6 GB · 60 fps with 1.3.0</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-legends-z-a.jpg" width="256" alt="Pokémon Legends: Z-A"><br>Pokémon Legends: Z-A<br><sub>~6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-sword.jpg" width="256" alt="Pokémon Sword"><br>Pokémon Sword<br><sub>~12.4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-shield.jpg" width="256" alt="Pokémon Shield"><br>Pokémon Shield<br><sub>~12.4 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-shield.jpg" width="256" alt="Pokémon Shield"><br>Pokémon Shield<br><sub>~12.4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/portal.jpg" width="256" alt="Portal"><br>Portal<br><sub>~2.6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/raging-loop.jpg" width="256" alt="Raging Loop"><br>Raging Loop<br><sub>~0.9 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/rhythm-heaven-groove.jpg" width="256" alt="Rhythm Heaven Groove"><br>Rhythm Heaven Groove<br><sub>~3 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/rhythm-heaven-groove.jpg" width="256" alt="Rhythm Heaven Groove"><br>Rhythm Heaven Groove<br><sub>~3 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/romance-of-the-three-kingdoms-8-remake.jpg" width="256" alt="ROMANCE OF THE THREE KINGDOMS 8 REMAKE"><br>ROMANCE OF THE THREE KINGDOMS 8 REMAKE<br><sub>~16.8 GB · controller only</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/sea-of-stars.jpg" width="256" alt="Sea of Stars"><br>Sea of Stars<br><sub>~5.1 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/shin-megami-tensei-v-vengeance.jpg" width="256" alt="SHIN MEGAMI TENSEI V: Vengeance"><br>SHIN MEGAMI TENSEI V: Vengeance<br><sub>~14.6 GB · 60 fps with 1.0.3</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/shin-megami-tensei-v-vengeance.jpg" width="256" alt="SHIN MEGAMI TENSEI V: Vengeance"><br>SHIN MEGAMI TENSEI V: Vengeance<br><sub>~14.6 GB · 60 fps with 1.0.3</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/stardew-valley.jpg" width="256" alt="Stardew Valley"><br>Stardew Valley<br><sub>~0.6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/steins-gate-reboot.jpg" width="256" alt="STEINS;GATE RE:BOOT"><br>STEINS;GATE RE:BOOT<br><sub>~6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/super-smash-bros-ultimate.jpg" width="256" alt="Super Smash Bros. Ultimate"><br>Super Smash Bros. Ultimate<br><sub>~17.5 GB · offline</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/super-smash-bros-ultimate.jpg" width="256" alt="Super Smash Bros. Ultimate"><br>Super Smash Bros. Ultimate<br><sub>~17.5 GB · offline</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/taiko-risshiden-v-dx.jpg" width="256" alt="Taiko Risshiden V DX"><br>Taiko Risshiden V DX<br><sub>~3.3 GB · Japanese · played by touch</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/unicorn-overlord.jpg" width="256" alt="Unicorn Overlord"><br>Unicorn Overlord<br><sub>~4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/nier-automata.png" width="256" alt="NieR:Automata The End of YoRHa Edition"><br>NieR:Automata The End of YoRHa Edition<br><sub>~10.9 GB · 60 fps with 1.0.3</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/nier-automata.png" width="256" alt="NieR:Automata The End of YoRHa Edition"><br>NieR:Automata The End of YoRHa Edition<br><sub>~10.9 GB · 60 fps with 1.0.3</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/zelda-echoes-of-wisdom.jpg" width="256" alt="The Legend of Zelda: Echoes of Wisdom"><br>The Legend of Zelda: Echoes of Wisdom<br><sub>~7.8 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/donkey-kong-country-returns-hd.png" width="256" alt="Donkey Kong Country Returns HD"><br>Donkey Kong Country Returns HD<br><sub>~11.6 GB</sub></td>
|
||||
</tr>
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# Kirby and the Forgotten Land (`01004D300C5AE000`)
|
||||
|
||||
Update 1.1.0 (v65536) over the base game, SDK 11.4.0. HAL Laboratory's own engine on NintendoWare (G3d, Font, Fxt) with `nn::atk2` sound and NEX online. Modules `main`, `subsdk0` (NintendoSDK_movie 11.4.0) and `sdk`; no other profile ships these `sdk` and `subsdk0` builds. Plays on the Mac at 30 fps, as on the Switch: the online and difficulty prompts, the 1080p opening movie, the beach and the walk through Point of Arrival with its guide movies, the reveal of the city, inhaling a Blade Knight for the Sword ability, fights with Awoofies and Bronto Burts, the title screen and continuing a file. An unsigned IPA is built; iOS is untested.
|
||||
|
||||
## What it needs
|
||||
|
||||
| Setting | Where | Why |
|
||||
|---|---|---|
|
||||
| 4 GB block: `mem_reserve 0x100000000`, `mem_alias 0x10000000`, `mem_stack 0x10000000`, `app_pool 0xCD500000` | [`game.tsv`](game.tsv) | no system resource; 256 MB in front + 256 MB alias region + 256 MB stack region + 3.25 GB heap region. See [Memory](#memory) |
|
||||
| `modules main subsdk0 sdk` | `game.tsv` | `subsdk0` is the movie library: the opening and ending movies and the guide movies |
|
||||
| `shader_groups /gfx/=gfx /ptcl/=ptcl /lyt/=lyt /shader/=shader` | `game.tsv` | the shaders are BNSH archives in HAL's `.cmp` containers; see [Files and shaders](#files-and-shaders) |
|
||||
|
||||
No code of the game is cut, and no run-time key is set. The game asks for present interval 1 and paces itself at 30 fps.
|
||||
|
||||
## Memory
|
||||
|
||||
- The NPDM declares no system resource, so there is no virtual address memory: the game sets a heap of the pool less the used memory, 3.14 GB (`[mem] heap 3.14 GB (heap region 3.25 GB)`), which the 3.25 GB heap region holds.
|
||||
- **Current:** 4.1 GB resident and 2.67 GB of Metal allocations on the Mac after 20 minutes in Point of Arrival (`footprint`: 2.6 GB untagged, the guest memory the game touches; 0.7 GB graphics; 0.6 GB large host mallocs). It grew with each new area (3.2 GB at the beach, 3.97 GB at the city) and then stayed nearly flat, 4.13 → 4.17 GB with Metal allocations steady at 2.67 GB, over a 15-minute soak in one area. The highest resident figure of the games so far; how much a phone allows is open.
|
||||
|
||||
## Files and shaders
|
||||
|
||||
- RomFS: 9,386 files, 5.72 GB. 7,201 `.cmp` and 430 `.cmpbin` containers (HAL's size-prefixed zstd and LZ4 frames), 52 `.bfstm` and 104 `.opus` streams, 36 WebM movies (`mov/EventMovie`, `mov/GuideMovie`).
|
||||
- 27,297 shaders in the map (18,460 pixel, 8,755 vertex, 72 compute, 10 geometry): 18,476 in `gfx/`, 8,313 in `ptcl/` (70 of them compute), 387 in `lyt/`, 67 in the loose `shader/` files and 54 in BNSH archives embedded in `main`. Before the split opened the `.cmp` containers it found 121 ([HAL's `.cmp` containers](../../../docs/reference/shaders.md#hals-cmp-containers-a-size-prefixed-frame)). All of them built for the Mac without a failure, and no run met a shader missing from the map (`shader keys not in the shader map … 0`).
|
||||
|
||||
## Rendering
|
||||
|
||||
- **Current, harmless:** `[RED] texture type mismatch: shader … declares slot 1 / 2 as Metal type 2, texture ID … is type 7 (NVN target 2, format 0x38 …); bound as is`, once per shader, for eight UI layout pixel shaders so far (`lyt/TitleMenu`, `lyt/Resident/TitleLogo`, `lyt/Cmn/HeroPanel`, `lyt/Step/ScopeCine`, `lyt/Arena/Entry`). The texture is a 16×16×16 RGBA8 sRGB 3D texture, one of the colour-grading lookup tables (`texlist 2`), left in slots the layout material does not use: the layout shader samples slots 1 and 2 only when its uniform `c3[0].y` is non-zero.
|
||||
|
||||
## Movies
|
||||
|
||||
- The SDK's demuxer (libwebm in `subsdk0`) runs natively; the host decodes VP9 with libvpx and Vorbis with stb_vorbis into the game's CPU buffers (NV12, mode 1). The opening movie is 1920×1080 VP9 with a Vorbis track (2,297 frames); the guide movies that show a move in a corner of the screen are 480×270 VP9 without sound.
|
||||
- **Resolved: the game froze at the first guide movie** ("A Jump", on the first walk from the beach, every time). A job worker (priority 46) returning an output buffer signalled the movie's events while the host decoder held its decode lock; the signal handed core 0 to the movie threads (priority 44), and the input thread among them was waiting for that lock in host code, which the handoff counted as running ([A host lock held across a same-core handoff](../../../docs/reference/threads.md#a-host-lock-held-across-a-same-core-handoff)).
|
||||
|
||||
## Frame rate
|
||||
|
||||
- **Current:** 30 fps on the Mac, as on the Switch (`power` lines: 28–30 fps; frame interval p99 33.6 ms in play). The game sets present interval 1 and paces its frames itself. CPU about one core and 5–5.6 W on an Apple M6 in Point of Arrival.
|
||||
|
||||
## Audio
|
||||
|
||||
- `nn::atk2` through an audio renderer (96 voices, 14 effects, one sink). 131,543 of the first 132,000 rendered frames were not silent.
|
||||
|
||||
## Mac routes
|
||||
|
||||
- Build with `scripts/build_ipa_from_nsp.py <NSP folder> -k <prod.keys> --platform mac`; its last line runs the game.
|
||||
- A fresh save, through `DM_CTL`: the online prompt is up about 20 s after boot: `Left`, `A` (Do Not Use), `A`; the difficulty: `Right`, `A` (Spring-Breeze Mode). The game saves here, then plays the opening movie and the beach scene (about 75 s) and gives control on the beach.
|
||||
- With that save, the title shows Continue about 25 s after boot; `A` puts Kirby on the beach. Forward (`LUp`) leads to the first guide movie in about 15 s; the path then turns right past a rock, crosses a ravine under a plank (float up on the right with `A`) and runs forward between trees to the city reveal, then right past the building's base to the first Blade Knight.
|
||||
- A `DM_CTL` press replaces the one being held: send the next key only after the hold ends.
|
||||
|
||||
## Testing
|
||||
|
||||
- 2026-10-26, Mac: boot, the online and difficulty prompts, the opening movie, the beach, Point of Arrival with three guide movies, the city reveal, inhaling a Blade Knight (Sword), fighting with the Sword, losing it to damage, the title screen and Continue, over five runs (the longest 30 minutes, with a 15-minute soak of walking, jumping, floating, inhaling and attacking, no crash). The only red lines were the layout shaders' [texture type mismatches](#rendering); a key dump of the title menu's shader (`DM_NVN_KEYDUMP=5de6e5dc52a56300`) shows `c3[0].y` = 0, so its reads of slots 1 and 2 do not run.
|
||||
|
||||
## Open
|
||||
|
||||
- The rest of the game is untested on the Mac: no stage was finished, so no Waddle Dee Town, boss or Mouthful Mode yet, and no save at a stage's end.
|
||||
- iOS: the unsigned IPA (`KirbyAndTheForgottenLand.ipa`, 6.4 GB, 6,846,727,151 bytes, with the fps counter; 27,297 iOS shader libraries, no pipeline archive) is untested on a device; its memory (4.1 GB resident on the Mac) may be the first limit there, above the 3.4 GB iOS gives an app without the Increased Memory Limit capability.
|
||||
|
||||
## Found on this game
|
||||
|
||||
- [HAL's `.cmp` containers: a size-prefixed frame](../../../docs/reference/shaders.md#hals-cmp-containers-a-size-prefixed-frame).
|
||||
- [A host lock held across a same-core handoff](../../../docs/reference/threads.md#a-host-lock-held-across-a-same-core-handoff).
|
||||
@@ -0,0 +1 @@
|
||||
# generated
|
||||
|
@@ -0,0 +1,39 @@
|
||||
# Game profile of Kirby and the Forgotten Land (Switch, update 1.1.0 = v65536). Plays on the Mac
|
||||
# Format: see dm/games/README.md
|
||||
# HAL Laboratory's own engine on NintendoWare (G3d, Font, Fxt) and nn::atk2 sound, SDK 11.4.0, NVN; subsdk0 is the movie module (VP9 / Vorbis WebM cinematics and guide movies).
|
||||
# The shaders are BNSH archives inside HAL's .cmp containers under gfx/, ptcl/ and lyt/, in the loose shader/ files and embedded in main (shader_split.py); no run met a shader missing from the map.
|
||||
# The game asks for present interval 1 and paces itself at 30 fps, as on the Switch
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 01004D300C5AE000
|
||||
program_id 01004D300C5AE000
|
||||
|
||||
# ---- App ----
|
||||
# Game name: only a label for profile listings and the Mac window title; the iOS app name is the game name in the base NSP, and the bundle ID comes from the base game title ID (build_ipa.sh / NSP script)
|
||||
name Kirby and the Forgotten Land
|
||||
ipa KirbyAndTheForgottenLand.ipa
|
||||
app_category public.app-category.games
|
||||
|
||||
# ---- Versions ----
|
||||
build v65536 1.1.0 main=a6ce40dc3aedb1becacd2f1ff4515307 subsdk0=535249a546887cac875a099e9c8ef367 sdk=b39add63421be525d352e576dab438d3
|
||||
package v65536
|
||||
shader_versions v65536
|
||||
|
||||
# ---- Files ----
|
||||
# load order as on Horizon: main, subsdk*, sdk (subsdk0 = NintendoSDK_movie 11.4.0)
|
||||
modules main subsdk0 sdk
|
||||
romfs_marker mov/EventMovie/Ending.webm
|
||||
zip_store .bfstm:.cmpbin:.opus:.webm:.metallib
|
||||
|
||||
# ---- Memory ----
|
||||
# The NPDM declares no system resource, so there is no virtual address memory: the game sets a heap of the pool less 64 MB of used memory (3.14 GB, measured), which fits the
|
||||
# 3.25 GB heap region of this 4 GB block (256 MB in front + 256 MB alias region + 256 MB stack region + heap region). The application pool is the Switch's 3285 MB
|
||||
mem_reserve 0x100000000
|
||||
mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
sys_resource 0x0
|
||||
|
||||
# ---- Shader split (shader_split.py; BNSH in HAL's .cmp containers, the loose shader/ files and main) ----
|
||||
shader_groups /gfx/=gfx /ptcl/=ptcl /lyt/=lyt /shader/=shader
|
||||
shader_note Kirby and the Forgotten Land shaders
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# Prewarm seed list (shipped in the app, next to shader_map.tsv). Rows: S<TAB>shader key<TAB>stage / R<TAB>render pipeline key (pso_skey hex) / C<TAB>compute shader key.
|
||||
# Provisional profile: empty until recorded on the Mac (DM_PSO_MANIFEST=off DM_PSO_RECORD).
|
||||
|
@@ -0,0 +1 @@
|
||||
# kind: evidence vertex / pixel shader pairs of agl program variations (BAHS) that shader_pairs.tsv does not hold, for the pipeline prebuild only (generated by dm/gfx/tools/shader_split.py, do not edit): vertex key <TAB> pixel key
|
||||
|
@@ -0,0 +1,73 @@
|
||||
# Answer table for translation queries (generated by dm/gfx/tools/shader_split.py, do not edit): key <TAB> fact JSON. This game has no PC version to pair with: only the thread group / shared memory of compute shaders (taken from the control section); other queries get the translator defaults
|
||||
03a154159cd9ddc3 {"local":[32,1,1],"shared":1152}
|
||||
0548816462332941 {"local":[32,1,1],"shared":0}
|
||||
0590a331e16c36f3 {"local":[32,1,1],"shared":640}
|
||||
074ed7f54681854b {"local":[32,1,1],"shared":640}
|
||||
0b3cf9712884aedd {"local":[32,1,1],"shared":0}
|
||||
0bb3a1724563783e {"local":[32,1,1],"shared":1152}
|
||||
15a1a79d8dc12bee {"local":[32,1,1],"shared":1152}
|
||||
1853ff7127111f47 {"local":[32,1,1],"shared":640}
|
||||
227f5eb5b2da3f16 {"local":[32,1,1],"shared":640}
|
||||
2903256916f6ef8e {"local":[32,1,1],"shared":640}
|
||||
2b06468b00d13c2e {"local":[32,1,1],"shared":0}
|
||||
38b56df9f4ae76ef {"local":[32,1,1],"shared":0}
|
||||
3ca20bdbefc67cd9 {"local":[32,1,1],"shared":640}
|
||||
3eb1e07b5de80718 {"local":[32,1,1],"shared":0}
|
||||
3f25add9dd386b70 {"local":[32,1,1],"shared":640}
|
||||
42b2ff1c920a542f {"local":[32,1,1],"shared":1664}
|
||||
49a79129946bf8f3 {"local":[32,1,1],"shared":640}
|
||||
5449caa443281fc8 {"local":[32,1,1],"shared":0}
|
||||
5469cb87afab6ec8 {"local":[32,1,1],"shared":640}
|
||||
5974019f6d5d98ce {"local":[32,1,1],"shared":1664}
|
||||
5bd12731b5720968 {"local":[32,1,1],"shared":1152}
|
||||
5eedafb8835f5384 {"local":[32,1,1],"shared":1152}
|
||||
606492f2659f43cd {"local":[32,1,1],"shared":1152}
|
||||
63ae5e8c010a4bc7 {"local":[32,1,1],"shared":0}
|
||||
6640c87661b31e13 {"local":[32,1,1],"shared":896}
|
||||
6d83c6b663c18d19 {"local":[32,1,1],"shared":640}
|
||||
6fe2311b67d2c198 {"local":[32,1,1],"shared":0}
|
||||
71f081c216c56798 {"local":[32,1,1],"shared":896}
|
||||
75d37f9a651fd6ed {"local":[32,1,1],"shared":640}
|
||||
776175a24eb1d5d2 {"local":[32,1,1],"shared":1152}
|
||||
81fe01b43ac15d22 {"local":[32,1,1],"shared":640}
|
||||
8224e89f90221ee0 {"local":[32,1,1],"shared":640}
|
||||
85921ccc8e3d0411 {"local":[32,1,1],"shared":1664}
|
||||
874c3badc9c3fc9c {"local":[32,1,1],"shared":1152}
|
||||
89456f159e1b31eb {"local":[32,1,1],"shared":1664}
|
||||
8e7fb9b21aaae837 {"local":[32,1,1],"shared":1152}
|
||||
8fe07f235ec04cc7 {"local":[32,1,1],"shared":640}
|
||||
90b1a8d6f6b6e7f2 {"local":[32,1,1],"shared":1152}
|
||||
92b0050e6fcc140c {"local":[32,1,1],"shared":640}
|
||||
97f519d00cc5b13d {"local":[32,1,1],"shared":640}
|
||||
99af0e77f5f33211 {"local":[32,1,1],"shared":1664}
|
||||
9cf3726b20c5353f {"local":[32,1,1],"shared":640}
|
||||
9fd277015b68a55d {"local":[8,5,1],"shared":0}
|
||||
a38cf5be425d9dac {"local":[32,1,1],"shared":640}
|
||||
a42eb4f5b694577b {"local":[32,1,1],"shared":1152}
|
||||
a5670ea977551354 {"local":[32,1,1],"shared":640}
|
||||
a9d63eded3662d63 {"local":[32,1,1],"shared":640}
|
||||
aacde8d8a7029d2b {"local":[32,1,1],"shared":0}
|
||||
b0e641f93378468f {"local":[32,1,1],"shared":0}
|
||||
b3b4182c3b8ec5ce {"local":[32,1,1],"shared":0}
|
||||
ba56c432c8b8cd9f {"local":[32,1,1],"shared":640}
|
||||
bbe75ebd4b48031c {"local":[32,1,1],"shared":1152}
|
||||
bea7594dee85c37e {"local":[8,5,4],"shared":0}
|
||||
c24c52032cdfbac6 {"local":[32,1,1],"shared":1152}
|
||||
c6c99351b648c1fc {"local":[32,1,1],"shared":0}
|
||||
c9e3ac35cf7aba71 {"local":[32,1,1],"shared":1152}
|
||||
ccce6225dba1e875 {"local":[32,1,1],"shared":0}
|
||||
d4bf6478b0884f21 {"local":[32,1,1],"shared":1408}
|
||||
d4d1d21be8560aca {"local":[32,1,1],"shared":0}
|
||||
d5e78ff9d0709952 {"local":[32,1,1],"shared":640}
|
||||
d618146fe673f9b5 {"local":[32,1,1],"shared":640}
|
||||
d8b43995cd963bdd {"local":[32,1,1],"shared":640}
|
||||
df7e62a7b5fe9adc {"local":[32,1,1],"shared":1152}
|
||||
e40ff6b43b861c4c {"local":[32,1,1],"shared":1664}
|
||||
e811d84fa52627a4 {"local":[32,1,1],"shared":640}
|
||||
e873937454fbfeda {"local":[32,1,1],"shared":0}
|
||||
e9093a151f9e26ae {"local":[32,1,1],"shared":640}
|
||||
ea68d447c57bdc6f {"local":[32,1,1],"shared":1152}
|
||||
f07ef11671f06077 {"local":[32,1,1],"shared":1152}
|
||||
f24c15eaa8d01cb5 {"local":[32,1,1],"shared":0}
|
||||
f60dee6271aa1633 {"local":[32,1,1],"shared":1664}
|
||||
fadcc38f6e42f111 {"local":[32,1,1],"shared":0}
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 19.
|
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
# kind: evidence control-section key conflict rulings: empty
|
||||
dm_ctl_sha16 chosen_pc class candidates records
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -74,6 +74,7 @@ One row per profile; the notes are each profile's `README.md`.
|
||||
| [Hades II](0100A00019DE0000/README.md) | `0100A00019DE0000` | base game | 19.3.9 | Supergiant Games' engine | NVN |
|
||||
| [Hollow Knight: Silksong](010013C00E930000/README.md) | `010013C00E930000` | 1.0.30000 | 20.5.8 | Unity 6000.0.50f1 IL2CPP | NVN |
|
||||
| [Hyrule Warriors: Age of Calamity](01002B00111A2000/README.md) | `01002B00111A2000` | 1.3.0 | 9.6.0 | Koei Tecmo's KTGL | NVN |
|
||||
| [Kirby and the Forgotten Land](01004D300C5AE000/README.md) | `01004D300C5AE000` | 1.1.0 | 11.4.0 | HAL Laboratory's engine (NintendoWare) | NVN |
|
||||
| [Luigi's Mansion 3](0100DCA0064A6000/README.md) | `0100DCA0064A6000` | 1.4.0 | 7.3.2 | Next Level Games' engine | NVN |
|
||||
| [Mario Kart 8 Deluxe](0100152000022000/README.md) | `0100152000022000` | 4.0.0 | 4.4.0 | Nintendo (sead / agl / gsys, NintendoWare, Pia) | NVN |
|
||||
| [Mario vs. Donkey Kong](0100B99019412000/README.md) | `0100B99019412000` | 1.0.1 (cartridge base) | 15.3.0 | Nintendo (sead / agl / gsys) | NVN |
|
||||
|
||||
+11
-8
@@ -137,12 +137,15 @@ static void md_event(mdec_t *d, int i) { // 0 input 1 output 2 format 3 error
|
||||
// and only later links the holders into its MultiWait. Signalled synchronously inside Start, the holders were not linked yet and SignalEvent crashed in MultiWaitImpl::NotifyAndWakeupThread on a null MultiWait (0x20)
|
||||
// (it showed only when a decoder was built a second time in the same memory). A guest thread now signals it about 2 ms later (SignalEvent is guest code and must be called on a guest thread); the destructor drops that decoder's pending signals under the same lock.
|
||||
// The movie player's dispatcher and worker must also finish their requests before the posting thread continues; the shared SVC wake handoff supplies that ordering (hle_thread.c).
|
||||
// g_dfmu and each decoder's dmu are held while guest code signals events, so guest threads take them with dm_host_lock: a thread waiting for one counts as blocked, and the
|
||||
// signaller's handoff does not wait for it (Kirby and the Forgotten Land froze at its first guide movie: the handoff waited for the input thread, which waited for dmu)
|
||||
void *dm_guest_thread_spawn(void (*fn)(void *), void *arg, int prio, size_t stack_size);
|
||||
void dm_host_lock(pthread_mutex_t *m); // hle_thread.c
|
||||
static pthread_mutex_t g_dfmu = PTHREAD_MUTEX_INITIALIZER; static pthread_cond_t g_dfcv = PTHREAD_COND_INITIALIZER;
|
||||
static struct { void *d; int ev; uint64_t due; } g_dfq[64]; static int g_ndfq, g_dfthr;
|
||||
static void md_defer_body(void *arg);
|
||||
static void md_defer_at(void *d, int ev, uint64_t due) {
|
||||
pthread_mutex_lock(&g_dfmu);
|
||||
dm_host_lock(&g_dfmu);
|
||||
if (!g_dfthr) { g_dfthr = 1; dm_guest_thread_spawn(md_defer_body, NULL, 16, 64 << 10); }
|
||||
for (int i = 0; i < g_ndfq; i++) if (g_dfq[i].d == d && g_dfq[i].ev == ev && g_dfq[i].due <= due) { pthread_mutex_unlock(&g_dfmu); return; } // already due by then
|
||||
if (g_ndfq < 64) g_dfq[g_ndfq++] = (__typeof__(g_dfq[0])){d, ev, due};
|
||||
@@ -151,14 +154,14 @@ static void md_defer_at(void *d, int ev, uint64_t due) {
|
||||
}
|
||||
static void md_defer(void *d, int ev) { md_defer_at(d, ev, now_ns() + 2000000); }
|
||||
static void md_defer_drop(void *d) {
|
||||
pthread_mutex_lock(&g_dfmu);
|
||||
dm_host_lock(&g_dfmu);
|
||||
int n = 0; for (int i = 0; i < g_ndfq; i++) if (g_dfq[i].d != d) g_dfq[n++] = g_dfq[i];
|
||||
g_ndfq = n; pthread_mutex_unlock(&g_dfmu);
|
||||
}
|
||||
// signal under the lock: the destructor first drops pending entries under it, so a signal never interleaves with destruction
|
||||
static void md_defer_body(void *arg) {
|
||||
(void)arg;
|
||||
pthread_mutex_lock(&g_dfmu);
|
||||
dm_host_lock(&g_dfmu);
|
||||
for (;;) {
|
||||
uint64_t now = now_ns(), next = 0; int k = -1;
|
||||
for (int i = 0; i < g_ndfq && k < 0; i++) { if (g_dfq[i].due <= now) k = i; else if (!next || g_dfq[i].due < next) next = g_dfq[i].due; }
|
||||
@@ -414,7 +417,7 @@ static void md_job(mdec_t *d, int job) {
|
||||
}
|
||||
// while packets are queued and an output slot is free, decode the head packet, until the queue is empty or every slot is taken; signal the input / output events by the result
|
||||
static void md_pump(mdec_t *d) {
|
||||
pthread_mutex_lock(&d->dmu);
|
||||
dm_host_lock(&d->dmu);
|
||||
for (;;) {
|
||||
pthread_mutex_lock(&d->mu);
|
||||
if (d->avc_drain) { // H.264 end of stream: the pictures still queued leave one per free slot; the last carries the end-of-stream flag
|
||||
@@ -503,7 +506,7 @@ static void md_free(void **self) {
|
||||
self[1] = NULL;
|
||||
LOG("[movie] decoder %p destroyed: %lu packets in, %lu frames out; decoded %llu packets, average %.2f ms per packet, longest %.2f ms", (void *)self, d->ncall[7], d->ncall[8],
|
||||
(unsigned long long)d->dec_n, d->dec_n ? d->dec_ns / 1e6 / d->dec_n : 0.0, d->dec_max / 1e6);
|
||||
pthread_mutex_lock(&d->dmu); md_job(d, MDJ_QUIT); pthread_mutex_unlock(&d->dmu);
|
||||
dm_host_lock(&d->dmu); md_job(d, MDJ_QUIT); pthread_mutex_unlock(&d->dmu);
|
||||
pthread_join(d->wk, NULL);
|
||||
for (int i = 0; i < MD_NIN; i++) free(d->in[i]);
|
||||
for (int k = 0; k < MD_NOUT; k++) free(d->out[k].frm);
|
||||
@@ -559,7 +562,7 @@ HLE_FORCE("_ZN5movie7Decoder5StartEv", "movie decoder start: all inputs empty, n
|
||||
}
|
||||
// clear the queues and slots; with close, the libvpx decoder is closed too (rebuilt at the next packet)
|
||||
static void md_reset(mdec_t *d, int close) {
|
||||
pthread_mutex_lock(&d->dmu); // wait for the packet being decoded
|
||||
dm_host_lock(&d->dmu); // wait for the packet being decoded
|
||||
if (close) md_job(d, MDJ_CLOSE);
|
||||
if (d->avc) mavc_drop(d->avc); // H.264 pictures waiting for their turn belong to the old position
|
||||
pthread_mutex_lock(&d->mu);
|
||||
@@ -576,7 +579,7 @@ static void md_reset(mdec_t *d, int close) {
|
||||
HLE_FORCE("_ZN5movie7Decoder4StopEv", "movie decoder stop", int, h_movie_Stop, void **self) {
|
||||
mdec_t *d = md_of(self); if (!d) return 0x2003;
|
||||
MD_TRACE(d, 3, "Stop");
|
||||
pthread_mutex_lock(&g_dfmu); pthread_mutex_lock(&d->mu); d->started = 0; pthread_mutex_unlock(&d->mu); pthread_mutex_unlock(&g_dfmu); // the deferred thread signals holding g_dfmu: taking it guarantees no signal afterwards
|
||||
dm_host_lock(&g_dfmu); pthread_mutex_lock(&d->mu); d->started = 0; pthread_mutex_unlock(&d->mu); pthread_mutex_unlock(&g_dfmu); // the deferred thread signals holding g_dfmu: taking it guarantees no signal afterwards
|
||||
md_defer_drop(d);
|
||||
md_reset(d, 1); return 0;
|
||||
}
|
||||
@@ -589,7 +592,7 @@ HLE_FORCE("_ZN5movie7Decoder5ResetEv", "movie decoder reset", int, h_movie_Reset
|
||||
HLE_FORCE("_ZN5movie7Decoder5FlushEv", "movie decoder flush: all inputs and outputs returned", int, h_movie_Flush, void **self) {
|
||||
mdec_t *d = md_of(self); if (!d) return 0x1000;
|
||||
MD_TRACE(d, 5, "Flush"); md_defer_drop(d); md_reset(d, 0);
|
||||
if (d->vb) { pthread_mutex_lock(&d->dmu); stb_vorbis_flush_pushdata(d->vb); pthread_mutex_unlock(&d->dmu); } // a seek: start the overlap afresh
|
||||
if (d->vb) { dm_host_lock(&d->dmu); stb_vorbis_flush_pushdata(d->vb); pthread_mutex_unlock(&d->dmu); } // a seek: start the overlap afresh
|
||||
return 0;
|
||||
}
|
||||
HLE_FORCE("_ZN5movie7Decoder23AcquireInputBufferIndexEPiS1_", "movie decoder: acquire a free input index", int, h_movie_AcqIn, void **self, int *idx, int *left) {
|
||||
|
||||
@@ -167,6 +167,16 @@ int dm_yield_to_core_peers(int all) {
|
||||
{ struct timespec ts = {0, 50000}; nanosleep(&ts, 0); } waited = 1;
|
||||
}
|
||||
}
|
||||
// Take a host lock that an HLE call holds while it runs guest code. A guest thread waiting for it is in a native HLE wait (dm_thread_native_wait), blocked as it would be
|
||||
// on the guest-visible lock the SDK's own code uses there, so it does not hold a same-core handoff: Kirby and the Forgotten Land's guide movies froze the game when the movie decoder's decode lock was held across the
|
||||
// "input available" SignalEvent, the signal woke the priority-44 movie threads of core 0, and the signaller (a priority-46 job worker) waited for its woken input thread,
|
||||
// which was itself waiting for that lock (hle_movie.c md_pump)
|
||||
void dm_host_lock(pthread_mutex_t *m) {
|
||||
if (!pthread_mutex_trylock(m)) return;
|
||||
dm_thread_native_wait(1);
|
||||
pthread_mutex_lock(m);
|
||||
dm_thread_native_wait(0);
|
||||
}
|
||||
|
||||
// ---------- Wait primitives (called holding G) ----------
|
||||
static void wait_unlink(thr_t *t) {
|
||||
|
||||
@@ -192,27 +192,39 @@ static void *scope_any_yield(void *arg) { // waits on event 1, then yields to
|
||||
while (!atomic_load(&scope_release)) { uint64_t r[32] = {(uint64_t)-2}; svc_thread(0x0B, r); }
|
||||
thread_exit(t); return NULL;
|
||||
}
|
||||
static pthread_mutex_t scope_mu = PTHREAD_MUTEX_INITIALIZER; // a host lock the signaller holds across its signal (hle_movie.c: the decoder's dmu)
|
||||
static void *scope_host_lock(void *arg) { // waits on event 1, then waits for the host lock, finishes its payload and exits
|
||||
thr_t *t = arg; pthread_setspecific(g_keythr, t);
|
||||
scope_wait(1);
|
||||
atomic_store(&scope_payload, 1);
|
||||
dm_host_lock(&scope_mu); atomic_store(&scope_payload, 2); pthread_mutex_unlock(&scope_mu);
|
||||
thread_exit(t); return NULL;
|
||||
}
|
||||
static void scope_until_waiting(thr_t *t) {
|
||||
for (;;) { pthread_mutex_lock(&G); int w = t->wkind == 4; pthread_mutex_unlock(&G); if (w) break; usleep(100); }
|
||||
}
|
||||
// kind 0: a busy unrelated peer and one woken worker; 1: a dispatcher that wakes the worker; 2: a woken thread that yields to any thread
|
||||
// kind 0: a busy unrelated peer and one woken worker; 1: a dispatcher that wakes the worker; 2: a woken thread that yields to any thread;
|
||||
// 3: a woken thread that waits for a host lock the signaller holds (it never returned before that wait counted as blocked)
|
||||
static int handoff_scope(int kind) {
|
||||
for (int e = 0; e < 2; e++) { uint64_t r[32] = {0}; svc_thread(0x45, r); assert(r[0] == 0); scope_ev[e][0] = r[1]; scope_ev[e][1] = r[2]; }
|
||||
atomic_store(&scope_payload, 0); atomic_store(&scope_release, 0);
|
||||
thr_t *busy = scope_new(&scope_peer[0], 30); // runnable in guest code the whole time, woken by nobody
|
||||
thr_t *w = scope_new(&scope_peer[1], 30), *d = kind == 1 ? scope_new(&scope_peer[2], 30) : NULL;
|
||||
pthread_t pw, pd;
|
||||
assert(!pthread_create(&pw, NULL, kind == 2 ? scope_any_yield : scope_worker, w)); scope_until_waiting(w);
|
||||
assert(!pthread_create(&pw, NULL, kind == 2 ? scope_any_yield : kind == 3 ? scope_host_lock : scope_worker, w)); scope_until_waiting(w);
|
||||
if (d) { assert(!pthread_create(&pd, NULL, scope_dispatcher, d)); scope_until_waiting(d); }
|
||||
if (kind == 3) pthread_mutex_lock(&scope_mu);
|
||||
scope_signal(d ? 0 : 1);
|
||||
pthread_mutex_lock(&G); int payload = atomic_load(&scope_payload), done = w->exited; pthread_mutex_unlock(&G);
|
||||
if (kind == 3) pthread_mutex_unlock(&scope_mu);
|
||||
atomic_store(&scope_release, 1);
|
||||
pthread_join(pw, NULL); if (d) pthread_join(pd, NULL);
|
||||
if (d) scope_free(d);
|
||||
scope_free(w); scope_free(busy);
|
||||
for (int e = 0; e < 2; e++) for (int i = 0; i < 2; i++) { uint64_t r[32] = {scope_ev[e][i]}; svc_thread(0x16, r); }
|
||||
int ok = kind == 2 ? payload == 1 : payload == 2 && done;
|
||||
static const char *what[] = {"woken worker beside a busy unrelated peer", "worker woken by the woken dispatcher", "woken thread yielding to any thread"};
|
||||
int ok = kind >= 2 ? payload == 1 : payload == 2 && done;
|
||||
static const char *what[] = {"woken worker beside a busy unrelated peer", "worker woken by the woken dispatcher", "woken thread yielding to any thread",
|
||||
"woken thread waiting for the signaller's host lock"};
|
||||
printf("%s: handoff scope, %s: signaller returned, payload %d, worker exited %d\n", ok ? "PASS" : "FAIL", what[kind], payload, done);
|
||||
return ok;
|
||||
}
|
||||
@@ -224,7 +236,7 @@ int main(int argc, char **argv) {
|
||||
main_handle = svc_thread_main_init();
|
||||
condition_order();
|
||||
int ok = 1;
|
||||
if (argc == 1) for (int kind = 0; kind <= 2; kind++) ok &= handoff_scope(kind); // with every runnable peer checked (core_preempt), the busy peer holds the signaller by design
|
||||
if (argc == 1) for (int kind = 0; kind <= 3; kind++) ok &= handoff_scope(kind); // with every runnable peer checked (core_preempt), the busy peer holds the signaller by design
|
||||
for (int kind = 0; kind <= 4; kind++) if (kind != 3) {
|
||||
ok &= delayed_wake(1000, kind, 0);
|
||||
ok &= delayed_wake(20000, kind, 0);
|
||||
|
||||
@@ -83,6 +83,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
|
||||
- the main thread waits forever on an event right after `nnMain` starts, with only three or four threads created; `DM_THR=1` shows no `CreateThread` for the thread it waits on. → [Free thread count (`svcGetInfo` type 24)](../reference/threads.md#free-thread-count-svcgetinfo-type-24)
|
||||
- in garbage-collected titles (Unity / IL2CPP), live objects freed, with their finalizers releasing native resources. → [Pausing threads and reading their context for garbage collectors](../reference/threads.md#pausing-threads-and-reading-their-context-for-garbage-collectors)
|
||||
- crashes around movie changes in the game's own player threads, about one run in two: `SignalEvent` → `MultiWaitImpl::NotifyAndWakeupThread` on a null MultiWait at address 0x20; `WaitAny` walking freed holders; a worker reading a freed object at `main+0x225338`. The other way round, a handoff that waits too widely: a black screen at boot that never loads a texture (`packaged textures: first uploads 0`), or a load that never ends behind an animated loading screen, with the thread dump's `same-core handoffs N, total T ms` larger than the run itself. → [Same-core wake handoffs](../reference/threads.md#same-core-wake-handoffs)
|
||||
- the game freezes when a movie starts playing beside the game (a guide movie in a corner of the screen): the last frame stays, the process idles at a few percent of a core, and the thread dump shows a lower-priority job thread `running/not waiting` with its backtrace in the movie player (`ReleaseOutputBufferIndex`), the main thread waiting for it, and the movie's input thread `running/not waiting` too. A `sample` of the process shows the job thread in `md_pump` → `md_event` → `dm_yield_to_core_peers`, and the input thread in `h_movie_SendIn` → `md_pump` waiting for a pthread mutex. → [A host lock held across a same-core handoff](../reference/threads.md#a-host-lock-held-across-a-same-core-handoff)
|
||||
- several threads crash at once and the dumps interleave, then the process hangs instead of exiting. → [Simultaneous crashes of several threads](../reference/threads.md#simultaneous-crashes-of-several-threads)
|
||||
- none seen; found while investigating Fire Emblem Engage's demo crash, which it did not change (that was the address-space shortage, Size-aligned regions exhaust the alias address space). → [`SignalToAddress` type 2 with count ≤ 0](../reference/threads.md#signaltoaddress-type-2-with-count--0)
|
||||
- a 64-bit wait could return at once or block when it should not (not met in a game). → [`WaitForAddress` type 3: 64-bit wait if equal](../reference/threads.md#waitforaddress-type-3-64-bit-wait-if-equal)
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 212 KiB |
@@ -33,7 +33,7 @@ Finding, keying and translating shaders (`dm/gfx/tools/shader_split.py`, `dmsh_b
|
||||
- **Symptom:** the game asks for shaders the map does not have (`[RED] shader ... is not in the shader map`, hundreds of lines), and the split finds far fewer shaders than the game uses (121 against 27,288 on Kirby and the Forgotten Land).
|
||||
- **Cause:** HAL keeps its assets in `.cmp` containers: a 4-byte little-endian uncompressed size, then a frame (zstd, or LZ4 in the `ubin/` map data). The split decompressed a zstd file by its magic at offset 0, so a frame that starts at offset 4 was read raw, and the shaders inside the BFRES archives of those containers were never seen.
|
||||
- **Fix:** the split opens a size-prefixed frame as well (`unsized`: zstd, with a dictionary as for a plain zstd file, or LZ4) when the file's bytes 4 to 8 are a frame magic and bytes 0 to 4 are not the zstd magic. A frame whose header records a size other than the prefix is not such a container, and the file is read as it is: Tears of the Kingdom's 1,202 `Cave/…/C.crbin.<hash>/*.quad` files are a 4-byte hash (the one in their folder name) and a zstd frame, the only files with a frame at +4 in the RomFS of 39 other games on hand. A frame that records no size must decompress to exactly the prefix's size; otherwise (it does not decompress, it holds more or less, or it names a dictionary `Pack/ZsDic.pack.zs` does not have) the split prints a `[RED]` line and reads the file as it is.
|
||||
- **Found on:** Kirby and the Forgotten Land (`01004D300C5AE000`, in bring-up): 7,201 `.cmp` files, 5,121 of them zstd frames with 19,364 BNSH inside (11.2 GB decompressed) and 2,080 LZ4 frames, all of the latter in `ubin/` (0.3 GB, and no BNSH); the same layout is in 430 `.cmpbin` files (76 zstd in `lyt/`, 354 LZ4 in `snd/`). Every zstd frame records its size, the same as its prefix; no LZ4 frame does, and each decompresses to its prefix's size. The split now finds 27,288 shaders in the base game (18,476 in `gfx/`, 8,313 in `ptcl/`, 378 in `lyt/`, 67 in the loose `shader` files, 54 in `main`), against 121 before, and 27,302 with update 1.1.0 (v65536: 14 more in `lyt/`).
|
||||
- **Found on:** [Kirby and the Forgotten Land](../../dm/games/01004D300C5AE000/README.md): 7,201 `.cmp` files, 5,121 of them zstd frames with 19,364 BNSH inside (11.2 GB decompressed) and 2,080 LZ4 frames, all of the latter in `ubin/` (0.3 GB, and no BNSH); the same layout is in 430 `.cmpbin` files (76 zstd in `lyt/`, 354 LZ4 in `snd/`). Every zstd frame records its size, the same as its prefix; no LZ4 frame does, and each decompresses to its prefix's size. The split now finds 27,288 shaders in the base game (18,476 in `gfx/`, 8,313 in `ptcl/`, 378 in `lyt/`, 67 in the loose `shader` files, 54 in `main`), against 121 before, and 27,302 with update 1.1.0 (v65536: 14 more in `lyt/`).
|
||||
|
||||
### BNSH in Bezel Engine archives
|
||||
|
||||
|
||||
@@ -49,6 +49,14 @@ Guest threads, their synchronization and how the host schedules them. Each entry
|
||||
- **Found on:** [Metroid Dread](../../dm/games/010093801237C000/README.md), whose per-build movie request-ring gates and `core_preempt on` setting are now removed; the whole-core wait on [The Legend of Zelda: Tears of the Kingdom](../../dm/games/0100F2C0115B6000/README.md) ([issue #247](https://github.com/seedds/nxnative/issues/247)) and Donkey Kong Country: Tropical Freeze ([issue #259](https://github.com/seedds/nxnative/issues/259), section 6; no profile here).
|
||||
- **Scope:** these are cooperative thread-SVC handoffs, not a complete asynchronous guest scheduler. The optional `core_preempt` profile key (`DM_PREEMPT=0` / `1` overrides it) additionally checks already-runnable peers after every thread SVC; it defaults off. Historically that broader policy moved [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md)'s first frame from 0.6 s to 2.9 s by waiting for busy decompression threads. Wake-triggered handoffs apply with either setting.
|
||||
|
||||
### A host lock held across a same-core handoff
|
||||
|
||||
- **Symptom:** the game freezes when a movie starts playing beside the game (a guide movie in a corner of the screen): the last frame stays, the process idles at a few percent of a core, and the thread dump shows a lower-priority job thread `running/not waiting` with its backtrace in the movie player (`ReleaseOutputBufferIndex`), the main thread waiting for it, and the movie's input thread `running/not waiting` too. A `sample` of the process shows the job thread in `md_pump` → `md_event` → `dm_yield_to_core_peers`, and the input thread in `h_movie_SendIn` → `md_pump` waiting for a pthread mutex.
|
||||
- **Cause:** the host's movie decoder holds its decode lock (`dmu`) across guest code: the "input available" and "output ready" events are signalled with the SDK's `SignalEvent` from inside `md_pump`. The signal woke the movie's higher-priority threads pinned to the signaller's core, so the signaller's [same-core wake handoff](#same-core-wake-handoffs) waited for them to block before its SVC returned. One of them, the input thread, had called `SendInputBufferForDecode` and was waiting for `dmu` in host code. A thread waiting for a host lock still counted as runnable, so the handoff never ended and the lock was never released. On Horizon that lock is guest-visible (the SDK's decoder holds an `nn::os` mutex there), and a thread waiting for it is blocked.
|
||||
- **Fix:** `dm_host_lock` (`hle_thread.c`) takes a host lock that HLE code holds while it runs guest code. A guest thread that has to wait for it is in a [native HLE wait](#native-hle-waits-release-the-guest-core) (`dm_thread_native_wait`) while it waits: blocked for `thr_runnable`, and out of the handoff that made it runnable, so the handoff does not wait for it. The movie decoder takes `dmu` and the deferred-event lock `g_dfmu` with it (`hle_movie.c`). Releasing `dmu` before signalling instead would let a signal land after `Stop` has returned, on events the game is tearing down (the reason `md_defer_body` signals under its lock).
|
||||
- **Check:** `make -C dm/host/tests GAME=010093801237C000 preempt`, handoff scope "woken thread waiting for the signaller's host lock": the signaller holds a host lock across its signal and the woken higher-priority worker waits for it with `dm_host_lock`. The signaller must return before the worker finishes; without the native wait around the lock the test never returns.
|
||||
- **Found on:** [Kirby and the Forgotten Land](../../dm/games/01004D300C5AE000/README.md): the first guide movie ("A Jump", 480×270 VP9) froze the game on the first walk from the beach, every time. The job worker (priority 46) and the movie threads (priority 44) are all pinned to core 0.
|
||||
|
||||
### Simultaneous crashes of several threads
|
||||
|
||||
- **Symptom:** several threads crash at once and the dumps interleave, then the process hangs instead of exiting.
|
||||
|
||||
Reference in new issue
Block a user