diff --git a/dm/games/01002B00111A2000/README.md b/dm/games/01002B00111A2000/README.md index 78a9c05..5143eb8 100644 --- a/dm/games/01002B00111A2000/README.md +++ b/dm/games/01002B00111A2000/README.md @@ -8,7 +8,6 @@ Update 1.3.0 (v327680) over the base game, with four DLC packages (one is only a |---|---|---| | Frame step, `fps_target 60` | [`patches.py`](patches.py) `FRAME_STEP`, [`game.tsv`](game.tsv) | three stubs run the game's fixed-step updates at the cap; see [Frame rate](#frame-rate) | | `fps_hold off` | `game.tsv` | the updates step by the cap whatever the hold, so holding late frames only paces the screen: each frame is shown as soon as it is ready (the 60fps IPA ran battles at 45–57) | -| Sound heap cut: engine heap +4 MB, sound data heap −4 MB | `patches.py` `CUTS`, `main` build `c3cf52bf…` | the KTSL sound setup gave up quietly and the main thread waited forever; see [Audio](#audio) | | `mem_used 0x4800000` (72 MB) | `game.tsv` | the three modules take 66.6 MB, more than the default 64 MB, so `SetHeapSize` failed; see [Memory](#memory) | | 4 GB layout: `mem_reserve 0x100000000`, `mem_alias 0x10000000`, `mem_stack 0x10000000`, `app_pool 0xCD500000` | `game.tsv` | the NPDM declares no system resource (META +0x14 = 0), so the game uses `SetHeapSize`. Link's Awakening's partition, not measured for this game | | `tex_flush_upload on` | `game.tsv` | movie frames are CPU-written and published with `nvnMemoryPoolFlushMappedRange`; without the upload the intro movie stayed black under its "+ Skip" prompt | @@ -37,9 +36,11 @@ Update 1.3.0 (v327680) over the base game, with four DLC packages (one is only a ## Audio -- **Resolved: the main thread spun at start-up in a resource wait** (a `YieldThread` loop, no crash, right after `OpenAudioRenderer`; nnMain's resource wait at `main+0x8d9bc8`). Koei Tecmo's sound engine (KTSL) failed its setup quietly. The init (`main+0x8d9610`) hands it three heaps cut from one block: 15 MB engine memory, 2 MB and 142 MB sound data (sizes built by `mov x8,#0xf00000; movk x8,#0x20,lsl #48` and `mov w9,#0x8e00000`; constructor `main+0x12647a0`, heaps created at `main+0x1a2ba58`). In the 15 MB heap the setup allocates the renderer config and work buffer (0x760000: `AudioRenderSystem::GetWorkBufferSize`, 0x40a9c8 of it command buffer) and one block per player (up to 0x1b5860; three of the four mix 5.1), about 16.9 MB in all. The fourth player block (`main+0x1a32f70`) came back null, the setup released its memory pools (`RequestAttachMemoryPool` then `Detach` with no update between) and returned −51, the first sound archive (ASRS) never loaded, and the resource the main thread waited for depended on it. +- **Resolved: the main thread spun at start-up in a resource wait** (a `YieldThread` loop, no crash, right after `OpenAudioRenderer`; nnMain's resource wait at `main+0x8d9bc8`). Koei Tecmo's sound engine (KTSL) failed its setup quietly. The init (`main+0x8d9610`) hands it three heaps cut from one block: 15 MB engine memory, 2 MB and 142 MB sound data (sizes built by `mov x8,#0xf00000; movk x8,#0x20,lsl #48` and `mov w9,#0x8e00000`; constructor `main+0x12647a0`, heaps created at `main+0x1a2ba58`). In the 15 MB heap the setup allocated the renderer config and local server work buffer (0x760000, including a 0x40a9c8 command buffer) and one block per player. The fourth player block (`main+0x1a32f70`, size 0x1b5860) came back null, setup released its memory pools (`RequestAttachMemoryPool` then `Detach` with no update between) and returned −51, and the first sound archive (ASRS) never loaded. - How it was found: probe the pump the wait calls (`main+0x8e1c0`: first missing stage bit → the type's stage handler), the handler's dependency list (an incomplete entry of type `0xbbd39f2d`, ASRS), then the memory pool states (`MemoryPoolInfo` +0x10: 1 new, 3 detached, 4 request attach, 5 attached, 6 unused) and the engine allocator's arguments and results. -- **Current:** a per-build cut moves 4 MB from the sound data heap to the engine heap (`mov x8,#0xf00000` → `#0x1300000` at `main+0x8d9610`, `mov w9,#0x8e00000` → `#0x8a00000` at `main+0x8d9618`), so the block the three come from keeps its size. Why hardware fits in 15 MB is not known. +- **Measured, 2026-10-26:** with both heap cuts restored to the game's original instructions, `DM_PROBE=main:1a32f74` reproduced the null fourth allocation. The unchanged renderer parameter is 48 kHz, 240 samples, 132 mix buffers, 19 submixes, 138 voices, 1 sink, 16 effects, 0 performance frames, 85 splitters, 1800 splitter destinations and revision `REV8` (`0x38564552`). The SDK 9.6 local `AudioRenderSystem::GetWorkBufferSize` (`sdk+0x160320`) returns **0x760000 for DSP/auto, CPU/auto and CPU/manual**; the public CPU/manual query returns the same. The internal parameter was packed as the public wrapper (`sdk+0x1533a0`) does: device / mode bytes +0x22 / +0x23, revision +0x30. Comparing these local modes alone did not explain the shortage. +- **Resolved root cause: the host exposed the local SDK server's larger command representation as the DSP service's requirement.** The installed Horizon audio sysmodule (`0100000000000014`, NSO build `3a75b4a62dad189b2c8b30e4dedb4e15`, process renderer revision 15) was extracted into this game's work folder. Its service size method (`audio+0x47be4`) directly returns `AudioRenderSystem`'s calculation (`audio+0x47c14`). Isolated execution of the relocated machine code gives **0x61d000** for the same parameter, in all three modes. Its command-buffer calculation (`audio+0x58344`) returns **0x2c9a60**, while the SDK 9.6 local one (`sdk+0x166f10`) returns **0x40a9c8**. The command contribution accounts for 0x140f68 of the total **0x143000 excess**; other state sizes and page rounding account for the rest. The local routine sizes its compiled CPU structures even when passed DSP/auto, so that earlier comparison could not measure the service budget. Reporting the measured service size alone, with backend storage supplied separately, made all players fit: this explains why the original heap can work on Horizon. +- **Current fix:** the host computes the service's revision-dependent requirement and sizes the local backend separately ([service and backend work buffers](../../../docs/reference/audio.md#a-sound-engines-heap-too-small-for-its-renderer-and-players)). This game receives **0x61d000**, with the local SDK still given its required **0x760000**. The temporary one-page policy and both per-build heap cuts are removed. With the original 15 / 2 / 142 MB heaps, all five observed player allocations succeed (0x1220e8, three of 0x1b5860, then 0x84c80), the resource wait returns at `main+0x8d9be0`, and the renderer produces non-silent PCM. This is a windowless audio check without rebuilt shader libraries; iOS and battles have not been retested for this change. Independent fixture cases and 360 seeded parameter sets across REV1–REV15 match executed Horizon code; the tests also cover the public size/open/close contract and buffer lifetime. Firmware is used only for the investigation, never by the host. - The multi-stream hardware Opus decoder is the SDK's software one (`HardwareOpusMultiStreamDecoder`, see [Services](#services)). ## Rendering @@ -86,6 +87,7 @@ General host changes first needed here; each applies to every game: - **Rewriter:** byte jump tables indexed by x30 (`main+0x138216c`; capstone names x30 `lr` and x29 `fp`, mapped back): [Byte tables indexed by x30 (capstone register names)](../../../docs/reference/rewriter.md#byte-tables-indexed-by-x30-capstone-register-names). - **`mem_used`** for games whose modules exceed 64 MB: [Heap sized from the application pool less used memory](../../../docs/reference/memory.md#heap-sized-from-the-application-pool-less-used-memory). - **Copyright frame buffer no-ops** and the **hardware Opus multi-stream decoder** as the SDK's software one (`hle_oe.c`, `hle_codec.c`): [Copyright frame buffer overlay](../../../docs/reference/services.md#copyright-frame-buffer-overlay), [Hardware Opus decoders built on the SDK's software decoders](../../../docs/reference/audio.md#hardware-opus-decoders-built-on-the-sdks-software-decoders). +- **Host ownership of the local audio renderer's work buffer:** [A sound engine's heap too small for its renderer and players](../../../docs/reference/audio.md#a-sound-engines-heap-too-small-for-its-renderer-and-players). - **Compute's own binding** (`S.cprog`, `g_pp_cprog`), **`nvnTexturePoolRegisterImage`**, **indirect draws and dispatches** (`ixc_run`, `DM_NVN_IXCLOG`), **dispatches in the copy encoder** (`dispatch_enc`, `DM_NVN_DISPATCH_OWN=1`) and **texture views cached by texture** (`vc_get`): [Compute programs bound apart from graphics programs](../../../docs/reference/rendering.md#compute-programs-bound-apart-from-graphics-programs), [Images registered on their own](../../../docs/reference/nvn-objects.md#images-registered-on-their-own), [Indirect draws and dispatches read by the GPU](../../../docs/reference/rendering.md#indirect-draws-and-dispatches-read-by-the-gpu), [Compute dispatches share the copy encoder](../../../docs/reference/rendering.md#compute-dispatches-share-the-copy-encoder), [Texture views cached by texture and view parameters](../../../docs/reference/rendering.md#texture-views-cached-by-texture-and-view-parameters). - **CPU vertex conversion over the draw's vertices** (`vconv_range`, `vconv_bind`): [dm/games/README.md](../README.md). - **Shadow vertex inputs** (`dmsh_vin.py`, `vin_ct` / `vin_on`) and **row wrap** for buffer textures read by 1D fetches (`fetch1d`, `tbuf_as_2d`, `nvn_tbuf_rows`): [Shadow vertex inputs](../../../docs/reference/rendering.md#shadow-vertex-inputs), [Buffer textures read by 1D integer fetches (row wrap)](../../../docs/reference/rendering.md#buffer-textures-read-by-1d-integer-fetches-row-wrap). diff --git a/dm/games/01002B00111A2000/patches.py b/dm/games/01002B00111A2000/patches.py index d6e9e89..e961542 100644 --- a/dm/games/01002B00111A2000/patches.py +++ b/dm/games/01002B00111A2000/patches.py @@ -1,19 +1,6 @@ # Hyrule Warriors: Age of Calamity (01002B00111A2000): the rewriter's patches of this game's own builds. dm/tools/patch_nso.py loads every profile's patches.py and applies those of a module's NSO build ID; # CUTS, OBS and FRAME_STEP as described there ("Registered patches"). Patches of Nintendo SDK module builds are in dm/tools/sdk_patches.py from nso_asm import ADRP, B -CUTS = { - # Hyrule Warriors: Age of Calamity 1.3.0 (v327680) main: the init (main+0x8d9610) hands the KTSL sound engine three heaps carved from one block: 15 MB engine memory, 2 MB, and - # 142 MB sound data (sizes built by mov x8,#0xf00000; movk x8,#0x20,lsl #48 and mov w9,#0x8e00000; constructor main+0x12647a0, heaps created at main+0x1a2ba58). The audio renderer - # work buffer in the 15 MB heap is 0x760000 (the SDK's AudioRenderSystem::GetWorkBufferSize, 0x40a9c8 of it command buffer), and the engine's setup asks for about 16.9 MB in - # all: the fourth 0x1b5860-byte player block (main+0x1a32f70; three of the four players mix 5.1) came back null, the setup released its memory pools and gave up (-51), - # the first sound archive (ASRS) never loaded and nnMain's resource wait (main+0x8d9bc8) spun forever. Where hardware differs is not known. The engine heap gets 4 MB from the - # sound data heap, so the block the three come from keeps its size - 'c3cf52bf2b05d7310884971b50bcccb8': [ - (0x8d9610, '081ea0d2', '0826a0d2', 'sound engine heap: mov x8,#0xf00000 → mov x8,#0x1300000 (+4 MB)'), - (0x8d9618, '091ca152', '0914a152', 'sound data heap: mov w9,#0x8e00000 → mov w9,#0x8a00000 (-4 MB; the block the three heaps come from keeps its total size)'), - ], -} - # Hyrule Warriors: Age of Calamity steps its game a fixed amount per update, from two 30 fps values. A KIDS task (main+0x166d980, on a KIDSTaskScheduler thread, once per update) # waits until 1 / fps has passed since the last update (fps at time +0x30: 30 from a frame-rate object, main+0x166daa4; csinc at main+0x166db80) and then steps the game clock # (system +0x3348, the step at +0x3350) by that object's step (x23 +0x34: 1/30 s times the game's timescale, a second time at +0x44, fps 30 at +0x48), not by the time that diff --git a/dm/host/hle_audren.c b/dm/host/hle_audren.c index f651213..4d53dbc 100644 --- a/dm/host/hle_audren.c +++ b/dm/host/hle_audren.c @@ -4,7 +4,8 @@ // +0x2c renderingDevice = 1 CPU, +0x30 executionMode = 1 manual). In this mode OpenAudioRenderer goes through the in-process DFC manager (CreateAudioRendererManagerByDfc), // RequestUpdate is parsed in-process, and rendering happens on the CPU of whichever thread calls nn::audio::ExecuteAudioRendererRendering, which then signals the game's renderer event. // The host does three things: -// 1. Takes over GetAudioRendererWorkBufferSize / OpenAudioRenderer: switches the parameter to CPU + manual and calls the SDK original (the work buffer size follows the modified parameter); +// 1. Takes over GetAudioRendererWorkBufferSize / OpenAudioRenderer: preserves the DSP service's work-buffer requirement, then runs the SDK's CPU + manual renderer. +// The local SDK uses a different command format whose storage can exceed the service's budget; that backend storage is supplied by the host. Explicit CPU renderers keep the SDK contract; // 2. A render thread (a guest thread started by the host): renders one frame whenever the output hands back a 5 ms buffer, so pacing follows the sound card clock; // 3. Takes over AudioRendererImpl::ExecuteAudioRendererRendering (reached through the PLT): generates commands and runs CommandListProcessor like the original, // then finds the DeviceSink command in the command list (a no-op for the CPU processor) and clamps its input mix buffers (int32 holding 16-bit range) to int16 stereo for AudioOut (hle_audio.c). @@ -27,16 +28,59 @@ void dm_pcmout_wait(void *k, int ms); // P_SIZE: bytes of the game's AudioRendererParameter copied before the two fields are changed. SDK 16's struct is 0x38 bytes; SDK 8's (Link's Awakening) has one more word, // its revision at +0x38: copying only 0x38 left the revision uninitialized and the SDK rejected it (CheckValidRevision, Result 0x499). The SDK reads only its own size of the copy -enum { P_DEVICE = 0x2c, P_MODE = 0x30, P_SIZE = 0x40, FRAME_MAX = 1024, NBUF = 4 }; +enum { P_DEVICE = 0x2c, P_MODE = 0x30, P_SIZE = 0x40, WORK_PAGE = 0x1000, FRAME_MAX = 1024, NBUF = 4 }; static void *gsym(const char *n) { void *p = guest_sym(n); if (!p) die("audren: missing SDK symbol %s", n); return p; } static void cpu_manual(uint8_t *dst, const uint8_t *src) { memcpy(dst, src, P_SIZE); *(int32_t *)(dst + P_DEVICE) = 1; *(int32_t *)(dst + P_MODE) = 1; } -HLE_FORCE("_ZN2nn5audio30GetAudioRendererWorkBufferSizeERKNS0_22AudioRendererParameterE", "renderer work buffer size: computed for CPU rendering + manual execution", +static size_t ar_align(size_t n, size_t alignment) { return (n + alignment - 1) & ~(alignment - 1); } +// Horizon's renderer service budget, through renderer revision 15. Its command/state representation is independent of the application's SDK: an older client still uses the +// service's compact commands. The SDK's local server instead sizes its own CPU command structures, so forwarding its size inflates the guest's service requirement. +// Derived from the service's sizing and revision gates; checked against its isolated machine-code execution. No firmware code is needed at runtime. Unknown revisions retain +// the SDK requirement. SDK 8 keeps the public parameter revision at +0x38, while SDK 9+ uses +0x34 (the copied parameter holds both). +static size_t ar_service_work_size(const uint8_t *p) { + uint32_t revision = *(const uint32_t *)(p + 0x34); + if ((revision & 0xffffff) != 0x564552) revision = *(const uint32_t *)(p + 0x38); + if ((revision & 0xffffff) != 0x564552 || revision < 0x31564552 || revision > 0x3f564552) return 0; + unsigned r = (revision >> 24) - 0x30; + size_t n[11]; + for (int i = 0; i < 11; i++) { + if (i == 8) { n[i] = p[0x20]; continue; } // voice-drop is a byte; the following public-struct padding is not initialized + int32_t v = ((const int32_t *)p)[i]; + if (v < 0 || v > 0x100000) return 0; // bound arithmetic even if the caller supplied an invalid parameter + n[i] = (size_t)v; + } + size_t mix = n[2], sub = n[3], voices = n[4], sinks = n[5], effects = n[6], perf = n[7], splitters = n[9], dest = n[10], mixes = sub + 1; + size_t size = ar_align(((sinks + sub) * 0x3c0 + n[1] * 4) * (mix + 6), 64); + size += ar_align(mix * 4, 64) + sub * 0x400 + mixes * 0x940 + ar_align(mixes * 8, 16); + size += voices * 0x1d0 + ar_align(voices * (r >= 15 ? 0x238 : 0x220), 16) + ar_align(voices * 8, 16); + if (r >= 2) { + size_t graph = mixes * mixes * 4 + mixes * 12 + 2 * ar_align(mixes, 64) / 8 + ar_align(mixes * mixes, 64) / 8; + size += ar_align(graph, 16); + size += splitters * 0x20 + ar_align(dest * (r >= 12 ? (r >= 15 ? 0x130 : 8) : 0xe0), 16); + if (r < 5) size += ar_align(dest * 4, 16); + } + size += (effects + voices * 4) * 0x20 + 0x50; + if (r >= 12 && splitters && dest) size += ar_align(dest * (r >= 14 ? 0xe0 : 0x80), 64); + size = ar_align(size + (r >= 9 ? effects * 0x80 : 0), 64); + size += (sinks + sub) * 0x280 + effects * (r >= 9 ? 0x530 : 0x4b0) + sinks * 0x170 + voices * 0x100 + 0x40; + if (perf) { + size_t frame = (sub + voices + sinks + effects + 1) * (r >= 5 ? 0x18 : 0x10) + (r >= 5 ? 0x990 : 0x658); + size += ar_align(frame * (perf + 1), 64) + 0xc0; + } + size_t commands = 0x18000; + if (r >= 5) commands = effects * 0x540 + sub * 0x3f24 + sinks * 0x130 + dest * 0x360 + + voices * (splitters * 0x50 + (r >= 8 ? 0x284 : 0x250)) + (voices + sub + effects + sinks + 0x65) * 0x28 + 0x2c4; + return ar_align(size + commands + 0x7e, WORK_PAGE); +} + +HLE_FORCE("_ZN2nn5audio30GetAudioRendererWorkBufferSizeERKNS0_22AudioRendererParameterE", "renderer work buffer: DSP service budget, with extra local CPU backend storage supplied by the host", size_t, h_ar_WorkSize, const uint8_t *param) { static size_t (*orig)(const uint8_t *); if (!orig) orig = gsym("_ZN2nn5audio30GetAudioRendererWorkBufferSizeERKNS0_22AudioRendererParameterE"); uint8_t p[P_SIZE]; cpu_manual(p, param); size_t n = orig(p); - { static int once; if (!once++) LOG("audren: work buffer %#zx", n); } - return n; + size_t service = *(int32_t *)(param + P_DEVICE) == 0 ? ar_service_work_size(p) : 0; + size_t guest_n = service ? service : n; + { static int once; if (!once++) LOG("audren: work buffer guest %#zx, local CPU/manual renderer %#zx", guest_n, n); } + return guest_n; } // ---------- rendering and output ---------- static void *g_handle[2]; // AudioRendererHandle (renderer object, client state) @@ -95,8 +139,23 @@ static Result ar_open(void **handle, void *ev, const uint8_t *param, void *work, LOG("audren: OpenAudioRenderer%s rate %d samples per frame %d mix buffers %d submixes %d voices %d sinks %d effects %d splitters %d (original device %d mode %d → CPU / manual) work buffer %p+%#zx", ev ? "" : " (no event)", *(int32_t *)param, *(int32_t *)(param + 4), *(int32_t *)(param + 8), *(int32_t *)(param + 0xc), *(int32_t *)(param + 0x10), *(int32_t *)(param + 0x14), *(int32_t *)(param + 0x18), *(int32_t *)(param + 0x24), *(int32_t *)(param + P_DEVICE), *(int32_t *)(param + P_MODE), work, sz); + // The application's buffer follows the DSP service contract; the local server's larger CPU command representation is a backend cost. Keep a full-size caller buffer + // when it fits the backend, otherwise supply the backend storage here. The nested no-event open receives this same buffer and starts no second thread. + void *owned = NULL; + if (*(int32_t *)(param + P_DEVICE) == 0 && work && !((uintptr_t)work & (WORK_PAGE - 1)) && sz >= WORK_PAGE && !(sz & (WORK_PAGE - 1))) { + size_t (*size)(const uint8_t *) = gsym("_ZN2nn5audio30GetAudioRendererWorkBufferSizeERKNS0_22AudioRendererParameterE"); + size_t need = size(p); + size_t service = ar_service_work_size(p), guest_need = service ? service : need; + if (sz < guest_need) { LOG("[RED] audren: work buffer %#zx is smaller than the service requirement %#zx", sz, guest_need); return RES_FAIL; } + if (need > sz) { + if (posix_memalign(&owned, WORK_PAGE, need)) { LOG("[RED] audren: cannot allocate local renderer work buffer %#zx", need); return RES_FAIL; } + memset(owned, 0, need); work = owned; sz = need; + LOG("audren: host backs local renderer work buffer %p+%#zx", work, sz); + } + } t_in_open = 1; Result r = ev ? orig_ev(handle, ev, p, work, sz) : orig(handle, p, work, sz); t_in_open = 0; - if (r) { LOG("[RED] audren: SDK OpenAudioRenderer failed %#x", r); return r; } + if (r) { free(owned); LOG("[RED] audren: SDK OpenAudioRenderer failed %#x", r); return r; } + if (owned) objmap_put(handle[0], owned); // Close releases it only after the render thread and the SDK have finished with it if (atomic_load(&g_open)) { LOG("[RED] audren: a second renderer (the host drives only one)"); return r; } g_handle[0] = handle[0]; g_handle[1] = handle[1]; atomic_store(&g_open, 1); g_render_tt = dm_guest_thread_spawn(render_main, NULL, 9, 0x10000); @@ -111,7 +170,9 @@ HLE_FORCE("_ZN2nn5audio18CloseAudioRendererENS0_19AudioRendererHandleE", "close // Join the render thread first (it checks g_open every round and at most every 10 ms while waiting for the output): it may be inside exec(), which uses the renderer the SDK is about to free. // A fixed 20 ms sleep used to stand in for this, and a renderer reopened inside that window ran a second thread over the first one's static buffers if (obj == g_handle[0] && g_render_tt) { atomic_store(&g_open, 0); dm_guest_thread_join(g_render_tt); g_render_tt = NULL; } + void *owned = objmap_del(obj); orig(obj, state); + free(owned); } // DeviceSink input mix buffers → int16 stereo. The command ID is 0x13 in SDK 16; SDK 20 inserted a command before it, making it 0x14 (written at sdk+0x233f44); SDK 8 (Link's Awakening) diff --git a/dm/host/tests/Makefile b/dm/host/tests/Makefile index 2640232..c3debc4 100644 --- a/dm/host/tests/Makefile +++ b/dm/host/tests/Makefile @@ -16,6 +16,7 @@ run: $(B)/t_heap_size $(B)/t_touch $(B)/t_gesture $(B)/t_oe $(B)/t_fs $(B)/t_fs_ TMPDIR="$(abspath $(B))" $(B)/t_fs_case 2>/dev/null $(B)/t_audio $(B)/t_audio_spsc + $(B)/t_audren_work $(B)/t_swkbd $(B)/t_altstack $(B)/t_condvar @@ -34,6 +35,12 @@ run: $(B)/t_heap_size $(B)/t_touch $(B)/t_gesture $(B)/t_oe $(B)/t_fs $(B)/t_fs_ $(B)/t_nro run: $(B)/t_nro run: $(B)/t_system_event +run: $(B)/t_audren_work +$(B)/t_audren_work: t_audren_work.c ../hle_audren.c ../hle.h ../dm.h $(GAME_H) | $(B) + clang -O1 -g -Wall -Werror -Wno-unused-function -I.. $(GINC) -o $@ t_audren_work.c +.PHONY: audren-work +audren-work: $(B)/t_audren_work + $(B)/t_audren_work $(B)/t_system_event: t_system_event.c ../hle_thread.c ../hle_sdk16.c ../hle.h ../dm.h $(GAME_H) | $(B) clang -O1 -g -Wall -Werror -Wno-unused-function -Wno-frame-address -I.. $(GINC) -o $@ t_system_event.c .PHONY: system-event diff --git a/dm/host/tests/t_audren_work.c b/dm/host/tests/t_audren_work.c new file mode 100644 index 0000000..dc7ea02 --- /dev/null +++ b/dm/host/tests/t_audren_work.c @@ -0,0 +1,161 @@ +// Exercise the public size -> caller allocation -> both SDK open routes -> close boundary. +// The service requirement must not be inflated by the local backend's different command representation. +#include "../hle.h" +#include +#include + +static void *backing; +static size_t backing_size; +static int allocations, frees, fail_alloc; +static int work_alloc(void **out, size_t align, size_t size) { + if (fail_alloc) return ENOMEM; + int r = posix_memalign(out, align, size); + if (!r) { assert(!backing); backing = *out; backing_size = size; allocations++; } + return r; +} +static void work_free(void *p) { + if (p) { assert(p == backing); backing = NULL; backing_size = 0; frees++; } + free(p); +} +#define posix_memalign work_alloc +#define free work_free +#include "../hle_audren.c" +#undef posix_memalign +#undef free + +enum { SDK_WORK = 0x760000, SERVICE_WORK = 0x61d000 }; +static int opens, starts, joins, closes, fail_open, nested; +static void *sdk_work; +static void *map_key, *map_value; +static size_t sdk_size(const uint8_t *param) { + assert(*(int32_t *)(param + P_DEVICE) == 1 && *(int32_t *)(param + P_MODE) == 1); + assert((*(uint32_t *)(param + 0x34) & 0xffffff) == 0x564552 || (*(uint32_t *)(param + 0x38) & 0xffffff) == 0x564552); + return SDK_WORK; +} +static Result sdk_open(void **handle, const uint8_t *param, void *work, size_t size) { + opens++; + size_t need = h_ar_WorkSize(param); // SDK opens can query the already converted parameter through the PLT + assert(need == SDK_WORK); + if (!work || ((uintptr_t)work & (WORK_PAGE - 1)) || size < need) return RES_FAIL; + if (fail_open) return RES_FAIL; + assert(!backing || (work == backing && size == backing_size)); + sdk_work = work; + memset(work, 0x5a, need); // touch the entire area, not just the handle + handle[0] = work; handle[1] = (uint8_t *)work + 0x80; + return RES_OK; +} +static Result sdk_open_ev(void **handle, void *ev, const uint8_t *param, void *work, size_t size) { + assert(ev); + return nested ? h_ar_Open0(handle, param, work, size) : sdk_open(handle, param, work, size); +} +static void sdk_close(void *obj, void *state) { + assert(obj == sdk_work && state == (uint8_t *)sdk_work + 0x80); + assert(((uint8_t *)obj)[SDK_WORK - 1] == 0x5a); // storage is alive through SDK teardown + closes++; +} +void *guest_sym(const char *name) { + if (!strcmp(name, hle_force_h_ar_WorkSize.name)) return sdk_size; + if (!strcmp(name, hle_force_h_ar_Open.name)) return sdk_open_ev; + if (!strcmp(name, hle_force_h_ar_Open0.name)) return sdk_open; + if (!strcmp(name, hle_force_h_ar_Close.name)) return sdk_close; + return NULL; +} +void *objmap_get(const void *key) { return key == map_key ? map_value : NULL; } +void objmap_put(const void *key, void *value) { assert(!map_key); map_key = (void *)key; map_value = value; } +void *objmap_del(const void *key) { + if (key != map_key) return NULL; + void *value = map_value; map_key = map_value = NULL; return value; +} +void *dm_guest_thread_spawn(void (*fn)(void *), void *arg, int prio, size_t stack) { + (void)fn; (void)arg; (void)prio; (void)stack; starts++; return (void *)1; +} +void dm_guest_thread_join(void *tt) { + assert(tt == (void *)1 && !atomic_load(&g_open)); + assert(((uint8_t *)sdk_work)[SDK_WORK - 1] == 0x5a); // storage also survives the render thread + joins++; +} +uint64_t now_ns(void) { return 0; } +void *dm_pcmout_open(int ch) { (void)ch; return NULL; } +void dm_pcmout_close(void *k) { (void)k; } +void dm_pcmout_start(void *k) { (void)k; } +int dm_pcmout_append(void *k, void *b) { (void)k; (void)b; return 0; } +void *dm_pcmout_released(void *k) { (void)k; return NULL; } +void dm_pcmout_wait(void *k, int ms) { (void)k; (void)ms; } +void hle_bind_system_event(void *ev, int ac) { (void)ev; (void)ac; } + +int main(void) { + uint8_t param[P_SIZE] = {0}, saved[P_SIZE]; + const uint32_t words[] = {48000, 240, 132, 19, 138, 1, 16, 0, 0, 85, 1800, 0, 0, 0x38564552}; + memcpy(param, words, sizeof words); + memcpy(saved, param, sizeof param); + // Expected results were obtained by executing Horizon audio build 3a75b4a6's sizing routine, + // independently of this implementation. A changed count must change the service requirement. + const struct { int revision, word, value; size_t expected; } cases[] = { + {8, 4, 138, 0x61d000}, {8, 4, 137, 0x61b000}, {8, 3, 18, 0x5f7000}, + {8, 10, 1799, 0x61c000}, {8, 7, 2, 0x622000}, {9, 4, 138, 0x61e000}, + {12, 4, 138, 0x5f7000}, {14, 4, 138, 0x621000}, {15, 4, 138, 0x6a4000} + }; + for (size_t i = 0; i < sizeof cases / sizeof *cases; i++) { + memcpy(param, saved, sizeof param); + *(uint32_t *)(param + 0x34) = 0x30564552 + ((uint32_t)cases[i].revision << 24); + ((uint32_t *)param)[cases[i].word] = cases[i].value; + size_t got = h_ar_WorkSize(param); + if (got != cases[i].expected) { fprintf(stderr, "service size case %zu: expected %#zx got %#zx\n", i, cases[i].expected, got); return 1; } + } + memcpy(param, saved, sizeof param); + *(uint32_t *)(param + 0x34) = 0x34564552; + *(uint32_t *)(param + 0x24) = *(uint32_t *)(param + 0x28) = 0; + assert(h_ar_WorkSize(param) == 0x308000); // fixed command buffer and performance format 1 + // The longer SDK 8 public layout, padding bytes, and unknown future revisions. + memcpy(param, saved, sizeof param); + *(uint32_t *)(param + 0x34) = 0; + *(uint32_t *)(param + 0x38) = 0x38564552; + param[0x21] = param[0x22] = param[0x23] = 0xff; + assert(h_ar_WorkSize(param) == SERVICE_WORK); + memcpy(param, saved, sizeof param); + *(uint32_t *)(param + 0x34) = 0x40564552; + assert(h_ar_WorkSize(param) == SDK_WORK); + memcpy(param, saved, sizeof param); + assert(h_ar_WorkSize(param) == SERVICE_WORK); + void *work; assert(!posix_memalign(&work, WORK_PAGE, SDK_WORK)); + memset(work, 0xa5, SDK_WORK); + void *handle[2], *event[8]; + for (int route = 0; route < 3; route++) { + nested = route == 2; + int a = allocations, s = starts, c = closes; + Result r = route ? h_ar_Open(handle, event, param, work, SERVICE_WORK) : h_ar_Open0(handle, param, work, SERVICE_WORK); + assert(!r && allocations == a + 1 && starts == s + 1); + assert(handle[0] != work && map_value == backing); + assert(((uint8_t *)work)[0] == 0xa5 && ((uint8_t *)work)[SERVICE_WORK - 1] == 0xa5); + assert(!memcmp(param, saved, sizeof param)); + h_ar_Close(handle[0], handle[1]); + assert(!backing && !map_key && allocations == frees && closes == c + 1 && joins == starts); + } + fail_open = 1; + int s = starts; + assert(h_ar_Open0(handle, param, work, SERVICE_WORK)); + assert(!backing && !map_key && allocations == frees && starts == s); + fail_open = 0; fail_alloc = 1; + int o = opens; + assert(h_ar_Open0(handle, param, work, SERVICE_WORK)); + assert(opens == o && starts == s && !backing); + fail_alloc = 0; + // Invalid caller storage is still rejected, without giving it a host allocation. + int a = allocations; + assert(h_ar_Open0(handle, param, NULL, WORK_PAGE)); + assert(h_ar_Open0(handle, param, (uint8_t *)work + 1, WORK_PAGE)); + assert(h_ar_Open0(handle, param, work, WORK_PAGE - 1)); + assert(h_ar_Open0(handle, param, work, SERVICE_WORK - WORK_PAGE)); + assert(allocations == a && starts == s); + // An existing full-size buffer and an explicitly CPU renderer retain caller ownership. + for (int cpu = 0; cpu < 2; cpu++) { + *(int32_t *)(param + P_DEVICE) = cpu; + if (cpu) assert(h_ar_WorkSize(param) == SDK_WORK); + assert(!h_ar_Open0(handle, param, work, SDK_WORK)); + assert(handle[0] == work && allocations == a && !map_key); + h_ar_Close(handle[0], handle[1]); + } + free(work); + puts("audren work buffer: Horizon sizing fixtures, both opens, nested open, failures and close lifetime passed"); + return 0; +} diff --git a/docs/reference/audio.md b/docs/reference/audio.md index 3234f7a..187fc58 100644 --- a/docs/reference/audio.md +++ b/docs/reference/audio.md @@ -8,11 +8,12 @@ The audio renderer (`dm/host/hle_audren.c`), AudioOut, audio devices and codecs. Check whether the SDK carries the renderer: list the `sdk` module's exports for `nn::audio::server::` and `nn::audio::dsp::CommandListProcessor`. If they are there ([Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md), SDK 16), do not write a mixer: -- switch `AudioRendererParameter` to rendering device = CPU (+0x2c = 1) and execution mode = manual (+0x30 = 1) in `GetAudioRendererWorkBufferSize` and both `OpenAudioRenderer` overloads; +- switch a copy of `AudioRendererParameter` to rendering device = CPU (+0x2c = 1) and execution mode = manual (+0x30 = 1) in `GetAudioRendererWorkBufferSize` and both `OpenAudioRenderer` overloads; - call the SDK originals (`guest_sym` returns the module's own code even for a forced symbol); - drive `ExecuteAudioRendererRendering` from a guest thread (`dm_guest_thread_spawn`). The open then goes through `CreateAudioRendererManagerByDfc`; no IPC. +For a replaced DSP renderer the size query keeps the service's work-buffer contract; a larger local CPU representation is a backend cost. See [A sound engine's heap too small for its renderer and players](#a-sound-engines-heap-too-small-for-its-renderer-and-players). ### SDK 8's longer `AudioRendererParameter` @@ -62,10 +63,10 @@ The open then goes through `CreateAudioRendererManagerByDfc`; no IPC. ### A sound engine's heap too small for its renderer and players - **Symptom:** the main thread spins at start-up in a resource wait (a `YieldThread` loop, no crash), right after `OpenAudioRenderer`. -- **Cause:** Koei Tecmo's sound engine (KTSL) failed its setup quietly. The init hands it three heaps cut from one block (15 MB engine memory, 2 MB, 142 MB sound data; the sizes are constants at `main+0x8d9610`). In the 15 MB heap the setup allocates the renderer config and work buffer (0x760000: `AudioRenderSystem::GetWorkBufferSize`, 0x40a9c8 of it command buffer) and one block per player (up to 0x1b5860; three of the four mix 5.1). The fourth block came back null, the setup released its memory pools (`RequestAttachMemoryPool` then `Detach` with no update between) and returned −51, the first sound archive (ASRS) never loaded, and the resource the main thread waited for depended on it. -- **Fix:** a per-build cut moves 4 MB from the sound data heap to the engine heap (its `patches.py`). +- **Cause:** the DSP service and an application's local SDK server use different command/state representations. The host forwarded the local CPU server's work-buffer requirement as the guest's DSP-service requirement. Larger local command structures could then exhaust a fixed sound heap that fits the service's budget. Changing only the device / execution fields in the local sizing routine does not reveal this difference: it still sizes the same compiled structures. The game note records the comparison against Horizon's actual sizing code and the resulting allocation failure. +- **Fix:** `ar_service_work_size` in `hle_audren.c` computes the Horizon service budget through renderer revision 15, with revision gates for graph/splitter storage, effect states, performance frames and command storage. A DSP caller receives that size. On open the host independently asks the SDK for its CPU/manual requirement and supplies a page-aligned, zeroed backend buffer if it exceeds the caller's capacity; a buffer below the declared service requirement is rejected. Both open forms share the allocation, including a nested no-event open. Failed opens free it; close joins the render thread, closes the SDK renderer, then frees it. Explicit CPU renderers keep their SDK contract. A sufficient caller buffer is reused; revisions beyond the verified service model retain the SDK size. No firmware is loaded or distributed at runtime, and no game's heap constants change. - **Found on:** [Hyrule Warriors: Age of Calamity](../../dm/games/01002B00111A2000/README.md). -- **Note:** how it was found: probe the pump the wait calls (`main+0x8e1c0`: first missing stage bit → the type's stage handler), the handler's dependency list (an incomplete entry of type `0xbbd39f2d`, ASRS), then the memory pool states (`MemoryPoolInfo` +0x10: 1 new, 3 detached, 4 request attach, 5 attached, 6 unused) and the engine allocator's arguments and results. Why hardware fits in 15 MB is not known. +- **Check:** `make -C dm/host/tests GAME= audren-work` checks independently obtained Horizon sizing fixtures, both public parameter layouts, uninitialized padding, unsupported revisions, nested opens, undersized buffers, failure cleanup, caller-owned storage and close lifetime. The sizing implementation also matched 360 seeded parameter sets executed through Horizon's original machine code across REV1–REV15. The game note records the original-heap reproduction and audio check. ## Command lists by SDK