diff --git a/README.md b/README.md index 3a7c5ac..b80e6d6 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ The project is built from the generic parts of [d2r-mobile](../d2r-mobile) (MIT) | Game | Profile | State | |---|---|---| | Super Mario Bros. Wonder (base game v0, update 1.2.1) | `smbw` | **Playable on the iPhone with sound through several levels**: an 11-minute session at 60 fps with no crash and no audio dropouts. Scripted Mac runs reach the opening course. Notes are in the [Super Mario Bros. Wonder](#super-mario-bros-wonder-smbw) section | -| Fire Emblem Engage | `fe_engage` | Bring-up (Unity 2020.3 IL2CPP, SDK 12.3.7): **on the Mac it reaches the title screen** and, left idle, its 3D attract demo. `shader_split.py` reads the shaders out of the Unity bundles (9,268 of 9,269 compile). Movies play through a host decoder (VP9 with libvpx). Scripted input starts a new game and plays into the prologue battle (a 12-minute run ends without a crash). Not yet: iOS (the profile fits iOS's 6 GB reservation) | +| Fire Emblem Engage | `fe_engage` | Bring-up (Unity 2020.3 IL2CPP, SDK 12.3.7): **on the Mac it reaches the title screen** and, left idle, its 3D attract demo. `shader_split.py` reads the shaders out of the Unity bundles (9,268 of 9,269 compile). Movies play through a host decoder (VP9 with libvpx). Scripted input starts a new game and reaches the prologue battle map; 12-minute runs end without a crash. Not yet: iOS (the guest block is 5 GB, the total proven on a phone) | ## Status diff --git a/dm/games/fe_engage/game.tsv b/dm/games/fe_engage/game.tsv index 598bbea..4065a49 100644 --- a/dm/games/fe_engage/game.tsv +++ b/dm/games/fe_engage/game.tsv @@ -26,22 +26,22 @@ romfs_marker Data/data.unity3d zip_store .bundle:.wem:.unity3d:.metallib # ---- 内存 | Memory ---- -# NPDM 声明系统资源 16 MB,所以 IsVirtualAddressMemoryEnabled 为真:nninitStartup 把整个空闲内存(约 3.2 GB)作虚拟地址内存交给 StandardAllocator, -# 要在别名区里一次分到这么大的地址区域(较新 SDK 的 AddressRegionManager 先占别名区开头约 1 GB 作节点区)。马力欧的 2.875 GB 别名区不够(StandardAllocator::Initialize 中止)。 -# The NPDM declares 16 MB of system resource, so IsVirtualAddressMemoryEnabled is true: nninitStartup hands all free memory (about 3.2 GB) to StandardAllocator as virtual address memory, -# which needs one address region that large in the alias region (newer SDKs' AddressRegionManager first takes about 1 GB at the start of the alias region as its node area). Mario's 2.875 GB alias region is too small (StandardAllocator::Initialize aborts). -# Engage 不调 SetHeapSize:全部内存经别名区(MapPhysicalMemory,实测映射一直用到别名区顶端),线程栈在栈区里随机放。所以堆区只留 128 MB, -# 整块 6 GB = 前置 256 MB + 别名区 5.375 GB + 栈区 256 MB + 堆区 128 MB,正好是 iOS 不带「扩展虚拟寻址」授权时单块预留的上限(原先 Mac 起机用 8 GB、别名区 5.5 GB) -# Engage never calls SetHeapSize: all its memory goes through the alias region (MapPhysicalMemory; measured mappings reach the top of the alias region), and thread stacks are placed at random in the stack region. So the heap region keeps only 128 MB, -# and the 6 GB block = 256 MB in front + 5.375 GB alias region + 256 MB stack region + 128 MB heap region, exactly the largest single reservation iOS allows without the "extended virtual addressing" entitlement (Mac bring-up used 8 GB with a 5.5 GB alias region) -# 别名区的地址也是 Unity 的:它的低层虚拟分配器自己 AllocateAddressRegion(每块按尺寸对齐,2.25 MB 占 4 MB、16.25 MB 占 20 MB),序章战斗前已占约 1.2 GB。 -# 节点区 1 GB 再占掉就不够了,所以 patch_nso.py 给本作 sdk 构建登记了剪口,把节点区缩到 62 MB(真机别名区 64 GB,不在乎这点)。 -# The alias region's address space is Unity's too: its low-level virtual allocator calls AllocateAddressRegion itself (each block aligned by size: 2.25 MB takes 4 MB, 16.25 MB takes 20 MB), about 1.2 GB before the prologue battle. -# With the 1 GB node area taken as well that is not enough, so patch_nso.py registers a cut for this game's sdk build that shrinks the node area to 62 MB (the hardware's alias region is 64 GB and does not care). -mem_reserve 0x180000000 -mem_alias 0x158000000 +# NPDM 声明系统资源 16 MB,所以 IsVirtualAddressMemoryEnabled 为真:nninitStartup 把整个空闲内存(应用池 − 64 MB)作虚拟地址内存交给 StandardAllocator, +# 在别名区里一次分一整块地址区域(按 1 GB 对齐)。Unity 的低层虚拟分配器也自己 AllocateAddressRegion(每块按尺寸对齐,2.25 MB 占 4 MB、16.25 MB 占 20 MB), +# 序章战斗时约 1.05 GB。较新 SDK 的 AddressRegionManager 先占别名区开头约 1 GB 作节点区;patch_nso.py 给本作 sdk 构建登记了剪口,把它缩到 62 MB(真机别名区 64 GB,不在乎这点)。 +# The NPDM declares 16 MB of system resource, so IsVirtualAddressMemoryEnabled is true: nninitStartup hands all free memory (application pool − 64 MB) to StandardAllocator as virtual address memory, +# one address region in the alias region (1 GB aligned). Unity's low-level virtual allocator also calls AllocateAddressRegion itself (each block aligned by size: 2.25 MB takes 4 MB, 16.25 MB takes 20 MB), +# about 1.05 GB in the prologue battle. Newer SDKs' AddressRegionManager first takes about 1 GB at the start of the alias region as its node area; patch_nso.py registers a cut for this game's sdk build that shrinks it to 62 MB (the hardware's alias region is 64 GB and does not care). +# 实测(序章到战斗,12 分钟):StandardAllocator 只提交 0.58 GB,Unity 自己的区域提交 0.97 GB,合计 1.55 GB。所以应用池报 2.5 GB(StandardAllocator 约 2.44 GB,是用量的 4 倍), +# 整块就能放进 5 GB = 前置 256 MB + 别名区 4.375 GB + 栈区 256 MB + 堆区 128 MB(Engage 不调 SetHeapSize,线程栈在栈区里随机放)。5 GB 是真机跑通过的总量(D2R); +# iOS 不带「扩展虚拟寻址」授权时单块最多 6 GB,但 Metal 等还要地址(Mac 上 Engage 的显存分配约 1 GB),原先的 6 GB 块在手机上没有余量。 +# Measured (prologue to the battle, 12 minutes): StandardAllocator commits only 0.58 GB and Unity's own regions 0.97 GB, 1.55 GB in all. So the pool is reported as 2.5 GB (StandardAllocator about 2.44 GB, four times its use), +# and the block fits in 5 GB = 256 MB in front + 4.375 GB alias region + 256 MB stack region + 128 MB heap region (Engage never calls SetHeapSize; thread stacks are placed at random in the stack region). 5 GB is the total proven on a phone (D2R); +# without the "extended virtual addressing" entitlement iOS allows one reservation of at most 6 GB, but Metal and the rest need address space too (Engage allocates about 1 GB of GPU memory on the Mac), so the former 6 GB block left no margin on a phone. +mem_reserve 0x140000000 +mem_alias 0x118000000 mem_stack 0x10000000 -app_pool 0xCD500000 +app_pool 0xA0000000 # ---- NVN 驱动常量段(dmsh_build.py 出包、nvn_mtl.m 运行时现填;按全库着色器实测) | NVN driver constant segments (dmsh_build.py at packaging, nvn_mtl.m filled at run time; measured over all shaders) ---- # 旧驱动布局(同 D2R):c2 贴图句柄表 0x168 + 8 r(128 槽)、分离采样器表 0x568 + 8 r;Unity 只用分离贴图 / 采样器(BindSeparateTexture / Sampler)。图像表与 c0 存储缓冲表未实测,先沿用 diff --git a/docs/agents/03-bringup.md b/docs/agents/03-bringup.md index e527cc6..be0914d 100644 --- a/docs/agents/03-bringup.md +++ b/docs/agents/03-bringup.md @@ -41,7 +41,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi - **C++ exception aborts in `nn::ro::detail::GetExceptionInfo`** → the SDK's unwinder finds a module from `svcQueryMemory`: it walks back over Code+RX regions to the module start, forward over Code then CodeData, and reads `MOD0` for `.eh_frame_hdr`. The host must report each loaded module's real layout and report other host memory as free (`hle_svc.c`). IL2CPP throws a C++ exception for every managed exception, so Unity games need this; Mario never threw. - **First `AllocateAddressRegion` fails with 0x1803** → SDK 16's AddressRegionManager takes 0x3fe00000 at the start of the alias region as a node area → the alias region must be larger than 1 GB. - **A null-based write minutes into play, at the pool size minus 0x20 (Engage: `addr=0x1fffe0` at `main+0x9a4664`)** → Unity's TLSF heap added a pool, and its low-level virtual allocator (vtable at `main+0x61ffeb0`; reserve `main+0x9aa0f4` → `nn::os::AllocateAddressRegion`) got no address region and returned 0. The alias region ran out of *address space*, not memory: the AddressRegionManager aligns each region by its size (64 KB under 2 MB, 2 MB under 4 MB, 4 MB under 32 MB, 32 MB under 1 GB, else 1 GB) and Unity asks for its block plus alignment slack, so a 2.25 MB block takes a 4 MB slot and a 16.25 MB block a 20 MB one. At the crash Engage held 4.17 of the 4.375 GB left after the node area (StandardAllocator 3.14 GB, Unity about 1.05 GB, the rest fragments). On hardware the alias region is 64 GB. → shrink the node area with a rewriter cut for that SDK build (`patch_nso.py`, Engage: the two `mov w8/w9, #0x3fe00000` in the constructor → 62 MB). To measure, probe the SDK's `VammManager::AllocateAddressRegion` at its success path (`ReleaseWriteLock` return: x21 = size, x22 = address) and `FreeAddressRegion` after its prologue (x1 = address), and sum the live regions by caller. -- **Sizing the reservation for a title that uses virtual address memory (Engage)** → such a title may never call `SetHeapSize`: everything comes from `MapPhysicalMemory` in the alias region. The `[内存]` log lines report the alias high-water mark (one line per 256 MB) and every heap growth. Engage's mappings reach +5.26 GB, so its profile uses a 5.375 GB alias region and a 128 MB heap region, 6 GB in all, iOS's single-reservation limit without the extended-virtual-addressing entitlement. +- **Sizing the reservation for a title that uses virtual address memory (Engage)** → such a title may never call `SetHeapSize`: everything comes from `MapPhysicalMemory` in the alias region. The `[内存]` log lines report the alias high-water mark (one line per 256 MB), the total actually mapped (page bitmap) and every heap growth. What fills the alias region first is address space, not memory: StandardAllocator reserves the whole free pool (pool − 64 MB, 1 GB aligned) but Engage commits only 0.58 GB of it, and Unity's own regions commit 0.97 GB. So Engage's profile reports a 2.5 GB pool (`app_pool`) and uses a 4.375 GB alias region in a 5 GB block (reserved address space peaks at 3.47 GB of 4.31 GB). iOS without the extended-virtual-addressing entitlement allows a single 6 GB reservation, but Metal needs address space too; 5 GB is the total proven on a phone. To measure commits, probe `VammManager::AllocateMemoryPages` / `FreeMemoryPages` (x1 address, x2 size) and split them by region. - **Large resources silently skipped, later a null deref where the resource is used** → the game reserves "virtual address memory" (Mario: 1.75 GB in `HeapDef.byml`) and maps it into the alias region with `MapPhysicalMemory` on demand; the alias region left after the node area was too small. Also visible as `SetHeapSize` asking for much less than the pool. → repartition the guest block (`hle_svc.c` `mem_init`): Mario uses alias 2.875 GB, stacks 256 MB, heap 1.625 GB (it asks for 1.47 GB), 5 GB in total. Measure before sizing (`DM_SVC=1`: the `SetHeapSize` request and the highest `MapPhysicalMemory` address). On iOS without the extended-virtual-addressing entitlement a single 6 GB reservation succeeds and 7 GB fails, but Metal and the rest of the process need address space too; 5 GB is the total proven on a phone (D2R). - **`MapPhysicalMemory` faults on part of a page** → round to 16 KB host pages. - **Some textures are re-uploaded every frame (several MB per frame)** → the dirty-page tracker registers each memory pool rounded to 16 KB host pages, but NVN pools are only 4 KB aligned, so neighbouring pools share their first / last host page and one address can lie in several registered ranges. Taking the first range that contains an address judged textures in the second pool untracked (always dirty), released only one range on a write fault, and could unregister the neighbour when a pool was destroyed. `nvn_dirty.c` now uses the range that holds the whole span, releases every range covering a faulting page, and removes a range by the pool's exact start (`t_dirty_slots` covers it). diff --git a/docs/agents/04-graphics.md b/docs/agents/04-graphics.md index e09decf..cc6b0e2 100644 --- a/docs/agents/04-graphics.md +++ b/docs/agents/04-graphics.md @@ -72,7 +72,7 @@ Consequence: any optimization that infers "which textures a draw reads" from bou - **Video**: Unity's `Hidden/VideoDecode` samples luma as `.w` of an `R8` texture with the swizzle `(0, 0, 0, R)` (Unity's Alpha8) and chroma from an `RG8` texture at half size. To check the movie path apart from the decoder, fill the frames with a gradient in `hle_movie.c`; it should appear on screen. - **One geometry shader** (in `unity_builtin_extra`): Metal has no geometry stage, so it fails to compile (`unknown main0_out`); nothing in the opening scenes uses it. - **Fixed: a crash a few minutes into the prologue battle (scripted run, about 357 s).** It wrote to `NULL + 0x1fffe0` at `main+0x9a4664`: Unity's TLSF heap allocator (`main+0x9a45d0`) got no address region for a new 2 MB pool. The alias region was out of address space, not memory; see [03](03-bringup.md), "A null-based write minutes into play". A 12-minute scripted run now plays through the battle. -- **Open: about one run in four stalled early** (around the opening movie or the title; no log output, not even the time-limit report). Sample the process when its log stops growing (`sample 3`) to see where every thread waits. The crash handler now writes one line with `write()` and exits if the stderr lock stays taken for 1 s, so a stall that holds that lock shows up as such. +- **Probably fixed: about one run in four stalled early** (around the opening movie or the title; no log output, not even the time-limit report). The stalls happened before the real thread pause for IL2CPP's GC ([03](03-bringup.md)); since then 12 runs (eight of them 200 s runs through the movie and title, four of 330 to 850 s) have not stalled once, where one in four used to. Without the pause the collector ran while other threads kept changing the heap. If a stall comes back, sample the process when its log stops growing (`sample 3`) to see where every thread waits; the crash handler now writes one line with `write()` and exits if the stderr lock stays taken for 1 s, so a stall that holds that lock shows up as such. - **Fixed: the idle attract demo crashed after about 220 s.** Unity's particle geometry jobs wrote through a NULL index pointer on several job threads at once (`main+0xdb0ad4`, `0xdaea58`, `0xdadb5c`). The scheduler (`main+0xd91e0c`) takes its vertex and index chunks from a fallback that creates and maps a fresh buffer each time (`main+0x7b9c64`; Unity's Switch renderer always takes it, as on hardware), and from about 180 s 32-bit mesh indices also turned up inside Unity's texture objects. The cause was the alias region running out of address space, as in the battle crash ([03](03-bringup.md), "A null-based write minutes into play"): past 2 MB the index buffers got no mapping. With the node-area cut an 850 s idle run is clean, with no corrupted texture objects; with the old SDK the same run dies at 159 s in Unity's heap allocator. Reproduce the old state: `boot.sh 245` with no input on an SDK rewritten without the cut. ## 7. Diagnostic switches (host)