Engage: shrink the SDK's address-region node area; the battle crash is gone

- The crash about 6 minutes into the prologue battle (null write at
  main+0x9a4664) was the alias region running out of address space, not
  memory: Unity's low-level virtual allocator got no region from
  AllocateAddressRegion and returned 0. The SDK's AddressRegionManager
  keeps the first 1 GB of the alias region as its node area and aligns
  each region by size; after StandardAllocator's 3.14 GB block only about
  1.2 GB was left for Unity, and it was full and fragmented.
- patch_nso.py: a host-only cut for Engage's sdk build (56ec2474…)
  shrinks the node area from 0x3fe00000 to 0x3e00000 (62 MB, about 1.35
  million nodes). Both constructor movs change together.
- A 12-minute scripted run now plays through the battle without a crash;
  reserved address space levels off at 4.17 of 5.31 GB.
- hle_svc.c: the alias-mapped total counts pages with a bitmap, so maps
  of already-mapped pages and oversized unmaps no longer skew it (it went
  negative). Engage really maps about 1.5 GB, not 0.5 GB.
- Docs: bring-up catalogue entry with the measuring recipe, profile
  memory notes, README Engage row. Mario regression t64 clean; the iOS
  project compiles.

README.md

@@ -12,7 +12,7 @@ The project is built from the generic parts of [d2r-mobile](../d2r-mobile) (MIT)
 | Game | Profile | State |
 |---|---|---|
 | Super Mario Bros. Wonder (base game v0, update 1.2.1) | `smbw` | **Playable on the iPhone with sound through several levels**: an 11-minute session at 60 fps with no crash and no audio dropouts. Scripted Mac runs reach the opening course. Notes are in the [Super Mario Bros. Wonder](#super-mario-bros-wonder-smbw) section |
-| Fire Emblem Engage | `fe_engage` | Bring-up (Unity 2020.3 IL2CPP, SDK 12.3.7): **on the Mac it reaches the title screen** and, left idle, its 3D attract demo. `shader_split.py` reads the shaders out of the Unity bundles (9,268 of 9,269 compile). Movies play through a host decoder (VP9 with libvpx). Scripted input starts a new game and reaches the prologue cutscene and first battle map. Not yet: a heap-allocator crash a few minutes into that battle, a crash in the attract demo, iOS (the profile now fits iOS's 6 GB reservation) |
+| Fire Emblem Engage | `fe_engage` | Bring-up (Unity 2020.3 IL2CPP, SDK 12.3.7): **on the Mac it reaches the title screen** and, left idle, its 3D attract demo. `shader_split.py` reads the shaders out of the Unity bundles (9,268 of 9,269 compile). Movies play through a host decoder (VP9 with libvpx). Scripted input starts a new game and plays into the prologue battle (a 12-minute run ends without a crash). Not yet: a crash in the attract demo, iOS (the profile now fits iOS's 6 GB reservation) |
 
 ## Status
 

dm/games/fe_engage/game.tsv

@@ -34,6 +34,10 @@ zip_store	.bundle:.wem:.unity3d:.metallib
 # 整块 6 GB = 前置 256 MB + 别名区 5.375 GB + 栈区 256 MB + 堆区 128 MB，正好是 iOS 不带「扩展虚拟寻址」授权时单块预留的上限（原先 Mac 起机用 8 GB、别名区 5.5 GB）
 # Engage never calls SetHeapSize: all its memory goes through the alias region (MapPhysicalMemory; measured mappings reach the top of the alias region), and thread stacks are placed at random in the stack region. So the heap region keeps only 128 MB,
 # and the 6 GB block = 256 MB in front + 5.375 GB alias region + 256 MB stack region + 128 MB heap region, exactly the largest single reservation iOS allows without the "extended virtual addressing" entitlement (Mac bring-up used 8 GB with a 5.5 GB alias region)
+# 别名区的地址也是 Unity 的：它的低层虚拟分配器自己 AllocateAddressRegion（每块按尺寸对齐，2.25 MB 占 4 MB、16.25 MB 占 20 MB），序章战斗前已占约 1.2 GB。
+# 节点区 1 GB 再占掉就不够了，所以 patch_nso.py 给本作 sdk 构建登记了剪口，把节点区缩到 62 MB（真机别名区 64 GB，不在乎这点）。
+# The alias region's address space is Unity's too: its low-level virtual allocator calls AllocateAddressRegion itself (each block aligned by size: 2.25 MB takes 4 MB, 16.25 MB takes 20 MB), about 1.2 GB before the prologue battle.
+# With the 1 GB node area taken as well that is not enough, so patch_nso.py registers a cut for this game's sdk build that shrinks the node area to 62 MB (the hardware's alias region is 64 GB and does not care).
 mem_reserve	0x180000000
 mem_alias	0x158000000
 mem_stack	0x10000000

dm/host/hle_svc.c

@@ -28,6 +28,7 @@ static uint8_t *g_heap; static size_t g_heap_cur, g_heap_cap;
 // The address range lies entirely within the currently opened (readable and writable) part of the guest heap: such reads need no kernel trap to check, memcpy directly (used where guest memory is read every tick, such as diag.c tk_rd)
 int dm_guest_heap_readable(uint64_t a, size_t n) { uint64_t b = (uint64_t)(uintptr_t)g_heap; size_t cur = g_heap_cur; return g_heap && a >= b && n <= cur && a - b <= cur - n; }
 static uint8_t *g_alias; static size_t g_alias_cap;
+static _Atomic uint64_t *g_alias_bits;   // 别名区页位图（alias_pages） | alias region page bitmap (alias_pages)
 static uint8_t *g_stack; static size_t g_stack_cap;
 
 static void mem_init(void) {
@@ -49,6 +50,7 @@ static void mem_init(void) {
     g_alias = g_as + 0x10000000; g_alias_cap = DM_GAME_MEM_ALIAS;
     g_stack = g_alias + g_alias_cap; g_stack_cap = DM_GAME_MEM_STACK;
     g_heap  = g_stack + g_stack_cap; g_heap_cap  = RESERVE - (size_t)(g_heap - g_as);
+    if (!(g_alias_bits = calloc((g_alias_cap >> 18) + 1, sizeof *g_alias_bits))) die("别名区页位图 calloc");
     LOG("guest AS %p +%#llx  heap %p alias %p stack %p", g_as, (unsigned long long)RESERVE, g_heap, g_alias, g_stack);
 }
 
@@ -92,6 +94,19 @@ static void map_add(uint64_t b, uint64_t n, uint64_t src, uint32_t st) {
 }
 static void map_del(int i) { pthread_mutex_lock(&g_map_lock); g_map[i] = g_map[--g_nmap]; pthread_mutex_unlock(&g_map_lock); }
 
+// 别名区每个 4 KB 页一位（已映射 = 1）：Map 置位、Unmap 清位，返回状态真变了的页数（带符号：Map 为正、Unmap 为负）
+// One bit per 4 KB page of the alias region (mapped = 1): Map sets bits, Unmap clears them; returns the number of pages whose state really changed (signed: positive for Map, negative for Unmap)
+static int64_t alias_pages(uint64_t off, uint64_t n, int map) {
+    int64_t ch = 0;
+    for (uint64_t p = off >> 12, e = (off + n + 0xFFF) >> 12; p < e; ) {
+        uint64_t sh = p & 63, k = 64 - sh < e - p ? 64 - sh : e - p, m = (k == 64 ? ~0ull : (1ull << k) - 1) << sh;
+        uint64_t old = map ? atomic_fetch_or(&g_alias_bits[p >> 6], m) : atomic_fetch_and(&g_alias_bits[p >> 6], ~m);
+        ch += __builtin_popcountll((map ? ~old : old) & m);
+        p += k;
+    }
+    return map ? ch : -ch;
+}
+
 int svc_thread(int num, uint64_t *regs);   // hle_thread.c：线程与同步类 | hle_thread.c: thread and synchronization kind
 
 uint64_t svc_dispatch(int num, uint64_t *regs, uint64_t orig_addr) {
@@ -205,8 +220,11 @@ uint64_t svc_dispatch(int num, uint64_t *regs, uint64_t orig_addr) {
         if (num == 0x2C) { static _Atomic uint64_t hw; uint64_t end = (uint64_t)(a + size - g_alias), old = atomic_load(&hw);
             while (end > old && !atomic_compare_exchange_weak(&hw, &old, end)) {}
             if (end > old && end >> 28 != old >> 28) LOG("[内存] 别名区映射到 +%.2f GB（别名区 %.2f GB）", end / 1073741824.0, g_alias_cap / 1073741824.0); }
-        // 别名区实际映射的总量（每跨 256 MB 记一行，涨跌都记）：看游戏用了多少物理内存、有没有一直涨 | total actually mapped in the alias region (one line per 256 MB crossed, up or down): how much physical memory the game uses and whether it keeps growing
-        { static _Atomic int64_t tot; int64_t d = num == 0x2C ? (int64_t)size : -(int64_t)size, before = atomic_fetch_add(&tot, d), after = before + d;
+        // 别名区实际映射的总量（每跨 256 MB 记一行，涨跌都记）：看游戏用了多少物理内存、有没有一直涨。按页位图计，只数状态真变了的页——
+        // SDK 会对已映射的页再 Map、对比映射更大的范围 Unmap（Engage 先解 0x1000000 再解同址 0x103f000），按请求大小计会算成负数
+        // Total actually mapped in the alias region (one line per 256 MB crossed, up or down): how much physical memory the game uses and whether it keeps growing. Counted with a page bitmap, only pages whose state really changed --
+        // the SDK maps pages that are already mapped and unmaps ranges larger than what it mapped (Engage unmaps 0x1000000 and then 0x103f000 at the same address), so counting requested sizes went negative
+        { static _Atomic int64_t tot; int64_t d = alias_pages((uint64_t)(a - g_alias), size, num == 0x2C) << 12, before = atomic_fetch_add(&tot, d), after = before + d;
           if (before >> 28 != after >> 28) LOG("[内存] 别名区已映射 %.2f GB（应用池 %.2f GB）", after / 1073741824.0, APP_POOL / 1073741824.0); }
         regs[0] = 0; return 0;
     }

dm/tools/patch_nso.py

@@ -164,7 +164,22 @@ CUTS = {
     (0x5d8040, 'a005fe97', '411100d4', '玩家进局取出生点调用点 bl 0x5596c0 → svc #0x8a：宿主原样代调，进局在城镇时出生点换成小站旁（hle.h SVC_HOST_JOIN_SPAWN）', '仅宿主'),
     (0x5d8400, 'b004fe97', '611100d4', '进局给尸体取落点调用点 bl 0x5596c0 → svc #0x8b：同上，尸体与人取同一个点（hle.h SVC_HOST_JOIN_CORPSE）', '仅宿主'),
     (0x11f0914, '77100094', '811000d4', '段读取 0x11f07d0 逐块解码 bl 0x11f4af0(ctx, 块, 长度) → svc #0x84：同上（hle.h SVC_HOST_GRANNY_B）', '仅宿主'),
-]}
+  ],
+  # 火焰纹章 Engage 2.0.0（v327680）sdk：AddressRegionManager 节点区 0x3fe00000（约 1 GB）→ 0x3e00000（62 MB）。
+  # 构造函数（sdk+0x346420）在别名区开头划出节点区（0x30 字节一个节点的单元堆，用到才映射），其后才给 AllocateAddressRegion 分区域。
+  # 真机别名区 64 GB，这 1 GB 无所谓；宿主别名区受 iOS 单块预留 6 GB 所限只有 5.375 GB，StandardAllocator 一块 3.14 GB（按 1 GB 对齐）之后
+  # Unity 的低层虚拟分配器（main+0x9aa0f4 → AllocateAddressRegion）只剩约 1.2 GB，序章战斗约 357 s 时分不到 → 返回 0 → main+0x9a4664 写空崩溃。
+  # 62 MB 够约 135 万个节点（实测活着的区域几百个）。两条 mov 必须一起改：+0x346430 定节点区上限，+0x3464cc 把它登记成已占区域。
+  # Fire Emblem Engage 2.0.0 (v327680) sdk: AddressRegionManager node area 0x3fe00000 (about 1 GB) → 0x3e00000 (62 MB).
+  # The constructor (sdk+0x346420) carves the node area out of the start of the alias region (a unit heap of 0x30-byte nodes, mapped only as used) before AllocateAddressRegion hands out regions.
+  # The hardware alias region is 64 GB, so the 1 GB does not matter there; the host's alias region is only 5.375 GB (iOS caps a single reservation at 6 GB), and after StandardAllocator's 3.14 GB block (1 GB aligned)
+  # Unity's low-level virtual allocator (main+0x9aa0f4 → AllocateAddressRegion) had only about 1.2 GB left; about 357 s into the prologue battle it got no region → returned 0 → null write crash at main+0x9a4664.
+  # 62 MB holds about 1.35 million nodes (a few hundred regions are live in measurements). Both movs must change together: +0x346430 sets the node area limit, +0x3464cc registers it as an occupied region.
+  '56ec24747ec9e7f19b3efb062483a19d': [
+    (0x346430, '08fca752', '087ca052', 'AddressRegionManager 构造：节点区上限 mov w8,#0x3fe00000 → mov w8,#0x3e00000', '仅宿主'),
+    (0x3464cc, '09fca752', '097ca052', 'AddressRegionManager 构造：节点区登记为已占区域的长度 mov w9,#0x3fe00000 → mov w9,#0x3e00000', '仅宿主'),
+  ],
+}
 # ---- 已登记观测门：{NSO 构建号: [(偏移, 编号, 原指令字, 理由)]}。编号与宿主 hle.h 的 OBS_* 一致；
 # 原指令须与位置无关（不得 PC 相对 / 分支 / 涉 x18·tpidr），离线断言；宿主未挂回调时门块进出一次即返回 ----
 # ---- Registered observation gates: {NSO build ID: [(offset, number, original instruction word, reason)]}. Numbers match OBS_* in the host's hle.h;

docs/agents/03-bringup.md

@@ -40,6 +40,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
 - **`StandardAllocator::Initialize` aborts in `nninitStartup`** → the NPDM declares a system resource (offset 0x14 of `META`), so `IsVirtualAddressMemoryEnabled` is true and the allocator reserves all free memory (about 3.2 GB for Engage) as one region in the alias region, after the ~1 GB node area → the profile's `mem_alias` / `mem_reserve` must hold it (Engage: 5.5 / 8 GB on the Mac).
 - **C++ exception aborts in `nn::ro::detail::GetExceptionInfo`** → the SDK's unwinder finds a module from `svcQueryMemory`: it walks back over Code+RX regions to the module start, forward over Code then CodeData, and reads `MOD0` for `.eh_frame_hdr`. The host must report each loaded module's real layout and report other host memory as free (`hle_svc.c`). IL2CPP throws a C++ exception for every managed exception, so Unity games need this; Mario never threw.
 - **First `AllocateAddressRegion` fails with 0x1803** → SDK 16's AddressRegionManager takes 0x3fe00000 at the start of the alias region as a node area → the alias region must be larger than 1 GB.
+- **A null-based write minutes into play, at the pool size minus 0x20 (Engage: `addr=0x1fffe0` at `main+0x9a4664`)** → Unity's TLSF heap added a pool, and its low-level virtual allocator (vtable at `main+0x61ffeb0`; reserve `main+0x9aa0f4` → `nn::os::AllocateAddressRegion`) got no address region and returned 0. The alias region ran out of *address space*, not memory: the AddressRegionManager aligns each region by its size (64 KB under 2 MB, 2 MB under 4 MB, 4 MB under 32 MB, 32 MB under 1 GB, else 1 GB) and Unity asks for its block plus alignment slack, so a 2.25 MB block takes a 4 MB slot and a 16.25 MB block a 20 MB one. At the crash Engage held 4.17 of the 4.375 GB left after the node area (StandardAllocator 3.14 GB, Unity about 1.05 GB, the rest fragments). On hardware the alias region is 64 GB. → shrink the node area with a rewriter cut for that SDK build (`patch_nso.py`, Engage: the two `mov w8/w9, #0x3fe00000` in the constructor → 62 MB). To measure, probe the SDK's `VammManager::AllocateAddressRegion` at its success path (`ReleaseWriteLock` return: x21 = size, x22 = address) and `FreeAddressRegion` after its prologue (x1 = address), and sum the live regions by caller.
 - **Sizing the reservation for a title that uses virtual address memory (Engage)** → such a title may never call `SetHeapSize`: everything comes from `MapPhysicalMemory` in the alias region. The `[内存]` log lines report the alias high-water mark (one line per 256 MB) and every heap growth. Engage's mappings reach +5.26 GB, so its profile uses a 5.375 GB alias region and a 128 MB heap region, 6 GB in all, iOS's single-reservation limit without the extended-virtual-addressing entitlement.
 - **Large resources silently skipped, later a null deref where the resource is used** → the game reserves "virtual address memory" (Mario: 1.75 GB in `HeapDef.byml`) and maps it into the alias region with `MapPhysicalMemory` on demand; the alias region left after the node area was too small. Also visible as `SetHeapSize` asking for much less than the pool. → repartition the guest block (`hle_svc.c` `mem_init`): Mario uses alias 2.875 GB, stacks 256 MB, heap 1.625 GB (it asks for 1.47 GB), 5 GB in total. Measure before sizing (`DM_SVC=1`: the `SetHeapSize` request and the highest `MapPhysicalMemory` address). On iOS without the extended-virtual-addressing entitlement a single 6 GB reservation succeeds and 7 GB fails, but Metal and the rest of the process need address space too; 5 GB is the total proven on a phone (D2R).
 - **`MapPhysicalMemory` faults on part of a page** → round to 16 KB host pages.