Host: real thread pause for IL2CPP's GC; Engage reaches its first battle
- SetThreadActivity and GetThreadContext3 were placeholders that only reported success. IL2CPP's garbage collector pauses each thread and reads its registers for roots, so threads kept running and roots held in registers were missed. A thread inside an SVC now counts as paused at once with the gate frame as its context (thr_svc_enter / leave around every gate SVC; it waits at the SVC exit while paused). A thread in guest code is stopped with SIGUSR2 and parks in the handler. The context is Horizon's 0x320-byte ThreadContext. - nn::pctl::IsFreeCommunicationAvailable answers "not available" like the other free-communication calls; the SDK aborted in sm when Engage entered its first battle map. - Memory log lines: heap growth, alias-region high-water mark and the total mapped in the alias region (one line per 256 MB). - Engage profile: a 6 GB reservation (5.375 GB alias region, 128 MB heap region), iOS's single-reservation limit without the extended virtual addressing entitlement. Engage never calls SetHeapSize; its mappings reach +5.26 GB. - Engage on the Mac: with scripted input and DM_KBD answering the name keyboard, a new game reaches the prologue cutscene and first battle map. Open: a heap-allocator crash a few minutes into that battle, and occasional early stalls (docs/agents/04-graphics.md). - Docs: catalogue entries, README rows. Mario 1.2.1 regression (120 s scripted) clean; the iOS project compiles.
This commit is contained in:
seedds committed
1 parent
eecc7fa168
commit
72aa5ca40b
8 files changed
+135
-10
No files matched your search
@@ -12,7 +12,7 @@ The project is built from the generic parts of [d2r-mobile](../d2r-mobile) (MIT)
|
||||
| Game | Profile | State |
|
||||
|---|---|---|
|
||||
| Super Mario Bros. Wonder (base game v0, update 1.2.1) | `smbw` | **Playable on the iPhone with sound through several levels**: an 11-minute session at 60 fps with no crash and no audio dropouts. Scripted Mac runs reach the opening course. Notes are in the [Super Mario Bros. Wonder](#super-mario-bros-wonder-smbw) section |
|
||||
| Fire Emblem Engage | `fe_engage` | Bring-up (Unity 2020.3 IL2CPP, SDK 12.3.7): **on the Mac it reaches the title screen** and, left idle, its 3D attract demo. `shader_split.py` reads the shaders out of the Unity bundles (9,268 of 9,269 compile). Movies play through a host decoder (VP9 with libvpx), and the menus respond to the controller as far as a new game's appearance choice. Not yet: a crash in the attract demo, some render targets and textures, iOS |
|
||||
| Fire Emblem Engage | `fe_engage` | Bring-up (Unity 2020.3 IL2CPP, SDK 12.3.7): **on the Mac it reaches the title screen** and, left idle, its 3D attract demo. `shader_split.py` reads the shaders out of the Unity bundles (9,268 of 9,269 compile). Movies play through a host decoder (VP9 with libvpx). Scripted input starts a new game and reaches the prologue cutscene and first battle map. Not yet: a heap-allocator crash a few minutes into that battle, a crash in the attract demo, iOS (the profile now fits iOS's 6 GB reservation) |
|
||||
|
||||
## Status
|
||||
|
||||
@@ -22,7 +22,7 @@ The project is built from the generic parts of [d2r-mobile](../d2r-mobile) (MIT)
|
||||
| Mac host | Runs Mario through the menus into the opening course: the course, effects and HUD render. The title screen runs at 60 fps with flat memory (about 900 MB physical footprint). Menu and dialogue text currently render blank on the Mac (they are fine on the iPhone); this is not investigated yet |
|
||||
| Shaders | `shader_split.py` extracts the shaders from Nintendo's containers (bfsha archives, particle effects, `agl` packages, UI layouts, shaders embedded in `main`) and from Unity's `Shader` objects. They are translated offline, and each vertex shader gets the outputs its paired pixel shader reads. For Mario, 16,266 of 16,268 shaders compile; 2 `agl` post-effect compute shaders make SPIRV-Cross throw. All 14,603 pairs link |
|
||||
| Audio | The game's own SDK renderer runs in-process in its CPU / manual-execution mode (`hle_audren.c`). The host renders one 5 ms frame per buffer the output hands back. It sends the device-sink mix (5.1 folded to stereo) through Core Audio (`DM_AUDIO=1` on Mac, default on iOS). This costs about 0.1 ms of CPU per frame on an M-series Mac |
|
||||
| Input | On iOS, a touch gamepad and physical controllers. On the Mac, scripted input (`DM_KEYS=<seconds>:<button>,…`) or `DM_WINDOW=1` for a window with keyboard input |
|
||||
| Input | On iOS, a touch gamepad and physical controllers. On the Mac, scripted input (`DM_KEYS=<seconds>:<button>,…`, with `DM_KBD=<text>` answering the system keyboard) or `DM_WINDOW=1` for a window with keyboard input |
|
||||
| iOS bundle | `build_ipa.sh` builds one unsigned IPA for iPhone and iPad. The game code is embedded in the executable, and the iOS shader libraries and the RomFS are inside the app |
|
||||
| Known gaps | Partial / mid-pass clears; sparse virtual-pool mappings; conditional render; `DrawElementsIndirect`. TBDR optimizations are off by default because they don't understand bindless textures yet. There is no amiibo (no NFC reader). The bundled prewarm list is empty, so the first run after an install stutters while pipelines compile (see [Prewarm lists](#prewarm-lists)) |
|
||||
|
||||
|
||||
@@ -28,12 +28,14 @@ zip_store .bundle:.wem:.unity3d:.metallib
|
||||
# ---- 内存 | Memory ----
|
||||
# NPDM 声明系统资源 16 MB，所以 IsVirtualAddressMemoryEnabled 为真：nninitStartup 把整个空闲内存（约 3.2 GB）作虚拟地址内存交给 StandardAllocator，
|
||||
# 要在别名区里一次分到这么大的地址区域（较新 SDK 的 AddressRegionManager 先占别名区开头约 1 GB 作节点区）。马力欧的 2.875 GB 别名区不够（StandardAllocator::Initialize 中止）。
|
||||
# 先给 Mac 起机用：预留 8 GB、别名区 5.5 GB。iOS 没有「扩展虚拟寻址」授权时单块预留最多 6 GB，到时再定（授权，或少报内存池）
|
||||
# The NPDM declares 16 MB of system resource, so IsVirtualAddressMemoryEnabled is true: nninitStartup hands all free memory (about 3.2 GB) to StandardAllocator as virtual address memory,
|
||||
# which needs one address region that large in the alias region (newer SDKs' AddressRegionManager first takes about 1 GB at the start of the alias region as its node area). Mario's 2.875 GB alias region is too small (StandardAllocator::Initialize aborts).
|
||||
# For Mac bring-up for now: an 8 GB reservation with a 5.5 GB alias region. On iOS without the "extended virtual addressing" entitlement a single reservation is at most 6 GB; decide then (the entitlement, or a smaller reported pool)
|
||||
mem_reserve 0x200000000
|
||||
mem_alias 0x160000000
|
||||
# Engage 不调 SetHeapSize：全部内存经别名区（MapPhysicalMemory，实测映射一直用到别名区顶端），线程栈在栈区里随机放。所以堆区只留 128 MB，
|
||||
# 整块 6 GB = 前置 256 MB + 别名区 5.375 GB + 栈区 256 MB + 堆区 128 MB，正好是 iOS 不带「扩展虚拟寻址」授权时单块预留的上限（原先 Mac 起机用 8 GB、别名区 5.5 GB）
|
||||
# Engage never calls SetHeapSize: all its memory goes through the alias region (MapPhysicalMemory; measured mappings reach the top of the alias region), and thread stacks are placed at random in the stack region. So the heap region keeps only 128 MB,
|
||||
# and the 6 GB block = 256 MB in front + 5.375 GB alias region + 256 MB stack region + 128 MB heap region, exactly the largest single reservation iOS allows without the "extended virtual addressing" entitlement (Mac bring-up used 8 GB with a 5.5 GB alias region)
|
||||
mem_reserve 0x180000000
|
||||
mem_alias 0x158000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
|
||||
|
||||
|
Can't render this file because it contains an unexpected character in line 34 and column 96.
|
+3
-1
@@ -91,7 +91,9 @@ uint64_t gate_dispatch(uint64_t ret_addr, uint64_t *regs) {
|
||||
return t;
|
||||
}
|
||||
case G_OBS: if (g_obs_fn[g->arg]) g_obs_fn[g->arg](regs); return 0; // 只读寄存器；dm_gate 回写的 x0–x8 即原值 | Registers are read-only; the x0–x8 that dm_gate writes back are the original values
|
||||
default: return svc_dispatch(g->arg, regs, g->orig);
|
||||
default: { // 进 / 出 SVC 登记门帧：暂停线程时用它作现场（hle_thread.c） | register the gate frame on SVC entry / exit: it is the context of a paused thread (hle_thread.c)
|
||||
void thr_svc_enter(uint64_t *, uint64_t), thr_svc_leave(void);
|
||||
thr_svc_enter(regs, g->orig); uint64_t r = svc_dispatch(g->arg, regs, g->orig); thr_svc_leave(); return r; }
|
||||
}
|
||||
if (!g->ret) die("unknown gate at %#llx", ret_addr);
|
||||
}
|
||||
|
||||
@@ -70,6 +70,8 @@ HLE_FORCE("_ZN2nn3nsd9ResolveExEPNS0_4FqdnERKS1_", "服务器名解析，联网
|
||||
// bool arguments only look at the low 8 bits; returns bool
|
||||
HLE_FORCE("_ZN2nn4pctl25TryBeginFreeCommunicationEb", "家长控制·与他人通信，联网排除；恒不允许", unsigned, h_pctl_TryBegin, unsigned show_ui) { (void)show_ui; return 0; }
|
||||
HLE_FORCE("_ZN2nn4pctl20EndFreeCommunicationEv", "家长控制·与他人通信", void, h_pctl_End, void) {}
|
||||
// Engage 进战斗地图时查询（不弹界面）；不接管时 SDK 经 sm 连 pctl 服务，宿主没有，中止 | Engage asks on entering a battle map (no UI); without this the SDK connects to the pctl service through sm, which the host does not have, and aborts
|
||||
HLE_FORCE("_ZN2nn4pctl28IsFreeCommunicationAvailableEv", "家长控制·与他人通信可用否，联网排除；恒否", unsigned, h_pctl_IsFree, void) { return 0; }
|
||||
|
||||
// swkbd 软键盘：见 hle_swkbd.c
|
||||
// swkbd software keyboard: see hle_swkbd.c
|
||||
@@ -9,6 +9,7 @@
|
||||
#include <unistd.h>
|
||||
#include <sched.h>
|
||||
#include <time.h>
|
||||
#include <stdatomic.h>
|
||||
|
||||
// 预留一整块客体地址空间，按 Horizon 分区（地址值直接用宿主 mmap 得到的真地址）。只占地址、不占内存。
|
||||
// 大小 5 GB：游戏要的是开头 1 GB（别名区、栈）+ 一整块堆（应用内存池 3285 MiB，见下面 APP_POOL），合计约 4.2 GB；
|
||||
@@ -107,6 +108,7 @@ uint64_t svc_dispatch(int num, uint64_t *regs, uint64_t orig_addr) {
|
||||
if (size > g_heap_cur) rc = mprotect(g_heap + g_heap_cur, size - g_heap_cur, PROT_READ | PROT_WRITE);
|
||||
else if (size < g_heap_cur) rc = mprotect(g_heap + size, g_heap_cur - size, PROT_NONE);
|
||||
if (g_flag.svc) LOG("SetHeapSize %#zx -> base %p mprotect=%d errno=%d", size, g_heap, rc, errno);
|
||||
else if (size > g_heap_cur) LOG("[内存] 堆 %.2f GB（堆区 %.2f GB）", size / 1073741824.0, g_heap_cap / 1073741824.0);
|
||||
g_heap_cur = size;
|
||||
regs[0] = 0; regs[1] = (uint64_t)g_heap; return 0;
|
||||
}
|
||||
@@ -199,6 +201,13 @@ uint64_t svc_dispatch(int num, uint64_t *regs, uint64_t orig_addr) {
|
||||
: (madvise((void *)mlo, mhi - mlo, MADV_FREE), mprotect((void *)mlo, mhi - mlo, PROT_NONE));
|
||||
if (rc) die("SVC %#x mprotect %p+%#zx errno=%d", num, a, size, errno);
|
||||
if (g_flag.svc) LOG("%sPhysicalMemory %p+%#zx", num == 0x2C ? "Map" : "Unmap", a, size);
|
||||
// 别名区用到多深（每过 256 MB 记一行）：定预留块与别名区大小用（iOS 单块预留上限 6 GB） | how deep the alias region is used (one line per 256 MB crossed): for sizing the reservation and alias region (iOS caps a single reservation at 6 GB)
|
||||
if (num == 0x2C) { static _Atomic uint64_t hw; uint64_t end = (uint64_t)(a + size - g_alias), old = atomic_load(&hw);
|
||||
while (end > old && !atomic_compare_exchange_weak(&hw, &old, end)) {}
|
||||
if (end > old && end >> 28 != old >> 28) LOG("[内存] 别名区映射到 +%.2f GB（别名区 %.2f GB）", end / 1073741824.0, g_alias_cap / 1073741824.0); }
|
||||
// 别名区实际映射的总量（每跨 256 MB 记一行，涨跌都记）：看游戏用了多少物理内存、有没有一直涨 | total actually mapped in the alias region (one line per 256 MB crossed, up or down): how much physical memory the game uses and whether it keeps growing
|
||||
{ static _Atomic int64_t tot; int64_t d = num == 0x2C ? (int64_t)size : -(int64_t)size, before = atomic_fetch_add(&tot, d), after = before + d;
|
||||
if (before >> 28 != after >> 28) LOG("[内存] 别名区已映射 %.2f GB（应用池 %.2f GB）", after / 1073741824.0, APP_POOL / 1073741824.0); }
|
||||
regs[0] = 0; return 0;
|
||||
}
|
||||
case 0x26: { // Break(reason, ptr, size)：SDK 致命中止 | Break(reason, ptr, size): SDK fatal abort
|
||||
|
||||
+108
-3
@@ -27,6 +27,7 @@
|
||||
#include <mach/mach.h>
|
||||
#include <pthread/qos.h>
|
||||
#include <dlfcn.h>
|
||||
#include <stdatomic.h>
|
||||
|
||||
#define R_TIMEOUT 0xEA01u // 内核模块 1，描述 117 | Kernel module 1, description 117
|
||||
#define R_CANCELLED 0xEC01u // 118
|
||||
@@ -55,6 +56,14 @@ typedef struct thr {
|
||||
uint64_t nsvc; // SVC 次数（现场导出用） | SVC count (for the state dump)
|
||||
mach_port_t mport; // 宿主线程端口（取 CPU 时间） | Host thread port (to get CPU time)
|
||||
uint32_t qos;
|
||||
// 暂停（SetThreadActivity，IL2CPP 的垃圾回收停世界用）：pause_req 请求、paused 已停住；停在 SVC 里时现场取门帧 svc_regs（进 SVC 时登记），
|
||||
// 停在客体代码里时现场由信号处理存进 ctx_*；pcv 让请求方等「已停住」、被停线程等「恢复」
|
||||
// Pausing (SetThreadActivity, used by IL2CPP's garbage collector to stop the world): pause_req = requested, paused = stopped; when stopped inside an SVC the context comes from the gate frame svc_regs (registered on SVC entry),
|
||||
// when stopped in guest code the signal handler saves it into ctx_*; pcv lets the requester wait for "stopped" and the paused thread wait for "resume"
|
||||
_Atomic int pause_req; int paused;
|
||||
uint64_t *_Atomic svc_regs; uint64_t svc_pc;
|
||||
int ctx_ok; uint64_t ctx_x[29], ctx_fp, ctx_lr, ctx_sp, ctx_pc; uint32_t ctx_pstate, ctx_fpcr, ctx_fpsr; __uint128_t ctx_v[32];
|
||||
pthread_cond_t pcv;
|
||||
} thr_t;
|
||||
static thr_t *g_all;
|
||||
|
||||
@@ -184,7 +193,7 @@ static void *thread_main(void *p) {
|
||||
}
|
||||
static void thr_free(thr_t *t) { // 持 G；从全部线程链摘下并释放 | Holding G; unlink from the all-threads chain and free
|
||||
for (thr_t **p = &g_all; *p; p = &(*p)->all) if (*p == t) { *p = t->all; break; }
|
||||
pthread_cond_destroy(&t->cv); free(t);
|
||||
pthread_cond_destroy(&t->cv); pthread_cond_destroy(&t->pcv); free(t);
|
||||
}
|
||||
static const char *thr_name(thr_t *t, char *buf, size_t n);
|
||||
static void thread_exit(thr_t *t) {
|
||||
@@ -201,12 +210,77 @@ static void thread_exit(thr_t *t) {
|
||||
}
|
||||
void dm_thread_returned(void) { die("客体线程入口函数返回（Horizon 要求 ExitThread）"); }
|
||||
|
||||
// ---------- 暂停线程（SetThreadActivity / GetThreadContext3）----------
|
||||
// IL2CPP 的 Boehm 垃圾回收停世界：逐个 SetThreadActivity(句柄, 暂停)、GetThreadContext3 读寄存器找根、扫栈，再恢复。原先两者都是占位成功：
|
||||
// 线程照跑、现场是空的，回收器漏掉只在寄存器里的指针、提前回收还在用的对象（终结器连带释放原生资源）。现在照 Horizon 语义真停：
|
||||
// · 在 SVC 里（门帧已存全寄存器）：立即算停住，现场 = 门帧；SVC 返回前若仍在暂停就停在出口等恢复；
|
||||
// · 在跑客体代码：发 SIGUSR2，处理函数存下 ucontext 后原地等恢复（客体代码不持宿主锁，可在信号处理里取 G）；
|
||||
// · 在宿主代码里（HLE 函数等）：处理函数不停，请求方隔 100 µs 重发，直到它回到客体代码或进 SVC。
|
||||
// IL2CPP's Boehm garbage collector stops the world: SetThreadActivity(handle, paused) on each thread, GetThreadContext3 to read registers for roots, scan the stack, then resume. Both used to be placeholders that reported success:
|
||||
// threads kept running and the context was empty, so the collector missed pointers held only in registers and freed objects still in use (finalizers releasing native resources with them). They now really stop, as on Horizon:
|
||||
// · inside an SVC (the gate frame holds every register): stopped at once, context = the gate frame; if still paused when the SVC returns, the thread waits at the exit for the resume;
|
||||
// · running guest code: send SIGUSR2; the handler saves the ucontext and waits there for the resume (guest code holds no host locks, so the handler may take G);
|
||||
// · inside host code (HLE functions etc.): the handler does not stop it; the requester resends every 100 µs until the thread is back in guest code or enters an SVC.
|
||||
module_t *module_of(const void *p);
|
||||
static void on_pause_sig(int sig, siginfo_t *si, void *uc_) {
|
||||
(void)sig; (void)si;
|
||||
thr_t *t = pthread_getspecific(g_keythr);
|
||||
if (!t || !atomic_load(&t->pause_req) || atomic_load(&t->svc_regs)) return;
|
||||
ucontext_t *uc = uc_; const struct __darwin_arm_thread_state64 *ss = &uc->uc_mcontext->__ss;
|
||||
if (!module_of((void *)__darwin_arm_thread_state64_get_pc(*ss))) return; // 在宿主代码里：不停，请求方重发 | in host code: do not stop; the requester resends
|
||||
pthread_mutex_lock(&G);
|
||||
if (t->pause_req) {
|
||||
for (int i = 0; i < 29; i++) t->ctx_x[i] = ss->__x[i];
|
||||
t->ctx_fp = __darwin_arm_thread_state64_get_fp(*ss); t->ctx_lr = __darwin_arm_thread_state64_get_lr(*ss);
|
||||
t->ctx_sp = __darwin_arm_thread_state64_get_sp(*ss); t->ctx_pc = __darwin_arm_thread_state64_get_pc(*ss); t->ctx_pstate = ss->__cpsr;
|
||||
memcpy(t->ctx_v, uc->uc_mcontext->__ns.__v, sizeof t->ctx_v); t->ctx_fpcr = uc->uc_mcontext->__ns.__fpcr; t->ctx_fpsr = uc->uc_mcontext->__ns.__fpsr;
|
||||
t->ctx_ok = 1; t->paused = 1; pthread_cond_broadcast(&t->pcv);
|
||||
while (t->pause_req) pthread_cond_wait(&t->pcv, &G);
|
||||
t->paused = 0; t->ctx_ok = 0;
|
||||
}
|
||||
pthread_mutex_unlock(&G);
|
||||
}
|
||||
// 门进 SVC / 出 SVC 时调（hle.c gate_dispatch） | called by the gate on SVC entry / exit (gate_dispatch in hle.c)
|
||||
void thr_svc_enter(uint64_t *regs, uint64_t pc) {
|
||||
thr_t *t = pthread_getspecific(g_keythr); if (!t) return;
|
||||
t->svc_pc = pc; atomic_store(&t->svc_regs, regs);
|
||||
if (atomic_load(&t->pause_req)) { pthread_mutex_lock(&G); if (t->pause_req && !t->paused) { t->paused = 1; pthread_cond_broadcast(&t->pcv); } pthread_mutex_unlock(&G); }
|
||||
}
|
||||
void thr_svc_leave(void) {
|
||||
thr_t *t = pthread_getspecific(g_keythr); if (!t) return;
|
||||
pthread_mutex_lock(&G);
|
||||
while (t->pause_req) { t->paused = 1; pthread_cond_broadcast(&t->pcv); pthread_cond_wait(&t->pcv, &G); }
|
||||
t->paused = 0; atomic_store(&t->svc_regs, NULL);
|
||||
pthread_mutex_unlock(&G);
|
||||
}
|
||||
static void pause_install(void) {
|
||||
struct sigaction sa = {.sa_sigaction = on_pause_sig, .sa_flags = SA_SIGINFO | SA_ONSTACK | SA_RESTART};
|
||||
sigemptyset(&sa.sa_mask);
|
||||
sigaction(SIGUSR2, &sa, NULL);
|
||||
}
|
||||
// Horizon svc::ThreadContext（arm64，0x320 字节） | Horizon svc::ThreadContext (arm64, 0x320 bytes)
|
||||
typedef struct { uint64_t r[29], fp, lr, sp, pc; uint32_t pstate, pad; __uint128_t v[32]; uint32_t fpcr, fpsr; uint64_t tpidr; } hz_ctx_t;
|
||||
_Static_assert(sizeof(hz_ctx_t) == 0x320, "ThreadContext");
|
||||
static void ctx_fill(const thr_t *t, hz_ctx_t *o) { // 持 G、t 已停住 | holding G, t is stopped
|
||||
memset(o, 0, sizeof *o);
|
||||
const uint64_t *r = atomic_load(&t->svc_regs);
|
||||
if (r) { // 门帧：x0..x30 在 +0、q0..q31 在 +256、nzcv / fpsr / fpcr 在 +768 / +776 / +784；进门前的栈顶 = 门帧 + 800 | gate frame: x0..x30 at +0, q0..q31 at +256, nzcv / fpsr / fpcr at +768 / +776 / +784; the stack top before the gate = frame + 800
|
||||
for (int i = 0; i < 29; i++) o->r[i] = r[i];
|
||||
o->fp = r[29]; o->lr = r[30]; o->sp = (uint64_t)r + 800; o->pc = t->svc_pc; o->pstate = (uint32_t)r[96];
|
||||
memcpy(o->v, r + 32, sizeof o->v); o->fpsr = (uint32_t)r[97]; o->fpcr = (uint32_t)r[98];
|
||||
} else if (t->ctx_ok) {
|
||||
memcpy(o->r, t->ctx_x, sizeof o->r); o->fp = t->ctx_fp; o->lr = t->ctx_lr; o->sp = t->ctx_sp; o->pc = t->ctx_pc; o->pstate = t->ctx_pstate;
|
||||
memcpy(o->v, t->ctx_v, sizeof o->v); o->fpcr = t->ctx_fpcr; o->fpsr = t->ctx_fpsr;
|
||||
}
|
||||
}
|
||||
|
||||
// 主线程登记，返回其句柄（交给 nnosInitialize 作主线程句柄）
|
||||
// Register the main thread and return its handle (passed to nnosInitialize as the main thread handle)
|
||||
uint32_t svc_thread_main_init(void) {
|
||||
pthread_key_create(&g_keythr, NULL);
|
||||
pause_install();
|
||||
thr_t *t = calloc(1, sizeof *t);
|
||||
pthread_cond_init(&t->cv, NULL);
|
||||
pthread_cond_init(&t->cv, NULL); pthread_cond_init(&t->pcv, NULL);
|
||||
t->started = 1; t->prio = 44; t->core = 0; t->mask = 7; t->pt = pthread_self(); t->mport = pthread_mach_thread_np(t->pt);
|
||||
pthread_mutex_lock(&G); t->handle = h_new(O_THREAD, t); t->id = g_next_id++; t->all = g_all; g_all = t; pthread_mutex_unlock(&G);
|
||||
pthread_setspecific(g_keythr, t);
|
||||
@@ -441,7 +515,7 @@ int svc_thread(int num, uint64_t *regs) {
|
||||
switch (num) {
|
||||
case 0x08: { // CreateThread(→w1 句柄; x1 入口, x2 参数, x3 栈顶, w4 优先级, w5 核) | CreateThread(→w1 handle; x1 entry, x2 argument, x3 stack top, w4 priority, w5 core)
|
||||
thr_t *t = calloc(1, sizeof *t);
|
||||
pthread_cond_init(&t->cv, NULL);
|
||||
pthread_cond_init(&t->cv, NULL); pthread_cond_init(&t->pcv, NULL);
|
||||
t->entry = regs[1]; t->arg = regs[2]; t->stack_top = regs[3] & ~0xFull;
|
||||
// 核 -2（IdealCoreUseProcessValue）= 进程缺省理想核（本作 0 号），亲和掩码只含它——同 Horizon 内核。原先记成 -2，GetCurrentProcessorNumber 也答 -2，
|
||||
// SMBW 的字体引擎按当前核号取每核引擎槽，拿 -2 当下标越界崩（main +0x107dbc0，SequenceBackgroundWorker 线程）
|
||||
@@ -495,6 +569,37 @@ int svc_thread(int num, uint64_t *regs) {
|
||||
regs[0] = t ? 0 : R_BAD_HANDLE; return 1;
|
||||
}
|
||||
case 0x10: { int c = cur()->core; regs[0] = (uint64_t)(uint32_t)(c >= 0 ? c : 0); return 1; } // 当前核号总在 0..2（核号从不为负） | The current core number is always 0..2 (never negative)
|
||||
case 0x32: { // SetThreadActivity(w0 线程, w1 0 可运行 / 1 暂停)：见上「暂停线程」 | SetThreadActivity(w0 thread, w1 0 runnable / 1 paused): see "Pausing threads" above
|
||||
thr_t *me = cur(); int act = (int)regs[1];
|
||||
pthread_mutex_lock(&G);
|
||||
thr_t *t = h_get((uint32_t)regs[0], O_THREAD); int64_t r = 0;
|
||||
if (!t) r = R_BAD_HANDLE;
|
||||
else if (t == me) r = R_BAD_STATE;
|
||||
else if (act == 1) {
|
||||
atomic_store(&t->pause_req, 1);
|
||||
if (!t->started || t->exited || atomic_load(&t->svc_regs)) t->paused = 1;
|
||||
uint64_t t0 = now_ns(); int warned = 0;
|
||||
while (!t->paused && !t->exited) {
|
||||
pthread_mutex_unlock(&G); pthread_kill(t->pt, SIGUSR2); pthread_mutex_lock(&G);
|
||||
if (t->paused || t->exited) break;
|
||||
struct timespec dl; clock_gettime(CLOCK_REALTIME, &dl); dl.tv_nsec += 100000; if (dl.tv_nsec >= 1000000000) { dl.tv_sec++; dl.tv_nsec -= 1000000000; }
|
||||
pthread_cond_timedwait(&t->pcv, &G, &dl);
|
||||
if (!warned && now_ns() - t0 > 1000000000ull) { warned = 1; LOG("【红】暂停线程 %#x 超过 1 秒还没停住（一直在宿主代码里？）", t->handle); }
|
||||
}
|
||||
} else { atomic_store(&t->pause_req, 0); pthread_cond_broadcast(&t->pcv); }
|
||||
pthread_mutex_unlock(&G);
|
||||
{ static int n; if (n++ < 4) LOG("SetThreadActivity %#x → %s（%s）", (uint32_t)regs[0], act ? "暂停" : "恢复", r ? "失败" : "成功"); }
|
||||
regs[0] = (uint64_t)r; return 1;
|
||||
}
|
||||
case 0x33: { // GetThreadContext3(x0 出参 ThreadContext*, w1 线程)：只对已停住的线程 | GetThreadContext3(x0 out ThreadContext*, w1 thread): only for stopped threads
|
||||
pthread_mutex_lock(&G);
|
||||
thr_t *t = h_get((uint32_t)regs[1], O_THREAD); int64_t r = 0;
|
||||
if (!t) r = R_BAD_HANDLE;
|
||||
else if (!t->paused) r = R_BAD_STATE;
|
||||
else ctx_fill(t, (hz_ctx_t *)regs[0]);
|
||||
pthread_mutex_unlock(&G);
|
||||
regs[0] = (uint64_t)r; return 1;
|
||||
}
|
||||
case 0x25: { thr_t *t = h_get((uint32_t)regs[1], O_THREAD); regs[0] = t ? 0 : R_BAD_HANDLE; regs[1] = t ? t->id : 0; return 1; }
|
||||
|
||||
case 0x1A: { // ArbitrateLock(w0 持有者句柄, x1 地址, w2 本线程标签) | ArbitrateLock(w0 owner handle, x1 address, w2 this thread's tag)
|
||||
|
||||
@@ -40,6 +40,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **`StandardAllocator::Initialize` aborts in `nninitStartup`** → the NPDM declares a system resource (offset 0x14 of `META`), so `IsVirtualAddressMemoryEnabled` is true and the allocator reserves all free memory (about 3.2 GB for Engage) as one region in the alias region, after the ~1 GB node area → the profile's `mem_alias` / `mem_reserve` must hold it (Engage: 5.5 / 8 GB on the Mac).
|
||||
- **C++ exception aborts in `nn::ro::detail::GetExceptionInfo`** → the SDK's unwinder finds a module from `svcQueryMemory`: it walks back over Code+RX regions to the module start, forward over Code then CodeData, and reads `MOD0` for `.eh_frame_hdr`. The host must report each loaded module's real layout and report other host memory as free (`hle_svc.c`). IL2CPP throws a C++ exception for every managed exception, so Unity games need this; Mario never threw.
|
||||
- **First `AllocateAddressRegion` fails with 0x1803** → SDK 16's AddressRegionManager takes 0x3fe00000 at the start of the alias region as a node area → the alias region must be larger than 1 GB.
|
||||
- **Sizing the reservation for a title that uses virtual address memory (Engage)** → such a title may never call `SetHeapSize`: everything comes from `MapPhysicalMemory` in the alias region. The `[内存]` log lines report the alias high-water mark (one line per 256 MB) and every heap growth. Engage's mappings reach +5.26 GB, so its profile uses a 5.375 GB alias region and a 128 MB heap region, 6 GB in all, iOS's single-reservation limit without the extended-virtual-addressing entitlement.
|
||||
- **Large resources silently skipped, later a null deref where the resource is used** → the game reserves "virtual address memory" (Mario: 1.75 GB in `HeapDef.byml`) and maps it into the alias region with `MapPhysicalMemory` on demand; the alias region left after the node area was too small. Also visible as `SetHeapSize` asking for much less than the pool. → repartition the guest block (`hle_svc.c` `mem_init`): Mario uses alias 2.875 GB, stacks 256 MB, heap 1.625 GB (it asks for 1.47 GB), 5 GB in total. Measure before sizing (`DM_SVC=1`: the `SetHeapSize` request and the highest `MapPhysicalMemory` address). On iOS without the extended-virtual-addressing entitlement a single 6 GB reservation succeeds and 7 GB fails, but Metal and the rest of the process need address space too; 5 GB is the total proven on a phone (D2R).
|
||||
- **`MapPhysicalMemory` faults on part of a page** → round to 16 KB host pages.
|
||||
- **Some textures are re-uploaded every frame (several MB per frame)** → the dirty-page tracker registers each memory pool rounded to 16 KB host pages, but NVN pools are only 4 KB aligned, so neighbouring pools share their first / last host page and one address can lie in several registered ranges. Taking the first range that contains an address judged textures in the second pool untracked (always dirty), released only one range on a write fault, and could unregister the neighbour when a pool was destroyed. `nvn_dirty.c` now uses the range that holds the whole span, releases every range covering a faulting page, and removes a range by the pool's exact start (`t_dirty_slots` covers it).
|
||||
@@ -49,6 +50,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
### Threads
|
||||
|
||||
- **Per-core job queues never drained** → a `SetThreadCoreMask` that pins a thread to other cores (ideal core −3 = keep) must move the ideal core into the new mask, as Horizon does, so `GetCurrentProcessorNumber` reports a core the thread can run on.
|
||||
- **Garbage-collected titles (Unity / IL2CPP): `SetThreadActivity` (0x32) and `GetThreadContext3` (0x33) must really pause and report registers** → the Boehm collector pauses each thread, reads its registers for roots and scans its stack. As placeholders that only reported success, threads kept running and the context was empty: roots held only in registers were missed and live objects were freed, with their finalizers releasing native resources. In `hle_thread.c` a thread inside an SVC counts as paused at once, with the gate frame as its context (`thr_svc_enter` / `thr_svc_leave` around every gate SVC; it waits at the SVC exit while paused). A thread in guest code gets SIGUSR2, and the handler saves the ucontext and waits there. A thread in host code is retried every 100 µs. The context is Horizon's 0x320-byte `ThreadContext`.
|
||||
- **`SignalToAddress` type 2 (`SignalAndModifyByWaitingCountIfEqual`) with count ≤ 0 and waiters present** → the new value is `value − 2`, not `value − 1` (mesosphere `KAddressArbiter`; Ryujinx and yuzu agree). The host had −1. Fixing it did not change Engage's demo crash below, but it is the specified behaviour.
|
||||
- **Several threads crash at once and the dumps interleave, then the process hangs instead of exiting** → Unity's job threads tend to crash together. `crash.c` lets only the first crashing thread print (`crash_gate`); the others stop in `pause()`, and a 15 s alarm ends the process if the dump itself stalls.
|
||||
- **Crash indexing a per-core table with a huge offset (index −2)** → threads created with ideal core −2 (*use process default*) were recorded as core −2 and `GetCurrentProcessorNumber` returned −2 → resolve −2 to the process default core (0) and never return a negative core (`hle_thread.c`). Seen in Mario's font engine (`main+0x107dbc0`).
|
||||
@@ -61,6 +63,7 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi
|
||||
- **SDK 12 names** → older SDKs export other names for the same calls (`ListAudioDeviceName` / `GetActiveAudioDeviceName`, singular `hid::GetNpadState`, `OpenDefaultAudioOut`); hook both generations.
|
||||
- **Abort in NVIDIA's video path (`NvRmChannelOpen` → `NvOsDrvOpen`)** → the SDK movie library's hardware decoder opens nvdrv directly. The host takes over `movie::Decoder` (`hle_movie.c`, next entry); nvdrv opens fail (`hle_nvdrv.c`).
|
||||
- **The game waits forever at a movie (Engage's 94 s opening `S01F.webm`; frames keep presenting, nothing happens)** → Unity demuxes the WebM itself and feeds VP9 packets to `movie::Decoder`; with no working decoder the movie never ends. `movie::Decoder` is a thin wrapper over Android's MediaCodec, and its vtable entries are symbol relocations, so `HLE_FORCE` on each method also catches Unity's virtual calls. The host implements the protocol: four `nn::os` events (`DecoderEvents` = input available, output ready, format changed, error), input indices (`AcquireInputBufferIndex`, none = 0x1011; `GetInputBuffer` fills the client's `movie::Buffer`; `SendInputBufferForDecode(index, offset, size, µs, flags)`), output indices (`AcquireOutputBufferIndex`, none = 0x100e; one NV12 frame per packet copied into the buffer Unity allocated, w × h × 1.5; `ReleaseOutputBufferIndex`). While every output slot is taken, packets wait and their input indices are not returned, as with MediaCodec; dropping them instead lost frames. `Buffer`, `MediaData` and events are handled through the SDK's own exported functions (`SetDataAndCapacity`, `SetRange`, `SetInt32`, `SetRect`, `SignalEvent`). Unity reads `width`, `height`, `crop`, `nv12-y-stride`, `nv12-y-offset`, `nv12-uv-offset` and `slice-height`. VP9 packets are decoded with libvpx (`dm/third_party/libvpx`) on one host thread per decoder: guest threads run on the stacks the SDK gives them, too small for libvpx. The guest thread that sends a packet or returns an output index waits for the decode and signals the events itself, because `SignalEvent` is guest code and cannot run on a host thread. Packets are decoded in arrival order from one queue. When the movie ends the game continues to its title screen.
|
||||
- **Abort in `nn::sm::Initialize` under `nn::pctl::IsFreeCommunicationAvailable` (Engage, entering its first battle map)** → the SDK connects to the parental-control service through `sm`. The host answers it like the other free-communication calls: not available (`hle_misc.c`).
|
||||
- **Abort inside `nn::sm::Initialize`** → an SDK path reached a system service the host doesn't replace → hook the public SDK entry point named in the backtrace (`HLE_FORCE` in the relevant `hle_*.c`). The log shows it as `SVC 0x1f` (ConnectToNamedPort, port name `sm:`) and `SVC 0x21` answered by placeholders, then `nn::diag` abort.
|
||||
- **A game update aborts at boot where the base game ran** → the update imports SDK functions the base game didn't. Diff the undefined dynamic symbols of the two `main` modules before the first run. Mario 1.2.1 added 28, among them `nn::nfp` (amiibo, initialized at boot), `pl::IsRunningOnOunce` (is this a Switch 2), `ec::ShowShopApplicationInformationForUpgrade`, `account::PopOpenUsers` and `aoc::CountAddOnContent`. Read the call sites before choosing answers: Mario only reads the count from `PopOpenUsers` and falls back to `TryOpenPreselectedUser` on 0, and only counts add-on content when running on a Switch 2. The host answers as a Switch 1 with no amiibo and no add-on content (`hle_sdk16.c`, `hle_account.c`, `hle_misc.c`).
|
||||
- **Code inherited from another title calls fixed addresses** → harmless on the build it was written for, silent memory corruption on any other. d2r-mobile's 3D render scale read a renderer pointer at `main+0x31e0848` and called `main+0x8582f0`; in Mario v0 that is the middle of a static initializer and only did nothing because the pointer happened to read 0. Such code checks `module_t.build_id` now (written into `fixups.tsv` by the rewriter) and stays off on other builds.
|
||||
@@ -112,4 +115,5 @@ The game may also write into an NVN object directly, without an NVN call, throug
|
||||
|
||||
### Game state
|
||||
|
||||
- **In a scripted run, a screen goes back by itself right after a confirm (Engage's appearance choice)** → the confirm opened the system keyboard (`swkbd`, name entry). Without a window the host answers "cancel" by design; pass `DM_KBD=<text>` to answer it. The log shows `swkbd：… → 无窗口按取消`.
|
||||
- **"Stuck" at a screen with one draw per frame** → usually not stuck: most draws are in recorded command buffers replayed later; dump the passes ([04](04-graphics.md)) before assuming the game logic is blocked.
|
||||
@@ -71,6 +71,7 @@ Consequence: any optimization that infers "which textures a draw reads" from bou
|
||||
- **CPU-written block-linear textures** (`nvnTextureWriteTexels`: video frames, engine default textures) are uploaded and re-uploaded from guest memory like packaged textures ([03](03-bringup.md)).
|
||||
- **Video**: Unity's `Hidden/VideoDecode` samples luma as `.w` of an `R8` texture with the swizzle `(0, 0, 0, R)` (Unity's Alpha8) and chroma from an `RG8` texture at half size. To check the movie path apart from the decoder, fill the frames with a gradient in `hle_movie.c`; it should appear on screen.
|
||||
- **One geometry shader** (in `unity_builtin_extra`): Metal has no geometry stage, so it fails to compile (`unknown main0_out`); nothing in the opening scenes uses it.
|
||||
- **Open: a crash a few minutes into the prologue battle (scripted run, about 357 s).** It writes to `NULL + 0x1fffe0` at `main+0x9a4664`. Unity's TLSF heap allocator (`main+0x9a45d0`) asks its low-level virtual allocator (`[allocator + 0x168]`) for a new 2 MB pool, and the block it gets back has a NULL pointer. The host's memory SVCs all succeed up to that point, and only about 0.5 GB is mapped, so Unity's own reserved address range is the suspect. Separately, about one run in four stalls early (no log output, not even the time-limit report). `hangwatch.sh`-style sampling (`sample <pid>` when the log stops growing) has not caught one yet.
|
||||
- **Open: the idle attract demo crashes after about 220 s.** Unity's particle geometry jobs write through a NULL index pointer, on several job threads at once (`main+0xdb0ad4`, `0xdaea58`, `0xdadb5c`). The scheduler (`main+0xd91e0c`) takes its vertex and index chunks from a fallback that creates and maps a fresh buffer each time (`main+0x7b9c64`; Unity's Switch renderer always takes it, as on hardware). The job (`main+0xd92eb0` → `0xd93194` → the index writers `0xdb0100` / `0xdad270` / `0xdae0c0`) then copies mesh indices per particle. Earlier, from the demo's scene load at about 180 s, those 32-bit mesh indices already appear inside Unity's texture objects (the bad render targets in [03](03-bringup.md)): the job writes more indices than the main thread sized for. By about 223 s the requested index buffers pass 2 MB, the map returns NULL, and the next write crashes. Why the main thread's count and the job's disagree on the host is not found yet; the arbiter fix in [03](03-bringup.md) did not change it. Reproduce: `boot.sh <log> 245` with no input.
|
||||
|
||||
## 7. Diagnostic switches (host)
|
||||
|
||||
Reference in new issue
Block a user