From 53642a6f9cfdb05bc436880d5ddd32f25a912ae7 Mon Sep 17 00:00:00 2001 From: seedds Date: Tue, 13 Oct 2026 08:56:30 +0800 Subject: [PATCH] Opening course playable: integer vertex inputs for skinned models, sky colour ramps - dmsh_build.py: vertex shaders whose inputs are only used as integer bit casts are re-translated with uint inputs (529 shaders); bone indices were converted to floats and every character was stretched off screen - Host: Metal vertex formats follow the shader input type (signedness swap, 32-bit float reinterpretation), mismatches reported - LINEAR_RENDER_TARGET textures in CPU-visible pools are zero-copy too (sky colour ramps) - Pass dump lists all vertex attributes; docs and README updated --- README.md | 4 +++- dm/gfx/nvn/c1_full.tsv | 2 +- dm/gfx/nvn/dmsh_chain.h | 2 +- dm/gfx/nvn/nvn_mtl.m | 27 +++++++++++++++++++++--- dm/gfx/tools/dmsh_build.py | 42 +++++++++++++++++++++++++++++++++++++- docs/agents/03-bringup.md | 2 ++ docs/agents/04-graphics.md | 3 ++- 7 files changed, 74 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index aff40e8..238d833 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ Same approach as d2r-mobile: the game's own aarch64 code runs natively. A rewrit | Mac host | **Plays through the menus (player count, character select) into the opening course**: the course, effects, HUD, menu and dialogue text render in Chinese or English. Title screen at 60 fps, memory flat (about 900 MB physical footprint on the title screen) | | Shaders | All 12,811 shaders the game ships (bfsha archives, particle effects, engine `agl` packages, UI layouts, `main`) are extracted by `shader_split.py` and translated offline; 12,809 compile. 2 `agl` post-effect compute shaders make SPIRV-Cross throw | | Input | Scripted input `DM_KEYS="12:A,14:A,20:A,26:A,32:A"` (seconds:button) reaches the opening course; `DM_WINDOW=1` for a window with keyboard input | -| Current frontier | In the opening course the sky is black and no character models (player, NPCs) are visible; controller-button icons in text show as `?` (the host's stand-in system font has no Nintendo-extension glyphs) | +| Current frontier | **The opening course is playable** (run, jump, collect coins; characters, NPCs, sky, text all render). Open: an intermittent water / refraction effect over the fountain draws as tall blue panels (sometimes with a wrong texture); controller-button icons in text show as `?` (the host's stand-in system font has no Nintendo-extension glyphs); no audio | | Known gaps | Partial / mid-pass clears; sparse virtual-pool mappings; conditional render; `DrawElementsIndirect`; TBDR optimizations are off by default (they don't understand bindless textures yet); resource heaps get +16 KB slack because one model's setup came up 0x70 bytes short of the size table (cause unknown) | | Audio | Silent AudioRenderer (`hle_audren.c`); real mixing comes later | | iOS bundle | Not started. Plan: reuse `dm/ios` from d2r-mobile, bundle the RomFS (3.5 GB) inside the app, standard-controller virtual gamepad | @@ -44,9 +44,11 @@ Same approach as d2r-mobile: the game's own aarch64 code runs natively. A rewrit - Linear textures in CPU-visible pools are created directly on the pool buffer (zero copy). Mario's glyph cache (2048×8192 R8) is filled by the CPU; as a private GPU texture all menu and dialogue text was blank. - Host command-memory segments (lent when a command buffer outgrows the memory the game gave it) grow on demand instead of a fixed 64 × 1 MB. - `RGB10A2SN` vertex format; instanced `DrawArrays`. + - Vertex formats follow the shader input type (integer inputs get integer formats of the matching sign). - Shader chain: - `dm/gfx/tools/shader_split.py` (new) scans RomFS and `main` for BNSH and BAHS (agl `sharcb`) containers and writes the package directory, the shader map and the answer table (compute thread-group size from control +0x748, shared memory from +0x754). - The translator uses GL depth (`-1..1`) and no depth clip control, so vertex shaders remap z' = z/2 + w/2: Mario sets `nvnDeviceSetDepthMode` to near = -w, and without the remap its full-screen copy was clipped away. + - Vertex shaders whose inputs are only used as integers are re-translated with `uint` inputs (`int_inputs`): skinned models read bone indices as raw integer bits, and the float default made every character invisible. - Compute code areas (magic 0x12345679) start their instructions at +0x100 (0x30 header + zero padding); `dmsh_build.py` strips all of it. Stripping only 0x30 misaligned the translator's 32-byte instruction groups: 12 particle / post-effect compute shaders became endless loops and hung the GPU at the first gameplay scene. - Host: - Guest address space repartitioned: alias region 2.875 GB, heap 2.625 GB. Mario reserves 1.75 GB of `VirtualAddressMemory` (HeapDef) that it maps into the alias region on demand; with 1.25 GB, large layout archives got no address and were skipped. diff --git a/dm/gfx/nvn/c1_full.tsv b/dm/gfx/nvn/c1_full.tsv index 88d8ee3..120d12b 100644 --- a/dm/gfx/nvn/c1_full.tsv +++ b/dm/gfx/nvn/c1_full.tsv @@ -1,4 +1,4 @@ -# chain b-1b3a96e6a7d3 2026-10-13T08:05:33+08:00 +# chain b-3620e9b5e1bd 2026-10-13T08:39:12+08:00 35cbc686dd8461b8 4872bec9a5567f8b 54a618792b2d5f0f diff --git a/dm/gfx/nvn/dmsh_chain.h b/dm/gfx/nvn/dmsh_chain.h index 03dbf63..8a11090 100644 --- a/dm/gfx/nvn/dmsh_chain.h +++ b/dm/gfx/nvn/dmsh_chain.h @@ -1,2 +1,2 @@ // 生成文件（dm/gfx/tools/dmsh_chain.py --header）：本源码树期望的 B 着色器库翻译链版本（缺省选项）；宿主只加载 <库目录>/b-/ -#define DMSH_CHAIN "1b3a96e6a7d3" +#define DMSH_CHAIN "3620e9b5e1bd" diff --git a/dm/gfx/nvn/nvn_mtl.m b/dm/gfx/nvn/nvn_mtl.m index e8201d2..591bd19 100644 --- a/dm/gfx/nvn/nvn_mtl.m +++ b/dm/gfx/nvn/nvn_mtl.m @@ -1199,7 +1199,8 @@ static id pkg_upload_(const tex_t *t, uint64_t *stamp) { // Linear textures (LINEAR) in a CPU-visible pool: the game writes the pool memory directly from the CPU (SMBW's glyph cache: 2048×8192 R8, IMAGE|LINEAR, pool CPU_UNCACHED|GPU_CACHED, // the font engine rasterizes glyphs into it on the CPU). A GPU-private texture never sees those writes → all menu / dialog text is blank. On unified memory, create a linear texture directly on the pool buffer (zero copy): the GPU reads the current content at execution time, same as NVN. static id tex_linear(const tex_t *t) { - if (!(t->d.flags & NVN_TEXTURE_FLAGS_LINEAR) || !t->mem || !t->pool || (t->pool->flags & NVN_MEMORY_POOL_FLAGS_CPU_NO_ACCESS)) return nil; + // LINEAR_RENDER_TARGET 也是线性排布（SMBW 的天空色带：4 张 64×1 RGBA16F，CPU 每帧写）| LINEAR_RENDER_TARGET is a linear layout too (SMBW's sky colour ramps: four 64×1 RGBA16F, written by the CPU every frame) + if (!(t->d.flags & (NVN_TEXTURE_FLAGS_LINEAR | NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET)) || !t->mem || !t->pool || (t->pool->flags & NVN_MEMORY_POOL_FLAGS_CPU_NO_ACCESS)) return nil; if (t->d.target != NVN_TEXTURE_TARGET_2D || t->d.levels > 1 || t->d.samples > 1) return nil; uint32_t pf = 0; uint8_t swz[4]; int bw, bh, bpb = nvn_fmt_bpb(t->d.format, &bw, &bh); if (!nvn_fmt_to_mtl(t->d.format, &pf, swz) || !pf || !bpb || bw != 1 || bh != 1) return nil; @@ -2384,6 +2385,25 @@ static void state_reset(void) { S.st_vmask[0] = S.st_vmask[1] = S.st_wmask[0] = S.st_wmask[1] = 0xff; S.dr[1] = 1; } +// 顶点格式按着色器输入的分量类型校正：Metal 要求整数输入配同符号的整数格式、浮点输入配浮点 / 归一化格式。 +// 整数输入（翻译时按「只按整数用」推断，dmsh_build.py int_inputs）配整数格式：符号不同则换同宽的另一符号（位模式不变）；配 32 位浮点格式则按同宽整数读（位模式不变）。 +// 其余不匹配（整数输入配归一化 / 半精度格式，浮点输入配整数格式）位模式保不住，报红一次照原格式。 +// Correct the vertex format by the shader input's component type: Metal requires integer inputs to get integer formats of the same signedness, and float inputs float / normalized formats. +// Integer input (inferred at translation time as "only used as an integer", dmsh_build.py int_inputs) with an integer format: on a signedness mismatch switch to the same width with the other sign (bit pattern unchanged); with a 32-bit float format read it as an integer of the same width (bit pattern unchanged). +// Other mismatches (integer input with a normalized / half format, float input with an integer format) cannot keep the bit pattern: report once and keep the format. +static MTLVertexFormat vfmt_for_ct(MTLVertexFormat f, int ct, int nvn_fmt) { + static const MTLVertexFormat U[] = { MTLVertexFormatUChar, MTLVertexFormatUChar2, MTLVertexFormatUChar3, MTLVertexFormatUChar4, MTLVertexFormatUShort, MTLVertexFormatUShort2, MTLVertexFormatUShort3, MTLVertexFormatUShort4, MTLVertexFormatUInt, MTLVertexFormatUInt2, MTLVertexFormatUInt3, MTLVertexFormatUInt4 }; + static const MTLVertexFormat I[] = { MTLVertexFormatChar, MTLVertexFormatChar2, MTLVertexFormatChar3, MTLVertexFormatChar4, MTLVertexFormatShort, MTLVertexFormatShort2, MTLVertexFormatShort3, MTLVertexFormatShort4, MTLVertexFormatInt, MTLVertexFormatInt2, MTLVertexFormatInt3, MTLVertexFormatInt4 }; + static const MTLVertexFormat F[] = { MTLVertexFormatFloat, MTLVertexFormatFloat2, MTLVertexFormatFloat3, MTLVertexFormatFloat4 }; + int iu = -1, ii = -1, iff = -1; + for (int i = 0; i < 12; i++) { if (U[i] == f) iu = i; if (I[i] == f) ii = i; } + for (int i = 0; i < 4; i++) if (F[i] == f) iff = i; + if (ct == NVN_CT_UINT) { if (iu >= 0) return f; if (ii >= 0) return U[ii]; if (iff >= 0) return U[8 + iff]; } + else if (ct == NVN_CT_INT) { if (ii >= 0) return f; if (iu >= 0) return I[iu]; if (iff >= 0) return I[8 + iff]; } + else { if (iu < 0 && ii < 0) return f; } + RED_ONCE(66, "顶点属性格式 %#x 与着色器输入类型 %s 对不上，位模式保不住（照原格式）", nvn_fmt, ct == NVN_CT_UINT ? "uint" : ct == NVN_CT_INT ? "int" : "float"); + return f; +} static MTLVertexFormat vfmt(int f) { switch (f) { case 0x01: return MTLVertexFormatUCharNormalized; case 0x02: return MTLVertexFormatCharNormalized; case 0x03: return MTLVertexFormatUChar; case 0x04: return MTLVertexFormatChar; @@ -2468,6 +2488,7 @@ static MTLRenderPipelineDescriptor *pso_desc(const pso_key *k, NvShader *vs, NvS continue; } int st = k->attr[a].stream; + vf = vfmt_for_ct(vf, rf->v[v].ct, k->attr[a].fmt); vd.attributes[loc].format = vf; vd.attributes[loc].offset = k->attr[a].offset; vd.attributes[loc].bufferIndex = 16 + st; vd.layouts[16 + st].stride = k->stream[st].stride ? k->stream[st].stride : 4; vd.layouts[16 + st].stepFunction = k->stream[st].stride == 0 ? MTLVertexStepFunctionConstant : k->stream[st].divisor ? MTLVertexStepFunctionPerInstance : MTLVertexStepFunctionPerVertex; @@ -4548,8 +4569,8 @@ frz_plain:; if (hn < 12 && u->size <= 608 && v->rf.b[i].reg == 3) { hn++; char hx[1300]; for (uint64_t q = 0; q < u->size; q++) snprintf(hx + 2 * q, 3, "%02x", base[mo + q]); PD("x %llu r%d %s", g_stat_pass, v->rf.b[i].reg, hx); } } o += snprintf(ub + o, sizeof ub - o, " r%d=%016llx/%llu/%016llx", v->rf.b[i].reg, (unsigned long long)h, (unsigned long long)u->size, (unsigned long long)h64); } PD("u %llu%s", g_stat_pass, ub); } - if (all) { char vb[512]; size_t o = 0; o += snprintf(vb + o, sizeof vb - o, "nattr=%d", S.nattr); - for (int i = 0; i < S.nattr && i < 6; i++) { int st = S.attr[i].stream; + if (all) { char vb[1600]; size_t o = 0; o += snprintf(vb + o, sizeof vb - o, "nattr=%d", S.nattr); + for (int i = 0; i < S.nattr && i < 16 && o < sizeof vb - 100; i++) { int st = S.attr[i].stream; if (!S.attr[i].fmt) continue; // 只打有格式的属性 | only attributes with a format o += snprintf(vb + o, sizeof vb - o, " a%d{fmt=%#x off=%d s=%d stride=%u div=%u vb=%#llx/%llu}", i, S.attr[i].fmt, S.attr[i].offset, st, st < 16 ? S.stream[st].stride : 0, st < 16 ? S.stream[st].divisor : 0, st < 16 ? (unsigned long long)S.vbuf[st].address : 0ull, st < 16 ? (unsigned long long)S.vbuf[st].size : 0ull); } PD("v %llu %s", g_stat_pass, vb); } diff --git a/dm/gfx/tools/dmsh_build.py b/dm/gfx/tools/dmsh_build.py index 29ee12e..ee52a20 100644 --- a/dm/gfx/tools/dmsh_build.py +++ b/dm/gfx/tools/dmsh_build.py @@ -411,13 +411,53 @@ RULES = [] # 按着色器名挂的 MSL 改写规则（函数, 适用判定） CODE_FP = hashlib.sha256(source_without_rule_bodies().encode() + b''.join(b'\0' + open(m.__file__, 'rb').read() for m in (dmsh_chain,) + ((ios_metal,) if IOSOUT else ()))).hexdigest() TOOL_FP = tool_fingerprint().encode() +_CAST = ('as_type(', 'as_type(') +def int_inputs(src): + """顶点着色器里只按整数用的输入（float 声明下每次使用都是 as_type 位转换，或原样拷走）→ {位置: 'uint'}。 + Maxwell 读顶点属性寄存器不做类型转换：整数格式（骨骼索引 R8UI / RG8UI / RGBA8UI 等）的位模式原样进寄存器，着色器再按整数用。 + 本作没有 PC 版反射可配对，翻译器的属性类型查询缺省全答 float：Metal 把整数格式转成浮点值送进来，位转换后成了乱值——骨骼索引全错，蒙皮模型（角色）的顶点被拉成长条或飞出画面。 + Inputs that a vertex shader only uses as integers (under a float declaration every use is an as_type bit cast, or a plain copy) → {location: 'uint'}. + Maxwell does no type conversion when reading vertex attribute registers: the bit pattern of integer formats (bone indices R8UI / RG8UI / RGBA8UI etc.) goes into the register as is and the shader uses it as an integer. + This game has no PC version to pair reflection with, so the translator's attribute type queries all defaulted to float: Metal converts the integer format to a float value, and after the bit cast it is garbage - every bone index is wrong and skinned models (the characters) are stretched into streaks or thrown off screen.""" + ins = {m.group(2): int(m.group(3)) for m in re.finditer(r'(\w+)\s+(\w+)\s*\[\[attribute\((\d+)\)\]\]', src)} + i = src.find('vertex main0_out main0(') + if i < 0: return {} + body = src[i:]; out = {} + for name, loc_ in ins.items(): + uses = list(re.finditer(r'in\.%s\b' % re.escape(name), body)) + nint = 0; bad = False + for u in uses: + ls = body.rfind('\n', 0, u.start()) + 1; le = body.find('\n', u.end()) + m = re.match(r'\s*float\s+(_\w+)\s*=\s*in\.%s\.[xyzw];$' % re.escape(name), body[ls:le]) + if not m: + if body[:u.start()].endswith(_CAST): nint += 1; continue + bad = True; break + v = m.group(1) + for o in list(re.finditer(r'\b%s\b' % re.escape(v), body))[1:]: + if body[:o.start()].endswith(_CAST): nint += 1; continue + l2s = body.rfind('\n', 0, o.start()) + 1; l2e = body.find('\n', o.end()) + if re.match(r'\s*[\w.\[\]]+\s*=\s*%s;$' % re.escape(v), body[l2s:l2e]): continue # 原样拷走：两种声明下位模式都不变 | plain copy: the bit pattern is the same under either declaration + bad = True; break + if bad: break + if not bad and nint: out[str(loc_)] = 'uint' + return out + def finish(key, w, j, c1): name, k = loc[key] - res = json.load(open(os.path.join(w, key + '.res.json'))) msl = os.path.join(w, key + '.metal'); air = msl[:-6] + '.air'; lib = os.path.join(out, key + '_b.metallib') r = subprocess.run(['spirv-cross', *XC, os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True) if r.returncode: return 'MSL 失败', r.stderr.strip()[:160], None src = open(msl).read() + if j['stage'] == 'vs': # 第二遍：只按整数用的输入改报 uint 重新翻译（见 int_inputs） | second pass: re-translate with inputs that are only used as integers reported as uint (see int_inputs) + extra = {l: t for l, t in int_inputs(src).items() if l not in j['attr']} + if extra: + j['attr'].update(extra); jp = os.path.join(w, f'job1_{key}.json'); json.dump([j], open(jp, 'w')) + rx = subprocess.run([XLATE, jp, w], capture_output=True, text=True) + if rx.returncode or not os.path.exists(os.path.join(w, key + '.spv')): return '翻译失败', ('整数输入重翻：' + rx.stderr.strip())[:160], None + r = subprocess.run(['spirv-cross', *XC, os.path.join(w, key + '.spv'), '--output', msl], capture_output=True, text=True) + if r.returncode: return 'MSL 失败', r.stderr.strip()[:160], None + src = open(msl).read() + res = json.load(open(os.path.join(w, key + '.res.json'))) try: subprocess.run(['xcrun', '-sdk', 'macosx', 'metal', '-c', msl, '-o', air], check=True) tmp_lib = lib + '.tmp-' + str(os.getpid()) diff --git a/docs/agents/03-bringup.md b/docs/agents/03-bringup.md index b62cb20..c671ada 100644 --- a/docs/agents/03-bringup.md +++ b/docs/agents/03-bringup.md @@ -64,6 +64,8 @@ Each entry: symptom → root cause → fix (where the fix lives). Check these fi - **Frames stop, no crash, the main thread sits in `nvnSyncWait(sync, -1)`, and the watchdog's wait-state line shows the main fence one short (`已提交 N / 已完成 N-1`)** → a GPU command buffer never completes. Check GPU load from outside: `ioreg -r -d 1 -c IOAccelerator | grep 'Device Utilization'` at 100 % means a shader is looping forever. Find the shaders first used in the last frames (`DM_NVN_PASSDUMP`, compare the `K` keys) and look at their `.metal` for an empty `do {} while (true);` with no `return` (see [04](04-graphics.md) section 1: the compute program start). - **Menus and dialogue boxes draw but contain no text; images and pictogram digits are fine** → the game rasterizes glyphs on the CPU into a *linear* texture in a CPU-visible pool (Mario: 2048×8192 R8, flags IMAGE|LINEAR). A private Metal texture never sees those writes. Linear textures in CPU-visible pools are now created on the pool buffer (`tex_linear` in `nvn_mtl.m`). `DM_NVN_TEXCPU=1` lists non-packaged textures in CPU-visible pools. +- **Characters (skinned models) invisible or stretched into streaks while the level renders** → integer vertex inputs (bone indices) translated as float; see [04](04-graphics.md) section 6. To find the draws: log model file ranges with `DM_PROBE=main:480f0` (x1 = file data, x2 = size), match the vertex/index buffer addresses in `DM_NVN_PASSDUMP_ALL` against them, then `DM_NVN_HILITE` their fragment shader. +- **Sky black** → CPU-written linear colour ramps (see the linear-texture entry above). - **Controller-button icons in text show as `?`** → those glyphs come from the system shared font's Nintendo-extension set; the host serves a desktop font for every shared-font type. Open. ### NVN object sizes (memory corruption) diff --git a/docs/agents/04-graphics.md b/docs/agents/04-graphics.md index d819521..b6069ef 100644 --- a/docs/agents/04-graphics.md +++ b/docs/agents/04-graphics.md @@ -52,7 +52,8 @@ Consequence: any optimization that infers "which textures a draw reads" from bou - **Compute code areas have a header** in Mario (`0x12345679`), not in D2R; `dmsh_build.py` strips it. - **Virtual memory pools** (flag 0x200): aliased onto the physical pool named by `MapVirtual` when all requests share one base (Mario's case); otherwise unimplemented. - **SPIRV-Cross failures**: 2 agl post-effect compute shaders still produce SPIR-V that SPIRV-Cross can't parse (`nullptr` exception). Before the compute start-offset fix it was 7: misdecoded code produced the bad SPIR-V. -- **CPU-written linear textures**: a texture with `NVN_TEXTURE_FLAGS_LINEAR` in a CPU-visible pool is created on the pool's `MTLBuffer` (`tex_linear`), so CPU writes (Mario's glyph cache) are visible to the GPU. Block-linear CPU-written textures still need the packaged-texture re-upload path. +- **CPU-written linear textures**: a texture with `NVN_TEXTURE_FLAGS_LINEAR` or `LINEAR_RENDER_TARGET` in a CPU-visible pool (Mario's glyph cache; its four 64×1 RGBA16F sky colour ramps, without which the sky is black) is created on the pool's `MTLBuffer` (`tex_linear`), so CPU writes (Mario's glyph cache) are visible to the GPU. Block-linear CPU-written textures still need the packaged-texture re-upload path. +- **Integer vertex inputs**: without PC reflection the translator declares every vertex input `float4`. Skinned models read their bone index (`R8UI` / `RG8UI` / `RGBA8UI`) as raw integer bits (`as_type(in.m_N.x) * 48`); Metal converted the integers to floats, every bone index was garbage, and all characters were stretched into streaks. `dmsh_build.py` (`int_inputs`) now re-translates a vertex shader with `uint` inputs when every use of an input is an integer bit cast (529 of 4,854 in Mario), and the host makes the Metal vertex format agree with the declared type (`vfmt_for_ct`; signedness swaps and 32-bit float→int reinterpretation keep the bits). A remaining mismatch is reported as `顶点属性格式 … 与着色器输入类型 … 对不上`. - **Vertex formats**: Mario uses `RGB10A2SN` (0x69) → `MTLVertexFormatInt1010102Normalized`. Unmapped formats fall back to a 4-component read and print `顶点属性格式 … 未映射` once. - **Instanced array draws** (`DrawArraysInstanced`) carry instance count and base instance in the `OP_DRAW` record.