Port Persona 5 Royal with Needle shaders and NVN texture fixes
This commit is contained in:
seedds committed
1 parent
c3a0880304
commit
be5b197640
25 files changed
+34399
-18
No files matched your search
@@ -114,22 +114,25 @@ Under each game: the IPA's size, and "60 fps with 1.2.0" where the game runs at
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/patricks-parabox.jpg" width="256" alt="Patrick's Parabox"><br>Patrick's Parabox<br><sub>~0.2 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/persona-5-royal.jpg" width="256" alt="Persona 5 Royal"><br>Persona 5 Royal<br><sub>~14.9 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pikmin-4.jpg" width="256" alt="Pikmin 4"><br>Pikmin 4<br><sub>~10.8 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-legends-z-a.jpg" width="256" alt="Pokémon Legends: Z-A"><br>Pokémon Legends: Z-A<br><sub>~6 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-legends-z-a.jpg" width="256" alt="Pokémon Legends: Z-A"><br>Pokémon Legends: Z-A<br><sub>~6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/pokemon-sword.jpg" width="256" alt="Pokémon Sword"><br>Pokémon Sword<br><sub>~12.4 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/raging-loop.jpg" width="256" alt="Raging Loop"><br>Raging Loop<br><sub>~0.9 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/rhythm-heaven-groove.jpg" width="256" alt="Rhythm Heaven Groove"><br>Rhythm Heaven Groove<br><sub>~3 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/rhythm-heaven-groove.jpg" width="256" alt="Rhythm Heaven Groove"><br>Rhythm Heaven Groove<br><sub>~3 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/romance-of-the-three-kingdoms-8-remake.jpg" width="256" alt="ROMANCE OF THE THREE KINGDOMS 8 REMAKE"><br>ROMANCE OF THE THREE KINGDOMS 8 REMAKE<br><sub>~16.8 GB · controller only</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/stardew-valley.jpg" width="256" alt="Stardew Valley"><br>Stardew Valley<br><sub>~0.6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/steins-gate-reboot.jpg" width="256" alt="STEINS;GATE RE:BOOT"><br>STEINS;GATE RE:BOOT<br><sub>~6 GB</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/steins-gate-reboot.jpg" width="256" alt="STEINS;GATE RE:BOOT"><br>STEINS;GATE RE:BOOT<br><sub>~6 GB</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/super-smash-bros-ultimate.jpg" width="256" alt="Super Smash Bros. Ultimate"><br>Super Smash Bros. Ultimate<br><sub>~17.5 GB · offline</sub></td>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/taiko-risshiden-v-dx.jpg" width="256" alt="Taiko Risshiden V DX"><br>Taiko Risshiden V DX<br><sub>~3.3 GB · Japanese · played by touch</sub></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" valign="top" width="33%"><img src="docs/images/unicorn-overlord.jpg" width="256" alt="Unicorn Overlord"><br>Unicorn Overlord<br><sub>~4 GB</sub></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# Persona 5 Royal (`01005CA01580E000`)
|
||||
|
||||
Update 1.0.2 (`v65536`) over the base game, SDK 13.3.2. Atlus GFD / Needle on NVN, with CRI middleware. Modules `main`, `subsdk0` (NintendoSDK_movie) and `sdk`.
|
||||
|
||||
**Current:** opening casino gameplay tested on Mac with the complete shader library. The unsigned 1.0.2 IPA is built (15,969,549,546 bytes, about 14.9 GiB). iOS and later battles have not been tested.
|
||||
|
||||
## What it needs
|
||||
|
||||
| Setting | Where | Why |
|
||||
|---|---|---|
|
||||
| `mem_reserve 0x100000000`, `mem_alias 0x10000000`, `mem_stack 0x10000000`, `app_pool 0xCD500000` | [`game.tsv`](game.tsv) | the NPDM declares no system resource; the game sets a heap in the existing 4 GB layout |
|
||||
| `mem_used 0x4c00000` | `game.tsv` | the rewritten modules require 76 MB, exceeding the default 64 MB deducted from the pool |
|
||||
| `pad_layout full` | `game.tsv` | movement and camera use both sticks |
|
||||
|
||||
## Rendering
|
||||
|
||||
- **Resolved:** the first boot trapped in `SetupSurfaceInfoMemoryInside`, at `main+0x1028fbc`, asserting `ctx.memorySize <= localmemsetupinfo.memorySize`. A 16x16 RGBA8 texture with flags `0x44` (`IMAGE | LINEAR_RENDER_TARGET`) and stride 128 received only 1 KB from the host; the game independently calculated 2 KB. The general fix is linked below. No executable cut is needed.
|
||||
|
||||
- **Resolved:** the casino clears only alpha on a 1920x1080 RGBA8 texture (`mask 0x8`). The general partial-channel texture clear implementation preserves RGB and supports explicit regions; no game-specific override is needed.
|
||||
|
||||
## Files and builds
|
||||
|
||||
- `CPK/ALL_USEU.CPK` contains 94,631 files, 74,521 CRILAYLA-compressed; `CPK/PATCH1.CPK` contains one USM movie. The host reads the archives through the game's own CRI file system.
|
||||
- The available release supports English, French, German, Spanish and Italian language codes. The build defaults to American English.
|
||||
- Only the merged update is extracted and registered. Fixed addresses in this note refer to `main` build `d4b150b29a931cd381e2a30318fc299f`.
|
||||
|
||||
## Shaders
|
||||
|
||||
- **Resolved:** the initial split found no shaders. NEEDLE `HNSHV002` containers hold 228 shader records in `main` and 30,418 in nine GSC files inside `ALL_USEU.CPK`. The descriptor-based reader extracts 30,646 unique keys: 29,526 pixel, 1,114 vertex and 6 compute, with 3,342 single-stage cover pairs.
|
||||
- All 13 shader control/code pairs captured during startup match the extracted package bytes and keys (8 from `main`, 5 from GSC files).
|
||||
- **Resolved:** all 30,646 keys translated successfully for both Mac and iOS, producing 27,501 variants. The full libraries use chain `dec5ee141a9f` and profile-input stamp `f53e8a12f4ad`; there are no unfillable driver reads or undecoded bindings. Final Mac gameplay validation loaded this full stamped library on base `7a3b958`. The port was subsequently integrated onto `c3a0880`, whose upstream image-atomic changes advance the chain to `467efdfd0b62`; the affected Metal regression tests were rebuilt and passed there, but a full game shader rebuild and gameplay replay on that chain have not been performed. The existing IPA remains the validated earlier-chain build.
|
||||
|
||||
## Testing and running
|
||||
|
||||
On 2026-10-25, update 1.0.2 was tested on an M4 Pro Mac with Metal API validation enabled. Boot, startup movies, title and language menus, opening dialogue, and casino gameplay rendered. Movement, camera control and the first two platform jumps were exercised. The full-library run exceeded 35 minutes without a crash, missing shader or Metal validation error. Recent casino frame-time windows measured p50 33.33 ms and p95 33.36 ms (about 30 fps); occasional longer frames occurred during the run. Resident memory was about 1.9 GB and GPU allocations about 1.75 GB. This is an opening-sequence check, not a full playthrough.
|
||||
|
||||
Build with `scripts/build_ipa_from_nsp.py` pointed at the NSP folder and the local keys file. The output is `Persona5Royal.ipa`; it contains no code signature or provisioning profile. For Mac testing use the generated `<title ID>_build/tools/dmhost` and `patched_v65536`, with `DM_ROOT` set to the build directory, `DM_ROMFS` to `extract/v65536/RomFS`, `DM_MSL_DIR` to `shader/cache`, and `DM_FONT_DIR` to `font`. Set `DM_NVN_SHMAP` to this profile's `shader_map.tsv`, `DM_LANG=en-US`, and writable `DM_SAVE` / `DM_TEMP` directories under the NSP folder. See the [control-file recipe](../../../docs/agents/05-debugging.md) for `DM_CTL`.
|
||||
|
||||
Choose offline mode at the network prompt, then New Game and English voices. Advance the opening dialogue; Plus shows and confirms movie skipping. In the casino, the left stick moves, the right stick controls the camera, and A jumps at marked platform edges.
|
||||
|
||||
## Found on this game
|
||||
|
||||
- [Linear render-target storage without the LINEAR flag](../../../docs/reference/nvn-objects.md#linear-render-target-storage-without-the-linear-flag): sizing, row pitch and deswizzle eligibility must all recognize `LINEAR_RENDER_TARGET` independently.
|
||||
- [Needle shader containers in main and GFD archives](../../../docs/reference/shaders.md#needle-shader-containers-in-main-and-gfd-archives): descriptor-based extraction of NVN control/code sections.
|
||||
|
||||
- [Texture clears with partial channel masks or regions](../../../docs/reference/rendering.md#texture-clears-with-partial-channel-masks-or-regions): a load/store draw applies the requested write mask and rectangle.
|
||||
|
||||
## Open
|
||||
|
||||
- Rebuild the complete game libraries and host on the current upstream chain, then repeat gameplay validation.
|
||||
- Test battles, later story scenes and save/load over a longer playthrough.
|
||||
- `SetRenderEnableConditional` still reports that the host draws unconditionally. No visual failure was identified in the tested opening; conditional rendering remains unvalidated.
|
||||
- Device validation remains for the user after sideloading.
|
||||
@@ -0,0 +1 @@
|
||||
# generated
|
||||
|
@@ -0,0 +1,40 @@
|
||||
# Persona 5 Royal 1.0.2, SDK 13.3.2. Atlus GFD / Needle, NVN, CRI middleware.
|
||||
# Mac bring-up and validation: README.md beside this profile.
|
||||
|
||||
# ---- Identity ----
|
||||
title_id 01005CA01580E000
|
||||
program_id 01005CA01580E000
|
||||
|
||||
# ---- App ----
|
||||
name Persona 5 Royal
|
||||
ipa Persona5Royal.ipa
|
||||
app_category public.app-category.role-playing-games
|
||||
pad_layout full
|
||||
|
||||
# ---- Versions ----
|
||||
build v65536 1.0.2 main=d4b150b29a931cd381e2a30318fc299f subsdk0=a8cb97803f7f442a3b6cce1c7cd4ee4a sdk=357efc198e728cb5d55265aff304f03c
|
||||
package v65536
|
||||
shader_versions v65536
|
||||
|
||||
# ---- Files ----
|
||||
# every module main imports from
|
||||
modules main subsdk0 sdk
|
||||
romfs_marker CPK/ALL_USEU.CPK
|
||||
zip_store .cpk:.metallib
|
||||
|
||||
# ---- Memory ----
|
||||
# main sets a heap of at most the pool − 64 MB: a 3328 MB heap region
|
||||
mem_reserve 0x100000000
|
||||
mem_alias 0x10000000
|
||||
mem_stack 0x10000000
|
||||
app_pool 0xCD500000
|
||||
# the NPDM declares 0 MB of system resource: IsVirtualAddressMemoryEnabled is false, as on Horizon
|
||||
sys_resource 0x0
|
||||
|
||||
# ---- Shader split ----
|
||||
shader_groups /CPK/=cpk /app_config/=appconfig
|
||||
shader_note Persona 5 Royal shaders
|
||||
|
||||
# ---- After the rewrite ----
|
||||
# The rewritten modules require 76 MB, more than the default 64 MB deducted from the pool.
|
||||
mem_used 0x4c00000
|
||||
|
@@ -0,0 +1,2 @@
|
||||
# Prewarm seed list (shipped in the app, next to shader_map.tsv). Rows: S<TAB>shader key<TAB>stage / R<TAB>render pipeline key (pso_skey hex) / C<TAB>compute shader key.
|
||||
# Provisional profile: empty until recorded on the Mac (DM_PSO_MANIFEST=off DM_PSO_RECORD).
|
||||
|
@@ -0,0 +1,7 @@
|
||||
# Answer table for translation queries (generated by dm/gfx/tools/shader_split.py, do not edit): key <TAB> fact JSON. This game has no PC version to pair with: only the thread group / shared memory of compute shaders (taken from the control section); other queries get the translator defaults
|
||||
2e4f1850a6608b13 {"local":[32,1,1],"shared":0}
|
||||
2fded99906f1e547 {"local":[64,1,1],"shared":0}
|
||||
6601414b49fb0a19 {"local":[32,1,1],"shared":0}
|
||||
6ad47de9a3c6f402 {"local":[32,1,1],"shared":0}
|
||||
86f58a25fc7a13e4 {"local":[32,1,1],"shared":0}
|
||||
c8595847942efa37 {"local":[64,1,1],"shared":0}
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 19.
|
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
# kind: evidence control-section key conflict rulings: empty
|
||||
dm_ctl_sha16 chosen_pc class candidates records
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -81,6 +81,7 @@ One row per profile; the notes are each profile's `README.md`.
|
||||
| [OCTOPATH TRAVELER II](0100A3501946E000/README.md) | `0100A3501946E000` | 1.1.1 | 12.3.2 | Unreal Engine 4.27 | NVN |
|
||||
| [Paper Mario: The Thousand-Year Door](0100ECD018EBE000/README.md) | `0100ECD018EBE000` | 1.0.1 | 15.3.2 | Intelligent Systems' engine on NintendoWare | NVN |
|
||||
| [Patrick's Parabox](0100E44016C22000/README.md) | `0100E44016C22000` | 1.0.1 | 13.3.7 | Unity 2020.3.30f1 IL2CPP | NVN |
|
||||
| [Persona 5 Royal](01005CA01580E000/README.md) | `01005CA01580E000` | 1.0.2 | 13.3.2 | Atlus GFD / Needle (CRI middleware) | NVN |
|
||||
| [Pikmin 4](0100B7C00933A000/README.md) | `0100B7C00933A000` | 1.1.1 | 10.4.8 | Unreal Engine 4.26.2 | NVN |
|
||||
| [Pokémon Legends: Z-A](0100F43008C44000/README.md) | `0100F43008C44000` | 2.0.2 | 19.3.11 | Game Freak's Trinity engine | NVN |
|
||||
| [Pokémon Sword](0100ABF008968000/README.md) | `0100ABF008968000` | 1.3.2 | 7.3.2 | Game Freak's engine (before Trinity) | NVN |
|
||||
|
||||
+38
-8
@@ -1986,7 +1986,7 @@ static id<MTLTexture> tex_linear(const tex_t *t) {
|
||||
if (t->d.target != NVN_TEXTURE_TARGET_2D || t->d.levels > 1 || t->d.samples > 1) return nil;
|
||||
uint32_t pf = 0; uint8_t swz[4]; int bw, bh, bpb = nvn_fmt_bpb(t->d.format, &bw, &bh);
|
||||
if (!nvn_fmt_to_mtl(t->d.format, &pf, swz) || !pf || !bpb || bw != 1 || bh != 1) return nil;
|
||||
size_t row = t->d.stride ? t->d.stride : (size_t)t->d.width * bpb;
|
||||
size_t row = t->L.layer_size / (t->d.height ? t->d.height : 1); // same explicit or aligned stride as GetStorageSize
|
||||
void *mb; uint64_t mo;
|
||||
if (!mtl_addr((uint64_t)t->mem, (uint64_t)row * (t->d.height ? t->d.height : 1), &mb, &mo)) { RED_ONCE(64, "linear texture storage is not in a memory pool (%ux%u format %#x)", t->d.width, t->d.height, t->d.format); return nil; }
|
||||
id<MTLBuffer> b = (__bridge id<MTLBuffer>)mb;
|
||||
@@ -7850,7 +7850,8 @@ static unsigned mtl_pf_chmask(MTLPixelFormat pf) {
|
||||
}
|
||||
// GPU: clear a texture region with the given value (previously mislisted as a no-op, called 5 times at initialization; the initial value of a Metal private texture is undefined, and the game relies on the cleared initial value).
|
||||
// Whole level and whole face, all four channels written -> open one clear-only render pass per layer / depth slice (the hardware converts the clear value by the texture format, including integer and depth);
|
||||
// view with changed format: clear on a view of that format; partial region, partial channels, compressed / packaged textures (no render target usage): report a failure, to be implemented
|
||||
// View with changed format: clear on that view. Partial colour regions / channel masks use a triangle and preserve everything else.
|
||||
static void clear_tex_tri(id<MTLTexture> t, unsigned lv, unsigned l0, const p_clear_tex *c);
|
||||
static void clear_tex(const p_clear_tex *c) {
|
||||
if (c->tex) zskip_del(c->tex->serial);
|
||||
id<MTLTexture> t = tex_mtl(c->tex);
|
||||
@@ -7860,8 +7861,8 @@ static void clear_tex(const p_clear_tex *c) {
|
||||
unsigned lv = c->has_view && c->view.set_levels ? c->view.base_level : 0, l0 = c->has_view && c->view.set_layers ? c->view.min_layer : 0;
|
||||
NSUInteger w = MAX(t.width >> lv, 1u), h = MAX(t.height >> lv, 1u);
|
||||
int is3d = t.textureType == MTLTextureType3D, nz = c->r.depth ? c->r.depth : 1;
|
||||
if (!(t.usage & MTLTextureUsageRenderTarget) && !mtl_fmt_block(t.pixelFormat) && t.sampleCount == 1 && !c->r.xoffset && !c->r.yoffset && (NSUInteger)c->r.width == w && (NSUInteger)c->r.height == h && (c->mask & mtl_pf_chmask(t.pixelFormat)) == mtl_pf_chmask(t.pixelFormat))
|
||||
t = tex_upgrade(c->tex, t, MTLTextureUsageRenderTarget, t.pixelFormat); // a whole-level clear of a texture made without render-target use (packaged): give it that use
|
||||
if (!(t.usage & MTLTextureUsageRenderTarget) && !mtl_fmt_block(t.pixelFormat) && t.sampleCount == 1)
|
||||
t = tex_upgrade(c->tex, t, MTLTextureUsageRenderTarget, t.pixelFormat); // a clear of a texture made without render-target use (packaged): give it that use
|
||||
if (c->has_view && c->view.set_format) { // View with changed format: the clear value is interpreted in the view format (all 5 of the game's calls are the ui version with an integer view), and the clear is done on a view of the same format
|
||||
uint32_t vpf = 0; uint8_t swz[4];
|
||||
if (!nvn_fmt_to_mtl(c->view.format, &vpf, swz) || !vpf) { RED_ONCE(42, "texture clear view format %#x has no equivalent", c->view.format); return; }
|
||||
@@ -7876,6 +7877,7 @@ static void clear_tex(const p_clear_tex *c) {
|
||||
int depth = pf == MTLPixelFormatDepth32Float || pf == MTLPixelFormatDepth16Unorm || pf == MTLPixelFormatDepth32Float_Stencil8, stencil = pf == MTLPixelFormatStencil8 || pf == MTLPixelFormatDepth32Float_Stencil8;
|
||||
if (c->r.xoffset || c->r.yoffset || (NSUInteger)c->r.width != w || (NSUInteger)c->r.height != h || (c->mask & mtl_pf_chmask(pf)) != mtl_pf_chmask(pf)
|
||||
|| !(t.usage & MTLTextureUsageRenderTarget)) {
|
||||
if (!depth && !stencil && (t.usage & MTLTextureUsageRenderTarget)) { clear_tex_tri(t, lv, l0, c); return; }
|
||||
RED_ONCE(44, "texture clear case not implemented: region %d,%d %dx%d (level size %lux%lu) mask %#x render-target capable %d (format %#x flags %#x)", c->r.xoffset, c->r.yoffset, c->r.width, c->r.height,
|
||||
(unsigned long)w, (unsigned long)h, c->mask, !!(t.usage & MTLTextureUsageRenderTarget), c->tex->d.format, c->tex->d.flags);
|
||||
return;
|
||||
@@ -8002,7 +8004,7 @@ static int clr_scissor(MTLScissorRect *sc) { // the clear's scissor in the ope
|
||||
if (x1 <= x0 || y1 <= y0) return 0;
|
||||
*sc = (MTLScissorRect){(NSUInteger)x0, (NSUInteger)y0, (NSUInteger)(x1 - x0), (NSUInteger)(y1 - y0)}; return 1;
|
||||
}
|
||||
static void clr_tri(id<MTL4RenderCommandEncoder> en, id<MTLRenderPipelineState> ps, id<MTLDepthStencilState> ds, MTLScissorRect sc, const uint32_t val[4], float z, uint32_t sref) {
|
||||
static void clr_tri(id<MTL4RenderCommandEncoder> en, id<MTLRenderPipelineState> ps, id<MTLDepthStencilState> ds, MTLScissorRect sc, const uint32_t val[4], float z, uint32_t sref, NSUInteger tw, NSUInteger th) {
|
||||
static id<MTL4ArgumentTable> vt, ft;
|
||||
if (!ft) { NSError *e = nil; MTL4ArgumentTableDescriptor *ad = [MTL4ArgumentTableDescriptor new]; ad.maxBufferBindCount = 1;
|
||||
vt = rec_table([g_dev newArgumentTableWithDescriptor:ad error:&e]); ft = vt ? rec_table([g_dev newArgumentTableWithDescriptor:ad error:&e]) : nil;
|
||||
@@ -8012,11 +8014,39 @@ static void clr_tri(id<MTL4RenderCommandEncoder> en, id<MTLRenderPipelineState>
|
||||
[en setArgumentTable:vt atStages:MTLRenderStageVertex]; [en setArgumentTable:ft atStages:MTLRenderStageFragment];
|
||||
enc_r_pso(en, ps); g_enc_pso = nil; enc_r_dss(en, ds); g_enc_dss = nil; // the next draw sets its own again
|
||||
if (g_enc_vis) { enc_r_visibility(en, MTLVisibilityResultModeDisabled, 0); g_enc_vis = 0; } // not counted by an occlusion query; the next counted draw turns counting back on
|
||||
enc_r_viewport(en, (MTLViewport){0, 0, (double)g_pass.tw, (double)g_pass.th, 0, 1}); enc_r_scissor(en, sc);
|
||||
enc_r_viewport(en, (MTLViewport){0, 0, (double)tw, (double)th, 0, 1}); enc_r_scissor(en, sc);
|
||||
enc_r_cull(en, MTLCullModeNone); enc_r_fill(en, MTLTriangleFillModeFill); enc_r_depth_bias(en, 0, 0, 0); enc_r_depth_clip(en, YES); enc_r_stencil_ref(en, sref, sref); g_ed.ok = 0;
|
||||
enc_r_draw(en, MTLPrimitiveTypeTriangle, 0, 3);
|
||||
[en setArgumentTable:g_vt atStages:MTLRenderStageVertex]; [en setArgumentTable:g_ft atStages:MTLRenderStageFragment];
|
||||
}
|
||||
static void clear_tex_tri(id<MTLTexture> t, unsigned lv, unsigned l0, const p_clear_tex *c) {
|
||||
int mask = c->mask & mtl_pf_chmask(t.pixelFormat); if (!mask) return;
|
||||
NSUInteger w = MAX(t.width >> lv, 1u), h = MAX(t.height >> lv, 1u);
|
||||
long x0 = MAX(c->r.xoffset, 0), y0 = MAX(c->r.yoffset, 0);
|
||||
long x1 = MIN((long)c->r.xoffset + c->r.width, (long)w), y1 = MIN((long)c->r.yoffset + c->r.height, (long)h);
|
||||
if (x1 <= x0 || y1 <= y0) return;
|
||||
int kind = pf_int_kind(t.pixelFormat);
|
||||
clr_key k = {0}; k.kind = kind; k.mask = mask; k.cfmt[0] = t.pixelFormat; k.samples = t.sampleCount;
|
||||
id<MTLRenderPipelineState> ps = clr_pso(&k); id<MTLDepthStencilState> ds = clr_dss(0, 0); if (!ps || !ds) return;
|
||||
uint32_t v[4];
|
||||
for (int i = 0; i < 4; i++) {
|
||||
if (c->is_uint && kind) { v[i] = c->c.u[i]; continue; }
|
||||
float f = c->is_uint ? (float)c->c.u[i] : c->c.f[i]; int32_t si;
|
||||
if (kind == 1) v[i] = f > 0 ? (f < 4294967040.f ? (uint32_t)f : 0xffffffffu) : 0;
|
||||
else if (kind == 2) { si = !(f == f) ? 0 : f >= 2147483520.f ? INT32_MAX : f <= -2147483648.f ? INT32_MIN : (int32_t)f; memcpy(&v[i], &si, 4); }
|
||||
else memcpy(&v[i], &f, 4);
|
||||
}
|
||||
rp_end(); if (g_blit) { enc_b_end(g_blit); g_blit = nil; }
|
||||
for (int z = 0, nz = c->r.depth ? c->r.depth : 1; z < nz; z++) {
|
||||
MTL4RenderPassDescriptor *rp = [MTL4RenderPassDescriptor new];
|
||||
rp.colorAttachments[0].texture = t; rp.colorAttachments[0].level = lv;
|
||||
if (t.textureType == MTLTextureType3D) rp.colorAttachments[0].depthPlane = c->r.zoffset + z;
|
||||
else rp.colorAttachments[0].slice = l0 + c->r.zoffset + z;
|
||||
rp.colorAttachments[0].loadAction = MTLLoadActionLoad; rp.colorAttachments[0].storeAction = MTLStoreActionStore;
|
||||
ser_gate(); id<MTL4RenderCommandEncoder> en = enc_r_begin(rp);
|
||||
clr_tri(en, ps, ds, (MTLScissorRect){x0,y0,x1-x0,y1-y0}, v, 0, 0, w, h); enc_r_end(en);
|
||||
}
|
||||
}
|
||||
static void clr_note(int slot, int depth, int stencil) { // as udc_note_draw for a draw: the attachment was written (zero-source skip, unwritten-attachment store)
|
||||
if (slot >= 0 && R.rt.color[slot]) { zskip_del(R.rt.color[slot]->serial); g_zskip_cand_ok[slot] = 0; }
|
||||
for (int i = 0; i < g_nsd; i++) { sd_att *t = &g_sd[i];
|
||||
@@ -8044,7 +8074,7 @@ static void clear_color_tri(const p_clear_color *c) {
|
||||
if (kind == 1) v[i] = f > 0 ? (f < 4294967040.f ? (uint32_t)f : 0xffffffffu) : 0;
|
||||
else if (kind == 2) { si = !(f == f) ? 0 : f >= 2147483520.f ? INT32_MAX : f <= -2147483648.f ? INT32_MIN : (int32_t)f; memcpy(&v[i], &si, 4); }
|
||||
else memcpy(&v[i], &f, 4); }
|
||||
clr_tri(en, ps, ds, sc, v, 0, 0); clr_note(at, 0, 0);
|
||||
clr_tri(en, ps, ds, sc, v, 0, 0, g_pass.tw, g_pass.th); clr_note(at, 0, 0);
|
||||
if (!g_clr_tri[0]++) MLOG("mid-pass / partial-mask / partial-scissor color clear: drawn as a full-pass triangle (first case pass %llu slot %d mask %#x scissor %lu,%lu %lux%lu)", g_stat_pass, at, mask,
|
||||
(unsigned long)sc.x, (unsigned long)sc.y, (unsigned long)sc.width, (unsigned long)sc.height);
|
||||
PD("c %llu color clear drawn as a triangle slot%d ser=%llu mask=%#x scissor=%lu,%lu,%lu,%lu color=%g,%g,%g,%g", g_stat_pass, at, (unsigned long long)R.rt.color[at]->serial, mask,
|
||||
@@ -8063,7 +8093,7 @@ static void clear_ds_tri(const p_clear_ds *c) {
|
||||
id<MTLRenderPipelineState> ps = clr_pso(&k); id<MTLDepthStencilState> ds = clr_dss(dw, sm); if (!ps || !ds) return;
|
||||
float z = c->depth > 0 ? (c->depth < 1 ? c->depth : 1) : 0; // as clearDepth: within [0, 1] (NaN → 0)
|
||||
const uint32_t v[4] = {0};
|
||||
clr_tri(en, ps, ds, sc, v, z, (uint32_t)c->stencil & 0xff); clr_note(-1, dw, sm != 0);
|
||||
clr_tri(en, ps, ds, sc, v, z, (uint32_t)c->stencil & 0xff, g_pass.tw, g_pass.th); clr_note(-1, dw, sm != 0);
|
||||
if (!g_clr_tri[1]++) MLOG("mid-pass scissored depth-stencil clear: drawn as a full-pass triangle (first case pass %llu depth %d stencil mask %#x scissor %lu,%lu %lux%lu)", g_stat_pass, dw, sm,
|
||||
(unsigned long)sc.x, (unsigned long)sc.y, (unsigned long)sc.width, (unsigned long)sc.height);
|
||||
PD("c %llu depth-stencil clear drawn as a triangle ser=%llu depth=%g(%d) stencil=%d mask=%#x scissor=%lu,%lu,%lu,%lu", g_stat_pass, R.rt.depth ? (unsigned long long)R.rt.depth->serial : 0ull, z, dw, c->stencil, sm,
|
||||
|
||||
@@ -68,7 +68,7 @@ static void samples_xy(int s, uint32_t *sx, uint32_t *sy) {
|
||||
// its 1024×1024 ASTC 6×6 textures (48 GOBs wide instead of 43, level 1 at 0x90000 instead of 0x81000) read 2,183 all-zero blocks in level 0 and 3,043 in level 1, which ASTC
|
||||
// decodes as its error colour (magenta clothes)
|
||||
enum { SPARSE_TILE = 0x10000 };
|
||||
static int sparse_of(const nvn_texdesc *d) { return (d->flags & 0x20) && !(d->flags & 0x10) && d->target != 10; }
|
||||
static int sparse_of(const nvn_texdesc *d) { return (d->flags & 0x20) && !(d->flags & 0x50) && d->target != 10; }
|
||||
// Offset of level `level` from the start of its layer, and the GOB columns it is laid out with (*gobs, may be NULL)
|
||||
static uint64_t level_walk(const nvn_texdesc *d, unsigned level, unsigned dpl, uint32_t w, uint32_t h, uint32_t z,
|
||||
unsigned tileH, unsigned tileD, fmt_t t, uint32_t *gobs) {
|
||||
@@ -105,7 +105,10 @@ void nvn_tex_layout(const nvn_texdesc *d, nvn_texlayout *o) {
|
||||
o->layer_size = o->size = (uint64_t)t.bpb * d->width;
|
||||
o->align = t.bpb == 12 ? 4 : t.bpb; return;
|
||||
}
|
||||
if (d->flags & 0x10) { // LINEAR: row pitch = stride or aligned to 32
|
||||
// LINEAR_RENDER_TARGET (0x40) is linear by itself. Persona 5 Royal creates
|
||||
// a 16x16 RGBA8 target with flags 0x44 and stride 128; block-linear sizing
|
||||
// returned 1024 instead of 2048 bytes and tripped its surface-memory assertion.
|
||||
if (d->flags & 0x50) { // LINEAR or LINEAR_RENDER_TARGET
|
||||
o->align = (d->flags & 0x40) ? 128 : 32;
|
||||
uint32_t stride = d->stride ? d->stride : (t.bpb * blk(w, t.bw) + o->align - 1) & ~(o->align - 1);
|
||||
o->layer_size = o->size = (uint64_t)stride * blk(h, t.bh);
|
||||
@@ -134,7 +137,7 @@ uint64_t nvn_tex_level_offset(const nvn_texdesc *d, unsigned level) {
|
||||
fmt_t t = fmt_of(d->format);
|
||||
unsigned dpl; int layered; uint32_t w, h, z;
|
||||
shape(d, &dpl, &layered, &w, &h, &z);
|
||||
if (!t.bpb || (d->flags & 0x10) || d->target == 10) return 0;
|
||||
if (!t.bpb || (d->flags & 0x50) || d->target == 10) return 0;
|
||||
nvn_texlayout L; nvn_tex_layout(d, &L);
|
||||
return level_offset(d, level, dpl, w, h, z, L.tileH, L.tileD, t);
|
||||
}
|
||||
@@ -143,7 +146,7 @@ uint32_t nvn_tex_level_gobs(const nvn_texdesc *d, unsigned level) {
|
||||
fmt_t t = fmt_of(d->format);
|
||||
unsigned dpl; int layered; uint32_t w, h, z, g = 0;
|
||||
shape(d, &dpl, &layered, &w, &h, &z);
|
||||
if (!t.bpb || (d->flags & 0x10) || d->target == 10) return 0;
|
||||
if (!t.bpb || (d->flags & 0x50) || d->target == 10) return 0;
|
||||
nvn_texlayout L; nvn_tex_layout(d, &L);
|
||||
level_walk(d, level, dpl, w, h, z, L.tileH, L.tileD, t, &g);
|
||||
return g;
|
||||
|
||||
@@ -74,6 +74,8 @@ t_primitive_restart
|
||||
t_primitive_restart.dSYM/
|
||||
t_clear_ds
|
||||
t_clear_ds.dSYM/
|
||||
t_clear_tex
|
||||
t_clear_tex.dSYM/
|
||||
t_upgrade_alias
|
||||
t_upgrade_alias.dSYM/
|
||||
t_stencil_view
|
||||
|
||||
@@ -122,6 +122,12 @@ t_primitive_restart: t_primitive_restart.m $(MTL_T_DEP)
|
||||
clang $(CFLAGS) -fobjc-arc -o $@ t_primitive_restart.m $(MTL_T_SRC) $(FW)
|
||||
t_clear_ds: t_clear_ds.m $(MTL_T_DEP)
|
||||
clang $(CFLAGS) -fobjc-arc -o $@ t_clear_ds.m $(MTL_T_SRC) $(FW)
|
||||
t_clear_tex: t_clear_tex.m $(MTL_T_DEP)
|
||||
clang $(CFLAGS) -fobjc-arc -o $@ t_clear_tex.m $(MTL_T_SRC) $(FW)
|
||||
.PHONY: check_t_clear_tex
|
||||
check_t_clear_tex: t_clear_tex
|
||||
MTL_DEBUG_LAYER=1 ./t_clear_tex
|
||||
run: check_t_clear_tex
|
||||
t_upgrade_alias: t_upgrade_alias.m $(MTL_T_DEP)
|
||||
clang $(CFLAGS) -fobjc-arc -o $@ t_upgrade_alias.m $(MTL_T_SRC) $(FW)
|
||||
t_stencil_view: t_stencil_view.m $(MTL_T_DEP)
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
// Texture clears preserve channels and pixels outside their explicit mask and region.
|
||||
#include "../nvn/nvn_mtl.m"
|
||||
void *front_metal_layer(void) { return NULL; }
|
||||
module_t *module_of(const void *p) { (void)p; return NULL; }
|
||||
module_t g_mod[MAX_MOD]; int g_nmod;
|
||||
void svc_thread_dump(void) {}
|
||||
uint64_t now_ns(void) { return bp_now(); }
|
||||
static int checks, failed;
|
||||
#define CHECK(c, label) do { checks++; if (!(c)) { failed++; fprintf(stderr,"FAIL: %s\n",label); } } while (0)
|
||||
|
||||
int main(void) { @autoreleasepool {
|
||||
mtl_init(); id<MTLCommandQueue> queue=[g_dev newCommandQueue];
|
||||
struct { int mask, x, y, w, h, layer, level, ui; const char *label; } cases[]={
|
||||
{8,0,0,8,8,0,0,0,"alpha-only clear preserves RGB"},
|
||||
{5,2,1,3,2,0,0,0,"masked rectangle preserves other pixels and channels"},
|
||||
{15,2,1,3,2,0,0,0,"whole-channel rectangle"},
|
||||
{8,0,0,4,4,1,1,0,"mip and array layer selection"},
|
||||
{8,0,0,8,8,0,0,1,"integer alpha clear preserves exact values"},
|
||||
{0,0,0,8,8,0,0,0,"empty channel mask writes nothing"},
|
||||
{15,0,0,8,8,0,0,0,"whole clear keeps load-action path"},
|
||||
};
|
||||
for(unsigned i=0;i<sizeof cases/sizeof *cases;i++) {
|
||||
tex_t t={.magic=MAGIC('T','E','X',0),.serial=99601+i};
|
||||
t.d=(nvn_texdesc){.target=NVN_TEXTURE_TARGET_2D_ARRAY,.format=cases[i].ui?NVN_FORMAT_RGBA8UI:NVN_FORMAT_RGBA8,.width=8,.height=8,.depth=2,.levels=2,.samples=1};
|
||||
MTLTextureDescriptor *d=tex_desc(&t,0); d.usage|=MTLTextureUsageRenderTarget;
|
||||
id<MTLTexture> base=[g_dev newTextureWithDescriptor:d]; CHECK(base!=nil,"target allocates"); if(!base)continue;
|
||||
for(int layer=0;layer<2;layer++)for(int lv=0;lv<2;lv++) {
|
||||
MTLRenderPassDescriptor *rp=[MTLRenderPassDescriptor renderPassDescriptor];
|
||||
rp.colorAttachments[0].texture=base;rp.colorAttachments[0].slice=layer;rp.colorAttachments[0].level=lv;
|
||||
rp.colorAttachments[0].loadAction=MTLLoadActionClear;rp.colorAttachments[0].storeAction=MTLStoreActionStore;
|
||||
rp.colorAttachments[0].clearColor=cases[i].ui?MTLClearColorMake(17,34,51,68):MTLClearColorMake(17./255,34./255,51./255,68./255);
|
||||
id<MTLCommandBuffer> cb=[queue commandBuffer];[[cb renderCommandEncoderWithDescriptor:rp] endEncoding];[cb commit];[cb waitUntilCompleted];
|
||||
}
|
||||
res_add(base);mtl_tex_init(&t);serial_put(t.serial,base);
|
||||
state_reset();memset(&R,0,sizeof R);mtl_sub_begin();
|
||||
// ClearTexture uses its region, independently of the current draw scissor.
|
||||
R.has_sc=1;R.sc=(p_scissor){0,0,1,1};
|
||||
struct { op_hdr head; p_clear_tex clear; op_hdr end; } ops={0};
|
||||
ops.head=(op_hdr){OP_CLEAR_TEX,sizeof(op_hdr)+sizeof(p_clear_tex)};
|
||||
ops.clear=(p_clear_tex){.tex=&t,.has_view=1,.is_uint=cases[i].ui,.mask=cases[i].mask,
|
||||
.r={cases[i].x,cases[i].y,0,cases[i].w,cases[i].h,1}};
|
||||
ops.clear.view=(view_t){.set_levels=1,.base_level=cases[i].level,.set_layers=1,.min_layer=cases[i].layer};
|
||||
for(int ch=0;ch<4;ch++)if(cases[i].ui)ops.clear.c.u[ch]=201+ch;else ops.clear.c.f[ch]=(201.+ch)/255;
|
||||
ops.end=(op_hdr){OP_END,sizeof(op_hdr)};
|
||||
mtl_exec((const uint8_t *)&ops);mtl_sub_commit();mtl_drain();
|
||||
CHECK(!mtl_last_commit_error(),"production texture clear completes");
|
||||
int valid=1;
|
||||
for(int layer=0;layer<2;layer++)for(int lv=0;lv<2;lv++) {
|
||||
int size=8>>lv;id<MTLBuffer> out=[g_dev newBufferWithLength:256*size options:MTLResourceStorageModeShared];
|
||||
id<MTLCommandBuffer> cb=[queue commandBuffer];id<MTLBlitCommandEncoder> en=[cb blitCommandEncoder];
|
||||
[en copyFromTexture:base sourceSlice:layer sourceLevel:lv sourceOrigin:MTLOriginMake(0,0,0) sourceSize:MTLSizeMake(size,size,1) toBuffer:out destinationOffset:0 destinationBytesPerRow:256 destinationBytesPerImage:256*size];
|
||||
[en endEncoding];[cb commit];[cb waitUntilCompleted];CHECK(cb.status==MTLCommandBufferStatusCompleted,"readback completes");
|
||||
uint8_t *p=out.contents;
|
||||
for(int y=0;y<size;y++)for(int x=0;x<size;x++)for(int ch=0;ch<4;ch++) {
|
||||
int hit=layer==cases[i].layer&&lv==cases[i].level&&x>=cases[i].x&&x<cases[i].x+cases[i].w&&y>=cases[i].y&&y<cases[i].y+cases[i].h&&(cases[i].mask&(1<<ch));
|
||||
valid&=p[y*256+x*4+ch]==(hit?201+ch:17*(ch+1));
|
||||
}
|
||||
}
|
||||
CHECK(valid,cases[i].label);printf("%s: %s\n",cases[i].label,valid?"PASS":"FAIL");mtl_tex_release(&t);
|
||||
}
|
||||
printf("texture clears: %d checks, %d failed\n",checks,failed);return failed!=0;
|
||||
} }
|
||||
@@ -42,6 +42,32 @@ static void run_case(int format) {
|
||||
EXPECT("GPU image writes -> CPU pool map, 14400 literal pixels", 0, bad);
|
||||
h_nvnTextureFinalize(&tex); mtl_pool_del(mem); free(mem);
|
||||
}
|
||||
static void render_target_stride_case(void) {
|
||||
void *mem = NULL; if (posix_memalign(&mem, getpagesize(), 65536)) abort(); memset(mem, 0, 65536);
|
||||
pool_t pool = {.mem=mem, .size=65536, .flags=0x24}; mtl_pool_add(mem, 65536);
|
||||
NVNtextureBuilder tb; NVNtexture tex; h_nvnTextureBuilderSetDefaults(&tb);
|
||||
h_nvnTextureBuilderSetWidth(&tb, 16); h_nvnTextureBuilderSetHeight(&tb, 16);
|
||||
h_nvnTextureBuilderSetFormat(&tb, NVN_FORMAT_RGBA8);
|
||||
h_nvnTextureBuilderSetFlags(&tb, NVN_TEXTURE_FLAGS_LINEAR_RENDER_TARGET);
|
||||
h_nvnTextureBuilderSetStorage(&tb, (const NVNmemoryPool *)&pool, 0);
|
||||
EXPECT("linear target inferred stride storage", 2048, h_nvnTextureBuilderGetStorageSize(&tb));
|
||||
EXPECT("linear target init", 1, h_nvnTextureInitialize(&tex, &tb));
|
||||
id<MTLTexture> t = tex_mtl((const tex_t *)&tex);
|
||||
EXPECT("linear target has buffer backing", 1, t.buffer != nil);
|
||||
EXPECT("linear target inferred row pitch", 128, t.bufferBytesPerRow);
|
||||
id<MTLCommandBuffer> cb = q_cb(); MTLRenderPassDescriptor *pd = [MTLRenderPassDescriptor renderPassDescriptor];
|
||||
pd.colorAttachments[0].texture=t; pd.colorAttachments[0].loadAction=MTLLoadActionClear;
|
||||
pd.colorAttachments[0].storeAction=MTLStoreActionStore; pd.colorAttachments[0].clearColor=MTLClearColorMake(1,0,0,1);
|
||||
[[cb renderCommandEncoderWithDescriptor:pd] endEncoding]; [cb commit]; [cb waitUntilCompleted];
|
||||
EXPECT("linear target clear completed", MTLCommandBufferStatusCompleted, cb.status);
|
||||
static const uint8_t red[]={255,0,0,255}; int bad=0, padding=0;
|
||||
for (int y=0;y<16;y++) {
|
||||
for (int x=0;x<16;x++) bad += memcmp((uint8_t *)mem+y*128+x*4,red,4)!=0;
|
||||
for (int x=64;x<128;x++) padding += ((uint8_t *)mem)[y*128+x]!=0;
|
||||
}
|
||||
EXPECT("linear target clear uses padded rows", 0, bad); EXPECT("linear target leaves row padding alone", 0, padding);
|
||||
h_nvnTextureFinalize(&tex); mtl_pool_del(mem); free(mem);
|
||||
}
|
||||
static void scaled_copy_case(void) {
|
||||
void *mem = NULL; if (posix_memalign(&mem, getpagesize(), 262144)) abort(); memset(mem, 0, 262144);
|
||||
pool_t pool = {.mem=mem, .size=262144, .flags=0x24}; mtl_pool_add(mem, 262144);
|
||||
@@ -69,6 +95,7 @@ static void scaled_copy_case(void) {
|
||||
h_nvnTextureFinalize(&dst); mtl_tex_release(&src); mtl_pool_del(mem); free(mem);
|
||||
}
|
||||
int main(void) { @autoreleasepool { mtl_init(); run_case(NVN_FORMAT_RGBA8); run_case(NVN_FORMAT_RGBA8_SRGB);
|
||||
render_target_stride_case();
|
||||
scaled_copy_case();
|
||||
printf("LINEAR_READBACK cases=%d failed=%d\n",cases,failed); return failed != 0;
|
||||
} }
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Needle's embedded NVN shader records, without game data."""
|
||||
from pathlib import Path
|
||||
import struct
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
|
||||
import shader_split as split
|
||||
|
||||
|
||||
def shader(stage):
|
||||
ctl, code = bytearray(0x850), bytearray(0x100)
|
||||
ctl[:4] = split.CTL_MAGIC
|
||||
struct.pack_into('<I', ctl, 0x20, len(ctl))
|
||||
struct.pack_into('<I', ctl, 0x704, len(code))
|
||||
if stage == 'cs':
|
||||
struct.pack_into('<3I', ctl, 0x748, 8, 4, 1)
|
||||
else:
|
||||
code[:4] = split.GFX_MAGIC
|
||||
struct.pack_into('<I', code, 0x30, (1 if stage == 'vs' else 5) << 10 | 1)
|
||||
if stage == 'ps': struct.pack_into('<I', code, 0x48, 1) # pixel shader reads generic location 0
|
||||
header = bytearray(0x90)
|
||||
struct.pack_into('<4I', header, 0x30, 0x90, 0x90 + len(ctl), len(code), len(ctl))
|
||||
return bytes(header + ctl + code), bytes(ctl), bytes(code)
|
||||
|
||||
|
||||
def container(*stages):
|
||||
# Reflection/variant metadata can precede and separate the NVN records.
|
||||
d = bytearray(b'HNSHV002' + bytes(24) + b'variant metadata')
|
||||
for stage in stages:
|
||||
d += shader(stage)[0] + bytes(36)
|
||||
struct.pack_into('>I', d, 8, len(d))
|
||||
return bytes(d)
|
||||
|
||||
|
||||
class NeedleTests(unittest.TestCase):
|
||||
def test_embedded_variants_and_headerless_compute(self):
|
||||
data = b'prefix\0' + container('vs', 'ps', 'cs') + b'suffix'
|
||||
found = []
|
||||
split.scan_blob(data, 'main', found)
|
||||
self.assertEqual([s for _, s, _, _ in found], ['vs', 'ps', 'cs'])
|
||||
for (_, stage, ctl, code) in found:
|
||||
self.assertEqual((ctl, code), shader(stage)[1:])
|
||||
keys = split.uniq_keys(found)
|
||||
# These stages are independent programs; the main/GSC wrapper is not a pair.
|
||||
self.assertEqual(split.var_pairs(found), set())
|
||||
self.assertEqual(len(split.separable_pairs(keys)[0]), 1)
|
||||
self.assertTrue(all('#NEEDLE@' in label for label, *_ in found))
|
||||
|
||||
def test_container_bounds_reject_truncated_or_string_magic(self):
|
||||
self.assertEqual(split.needle_shaders(b'HNSHV002', 'short'), [])
|
||||
self.assertEqual(split.needle_shaders(container('vs')[:-1], 'truncated'), [])
|
||||
data = bytearray(container('vs'))
|
||||
struct.pack_into('>I', data, 8, 32)
|
||||
self.assertEqual(split.needle_shaders(data, 'short envelope'), [])
|
||||
|
||||
def test_descriptor_cannot_borrow_code_from_next_record(self):
|
||||
data = bytearray(container('vs'))
|
||||
c = data.index(split.CTL_MAGIC)
|
||||
struct.pack_into('<I', data, c - 0x90 + 0x34, len(data))
|
||||
with self.assertRaisesRegex(SystemExit, 'outside its HNSHV002 container'):
|
||||
split.needle_shaders(data + container('ps'), 'bad code offset')
|
||||
|
||||
def test_control_length_must_match_descriptor(self):
|
||||
data = bytearray(container('vs'))
|
||||
c = data.index(split.CTL_MAGIC)
|
||||
struct.pack_into('<I', data, c - 0x90 + 0x3c, 0x880)
|
||||
with self.assertRaisesRegex(SystemExit, 'control length'):
|
||||
split.needle_shaders(data, 'bad control length')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -27,7 +27,7 @@ static uint32_t mip(uint32_t s, unsigned l) { s >>= l; return s ? s : 1; }
|
||||
// Geometry of this level: width-in-blocks byte count, block row count, slice count, and where this level and layer starts in storage; returns -1 for unsupported textures
|
||||
int nvn_deswizzle_geom(const nvn_texdesc *d, unsigned level, unsigned layer,
|
||||
uint32_t *row_bytes, uint32_t *rows, uint32_t *slices, uint64_t *base, unsigned *th, unsigned *td, uint32_t *bpb) {
|
||||
if (d->target == 5 || d->target == 6 || d->target == 10 || (d->flags & 0x10) || d->samples > 1) return -1; // MS / BUFFER / LINEAR
|
||||
if (d->target == 5 || d->target == 6 || d->target == 10 || (d->flags & 0x50) || d->samples > 1) return -1; // MS / BUFFER / LINEAR / LINEAR_RENDER_TARGET
|
||||
int bw, bh; *bpb = (uint32_t)nvn_fmt_bpb(d->format, &bw, &bh);
|
||||
if (!*bpb || level >= (d->levels ? d->levels : 1)) return -1;
|
||||
nvn_texlayout L; nvn_tex_layout(d, &L);
|
||||
|
||||
@@ -46,6 +46,28 @@ static void roundtrip(const char *tag, int target, int fmt, uint32_t w, uint32_t
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
printf("L linear render targets without the separate LINEAR flag\n");
|
||||
{
|
||||
// Persona 5 Royal asks for 16x16 RGBA8, flags 0x44, stride 128: 2 KB,
|
||||
// including row padding. Treating it as block-linear returned only 1 KB.
|
||||
nvn_texdesc d = {1, 0x25, 0x44, 0, 16, 16, 1, 1, 128, 0};
|
||||
nvn_texlayout L; nvn_tex_layout(&d, &L);
|
||||
CHECK(L.size == 2048 && L.layer_size == 2048 && L.align == 128,
|
||||
"L render target storage %llu alignment %u, expected 2048 / 128", (unsigned long long)L.size, L.align);
|
||||
CHECK(nvn_tex_level_gobs(&d, 0) == 0 && nvn_tex_level_offset(&d, 0) == 0,
|
||||
"L linear render target must not have block-linear GOB columns");
|
||||
uint8_t storage[2048] = {0}, pixels[1024] = {0};
|
||||
CHECK(nvn_deswizzle(&d, 0, 0, storage, pixels, 64) == -1,
|
||||
"L linear render target must not be deswizzled");
|
||||
d.stride = 0; nvn_tex_layout(&d, &L);
|
||||
CHECK(L.size == 2048 && L.align == 128, "L implicit render target stride must align to 128");
|
||||
d.flags |= 0x10; nvn_tex_layout(&d, &L);
|
||||
CHECK(L.size == 2048 && L.align == 128, "L both linear flags must keep render target alignment");
|
||||
d.flags = 0x10; nvn_tex_layout(&d, &L);
|
||||
CHECK(L.size == 1024 && L.align == 32, "L ordinary linear texture keeps 32-byte alignment");
|
||||
d.flags = 0x44; d.stride = 256; nvn_tex_layout(&d, &L);
|
||||
CHECK(L.size == 4096, "L explicit padded stride must be preserved");
|
||||
}
|
||||
printf("A round trip (random data, non-power-of-2 sizes)\n");
|
||||
roundtrip("2D 100×37 R8 4 levels", 1, 0x01, 100, 37, 1, 4);
|
||||
roundtrip("2D 100×37 RG8 4 levels", 1, 0x0D, 100, 37, 1, 4);
|
||||
|
||||
@@ -29,6 +29,7 @@
|
||||
# · PKZ (PlatinumGames, Bayonetta 3): shader objects (VSO / PSO / CSO / HSO / DSO, a control section and a code area each) in the DAT archives of the PKZ entries (Oodle Kraken, decoded by the
|
||||
# game's own decoder in the Mac host), see pkz_entries, pgdat_members and pg_shader
|
||||
# · CPK (CRI's file system, Bayonetta 2): BNSH files ('.vso' / '.pso') in the PlatinumGames DAT archives of the CPK entries (CRILAYLA entries, cpk.py), see cpk_jobs
|
||||
# · HNSHV002 (Needle, Persona 5 Royal): NVN section descriptors in the shader containers embedded in main and the GSC files of its CPK archives, see needle_shaders
|
||||
# · GFPAK (Game Freak, Pokémon Sword): BNSH entries of the archives under bin/archive (LZ4 entries), see gfpak_members
|
||||
# · .tfxo (Silicon Studio's YEBIS post effects, Pokémon Sword): control sections each followed by its code area, in a zlib stream in main, see tfxo_streams and tfxo_shaders
|
||||
# · control sections loose in main, in no GLSLC output or BNSH file (Advance Wars 1+2: Re-Boot Camp's Bink 2 shaders): paired with their code areas by data records or adjacency, see raw_shaders
|
||||
@@ -207,6 +208,35 @@ def glslc_shaders(d, label, need=True):
|
||||
if not out and need: red(f'{label}: no GLSLC output')
|
||||
return out
|
||||
|
||||
NEEDLE = b'HNSHV002'
|
||||
def needle_shaders(d, label):
|
||||
"""Needle shader containers, embedded in main and in GFD's GSC files inside CRI CPKs (Persona 5 Royal). HNSHV002 +8 is the container's whole length, big-endian. Variant and reflection
|
||||
metadata varies; each NVN payload has a 0x90-byte little-endian section descriptor immediately before its control section: +0x30 control offset (0x90), +0x34 code offset, +0x38 code
|
||||
length, +0x3c control length, all relative to the descriptor. Both regions must fit this container, not merely the enclosing file. The 0x850-byte controls are followed by graphics
|
||||
code or headerless compute code. Each payload is a separate program, including the 121 post-effect variants in one main container; vertex/pixel pairs exist only at draw time."""
|
||||
out = []
|
||||
for m in re.finditer(NEEDLE, d):
|
||||
h = m.start()
|
||||
if h + 12 > len(d): continue
|
||||
size = struct.unpack_from('>I', d, h + 8)[0]
|
||||
if size < 0x90 + 0x760 or size > len(d) - h: continue # a literal magic string in executable code is not a container
|
||||
end = h + size; c = d.find(CTL_MAGIC, h + 12, end)
|
||||
while c >= 0:
|
||||
s = c - 0x90
|
||||
if s < h + 12 or c + 0x760 > end: red(f'{label} NEEDLE @{h:#x}: incomplete NVN section at {c:#x}')
|
||||
co, ko, kn, cn = struct.unpack_from('<4I', d, s + 0x30)
|
||||
if co != 0x90 or cn < 0x760 or cn != struct.unpack_from('<I', d, c + 0x20)[0]:
|
||||
red(f'{label} NEEDLE @{h:#x}: invalid control offset or control length at {s:#x}')
|
||||
if ko < co + cn or not kn or max(co + cn, ko + kn) > end - s:
|
||||
red(f'{label} NEEDLE @{h:#x}: NVN section at {s:#x} is outside its HNSHV002 container')
|
||||
ctl, code = d[c:c + cn], d[s + ko:s + ko + kn]
|
||||
st = glslc_stage(ctl, code)
|
||||
why = check(label, st, ctl, code) if st else 'unknown shader stage'
|
||||
if why: red(f'{label} NEEDLE @{h:#x}: NVN section at {s:#x}: {why}')
|
||||
out.append((f'{label}#NEEDLE@{s:#x}', st, ctl, code))
|
||||
c = d.find(CTL_MAGIC, s + ko + kn, end)
|
||||
return out
|
||||
|
||||
IDRK, PDRK = b'IDRK0000', b'PDRK0000'
|
||||
def idrk_entries(d):
|
||||
"""Koei Tecmo RDB data files (Hyrule Warriors: Age of Calamity: asset/*.rdb.bin*, asset/data/*.file) are a run of IDRK entries: 'IDRK0000', u64 entry length, u64 stored length, u64 length,
|
||||
@@ -1469,6 +1499,7 @@ def scan_blob(d, label, out):
|
||||
size = struct.unpack_from('<I', d, m.start() + 0x1c)[0]
|
||||
if m.start() + size <= len(d) and d[m.start() + 0x60:m.start() + 0x64] == b'grsc': out.extend(bnsh_shaders(d, m.start(), label))
|
||||
if MGFX in d: out.extend(mgfx_shaders(d, label))
|
||||
if NEEDLE in d: out.extend(needle_shaders(d, label))
|
||||
|
||||
def group(label):
|
||||
"""source group (second half of the package file name)"""
|
||||
@@ -1481,7 +1512,7 @@ def ps_inputs(code):
|
||||
"""Generic attributes a pixel shader reads: SPH words 6–13 are the input map, 8 bits for each of the 32 attributes (Ryujinx ShaderHeader.ImapTypes), non-zero = read; the translated MSL inputs are exactly user(locn<attribute>)"""
|
||||
return {i for i in range(32) if struct.unpack_from('<I', code, 0x30 + 4 * (6 + (i >> 2)))[0] >> ((i & 3) * 8) & 0xff}
|
||||
|
||||
SINGLE_STAGE = ('#GLSLC@', '#UE', '#MTSPK@', '#MGFX@', '#NLG', '#PGSO', '#RAW@', '#TFXO@') # sources whose every shader is a program of one stage: GLSLC outputs, Unreal's shader records, MT Framework's shader package (BNSH variations, Unity programs, MSHD and SDF pair theirs);
|
||||
SINGLE_STAGE = ('#GLSLC@', '#UE', '#MTSPK@', '#MGFX@', '#NLG', '#PGSO', '#RAW@', '#TFXO@', '#NEEDLE@') # sources whose every shader is a program of one stage: GLSLC outputs, Unreal's shader records, MT Framework's shader package (BNSH variations, Unity programs, MSHD and SDF pair theirs);
|
||||
# MonoGame's effects, whose passes may name no vertex shader and keep the one bound (Celeste: 10 of its 13 effects draw through SpriteBatch's vertex shader, which is in main); Next Level Games'
|
||||
# archives (Luigi's Mansion 3 makes 5,317 programs of one stage each at boot); PlatinumGames' shader objects (Bayonetta 3: one stage each, pg_shader); control sections loose in main
|
||||
# (Advance Wars 1+2: Re-Boot Camp's Bink shaders, raw_shaders), which no file pairs; and YEBIS's effect file (tfxo_shaders)
|
||||
@@ -1596,7 +1627,7 @@ def scan_file(args):
|
||||
import unityfs
|
||||
try: unity_shaders(d, '/' + rel, found, trees if not TREES else dict(TREES))
|
||||
except unityfs.NoTree: return rel, {}, set(), 'defer'
|
||||
elif b'BNSH\0\0\0\0' not in d and b'BAHS' not in d and d[:4] != b'MSHD' and not (d[:4] == b'SARC' and b'Yaz0' in d) and b'\x91\x68\x86\x19' not in d and MGFX not in d: return rel, {}, set(), {} # a SARC may hold Yaz0 members
|
||||
elif b'BNSH\0\0\0\0' not in d and b'BAHS' not in d and d[:4] != b'MSHD' and not (d[:4] == b'SARC' and b'Yaz0' in d) and b'\x91\x68\x86\x19' not in d and MGFX not in d and NEEDLE not in d: return rel, {}, set(), {} # a SARC may hold Yaz0 members
|
||||
else:
|
||||
scan_blob(d, '/' + rel, found)
|
||||
# whole GLSLC outputs in a file no other reader takes (FINAL FANTASY TACTICS - The Ivalice Chronicles: its engine's technique files, resources_nx/…/shader/*.tec, 261 outputs beside
|
||||
|
||||
@@ -237,6 +237,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
|
||||
- frames stop, no crash, the main thread sits in `nvnSyncWait(sync, -1)`, and the watchdog's wait-state line shows the main fence one short (`main fence submitted N / completed N-1`). → [A GPU command buffer that never completes](../reference/nvn-objects.md#a-gpu-command-buffer-that-never-completes)
|
||||
- loading screens many times slower on the iPhone than on the Mac (Three Houses' first loading screen 78 s on the iPhone, 5 s on the Mac). → [`nvnQueueFinish` waits for the GPU, not the display](../reference/nvn-objects.md#nvnqueuefinish-waits-for-the-gpu-not-the-display)
|
||||
- menus and dialogue boxes draw but contain no text, while images and pictogram digits are fine; the sky is black (CPU-written linear colour ramps). → [Linear textures in CPU-visible pools](../reference/nvn-objects.md#linear-textures-in-cpu-visible-pools)
|
||||
- a trap during graphics initialization in `SetupSurfaceInfoMemoryInside`, asserting `ctx.memorySize <= localmemsetupinfo.memorySize`. → [Linear render-target storage without the LINEAR flag](../reference/nvn-objects.md#linear-render-target-storage-without-the-linear-flag)
|
||||
- video frames, or a game's built-in default textures, are all zero. → [Textures written with `nvnTextureWriteTexels`](../reference/nvn-objects.md#textures-written-with-nvntexturewritetexels)
|
||||
- after a few minutes of play the main thread spends whole frames in `WriteTexels` (Metroid Dread's frames stopped at about 175 s). → [`WriteTexels` regions written in place by block-linear address](../reference/nvn-objects.md#writetexels-regions-written-in-place-by-block-linear-address)
|
||||
- black screen with every geometry right (`DM_NVN_HILITE` shows each sprite and glyph in place); the uniform buffer right (a `CBDUMP`), so every texture read 0. → [A pool's storage at creation is its textures' content](../reference/nvn-objects.md#a-pools-storage-at-creation-is-its-textures-content)
|
||||
@@ -286,6 +287,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
|
||||
- a black world: everything rendered through a 3D LUT built by layered draws comes out black (every plane went to plane 0); `DM_NVN_PASSDUMP` shows no `L` line. → [Layered passes for vertex shaders that write the layer](../reference/rendering.md#layered-passes-for-vertex-shaders-that-write-the-layer)
|
||||
- a scene drawn only in a small square in the top-left corner of a black screen; speckled and blocky patches; a full-screen overlay showing one tile; the texture dump right after the scene pass shows the picture only in the corner, and `DM_NVN_HILITE` on the scene's shaders lights only the corner. → [Clears of attachments larger than the render pass](../reference/rendering.md#clears-of-attachments-larger-than-the-render-pass)
|
||||
- `texture clear case not implemented … mask 0x1`. → [A channel mask that covers every channel is a whole clear](../reference/rendering.md#a-channel-mask-that-covers-every-channel-is-a-whole-clear)
|
||||
- `texture clear case not implemented … mask 0x8` while entering a rendered scene. → [Texture clears with partial channel masks or regions](../reference/rendering.md#texture-clears-with-partial-channel-masks-or-regions)
|
||||
- red lines for unimplemented mid-pass, partial-mask and partial-scissor clears (the clears were dropped); `depth-stencil clear with a scissor or partial stencil mask, cleared whole` (logged once). → [Clears inside a pass, with a partial mask or a partial scissor](../reference/rendering.md#clears-inside-a-pass-with-a-partial-mask-or-a-partial-scissor)
|
||||
- solid magenta blocks, exactly (255, 0, 255), hard-edged and aligned to the 16-pixel grid (16×16, sometimes two stacked): Super Mario 3D World showed them only on the iPhone; Captain Toad on a Mac, Apple M6, in about half the frames of the opening course, moving with the falling leaves. → [Same-pass feedback read through a snapshot](../reference/rendering.md#same-pass-feedback-read-through-a-snapshot)
|
||||
- text a game renders into textures for the title and the menus comes out upside down, while the scene looks right. → [Lower-left window origin](../reference/rendering.md#lower-left-window-origin)
|
||||
@@ -339,6 +341,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
|
||||
- the split found no shader in the game (0 shaders). → [GLSLC outputs holding several stages](../reference/shaders.md#glslc-outputs-holding-several-stages)
|
||||
- 24 shaders before the control-length fix, 1,355 after. → [SPKG packages in KTGL LINKDATA](../reference/shaders.md#spkg-packages-in-ktgl-linkdata)
|
||||
- `[RED] ... PKZ version 0x10000: only 0x20000 is known` stops the shader split. → [PKZ version 1 archives](../reference/shaders.md#pkz-version-1-archives)
|
||||
- the split finds no shaders, although `main` contains NVN control sections; it reports that no data record or code-address pair identifies their code areas. → [Needle shader containers in main and GFD archives](../reference/shaders.md#needle-shader-containers-in-main-and-gfd-archives)
|
||||
- shaders the game draws are missing from the map (`DM_NVN_SHDUMP`), in no RomFS file and no module, stored or LZ4, and not compiled at run time. → [Shaders in a zlib stream in main (YEBIS .tfxo)](../reference/shaders.md#shaders-in-a-zlib-stream-in-main-yebis-tfxo)
|
||||
- two `[RED] shader … is not in the shader map` lines and `program has no vertex shader` as the opening movie starts; every movie drew white. → [Loose control sections and code areas in main (Bink 2)](../reference/shaders.md#loose-control-sections-and-code-areas-in-main-bink-2)
|
||||
- the split stopped with `no bundle has a Shader type tree of the same hash to borrow`. → [Unity Shader layouts written out for players without bundles](../reference/shaders.md#unity-shader-layouts-written-out-for-players-without-bundles)
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 221 KiB |
@@ -181,6 +181,13 @@ Games read this count as a position in their own command memory and size their h
|
||||
- **Check:** `DM_NVN_TEXCPU=1` lists non-packaged textures in CPU-visible pools.
|
||||
- **Found on:** [Super Mario Bros. Wonder](../../dm/games/010015100B514000/README.md): its glyph cache (the text texture) and its four 64×1 RGBA16F sky colour ramps.
|
||||
|
||||
### Linear render-target storage without the LINEAR flag
|
||||
|
||||
- **Symptom:** a trap during graphics initialization in `SetupSurfaceInfoMemoryInside`, asserting `ctx.memorySize <= localmemsetupinfo.memorySize`.
|
||||
- **Cause:** `LINEAR_RENDER_TARGET` specifies linear storage independently of `LINEAR`. The Metal backing path recognized either flag, but the storage-size and deswizzle helpers tested only `LINEAR`, underallocating some render targets and treating their rows as block-linear.
|
||||
- **Fix:** `nvn_tex.c` recognizes either flag for linear sizing and excludes both from block-linear level offsets, GOB columns and sparse layout. `nvn_deswizzle_geom` rejects both. `tex_linear` uses the same calculated row pitch, including implicit alignment, as `GetStorageSize`. Render-target rows align to 128 bytes; ordinary linear rows align to 32. Explicit strides are preserved. `t_tex` covers the layout and `t_linear_readback` checks a GPU clear against padded CPU-visible rows.
|
||||
- **Found on:** [Persona 5 Royal](../../dm/games/01005CA01580E000/README.md#rendering): a 16x16 RGBA8 target with flags `0x44` and stride 128 needs 2 KB; block-linear sizing returned 1 KB.
|
||||
|
||||
### Textures written with `nvnTextureWriteTexels`
|
||||
|
||||
- **Symptom:** video frames, or a game's built-in default textures, are all zero.
|
||||
|
||||
@@ -223,6 +223,13 @@ The replay on Metal (`dm/gfx/nvn/nvn_mtl.m`) and the Vulkan and OpenGL layers. E
|
||||
- **Fix:** a mask that covers every channel the format has is a whole clear (`mtl_pf_chmask`).
|
||||
- **Found on:** [FINAL FANTASY TACTICS - The Ivalice Chronicles](../../dm/games/010038B015560000/README.md).
|
||||
|
||||
### Texture clears with partial channel masks or regions
|
||||
|
||||
- **Symptom:** `texture clear case not implemented … mask 0x8` while entering a rendered scene.
|
||||
- **Cause:** `ClearTexture` only handled whole-level clears whose mask covered every channel. Persona 5 Royal clears only alpha on a 1920x1080 RGBA8 target; a load-action clear cannot preserve its RGB channels.
|
||||
- **Fix:** partial colour texture clears use the existing clear-triangle shaders in a load/store pass, with the requested channel mask and rectangle. The viewport comes from the selected mip, the region selects the slices or depth planes, and the current draw scissor does not affect the operation. Whole-level whole-channel clears keep the load-action path. `t_clear_tex` checks pixel and channel preservation, mip/layer selection, integer values, an empty mask and the whole-clear path under Metal validation.
|
||||
- **Found on:** [Persona 5 Royal](../../dm/games/01005CA01580E000/README.md), the opening casino.
|
||||
|
||||
### Scissored colour clears
|
||||
|
||||
- **Cause:** NVN `ClearColor` honours the scissor. Clears are normally folded into the next pass's load action, which only works for a full-target clear; Mario clears sub-rectangles of small targets to zero (pyramid levels packed into 480×272, 256×256 and 64×36 targets).
|
||||
|
||||
@@ -128,6 +128,13 @@ Finding, keying and translating shaders (`dm/gfx/tools/shader_split.py`, `dmsh_b
|
||||
- **Found on:** [Bayonetta 2](../../dm/games/01007960049A0000/README.md): 1,856 distinct keys (1,547 pixel, 309 vertex) in `shader.dat`, `shader2.dat` and `shadereff.dat` of `data000.cpk` (the split about 2.5 minutes); [Bayonetta](../../dm/games/010076F0049A2000/README.md): 458 distinct keys (236 pixel, 222 vertex), the members of `shader.dat` in `data00.cpk` (about 1.7 minutes). All build for both.
|
||||
- **Not checked:** 13 Sentinels and Catherine: Full Body's CPKs (no files on the drives).
|
||||
|
||||
### Needle shader containers in main and GFD archives
|
||||
|
||||
- **Symptom:** the split finds no shaders, although `main` contains NVN control sections; it reports that no data record or code-address pair identifies their code areas.
|
||||
- **Cause:** Needle keeps independent shader programs in `HNSHV002` containers, embedded in `main` and in GFD's GSC files inside CRI CPK archives. These are neither GLSLC outputs nor BNSH files. A container can hold many variants separated by reflection metadata.
|
||||
- **Fix:** `shader_split.py` `needle_shaders` uses the container's big-endian whole length at +8, then the little-endian NVN section descriptors: 0x90 bytes immediately before each control section, with control offset, code offset, code length and control length at +0x30. Both regions must fit within that container, and the control/code headers and lengths must agree. Graphics stages come from SPH; headerless compute stages from the control's thread-group dimensions. Each program is paired with the single-stage cover, since its draw-time partner is not recorded by this wrapper. Synthetic fixtures check embedded variants, compute, length mismatches and container boundaries.
|
||||
- **Found on:** [Persona 5 Royal](../../dm/games/01005CA01580E000/README.md#shaders), with 0x850-byte control sections. The game note records counts and runtime coverage.
|
||||
|
||||
### Shaders in a zlib stream in main (YEBIS .tfxo)
|
||||
|
||||
- **Symptom:** shaders the game draws are missing from the map (`DM_NVN_SHDUMP`), in no RomFS file and no module, stored or LZ4, and not compiled at run time.
|
||||
|
||||
Reference in new issue
Block a user