Pack shader pairs and the offline pipeline archive into Mac apps (#241, #260)

The Mac app packer left out shader_pairs.tsv and pso_pairs.tsv, which the
host's pipeline prebuild reads beside the shader map and build_ipa.sh already
copies into the IPA. Every NVN game's Mac app therefore built no pulling
bases, had no hybrid stand-ins, and compiled first-seen pipelines on the
submit thread. Copy both tables, and the shader split's pso_pairs.tsv for a
profile without one.

An NVN game's Mac app also carries its pipeline archive made offline for this
Mac's GPU (pso_offline.py --platform macos), handed to the host as
DM_PSO_BUNDLE as on iOS; --gpu none leaves it out. Integrated with three
fixes: the archive is validated and counted in the free-space check before
anything is copied; a plan whose pairs all need function constants packs the
app without an archive instead of aborting; and a PSO_GPUS value that does
not name this Mac's GPU is reported.

Validation: test_nsp_mac_app 22 (pair tables, split copy, bad archive, free
space, GPU note, no-archive plan; the table and space tests fail without the
fixes), the tool suite, iOS packaging tests and test_docs. A Breath of the
Wild Mac app on an M4 Pro (applegpu_g16s, 629.4 MB archive): 16 of 16 probed
bases found, no pair base built, no red line; the run stayed in the title.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
Co-authored-by: blindlight86 <25404088+blindlight86@users.noreply.github.com>

dm/games/01007EF00011E000/README.md

@@ -145,6 +145,8 @@ Every kind of cold start also shows the game's own 130–220 ms frames every 9
 
 - **The pipeline archive made offline** ([development.md](../../../docs/development.md#the-pipeline-archive-made-offline)): every one of the 15,767 pairs makes a base (none left to the device, none left out); `metal-tt` built the iPhone 17 Pro Max's archive (`applegpu_g18p`, iOS 27.0) in 252 s on an M2 Max: 653 MB, 219 MB zipped. On that iPhone (iOS 27.0.1) a test app found all 200 bases of a 200-pair archive, at 0.06–0.3 ms each. On the Mac the same build for the M2 Max (`applegpu_g14s`, 615 MB, 253 s) held all 15,767 bases: the prebuild tool with it opened it in 1.8 s from the external drive, found 16 of 16 probed bases in 57 ms and built nothing, where the device's own prebuild took 187 + 19 s on 7 workers. The process's footprint rose by about 615 MB with the archive open. Resolved on the iPhone 17 Pro Max (iOS 27.0.1): the app with the archive (622.8 MB) opened it in 555 ms, found 16 of 16 probed bases in 17 ms, and its prepare step took 0.53 s with no base compiled (1,722 bases from the archives), where the device's own prebuild took 7–8 minutes on a first start; footprint 1,424 MiB at frame 300 and 2,579–2,679 MiB in play to frame 2,100, with the device's own earlier prebuild (932.7 MB in 30 files of `pso_archive/`, from before the archive was shipped) still open beside it.
 
+- **A Mac app** (`--platform mac-app`, M2 Max). Before the shader pairs went into the app: `pipeline prebuild: no …/Resources/dm/shader_pairs.tsv, nothing to build`, no hybrid draws, and first-seen pipelines compiled on the submit thread in frames of 100–546 ms ([Shader-pair tables in a Mac app](../../../docs/reference/shaders.md#shader-pair-tables-in-a-mac-app)). With the pairs and the archive for `applegpu_g14s` in the app (586.2 MB): 16 of 16 probed bases found in 20.6 ms, prepare step 0.23 s, no base compiled, no red line. The first launch of the freshly signed app waited about 100 s while macOS checked its signature; later launches start in a few seconds. The run stayed in the title and menus, so no hybrid draw occurred in it ([PR #241](https://github.com/seedds/nxnative/pull/241)). Repeated on an M4 Pro (`applegpu_g16s`) with an empty Application Support folder: `metal-tt` built the archive in 6:12 (629.4 MB, an APFS clone in the app; ad-hoc signing of the 16.9 GB app took 4:40). The first launch waited about 4 minutes for the signature check before the host started. The archive opened in 250.5 ms from the external drive and covered every pair (16 of 16 probed bases in 34.5 ms), so no pair base was built. The prepare step still compiled 871 bases in 2.80 s on 14 workers: the prewarm list's bases with a vertex layout, which the archive never holds, and which went into the app's own `pso_archive/` (19.9 MB). No red line in the 90 s after.
+
 On this Mac the background prewarm also ends before the save loads, so the step changes the first minute only where the prewarm is slower than the way into play: a slower device, fewer cores, a longer list, a player who skips through the title. Frames over 25 ms without pipeline work (20–71 per run) are the game's own and vary with the machine's other load.
 
 ## Mac routes
@@ -202,6 +204,7 @@ General host changes first needed here; each applies to every game:
 - **SDK 20 service calls** and **record sizes against `GetCommandMemoryUsed`** (`draw_rec`): [SDK 20 six-axis sensor and amiibo (`nfp`) calls](../../../docs/reference/services.md#sdk-20-six-axis-sensor-and-amiibo-nfp-calls), [Applet resource user ID, launch parameters and VR mode](../../../docs/reference/services.md#applet-resource-user-id-launch-parameters-and-vr-mode), [Shared font load request](../../../docs/reference/services.md#shared-font-load-request), [Model heaps sized by the host's command record sizes](../../../docs/reference/memory.md#model-heaps-sized-by-the-hosts-command-record-sizes). **SDK 20's audio renderer layout** (`render_once`): [SDK 20's `AudioRenderSystem` layout](../../../docs/reference/audio.md#sdk-20s-audiorendersystem-layout). **Running a 30 fps game at 60**: [sead's dynamic present interval, `ksys::VFR` and the host's frame step](../../../docs/reference/frame-rate.md#seads-dynamic-present-interval-ksysvfr-and-the-hosts-frame-step).
 - **Pipeline bases, hybrid pipelines, the prebuild and the prepare step** (`pso_build_ex`, `hybrid_get`, `pso_prebuild`, `warm_prepare`): [development.md, Hybrid pipelines](../../../docs/development.md#hybrid-pipelines), [Pulling vertex inputs](../../../docs/reference/rendering.md#pulling-vertex-inputs), [the prepare step](../../../docs/development.md#the-prepare-step); measured under [Pipelines on a cold start](#pipelines-on-a-cold-start).
 - **The pipeline archive made offline** (`dm/gfx/tools/pso_offline.py`, `pso_bundle_open`, `pso_bundle_covers`): [development.md, The pipeline archive made offline](../../../docs/development.md#the-pipeline-archive-made-offline); measured under [Pipelines on a cold start](#pipelines-on-a-cold-start).
+- **Resolved: a Mac app without its shader-pair tables**: [Shader-pair tables in a Mac app](../../../docs/reference/shaders.md#shader-pair-tables-in-a-mac-app); measured under [Pipelines on a cold start](#pipelines-on-a-cold-start).
 - **Resolved in synthetic archive review checks:** truncated archive containers, zero-probe coverage, stale archives after library bytes change, and plans whose pairs all require device specialization: [Offline pipeline archive reuse and fallback](../../../docs/reference/shaders.md#offline-pipeline-archive-reuse-and-fallback). Checked with generated libraries on the Mac; no new gameplay or iPhone validation.
 - **GPU measurement on the iPhone** (`DM_NVN_GPUPROF=every:n`, `DM_AB`, `DM_GPU_CAPTURE`, `dm/tools/gpuprof.py`) and **captures with pipelines built through the device** (`pso4_on`): [05, GPU time on the device](../../../docs/agents/05-debugging.md#gpu-time-on-the-device), [GPU frame captures need pipelines built through the device](../../../docs/reference/shaders.md#gpu-frame-captures-need-pipelines-built-through-the-device).
 - **Resolved in synthetic profiling review checks:** [Periodic GPU profiles close at frame boundaries](../../../docs/reference/rendering.md#periodic-gpu-profiles-close-at-frame-boundaries). No new gameplay or iPhone validation.

dm/games/01008CF01BAAC000/README.md

@@ -19,6 +19,8 @@ No game-specific rewriter cuts, SDK cuts or host branches are needed for the tes
 
 ## Rendering and shaders
 
+- **Resolved: the Mac app had no shader-pair tables** ([Shader-pair tables in a Mac app](../../../docs/reference/shaders.md#shader-pair-tables-in-a-mac-app)), so the host skipped the pipeline prebuild although all 9,858 shader libraries were built. The first app had a 308.4 ms frame, 283.5 ms of it pipeline acquisition and 5.9 ms GPU work. With the tables an offline prebuild built all 7,495 bases with no failure. A player first reported audio over half a second ahead of the picture and drops to about 50 fps; after repacking, completing the prebuild and restarting the same opening, audio in sync or only slightly off and no stutter seen. This is playtest feedback, not a controlled cold-cache A/B (the learned pipeline list and the restart changed too), so neither a general synchronization fix nor hitch-free later play is established. Found by [PR #260](https://github.com/seedds/nxnative/pull/260).
+- **Open: short dips below 50 fps on entering the castle or changing scenes** (a later playtest, not sustained slow movement). A nearby foreground frame took 212.0 ms (189.7 ms submit/present, 7.1 ms GPU) with six new pipelines, 46 new shader libraries, 134.3 ms of pipeline and library acquisition and 34.0 MB of first texture uploads (6.7 ms); 145 hybrid draws, so the prebuilt stand-ins were in use. This does not isolate compiler CPU time, and other long transition frames had little or no pipeline work. The pipeline archive bundled with the app (since [PR #241](https://github.com/seedds/nxnative/pull/241)) has not been measured on this game. Background-window timing is excluded.
 - **Current:** lower-left window origin and near-zero depth, both selected from the game's NVN calls. The existing host handling used by [Link's Awakening](../01006BB00C6F0000/README.md#rendering) applies.
 - **Resolved: a false red cube/2D warning at the first raised-door transition.** The particle fragment shader `aca252efc381a89d` (`region_common/effect/Common.ptcl`, BNSH at `0x59bc000`, variation 3868) chooses reflection cube slot 14 only when the first integer in uniform slot 13 is positive; otherwise it samples cube slot 13. The captured failing draw has that integer equal to zero and a valid 64×64 cube in slot 13. Slot 14 retains a 1408×792 2D depth-stencil texture (ID 272), so it is unused by this draw. The NVN descriptor itself is 2D: no cube view was lost. The existing zero-cube binding is valid; reporting the binding as a rendering error was wrong. Its [general handling](../../../docs/reference/rendering.md#a-2d-texture-in-a-cube-slot-binds-the-zero-cube) now reports an informational fallback. The particle shader and rendered output require no workaround.
 - **Warning validation:** the isolated replay first reproduced the red line in 52.6 s; tracing captured the conditional input and selector. After the fix, the same fallback and the following room render without red lines (63.3 s through frame 3600). The GPU regression has 42 checks under Metal API validation: all pass, versus six false diagnostic failures before the fix. Reproduction scripts, isolated saves, logs and the room screenshot remain under the game's work folder in `cube-repro/`. The unsigned IPA was rebuilt with the fix.
@@ -30,7 +32,7 @@ No game-specific rewriter cuts, SDK cuts or host branches are needed for the tes
 
 - **Current:** the SDK audio renderer runs at 48 kHz, 240 samples per frame, 96 voices and a six-input DeviceSink. The Mac callback reports non-silent output and no underruns in the tested route.
 - The first dungeon corridor settles around 1.7–1.8 GB resident with 1.3 GB allocated by Metal. Level transitions change the footprint; the seven-minute repeat ends at 1,831 MB resident / 1,261 MB Metal, with a stable 490-pool count and no continuing growth in the stationary corridor windows.
-- The corridor runs at the game's 60 fps when the Mac window is presenting. First-use pipeline compilation causes hitches. Window-inactive timing is not a gameplay frame-rate measurement.
+- The corridor runs at the game's 60 fps when the Mac window is presenting. First-use pipeline compilation causes hitches (see the shader-pair tables under [Rendering and shaders](#rendering-and-shaders)). Window-inactive timing is not a gameplay frame-rate measurement.
 - **Validation:** four-minute and seven-minute repeats through the opening corridor, followed by an 85-second run of the finalized profile with prewarming enabled. The final run has no red line. Timed runs end deliberately with SIGALRM (`CRASH sig=14`), rather than a game crash. The unsigned IPA's ZIP CRC check passed; it contains all 9,858 base shader libraries and no signing entries.
 
 ## Mac routes
@@ -48,4 +50,5 @@ The development build is `01008CF01BAAC000_build/` beside the NSPs, with runs an
 
 ## Found on this game
 
+- **Mac app pair tables:** [Shader-pair tables in a Mac app](../../../docs/reference/shaders.md#shader-pair-tables-in-a-mac-app).
 - **Unused conditional cube inputs are not rendering errors:** [A 2D texture in a cube slot binds the zero cube](../../../docs/reference/rendering.md#a-2d-texture-in-a-cube-slot-binds-the-zero-cube).

dm/mac/launcher.c

@@ -1,8 +1,8 @@
 // Mac app launcher (CFBundleExecutable of the .app that dm/mac/pack_app.py packs): sets the host's environment from the bundle, then runs the Mac host (Contents/MacOS/dmhost) as a child.
 // The host is not exec'ed: macOS privacy checks (TCC) attribute a child's file access to the app that launched it, while a launcher that hands its process over (a shell script exec'ing
 // dmhost) gets EPERM on every open on a removable volume, with no prompt, signed or not. The launcher forwards SIGTERM / SIGINT / SIGHUP to the host and returns its exit code.
-// Bundle layout: Resources/romfs (the game's RomFS), Resources/msl/<b-chain | vk | gl> (the Mac shader libraries), Resources/aoc/<index> (DLC), Resources/dm (the rewritten modules, the
+// Bundle layout: Resources/romfs (the game's RomFS), Resources/msl/<b-chain | vk | gl> (the Mac shader libraries), Resources/aoc/<index> (DLC), Resources/pso (the pipeline archive made
-// shader tables, font/, env.txt). Saves, the pipeline list and temporary files go to ~/Library/Application Support/nxnative/<title ID>/, the host's output to ~/Library/Logs/nxnative/<title ID>.log
+// offline, handed to the host as DM_PSO_BUNDLE as the iOS app's main.m does), Resources/dm (the rewritten modules, the shader tables and pairs, font/, env.txt). Saves, the pipeline list and temporary files go to ~/Library/Application Support/nxnative/<title ID>/, the host's output to ~/Library/Logs/nxnative/<title ID>.log
 // (the previous run's kept as <title ID>.prev.log). Environment switches: one KEY=VALUE per line in <Application Support folder>/env.txt, then Resources/dm/env.txt (the first one wins; neither
 // replaces a variable already set), as the iOS app reads Documents/env.txt and dm/env.txt.
 #include <CoreFoundation/CoreFoundation.h>
@@ -105,6 +105,7 @@ int main(void) {
     snprintf(p, sizeof p, "%s/font", dm); setenv("DM_FONT_DIR", p, 0);
     snprintf(p, sizeof p, "%s/shader_map.tsv", dm); if (!stat(p, &st)) setenv("DM_NVN_SHMAP", p, 0);
     snprintf(p, sizeof p, "%s/aoc", res); if (!stat(p, &st)) setenv("DM_AOC_DIR", p, 0);
+    snprintf(p, sizeof p, "%s/pso", res); if (!stat(p, &st)) setenv("DM_PSO_BUNDLE", p, 0);   // the bases of every shader pair: the first start builds none
     if (*lang) setenv("DM_LANG", lang, 0);
     setenv("DM_WINDOW", "1", 0); setenv("DM_AUDIO", "1", 0);
     if (chdir(sup)) { fprintf(stderr, "[RED] launcher: cannot enter %s\n", sup); return 1; }

dm/mac/pack_app.py

@@ -1,7 +1,8 @@
 #!/usr/bin/env python3
 # Pack a self-contained, double-clickable Mac app of a game (the NSP script's --platform mac-app): the launcher (dm/mac/launcher.c, compiled here) as CFBundleExecutable, the Mac host beside it,
 # and in Contents/Resources what the IPA carries (dm/ios/tools/build_ipa.sh): romfs/ (the RomFS), msl/<b-chain | vk | gl>/ (the Mac shader libraries, the folder named as in the IPA),
-# aoc/<index>/ (DLC), and dm/ (the rewritten modules and their rewriter tables, nro.tsv, the profile's shader_map.tsv / pso_manifest.tsv / c1_full.tsv / cb_extent.tsv, font/, env.txt),
+# aoc/<index>/ (DLC), pso/ (an NVN game's pipeline archive made offline for this Mac's GPU, dm/gfx/tools/pso_offline.py: the launcher hands it to the host as DM_PSO_BUNDLE),
+# and dm/ (the rewritten modules and their rewriter tables, nro.tsv, the profile's shader_map.tsv / pso_manifest.tsv / c1_full.tsv / cb_extent.tsv / shader_pairs.tsv / pso_pairs.tsv, font/, env.txt),
 # plus AppIcon.icns from the game's icon (sips, iconutil) and the Info.plist keys the launcher reads (DMTitleID, DMRomfsMarker, DMLanguage).
 # The large folders are APFS clones (cp -c) when the app is on the same APFS volume as the build, so they take no space; plain copies otherwise.
 # Signing: the inner host first, then the bundle (codesign --force --sign <identity>; '-' = ad-hoc), without the hardened runtime: the host maps the game's code executable, which the hardened
@@ -14,7 +15,10 @@ from pathlib import Path
 HERE = Path(__file__).resolve().parent
 MODULE_FILES = ('*.patched.bin', '*.fixups.tsv', '*.gates.tsv', '*.cuts.tsv', '*.obs.tsv', 'nro.tsv')   # what the host reads from the module folder (build_ipa.sh copies the same tables)
 MSL_FILES = ('*.metallib', '*.json', '*.tsv')   # what the host reads from msl/ (not the Mac build's .metal sources or _b_build.json)
-TABLES = ('shader_map.tsv', 'pso_manifest.tsv', 'c1_full.tsv', 'cb_extent.tsv')   # read beside DM_NVN_SHMAP; any of them may be absent (a Vulkan or GL game has no c1_full.tsv)
+# read beside DM_NVN_SHMAP; any of them may be absent (a Vulkan or GL game has no c1_full.tsv). The shader pairs are the pipeline prebuild's (nvn_mtl.m pso_prebuild): without them the
+# app makes no pulling bases, a hybrid draw has no stand-in and a new pipeline compiles on the submit thread (build_ipa.sh copies them into the IPA the same way)
+TABLES = ('shader_map.tsv', 'pso_manifest.tsv', 'c1_full.tsv', 'cb_extent.tsv', 'shader_pairs.tsv', 'pso_pairs.tsv')
+PSO_FILES = ('*.mtl4a', 'prebuilt.tsv', 'rest_pairs.tsv', 'archive.tsv')   # what the app takes of pso_offline.py's output folder
 EXE = 'nxnative'   # the launcher's file name (Contents/MacOS/nxnative)
 ICON_SIZES = (16, 32, 128, 256, 512)
 
@@ -113,9 +117,11 @@ def pack(a):
     if a.marker and not (romfs / a.marker).exists(): raise SystemExit(f'no RomFS at {romfs} (missing {a.marker})')
     msls = sorted(p for pat in MSL_FILES for p in msl.glob(pat) if p.is_file() and p.name != '_b_build.json') if msl.is_dir() else []
     if not any(p.suffix == '.metallib' for p in msls): raise SystemExit(f'no shader libraries at {msl}')
-    clone = can_clone(romfs, res)
+    pso = sorted({p for pat in PSO_FILES for p in Path(a.pso).glob(pat)}) if a.pso else []
-    if not clone:
+    if a.pso and (not any(p.suffix == '.mtl4a' for p in pso) or not (Path(a.pso) / 'prebuilt.tsv').is_file()): raise SystemExit(f'no pipeline archive with its prebuilt.tsv in {a.pso}')
-        need = sum(tree_bytes(d) for d in [romfs] + ([aoc] if aoc else [])) + sum(p.stat().st_size for p in msls)
+    clone = can_clone(romfs, res); pso_clone = bool(pso) and can_clone(Path(a.pso), res)
+    if not clone or (pso and not pso_clone):
+        need = (0 if clone else sum(tree_bytes(d) for d in [romfs] + ([aoc] if aoc else [])) + sum(p.stat().st_size for p in msls)) + (0 if pso_clone else sum(p.stat().st_size for p in pso))
         free = shutil.disk_usage(res).free
         if need + (1 << 30) > free: raise SystemExit(f'the app needs {need / 2**30:.1f} GB on the volume of {out.parent} (no APFS clones from the build there) and {free / 2**30:.1f} GB are free')
     step(f'launcher and host ({"clones" if clone else "copies"} of the game files)')
@@ -130,11 +136,13 @@ def pack(a):
     copy(mods, dm, clone)
     tables = [Path(a.tables) / n for n in TABLES if (Path(a.tables) / n).is_file()]
     if tables: copy(tables, dm, False)
+    if a.pso_pairs and not (dm / 'pso_pairs.tsv').is_file(): shutil.copyfile(a.pso_pairs, dm / 'pso_pairs.tsv')   # the split's copy, for a profile without the table
     fonts = sorted(Path(a.font).glob('*.ttf'))
     if fonts: copy(fonts, dm / 'font', False)
     if a.env_file: shutil.copyfile(a.env_file, dm / 'env.txt')
     step('shader libraries'); msl_dst = res / 'msl' / (a.msl_name or msl.name); msl_dst.mkdir(parents=True); copy(msls, msl_dst, clone)
     if aoc: step('add-on content'); copy([aoc], res / 'aoc', clone)
+    if pso: step('pipeline archive'); (res / 'pso').mkdir(); copy(pso, res / 'pso', pso_clone)
     step('game files'); copy([romfs], res / 'romfs', clone)
     icon = bool(a.icon and Path(a.icon).is_file())
     if icon: step('icon'); make_icns(a.icon, res / 'AppIcon.icns')
@@ -162,6 +170,8 @@ def parse(argv):
     ap.add_argument('--tables', required=True, help='the game profile folder (shader_map.tsv and the tables beside it)')
     ap.add_argument('--font', required=True, help='the folder of the shared-font stand-ins (nintendo_ext.ttf)')
     ap.add_argument('--aoc', help='the DLC RomFS folders by index')
+    ap.add_argument('--pso', help='the pipeline archive made offline for this Mac (pso_offline.py --platform macos output: <arch>.mtl4a, prebuilt.tsv)')
+    ap.add_argument('--pso-pairs', help='pso_pairs.tsv for a profile that has none (the shader split\'s copy)')
     ap.add_argument('--env-file', help='environment switches baked in (Resources/dm/env.txt)')
     ap.add_argument('--title-id', required=True); ap.add_argument('--name', required=True)
     ap.add_argument('--version'); ap.add_argument('--bundle-id'); ap.add_argument('--category'); ap.add_argument('--lang'); ap.add_argument('--marker', help='the profile\'s romfs_marker')

dm/tools/tests/test_nsp_mac_app.py

@@ -99,8 +99,10 @@ class App(unittest.TestCase):
         (t / 'cache/b-0123456789ab/k_b.json').write_text('{}'); (t / 'cache/b-0123456789ab/k_b.metal').write_text('msl'); (t / 'cache/b-0123456789ab/_b_build.json').write_text('[]')
         (t / 'cache/cas').mkdir()   # the Metal compile cache: not for the app
         for n in ('shader_map.tsv', 'pso_manifest.tsv', 'c1_full.tsv', 'game.tsv'): (t / 'profile' / n).write_text(n)
+        (t / 'profile/shader_pairs.tsv').write_text('vertex-key\tpixel-key\n'); (t / 'profile/pso_pairs.tsv').write_text('other-vertex\tother-pixel\n')
         (t / 'font/nintendo_ext.ttf').write_bytes(b'ttf')
         (t / 'aoc/1/x').write_bytes(b'dlc')
+        (t / 'pso').mkdir(); (t / 'pso/applegpu_g14s.mtl4a').write_bytes(b'arch'); (t / 'pso/prebuilt.tsv').write_text('chain\t0123456789ab\n'); (t / 'pso/key.x').write_text('not for the app')
         (t / 'env.txt').write_text('# baked in\nDM_BAKED=1\nDM_BOTH=bundle\n')
         subprocess.run(['sips', '-s', 'format', 'jpeg', '-z', '256', '256', '/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericApplicationIcon.icns',
                         '--out', t / 'icon.jpg'], check=True, capture_output=True)
@@ -109,7 +111,7 @@ class App(unittest.TestCase):
         (t / 'bin/osascript').chmod(0o755)
         cls.app = t / 'Game - Test.app'
         cls.args = pack_app.parse([str(cls.app), '--host', str(t / 'dmhost'), '--patched', str(t / 'patched'), '--romfs', str(t / 'romfs'), '--msl', str(t / 'cache/b-0123456789ab'),
-                                   '--tables', str(t / 'profile'), '--font', str(t / 'font'), '--aoc', str(t / 'aoc'), '--env-file', str(t / 'env.txt'), '--title-id', TID.lower(),
+                                   '--tables', str(t / 'profile'), '--font', str(t / 'font'), '--aoc', str(t / 'aoc'), '--pso', str(t / 'pso'), '--env-file', str(t / 'env.txt'), '--title-id', TID.lower(),
                                    '--name', 'Game: Test', '--version', '1.0.1', '--lang', 'fr', '--marker', 'region_common/audio/edward.bfsar', '--icon', str(t / 'icon.jpg'),
                                    '--category', 'public.app-category.games'])
         with contextlib.redirect_stdout(io.StringIO()): pack_app.pack(cls.args)
@@ -134,14 +136,30 @@ class App(unittest.TestCase):
         self.assertFalse(self.app.with_name(self.app.name + '.partial').exists())
         self.assertEqual(sorted(p.name for p in (c / 'MacOS').iterdir()), ['dmhost', 'nxnative'])
         self.assertEqual(sorted(p.name for p in (c / 'Resources/dm').iterdir()),
-                         ['c1_full.tsv', 'env.txt', 'font', 'main.cuts.tsv', 'main.fixups.tsv', 'main.gates.tsv', 'main.obs.tsv', 'main.patched.bin', 'pso_manifest.tsv', 'shader_map.tsv'])
+                         ['c1_full.tsv', 'env.txt', 'font', 'main.cuts.tsv', 'main.fixups.tsv', 'main.gates.tsv', 'main.obs.tsv', 'main.patched.bin', 'pso_manifest.tsv', 'pso_pairs.tsv',
+                          'shader_map.tsv', 'shader_pairs.tsv'])   # the shader pairs: the pipeline prebuild's, without which hybrid draws have no stand-in
         self.assertEqual([p.name for p in (c / 'Resources/msl').iterdir()], ['b-0123456789ab'])   # the libraries only, not the compile cache
         self.assertEqual(sorted(p.name for p in (c / 'Resources/msl/b-0123456789ab').iterdir()), ['inputs.tsv', 'k_b.json', 'k_b.metallib'])
         self.assertEqual((c / 'Resources/msl/b-0123456789ab/k_b.metallib').read_bytes(), b'lib')
         self.assertTrue((c / 'Resources/romfs/region_common/audio/edward.bfsar').is_file())
         self.assertTrue((c / 'Resources/aoc/1/x').is_file()); self.assertTrue((c / 'Resources/dm/font/nintendo_ext.ttf').is_file())
+        self.assertEqual(sorted(p.name for p in (c / 'Resources/pso').iterdir()), ['applegpu_g14s.mtl4a', 'prebuilt.tsv'])   # the pipeline archive and its stamp only
         self.assertEqual((c / 'Resources/AppIcon.icns').read_bytes()[:4], b'icns')
 
+    def test_pipeline_pair_tables(self):   # the pipeline prebuild reads them beside DM_NVN_SHMAP: without them it builds nothing, though every shader library is there
+        dm = self.app / 'Contents/Resources/dm'
+        self.assertEqual((dm / 'shader_pairs.tsv').read_text(), 'vertex-key\tpixel-key\n')
+        self.assertEqual((dm / 'pso_pairs.tsv').read_text(), 'other-vertex\tother-pixel\n')
+
+    def test_split_pso_pairs(self):   # a profile without pso_pairs.tsv: the shader split's copy (--pso-pairs); the profile's own wins when it has one
+        with tempfile.TemporaryDirectory() as d, mock.patch.object(pack_app, 'sign', lambda *a: None), mock.patch('sys.stdout'):
+            d = Path(d); (d / 'profile').mkdir(); (d / 'profile/shader_map.tsv').write_text('m'); (d / 'split.tsv').write_text('split-vertex\tsplit-pixel\n')
+            a = types.SimpleNamespace(**vars(self.args)); a.out = str(self.tmp / 'Split.app'); a.icon = None; a.tables = str(d / 'profile'); a.pso_pairs = str(d / 'split.tsv')
+            pack_app.pack(a)
+            self.assertEqual((self.tmp / 'Split.app/Contents/Resources/dm/pso_pairs.tsv').read_text(), 'split-vertex\tsplit-pixel\n')
+            a.out = str(self.tmp / 'Own.app'); a.tables = self.args.tables; pack_app.pack(a)
+            self.assertEqual((self.tmp / 'Own.app/Contents/Resources/dm/pso_pairs.tsv').read_text(), 'other-vertex\tother-pixel\n')
+
     def test_info_plist(self):
         p = plistlib.loads((self.app / 'Contents/Info.plist').read_bytes())
         self.assertEqual((p['CFBundleExecutable'], p['CFBundleName'], p['CFBundleIdentifier'], p['CFBundleShortVersionString']), ('nxnative', 'Game: Test', 'org.nxnative.01008cf01baac000', '1.0.1'))
@@ -162,6 +180,7 @@ class App(unittest.TestCase):
         self.assertEqual(r['argv0'], str(self.app / 'Contents/MacOS/dmhost')); self.assertEqual(r['argv1'], str(res / 'dm'))
         self.assertEqual((r['DM_ROMFS'], r['DM_MSL_DIR'], r['DM_NVN_SHMAP'], r['DM_FONT_DIR'], r['DM_AOC_DIR']),
                          (str(res / 'romfs'), str(res / 'msl'), str(res / 'dm/shader_map.tsv'), str(res / 'dm/font'), str(res / 'aoc')))
+        self.assertEqual(r['DM_PSO_BUNDLE'], str(res / 'pso'))   # the pipeline archive made offline, as the iOS app's main.m sets it
         self.assertEqual((r['DM_SAVE'], r['DM_TEMP'], r['DM_ROOT'], r['cwd']), (str(sup / 'save'), str(sup / 'temp'), str(sup), str(sup)))   # pso_user.tsv goes beside save/
         self.assertEqual((r['DM_WINDOW'], r['DM_AUDIO'], r['DM_LANG']), ('1', '1', 'ja'))   # the caller's DM_LANG wins over the packed language
         self.assertEqual((r['DM_BAKED'], r['DM_BOTH']), ('1', 'player'))   # the player's env.txt before the bundle's
@@ -244,6 +263,23 @@ class App(unittest.TestCase):
                 pack_app.pack(a)
             self.assertFalse(Path(a.out).exists())
 
+    def test_pack_bad_archive(self):   # a pipeline archive folder without its archive or stamp is refused before anything is copied
+        with tempfile.TemporaryDirectory() as src:
+            src = Path(src); (src / 'prebuilt.tsv').write_text('chain\t0123456789ab\n')
+            a = types.SimpleNamespace(**vars(self.args)); a.pso = str(src); a.out = str(self.tmp / 'NoArchive.app')
+            with contextlib.redirect_stdout(io.StringIO()) as o, self.assertRaisesRegex(SystemExit, 'no pipeline archive with its prebuilt.tsv'):
+                pack_app.pack(a)
+            self.assertEqual(o.getvalue(), ''); self.assertFalse(Path(a.out).exists())
+
+    def test_copies_count_archive(self):   # another volume: the archive's bytes count in the free-space check, which runs before anything is copied
+        a = types.SimpleNamespace(**vars(self.args)); a.out = str(self.tmp / 'Full.app')
+        msls = sum(p.stat().st_size for pat in pack_app.MSL_FILES for p in Path(a.msl).glob(pat) if p.is_file() and p.name != '_b_build.json')
+        pso = sum(p.stat().st_size for pat in pack_app.PSO_FILES for p in Path(a.pso).glob(pat))
+        with mock.patch.object(pack_app, 'can_clone', lambda *a: False), mock.patch.object(pack_app, 'tree_bytes', lambda d: 0), \
+             mock.patch.object(shutil, 'disk_usage', lambda p: types.SimpleNamespace(free=(1 << 30) + msls + pso - 1)), contextlib.redirect_stdout(io.StringIO()) as o:
+            with self.assertRaisesRegex(SystemExit, 'the app needs'): pack_app.pack(a)
+        self.assertEqual(o.getvalue(), ''); self.assertFalse(Path(a.out).exists())
+
     def test_copies(self):   # another volume (no APFS clone): plain copies, the same layout; a new app replaces the old one whole
         with mock.patch.object(pack_app, 'can_clone', lambda *a: False), mock.patch.object(pack_app, 'sign', lambda *a: None), mock.patch('sys.stdout'):
             a = types.SimpleNamespace(**vars(self.args)); a.out = str(self.tmp / 'Copy.app'); a.icon = None
@@ -263,26 +299,61 @@ class Script(unittest.TestCase):
         self.assertEqual(nsp.mac_app_name('AC/DC'), 'AC - DC'); self.assertEqual(nsp.mac_app_name(':'), 'nxnative')
 
     def pack_cmd(self, gfx, **kw):
-        ran, tmp = [], Path(tempfile.mkdtemp()); self.addCleanup(shutil.rmtree, tmp)
+        ran, tmp = [], Path(tempfile.mkdtemp()); self.addCleanup(shutil.rmtree, tmp); self.said = []
-        out = tmp / 'G.app'
+        if kw.get('work'): tmp = kw['work']
+        out = tmp / 'G.app'; shutil.rmtree(out, ignore_errors=True)
         def fake_run(cmd, log, env=None, cwd=None, on_line=None):
-            ran.append([str(c) for c in cmd]); (out / 'Contents/MacOS').mkdir(parents=True); (out / 'Contents/MacOS/dmhost').write_bytes(b'x')
+            ran.append([str(c) for c in cmd])
+            if not on_line:   # pso_offline.py: its archive, or none when every eligible pair has function constants
+                o = Path(cmd[cmd.index('--out') + 1]); o.mkdir(parents=True, exist_ok=True)
+                if not kw.get('no_bases'): (o / f'{cmd[cmd.index("--gpu") + 1]}.mtl4a').write_bytes(b'arch')
+                return
+            (out / 'Contents/MacOS').mkdir(parents=True); (out / 'Contents/MacOS/dmhost').write_bytes(b'x')
             on_line('step: launcher and host (clones of the game files)')
         g = types.SimpleNamespace(table=lambda n: tmp / 'profile' / n, get={'app_category': 'public.app-category.games', 'romfs_marker': 'a/b'}.get)
-        with mock.patch.object(nsp, 'run', fake_run), mock.patch.object(nsp, 'PROG', mock.Mock()), mock.patch.object(nsp, 'chain_of', lambda repo: '0123456789ab'):
+        g.id = '01008CF01BAAC000'
+        with mock.patch.object(nsp, 'run', fake_run), mock.patch.object(nsp, 'PROG', mock.Mock()), mock.patch.object(nsp, 'chain_of', lambda repo: '0123456789ab'), \
+             mock.patch.object(nsp, 'mac_gpu_arch', lambda *a: 'applegpu_g14s'), mock.patch.object(nsp, 'say', self.said.append):
             clones = nsp.mac_app(sys.executable, REPO, out, 0x01008CF01BAAC000, tmp, tmp / 'ex', tmp / 'patched', tmp / 'dmhost', tmp / 'font', tmp / 'shader/cache' if gfx == 'nvn' else tmp / 'shader',
-                                 kw.get('aoc'), gfx, 'en-US', g, 'Game', '1.0.0', 'org.nxnative.01008cf01baac000', None, kw.get('sign', ''), {}, tmp)
+                                 kw.get('aoc'), gfx, 'en-US', g, 'Game', '1.0.0', 'org.nxnative.01008cf01baac000', None, kw.get('sign', ''), {}, tmp, gpus=kw.get('gpus', ''))
         self.assertTrue(clones)
-        c = ran[0]; return {c[i]: c[i + 1] for i in range(3, len(c) - 1) if c[i].startswith('--')}, tmp
+        self.pso_cmd = ran[0] if len(ran) > 1 else None   # pso_offline.py, before the packer
+        c = ran[-1]; return {c[i]: c[i + 1] for i in range(3, len(c) - 1) if c[i].startswith('--')}, tmp
 
     def test_pack_command(self):
+        o, tmp = self.pack_cmd('nvn')
+        self.assertNotIn('--pso-pairs', o)   # no split copy
+        (tmp / 'shader').mkdir(); (tmp / 'shader/pso_pairs.tsv').write_text('v\tf\n')
+        o, tmp2 = self.pack_cmd('nvn', work=tmp)
+        self.assertEqual(o['--pso-pairs'], str(tmp / 'shader/pso_pairs.tsv'))   # the profile has no pso_pairs.tsv: the split's copy goes in
         o, tmp = self.pack_cmd('nvn')
         self.assertEqual((o['--msl'], o['--romfs'], o['--tables'], o['--title-id'], o['--sign'], o['--marker']),
                          (str(tmp / 'shader/cache/b-0123456789ab'), str(tmp / 'ex/RomFS'), str(tmp / 'profile'), TID, '-', 'a/b'))
         self.assertNotIn('--aoc', o); self.assertNotIn('--icon', o)
+        self.assertNotIn('--pso', o)   # no shader pairs: no pipeline archive
         o, tmp = self.pack_cmd('vulkan', aoc=Path('/x/aoc'), sign='Developer ID Application: X (T)')
         self.assertEqual((o['--msl'], o['--aoc'], o['--sign']), (str(tmp / 'shader/vk'), '/x/aoc', 'Developer ID Application: X (T)'))
 
+    def test_pso_archive(self):   # an NVN game with shader pairs: its pipeline archive for this Mac's GPU, made by pso_offline.py, goes into the app
+        tmp = Path(tempfile.mkdtemp()); self.addCleanup(shutil.rmtree, tmp)
+        (tmp / 'profile').mkdir(); (tmp / 'profile/shader_pairs.tsv').write_text('v\tf\n')
+        o, _ = self.pack_cmd('nvn', work=tmp)
+        self.assertEqual(o['--pso'], str(tmp / 'pso/mac'))
+        p = self.pso_cmd; self.assertTrue(p[1].endswith('dm/gfx/tools/pso_offline.py'))
+        self.assertEqual({p[i]: p[i + 1] for i in range(2, len(p) - 1) if p[i].startswith('--')},
+                         {'--game': '01008CF01BAAC000', '--msl': str(tmp / 'shader/cache/b-0123456789ab'), '--out': str(tmp / 'pso/mac'), '--work': str(tmp / 'pso/mac-work'),
+                          '--gpu': 'applegpu_g14s', '--platform': 'macos'})
+        self.assertFalse(any('does not name this Mac' in m for m in self.said))
+        o, _ = self.pack_cmd('nvn', work=tmp, gpus='g18p')   # PSO_GPUS set for the IPA: built as asked, with a note that this Mac's app builds its own bases
+        self.assertEqual(o['--pso'], str(tmp / 'pso/mac'))
+        self.assertTrue(any('does not name this Mac\'s GPU (applegpu_g14s)' in m for m in self.said))
+        o, _ = self.pack_cmd('nvn', work=tmp, gpus='applegpu_g14s,g18p'); self.assertFalse(any('does not name this Mac' in m for m in self.said))
+        o, _ = self.pack_cmd('nvn', work=tmp, gpus='none')
+        self.assertNotIn('--pso', o); self.assertIsNone(self.pso_cmd)   # --gpu none: no archive, the app builds its bases on its first start
+        shutil.rmtree(tmp / 'pso')
+        o, _ = self.pack_cmd('nvn', work=tmp, no_bases=True)   # every eligible pair has function constants: pso_offline.py makes no archive, the app is packed without one
+        self.assertNotIn('--pso', o); self.assertIsNotNone(self.pso_cmd)
+
 
 if __name__ == '__main__':
     unittest.main()

docs/agents/symptoms.md

@@ -431,6 +431,7 @@ One line per symptom in the [subsystem references](../reference/): what you see
 - Metal shader validation reports invalid image-atomic stores even with an image-sized scratch buffer; without validation, GPU page faults can freeze rendering. → [Bounded MSL 2.3 atomic emulation](../reference/shaders.md#bounded-msl-23-atomic-emulation)
 - shaders missing from the map that no game file holds. → [Built-in shaders compiled with the game's own GLSLC](../reference/shaders.md#built-in-shaders-compiled-with-the-games-own-glslc)
 - a null read at `main+0x3316dc` in `nnMain`, right after the window setup (the program object's interface pointers at +0xd0 / +0xd8 both null). → [GLSLC GPU code version against the driver's range](../reference/shaders.md#glslc-gpu-code-version-against-the-drivers-range)
+- a self-contained Mac app (`--platform mac-app`) logs `pipeline prebuild: no .../Resources/dm/shader_pairs.tsv, nothing to build` although every shader library is there; it draws no hybrid stand-ins, and each pipeline seen for the first time compiles on the submit thread (frames of 100–550 ms). → [Shader-pair tables in a Mac app](../reference/shaders.md#shader-pair-tables-in-a-mac-app)
 - a truncated `.mtl4a` reaches `MTL4ArchiveImpl::loadFromURL`; the bundle reports coverage with `0 of 0 probed bases found`; rebuilt shader libraries reuse an archive of their old binaries; or packing fails with `pso_offline: no base to build` when every eligible pair has function constants. → [Offline pipeline archive reuse and fallback](../reference/shaders.md#offline-pipeline-archive-reuse-and-fallback)
 - identical render pipelines compile repeatedly under different keys, lengthening startup prewarm and adding pipeline misses during play. → [Pipeline keys omit unused colour state](../reference/shaders.md#pipeline-keys-omit-unused-colour-state)
 - a vertex shader whose pairs changed was restored unpadded from `cas/`. → [Shader cache key covering the pairs](../reference/shaders.md#shader-cache-key-covering-the-pairs)

docs/development.md

@@ -200,10 +200,15 @@ The bundle holds what the IPA holds, with the Mac's shader libraries:
   Resources/romfs/           the packed version's RomFS
   Resources/msl/b-<chain>/   the Mac shader libraries, reflection JSON and runtime TSV tables (vk/ or gl/ for a Vulkan or GL game)
   Resources/aoc/<index>/     the DLC, when the folder has some
-  Resources/dm/              the rewritten modules and their rewriter tables, nro.tsv, the profile's shader_map.tsv, pso_manifest.tsv, c1_full.tsv and cb_extent.tsv, font/, env.txt (--env-file)
+  Resources/pso/             an NVN game's pipeline archive made offline for this Mac's GPU (<arch>.mtl4a, prebuilt.tsv; --gpu none leaves it out)
+  Resources/dm/              the rewritten modules and their rewriter tables, nro.tsv, the profile's shader_map.tsv, pso_manifest.tsv, c1_full.tsv, cb_extent.tsv, shader_pairs.tsv and pso_pairs.tsv, font/, env.txt (--env-file)
   Resources/AppIcon.icns
 ```
 
+The shader pairs are what the pipeline prebuild and the hybrid pipelines read ([Hybrid pipelines](#hybrid-pipelines)), as in the IPA; a profile without `pso_pairs.tsv` gets the shader split's copy (`--pso-pairs`). An app without them builds no base: [Shader-pair tables in a Mac app](reference/shaders.md#shader-pair-tables-in-a-mac-app).
+
+An NVN game's app also carries its pipeline archive made offline ([The pipeline archive made offline](#the-pipeline-archive-made-offline)), built by `pso_offline.py --platform macos` for this Mac's GPU (`MTLDevice.architecture`, e.g. `applegpu_g14s` for an M2 Max; `--gpu` names others, `none` builds none) into `<id>_build/pso/mac` and made again only when its inputs change. The launcher hands `Resources/pso` to the host as `DM_PSO_BUNDLE`, as the iOS app's `main.m` does: the first start builds no base (`pipeline archive bundled with the app: holds the bases of every shader pair (16 of 16 probed bases found …)`), and the bases (about 615 MB for Breath of the Wild) stay in the app instead of the player's `~/Library/Application Support`. On a Mac with another GPU, or after a macOS update that changed the GPU compiler, the app builds them there as before. `PSO_GPUS` is shared with the IPA, so a value that does not name this Mac's GPU (`g18p`, an iPhone's) still builds what it names, and the console says that this Mac's app will build its own bases. When every eligible pair declares function constants, `pso_offline.py` makes no archive and the app is packed without one.
+
 The RomFS, the shader libraries and the DLC are APFS clones (`cp -c`) of the build's files when the app is on the same APFS volume, so they take no space of their own; on another volume they are copied, after a check for free space. The app is written as `<app>.partial` and replaces the old one only when it is complete.
 
 The shader folder includes only `.metallib`, `.json` (except `_b_build.json`) and `.tsv` files. Translated `.metal` sources, build records and the compile cache stay in the build folder and do not count toward the free-space check. Copies of many shader files are batched by encoded argument bytes, including the inherited environment, to stay below the system's `ARG_MAX` even in long NSP folder paths.
@@ -377,7 +382,7 @@ On iOS the standard is the pipeline archive made on the Mac and shipped in the a
 2. `metal-tt` compiles the script into one archive per GPU architecture: `xcrun -sdk iphoneos metal-tt -arch applegpu_g18p -ios-version-min 27.0 bases.mtl4-json -o applegpu_g18p.mtl4a`. The script needs the `.mtl4-json` extension, from which `metal-tt` takes the format: as `.mtlp-json` (Metal 3's format) it rejected even the Metal 4 serializer's own capture ("expecting [ instead got {"), which it compiles under the right extension. The archive refers to the libraries by content, not by path, so the app's own copies (`msl/b-<chain>`) find their bases.
 3. The archives go into the app as `pso/` with `prebuilt.tsv`, the stamp the device's prebuild would write for the same chain version, depth mode and pair lists. With an archive, or for a game whose profile sets `pso_mtl4 on` (its hybrid pipelines draw with them), the IPA also packs the pulling libraries (`_bp`), which the bases are made of; an Xcode build copies them anyway.
 
-`main.m` hands `pso/` to the host as `DM_PSO_BUNDLE` (`off` ignores it). The host opens the file named after the device's GPU architecture (`MTLDevice.architecture`, e.g. `applegpu_g18p.mtl4a`) read only and looks it up first among the files of bases without a vertex layout (`pso_bundle_open`); it never writes or compacts it. Before the prebuild, `pso_bundle_covers` compares the bundled `prebuilt.tsv` with this run's stamp, then looks up the bases of 16 pairs spread over the list. Coverage needs at least one actual lookup and no misses; ineligible sampled pairs supply no evidence. If both hold, the device builds no base, only the pairs in `rest_pairs.tsv`; otherwise it runs its own prebuild as before, which still takes from the bundle every base it finds there. An archive for another GPU, one the system cannot open (made for a newer system, or an OS update changed the GPU compiler), a probe that finds nothing, or a stamp of another depth mode each give an info line (`pipeline archive bundled with the app: …`), not a red one. The on-device prebuild is the fallback: it runs when the app has no archive for the device's GPU, when the system cannot open the archive or take its bases (an iOS update that changed the GPU compiler, a system older than its minimum), or when its stamp is not this run's (another depth mode, other pairs). The device's own archive (`pso_archive/`) gets only what the bundle lacks: a draw, the prewarm and the prebuild compile a base, which the serializer then captures, only after the bundle missed it, and bases with a vertex layout, which the bundle never holds (`t_pso_bundle` checks a covered pair's draw: no compile, nothing captured). The Mac builds its own archive as before, with the NSP script's prebuild tool ([Hybrid pipelines](#hybrid-pipelines)). Metal crashed (`MTL4ArchiveImpl::loadFromURL`) when handed a file that was no archive, a text file or a bare Mach-O header, so `arch_file_ok` checks the Mach-O load commands and payload ranges, universal slice bounds and the serializer's bundled Metal library ranges before opening a file; this holds for the device's own files too. It checks the container structure; the compiler's private payload remains Metal's responsibility.
+`main.m` hands `pso/` to the host as `DM_PSO_BUNDLE` (`off` ignores it). The host opens the file named after the device's GPU architecture (`MTLDevice.architecture`, e.g. `applegpu_g18p.mtl4a`) read only and looks it up first among the files of bases without a vertex layout (`pso_bundle_open`); it never writes or compacts it. Before the prebuild, `pso_bundle_covers` compares the bundled `prebuilt.tsv` with this run's stamp, then looks up the bases of 16 pairs spread over the list. Coverage needs at least one actual lookup and no misses; ineligible sampled pairs supply no evidence. If both hold, the device builds no base, only the pairs in `rest_pairs.tsv`; otherwise it runs its own prebuild as before, which still takes from the bundle every base it finds there. An archive for another GPU, one the system cannot open (made for a newer system, or an OS update changed the GPU compiler), a probe that finds nothing, or a stamp of another depth mode each give an info line (`pipeline archive bundled with the app: …`), not a red one. The on-device prebuild is the fallback: it runs when the app has no archive for the device's GPU, when the system cannot open the archive or take its bases (an iOS update that changed the GPU compiler, a system older than its minimum), or when its stamp is not this run's (another depth mode, other pairs). The device's own archive (`pso_archive/`) gets only what the bundle lacks: a draw, the prewarm and the prebuild compile a base, which the serializer then captures, only after the bundle missed it, and bases with a vertex layout, which the bundle never holds (`t_pso_bundle` checks a covered pair's draw: no compile, nothing captured). A Mac app carries one for the Mac it was built on ([A Mac app](#a-mac-app)); a Mac-only build (`--platform mac`) builds its own archive as before, with the NSP script's prebuild tool ([Hybrid pipelines](#hybrid-pipelines)). Metal crashed (`MTL4ArchiveImpl::loadFromURL`) when handed a file that was no archive, a text file or a bare Mach-O header, so `arch_file_ok` checks the Mach-O load commands and payload ranges, universal slice bounds and the serializer's bundled Metal library ranges before opening a file; this holds for the device's own files too. It checks the container structure; the compiler's private payload remains Metal's responsibility.
 
 The settings:
 - `DM_PSO_GPUS` (`build_ipa.sh`; the NSP script's `--gpu` / `PSO_GPUS`): the architectures, comma-separated, with or without the `applegpu_` prefix. Unset (or `PSO_GPUS` empty), it follows the profile: `g18p`, the A19 and A19 Pro of the iPhone 17 family, for a game that sets `pso_mtl4 on`, and `none` for the others, whose app would carry an archive it never reads (a test with `DM_PSO_MTL4=1` in `Documents/env.txt` sets it explicitly). Another device's name is its `MTLDevice.architecture`, which its log names when the app has no archive for it (`pipeline archive bundled with the app: none for this GPU (…)`). `none` builds no archive. Each architecture increases build time and app size; see the measurements in the game note linked above.

docs/reference/shaders.md

@@ -697,6 +697,14 @@ shader on a GPU.
 
 ## Pipelines and caches
 
+### Shader-pair tables in a Mac app
+
+- **Symptom:** a self-contained Mac app (`--platform mac-app`) logs `pipeline prebuild: no .../Resources/dm/shader_pairs.tsv, nothing to build` although every shader library is there; it draws no hybrid stand-ins, and each pipeline seen for the first time compiles on the submit thread (frames of 100–550 ms).
+- **Cause:** `dm/mac/pack_app.py` copied the shader map, the prewarm manifest, `c1_full.tsv` and `cb_extent.tsv` beside `DM_NVN_SHMAP`, but not `shader_pairs.tsv` and `pso_pairs.tsv`, which `build_ipa.sh` copies into the IPA and the host's pipeline prebuild reads there (`pso_prebuild`). Without them the app makes no pulling bases. For a profile without `pso_pairs.tsv` a Mac run (`mac_run`) passes the shader split's copy as `DM_PSO_PAIRS`; the app had neither. A full shader-library build does not build the pipeline bases by itself.
+- **Fix:** the packer copies both tables when the profile has them, and the split's `pso_pairs.tsv` (`--pso-pairs`, which the NSP script passes) when the profile has none. They stay optional. An app packed before the fix needs repacking. An NVN game's app also carries its pipeline archive made offline for this Mac's GPU, so the first start builds no base: [A Mac app](../development.md#a-mac-app).
+- **Found on:** [The Legend of Zelda: Echoes of Wisdom](../../dm/games/01008CF01BAAC000/README.md#rendering-and-shaders), update 1.1.0: a 308.4 ms frame, 283.5 ms of it pipeline acquisition and 5.9 ms GPU work; with the tables the offline prebuild built all 7,495 bases. [The Legend of Zelda: Breath of the Wild](../../dm/games/01007EF00011E000/README.md#pipelines-on-a-cold-start): frames of 100–546 ms before, none of its bases compiled on the first start after.
+- **Check:** `test_nsp_mac_app`: `App.test_pipeline_pair_tables`, `App.test_split_pso_pairs` and `App.test_layout` fail without the tables in the packer; `Script.test_pack_command` checks the split's copy is passed only when the profile has no `pso_pairs.tsv`.
+
 ### Offline pipeline archive reuse and fallback
 
 - **Symptom:** a truncated `.mtl4a` reaches `MTL4ArchiveImpl::loadFromURL`; the bundle reports coverage with `0 of 0 probed bases found`; rebuilt shader libraries reuse an archive of their old binaries; or packing fails with `pso_offline: no base to build` when every eligible pair has function constants.

scripts/build_ipa_from_nsp.py

@@ -51,6 +51,7 @@ DM_ENV_FILE = ''                        # optional: environment switches for a d
 PSO_GPUS = ''                           # NVN games: GPU architectures the app gets the shader pairs' pipeline bases for, made on this Mac (metal-tt, comma-separated: 'g18p' = A19 / A19 Pro of the iPhone 17 family;
                                         #   another device's name is in its log when the app has none for it), so the device does not build them on its first start; each adds about 650 MB to the app (Breath of the Wild) and minutes to a first build; 'none' = no archive;
                                         #   empty = by the game's profile: 'g18p' when it sets pso_mtl4 on (the Metal 4 path, the only one that reads the archive on iOS, is on for it), else 'none'
+                                        #   PLATFORM 'mac-app': the Mac app's archive (Resources/pso), empty = this Mac's GPU (e.g. 'g14s' for an M2 Max), 'none' = the app builds its bases on its first start
 SHOW_FPS = True                         # True shows the game's frame rate on the iPhone (frames on screen per second, top left); DM_SHOW_FPS=0 / 1 in the app's Documents/env.txt changes it without a rebuild
 SURVEY_ONLY = False                     # a game with no profile in dm/games: only write the survey, no provisional profile and no IPA
 PROFILE_OUT = ''                        # where a provisional profile goes (empty = <NSP folder>/<title ID>_build/profile; the profile is <this>/<title ID>/)
@@ -1566,19 +1567,66 @@ def mac_run(work, game, patched, host, font, msl, aoc, gfx, lang, env, logs, g,
     return f'cd {q(rd)} && ' + ' '.join(f'{k}={q(v)}' for k, v in renv.items()) + f' {q(host)} {q(patched)}'
 
 
+MTL_ARCH_M = '''#import <Metal/Metal.h>
+int main(void) { @autoreleasepool { id<MTLDevice> d = MTLCreateSystemDefaultDevice(); if (!d) return 1; printf("%s\\n", d.architecture.name.UTF8String); return 0; } }
+'''
+
+
+def mac_gpu_arch(work, env, logs):
+    """This Mac's GPU architecture as Metal names it (MTLDevice.architecture, e.g. applegpu_g14s for an M2 Max): the archive file the host opens in an app's pso/ (pso_bundle_open)"""
+    src, exe = work / 'tools/mtl_arch.m', work / 'tools/mtl_arch'
+    if not (exe.is_file() and src.is_file() and src.read_text() == MTL_ARCH_M):
+        src.parent.mkdir(parents=True, exist_ok=True); src.write_text(MTL_ARCH_M)
+        run(['xcrun', 'clang', '-fobjc-arc', '-framework', 'Metal', '-framework', 'Foundation', src, '-o', exe], logs / 'mtl_arch.log', env=env)
+    r = subprocess.run([str(exe)], capture_output=True, text=True)
+    arch = r.stdout.strip()
+    if r.returncode or not re.fullmatch(r'applegpu_\w+', arch): raise BuildError(f'cannot tell this Mac\'s GPU architecture ({(r.stderr or r.stdout).strip() or r.returncode}); --gpu names it, none builds the app without a pipeline archive')
+    return arch
+
+
+def mac_pso(py, repo, work, lib, g, gpus, env, logs):
+    """PLATFORM 'mac-app', an NVN game with shader pairs: its pipeline archive made offline for this Mac's GPU (dm/gfx/tools/pso_offline.py --platform macos, metal-tt), which the app carries as
+    Resources/pso and the launcher hands to the host as DM_PSO_BUNDLE, as the iOS app does (docs/development.md, The pipeline archive made offline): the app's first start builds no base,
+    and the bases (about 615 MB for Breath of the Wild) do not go into the player's ~/Library/Application Support. gpus: --gpu / PSO_GPUS (empty: this Mac's GPU; none: no archive, the app
+    builds the bases on its first start). Made again only when its inputs change (pso_offline.py's key). Returns the folder, or None"""
+    if gpus == 'none': say('pipeline archive: none (--gpu none): the app builds the pipeline bases on its first start'); return None
+    if gpus and not re.fullmatch(r'(applegpu_)?g\w+(,(applegpu_)?g\w+)*', gpus): raise BuildError(f'--gpu / PSO_GPUS takes GPU architectures such as g14s, or none: {gpus}')
+    try: here = mac_gpu_arch(work, env, logs)
+    except BuildError:
+        if not gpus: raise
+        here = None
+    if gpus and here and here not in {x if x.startswith('applegpu_') else 'applegpu_' + x for x in gpus.split(',')}:   # PSO_GPUS is shared with the IPA (g18p is an iPhone's)
+        say(f'pipeline archive: --gpu / PSO_GPUS ({gpus}) does not name this Mac\'s GPU ({here}): on this Mac the app builds its pipeline bases on its first start (empty = this Mac\'s GPU)')
+    arch = gpus or here
+    PROG.add(1); PROG.stage('Pipeline archive', step=f'{arch} (metal-tt, minutes when its inputs changed)')
+    out = work / 'pso/mac'
+    cmd = [py, repo / 'dm/gfx/tools/pso_offline.py', '--game', g.id, '--msl', lib, '--out', out, '--work', work / 'pso/mac-work', '--gpu', arch, '--platform', 'macos']
+    if not g.table('pso_pairs.tsv').is_file() and (work / 'shader/pso_pairs.tsv').is_file():   # the split's copy, as the app gets it (--pso-pairs)
+        cmd += ['--pairs', g.table('shader_pairs.tsv'), '--pairs', work / 'shader/pso_pairs.tsv']
+    run(cmd, logs / 'pso_offline_mac.log', env=env)
+    if not any(out.glob('*.mtl4a')):   # every eligible pair declares function constants: no base to bundle, the host specializes them on the device (rest_pairs.tsv)
+        PROG.end('no base without function constants'); say('pipeline archive: none (every eligible shader pair has function constants): the app builds them on its first start'); return None
+    PROG.end(f'{arch}, {amount(tree_size(out)[1], "B")}')
+    return out
+
+
 def mac_app_name(name):   # the app's file name: the game name with the characters Finder shows otherwise (':' and '/') as ' - '
     return re.sub(r'\s*[:/]\s*', ' - ', name).strip(' .-') or 'nxnative'
 
 
-def mac_app(py, repo, out, app, work, game, patched, host, font, msl, aoc, gfx, lang, g, name, version, bundle_id, icon, sign, env, logs, env_file=''):
+def mac_app(py, repo, out, app, work, game, patched, host, font, msl, aoc, gfx, lang, g, name, version, bundle_id, icon, sign, env, logs, env_file='', gpus=''):
     """The last stage of PLATFORM 'mac-app': pack the Mac app (dm/mac/pack_app.py) from what a Mac-only build made: the Mac host and font (mac_host), the packed version's RomFS and rewritten
-    modules, the Mac shader libraries (msl: <shader>/cache, whose b-<chain> goes in, or a Vulkan or GL game's <shader>, whose vk / gl goes in), the DLC, the profile's tables and an environment file.
+    modules, the Mac shader libraries (msl: <shader>/cache, whose b-<chain> goes in, or a Vulkan or GL game's <shader>, whose vk / gl goes in), the DLC, the profile's tables (with the shader pairs
+    the pipeline prebuild reads), the pipeline archive made offline for this Mac (mac_pso; gpus: --gpu / PSO_GPUS) and an environment file.
     The save and the log go to the player's Library, keyed by the folder's title ID (app); a new app replaces the old one only when it is complete"""
-    PROG.stage('Mac app', step='starting')
     lib = msl / (f'b-{chain_of(repo)}' if gfx == 'nvn' else 'vk' if gfx == 'vulkan' else 'gl')
+    pso = mac_pso(py, repo, work, lib, g, gpus, env, logs) if gfx == 'nvn' and g.table('shader_pairs.tsv').is_file() else None   # a stage of its own, before the app's
+    PROG.stage('Mac app', step='starting')
     cmd = [py, repo / 'dm/mac/pack_app.py', out, '--host', host, '--patched', patched, '--romfs', game / 'RomFS', '--msl', lib, '--tables', g.table('shader_map.tsv').parent,
            '--font', font, '--title-id', f'{app:016X}', '--name', name, '--bundle-id', bundle_id, '--lang', lang, '--sign', sign or '-']
+    split_pairs = work / 'shader/pso_pairs.tsv'   # the split's copy of pso_pairs.tsv, for a profile without the table (as mac_run's DM_PSO_PAIRS)
     for opt, v in (('--version', version), ('--category', g.get('app_category')), ('--marker', g.get('romfs_marker')), ('--aoc', aoc), ('--icon', icon),
+                   ('--pso', pso), ('--pso-pairs', split_pairs if gfx == 'nvn' and not g.table('pso_pairs.tsv').is_file() and split_pairs.is_file() else None),
                    ('--env-file', Path(os.path.expanduser(env_file)).resolve() if env_file else None)):
         if v: cmd += [opt, v]
     steps = []
@@ -1620,7 +1668,8 @@ def parse_args(argv):
     ap.add_argument('--keep-extracted', action=argparse.BooleanOptionalAction, help='keep the extracted game files after the IPA is built, for the next build (default: the KEEP_EXTRACTED setting, on)')
     ap.add_argument('--env-file', help='environment switches for a diagnostic build (one KEY=VALUE per line)')
     ap.add_argument('--gpu', help='NVN games: GPU architectures the app gets its pipeline archive for, made on this Mac with metal-tt, comma-separated: g18p (A19 / A19 Pro), or the name a device\'s log gives when the app has no archive for its GPU; '
-                    'none = no archive, the device builds the bases on its first start (default: the PSO_GPUS setting; when that is empty, g18p for a game whose profile sets pso_mtl4 on, else none)')
+                    'none = no archive, the device builds the bases on its first start (default: the PSO_GPUS setting; when that is empty, g18p for a game whose profile sets pso_mtl4 on, else none). '
+                    'With --platform mac-app: the Mac app\'s archive, by default for this Mac\'s GPU (none: the app builds the bases on its first start)')
     ap.add_argument('--show-fps', action=argparse.BooleanOptionalAction, help='show the game\'s frame rate on the iPhone, frames on screen per second at the top left (default: the SHOW_FPS setting, on)')
     ap.add_argument('--survey-only', action='store_true', help='a game with no profile: only the survey, no provisional profile and no IPA')
     ap.add_argument('--allow-damaged', action='store_true', help='a damaged NSP (files failing the NCA hash check): write them as read and go on instead of stopping')
@@ -1838,7 +1887,8 @@ def main(args=None):
     if mac_pack:   # a Mac app: the Mac host, then the app packed from it and the build's files
         host, font = mac_host(py, repo, work, env, logs, g)
         if not icon: say('[RED] no game icon in the extraction (an older extraction did not save it: delete extract/ and run again); the app\'s icon will be blank')
-        clones = mac_app(py, repo, out_ipa, app, work, game, patched, host, font, ios, aoc, gfx, lang, g, app_name, app_version, bundle_id, icon, args.mac_sign or MAC_SIGN, env, logs, args.env_file or DM_ENV_FILE)
+        clones = mac_app(py, repo, out_ipa, app, work, game, patched, host, font, ios, aoc, gfx, lang, g, app_name, app_version, bundle_id, icon, args.mac_sign or MAC_SIGN, env, logs, args.env_file or DM_ENV_FILE,
+                         (args.gpu or PSO_GPUS).strip())
         rmtree(work / 'tmp')
         say(f'done: {out_ipa} ({tree_size(out_ipa)[1] / 2**30:.1f} GB{", its game files APFS clones of the extracted ones, which take no space of their own" if clones else ""}); '
             f'open it from Finder. Its save is in ~/Library/Application Support/nxnative/{app:016X}/, its log in ~/Library/Logs/nxnative/{app:016X}.log')