8 Commits
Author SHA1 Message Date
seedds 54647b08ea Make repeated IPA and Mac app builds incremental (#279)
Reuse unchanged IPA entries through the packing index and APFS clones,
and keep Mac app and derived-profile shader outputs while their inputs
match. Preserve main's shader regression checks and .NET discovery.

Count uncompressed bytes in streamed files so a size change aborts the
pack without replacing the previous IPA or index. Cover both deflated
and stored streams with growth and shrinkage regression tests, and
record the cause and fix in the packing reference.

Co-authored-by: Suraj Mandal <surajmandalcell@gmail.com>
(cherry picked from commit 09f3778c36)
2026-10-26 19:35:18 -06:00
seeddsandv0lumehi 92fc2f2711 Build and ship offline Metal pipeline archives (#226)
Compile eligible shader-pair bases on the Mac with metal-tt, package one
archive per selected iPhone GPU, and use a matching bundle before falling
back to on-device preparation. Carry the profile depth mode through packing
and include the pulling libraries required by the bundled bases.

Integrate PR #226 with four confirmed fixes: validate archive container
bounds before invoking Metal; require at least one actual successful bundle
probe; hash consumed library bytes into offline cache keys; and emit device
remainder lists for constant-only plans instead of aborting packaging.
Keep game measurements in the game note and document the general findings.

Validation on current main: 52 real Metal bundle checks, 57 archive checks,
79 prewarm checks on each pipeline path, 73 Python regressions covering
offline tools, packaging, documentation and chain checks, and compilation
of all 133 iOS source files. No new iPhone gameplay validation. The archive guard validates
container structure, not private Metal payloads.

Co-authored-by: v0lumehi <aljic.zijad@gmail.com>
2026-10-26 10:52:50 +08:00
seedds 6b947d9504 Fix Xcode workflow mock for the rewrite profile return
Return the fixture profile from the rewrite mock so the preparation pipeline keeps a valid game profile after the Portal change.

Validation: all three tests in dm/ios/tests/test_xcode_workflow.py pass; git diff --check passes.
2026-10-26 01:46:07 +08:00
seedds 156d3c2a18 CI on GitHub's macOS runners (from PR #132): .github/workflows/tests.yml runs, for pull requests, pushes to main and manual dispatch, three jobs on macos-26 in parallel: the host unit tests and the graphics tests that need no GPU (t_dirty_slots, t_dirty_race, t_pace, t_fstep, t_hmap, t_cover, t_pso_skey, t_refl and t_rt_range_ro, the last two skipping their sample checks); every test_*.py of dm/tools/tests, dm/ios/tests and dm/gfx/tests on Python 3.14 with lz4, capstone and fonttools; and a build check: dmhost for Wonder (NVN), Hades (Vulkan) and Digimon Story Cyber Sleuth (GL), every graphics test compiled but not run, and the iOS host sources compiled for the iPhone SDK by a new helper, dm/ios/tools/compile_check.py (the host Makefile's file list and flags as gen_build.py takes them, the host's own command first so its header paths come before libvpx's; 133 files, no game files, signing or Xcode project). Read-only permissions, superseded runs cancelled, a timeout per job, logs uploaded only on failure, actions pinned to commit SHAs (checkout v7.0.1, setup-python v7.1.0, upload-artifact v7.0.2, each checked against its tag). The GPU tests stay local (the hosted runners are virtual Macs without Metal 4, issue #112), and so does profile_guess.py --check (it reads game extractions). test_xcode_workflow.py: the stand-in profile carries the registered profile's title_id, which report_profile_status reads since ce17300 (the test had 3 errors on main). Checked by replaying the three jobs on this Mac on current main: all pass, the tool tests in a fresh Python 3.14 venv with only the three packages 2026-10-25 01:58:01 +08:00
seedds ebe75c9d5d test_xcode_workflow.py: the provisional-warning test calls report_profile_status with its artifact name ('Xcode project'), the argument 98a91a6 (--platform mac) put in place of prepare_xcode; it failed with a TypeError since. The five tool and workflow test files pass (37 tests) 2026-10-24 01:52:16 +08:00
seedds c392d56af0 Balatro (0100CD801CE5E000, update 1.1.3, LÖVE 11.4 "love-nx" with LuaJIT and SDL on SDK 18.3.2): game profile and the GL layer LÖVE needs, plus four general changes. Plays on the iPhone by touch; on the Mac from the intro and the title into a run and its tutorial, at 60 fps with sound, about 0.65 GB resident and flat over three minutes. The game is Lua source in the RomFS and renders with NVIDIA's desktop OpenGL through EGL, as Digimon does. GL layer (dm/gfx/gl/hle_gl.m): LÖVE reads GL_CONTEXT_PROFILE_MASK and takes its core-profile paths only when it says core (with a compatibility profile it streams vertices from client memory, which Metal cannot draw from), so the layer answers core and implements what those paths use, about 150 functions in all against Digimon's 79: persistently mapped buffers (glBufferStorage, glMapBufferRange: the shared MTLBuffer's own memory), glBufferSubData in command order, fences (a fence is the serial of the command buffer being recorded; a wait on one still recording commits it), glBindAttribLocation (the game's attribute index of each Metal input, set at the link) and current attribute values (glVertexAttrib4f: LÖVE's VertexColor and ConstantColor are disabled arrays), glGetActiveUniform and the uniform read-backs, every glUniform* form, glTexStorage2D / 3D with their level counts, 3D and 2D array textures (LÖVE makes a default of each type on 96 units), GL_TEXTURE_SWIZZLE_* (its font glyphs are RG8 read as R, R, R, G), glGenerateMipmap as a blit (the game's images are mipmapped), instanced and base-vertex draws, separate read and draw framebuffers, glBlitFramebuffer (colour, same size), glReadPixels, glClearBufferfv, glFrontFace and the EGL calls SDL makes; stack arguments of the functions with more than eight are 64-bit, as AAPCS64 lays them out. DM_GL_STUBS=1 gives every GL name the layer lacks a stub of its own that reports its first call, instead of a null pointer. Shaders: LÖVE puts its 35 GLSL sources together at run time in Lua, so the NSP script captures them on the Mac as it does Super Mario Galaxy's SPIR-V (capture_shaders now serves both, profile gl_capture.tsv; profile_guess.py's gl_capture rule: an OpenGL game with no GLSL source in its RomFS or main), and gl_shaders.py keeps a bool uniform's declared type (a uint in SPIR-V; LÖVE's Shader:send takes only a boolean for a bool) and takes from main only strings that start with '#version <number>' (Balatro's main has glslang's messages). General: profile_guess.py leaves vkGetInstanceProcAddr, SDL's Vulkan loader, out of its Vulkan count (Balatro's provisional profile said Vulkan; Hades and both Galaxies import 99 or 100 Vulkan functions); OpenAudioRenderer without an event now starts the host's render thread too (hle_audren.c ar_open: only the form with an event did, and such a game was silent); loader_census.py follows registers in a straight line (adrp, add, loads through relocated GOT entries, mov, csel), for LÖVE's glad, whose pointers are globals its code keeps the addresses of in registers; and the iPhone gamepad has a third layout, Tap (profile pad_layout touch, which needs op_mode handheld; DM_GAME_PAD_LAYOUT replaces DM_GAME_PAD_FULL): no control but the layout button, every touch to the game, offered in the layout cycle only in handheld mode. Profile: op_mode handheld (the Switch build plays by touch there), pad_layout touch, gl_capture.tsv. A Lua error makes Balatro quit (its Switch error handler), and love-nx's video worker is then deleted twice (main+0x339c00): the bring-up notes say how the errors were found. Checked: the NSP script packs it from the NSPs alone (the capture in under a second, 35 shaders for the Mac and iOS, an 82 MB IPA), tool tests pass, profile_guess.py --check changes no other profile; Digimon was not rerun (its files are not on hand). README (games table, a Mac picture of the title), AGENTS.md (47 → 48), docs/development.md, docs/agents and the READMEs of dm/games and dm/gfx say so 2026-10-24 01:50:19 +08:00
seedds a2e6f4a18c NSP script: a registered profile's chain files that a chain change left behind are made again (GitHub issue #78: after bb585cd, 45 of 47 profiles' dmsh_chain.h no longer held the version their source gives, and Breath of the Wild's pack stopped at dmsh_build). chain_header writes dmsh_chain.h again, before any host is built, when it does not hold the version dmsh_chain.py gives (it follows from the source alone; a current one is left untouched, so the host is not rebuilt). chain_tables makes c1_full.tsv (c1_guard.py) or cb_extent.tsv (cb_extent.py) again from the build's Mac outputs when its header names another chain, which the host would ignore (every c1 its own 64 KiB, whole uniform bindings copied), also when the libraries are reused; provisional and derived profiles make both as before. The two tables are the shader step's outputs, so its reuse key no longer counts them. Tests: dm/tools/tests/test_nsp_chain.py (stale, missing and current headers and tables), and chain_tables with the real tools on Breath of the Wild's Mac cache (47 s, the committed list's 2 entries) 2026-10-23 22:11:09 +08:00
seeddsandAnimax Deng 6da1882871 NSP script: --prepare-xcode makes a per-game Xcode project to sign and run with your own developer account (GitHub PR #59)
Each game gets its own project in <NSP folder>/<title ID>_build/xcode, registered in a shared NxNative.xcworkspace; a target phase copies the RomFS, shader libraries and add-on content into the app before Xcode signs it, and preparing again keeps the bundle ID and team chosen in Xcode. The unsigned IPA build is unchanged.
On top of the PR: the resource phase stops when the profile's shader chain is no longer the prepared shaders' (the host loads only its own chain), prepared projects ask for the Increased Memory Limit capability (DM_ENTITLEMENTS overrides), the source links are made again on every preparation, and scheme names keep non-ASCII letters. The PR's TOML launcher is left out.

Co-authored-by: Animax Deng <Animax.deng@gmail.com>
2026-10-23 00:37:52 +08:00