diff --git a/docs/concepts/payload.md b/docs/concepts/payload.md index 757b71a..c485524 100644 --- a/docs/concepts/payload.md +++ b/docs/concepts/payload.md @@ -103,6 +103,13 @@ An `--owner-trust-cache` bundle is copied without the `[payload] trust_cache_sha ## Data ramdisk +When the selected kernel configuration enables `CONFIG_XNU_OWNER_DATA_MOUNT`, +the payload requires `--data-ramdisk FILE` or `[build] data_ramdisk` in the local +build configuration. Without a generated `.config`, this check reads the +selected device defconfig. A persistent overlay still needs its original seed +to publish the Data volume; changing the seed is not a way to resume old Data. +Missing seeds stop packaging before the payload is written. + A Data seed can be delivered in two forms: | Form | ADT placeholder | Behavior | diff --git a/docs/drivers/ath12k-wifi.md b/docs/drivers/ath12k-wifi.md index 3bbfb6e..3c781d4 100644 --- a/docs/drivers/ath12k-wifi.md +++ b/docs/drivers/ath12k-wifi.md @@ -28,6 +28,22 @@ Status: joins WPA2-Personal, gets DHCP, Safari loads pages. Throughput is low. | `sheng-wifi=5` | Start Wi-Fi. Without it the module never runs. | | `sheng-wifi-aj=0` | No join at boot; Settings joins. | +### iPadOS 27 service startup + +The sheng board DTS exports WCN7850 as `/arm-io/wlan` using +`unixnu,adt-name = "wlan"`. The stock 24A446 `com.apple.wifid` job checks +`_LimitLoadToDeviceTree` before launchd loads it. Without a matching node, +the kernel's `join.txt` path can still connect and exchange packets while +Settings shows an empty Wi-Fi page and cannot enable the switch. +`/chosen/mac-address-wifi0` and the IOService network objects do not satisfy +this device-tree condition. + +The 27 payload defaults include `sheng-wifi=5`. After changing the DTS, +rebuild the payload and boot image, then inspect the ADT inside the image. +Host checks confirm the node satisfies the startup condition; starting +wifid, enabling the Settings switch and scanning still require a device +test. Remove `sheng-autoreset` from a manual test image's boot arguments. + ### Facts - Each of the six PMU rails needs RPMh mode, voltage and enable requests. Without the mode request two rails stay off and the link never trains. diff --git a/docs/drivers/xnu-prelink.md b/docs/drivers/xnu-prelink.md index 1409f45..5e73804 100644 --- a/docs/drivers/xnu-prelink.md +++ b/docs/drivers/xnu-prelink.md @@ -19,12 +19,44 @@ matching code signature before reusing a map. comparison. It transfers unambiguous names from unique code matches and resolves those names in the actual native kernel. The map records the reference hash. +The sheng iPadOS 27 build runs `build-fileset-release-reference.py --tree . --lto` +before prelinking. Keep LTO enabled for this comparison: the non-LTO reference +loses matches needed by original kexts, including corecrypto's startup call to +`kprintf`. With `--allow-missing`, such a call reaches `unixnu_fileset_unmapped` +at runtime even though packaging succeeds. + +The reference builder records its mode, source, SDK, archive, compiler and +version inputs and verifies the output digest before reusing it. Unrecorded +references are rebuilt. Different inputs use separate build directories; +LTO export inspection reads the generated LTO object and non-LTO file list. +After a reference change, regenerate the map, kernel, payload and boot image, +and inspect the unresolved-target report before hardware testing. + +The 24A446 fixture preserves 23 import identities recovered from the retained +RELEASE-LTO map and cross-checked against its boot image, plus the observed +corecrypto `kprintf` import. Their original function digests and recovery +provenance are recorded; the current native targets are resolved by symbol. +The fixture also identifies 13 previously missing vnode operation descriptors by +`vdesc_name`, four metaclass pointer slots by their chained targets, and the +`gIODTPHandleKey` and `audio_active` globals by their source consumers. Unbound +vnode descriptors can make APFS registration read a zone field as an operation +index in `vfs_fsadd`; they must resolve to the native descriptors initialized by +VFS, not preserve old collection data addresses. + + The emitter places native segments within the collection's code, constant and data ranges, applies native split-segment records, rebuilds chained fixups and function starts, and recomputes corecrypto's code HMAC. The collection root uses the native link address. The ordinary prelink `--slide` value does not apply to filesets. The native input executable is unchanged. +Native adapters to retained kext functions declare fixed targets as `N_ABS` +`.original` symbols and use `BTI c; B` stubs. These branches lack linker +split-segment records, so the emitter recomputes them after placement and checks +every destination against retained executable ranges. The report records each +binding in `native_original_branches`. Builders must not pre-subtract a cached +native slide. Absolute data pointers in GOT entries keep their fixed values. + Missing mappings stop emission by default. `--allow-missing` produces an experimental image and the JSON report lists every unresolved target by kext. An image that passes structural checks still requires complete mappings and @@ -43,3 +75,9 @@ The catalog records implementation scope, hardware and OS requirements, board de ``` [Component catalog](/reference/components) + +The 24A446 packaging profile additionally checks a regression policy against the +original (unslid) imported targets. New unresolved data or stubbed calls fail +before replacing the last kernel candidate. This caught two lost order-based +matches during a full rebuild: `IOCircularDataQueueCopyLatest` and `get_aiotask`; +the reviewed fixture now identifies both by original digest and native symbol. diff --git a/docs/porting/packaging.md b/docs/porting/packaging.md index 6cb3e79..04b19d5 100644 --- a/docs/porting/packaging.md +++ b/docs/porting/packaging.md @@ -124,3 +124,43 @@ Booting the artifacts is a separate step with its own record. 4. Record the run in the `packaging` area of `porting.toml`. [Build and boot](/building), [Device repository](/porting/device-repository) + +## Runtime inputs for sheng iPadOS 27 + +The payload's `runtime` table enables preparation before kernel compilation. +Apple static/System/App trust modules come from the selected IPSW unchanged. Turnip, +AdrenoMetal and the board audio HAL are built from source; their CDHashes are +appended as a separate owner trust module. Wallpaper overlays come from the +read-only OS image. The official Xiaomi fastboot archive still supplies the +firmware inputs through the board preflight scripts. + +An existing installation must reuse its original `data.dbr` with +`--data-ramdisk FILE`. GPU and audio updates travel in the addon and do not edit +that seed. The runtime step checks its APFS volume UUID against the generated +kernel configuration and records the input SHA-256 values in the package manifest. +A payload-only build refuses a different Data UUID and asks for a kernel rebuild. + +For a **first installation**, `scripts/package.sh --os 27 --init-data` creates an +APFS seed from the IPSW's initial `private/var`, uses the board's `data_gib` and +Setup preferences, and binds the new UUID into the kernel configuration. This +writes host build files only; it does not erase or flash a device. Keep the seed +at `out/sheng/ipados-27.0.1-24A446/sheng_ipados27/runtime/data.dbr` for every later +build of that installation. Its content identity, not just its UUID, identifies +the persistent Data store. Regenerating it cannot preserve an older installation. + +The package workflow restores this seed from an installation cache, or accepts +an original seed URL with a required SHA-256. A missing cache stops the build +unless `init_data` is explicitly selected. The workflow uploads a separate seed +backup with one-day retention. Download and retain it: Actions caches are not a +permanent backup. The current device's seed is not supplied by either firmware +archive. Automatic runtime generation currently targets the iPadOS 27 profile. + +`--gpu-addon FILE` and its matching `--owner-trust-cache FILE` remain available +for reproducing an existing image. Omit both to compile the addon and generate +its trust bundle. These generated binaries and installation seeds are not Git +inputs. + +For the 24A446 fileset, `regression_policy` bounds the unresolved references by +original kernel address and importing component. New gaps fail packaging even +with `--allow-missing`; this check does not qualify the remaining experimental +paths or replace a device boot test. diff --git a/docs/zh/concepts/payload.md b/docs/zh/concepts/payload.md index 7c282b4..b6bac22 100644 --- a/docs/zh/concepts/payload.md +++ b/docs/zh/concepts/payload.md @@ -41,6 +41,11 @@ initrd = "Payloads/scratch-data.img" args = "-v serial=3" ``` +所选内核配置启用了 `CONFIG_XNU_OWNER_DATA_MOUNT` 时,必须通过 +`--data-ramdisk FILE` 或本地构建配置的 `[build] data_ramdisk` 提供种子。 +没有生成 `.config` 时检查所选设备 defconfig。持久化覆盖层也需要原始种子来 +发布 Data 卷;更换种子不能恢复旧 Data。缺少种子会在写负载前报错。 + 提供 Data 种子时存在 `initrd`。`args` 是 `[payload] kernel_args` 和 `--kernel-arg` 合并后的命令行。 ## ADT sidecar diff --git a/docs/zh/drivers/ath12k-wifi.md b/docs/zh/drivers/ath12k-wifi.md index be53143..6e2e485 100644 --- a/docs/zh/drivers/ath12k-wifi.md +++ b/docs/zh/drivers/ath12k-wifi.md @@ -26,6 +26,20 @@ Ath12kWifi 在自己的兼容层(`shim/`)上编译 Linux 的 ath12k、MHI | `sheng-wifi=5` | 启动 Wi-Fi。没有它模块不会运行。 | | `sheng-wifi-aj=0` | 启动时不加入网络,由设置加入。 | +### iPadOS 27 服务启动 + +sheng 的板级 DTS 通过 `unixnu,adt-name = "wlan"` 将 WCN7850 导出为 +`/arm-io/wlan`。24A446 原生 `com.apple.wifid` 任务带有 +`_LimitLoadToDeviceTree` 条件,launchd 加载前会检查。缺少匹配节点时, +内核的 `join.txt` 路径仍能连接并收发数据,但设置中的 Wi-Fi 页面为空, +开关无法打开。`/chosen/mac-address-wifi0` 和 IOService 网络对象不能满足 +这个设备树条件。 + +27 的默认 payload 参数包含 `sheng-wifi=5`。修改 DTS 后需要重新生成 +payload 和 boot 镜像,并检查镜像内的 ADT。主机检查确认节点满足启动条件; +wifid 启动、设置开关和扫描功能仍需上机验证。手动测试镜像的启动参数应移除 +`sheng-autoreset`。 + ### 事实 - 六条 PMU 电源轨各需要 RPMh 的模式、电压和使能请求。缺少模式请求时有两条保持关闭,链路无法训练。 diff --git a/docs/zh/drivers/xnu-prelink.md b/docs/zh/drivers/xnu-prelink.md index dcfbd75..874ffc2 100644 --- a/docs/zh/drivers/xnu-prelink.md +++ b/docs/zh/drivers/xnu-prelink.md @@ -5,6 +5,27 @@ `--fileset-reference FILE` 可以提供带符号的对照内核。唯一的代码匹配可提供符号名, 最终地址仍从实际编译的原生内核获取。映射记录中会保存对照文件的哈希。 +sheng 的 iPadOS 27 构建在预链接前运行 +`build-fileset-release-reference.py --tree . --lto`。该对照内核需要启用 +LTO;非 LTO 对照会丢失原生 kext 所需的匹配,包括 corecrypto 启动时对 +`kprintf` 的调用。启用 `--allow-missing` 时,打包可以成功,但该调用会在 +运行时进入 `unixnu_fileset_unmapped` 并触发 panic。 + +对照内核构建脚本记录模式、源码、SDK、归档库、编译器和版本输入,复用前还 +会核对输出哈希。没有记录的旧产物会重建;输入改变时使用独立构建目录。 +LTO 模式从生成的 LTO 对象和非 LTO 文件列表检查导出符号。对照内核改变后, +需要重新生成映射、内核、payload 和 boot 镜像,并在上机前检查未解析目标报告。 + +24A446 的符号记录保留了从旧 RELEASE-LTO 映射恢复、并与对应启动镜像交叉 +核对的 23 个导入函数身份,以及此次定位的 corecrypto `kprintf` 导入。 +记录包含原始函数字节哈希和恢复依据;当前原生内核的目标地址仍按符号重新解析。 + +固定记录还按 `vdesc_name` 识别 13 个此前缺失的 vnode 操作描述符,按链式 +指针目标识别 4 个 metaclass 指针槽,并通过源码调用方识别 `gIODTPHandleKey` +和 `audio_active`。APFS 注册时会读取这些描述符的操作下标;保留旧地址可能 +把 zone 数据当成下标,导致 `vfs_fsadd` 越界。必须绑定到由新内核 VFS 初始化 +的描述符。 + 对于 arm64e fileset 内核集合,`emit_prelink.py` 会选择 fileset 路径并生成 函数和数据映射。`--fileset-map FILE` 可以指定已有映射,输入哈希和匹配器版本 必须一致。输出保留所选根组件及其全部依赖。 @@ -14,6 +35,12 @@ HMAC。集合根头使用原生内核的链接地址,普通预链接的 `--slide` 不用于 fileset。 原生内核输入文件保持不变。 +原生适配器通过 `BTI c; B` 跳板调用保留的 kext 函数,固定目标由 `N_ABS` +符号 `.original` 声明。这类跳转没有链接器分段重定位记录,因此输出工具 +在布局完成后重新计算跳转,并核验目标位于保留组件的可执行范围内。报告中的 +`native_original_branches` 记录每条绑定。构建脚本不得预先减去缓存的原生内核 +位移;GOT 中的绝对数据指针保持固定值。 + 默认遇到缺失映射就停止。`--allow-missing` 可以生成实验映像,JSON 报告按 kext 列出所有未解析目标。结构验证通过后,仍需要补全映射并进行硬件验证, 才能确认启动兼容性。 @@ -39,3 +66,8 @@ XnuPrelink 将构建的 XNU 内核绑定到所选原始 kext,并写出预链 ``` [组件目录](/zh/reference/components) + +24A446 打包配置还会按原始、未滑动的导入地址检查回归清单。新增未解析数据或 +stub 调用会在替换上次内核产物前报错。完整重编曾丢失两个依赖顺序的匹配: +`IOCircularDataQueueCopyLatest` 和 `get_aiotask`;现在已根据源码和反汇编, +用原函数摘要及 native 符号固定其身份。 diff --git a/docs/zh/porting/packaging.md b/docs/zh/porting/packaging.md index b4b43f2..de50494 100644 --- a/docs/zh/porting/packaging.md +++ b/docs/zh/porting/packaging.md @@ -106,3 +106,32 @@ remove = [ ... ] 4. 把运行记录到 `porting.toml` 的 `packaging` 区域。 [构建与启动](/zh/building)、[设备仓库](/zh/porting/device-repository) + +## sheng iPadOS 27 运行输入 + +`payload.runtime` 在编译内核前准备运行输入。Apple 静态、System 和 App 信任模块从所选 +IPSW 原样提取;Turnip、AdrenoMetal 和设备音频 HAL 从源码编译,其 CDHash 写入 +单独的 owner 信任模块。壁纸覆盖文件从只读 OS 镜像生成。小米官方 fastboot 固件 +继续由设备 preflight 脚本下载、校验和提取。 + +保留现有安装的数据时,必须通过 `--data-ramdisk FILE` 复用原来的 `data.dbr`。 +GPU 和音频更新放在 addon 中,不修改 seed。构建会检查 APFS 卷 UUID,将它绑定到 +本次生成的内核配置,并把输入 SHA-256 写入打包清单。只打包 payload 时,若 UUID +与内核配置不同则报错,要求重新编译内核。 + +**首次安装**可用 `scripts/package.sh --os 27 --init-data`,从 IPSW 的初始 +`private/var` 生成 APFS seed,采用设备配置的 `data_gib` 和 Setup 偏好,并把新 +UUID 写入内核配置。该命令只写本机构建文件,不擦除或刷写设备。此后应一直保留 +`out/sheng/ipados-27.0.1-24A446/sheng_ipados27/runtime/data.dbr`。 +持久化 Data 绑定的是 seed 的内容身份,仅保留 UUID 或从同一 IPSW 重做都不能替代原文件。 + +Actions 会恢复安装对应的 seed 缓存,也支持原 seed 的下载 URL 加必填 SHA-256。 +缓存丢失时默认停止,只有明确选择 `init_data` 才会新建。工作流另提供保留一天的 +seed 备份下载;请自行长期保存,Actions 缓存不能作为永久备份。当前设备的原 seed +无法从官方固件包恢复。自动生成运行输入目前只适用于 iPadOS 27。 + +要复现已有镜像,可同时指定 `--gpu-addon FILE` 和配套的 +`--owner-trust-cache FILE`;省略二者则从源码构建。生成的二进制和 seed 不提交 Git。 + +24A446 的 `regression_policy` 按原内核地址和导入组件限制未解析引用;即使启用 +`--allow-missing`,新增缺口也会使打包失败。这不代表剩余实验路径已验证,仍需上机启动测试。 diff --git a/kits/XnuPrelink b/kits/XnuPrelink index b74ed7c..b1888c1 160000 --- a/kits/XnuPrelink +++ b/kits/XnuPrelink @@ -1 +1 @@ -Subproject commit b74ed7c74a07f0eb5f81089c660905ecc4cabb7f +Subproject commit b1888c12456c46a2226d63c794465dfcee08d465 diff --git a/profiles/os/ipados-27.0.1-24A446/profile.toml b/profiles/os/ipados-27.0.1-24A446/profile.toml index a0a2561..5e2c0c4 100644 --- a/profiles/os/ipados-27.0.1-24A446/profile.toml +++ b/profiles/os/ipados-27.0.1-24A446/profile.toml @@ -31,6 +31,7 @@ "kernel" = "build/xnu/ios-obj/DEVELOPMENT_ARM64_BCM2837/mach.development.bcm2837.unstripped.noctf" "kernelcache_sha256" = "38e6d9fa5f789b8228f4617ef4c6d2a3fd8ab564abdc9b62920f1db53fc1780a" "slide" = "-0x1000000" +regression_policy = "scripts/policies/ipados-27.0.1-24A446/prelink-regression.json" # The AMFI wrapper that calls _initializeAppleMobileFileIntegrity and returns 0; the kernel patch calls it early. "early_security_init" = "com.apple.driver.AppleMobileFileIntegrity=0xfffffe00092551ac" # Bring-up: match against the symbol-bearing RELEASE build of the same XNU source diff --git a/scripts/build-adreno-metal-plugin.py b/scripts/build-adreno-metal-plugin.py index 07b64e2..efa16b4 100644 --- a/scripts/build-adreno-metal-plugin.py +++ b/scripts/build-adreno-metal-plugin.py @@ -5,6 +5,7 @@ and the launchd probe daemon with its job plist (NativeAdrenoProbeDaemon.cpp). Output (ignored, never committed): build/adreno-metal-plugin/NativeAdrenoMetal.bundle with a 4 KiB-page ad-hoc signature, and trust.json with its CDHash. The bundle is staged into the Data seed by add-gpu-plugin-data.py and trusted through an extra owner trust module (make-owner-trust-bundle.py); neither changes the original trust modules, AMFI or any Apple signature.""" +import argparse import json import plistlib import re @@ -15,7 +16,11 @@ from pathlib import Path ROOT = Path(__file__).resolve().parents[1] # --target ios27: the arm64e build for iPadOS 27 (24A446), whose render server is an arm64e process. -IOS27 = sys.argv[1:] == ['--target', 'ios27'] +parser = argparse.ArgumentParser(description=__doc__) +parser.add_argument('--target', choices=('ios', 'ios27'), default='ios') +parser.add_argument('--no-replay', action='store_true', help='omit the optional host simulation capture used by diagnostic probes') +args = parser.parse_args() +IOS27 = args.target == 'ios27' OUT = ROOT / ('build/adreno-metal-plugin27' if IOS27 else 'build/adreno-metal-plugin') bundle = OUT / 'NativeAdrenoMetal.bundle' if OUT.exists(): @@ -30,7 +35,10 @@ if not TURNIP.is_file(): raise SystemExit('Build Turnip first: python3 scripts/build-turnip-darwin.py --target ' + ('ios27' if IOS27 else 'ios')) subprocess.run(['python3', ROOT / 'scripts/gen-turnip-smoke-shaders.py'], check=True) # Turnip's first submission, recorded on the host, for the plugin's prefix replay (build/turnip-smoke/TurnipReplay.h). -subprocess.run(['python3', ROOT / 'scripts/gen-turnip-replay.py'], check=True) +if args.no_replay: + (ROOT / 'build/turnip-smoke/TurnipReplay.h').unlink(missing_ok=True) +else: + subprocess.run(['python3', ROOT / 'scripts/gen-turnip-replay.py'], check=True) BRIDGE_TARGET = 'ios27' if IOS27 else 'ios' subprocess.run(['python3', ROOT / 'scripts/build-adreno-metal-bridge.py', '--target', BRIDGE_TARGET], check=True) BRIDGE = ROOT / 'build/adreno-metal-bridge' / BRIDGE_TARGET / 'libadreno_metal_bridge.a' diff --git a/scripts/build-adreno740-accelerator.py b/scripts/build-adreno740-accelerator.py index 061101d..f1f64be 100644 --- a/scripts/build-adreno740-accelerator.py +++ b/scripts/build-adreno740-accelerator.py @@ -92,11 +92,9 @@ sdk = run(['xcrun', '--sdk', 'iphoneos', '--show-sdk-path'], text=True).strip() binding_objects = [OUT / 'original-bindings.o'] if FILESET: # As for the Legacy display (build-gop-legacy-display-27.py): original functions are reached through owned stubs in kernel text - # (signed and moved like native code) whose branch lands on the original after the collection slide, less the measured move of the - # native kernel (native-slide); the metaclass is data, reached through the GOT at its plain address. + # (signed and moved like native code). XnuPrelink retargets their branches after placing native text. + # The metaclass is data, reached through the GOT at its fixed collection address. slide = int(plan['collection_slide'], 16) - native_slide_file = ROOT / 'build/native-legacy-display/native-slide' - native_slide = int(native_slide_file.read_text(), 16) if native_slide_file.exists() else 0 text, addresses = '.text\n.p2align 2\n', '' for n, a in sorted(bindings.items()): if n not in undefined: @@ -104,7 +102,7 @@ if FILESET: if n.endswith('gMetaClassE'): addresses += f'.globl {n}\n.set {n}, {a + slide:#x}\n' else: - addresses += f'.globl {n}.original\n.set {n}.original, {(a + slide - native_slide) & ((1 << 64) - 1):#x}\n' + addresses += f'.globl {n}.original\n.set {n}.original, {a + slide:#x}\n' text += f'.globl {n}\n{n}:\n bti c\n b {n}.original\n' asm.write_text(text) (OUT / 'original-addresses.S').write_text(addresses) diff --git a/scripts/build-boot-runtime.py b/scripts/build-boot-runtime.py new file mode 100644 index 0000000..3bf7d84 --- /dev/null +++ b/scripts/build-boot-runtime.py @@ -0,0 +1,177 @@ +#!/usr/bin/env python3 +"""Build payload runtime inputs from a pinned IPSW and the selected sources. + +Invoked by unixnu.runtime with a JSON request; no device ID or private binary is +stored in source control. Existing Data seeds are copied into no writable mount. +""" +import argparse +import contextlib +import json +import os +from pathlib import Path +import plistlib +import re +import struct +import subprocess +import sys +import tempfile +import zlib + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / 'scripts')) +from mesa_patch_identity import validate_recorded_patches + + +def run(*command, **kwargs): + subprocess.run([str(part) for part in command], check=True, **kwargs) + + +def script(name, *args, **kwargs): + run(sys.executable, ROOT / 'scripts' / name, *args, **kwargs) + + +@contextlib.contextmanager +def os_mount(image, work): + # A developer may already have this exact OS mounted for inspection. Reuse + # only a read-only attachment, and leave its lifetime with its owner. + info = plistlib.loads(subprocess.check_output(['hdiutil', 'info', '-plist'])) + for entry in info.get('images', []): + if Path(entry.get('image-path', '')).resolve() != Path(image).resolve(): + continue + if entry.get('writeable', True): + raise SystemExit('The IPSW OS image is already attached writable; require an unchanged read-only input') + mounts = [Path(item['mount-point']) for item in entry.get('system-entities', []) + if item.get('mount-point') and (Path(item['mount-point']) / 'sbin/launchd').is_file()] + if len(mounts) == 1: + yield mounts[0] + return + mount = work / 'os-mount' + mount.mkdir() + attached = plistlib.loads(subprocess.check_output(['hdiutil', 'attach', '-readonly', '-nobrowse', + '-mountpoint', str(mount), '-plist', str(image)])) + device = attached['system-entities'][0]['dev-entry'] + try: + yield mount + finally: + run('hdiutil', 'detach', device) + + +def checkout(path, pin): + if not path.exists(): + path.mkdir(parents=True) + run('git', 'init', '-q', path) + run('git', '-C', path, 'fetch', '--depth=1', pin.get('mirror', pin['repository']), pin['commit']) + run('git', '-C', path, 'checkout', '--detach', 'FETCH_HEAD') + return True + revision = subprocess.check_output(['git', '-C', str(path), 'rev-parse', 'HEAD'], text=True).strip() + if revision != pin['commit']: + raise SystemExit(f'{path}: source revision differs from its pin; preserve and inspect the checkout') + return False + + +def build_plugin(target): + pins = json.loads((ROOT / 'scripts/data/adreno-compiler-inputs.json').read_text()) + mesa = ROOT / '.cache/mesa-work' + fresh = checkout(mesa, pins['mesa']) + if fresh: + for patch in pins['patches']: + run('git', '-C', mesa, 'apply', ROOT / patch) + validate_recorded_patches(ROOT, mesa, pins) + translator = json.loads((ROOT / 'scripts/data/gpu-metal-vulkan-inputs.json').read_text())['translator'] + checkout(ROOT / '.cache/gpu-reference/metal2vulkan', translator) + venv = ROOT / '.cache/mesa-venv' + if not (venv / 'bin/python').is_file(): + run(sys.executable, '-m', 'venv', venv) + run(venv / 'bin/python', '-m', 'pip', 'install', '-r', ROOT / 'scripts/requirements-adreno-compiler.txt') + script('build-turnip-darwin.py', '--target', target) + # Replay is a diagnostic capture from a host GPU simulation, not a runtime + # dependency. Do not require unrelated test fixtures in a release build. + script('build-adreno-metal-plugin.py', '--target', target, '--no-replay') + return ROOT / 'build' / ('adreno-metal-plugin27' if target == 'ios27' else 'adreno-metal-plugin') + + +def addon_hashes(path, work): + data = path.read_bytes() + if data[:8] != b'NGPUADD2' or len(data) < 24 or struct.unpack_from(' allowed: + raise Error(f'prelink regression: {kind} {owner} original {original}, {count} sites (allowed {allowed})') + branches = {entry['symbol'] for entry in record.get('native_original_branches', [])} + if set(policy.get('required_original_branches', [])) - branches: + raise Error('prelink lacks required absolute adapter branch relocations') + + def prelink(ctx): """Produce out//kernel/kernel.macho from the XNU build and the original kernelcache.""" spec = ctx.board.get('kernel', {}).get('prelink') @@ -277,6 +292,11 @@ def prelink(ctx): run(command + ['--output', candidate], cwd=ctx.tree, log_file=ctx.logs / 'prelink.log') if not candidate.is_file(): raise Error('prelink completed without writing a kernel candidate') + if spec.get('regression_policy'): + policy = json.loads((ctx.tree / spec['regression_policy']).read_text()) + if policy['kernelcache_sha256'] != sha256_file(cache): + raise Error('prelink regression policy names another kernelcache') + check_prelink_regressions(json.loads(candidate.with_suffix('.json').read_text()), policy) success = True finally: if candidate_map.is_file(): diff --git a/scripts/unixnu/payload.py b/scripts/unixnu/payload.py index d7a9867..0774c8e 100644 --- a/scripts/unixnu/payload.py +++ b/scripts/unixnu/payload.py @@ -38,6 +38,21 @@ def check_args(arguments): return line +def check_data_input(ctx, data): + """An owner-Data kernel needs a seed even when it resumes a persistent overlay.""" + if data: + return + from .config import parse_config + config = ctx.config + if not ctx.config_path.is_file() and ctx.config_name: + selected = ctx.device / 'configs' / ctx.config_name + if selected.is_file(): + config = parse_config(selected.read_text()) + if config.get('XNU_OWNER_DATA_MOUNT'): + raise Error('CONFIG_XNU_OWNER_DATA_MOUNT requires --data-ramdisk FILE (or [build] data_ramdisk); ' + 'persistent Data still needs its original seed to publish the volume') + + def check_native_root(kernel, arguments, vmm_present=False): """Refuse the known bare-metal AMFI panic before the volume is written. @@ -111,6 +126,7 @@ def build(ctx, kernel=None): if allowed and not owner_trust and sha256_file(trust) not in allowed: raise Error(f'{trust} is not one of the trust caches this board accepts (board.toml [payload] trust_cache_sha256)') data = expand(args.data_ramdisk) if getattr(args, 'data_ramdisk', None) else None + check_data_input(ctx, data) if data and not data.is_file(): raise Error(f'Data ramdisk not found: {data}') if getattr(args, 'skip_setup', False): diff --git a/scripts/unixnu/pipeline.py b/scripts/unixnu/pipeline.py index c0ebd07..fbeac31 100644 --- a/scripts/unixnu/pipeline.py +++ b/scripts/unixnu/pipeline.py @@ -38,6 +38,11 @@ def write_manifest(ctx, modules, patch_order, files): 'revisions': revisions, 'patches': [str(path) for path in patch_order], 'kernel_command_line_bytes': len(command_line(ctx)), 'outputs': {name: {'bytes': path.stat().st_size, 'sha256': sha256_file(path)} for name, path in files.items()}} + runtime = ctx.work / 'runtime/inputs.json' + if runtime.is_file(): + inputs = json.loads(runtime.read_text()) + manifest['runtime'] = {'data_volume_uuid': inputs['data_volume_uuid'], + 'inputs': {key: value['sha256'] for key, value in inputs['inputs'].items()}} target = ctx.work / 'package.manifest.json' target.write_text(json.dumps(manifest, indent=2, sort_keys=True) + '\n') return target diff --git a/scripts/unixnu/runtime.py b/scripts/unixnu/runtime.py new file mode 100644 index 0000000..1edb0fd --- /dev/null +++ b/scripts/unixnu/runtime.py @@ -0,0 +1,112 @@ +"""Prepare reproducible runtime inputs before compiling the kernel. + +The board supplies a builder. An existing Data seed is immutable: generating a +replacement is an explicit first-install operation, never a cache-miss fallback. +""" +import importlib.util +import json +import os +import re +import sys +import uuid +from pathlib import Path + +from .ipsw import Ipsw +from .util import Error, log, run, sha256_file + + +def seed_uuid(tree, path): + spec = importlib.util.spec_from_file_location('runtime_packed_ramdisk', tree / 'drivers/DarwinBoot/Test/packed_ramdisk.py') + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + _, extents = module.regions(path.read_bytes()) + identities = set() + for _, data in extents: + for at in range(0, len(data), 4096): + block = data[at:at + 4096] + if block[32:36] == b'APSB': + identities.add(str(uuid.UUID(bytes=bytes(block[240:256]))).upper()) + if len(identities) != 1: + raise Error(f'{path}: require a packed APFS seed with exactly one volume UUID') + return identities.pop() + + +def prepare(ctx, kernel_build=False): + spec = ctx.board.get('payload', {}).get('runtime') + if not spec or not spec.get('enabled', True): + from .payload import check_data_input + check_data_input(ctx, getattr(ctx.args, 'data_ramdisk', None)) + return + if getattr(ctx, '_runtime_prepared', False): + return + args = ctx.args + out = ctx.work / 'runtime' + out.mkdir(parents=True, exist_ok=True) + seed = Path(args.data_ramdisk).expanduser().resolve() if args.data_ramdisk else out / 'data.dbr' + init = bool(getattr(args, 'init_data', False)) + if init and args.data_ramdisk: + raise Error('--init-data cannot replace --data-ramdisk; use one first-install seed or reuse the original') + if init and not kernel_build: + raise Error('--init-data needs a kernel build to bind its new Data volume UUID') + if not seed.is_file() and not init: + raise Error('No preserved Data seed. Pass --data-ramdisk FILE for an existing installation, ' + 'or --init-data for a FIRST installation; a missing cache never regenerates Data automatically') + previous = out / 'inputs.json' + if not args.data_ramdisk and seed.is_file() and previous.is_file(): + expected = json.loads(previous.read_text())['inputs']['data_ramdisk']['sha256'] + if sha256_file(seed) != expected: + raise Error('Saved Data seed differs from its recorded SHA-256; restore the original installation seed') + owner = getattr(args, 'owner_trust_cache', None) + trust = owner or getattr(args, 'trust_cache', None) + addon = getattr(args, 'gpu_addon', None) + for name, path in [('Data seed', args.data_ramdisk), ('trust bundle', trust), ('GPU addon', addon)]: + if path and not Path(path).expanduser().is_file(): + raise Error(f'{name} not found: {path}') + # Preserve explicit inputs. A source-built addon must be paired with newly + # collected CDHashes, so an explicit trust bundle alone is not sufficient. + if trust and not addon: + raise Error('Pass both --gpu-addon and its --owner-trust-cache, or omit both to build them from source') + request = dict(spec, tree=str(ctx.tree), device=str(ctx.device), output=str(out), + data_ramdisk=str(seed), init_data=init, + gpu_addon=str(Path(addon).expanduser().resolve()) if addon else None, + trust_cache=str(Path(trust).expanduser().resolve()) if trust else None) + if not seed.is_file() or not trust or not addon: + source = Ipsw(ctx) + if not seed.is_file() or not addon: + request['os_image'] = str(source.disk_image('OS')) + if not trust: + request['trust_modules'] = [str(source.extract(name)) for name in spec['trust_components']] + request_path = out / 'request.json' + request_path.write_text(json.dumps(request, indent=2) + '\n') + report = out / 'generated.json' + report.unlink(missing_ok=True) + run([sys.executable, ctx.tree / spec['tool'], '--request', request_path], cwd=ctx.tree, + env=dict(os.environ, UNIXNU_DEVICE=str(ctx.device)), log_file=ctx.logs / 'runtime.log') + if not report.is_file(): + raise Error('runtime builder did not produce generated.json') + generated = json.loads(report.read_text()) + seed = Path(generated['data_ramdisk']) + trust, addon = generated['trust_cache'], generated['gpu_addon'] + owner = trust + identity = seed_uuid(ctx.tree, seed) + config = ctx.need_config() + if identity != config.get('XNU_OWNER_DATA_VOLUME_UUID'): + if not kernel_build: + raise Error('Data seed UUID differs from the compiled configuration; rebuild the kernel with this seed') + text = ctx.config_path.read_text() + text, count = re.subn(r'^CONFIG_XNU_OWNER_DATA_VOLUME_UUID=.*$', + f'CONFIG_XNU_OWNER_DATA_VOLUME_UUID="{identity}"', text, flags=re.M) + if count != 1: + raise Error('runtime Data requires exactly one CONFIG_XNU_OWNER_DATA_VOLUME_UUID') + ctx.config_path.write_text(text) + ctx.need_config() + args.data_ramdisk, args.gpu_addon = str(seed), str(addon) + if owner: + args.owner_trust_cache = str(trust) + else: + args.trust_cache = str(trust) + inputs = {name: {'path': str(path), 'sha256': sha256_file(Path(path))} + for name, path in [('data_ramdisk', seed), ('trust_cache', trust), ('gpu_addon', addon)]} + (out / 'inputs.json').write_text(json.dumps(dict(inputs=inputs, data_volume_uuid=identity), indent=2) + '\n') + log(f'runtime: Data UUID {identity}; preserved seed sha256 {inputs["data_ramdisk"]["sha256"]}') + ctx._runtime_prepared = True