main
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 西风逍遥游
|
31880f0021 |
feat(examples): gate examples in CI and make repository scripts UTF-8 on every host
`examples/<name>/` was a tested surface in name only. The source-quality job had no examples check at all, and scripts/smoke_examples.sh accepted any process that had not printed one of its error markers: a project that died during startup, exited after a single frame, or crashed without a marker was reported as PASS. Nothing stopped a malformed or unregistered example from reaching dev, which is how 14 examples arrived without READMEs and 20 were never registered in the overview table. Source side - scripts/check_examples.py classifies every directory under examples/ as either runnable (main.nut plus a config.nut that assigns the engine `config` table) or as a reviewed non-runnable sample listed in NON_RUNNABLE_EXAMPLES with a reason. Both kinds must own a README.md and appear in examples/README.md. Stale exemptions and overview links to missing directories fail as well. - `make check/examples` and a new source-quality CI step run the gate. - scripts/tests/test_check_examples.py covers it and pins each failure mode against fixtures. Runtime side - scripts/smoke_examples.sh now enforces liveness, not just marker absence: every runnable example must still be running after MIN_RUN_SECONDS (default 2s; CI passes 2s), and a process that dies inside that window fails whatever its exit code. A clean self-exit after the window passes with an explicit note instead of being silently accepted as a pass. - The Linux CI step sets MIN_RUN_SECONDS=2 and keeps RUN_SECONDS=6, so the job's wall clock is unchanged (13.7 min of its 20 min budget on dev). Windows / GBK - scripts/utf8_stdio.py re-encodes stdout/stderr as UTF-8 and every scripts/*.py program calls it as the first statement of its __main__ block. On a Chinese Windows a redirected stream falls back to the ANSI code page (cp936), and printing a character that code page cannot encode (arrows, superscripts, bullet separators, emoji - all common in this repository) raised UnicodeEncodeError and turned a checker into a traceback. - The mirror image is input: read_text() without encoding= decoded UTF-8 repository text as cp936 and raised UnicodeDecodeError. - scripts/tests/test_utf8_stdio.py enforces both conventions for every script and test module; text file I/O and child-process decoding now pass encoding="utf-8" explicitly. - Unit-test invocations run with `python3 -X utf8` so the runner's own output never follows the host code page. Violations the new gates found on top of dev, fixed here: 14 examples shipped without a README (fluid3d, mesh-impact-lab, mesh-modifier-lab, spline-tube-lab on the earlier dev tip, plus pcg-depth-of-field, pcg-detail-overwrite, pcg-layer-culling, pcg-location-system, pcg-runtime-orchestration, pcg-scene-player-culling, pcg-snow-wind, pcg-thunder-strike, pcg-ui-scaler and pcg-underwater added by #397), 20 were missing from the overview table (those 14 plus archspace-editor, geometry-stroke-lab, interior-mapping, layered-animation, triplanar-decal, terrain-stamping), and scripts/export_meshret_onnx.py had no UTF-8 stdio guard. Verified locally: `python -m ruff check scripts` (All checks passed); scripts/check_test_manifest.py; scripts/check_quality_metadata.py; scripts/check_architecture_contracts.py --base HEAD; scripts/check_nodiscard.py; `make -n check/examples`; scripts/check_examples.py (125 examples: 118 runnable, 7 reviewed non-runnable); and `python -X utf8 -m unittest discover -s scripts/tests -p "test_*.py"` (155 tests, OK). The runtime half of the contract needs a Linux headless build, so it is verified by the CI examples smoke step rather than locally. |
||
|
西风逍遥游
|
cff9bb4d0c |
tensor: AITemplate-style compile pipeline, gpgpu Sequence, mini-LLM demo and weight-only quantization
Refactor the tensor module around AITemplate-style static compilation: - Optimizer: DCE, constant folding, elementwise chain fusion, matmul/conv bias+activation epilogue fusion, liveness-based static memory planning, topological group execution order. - KernelGen: shape-baked GLSL per fused group; two-pass softmax/layernorm; shared-memory SDPA; naive + 16x16 tiled matmul templates. - GpuBackend: arena buffer allocation per memory plan, matmul autotuning, single command-buffer submission via the new gpgpu::Sequence (uploads, dispatches and downloads recorded once per run, Kompute-style). - shaderc in-process GLSL->SPIR-V on Windows (no glslc.exe dependency), with glslc spawn fallback. - CPU reference kernels shared by eager and the compiled interpreter; no-Vulkan fallback keeps compiled graphs runnable headless. Neural-network coverage: softmax/logsoftmax, layernorm/rmsnorm (with weight/bias), gelu/silu, conv1d/conv2d, pool, embedding, concat/slice, axis reduces, argmax, cast, fused SDPA (+masked), resize2d, batched matmul, seeded random. Weight-only quantization (activations stay fp32): - fp16 (IEEE half), fp8 (e4m3), fp4 (e2m1), int8, int4 with per-group block scales (MXFP-style: scale = maxAbs / format max magnitude). - Quantized weights capture into graph Const nodes; CPU interpreter dequantizes at materialization; GPU matmul/embedding kernels dequantize inline from packed uint buffers (naive matmul variant). - TF::quantizeWeight API + script binding. Mini-LLM validation (TinyStories-gpt2-3M, 8 layers d=64): - test/tensor_llm.cpp builds the full GPT-2 graph from engine ops and matches numpy reference: probs within 3e-7, greedy generation 24/24 identical (~100 ms/token CPU), compiled path exact. - tensor.llm.quantizedDtypes: fp16/int8/fp8/int4 all keep 16/16 greedy top-1 (fp4 16/16 with proper block scaling), weights 27.6 MB -> 3.3-13 MB. - scripts/export_tiny_gpt2.py converts a PyTorch .bin checkpoint (no torch dependency) plus GPT-2 tokenizer into engine assets + numpy reference. Tests: gpgpu 8/8, tensor 31/31 (original suite) + tensor.llm.* + tensor.quant.* Assets in test/assets/mini_llm are gitignored (regenerate via the script). |