ASan (Linux CI) caught a heap-use-after-free in
JobSystemThreadPool::releaseDependents (voxel.world.cross_chunk_seam_culled):
completeJob set completionDone=true and released the mutex, then iterated
job->dependents. A waiter could acquire the mutex in between, see
completionDone, return from wait() and delete the heap job (owned by the
caller), freeing the dependents vector mid-iteration.
Move releaseDependents inside the same critical section and set
completionDone last, so no waiter can observe completion before the
dependents are released. Add a stress regression test that waits on a
completed child and deletes it immediately, 200 iterations.
- EVENGINE_ENABLE_STRICT_WARNINGS option: MSVC /W4 /WX, GCC/Clang
-Wall -Wextra -Werror, with documented noise suppressions and
-isystem/-external:I for third-party headers; enabled on the
Windows/Linux debug CI jobs.
- Fix warnings across engine, modules and tests exposed by the gate,
including real bugs (incomplete-type deletes, Timespan microsecond
overflow, overload hiding, enum switch gaps, dead code).
- Add Linux ASan+UBSan job, example smoke step, python-tools test
job, ruff config (scripts+tools), weekly Linux coverage job and a
CodeQL workflow.
Reproduced the intermittent CI segfault (different voxel.render tests each
run, ~1/80 processes under concurrent load) locally and traced it to a null
std::function call in JobSystemThreadPool::fireCompletion.
Root cause: completeJob released dependents (making a parallelFor join
runnable) before the child decremented its own counters and finished
touching its memory. The join could then complete on another thread and,
for heap jobs, be deleted by the caller — freeing the child while its
worker was still inside completeJob's tail; for frame jobs the same gap
let waitFrameJobs observe outstandingFrame == 0 and reset the arena early.
Both paths read freed memory, which manifested as the empty function call.
Fix:
- Add a per-State 'completing' count pinned around the whole completeJob
tail; waitFrameJobs refuses to reset the arena while it is > 0.
- Mark each job completionDone under the mutex before releasing
dependents, and waitJob()/isDone() now wait on completionDone rather
than the status flag, so callers deleting a join never free a child
still being completed.
Verified: pre-fix build crashed within ~25 rounds of a 4-way concurrent
voxel.render loop; post-fix survived 195 rounds (~780 process runs) with
zero crashes, and the full Linux ctest suite passes 1654/1654.
Stage D (shadow part): each in-flight slot's FrameGraph now owns both the
shadow and G-buffer passes as declarative, dependency-free passes in one
layer. The CSM array is imported per cascade via its layer view with
arrayLayers=3 so barriers cover the whole array; every cascade ends in
DepthStencilReadOnlyOptimal for the engine-side forward pass to sample.
- recordPendingShadowPasses / recordPendingGBufferPass are gone, merged
into recordDeferredFrameGraph(): graph record + submit before the
swapchain pass, with a once-per-slot-per-frame guard so repeated
render3D() calls in one script frame never re-record the same command
buffers (resetting in-flight CBs hung the driver).
- buildDeferredFrameGraphs() rebuilds one graph per slot from the current
shadow array + G-buffer targets; shadows can be enabled without a
G-buffer pass and vice versa (graphs are built on demand).
- JobSystem fix (PR #134 scheduler bug): completeJob now decrements
outstanding/outstandingFrame only after fireCompletion +
releaseDependents have finished using the job. Previously a waiter
(beginFrame/endFrame) could observe outstandingFrame == 0 and reset the
per-frame arena while a worker still touched the job (use-after-free:
lost forked job -> hang, or crash). Two stress tests
(render_graph.jobSystemGroupStress, render_graph.jobSystemEngineFramePattern)
cover the engine's exact beginFrame/parallel_for/task-group frame
pattern.
- Recording stays serial (graph->record()) for now: the JobSystem
help-execution path still has a residual timing-sensitive race that can
leave a worker stuck in a pass callback. The parallel executor
(jobSystemPassExecutor) is ready and CPU-tested; enable it once that
race is root-caused upstream (documented in FrameGraphJobs.h).
Verified locally (win32-debug, full clean rebuild): render_graph.*,
thread.*, Shadow3D.*, RenderSystem3D.*, Lighting3D.*, Mesh.* and
AO/AA/Outline/GI — 77 cases pass.
- JobSystem abstraction over the existing ThreadPool worker pool; the
factory in JobSystem.cpp is the single backend switch point (TBB later).
- Job carries a dependency count and completion callback so a render graph
can be wired on top without rewriting the scheduler.
- parallel_for(first, last, body, chunk) with static partitioning, auto
chunk sizing, and a join handle; task_group provides fork/join with
help-execution so workers can wait without deadlocking.
- Per-frame arena (beginFrame/endFrame) for frame-scoped jobs: no per-frame
new/delete; frame jobs are recycled by the arena.
- Thread module exposes a shared engine JobSystem (getJobSystem).
- VoxelWorld::remeshDirty now uses parallel_for instead of raw std::thread.
- 24 new zeroerr tests in test/thread.cpp; verified with a standalone MSVC
harness (24 cases x 30 runs, no flakes).
Normalize all header comments to Doxygen @brief form and fill in missing class/method docs across engine common, cmdline, and module headers (~2368 @brief entries). Add root Doxyfile, 'make docs' target, CMake docs target, .gitignore for docs/api, README/docs/dev updates, and a Windows (native) + code-conventions section in AGENTS.md.
Introduces eve::cap in common/Capability.h: a name-keyed registry where the
consumer owns the interface, the provider registers an implementation, and
neither includes the other. It generalises what common/RenderTrace.h already
does for the render tracer, and supports both a single provider (provide/query)
and many ordered ones (addListener/forEach), keyed by string so a provider
compiled into a plugin lands in the same slot as the host.
Two upward dependencies go away:
system -> graphics IGpuInfo. The Vulkan physical-device fields move into
graphics/vulkan/GpuInfoProvider.cpp, with a WebGPU
counterpart, so System.cpp loses both the graphics
include and its EVENGINE_WEBGPU branches.
thread -> event IMainThreadPost. postMain and submitPost now enqueue
through the capability. The interface makes explicit
what was previously only a comment in submitPost: the
submitting thread calls prepare() so the worker never
reaches the module registry.
Both now report an error when no provider is linked, rather than lazily
creating a module the build may not contain.
Step S3 of docs/dev/模块编排与裁剪架构.md; back-edge whitelist 16 -> 14.
Co-authored-by: 西风逍遥游 <sunxfancy@gmail.com>
- Updated ThreadPool destructor to handle exceptions during stop() gracefully, ensuring no unhandled exceptions disrupt the program.
- Improved stop() method to detach threads that are currently executing, preventing potential deadlocks when stopping the thread pool.
- Adjusted worker thread joining logic to ensure proper cleanup and resource management.
Additionally, modified occupancy threshold in RenderImageAudit to improve detection of incomplete frames, ensuring better handling of sparse frames.
- Updated the math module to include vector and matrix operations, noise generation, Bezier curves, and random number utilities.
- Introduced the tensor module, providing TensorFlow 2-style eager execution and graph building capabilities.
- Enhanced documentation to reflect the new features in the math and tensor modules, including usage examples and API references.
- Modified CMakeLists.txt to include the new math and tensor modules, ensuring proper build configurations.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Added the system module to provide OS, CPU, RAM, wall clock, sleep, clipboard, battery, and GPU information.
- Enhanced the thread module to support thread pools, tasks, and named channels for inter-thread communication.
- Updated documentation to reflect the new features in the system and thread modules, including usage examples and API references.
- Modified CMakeLists.txt to include the new system and thread modules, ensuring proper build configurations.
- Updated load.nut script to initialize the new system and thread functionalities.
Co-authored-by: Cursor <cursoragent@cursor.com>
西风逍遥游