145 lines
31 KiB
Markdown
145 lines
31 KiB
Markdown
# WLocate implementation record
|
||
|
||
## Feature — configurable global window hotkey, 2026-10-08
|
||
|
||
- Added native “工具 → 快捷键设置” dialog and the same entry in the tray context menu. Users can choose Ctrl/Alt/Shift/Win modifiers and letters, digits, function/navigation/punctuation/numpad keys, disable the shortcut, or restore the Ctrl+Alt+L default. Changes apply on OK and persist immediately; Cancel/default preview do not alter active settings. Existing installations retain the old default until changed.
|
||
- `gui_hotkeys.hpp` owns registration and one `[Keyboard] ShowWindow` INI value for enabled/modifiers/key. A replacement is registered under an alternate ID before releasing the active registration; occupied/reserved combinations or a failed settings write leave the old binding and saved value intact. Disable unregisters, restart loads saved preferences, malformed settings fall back to the default. Startup registration failures are shown in the tray notification/tooltip and settings dialog. Tray tooltip/About show the configured shortcut instead of hard-coded text. WM_HOTKEY validates current ID and chord and brings an already-open modal dialog forward.
|
||
- Native dialog uses the shared UTF-8 resource build and Windows dialog font/DPI scaling. Bare typing keys require Ctrl/Alt/Win; function keys can stand alone. F12 is excluded per Windows' documented debugger reservation. References: https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-registerhotkey and https://learn.microsoft.com/en-us/windows/win32/inputdev/wm-hotkey .
|
||
- Added 32 checks using actual RegisterHotKey ownership and a separate temporary INI: default/custom/disabled reload, invalid configuration, switching/releasing keys, unchanged selection, conflicts, failed-save rollback, startup failure and retry, dialog default/cancel/OK/disabled controls, inline conflict message. Tests reserve unused Ctrl+Alt+Shift+F13–F24 combinations and send no keyboard input to the desktop. Default UI integration tests no longer try to register the user's global shortcut. Dialog/conflict renders inspected at `test-output/hotkey-ui/hotkey-settings.png` and `hotkey-conflict.png`.
|
||
- Release build and all four CTest targets pass. Program Files/PATH executables, installed README and release ZIP refreshed; verified hashes in `test-output/installation-hotkey.json`. User's chosen settings are preserved; GUI restarted hidden, retaining the configured/default global shortcut.
|
||
|
||
## Fix — relative literal path fragment planning, 2026-10-08
|
||
|
||
- User reported `locate.old.exe foo\bar` was much faster. Reproduced with identical 17 paths: old wrapper 252.5 ms, native CLI 3779.2 ms (3765.1 ms service), GUI-equivalent uncached query 4866.1 ms service. The previous absolute-glob bound could not apply to literal relative fragments, which still reconstructed every indexed full path.
|
||
- `Query::path_literals()` chooses a necessary positive literal path term in every OR branch. `Volume::find_literals()` uses the last component as a name-prefix candidate range (exact component for a trailing separator), verifies the preceding path suffix, and walks matching directory descendants with row deduplication. A lone `foo\` uses Bloom-assisted name-suffix filtering. Actual predicates still enforce case, whole word, OR/AND, negation and metadata. Root/drive fragments, Unicode, partial first/last components, hardlink rows, late unique names and overlapping seed subtrees retain previous semantics. General matching remains the fallback when a branch is unbounded. Existing absolute-glob planning stays ahead of this path.
|
||
- No persistent index or snapshot format changes: reuse sorted names, new-name delta chains and parent/child relationships. Temporary row visitation bytes are allocated only when candidate seeds exist. Directory/name generation invalidates cached results as before.
|
||
- Added exhaustive full-path oracle comparisons across relative/absolute literals, slash forms, trailing separators, case collisions, Unicode, whole-word/OR/metadata, multi-pattern CLI, ordering, count-only, limits and GUI pagination. Generated fragments cross arbitrary component boundaries. Mutation checks cover directory moves/renames, changed hardlinks, orphan/late parents, deletions, compaction and snapshot restore. Bounded-work checks use 30000 unrelated names. All three CTest targets pass: 2386 core checks, existing 30 GUI cache and 16 icon cache checks.
|
||
- Extended live integration to compare all three query paths (name, absolute glob subtree, relative literal subtree) after create/rename/move/delete, 30 hardlinks, metadata changes, long Unicode paths and 500-file bursts; all passed. Logs: `test-output/build-relative-path.log`, `ctest-relative-path.log`, `integration-relative-path.log`.
|
||
- Deployed service benchmark: `foo\bar` native median 17.2717 ms total / 1.193 ms service, old wrapper 248.9998 ms total; uncached GUI-equivalent 1.374 ms service. `Windows\Logs`: 18.8648 vs 218.8839 ms, 364 identical paths; trailing slash: 14.6512 vs 247.6539 ms, 363 paths; `Windows\System32\drivers\etc`: 17.5668 vs 235.9423 ms, 7 paths. All four complete result sets agree among old/native/GUI. Four runs with first excluded, warm resident index; no GUI rendering/debounce included. Data and reusable script: `test-output/relative-path-{before,after}.json`, `tests/relative_path_benchmark.py`.
|
||
- Installed Program Files/PATH binaries, updated installed performance notes and release ZIP, verified hashes in `test-output/installation-relative-path.json`. GUI restarted hidden on `foo\bar` (Ctrl+Alt+L restores it). Original `locate.old.exe` unchanged. Prior delivery hashes and timings below are historical.
|
||
|
||
## Fix — anchored path glob subtree planning, 2026-10-07 22:51
|
||
|
||
- Reproduced `C:\Windows\Logs\*.log`: previous native CLI median 2598.9 ms end-to-end (2581.8 ms service), versus legacy wrapper 152.8 ms, identical 62 paths. GUI-equivalent uncached name-sorted query took 3638.2 ms inside the service. Path glob terms were Unknown during name evaluation, so the engine reconstructed/matched paths across both complete volumes.
|
||
- Query planning now derives a conservative absolute directory prefix for every positive OR branch, folds/merges overlapping bounds, and falls back to general scanning if any branch is unbounded. `Volume` resolves those prefixes and walks child chains keyed by parent NTFS file reference. Case-colliding directory names are all considered; original predicates still enforce case, glob, metadata, AND/OR and negation. Only matching subtree rows are sorted for GUI name order. Existing wildcard semantics include nested descendants.
|
||
- Four-byte-per-row sibling links plus one hash head per parent are maintained on insertion, deletion, hardlink replacement and directory moves; directory renames retain descendant references. `finish()` rebuilds chains after snapshot restore/compaction. No disk-format change or full reindex is required.
|
||
- Expanded core tests from 86 to 442 checks: exhaustive-path oracle across glob/case/slash/Unicode/OR/negation/sorting variants, limits/cancellation, hardlinks and mutations, late parents, snapshot/compaction, cross-volume ordering and GUI pagination. A bounded-work regression includes 30000 unrelated names. All three CTest targets pass (442 core + 30 GUI cache + 16 icon cache). Live integration now compares both name and subtree queries and passed with 30 hardlinks, rename/move/delete, long Unicode paths and a 500-file create burst.
|
||
- After installing the service: exact-query native CLI median 14.6785 ms total, 0.486 ms service; legacy wrapper 160.0808 ms total. GUI-equivalent query with its completed-result cache cleared before each run: 0.571 ms service. All 62 paths match across old/native/GUI. Five measured runs after one warmup; index already resident. Scripts/data: `tests/path_benchmark.py`, `test-output/path-before.json`, `path-after.json`, `integration-path-subtree.log`, `ctest-path-subtree.log`. These figures exclude actual GUI input debounce/rendering and are not a general Everything-engine comparison.
|
||
- Program Files/PATH executables and release ZIP refreshed; deployment hashes recorded in `test-output/installation-path-subtree.json`. GUI restarted hidden on the user's path query to avoid taking keyboard focus; Ctrl+Alt+L restores it. Previous executable hashes below are historical.
|
||
|
||
## Fix — actual file icons and full-path cache, 2026-10-07
|
||
|
||
- User observed that all three `Figma.exe` results stayed generic even when left idle. Confirmed in the installed UI. Previous async implementation only requested generic extension icons (`file.exe` with SHGFI_USEFILEATTRIBUTES); it never requested actual executable resources, so waiting could not change them. This was an incomplete follow-up load, not a missing repaint notification.
|
||
- `AsyncIcons` now has two layers: generic type icons provide the immediate fallback; the same dedicated COM worker loads actual file icons using the complete indexed path without SHGFI_USEFILEATTRIBUTES. Completion posts to the existing GUI handler and repaints automatically. File/path lookups remain off the GUI and search threads.
|
||
- Added a 4096-entry LRU cache keyed by full path, separate from type entries. Pending requests are deduplicated. Modification/creation timestamps, size and attributes invalidate a file entry; revision numbers reject outdated completions. Failed loads retain the generic fallback and retry after five seconds instead of poisoning the cache permanently or retrying every paint.
|
||
- Added 16 icon-cache checks covering fallback-to-file replacement, identical basename/extension in different directories, reuse, metadata invalidation, stale completion rejection, failure throttling/recovery, bounded eviction, and a real embedded EXE icon. Existing 30 GUI checks and 86 core checks also pass. Full UI integration/typing replay passed with a 250 ms maximum heartbeat gap on the 200 ms timer.
|
||
- Added `wlocate_ui_tests.exe <output> --file-icons` for the user's exact query. All three Figma paths loaded actual icons (system image indices 50/51/52 in that test process) without further input. Render inspected: `test-output/ui-figma-icons/file-icons.png`; report: `test-output/ui-figma-icons/report.txt`. File-specific Figma logos are visible on every row.
|
||
- Rebuilt/installed Program Files and PATH copies and refreshed the release ZIP; hashes are in `test-output/installation-file-icons.json`. GUI restarted hidden on `figma.exe` with its tray/hotkey available to avoid taking keyboard focus. Earlier icon/installation notes below are historical.
|
||
|
||
## Fix — input stalls with broad/intermediate queries, 2026-10-07
|
||
|
||
- User reported a several-second freeze while typing `python*`, before entering the final `n`. Isolated timings on the pre-fix controller: `pytho` service query ~84 ms and result publication ~7 ms; empty query ~0.93 s and `p` ~0.61 s on the worker. Those timings alone did not reproduce the reported UI stall (`test-output/gui-input-profile-before.txt`).
|
||
- Captured two installed-GUI stalls with CDB: the UI thread was inside `UIAutomationCore` bulk traversal of the stock owner-data ListView proxy. Intermediate `p` has ~1.61 million results, empty search ~3.27 million; the proxy walks every row and subitem. This also affected computer-use observation/input and could leave the application unresponsive for minutes. Stack captures: `gui-input-stack-before.txt`, `gui-input-stack-new.txt`. Do not attribute the user's original transient exclusively to icon loading; the directly observed long stall was UI Automation traversal.
|
||
- Added `gui_accessibility.hpp`: a server-side UIA fragment provider exposes visible rows only, with ItemContainer/VirtualizedItem for offscreen discovery and realization, native selection, focus, scroll-to-item and invoke. Row identities expire on query changes; detached providers cannot access destroyed windows. Calls use COM STA threading, keeping controls/cache access on the GUI thread. Native header accessibility remains available. A large select-all can be iterated through ItemContainer's IsSelected property; GetSelection avoids constructing arrays above 4096 elements. Reference: https://learn.microsoft.com/en-us/windows/win32/winauto/uiauto-workingwithvirtualizeditems
|
||
- Removed synchronous Shell icon lookup from startup/painting. `gui_icons.hpp` resolves generic type icons on a separate COM thread, deduplicates/cache requests, and posts completion to the GUI. It uses type names rather than probing indexed file paths. The worker owns detached shared state, so even a stuck Shell call cannot block search or window shutdown. Shared system image lists now use LVS_SHAREIMAGELISTS. Reference: https://learn.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-shgetfileinfow
|
||
- Text changes now immediately cancel/retire the old query; only launching the newest query is debounced. Late replies and missing-page requests cannot repaint results for previous text during the debounce interval.
|
||
- Expanded the existing cache regression target to 30 checks: controlled blocked icon provider, input/cancel/shutdown, visible-row accessibility over 3,000,000 rows, real UI Automation FindAll, offscreen realization/selection, and provider invalidation/lifetime. Real UIA traversal visited 16 visible rows in ~47 ms on five consecutive runs. Core tests (86 checks), CTest and UI integration pass. Typing replay `p -> py -> pyt -> pyth -> pytho -> python -> python*` had a maximum 204 ms heartbeat gap on a 200 ms timer. Reports: `test-output/gui-input-regression-final.txt`, `ui-input-uia-final/report.txt`.
|
||
- During development one test process exited with 0xC0000409; it did not recur under CDB, in five subsequent runs, CTest, or final GUI integration. The earlier same-process UIA test used Sleep polling and exposed costly cross-thread control calls; final implementation uses STA marshalling and the test pumps messages without polling delay.
|
||
- Updated Program Files/PATH binaries and the release ZIP. GUI restarted hidden on `python*` with its tray icon/hotkey available, because live desktop input was detected during verification. Current final hashes are recorded in `test-output/installation-input-fix.json` and the package SHA256SUMS; previous hashes below are historical.
|
||
|
||
## Fix — repeated GUI queries after dismissing a context menu, 2026-10-07
|
||
|
||
- Reproduced the user's exact `figma.exe` scenario in the installed GUI with computer-use after its native pipe recovered: right-click the last of three results, dismiss onto blank list space, then move/click in the blank area. The row retained keyboard focus, and the status bar updated continuously. GUI CPU use was about 2.59 CPU seconds over a 3-second sample.
|
||
- Root cause: `receive()` removed the pending request and cleared page zero, then called `ListView_SetItemCountEx` before publishing the new page. The real Windows owner-data ListView synchronously requested the focused row (`LVN_GETDISPINFO`), found no cached data and queued page zero again. Each completion repeated the same cycle. Publish the page before notifying the control. Cache hints now reject negative/reversed/out-of-range intervals and clamp prefetch to the actual result count.
|
||
- Added `tests/gui_cache_tests.cpp` and the `gui_cache` CTest target. It exercises the production controller with actual common controls and deterministic replies, without a service or worker. The focused-last-row regression failed on the old code (one unwanted query), then passed after the fix. All 9 checks pass, including 100 refresh/repaint cycles without extra queries, preserved focus, empty results, bounded/deduplicated last-page fetches, and stale completions. Logs: `test-output/gui-cache-before.txt`, `gui-cache-after.txt`.
|
||
- Release build and both CTest targets pass (86 core checks plus 9 GUI cache checks). Existing UI integration passes search/filter grouping/paging/native Shell menus/Unicode rename; report and renders are in `test-output/ui-cache-fix`.
|
||
- Installed the corrected executable in Program Files and `D:\tools\bin`, restarted the service, and repeated the original mouse sequence in the installed GUI. Continuous refresh is gone; the normal two-second refresh remains. GUI CPU use fell to about 0.17 CPU seconds over the same 3-second sample (`test-output/gui-cache-cpu.json`). The window remains open on `figma.exe` for the user. `locate -n 10 "python*.exe"` still succeeds.
|
||
- WLocate SHA256: `30B925FF2F85719F2F046FC718F58DD8455B7749A957B2DD0009AC3D6E071848`; CLI unchanged. Release package regenerated with matching binaries. The earlier executable hashes below are historical.
|
||
|
||
## Update — green icon and UTF-8 manifest, 2026-10-07 21:30
|
||
|
||
- User clarified the icon reference is Everything, not Evernote. Created original green magnifying-glass vector artwork with the same general orientation; no original icon pixels are used in the release. `assets/wlocate.svg` and `tools/Build-Icon.ps1` regenerate the nine-size (16–256 px) `assets/wlocate.ico`.
|
||
- Embedded the icon in both EXEs. Window, taskbar, tray, per-monitor DPI icon reload and Start-menu shortcut use it. UI screenshot verified at `test-output/ui-green-utf8/search.png`.
|
||
- Merged the `activeCodePage` UTF-8 setting from the user's `D:\work\dev3\utillib\utf8.manifest` into the existing shared manifest, preserving DPI, long paths, common controls and execution level. Both entry points enforce `GetACP() == 65001`; internal narrow strings/index/IPC remain UTF-8 with UTF-16 conversion at Windows W/Shell/UI boundaries. Compiler already uses `/utf-8`.
|
||
- Both actual EXE manifests extracted with mt.exe and checked. 86 core checks pass, including process ACP, implicit-code-page decoding and UTF-8 CreateFileA with Chinese/emoji. UI integration and live Unicode/long-path/hardlink integration pass.
|
||
- Reinstalled both Program Files and PATH copies, regenerated release ZIP, verified all deployed hashes match. wlocate SHA256: 44DBEE1555D0A1B7DD76C61C09DF6F104A5D1CFCFF2543A55AE8328C426BD1EE; locate: 0717B40DB2CEBFB598D4C9CE7DBBDF4079FE5C9877E1703595FE55EB6692E36E. Service Running; C:/D: ready, pending=0. GUI reopened for the user.
|
||
|
||
The older delivery hashes and test counts below are historical.
|
||
|
||
## Delivery status — 2026-10-07 21:10 (supersedes older notes below)
|
||
|
||
- Both native executables built, tested, packaged and installed. `WLocateService` is Running/Auto from `C:\Program Files\WLocate\wlocate.exe --service`; both executables also installed in `D:\tools\bin`. Default `locate` now resolves to the native program, version 1.0.0. C: and D: report ready, pending=0, ~3.20 million records.
|
||
- Release: `out/WLocate-1.0.0-win-x64.zip`, unpacked beside it. README, performance report, installer, complete PCRE2/SLJIT/reference notices and SHA256SUMS included. Source and build/test scripts remain here; no commit or remote publication performed.
|
||
- Original Everything wrapper `D:\tools\bin\locate.old.exe` hash remains C9AB737C1F30183F954CD899F6B6A03380A00ABD4DDE72E832C66BB872959FA3. Pre-native C# wrapper backup: `D:\tools\bin\locate.exe.backup-20261007-210549`. `C:\Program Files\WLocate\install-state.json` preserves that original rollback path across native upgrades.
|
||
- Final executable hashes: wlocate 6E083E51C1BFD59D5CED5417570A4D71651CAE9BE68E0DB6AAEB03D72773BE35; locate 33E1D35F42B916E00B570006C102F7CAE53BF26F9E96C6D184FD77E46BF93F87. Build, Program Files, PATH and release copies match (`test-output/installation.json`).
|
||
- 83 core checks pass; live integration (Unicode, 30 long hardlinks, rename/move/delete/metadata, >300-character paths, 500-file create burst), protocol (60 concurrent requests, 20k rows, malformed requests, timeout recovery, admin checks), restart and VHD hotplug tests pass. UI self-test passed native menus across folders, Unicode Shell rename, filters, paging; rendered images inspected at `test-output/ui-final`.
|
||
- Stable same-output benchmark: python*.exe 66 identical paths, legacy median 68.86 ms vs native 9.10 ms. Native -n10 python 8.40 ms; -n10 *.exe 7.61 ms. See PERFORMANCE.md for measured broad-query limitations (uncached all-name sort 1.28 s, all-path 3.97 s, regex ~0.28 s; cached first pages ~0.2 ms). Do not claim universal Everything engine parity.
|
||
- Final critical correctness fix: NTFS header hardlink count includes separate DOS/8.3 FILE_NAME attributes. Track filename_attribute_count including DOS aliases while emitting only long names. Previous mismatch triggered permanent Access Denied fallback retries on protected files; after fix pending=0. New unit fixture verifies this, and 30-link live tests still pass.
|
||
- Snapshot v4 uses SHA-256 and atomic replacement. Keeps an earlier USN checkpoint for unresolved record retries. Name-only generation avoids metadata-only invalidation. Parent-path interning reduces sort allocations. PCRE2 10.49 JIT is statically linked into service. Buffered 64 KiB IPC, timeout/cancel and final acknowledgement protect output completeness. Compaction releases unused interned names too.
|
||
- Dynamic local NTFS drive manager polls every 2 s; independent indexers poll USN at 250 ms. One unavailable volume does not block ready ones; CLI marks partial results with stderr and exit 2. Protected Program Files and ProgramData directories; service repair always installs to Program Files.
|
||
- Unsupported: non-NTFS/network/folder indexes, no-letter volumes, content search, ETP/HTTP, all Everything advanced functions, exact GNU database compatibility. README documents support and limits. All-match initial sorts and memory footprint remain below Everything's efficiency.
|
||
- IDA analysis completed; Everything IDB saved at `..\reference\Everything-1.4.1.1032.exe.i64`, both IDA instances closed. Computer-use native pipe was unavailable; GUI validation used the application's own test target, not external mouse automation. See notices for IDA observations and references.
|
||
- psuac recipe: `Start-Process D:\tools\bin\psuac.exe -ArgumentList '-w powershell.exe -NoProfile -NonInteractive -ExecutionPolicy Bypass -File <script>' -WindowStyle Hidden -PassThru -Wait`. Do NOT use psuac -o for PowerShell: invalid stdin makes PS silently fail startup. Use transcripts. -o works for simple native programs.
|
||
|
||
Older chronological development notes follow.
|
||
|
||
## Objective
|
||
|
||
Build an independent native Windows file finder, referring to SwiftList's indexing/service design and Everything's observable behavior. Deliver `wlocate.exe` (GUI and Windows-service modes) and `locate.exe` (command line). Keep existing Everything and SwiftList operational during development. Use `psuac -w -o <log> ...` for authorized administrative deployment and diagnostics.
|
||
|
||
## Acceptance
|
||
|
||
- Index all mounted local NTFS volumes, including every hard-link name, using MFT metadata; persist snapshots and replay USN changes after restart.
|
||
- Apply create, rename, move, delete and hard-link changes without a full rescan; detect journal gaps and rebuild safely.
|
||
- Search in the service, return only final matches, and stop limited CLI queries promptly.
|
||
- Native GUI with incremental search, virtual result list, common filters/syntax, sorting, standard keyboard actions and real Windows Shell context menus.
|
||
- CLI supports the prior locate-style options, UTF-8 and NUL output, clear exit codes and errors.
|
||
- Compare result sets and latency to the installed Everything-based `locate.old.exe`, including hard links and live-update fixtures.
|
||
- Exercise GUI, service restart/cache recovery and command replacement; retain rollback copies.
|
||
- Document implemented functions, measured performance, and remaining differences from Everything honestly.
|
||
|
||
## Current state
|
||
|
||
- Native source root: this directory. Previous C# adapter remains in `../SwiftList/Locate` for reference and rollback.
|
||
- Reference Everything: version 1.4.1.1032, SHA-256 F191F756996A14A11E5445FA7103D302EFD510CF2FBF920E6C0C8ED51D512E36. A private local analysis copy is under `../reference`.
|
||
- Old adapter baseline for `python*.exe`: 91–109 ms, 62 results. C# enumeration adapter: 3439–3629 ms, 58 results; ~71.7 MB of unnecessary pipe traffic. Four missing paths are hard links. SwiftList's MFT scanner currently skips extension records, an indexing issue to investigate.
|
||
- MSVC 14.50, Windows SDK, CMake and psuac are available. The shell is not elevated.
|
||
- IDA MCP open_database failed with worker exit 106; investigating without modifying the reference binary.
|
||
|
||
## Implementation plan
|
||
|
||
1. Native query engine, index records and protocol; unit tests.
|
||
2. MFT scanner, persistent cache and USN monitor; live service tests.
|
||
3. CLI and benchmarks; tune hot paths and validate hard links.
|
||
4. GUI, Shell operations and preferences; visual and interaction verification.
|
||
5. Installation, recovery, regression benchmarks and final delivery.
|
||
|
||
No commits or remote publication are authorized or planned.
|
||
# Progress 2026-10-07 20:04
|
||
|
||
## Update 20:16 (latest)
|
||
|
||
- Valid baseline `python*.exe`: old median 71.03 ms vs native 8.12 ms, exactly the same 66 paths. Later indexes gained temporary Python environments. Legacy wrapper DOES NOT support `-n`; fixed benchmark.py to exclude that invalid comparison. Do not cite old-vs-new limited-query results.
|
||
- `tests/integration.py` now passes live Unicode+emoji create, 30 long hardlinks forcing MFT extensions, directory rename, link removal, move, metadata-size updates, NUL/count/limit, errors/no-match exit codes, recursive fixture cleanup. Changes observed ~0.13–0.35 s. Script safely limits cleanup to its own UUID fixture under test-output.
|
||
- Fixed live hardlinks: nonresident attribute lists can be dirty in NTFS cache while raw-volume reads are stale. On link-count mismatch, enumerate current links using OpenFileById/GetFinalPathNameByHandle/FindFirstFileNameW and resolve parent IDs. Also merge unnamed DATA size from extension records and FILE_STANDARD_INFO. Initial raw scan now reconciles all extended records before publishing, including DATA-only extensions.
|
||
- Query optimization in progress: allocated-order name scanning and 64-bit trigram Bloom presence masks reduced uncached `python` basename searches from ~190 ms to ~25 ms. No false-negative filter intended; bracket patterns bypass masks. Plain full-path `python` was 4.55 s, then 1.07 s with parent memoization. Latest change uses component match flags + row-indexed ancestry (not yet benchmarked).
|
||
- Latest index changes: Entry uses its old 4-byte padding for parent_row cache, directory rename mutates stable row identity, full_path uses bounded stack array and rejects cycles/orphans. Separate name_generation avoids invalidating name-query caches for metadata-only USN updates. Name sort merges already-sorted volume results rather than resorting all 3.2M rows (prior empty GUI name sort was 5.9 s; latest not yet measured).
|
||
- Snapshot format bumped WLOCIDX3/version 3 to rebuild stale metadata and parent-row caches. Latest build passed 47 checks and just installed StageOnly; full rescans may be running. Build command CMake as above, no pending build session.
|
||
- IDA Everything analysis found file-query dispatcher 0x1400af3b0 divides file index into chunks, chooses worker count based on indexed file count/limits and merges per-thread results. Worker 0x140089ef0 failed decompilation (large/complex). Folder counterpart 0x1400ae4b0. NTFS file-record reader 0x1400cfe80. Document these as analysis observations, not copied source.
|
||
- IDA xrefs API actual `db.xrefs.to_ea(addr)` with `from_ea`/`to_ea` fields (reference's `to`/`frm` example is stale). Strings/functions get_at/get_pseudocode work. Avoid giant unfiltered outputs from Everything help string or old locate `-h` (old treats -h as a search).
|
||
|
||
|
||
Native C++ implementation now builds both executables with MSVC 19.50 / SDK 10.0.28000.0, CMake VS 18 2026. Source in this directory (not subject to cloned SwiftList AGENTS). 47 core checks pass via `ctest --test-dir build -C Release --output-on-failure`. All core/UI/service sources implemented, but real integration validation and performance optimization still in progress. Do not treat this as final delivery yet.
|
||
|
||
- Implemented raw NTFS MFT + extension FILE_NAME hardlinks, metadata, USN poll, snapshots, name interning/index, native pipe protocol, CLI options, Win32 virtual ListView GUI, Shell IContextMenu/clipboard/drag/rename, tray/hotkey/bookmarks/export.
|
||
- Native service `WLocateService` installed and running from `C:\Program Files\WLocate\wlocate.exe --service`. Installation via `Install.ps1 -StageOnly`; existing D:\tools\bin\locate.exe remains the old C# adapter, original Everything wrapper remains locate.old.exe.
|
||
- IMPORTANT elevation recipe: `Start-Process D:\tools\bin\psuac.exe -ArgumentList '-w powershell.exe -NoProfile -NonInteractive -ExecutionPolicy Bypass -File D:\work\gitprojs\swiftlist-locate\wlocate\Install.ps1 -StageOnly' -WindowStyle Hidden -PassThru -Wait`. Avoid psuac `-o` for PowerShell: its invalid stdin HANDLE prevents PS startup; transcript is `install-transcript.log`. For simple native commands -o works. Direct invoking GUI psuac via & can return before completion; Start-Process -Wait is necessary.
|
||
- Actual first scan: C ~990104 records /818773 file links, D ~2145178 records /1963931 file links. Service working set ~628 MB, private ~708 MB, not optimized yet. Data `%ProgramData%\WLocate`.
|
||
- Corrected `FSCTL_GET_NTFS_FILE_RECORD` handling: these are already fixed-up in-memory records, must NOT apply raw sector USA fixup (tails may change after previous disk write). Raw MFT scan still validates fixup.
|
||
- Corrected pipe lifetime: DisconnectNamedPipe discarded unread bytes; now client acknowledges fully received reply before server disconnects. Current latest native binaries installed with that fix. Await real CLI check.
|
||
- IDA MCP repaired: plugin `.venv` lacked pyvenv.cfg and dependencies. Restored pyvenv.cfg to actual Python 3.14.3 home, installed `ida-nexus 0.13.3`, `ida-domain 0.5.1`, `idapro 0.0.11`, zeromcp/packaging via uv pip. **Do not `uv sync` while server runs** (attempted recreation failed on running Scripts and removed libs; libs now restored). Main server remains running. Future uv run can restore ida-mcp dist metadata if needed.
|
||
- IDA active Everything instance `04e114d6f38c`, binary `..\reference\Everything-1.4.1.1032.exe`; psuac instance `44a40ce58704`. API reference already read. Can query strings/functions/xrefs and decompile. Found extensive Everything NTFS USN debug strings; next inspect relevant routines for indexing/query ideas. No Everything code copied into native program.
|
||
- Computer-use skill read + guidance/api/confirmations. Initialized @oai/sky via node_repl, but list_windows consistently fails `Computer Use native pipe is unavailable ... os error 2`. Per skill retried, reset, reinitialized; still fails. No Windows UI input performed. Need GUI testing when available or program-level integration checks, explicitly report any residual visual-test limitation.
|
||
|
||
Remaining important work: real result comparisons/benchmarks (python*.exe incl all hardlinks), live filesystem create/rename/move/hardlink/delete/metadata tests, restart snapshot test, query correctness/security tests, optimize broad GUI and path queries, add grouping/regex handling and missing common features where feasible, UI functional/visual verification, secure/final installation and replace default locate with backup, README/support matrix and measurements. Known concerns: GUI row cache/selection on refresh, CLI cancellation pending request, name-only orphan counting, full_path depth/cycles, unbuffered per-row IPC overhead, name sort unnecessarily sorts pre-sorted volume hits, no dynamic drive handling yet, unsupported filesystem folder indexes, incomplete grouped syntax. Core query tests cover current supported syntax only.
|