When `root` has at least one direct symlinked child, default to following
symlinks instead of silently dropping their contents. This makes "fake
working dir" patterns (a folder full of symlinks pointing at scattered
source dirs) work transparently — the caller no longer has to know to
pass `follow_symlinks=True`.
Concrete motivating shape:
~/projects/research-corpus/
├── papers -> /external/drive/papers
├── notes -> /external/drive/notes
└── code -> /external/drive/code
Before: `--update` invoked `detect_incremental(root)` without passing
`follow_symlinks=True`, so the scan saw the small set of files actually
inside the root and missed everything reachable only through a symlink.
Result: every legitimate new file was missed, and the manifest paths
were marked as deleted.
After: when `follow_symlinks` is left at its default `None`, `detect()`
runs `_auto_follow_symlinks(root)` (one cheap `iterdir()` + `is_symlink()`
loop) and follows symlinks if any direct child is one. Behaviour is
unchanged for ordinary scans (no direct symlinks → False, as before).
Override is always possible by passing an explicit `follow_symlinks=True`
or `follow_symlinks=False`; existing tests confirming the explicit
behaviour continue to pass unchanged.
Backwards compatibility:
- Type annotation: `bool` -> `bool | None`. Callers passing `True`/`False`
continue to work identically. Callers passing nothing get the new
auto-detect.
- No new dependencies.
- Cheap: one `iterdir()` call once per detect() invocation.
Tests:
- 3 new tests in tests/test_detect.py:
- test_detect_auto_detects_direct_symlink_child
- test_detect_default_does_not_follow_when_no_symlinks
- test_detect_explicit_false_overrides_auto_detect
- Full tests/test_detect.py + tests/test_incremental.py: 49/49 pass.
#873: Remove blanket dot-prefix exclusion from detect.py and
extract.py collect_files(). Add framework caches (.next, .nuxt,
.turbo, .angular, .idea, .cache, .parcel-cache, .svelte-kit,
.terraform, .serverless, .graphify) to _SKIP_DIRS so they stay
blocked. Meaningful dot dirs (.github, .claude, etc.) are now
indexed.
#874: Add _maybe_reload() with mtime+size stat key and threading.Lock
to serve.py. call_tool and read_resource call _maybe_reload() on
every request; the graph reloads automatically when graph.json changes
without restarting the MCP server.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add .sh, .bash, .json to CODE_EXTENSIONS in detect.py so files reach extractor
- Fix test_detect_incremental manifest path collision with new .json extension
- Update test_watch to reflect .json/.sh are now watched extensions
- B-1: only emit source imports for paths that exist on disk
- J-1: replace stat()+read() with bounded read to eliminate TOCTOU
- J-3: move pair_count cap inside loop so it is honoured exactly
- J-4: namespace $ref/extends refs with "ref_" prefix to prevent ID collision
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`detect_incremental(root)` always called `detect(root)` without forwarding
the `follow_symlinks` kwarg. As a result, corpora that include symlinked
sub-trees pointing to directories outside the scan root (e.g. a
`state_of_truth/` symlink pointing at `~/.hermes/state_of_truth/`) were
visible to a full `detect()` run with `follow_symlinks=True` but invisible
to any subsequent `--update` run. The incremental scan would then either
report no changes (silently dropping legitimate new files) or repeatedly
re-extract a phantom subset, depending on what was reachable without
crossing symlinks.
Add a keyword-only `follow_symlinks` parameter to `detect_incremental()`
and forward it. Default stays `False` for backwards compatibility — only
callers that already opt in to symlink following on `detect()` pick up
the new behaviour for incremental runs too.
Test: a corpus with a symlinked directory is invisible with
`follow_symlinks=False`, fully indexed with `follow_symlinks=True`, and
correctly reports zero new files on a second incremental scan after the
manifest is saved.
Alpha Nury