1 Commits
Author SHA1 Message Date
SafiandClaude Sonnet 4.6 5504c84324 perf/fix: replace datasketch with pure-numpy MinHash; memoize detect ignore checks
- graphify/_minhash.py: self-contained MinHash/MinHashLSH using pure numpy,
  byte-identical hash math to datasketch (sha1_hash32, Mersenne-prime permutation).
  Drops datasketch + scipy transitive dep — eliminates EDR hang on Windows where
  numpy.testing platform.machine() subprocess spawn was intercepted at import time
- dedup.py: import from graphify._minhash instead of datasketch
- pyproject.toml: replace datasketch>=1.6 with numpy>=1.21
- detect.py: memoize _is_ignored/_eval results in a dict[Path,bool] cache per
  detect() call; each unique ancestor dir evaluated once across all sibling files,
  eliminating ~42M redundant fnmatch calls on large repos (~34% whole-run speedup)
- tests/test_minhash.py: 11 tests including import-isolation guard asserting scipy
  and numpy.testing are not loaded after import graphify.dedup
- tests/test_detect.py: 2 cache tests — correctness (cached==uncached with negation
  patterns) and hit-count (each dir evaluated exactly once across siblings)

Closes #1234, closes #1235

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 15:50:38 +01:00