fix(detect): stop the sensitive-file filter from dropping topic docs and real source (#2106)

The heuristic over-matched and silently dropped legitimate files:
- prose `.md`/`.rst` whose topic slug ends in a keyword (privacy-tokens.md,
  token-economics.md) — only code was exempt, not prose;
- the unbounded Stage-2 `service.account` substring (regex `.` wildcard) matched
  real source (google/oauth2/service_account.py) and prose slugs.
It also MISSED real secrets (.npmrc, .pypirc, secring, .git-credentials, and
case variants on case-insensitive filesystems), which were being indexed.

Fix: move service_account/aws_credentials to the boundary-checked keyword path
(so real source is spared, downloaded key files still drop), add a prose-note
carve-out (multi-word slugs indexed, bare `secrets.md`/`token.md` still dropped),
tighten id_rsa with a left boundary, add the missed secret dotfiles + secring,
lowercase the dir/segment comparisons, and count multi-dot slugs as multi-word.
Net effect is stricter on real secrets and stops the false-positive data loss.

Traceability: `graphify extract` now names the files skipped as sensitive (not
just a count), so a wrongly-flagged file is visible.
This commit is contained in:
safishamsi
2026-07-22 18:40:11 +01:00
parent 82c46e5358
commit 36b5e770eb
4 changed files with 124 additions and 13 deletions
+14
View File
@@ -244,3 +244,17 @@ def test_explicit_exclude_replaces_persisted_setting_with_custom_out(tmp_path):
assert json.loads((graph_out / ".graphify_build.json").read_text()) == {
"excludes": ["generated"]
}
def test_extract_names_skipped_sensitive_files(tmp_path):
"""#2106 traceability: a file dropped by the sensitive-file filter is reported
by NAME (not just a count), so a wrongly-flagged file is visible."""
repo = tmp_path / "repo"
repo.mkdir()
(repo / "app.py").write_text("def hello():\n return 1\n")
(repo / "github_token.txt").write_text("ghp_secretvalue\n") # real secret -> skipped
r = _run(repo, "--code-only", "--no-cluster")
assert r.returncode == 0, r.stderr
out = r.stdout + r.stderr
assert "skipped as potentially sensitive" in out
assert "github_token.txt" in out, "the skipped filename must be surfaced (#2106)"