harden(export): widen dedup reserve to 5 and add multibyte budget test (#2655)

Widens _SLUG_SUFFIX_RESERVE/_DEDUP_SUFFIX_RESERVE from 4 to 5 so a four-digit
collision suffix (_1000..) can't push a truncated stem past MAX_PATH; the
suffix is technically unbounded but 5 chars covers ~10k identical stems. Adds
an end-to-end test that CJK labels at a tight budget stay within the window,
keep their non-ASCII characters, and produce links that resolve on disk.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
safishamsi
2026-08-14 14:26:12 +01:00
co-authored by Claude Opus 4.8
parent b7c709e13d
commit d79520198d
4 changed files with 35 additions and 5 deletions
+1
View File
@@ -6,6 +6,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu
- Fix: a cross-file INFERRED `uses` edge now binds to the symbol whose body actually references the imported name (a module-level function is a valid source; a co-located class that never touches the import gets no edge), instead of fanning out from the import line to every class in the importing file (#2652, thanks @ousamabenyounes). A reference at module top level, with no enclosing symbol, emits no edge.
- Fix: a wiki article link now targets the article's filename verbatim instead of a percent-encoded twin, so a label with `( ) & #` or non-ASCII characters no longer produces a link that names no file on disk; the link and the on-disk filename share one canonicalization (#2597, thanks @abhay-codes07).
- Fix: export filenames are now budgeted against the full destination path rather than only the per-component `NAME_MAX`, so a long output directory on Windows no longer pushes an Obsidian/wiki note path past `MAX_PATH` and aborts the export mid-write (#2655, thanks @abhay-codes07). The collision-suffix reserve was widened so a four-digit dedup suffix can't overrun the budget.
- Feature: OCaml `.ml`/`.mli` extraction via tree-sitter-ocaml (optional `[ocaml]` extra). Extracts modules, top-level and module-level values/functions, types and their variant constructors, `open` imports, and function calls; qualified calls (`Geo.area`) resolve to the value, and cross-file `open`/call targets collapse onto the unique real definition via the corpus stub rewire.
- Fix: a JS/TS `for...of` / `for...in` loop binding is now shadowed, so passing it as a call argument no longer fabricates an `indirect_call` edge to an unrelated same-named callable (#2685, thanks @ousamabenyounes); completes the loop/closure/catch shadow family (#2568/#2569/#2517).
- Fix: graph provenance (`built_at_commit`) is stamped from the analysed repository rather than the shell's working directory, so `graphify extract` run from elsewhere records the target's commit, not the caller's (#2534 family; #2699, thanks @C0KERNEL).
+4 -2
View File
@@ -512,8 +512,10 @@ def _obsidian_safe_stem(label: str, limit: int = 200) -> str:
# Room _dedup_node_filenames / the community loop need for a collision suffix
# ("_1" … "_999") appended AFTER the stem was capped.
_DEDUP_SUFFIX_RESERVE = 4
# ("_1" … "_9999") appended AFTER the stem was capped. The suffix is technically
# unbounded, but 5 chars ("_" + 4 digits) covers ~10k identical stems, far past
# anything real; sizing it to 3 digits let a 1000th collision overrun MAX_PATH.
_DEDUP_SUFFIX_RESERVE = 5
# Prefix the community overview notes carry ("_COMMUNITY_Backend.md").
_COMMUNITY_PREFIX = "_COMMUNITY_"
+5 -3
View File
@@ -9,9 +9,11 @@ import networkx as nx
from graphify.build import edge_data
from graphify.paths import stem_filename_budget
# Room _unique_slug needs for the collision suffix ("_2" … "_999") it appends
# after _safe_filename has already capped the slug.
_SLUG_SUFFIX_RESERVE = 4
# Room _unique_slug needs for the collision suffix ("_2" … "_9999") it appends
# after _safe_filename has already capped the slug. The suffix is technically
# unbounded, but 5 chars ("_" + 4 digits) covers ~10k identical stems, far past
# anything real; sizing it to 3 digits let a 1000th collision overrun MAX_PATH.
_SLUG_SUFFIX_RESERVE = 5
# Characters a slug may not contain, because the article's LINK and its ON-DISK
# NAME have to be the same string (#2597). Anything left here must be legal,
+25
View File
@@ -166,6 +166,31 @@ def test_wiki_respects_a_small_budget(tmp_path, monkeypatch):
assert len(p.stem) <= 40, p.name
def test_wiki_multibyte_labels_stay_within_budget_and_links_resolve(tmp_path, monkeypatch):
"""A CJK label at a tight budget: the stem must stay within budget counted in
CHARACTERS (wiki slices by character), links must resolve on disk, and the
non-ASCII characters must survive rather than being reduced to underscores."""
monkeypatch.setattr(wiki_mod, "stem_filename_budget", lambda out, **kw: 40 - kw.get("reserve", 0))
G, comms = _graph(["文档索引" * 50, "配置解析器" * 50])
out = tmp_path / "wiki"
to_wiki(G, comms, str(out), community_labels={0: "模块" * 100})
written = list(out.glob("*.md"))
assert written
import re
target_re = re.compile(r"\]\(([^)\s]+)\)")
for p in written:
if p.name != "index.md": # index is a fixed filename, not a label slug
# 40-char window: the stem is capped at 40 - reserve, and a collision
# suffix can add back up to the reserve, so the whole stem stays <= 40.
assert len(p.stem) <= 40, p.name
assert any("" <= ch <= "鿿" for ch in p.stem), f"CJK stripped from {p.name}"
for target in target_re.findall(p.read_text(encoding="utf-8")):
if "://" in target:
continue
assert (out / target).exists(), f"{p.name}: dangling link {target!r}"
# ---------------------------------------------------------------------------
# End-to-end on the platform that actually has the ceiling
# ---------------------------------------------------------------------------