New graphify/extractors/ocaml.py handles both the implementation grammar
(language_ocaml, .ml) and the interface grammar (language_ocaml_interface,
.mli). Emits nodes for modules, top-level/module-level values and functions,
types and their variant constructors; edges for defines/contains, open ->
imports_from, and application -> calls. Qualified paths (Geo.area) resolve to
the final value name, not the module qualifier; local let ... in bindings do
not mint nodes or steal call attribution. Cross-file open/call targets are
sourceless stubs so the corpus rewire collapses them onto the unique real
definition (no #1402 sourced-stub leak).
Wired into detect.py (CODE_EXTENSIONS), extract.py (dispatch +
_EXTRA_FOR_EXTENSION), pyproject.toml ([ocaml] extra + all + dev dep), and
README. Adds tests/test_ocaml.py (behind importorskip).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2475 fix recomputed the match path against the explicit scan root but
skipped the _nfc() normalization rel_anchor gets, so an NFD-named path
could miss a parent ignore pattern under the #2544 NFC-matching regime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #2600 title helper preferred a cwd-relative label, which still leaks
host/user path segments when the graph is built from a directory above the
project. Keep from the graphify-out / GRAPHIFY_OUT marker onward first
(portable in every case); fall back to cwd-relative only for a fully custom
output path with no marker.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The <title> previously embedded str(output_path), so Windows absolute
host paths leaked into a tracked artifact (regression of #433, #2598).
Prefer a cwd-relative label, else keep from the graphify-out segment
onward, else the filename only.
tree-sitter gives an arrow with one unparenthesised parameter a `parameter`
field (singular) and no `parameters` list node, so `_js_local_bound_names`
never saw it: `x => sink(x)` bound nothing, and `x` read as a by-name
reference to any same-named callable in the corpus, fabricating an
indirect_call edge (INFERRED, 0.8).
The parenthesised form was always handled, so `(x) => …` and `x => …`
behaved differently. Same singular/plural trap as `catch_clause.parameter`.
Strictly subtractive on real code: +0/-88 indirect_call edges over 5,402
files of node_modules, +0/-98 over a 6,000-file mixed corpus.
An ignore rule naming a directory with an accent silently does nothing on
macOS, and the files it was meant to exclude get scanned anyway.
macOS (APFS/HFS+) returns filenames in NFD — "ç" comes back as "c" + U+0327
COMBINING CEDILLA — while editors write ignore files in NFC, where the same
"ç" is the single codepoint U+00E7. The two render identically and compare
unequal, so `fnmatch` never matches and the pattern is a no-op.
Found in a real repo: a `.graphifyignore` containing `Orçamento/` failed to
exclude that directory, and 9 client contract PDFs were picked up for semantic
extraction — i.e. queued to be sent to an LLM — despite an explicit rule
against it. The failure is silent: there is no warning, and the only symptom
is a file count that does not match what you expect. A user who does not
count would never know. That is what makes this worth fixing rather than
documenting: the rule appears to work.
Both sides are now normalized to NFC before matching, at three boundaries:
the pattern (in `_parse_gitignore_line`, so it covers .graphifyignore,
.gitignore and $GIT_DIR/info/exclude alike) and the two path forms used in
`_is_ignored` (`target.name` and the anchor-relative path).
NFC is already the form Linux and Windows produce, so this is a no-op there
and only repairs the macOS mismatch.
Tests: two regression tests cover both directions (NFC pattern vs NFD path on
disk, and the reverse); both fail before this change and pass after. A third
asserts ASCII patterns are unaffected, so the normalization cannot regress
existing behavior.
Full suite: 3833 passed. The 13 failures in tests/test_terraform.py are
pre-existing on a clean upstream checkout (optional tree_sitter_hcl not
installed) and unrelated to this change.
`_find_node_tiers` builds two normalizations of the query: `term`, which
tokenizes on \w+ so punctuation becomes a space, and `norm_query`, which
keeps it. The exact tier compared the node id against `term` only, so
`term == nid_lower` was false for every id carrying punctuation, and
`norm_query` — which already held the right form, and is even one of the
two trigram needles — was never compared against the id at all. Comparing
`norm_query` to the folded id closes that half.
It does not reach ids carrying non-ASCII text. `_node_search_text`
indexed the id raw while every query path folds through
`_strip_diacritics`, which NFKD-decomposes. Hangul syllables decompose
into conjoining jamo, and jamo have combining class 0, so they survive
the combining-character filter: the needle's trigrams and the posting's
trigrams were disjoint, `_trigram_candidates` returned a candidate list
without the node, and it was dropped before any predicate ran. The
folded id is now part of the indexed text.
Both halves are additive. An id that resolved before resolves to the
same node; only ids that previously resolved to nothing can now resolve.
The folded field is appended, and only when the fold actually differs,
so field positions do not move and an all-ASCII graph indexes byte for
byte what it indexed before. Index build, median of 7 runs:
graph trigrams postings build
5k all-ASCII before 1723 238876 96ms
5k all-ASCII after 1723 238876 97ms
17k real before 33442 2311784 998ms
17k real after 33490 2312462 990ms
5k half-Hangul before 1735 256381 116ms
5k half-Hangul after 1739 266381 124ms
The real graph is the 17269-node one measured below; 354 of its ids are
non-ASCII, so the index grows 0.03% and the build stays inside run-to-run
noise. The half-Hangul row is a deliberate worst case — every other node
id Korean — and even there the cost is paid once per graph load, on a
graph where id lookup previously returned nothing at all.
On a real 17269-node graph with Korean source filenames, every node id
fed back to itself, full population:
id class total before after
contains punctuation 2326 0 2326
contains Hangul 354 0 354
ASCII, no punctuation 14589 14589 14589
And every query that graph can produce — all 17269 ids plus all 16537
distinct labels — through `_find_node_tiers` on both variants in one
process: 31126 identical, 2680 that returned nothing before and resolve
now, 0 with a changed first result, 0 lost, 0 with a widened exact tier.
Every difference is a query that previously returned nothing.
Left alone deliberately: `_score_query` compares the id raw in the same
way, so `path` and query seeding still cannot take a punctuated id, and
the prefix tier also matches ids against `term` only. Both are behaviour
changes beyond this defect rather than part of it.
One note for the regression tests: the non-ASCII case needs a graph of
at least ~10 nodes. `_trigram_candidates` bails out to a full scan when
`min(present) > int(n * 0.10)`, so on a small synthetic graph the index
path is never taken and the test passes with the defect still present.
