_get_docstring's loop checked only the FIRST child of a module/class/
function body for a docstring expression_statement, then unconditionally
broke -- so a leading comment (the #! shebang essentially every
executable Python script starts with, a coding declaration, a license
header) sat in that first-child slot instead of the docstring, and the
real docstring one position later was never seen. No warning, no
partial-extraction report, exit 0 -- module rationale for an entire
class of file (any shebang-led or comment-led script) was silently
absent from the graph.
Comments are not statements: a docstring must be the first STATEMENT
in the body, so the fix skips leading `comment` children before
checking whether the first real statement is a docstring. The same
shared helper also serves class/function bodies, so a comment as the
first line of a function/class body is fixed too, not just the
module-level case.
Fixes#3312.
(cherry picked from commit 87d4cb49b7)
_extract_python_rationale / _extract_js_rationale sliced the raw
docstring/comment text to 80 characters before collapsing whitespace,
so the cut could land mid-word, leave a run of literal spaces where a
newline + indentation used to be, and, when the cut landed on a ".",
produce an Obsidian export filename ending in "..md".
Both _add_rationale sites now share _shorten_rationale_label, which
normalizes whitespace first via textwrap.shorten (word-boundary safe,
adds a placeholder only when it actually truncates) and falls back to
a plain character truncation when shorten collapses to a bare
placeholder -- which it does when the first word alone is already
>= 80 chars (e.g. a comment opening with one long URL), a case that
would otherwise regress to a content-free label.
Parity with _extract_python_rationale: Python files get rationale nodes
from docstrings and '# NOTE:'-style comments, but JS/TS comments were
discarded entirely. This adds a post-pass to extract_js that:
1. extracts rationale comments ('// NOTE:', '// WHY:', block-comment
'* NOTE:' variants) as rationale nodes with rationale_for edges,
matching the Python behavior;
2. first-classes architecture-decision references (ADR-NNNN, RFC NNNN)
found in comments as doc_ref nodes with 'cites' edges from the file.
The doc_ref pass is the natural join point between code and design docs
in mixed corpora: teams conventionally cite ADR ids in file headers, but
today those citations produce no edges, so code<->ADR connections never
form even when the discipline exists. Spellings are normalized
(ADR-11 / ADR 0011 -> ADR-0011) so references to the same document
collapse to one node, and string literals are excluded (comment-shaped
lines only).
Tested on a real mixed corpus (Flutter/Supabase monorepo): router.ts
alone yields 10 ADR citations that previously produced zero edges.
Alembic/Flask-Migrate revisions, Django migrations, and protobuf/OpenAPI
generated files produce hundreds of degree-1 rationale nodes labeled as
'possible documentation gaps'. Their module docstrings are revision
annotations or boilerplate, not architectural rationale.
- Add _is_autogenerated_python() in extract.py detecting Alembic, Django
migrations, and generic DO-NOT-EDIT markers; skip module docstring only
- Function/class docstrings inside those files still extracted as normal
- report.py: exclude file_type=rationale nodes from isolated-node gaps
section — rationale nodes are degree-1 by construction; flagging them
as missing edges was always wrong
- 5 new tests covering Alembic, Django, protobuf, false-positive guard,
and function-docstring passthrough
ayushcodes10