Two related fixes in the community-labeling path:
#1690 (thanks @vdgbcrypto): a truncated or slightly malformed reply no
longer discards the whole batch with "Expecting value: line 1 column 6".
`_parse_label_response` now salvages the complete `"id": "name"` pairs from
a reply that failed a strict `json.loads` (e.g. one truncated mid-object),
raising only when no pairs can be recovered. The per-batch token budget was
also raised (256 + 48*n, was 64 + 24*n) so models that prepend a short
preamble have headroom to finish the JSON. The exact provider truncation
could not be reproduced without a live key; the parser and budget address
the mechanism.
#1694 (thanks @sub4biz): cluster-only mode reported a hardcoded
`0 input * 0 output` token cost because the labeling LLM calls were never
accounted for. `_call_llm` now accumulates per-response usage into an
optional accumulator threaded through the labeling path and surfaced in
GRAPH_REPORT.md. Backends that do not return usage (the Claude Code CLI)
contribute nothing, which is honest rather than estimated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`graphify label --missing-only` restricts LLM labeling to communities that are
unnamed or still hold a `Community N` placeholder, preserving existing
non-placeholder labels read from .graphify_labels.json and merging new labels over
them. Lets a large graph be relabeled incrementally without re-naming (and paying
for) communities that already have good names.
Ported from PR #1481 by @jiangyq9. This supersedes the earlier #1421 by
@matiasduartee, which proposed the same flag — credit to @matiasduartee for the
original; #1481 is written against the current label signature (post-#1390
max-concurrency/batch-size) and merges clean, where #1421 had drifted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
label_communities ran batches one LLM call at a time, so a large graph needed
hundreds of sequential calls even on backends that allow heavy concurrency. It
now fans batches out across a thread pool, mirroring extract_corpus_parallel:
results are returned per batch and merged on the main thread (labels dict is
never mutated concurrently, no lock), and workers==1 keeps the original
sequential path verbatim. ollama and claude-cli are forced serial unless the
matching GRAPHIFY_*_PARALLEL env opt-in is set (same guard as extract).
generate_community_labels threads max_concurrency + batch_size through, and the
cluster-only/label CLI parses --max-concurrency and --batch-size (both `--flag N`
and `--flag=N` forms; the space form is parsed explicitly so the value is not
mistaken for the positional scan path by the arg-walk's catch-all).
Output is deterministic regardless of concurrency (keyed by community id). Tests:
parallel == sequential result, batch-size controls batch count, batches actually
run concurrently, ollama forced serial, and the CLI parses both new flags. Full
suite 2393 passed; ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds extra_body parameter support for custom/OpenAI-compat providers so users can pass provider-specific params (e.g. thinking budget for Claude via Bedrock compat). Adds multi-batch label_communities for 16k-context models — batches multiple community descriptions into a single LLM call instead of one per community. Partial batch failures are handled gracefully.
Co-authored-by: EirikWolf <EirikWolf@users.noreply.github.com>
No behavior change. label_communities / generate_community_labels live next
to the backend dispatch (_call_llm, detect_backend) they use, so the lazy
import and separate file are no longer needed. Tests unchanged except the
import path; they still monkeypatch graphify.llm._call_llm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Community labeling was an agent-only step (skill.md Step 5): inside Claude Code
or Gemini CLI the agent names communities itself. Run as a bare CLI
(python -m graphify extract . --backend X), no agent does Step 5, so labels
stayed Community 0/1/2 for every backend.
Add graphify/labeling.py: one batched _call_llm asking the backend for a
{cid: name} map from each community's top node labels (god nodes first),
with per-community placeholder fallback and graceful degradation (no backend,
API error, or malformed reply -> Community N, never crashes).
Wire it in:
- cluster-only auto-labels when no .graphify_labels.json exists (the reported
repro path); --no-label opts out, --backend=<name> overrides auto-detect
- new `graphify label <path>` subcommand force-regenerates names
- extract prints a hint pointing standalone users at cluster-only
Tests mock the backend (no network): happy path, partial/fenced/malformed
replies, no-backend and error fallback, god_nodes dict shape.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
safishamsi