mirror of
https://github.com/safishamsi/graphify.git
synced 2026-07-13 10:57:13 +00:00
22a58ffc20
label_communities ran batches one LLM call at a time, so a large graph needed hundreds of sequential calls even on backends that allow heavy concurrency. It now fans batches out across a thread pool, mirroring extract_corpus_parallel: results are returned per batch and merged on the main thread (labels dict is never mutated concurrently, no lock), and workers==1 keeps the original sequential path verbatim. ollama and claude-cli are forced serial unless the matching GRAPHIFY_*_PARALLEL env opt-in is set (same guard as extract). generate_community_labels threads max_concurrency + batch_size through, and the cluster-only/label CLI parses --max-concurrency and --batch-size (both `--flag N` and `--flag=N` forms; the space form is parsed explicitly so the value is not mistaken for the positional scan path by the arg-walk's catch-all). Output is deterministic regardless of concurrency (keyed by community id). Tests: parallel == sequential result, batch-size controls batch count, batches actually run concurrently, ollama forced serial, and the CLI parses both new flags. Full suite 2393 passed; ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>