From ef8c2fef505a6bb0cfb402a31284ed558119a443 Mon Sep 17 00:00:00 2001 From: Safi Date: Sun, 5 Apr 2026 13:42:11 +0100 Subject: [PATCH] perf: larger chunks + code-only fast path + timing estimates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Semantic extraction chunks: 12-15 → 20-25 files (fewer subagent round trips) - Code-only corpora skip semantic dispatch entirely (AST covers it) - Print estimated time before extraction so the wait feels intentional --- graphify/skill.md | 13 ++++++++----- skills/graphify/skill.md | 13 ++++++++----- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/graphify/skill.md b/graphify/skill.md index dc3bc4da..25441d6c 100644 --- a/graphify/skill.md +++ b/graphify/skill.md @@ -124,12 +124,15 @@ else: #### Part B - Semantic extraction (parallel subagents) +**Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. + **MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.** -Before dispatching subagents, print a cost estimate: -- Load `total_words` from `.graphify_detect.json` -- Estimate: ~(total_words / 750) input tokens per file on average, output ~20% of that -- Print: "Semantic extraction: ~N files, estimated ~X input tokens" +Before dispatching subagents, print a timing estimate: +- Load `total_words` and file counts from `.graphify_detect.json` +- Estimate agents needed: `ceil(uncached_non_code_files / 22)` (chunk size is 20-25) +- Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit)) +- Print: "Semantic extraction: ~N files → X agents, estimated ~Ys" **Step B0 - Check extraction cache first** @@ -157,7 +160,7 @@ Only dispatch subagents for files listed in `.graphify_uncached.txt`. If all fil **Step B1 - Split into chunks** -Load files from `.graphify_uncached.txt`. Split into chunks of 12-15 files each. Each image gets its own chunk (vision needs separate context). +Load files from `.graphify_uncached.txt`. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context). **Step B2 - Dispatch ALL subagents in a single message** diff --git a/skills/graphify/skill.md b/skills/graphify/skill.md index dc3bc4da..25441d6c 100644 --- a/skills/graphify/skill.md +++ b/skills/graphify/skill.md @@ -124,12 +124,15 @@ else: #### Part B - Semantic extraction (parallel subagents) +**Fast path:** If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do. + **MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.** -Before dispatching subagents, print a cost estimate: -- Load `total_words` from `.graphify_detect.json` -- Estimate: ~(total_words / 750) input tokens per file on average, output ~20% of that -- Print: "Semantic extraction: ~N files, estimated ~X input tokens" +Before dispatching subagents, print a timing estimate: +- Load `total_words` and file counts from `.graphify_detect.json` +- Estimate agents needed: `ceil(uncached_non_code_files / 22)` (chunk size is 20-25) +- Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit)) +- Print: "Semantic extraction: ~N files → X agents, estimated ~Ys" **Step B0 - Check extraction cache first** @@ -157,7 +160,7 @@ Only dispatch subagents for files listed in `.graphify_uncached.txt`. If all fil **Step B1 - Split into chunks** -Load files from `.graphify_uncached.txt`. Split into chunks of 12-15 files each. Each image gets its own chunk (vision needs separate context). +Load files from `.graphify_uncached.txt`. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context). **Step B2 - Dispatch ALL subagents in a single message**