Add video/audio corpus support with yt-dlp download and Whisper transcription

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
SafiandClaude Sonnet 4.6 committed 2026-04-10 15:40:18 +01:00
1 parent e3adc0417a
commit cf5d721bee
17 files changed
+865 -14

No files matched your search

+48 -1
View File
@@ -98,13 +98,60 @@ Corpus: X files · ~Y words
docs: N files (.md .txt ...)
papers: N files (.pdf ...)
images: N files
video: N files (.mp4 .mp3 ...)
```
Omit any category with 0 files from the summary.
Then act on it:
- If `total_files` is 0: stop with "No supported files found in [path]."
- If `skipped_sensitive` is non-empty: mention file count skipped, not the file names.
- If `total_words` > 2,000,000 OR `total_files` > 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding.
- Otherwise: proceed directly to Step 3 - no need to ask anything.
- Otherwise: proceed directly to Step 2.5 if video files were detected, or Step 3 if not.
### Step 2.5 - Transcribe video / audio files (only if video files detected)
Skip this step entirely if `detect` returned zero `video` files.
Video and audio files cannot be read directly. Transcribe them to text first, then treat the transcripts as doc files in Step 3.
**Strategy:** Run non-video semantic extraction first (Step 3B) to get god nodes, use those to build a domain hint for Whisper, then transcribe. This keeps the prompt relevant without guessing the corpus topic from filenames.
**However**, if the corpus has *only* video files and no other docs/code, skip the god-node step and transcribe with the generic fallback prompt immediately.
**Transcription command:**
```bash
$(cat graphify-out/.graphify_python) -c "
import json
from pathlib import Path
from graphify.transcribe import build_whisper_prompt, transcribe_all
detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text())
video_files = detect.get('files', {}).get('video', [])
# Try to load god nodes from a previous partial run or pass [] if not yet available
try:
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text())
god_nodes = analysis.get('god_nodes', [])
except Exception:
god_nodes = []
prompt = build_whisper_prompt(god_nodes)
print(f'Whisper prompt: {prompt}')
transcript_paths = transcribe_all(video_files, initial_prompt=prompt)
print(json.dumps(transcript_paths))
" > graphify-out/.graphify_transcripts.json
```
After transcription:
- Read the transcript paths from `graphify-out/.graphify_transcripts.json`
- Add them to the docs list before dispatching semantic subagents in Step 3B
- Print how many transcripts were created: `Transcribed N video file(s) -> treating as docs`
- If transcription fails for a file, print a warning and continue with the rest
**Whisper model:** Default is `base`. If the user passed `--whisper-model <name>`, set `GRAPHIFY_WHISPER_MODEL=<name>` in the environment before running the command above.
### Step 3 - Extract entities and relationships