2 Commits
Author SHA1 Message Date
safishamsi ada1c095ab fix(llm): bisect PDF slices on extracted text; reconcile splittable-PDF tests (#2906)
Three fixes landing the PDF-slicing set coherently:
- bisect_slice read raw container bytes, so the adaptive-retry path (a lone oversized
  slice that still overflows, #2880) searched for the newline cut in binary coordinates
  and could cut mid-line for a compressed PDF. Index unit_source_text instead, matching
  read_slice_text and expand_oversized_files; any converter failure falls back to None.
- #2900's test asserted a PDF is not splittable, which #2906 overturns; rewrite it to
  guard the real invariant (an image has no addressable text and is never sliced). The
  unreadable-PDF passthrough case is covered by test_pdf_slicing.
- add a bisect_slice-on-PDF regression test: the cut lands on an extracted-text line
  boundary and the halves tile the slice exactly.
2026-08-21 16:57:29 +01:00
abhay-codes07 4faa86c8d5 fix(llm): slice oversized PDFs through the converter (#2906)
An oversized PDF was sliced as raw container bytes, so slice offsets indexed binary data
instead of the extracted text the prompt carries. Introduce unit_source_text so both the
oversized-file expansion and per-slice reads index the same extracted text, and make PDFs
splittable through the converter.
2026-08-21 16:51:17 +01:00