mirror of
https://github.com/internetarchive/heritrix3.git
synced 2026-11-22 13:47:02 +00:00
Replaces startsWith() prefix checks, which also matched longer subtypes,
e.g. "application/vnd.wap.wml" matched the binary
"application/vnd.wap.wmlc". The truncated "application/xhtml" and
"application/vnd.wap.xhtml" checks now name the full "+xml" types. Also
fixes case-sensitive comparisons in ExtractorPDF and ExtractorPDFContent,
and drops BotBlockDetector's private equivalent.
Substring heuristics like ExtractorJS's indexOf("javascript") and
ExtractorXML's indexOf("xml") are intentionally left as they are.