Files
heritrix3/modules
Alex Osborne 7a9e2689b4 Use CrawlURI.hasContentType() for exact media type checks
Replaces startsWith() prefix checks, which also matched longer subtypes,
e.g. "application/vnd.wap.wml" matched the binary
"application/vnd.wap.wmlc". The truncated "application/xhtml" and
"application/vnd.wap.xhtml" checks now name the full "+xml" types. Also
fixes case-sensitive comparisons in ExtractorPDF and ExtractorPDFContent,
and drops BotBlockDetector's private equivalent.

Substring heuristics like ExtractorJS's indexOf("javascript") and
ExtractorXML's indexOf("xml") are intentionally left as they are.
2026-11-05 09:58:58 +09:00
..