mirror of
https://github.com/internetarchive/heritrix3.git
synced 2026-11-22 13:47:02 +00:00
getting-started: - Use a Java 17 path in the JAVA_HOME example (17 is the minimum) - Link to the local Configuring Jobs page instead of the wiki - Fix typos operating: - Correct --jobs-dir option name and explain its default - Document --proxy-host, --proxy-port and --sni-host-check - Fix option list formatting that broke rendering (-c, --web-auth) - Remove contradictory note about credentials in heritrix_out.log - crawl.log fetch duration is separated by '+', not ';' - Renumber progress-statistics.log fields (field 3 was duplicated) - Fix sourceTagSeeds property name in the XML example - ActionDirectory is already in the default profile - Use the correct frontier.recover.gz file name throughout - The action directory is inside the job directory, not beside bin/ configuring-jobs: - Fix maxDocumentsDownload, obeyRelNofollow and logger class names - Replace removed/renamed decide rules with their current names - Fix javadoc links for decide rules in subpackages and in engine - Fix malformed useHTTP3 XML example and FTP/SFTP seed examples - Replace dead Confluence glossary link - Drop DoH caveats that only applied to Java 11 / DisableExplicitGC api: - Fix Create Job field list not rendering (missing blank line) - Script engines: list Groovy/BeanShell; js and AppleScript are not available on supported Java versions - Make JSON sample response consistent with the XML one glossary: - SHA-1 is used for content digests, not encryption - Fix garbled Host entry (logical vs physical host) - Fix SURT and SURT prefix examples - Status -6 refers to DNS, not WHOIS; add -9, 1001, 2001, 2002 - Link status codes to RFC 9110 instead of RFC 2616
Heritrix Documentation
This directory contains structured documentation we're in the process of migrating and reformatting. It can be viewed online.
The main bulk of Heritrix's documentation currently lives on the Github wiki.