mirror of
https://github.com/internetarchive/heritrix3.git
synced 2026-10-08 21:31:20 +00:00
Restlet's Directory gives every file an Expires header 10 minutes in the future. Job files (logs, reports, crawler-beans.cxml) change constantly, so browsers could show stale copies. Worst case, the config editor loads a cached crawler-beans.cxml and saving it silently reverts newer changes. Clear the expiration date and send Cache-Control: no-cache instead, so browsers revalidate against Last-Modified on each use.