mirror of
https://github.com/internetarchive/heritrix3.git
synced 2026-10-09 13:51:31 +00:00
stats & logging checkpoint support
* StatisticsTracker.java
implement Checkpointable; save small state in JSON, recycle BDB data when recovering
* CrawlerLoggerModule.java
implement Checkpointable; rotate logs on checkpoint
* GenerationFileHandler.java
avoid clobbering leftover logs from earlier futures via moveAside
* CrawlStatSnapshot.java
(sameProgressAs) test for lack-of-progress
* ObjectIdentityMemCache.java, TopNSet.java
offer deeper access to assist checkpoint/resume