lee/heritrix3
1
0 Fork 0
mirror of https://github.com/internetarchive/heritrix3.git synced 2026-11-14 09:50:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
283 Commits 47 Branches 34 Tags
fbd3af35ac1f4fd6f5e2fc21ef5811cd38b8d43f
Commit Graph
2 Commits
Author SHA1 Message Date
gojomo f01ad9ada8 [HER-1546] Springify(5): Update checkpointing to work smoothly with spring-configured crawls 
stats & logging checkpoint support
* StatisticsTracker.java
    implement Checkpointable; save small state in JSON, recycle BDB data when recovering
* CrawlerLoggerModule.java
    implement Checkpointable; rotate logs on checkpoint
* GenerationFileHandler.java
    avoid clobbering leftover logs from earlier futures via moveAside
* CrawlStatSnapshot.java
    (sameProgressAs) test for lack-of-progress
* ObjectIdentityMemCache.java, TopNSet.java
    offer deeper access to assist checkpoint/resume
 2009-11-19 23:18:15 +00:00
gojomo e047bf68e0 copy 'springy' branch to heritrix3 2009-05-11 22:56:36 +00:00