Noah Levitt b137022b79 Include in checkpoint data a list of ".open" WARCs and their current raw on-disk length, so the exact extent of pre-checkpoint content can be deduced/reconstructed. Also when flushing or closing writers, block to wait on any that remain active. (In context of heritrix crawl this is not normally necessary because of the dispositionInProgressLock managed by the frontier, but seems like a good idea in case WriterPool is used elsewhere.)

2013-10-28 17:35:55 -07:00

..

src

Include in checkpoint data a list of ".open" WARCs and their current raw on-disk length, so the exact extent of pre-checkpoint content can be deduced/reconstructed. Also when flushing or closing writers, block to wait on any that remain active. (In context of heritrix crawl this is not normally necessary because of the dispositionInProgressLock managed by the frontier, but seems like a good idea in case WriterPool is used elsewhere.)

2013-10-28 17:35:55 -07:00

pom.xml

update source/target version to 1.6 since we use 1.6 features, specifically @Override for methods only implementing an interface (not sure why maven wasn't enforcing this); also remove maven-antrun-plugin created timestamp.txt, doesn't seem to be used anywhere

2013-03-20 14:25:22 -07:00