mirror of
https://github.com/internetarchive/heritrix3.git
synced 2026-07-15 20:17:20 +00:00
gojomo
8d1083922d
* .classpath, pom.xml, org/apache/commons/pool/impl/*
remove references to and patched classes based on commons-pool
* WriterPool.java
move instance creation/round-robin-pool/dispose into this class
close (rather then recycle) extra writers when crawl slows (via tracking time of last need/rollover)
* WriterPoolMember.java
add isOversize() for external check of ready-to-rollover file
avoid rollover before every record (so that set of related WARC records aren't split between files)
* ARCWriterPool.java, WARCWriterPool.java
remove BasePoolableObjectFactory reference, implement class-appropriate makeWriter()
* ARCWriterPoolTest.java
adjust for new never-timeout behavior
* ARCWriterPoolProcessor.java WARCWriterPoolProcessor.java
adjust constructor calls
* WriterPoolProcessor.java
replace poolMaxWaitMs (which caused eventual complete timeout from waiting for writer) with maxWaitForIdleMs, a much smaller value which only controls how long a thread waits for an reusable writer before considering creating a new one. (If creation isn't allowed, threads will now wait indefinitely for a writer to become available.)