* BasicProfileTest.java, SelfTestBase.java, CrawlControllerTest.java, PrecedenceLoader.java, MigrateH1to3Tool.java, CrawlerLoggerModule.java, StatisticsTracker.java, CheckpointUtils.java, BdbUriUniqFilter.java, ARCWriterProcessorTest.java, WARCWriterProcessorTest.java, PersistProcessor.java, WriterPoolProcessor.java, PrefixFinderTest.java, StoredQueueTest.java, FileUtilsTest.java, ObjectIdentityBdbManualCacheTest.java, ObjectIdentityBdbCacheTest.java, ObjectPlusFilesOutputStream.java, TestUtils.java, TmpDirTestCase.java, Engine.java
Replace calls to File.mkdirs() with FileUtils.ensureWriteableDirectory(dir). In these cases the calls were either already in a spot where the possible IOException would be handled appropriately, or the line was trivially moved into such a block.
* ActionDirectory.java, Engine.java
replace calls to File.mkdirs() with FileUtils.ensureWriteableDirectory(dir), and throw IllegalStateException on failure
* BdbModule.java
setup() - replace call to File.mkdirs() with FileUtils.ensureWriteableDirectory(dir) and add "throws IOException" - conveniently the place where this method is called was already in a try block that catches IOException
* Checkpoint.java
generateFrom() - replace call to File.mkdirs() with FileUtils.ensureWriteableDirectory(dir) and add "throws IOException"
* CheckpointService.java
move call to Checkpoint.generateFrom() inside existing try block since it now can throw IOException
* Recorder.java
ensure(File) - replace call to File.mkdirs() with FileUtils.ensureWriteableDirectory(dir), and throw IllegalStateException on failure
new Recorder(File,String,int,int) - call ensure() on the correct object, the containing directory; and remove redundant call to ensure()
offer html
* XmlMarshaller.java
new class to write xml based on nested map structure
* BeanBrowseResource.java ScriptResource.java EngineResource.java
JobResource.java JobRelatedResource.java
support xml representation
* WorkQueueFrontier.java PrecedenceProvider.java WorkQueue.java ToeThread.java
ToePool.java CrawlJob.java Engine.java Frontier.java CrawlController.java
StatisticsTracker.java CrawledBytesHistotable.java FetchStats.java
ProcessorChain.java CrawlURI.java Reporter.java
offer same data that is returned in the single line reports in a map
suitable for use with XmlMarshaller
* FileUtils.java
new method tryToCanonicalize(File)
* IoUtils.java
remove class that confusingly-overlaps with commons IOUtils; move remaining methods to ArchiveUtils or archive's FileUtils
* ArchiveUtils.java, FileUtils.java
receive relocated methods; eliminated deprecated or unused methods
* JSONUtils.java
new class to collect common JSON actions
* Iteratorable.java
wrap Iterator as Iterable for foreach usage
* (many)
update to use alternate utils methods
Fix for [HER-1520] Windows mapped-files cause intermittent ReplayCharSequenceTest.testReplayCharSequenceByteToStringMulti failures, risk other problems
* GenericReplayCharSequence.java
use more diverse filenames, so subsequent decodes don't run into lingering open mapped sections of previous decodes
do delete via new FileUtils.deleteSoonerOrLater utility
* FileUtils.java
(deleteSoonerOrLater) try immediate delete, but if fails, add to a list of files to be retried later