* Arc2Warc.java
when HTTP response, write response record rather than resource
enable reconstruction of ARC from WARC via namedFields
* ARCRecord.java
added headerString and accessor to save verbatim ARC header string
* CrawlJob.java
added boolean partialJobLogScan, and print "at least X launches" when true
* JobResource.java
print "at least X launches" when partialJobLogScan
* Heritrix.java
added CLAP protocol to access representations via classloader
* EngineApplication.java
added staticDir resource to serve static files under "/engine/static"
also added org.archive.crawler.restlet package to hold static files
* EngineResource.java
added <link> to static CSS resource
* Engine.css
style for flashACK/NACK/ADVISORY divs
* Engine.java
removed ERROR and WARNING strings, rely on Flash class
log jobsDir check failure as SEVERE and return early
collapse error-handling in considerAsJobPath() and log SEVERE
renamed considerAsJobDirectory() to addJobDirectory()
remove catch-and-rethrow from leaveJobPathFile() when writing,
instead log SEVERE
renamed leaveJobPathFile() to writeJobPathFile()
added boolean "userRequest" arg to addJobDirectory(), and moved
call to writeJobPathFile() inside of addJobDirectory()
changed void considerAsJobPath(File) to String getJobPathFromFile
and removed call to addJobDirectory
make seperate calls to addJobDirectory() in findJobConfigs()
* EngineResource.java
restructured "create" logic for clarity
eliminate style code, rely on Flash class
remove limit on "addpath" input, change to "createpath"
removed all-caps and bangs ("!") from error messages
removed platform-specific path separators ("/")
call Engine.addJobDirectory() with userRequest=true in when
form action=add
* **/pom.xml
bump version identifiers to "3.1.1-SNAPSHOT" (odd minor number meaning dev version, odd patch number indicating expected next patch number of 3.0.x release if before a 3.2.0 release)
* AbstractFrontier.java
close recovery log in stop() (comes later, usu. from managerThread) rather than terminate() (earlier, may occur in other threads like progress-stats-snapshot thread)
iPhone Safari, Chrome) use a screwy 'Accept' header that if taken literally
means they prefer 'application/xml' content. Thus trying to view web UI in
these browsers results in seeing the XML version, without element markup
* BaseResource.java
override getPreferredVariant() to bump up preference level of text/html for
any client can accept it
* EngineResource.java, JobResource.java
inherit from BaseResource
* WorkQueueFrontier.java
add snoozedOverflow map of (wakeTime)->(DelayedWorkQueue) for snoozes over a capped size
maintain in parallel to snoozedClassQueues DelayQueue, maintain atomic size count
in standard report, presort snoozedClassQueues before displaying
* DelayedWorkQueue.java
move to top-level so serialization doesn't pull out whole frontier
* BdbFrontier.java
initialize snoozedOverflow
* BdbMultipleWorkQueues.java
improve error message seen on problematic checkpoint-recovery
* EngineResource.java
give add-job options their own header, reorder/reword for clarity
* JobResource.java
return to always showing 'checkpoint' button, disable if necessary, so main control buttons aren't appearing/disappearing or moving
* WriterPoolProcessor.java
change description of WriterPoolProcessor.directory ConfigPath to not imply as strongly that is where writing much be occurring
szznax