* MultiReporter
deleted. direct method calls are a better way to specify the kind of report desired
getReports was never used to choose from the array of strings. no use case for MultiReporter.
* ArchiveUtils
writeReportToString was not used, and it used the now deleted MultiReporter#reportTo
* DevUtils, Frontier, ToePool, AbstractFrontier, WorkQueueFrontier
MultiReporter -> Reporter and make methods that implemented named multireport public
* FrontierNonemptyReport, FrontierSummaryReport
call the appropriate report method instead of passing a string constant
* MultiReporter.java
deprecated
* ToePool, ToeThread, WorkQueue, WorkQueueFrontier, PrecedenceProvider, CrawlURI, FetchStats.java
remove MultiReporter#getReports() and reportTo(String, PrintWriter)
these are not used as MultiReporter specifies, and MultiReporter is going away anyhow
This enables syntax-highlighting, line-numbering and auto-indent for the crawl
configuration editor and the script console (for BeanShell, JavaScript and
Groovy).
* BaseResource.java
add a more generic getStaticRef() in addition to getStylesheetRef()
* EditRepresentation.java
wrap textarea with CodeMirror in purexml mode.
disable the "save changes" button until config is edited.
dynamically resize CodeMirror edit element to fit window height.
* ScriptResource.java
wrap textarea with CodeMirror.
switch CodeMirror language modes depending on selected script engine.
* resources/.../restlet/codemirror/
include just the files from codemirror-2.23.zip necessary for the above
plus the CodeMirror LICENSE file (MIT)
plus a README linking to the codemirror website
- added missing <html> and <body>
- fixed non-working CSS link, added CSS link to all pages
- replaced layout by <br> and with structural tags with CSS
- removed superfluous output, notably in BeanBrowseResource
- <code>-ify variable names in ScriptResource
- added special handling of NaN in doubleToString()
* SurtPrefixedDecideRule.java
make class implement Checkpointable, save surt prefixes to json on checkpoint and load them on recover
* profile-crawler-beans.cxml
make main SurtPrefixedDecideRule a top-level bean so that it can be checkpointed
* Checkpoint.java, CheckpointService.java
add some FINE level logging during checkpointing and recovery
Note that the name of the engine may be different than the name of the language
it parses. A good example is the Rhino script engine that parses javascript.
* PathSharingContext.java
new member variable ConcurrentHashMap data and accessor getData()
* ScriptedProcessor.java, ScriptedDecideRule.java
make appCtx available to scripts; also remove unused member sharedMap
* ActionDirectory.java
formatting fix
default false is old behavior, skip candidates-handling of outlinks from response codes <200 and >=400
if true, these outlinks will be treated the same as others (get scope-tested and enqueued)
BaseQueuePrecedencePolicy.basePrecedence) doesn't stick
(CrawlURI) don't null overlayNames in processingCleanup
(WorkQueueFrontier) move queue precedence recalc to only in handleQueue,
not all (post-wake) reenqueues
an HTTP status between 200 and 299.
Altered the QuotaEnforcer to enforce quotas on the number of URIs that
have an HTTP status between 200 and 299.
Removed confusingly named variables (SUCCESS, SUCCESS_KB, RESPONSES,
RESPONSES_KB) and replaced their references.
* HostsReport.java
write() - remove obsolete code described by this no longer correct comment - "StatisticsTracker doesn't know of zero-completion hosts; so supplement report with those entries from host cache" - StatisticsTracker does know of zero-completion hosts
* SheetOverlaysManager.java
getOverlayMap(String) - return null if sheet missing instead of triggering npe
* KeyedProperties.java
get(String) - check for null return value from getOverlayMap() and log warning
* WorkQueueFrontier.java
activateInactiveQueue(), deactivateQueue() - synchronize around updates of inactiveQueuesByPrecedence and highestPrecedenceWaiting
* Heritrix.java
instanceMain() - move logging of java vendor/version ahead of version check, since the information could be particularly helpful when version check doesn't pass
* CrawlController.java
requestCrawlResume() - do not create ToePool (creation here appeared to be inherited cruft), paused crawl must already have toe pool
Travis Wellman