* BeanBrowseResource.java
offers browsing (and limited editting of simple values) of beans in built crawl
* JobRelatedResource.java, ReportGenResource.java
refactor for reuse by other job sub-resources
* JobResource.java
link to job/beans and job/script
* DescriptorUpdater.java
helper for marking-up PropertyDescriptors to aid UI bean access
* CrawlJob.java
don't auto-instantiate context on request
* EngineApplication.java
add uri routes
* FetchHTTP.java, BdbCookieStorage.java, LongWrapper.java WorkQueueFrontier.java, BdbUriUniqFilter.java
improve display when browsed (esp. before launch)
* Sheet.java
make non-abstract; allow for name that might not be beanName
* SheetAssociation.java
define targets as list of names, rather than Sheets themselves
* SheetForDecideRuled.java -> DecideRuledSheetAssociation.java; SheetForSurtPrefixes.java -> SurtPrefixesSheetAssociation.java
make association separate from Sheet -- so same Sheet can be associated more than one place, more than one way
* SheetOverlaysManager.java
match changes to Sheet and SheetAssociations
offer convenience methods for post-construction addition of Sheets, associations
use internal name-to-Sheet mapping, allowing for mid-crawl Sheet additions not in bean factory
* Precedence1SelfTest.java, Precedence4SelfTest.java
update configs for Sheet changes
* ReportGenResource.java
restlet to generate-report-to-file and redirect to view file
* JobResource.java
links to reports from appropriate places for built job
clarify profiles, simplify into-job-dir path-construction
* CrawlJob.java
remove leading '/' from job-relative paths
* EngineApplication.java
install ReportGenResource at appropriate in-job path
* StatisticsTracker.java
fortify for reporting even pre-start/post-stop
add new report types, new groupings: ALL_, LIVE_, END_
dump END_ reports at stop()
dump report by short (class) name
Fix for [HER-1550] set-source-tag option nonfunctional
* CrawlURI.java
ensure 'heritable' key is itself heritable; also change heritable collection to a set
Fix for [HER-1533] bottleneck in StatisticsTracker.saveSourceStats
* StatisticsTrackerImpl.java
saveSourceStats() -- call put() on the CachedBdbMap only when entry is
added, not on every change
* FrontierJournal.java
refactor to always operate on CrawlURI instances (with hops-path and via info), and to write longer info on success/failure lines
* AbstractFrontier.java
change boolean 'respect-crawl-delay' to integer 'respect-crawl-delay-up-to-secs'
Fix for [HER-1516] in datamodel.Robotstxt, Integer.parseInt generates NumberFormatException on floating point Crawl-delay strings
* RobotsDirectives.java, RobotsTxt.java, RobotsExclusionPolicy.java
parse, store, return crawlDelay as float
* RobotsTxtTest.java
compare value as float
[HER-1510] meta refresh url from a seed considered as a seed
* LinksScoper.java
(considerAsSeed) return true any seed with hop type Hop.REFER, which
includes meta refreshes; also simplify the code a bit
* SelfTestBase.java
don't allow negative random passwords, in case leading '-' is causing problems
* Heritrix.java
echo arguments when showing usage info -- to understand what went wrong
gojomo