* ConfigPathConfigurer.java
work in two passes: one (BeanPostProcessor) collects all candidates for fixup, two (ApplicationListener) actually performs fixup
suppress fixup of own base path, preventing circulare ref/StackOverflow
* ConfigPath.java
change sense of 'merge' to merge into receiver
* SurtPrefixedDecideRule.java
use ConfigFile, use 'merge' in assignment
* CrawlerLoggerModule.java
use merge in assignment
* CrawlJob.java
(getBeanpathTarget) utility method
* JobResource.java
respect unset ConfigPaths
* ConfigPathEdito.java, ConfigFileEditor.java
better comments, ConfigFile variant
* CrawlController.java
(isFinished) added
* CrawlJob.java
rename reset to teardown
wait for CrawlController to report finished before discarding AppContext
if not finished in time, fail reporting 'false'
* JobResource.java
show Flash if teardown did not succeed; user must retry
* BdbUriUniqFilter.java
don't throw DatabaseException from simple accessor
rename 'discard' to 'teardown'
enable browse/edit of paths outside jobdir via new /anypath/ service
* EngineResource.java, Engine.java
enable addition of job dirs from outside main jobs directory
* EngineApplication.java
register new /engine/anypath service for outside-job-dir paths
Fix for [HER-1568] ARCReader fails on some newer Alexa ARC files
* ARCConstants.java
added ArcRecordErrors enum for known reading errors
* ARCReaderFactory.java
only check for 'LX' extension in GZIP extra field
* ARCRecord.java
added errors, hasErrors(), and getErrors()
* RecordingOutputStream.java
close ReplayInputStream used to create GenericReplayCharSequence
use 'UTF-8' and not possibly-varying JVm default encoding when none specified
* HOWTO-Launch-Heritrix.txt
removed; info mostly about H2 variants
* README.txt
updated wiki link, launch info to match H3
* LICENSE.txt
updated to Apache2
* HopCrossesAssignmentLevelDomainDecideRule.java
port H2 IsCrossTopmostAssignedSurtHopDecideRule to H3 under this new name
* PublicSuffixes.java
update comment, public method for 'Assignment Level Domain'/'Assignment Level SURT' term
* AssignmentLevelSurtQueueAssignmentPolicy
renamed TopmostAssignedSurtQueueAssignmentPolicy
(from nlevitt's H2 commit message:)
* InMemoryReplayCharSequence.java
new, simple ReplayCharSequence that supports any encoding, keeping everything in memory
* GenericReplayCharSequence.java
If the encoding supports random access, memory maps the backing file directly; otherwise decodes to UTF-16 and maps that. Supports the first Integer.MAX_VALUE bytes of the file. Maps up to 64M; moves the map around as necessary for larger files.
* Latin1ByteReplayCharSequence.java
removed, functionality split between InMemoryReplayCharSequence and GenericReplayCharSequence
* ReplayCharSequenceTest.java
xestHugeReplayCharSequence() - uncomment to test a huge replay char sequence
testReplayCharSequenceByteToStringOverflow() - test both UTF-8 and windows-1252
* RecordingOutputStream.java
use the new ReplayCharSequences
* (all, especially CachedBdbMap.java)
pbaclace-contributed patch, ported to H3, plus...
* StatisticsTracker.java
replace LongWrapper with AtomicLong
use non-recursive putIfAbsent-based idiom for get-or-create
* DefaultServerCache.java
clean up host/server get-or-create around non-recursive concurrent idiom
* BdbModule.java
add expectedConcurrency property & je.lock.nLockTables tuning
* TopNSet.java
add synchronization, as calling StatisticsTracker method has lost it
* Heritrix.java
fail-fast with helpful error message if launched with Java < 1.6
* EditRepresentation.java
"save changes" rather than "update content"
* EngineApplication.java
customize restlet error to include back-button and full stack (instead of link to HTTP spec!)
* BeanBrowseResource.java
offers browsing (and limited editting of simple values) of beans in built crawl
* JobRelatedResource.java, ReportGenResource.java
refactor for reuse by other job sub-resources
* JobResource.java
link to job/beans and job/script
* DescriptorUpdater.java
helper for marking-up PropertyDescriptors to aid UI bean access
* CrawlJob.java
don't auto-instantiate context on request
* EngineApplication.java
add uri routes
* FetchHTTP.java, BdbCookieStorage.java, LongWrapper.java WorkQueueFrontier.java, BdbUriUniqFilter.java
improve display when browsed (esp. before launch)
* Sheet.java
make non-abstract; allow for name that might not be beanName
* SheetAssociation.java
define targets as list of names, rather than Sheets themselves
* SheetForDecideRuled.java -> DecideRuledSheetAssociation.java; SheetForSurtPrefixes.java -> SurtPrefixesSheetAssociation.java
make association separate from Sheet -- so same Sheet can be associated more than one place, more than one way
* SheetOverlaysManager.java
match changes to Sheet and SheetAssociations
offer convenience methods for post-construction addition of Sheets, associations
use internal name-to-Sheet mapping, allowing for mid-crawl Sheet additions not in bean factory
* Precedence1SelfTest.java, Precedence4SelfTest.java
update configs for Sheet changes
* BeanShellProcessor.java -> ScriptedProcessor.java; BeanShellDecideRule.java -> ScriptedDecideRule.java
update to use any present JSR-223 scripting engine (requested by free-text name)
update to allow script to come from inline string or file (just as with seeds)
* modules/pom.xml
include groovy and jsr-223-capable JARs
* .classpath
add new JARs to classpath
* BeanShellProcessorTest.java -> ScriptedProcessorTest.java
rename (empty) test class
* DefaultProcessorURI.java
add constructor that supports setting via
* CustomSWFTags.java, CrawlUriSWFAction.java
look for links in a couple more places
* ExtractorSWF.java
ExtractorTagParser.parsePlaceObject2() - read 32 bit clip event
flags when flash version >= 6
* ExtractorSWFTest.java
xestHer1509() - test this fix (rename to enable the test)
* Extractor.java
added instance initializer for DEFAULT_PARAMETERS, and set
@Autowired(required=false) eliminates the need for test
classes to set them explicitly.
* ExtractorCSSTest.java
remove explicit setting of default parameters
* ExtractorHTMLTest.java
remove explicit setting of default parameters
* JerichoExtractorHTMLTest.java
remove explicit setting of default parameters
gojomo