lee/heritrix3
1
0
Fork 0
mirror of https://github.com/internetarchive/heritrix3.git synced 2026-08-03 21:31:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
340 Commits 40 Branches 31 Tags
bf2eaef6e2f9c9c78bf9515591d343386b2c1b79
Commit Graph
17 Commits
Author SHA1 Message Date
szznax bf2eaef6e2 Part of fix for [HER-1725] on engine page, 'create new job' form gives NPE 
* Engine.java
  made protected getProfileCxmlResource() for clear testability
* EngineTest.java
  added testGetProfileCxmlResource()
 2009-12-11 00:47:06 +00:00
gojomo e42e2ed7f6 Cleanup/consolidation/expansion of *Utils classes 
* IoUtils.java
    remove class that confusingly-overlaps with commons IOUtils; move remaining methods to ArchiveUtils or archive's FileUtils
* ArchiveUtils.java, FileUtils.java
    receive relocated methods; eliminated deprecated or unused methods
* JSONUtils.java
    new class to collect common JSON actions
* Iteratorable.java
    wrap Iterator as Iterable for foreach usage
* (many)
    update to use alternate utils methods
 2009-11-19 22:39:53 +00:00
gojomo 9059395617 Make Bloom size configurable without system properties 
* BloomUriUniqFilter.java
    make bloom size configurable via bean properties
    change implementing class to BloomFilter64bit (given growing prevalance of 64bit JVMs and larger heaps)
* BloomUriUniqFilterTest.java
    update test for property-set sizing
* BloomFilter64bit.java
    improve comment
 2009-11-16 22:42:18 +00:00
gojomo 0497efbb32 [HER-1561] update license to Apache License 2.0 
* (many)
    update license boilerplate
* (few)
    delete no-longer used classes
 2009-11-10 21:03:27 +00:00
gojomo dbc1c1b411 * KeyWordProcessor.java 
update shouldProcess() to be based on via CrawlURI's type
 2009-10-06 02:28:47 +00:00
gojomo 4ce8aef47a Remove obsolete test class 2009-10-06 01:40:22 +00:00
gojomo 047a157fe8 [HER-1605] Refactor parameterizable policies out of frontier 
* selftest-crawler.beans.cxml, FormAuthSelfTest.java, Precedence[134]SelfTest.java
    update self-tests for new chains refactoring
 2009-10-05 23:44:30 +00:00
gojomo 16538f175c Cleanup, refactoring 
* (many)
   update license notice to Apache where appropriate
   eliminate ProcessorURI, DefaultProcessorURI in favor of CrawlURI (now in modules package)
   other comment/warning/unused code cleanup
 2009-10-05 22:49:19 +00:00
gojomo 4d06b239ee Push CrawlURI and related constants down to modules subproject/package 
(many)
 2009-08-06 20:22:00 +00:00
gojomo 4251e5311f * SelfTestBase.java 
report full path of unclosed ARC for manual checking -- truly unclosed or timing issue?
 2009-07-08 01:21:49 +00:00
gojomo 4583ff7306 Rename Rrc2617Credential to HttpAuthenticationCredential 
* (multiple)
    use more-familiar name for HTTP authentication
 2009-07-06 22:09:52 +00:00
gojomo 771d23240c refactor sheets to better enable mid-crawl changes 
* Sheet.java
    make non-abstract; allow for name that might not be beanName
* SheetAssociation.java
    define targets as list of names, rather than Sheets themselves
* SheetForDecideRuled.java -> DecideRuledSheetAssociation.java; SheetForSurtPrefixes.java -> SurtPrefixesSheetAssociation.java
    make association separate from Sheet -- so same Sheet can be associated more than one place, more than one way
* SheetOverlaysManager.java
    match changes to Sheet and SheetAssociations
    offer convenience methods for post-construction addition of Sheets, associations
    use internal name-to-Sheet mapping, allowing for mid-crawl Sheet additions not in bean factory
* Precedence1SelfTest.java, Precedence4SelfTest.java
    update configs for Sheet changes
 2009-06-05 02:12:32 +00:00
gojomo 81a9a9f0f3 Fix unit test 
* BadURIsStopPageParsingSelfTest.java
    update expected URIs to match H2 branch / fixed code
 2009-05-15 00:27:27 +00:00
gojomo 2c11eb1dc2 Remove cruft comments 2009-05-12 19:50:05 +00:00
gojomo 8de6b2a9c5 Fix for [HER-1502] selftests break if 'localhost' does not resolve to 127.0.0.1 
* (all)
    change 'localhost' to '127.0.0.1'
 2009-05-12 18:40:18 +00:00
gojomo c8a5446695 Test robustness 
* SelfTestBase.java
    don't allow negative random passwords, in case leading '-' is causing problems
* Heritrix.java
    echo arguments when showing usage info -- to understand what went wrong
 2009-05-12 02:10:37 +00:00
gojomo e047bf68e0 copy 'springy' branch to heritrix3 2009-05-11 22:56:36 +00:00