lee/heritrix3
1
0
Fork 0
mirror of https://github.com/internetarchive/heritrix3.git synced 2026-09-22 13:46:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
11 Commits 38 Branches 33 Tags
2dec5dfd1a00eccc7eb8bb43106f8fbfc72834e7
Commit Graph
11 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
gojomo 2dec5dfd1a Fix for [HER-1268] ExtractorHTML not able to extract some links (due to them violating RFC2396) 
* ExtractorHTMLTest.java, UURIFactoryTest.java
    test that relative-URIs with late-position colons aren't interpreted as absolute URIs with long, illegal schemes
* UURIFactory.java
    update RFC2396REGEX to require legal scheme
* LaxURI.java
    allow illegal candidate scheme to just be interpreted as something else
 2009-05-12 22:22:11 +00:00
gojomo 252cf63863 Fix for [HER-1524] speculativeFixup in ExtractorJS should maintain URL scheme 
* ExtractorHTMLTest.java
    test for maintaining 'https' when speculative, leading-domain URI is on same host
* ExtractorJS.java
    (speculativeFixup) maintain origin URI scheme if apparent target host is same

Fix for [HER-1526] SCRIPT writing script TYPE common trigger of bogus links (eg. 'text/javascript')
Fix for [HER-1611] more javascript false-positives from Google Analytics insert
* ExtractorJS.java
    port of Steve's work in h2: blacklist for URIs, extracted strings to handle common mis-extractions
* ExtractorHTMLTest.java
    port of test for 'text/javascript' false extract
 2009-05-12 22:09:22 +00:00
gojomo 2a73339211 Forward port for [HER-1525] Bump WARC version to match latest (0.18, June-2008) ISO draft 
* WARCConstants.java, WARCWriter.java, WARCWriterProcessor.java
    remove/update references to pre-0.18 versions or old class names
 2009-05-12 19:51:16 +00:00
gojomo 2c11eb1dc2 Remove cruft comments 2009-05-12 19:50:05 +00:00
gojomo fb97744dfd Fix for [HER-1495] ARCRecord throws null pointer when trying to generate a RecoverableIOException 
* ARCRecord.java
    guard against bull statusBytes in exception message
 2009-05-12 19:43:20 +00:00
gojomo 8de6b2a9c5 Fix for [HER-1502] selftests break if 'localhost' does not resolve to 127.0.0.1 
* (all)
    change 'localhost' to '127.0.0.1'
 2009-05-12 18:40:18 +00:00
gojomo 3f7d85d7aa Fix for [HER-1528] NumberFormatException in UURIFactory.checkPort() 
* UURIFactoryTest.java
    test that port-looking-part that's out of integer bounds throws URIException not NumberFormatException
* UURIFactory.java
    catch NumberFormatException; allow to surface as URIException
 2009-05-12 18:21:17 +00:00
gojomo 4082195690 Fix for [HER-1529] AggressiveExtractorHTML reports back as ExtractorHTML2 in processor reports 
* AggressiveExtractorHTML.java
    correct self-reported name
 2009-05-12 18:04:14 +00:00
gojomo c8a5446695 Test robustness 
* SelfTestBase.java
    don't allow negative random passwords, in case leading '-' is causing problems
* Heritrix.java
    echo arguments when showing usage info -- to understand what went wrong
 2009-05-12 02:10:37 +00:00
gojomo daf5ed7590 Update pom.xml * 4 
more naming improvements
    ignore test cruft
    update SCM url
 2009-05-12 00:17:01 +00:00
gojomo e047bf68e0 copy 'springy' branch to heritrix3 2009-05-11 22:56:36 +00:00