heritrix3

mirror of https://github.com/internetarchive/heritrix3.git synced 2026-08-23 15:06:46 +00:00

T

nlevitt 3cfc315de9 Fix bug reported by Travis where each url in action directory ".include" file is reported as "not canonicalized" in heritrix_out, and some end up crawled in spite of being considered included

* WorkQueueFrontier.java
    considerIncluded(CrawlURI) - call FrontierPreparer.prepare(CrawlURI) so that CrawlURI is canonicalized etc before being considered included (similar to what schedule() does)
* CrawlURI.java
    getCanonicalString(), getPolitenessDelay() - use logger instead of System.err

2011-08-02 01:07:11 +00:00

.settings

commit project-specific eclipse settings

2009-11-11 02:05:30 +00:00

commons

Fix HER-1913 cookies not being sent

2011-07-26 19:07:26 +00:00

dist

Post 3.1.0-RC1

2011-07-26 03:42:02 +00:00

engine

Fix bug reported by Travis where each url in action directory ".include" file is reported as "not canonicalized" in heritrix_out, and some end up crawled in spite of being considered included