lee/heritrix3
1
0
Fork 0
mirror of https://github.com/internetarchive/heritrix3.git synced 2026-07-31 03:41:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

[HER-747] Ensure 'retired' status for queues is operator-controllable

Browse Source 
[HER-1768] Changing queue totalBudget not retiring queue as expected
* CrawlURI.java
    (includesRetireDirective) internalize to CrawlURI
* AbstractFrontier.java
    allow re-enqueuing of untried URI with retire-driective
    clean up terminology/method-names
* WorkQueue.java, WorkQueueFrontier.java, BdbFrontier.java
    adjust budget/sessionBalance handling to always count up, always consult latest values, always refresh values from latest URI
    (processFinish) reorganize to reduce redundancy
* DispositionProcessor.java
    offer forceRetire setting
* QuotaEnforcer.java
    set failed-status *or* forceRetire, not both
[HER-1823] H3: adding SheetAssociations mid-crawl won't affect already queued CrawlURIs (sheet liveness) 
* SheetsOverlayManager.java
    (applyOverlaysTo) rename, unify, reapply whenever called to ensure liveness
* CandidatesProcessor.java
    adjust to rename
This commit is contained in:
gojomo
2010-09-21 10:55:07 +00:00
parent c701cd391a
commit 1ce23ecbcd
9 changed files with 215 additions and 222 deletions
Download Patch File Download Diff File Expand all files Collapse all files
modules/src/main/java/org/archive/modules/CrawlURI.java
+5
View File
@@ -1863,4 +1863,9 @@ implements MultiReporter, Serializable, OverlayContext {
// deferrals from now don't count against future try
resetDeferrals();
}
public boolean includesRetireDirective() {
return containsDataKey(A_FORCE_RETIRE)
&& (Boolean)getData().get(A_FORCE_RETIRE);
}
}