vonrosen
f2bc21435f
Merge pull request #71 from nlevitt/checkpoint-no-ui-hang
...
avoid calling CheckpointService.hasAvailableCheckpoints() when crawl not...
2014-05-22 16:44:22 -07:00
Noah Levitt
39814d1efd
avoid calling CheckpointService.hasAvailableCheckpoints() when crawl not launchable and therefore the info is not needed, because it calls isRunning() which is synchronized and blocks when checkpoint is in progress
2014-05-22 16:40:44 -07:00
vonrosen
1c0d6c60fb
Merge pull request #70 from nlevitt/ari-3712
...
for ARI-3712, add extracted links relative to both via and base, and annotate with "extractorSWFRelToVia", "extractorSWFRelToBase", or "extractorSWFRelToBoth" if resulting link is the same whether relative to base or via
2014-05-22 10:58:21 -07:00
Noah Levitt
26222ef660
Merge pull request #69 from vonrosen/ari-3853
...
For https://webarchive.jira.com/browse/ARI-3865
2014-05-15 17:38:41 -07:00
Hunter Stern
d570498fd9
New Unit tests added now.
2014-05-15 17:37:26 -07:00
Hunter Stern
80880b4409
Unit test and other changes.
2014-05-15 17:34:43 -07:00
Noah Levitt
aff5b9ac4e
for ARI-3712, add extracted links relative to both via and base, and annotate with "extractorSWFRelToVia", "extractorSWFRelToBase", or "extractorSWFRelToBoth" if resulting link is the same whether relative to base or via
2014-05-15 13:31:29 -07:00
Noah Levitt
519717fd7b
make addRelativeToBase() and addRelativeToVia() return the link added, or null; copy getAnnotations() from CrawlURI
2014-05-15 13:27:50 -07:00
Hunter Stern
f24cfff6af
New youtube channel video extractor for https://webarchive.jira.com/browse/ARI-3865
2014-05-13 17:41:42 -07:00
Adam Miller
3e213c82bc
Merge pull request #67 from nlevitt/umbra
...
don't log severe with stack trace on normal amqp shutdown
2014-05-13 14:44:09 -07:00
Noah Levitt
08a4aafa23
don't log severe with stack trace on normal amqp shutdown
2014-05-13 14:39:35 -07:00
Noah Levitt
38cd14ecac
Merge pull request #63 from adam-miller/ExtractorPDFContent
...
Contrib - ExtractorPDFContent
2014-05-13 14:09:20 -07:00
Adam Miller
aaecd6d78c
Merge branch 'master' of git://github.com/internetarchive/heritrix3 into ExtractorPDFContent
2014-05-13 14:04:29 -07:00
Adam Miller
fb42c07c92
bugfix, new test case.
2014-05-13 14:03:03 -07:00
Adam Miller
ab7a60f9e3
Merge pull request #66 from nlevitt/cosmetics
...
oops, make "exit java process" button work again
2014-05-13 13:31:11 -07:00
Adam Miller
85e7a5ee78
Bugfix & new test cases.
2014-05-12 16:52:32 -07:00
Noah Levitt
ba68649ba0
Merge remote-tracking branch 'origin/master' into cosmetics
2014-05-07 10:02:06 -07:00
Noah Levitt
b7f8a0669f
Merge branch 'cosmetics' of github.com:nlevitt/heritrix3 into cosmetics
2014-05-07 10:02:00 -07:00
Noah Levitt
33e782e2fd
oops, make "exit java process" button work again
2014-05-07 10:01:47 -07:00
Adam Miller
eb32d7a8b8
Merge pull request #65 from nlevitt/umbra
...
shut down the starter-restarter thread at crawl finish!!
2014-05-02 15:47:44 -07:00
Noah Levitt
949c9f086c
shut down the starter-restarter thread at crawl finish!!
2014-05-02 15:42:29 -07:00
Noah Levitt
dd510cbf3b
Merge pull request #64 from adam-miller/ViaSurtPrefixedDecideRule
...
Via surt prefixed decide rule
2014-05-01 18:14:25 -07:00
Adam Miller
ae878d29cf
Code cleanup
2014-05-01 15:22:16 -07:00
Adam Miller
f2bebd41b0
Cleanup / Optimization
2014-05-01 12:48:23 -07:00
Adam Miller
753404f662
Adding ViaSurtPrefixedDecideRule. Returns true if a surtform 'via' matches any surt in a specified list
2014-05-01 12:43:26 -07:00
Adam Miller
2b1c330be6
Further cleanup
2014-04-30 12:18:14 -07:00
Adam Miller
e6d646d4b4
Cleanup / optimization
2014-04-30 12:14:32 -07:00
Adam Miller
34a9172e69
Organize Imports
2014-04-29 17:25:43 -07:00
Adam Miller
5181c1ef64
Cleaning up test cases and removing unused code
2014-04-29 17:11:18 -07:00
Adam Miller
33277445f6
Merge branch 'master' of git://github.com/internetarchive/heritrix3 into ExtractorPDFContent
2014-04-29 17:06:55 -07:00
Adam Miller
49069f2dd1
Adding comment for library dependencies
2014-04-29 17:06:11 -07:00
Adam Miller
7e8a66e7d8
New Test cases, fixing bugs
2014-04-29 10:50:04 -07:00
vonrosen
747bbeba97
Merge pull request #62 from nlevitt/ari-3765
...
Ari 3765 gracefully handle amqp server going up and down
2014-04-28 09:39:37 -07:00
Kenji Nagahashi
50a38e1376
Merge pull request #61 from nlevitt/her-2065
...
HER-2065 synchronize on inactiveQueuesByPrecedence inside of synchronize...
2014-04-25 11:55:14 -07:00
Noah Levitt
381442dfb6
ARI-3765 - make AMQPUrlReceiver gracefully handle amqp server going up and down
2014-04-24 21:23:03 -07:00
Noah Levitt
10e0f262b3
ARI-3765 - make AMQPPublishProcessor gracefully handle amqp server going up and down
2014-04-24 20:31:40 -07:00
Noah Levitt
d777c198bd
HER-2065 - in activateInactiveQueue() close synchronized block earlier, and move updateHighestWaiting() up to keep it inside the block; now the call to deactivateQueue() is not in the synchronized block, so the deadlock should be avoided (fix suggested by kenji)
2014-04-24 17:23:01 -07:00
Adam Miller
3a456638ca
Merge pull request #60 from nlevitt/cosmetics
...
Cosmetics
2014-04-21 15:52:43 -07:00
Noah Levitt
0cd1043cb1
change memory bar color, use consistent non-capitalization on buttons
2014-04-21 11:53:40 -07:00
Noah Levitt
1b72ce5dd8
remove extra </div>
2014-04-21 11:53:05 -07:00
Noah Levitt
cfb9474f7a
cosmetic changes
2014-04-21 09:06:15 -07:00
Adam Miller
a48a29f9bd
Adding ExtractorPDFContent to find nav links from the text content of PDFs
2014-04-14 16:02:35 -07:00
vonrosen
67a7a4fb6c
Merge pull request #59 from nlevitt/ari-3723
...
fix unit test now that we accept speculative urls with query params with...
2014-04-10 20:22:15 -07:00
Noah Levitt
e73810d0a6
fix unit test now that we accept speculative urls with query params with no value
2014-04-10 17:25:27 -07:00
Eldon
5e75ba6506
Merge pull request #57 from nlevitt/umbra
...
AMQPUrlReceiver - improve handling of case where rabbitmq is unreachable...
2014-04-10 15:35:48 -04:00
vonrosen
63c4264de6
Merge pull request #58 from nlevitt/ari-3723
...
for ARI-3723, accept speculative urls with query params with no value
2014-04-10 12:09:16 -07:00
Noah Levitt
339e6ec87a
for ARI-3723, accept speculative urls with query params with no value
2014-04-04 10:15:07 -07:00
Noah Levitt
ac9d8e6ba7
AMQPUrlReceiver - improve handling of case where rabbitmq is unreachable at crawl startup - spawn off a thread to start up the amqp consumer to avoid the crawl launch thread getting stuck
2014-04-03 11:39:36 -07:00
vonrosen
030063f7b3
Merge pull request #56 from nlevitt/fix-form-login
...
fix FormLoginProcessor checkpointing
2014-04-03 09:10:44 -07:00
Noah Levitt
36a207ce07
fix FormLoginProcessor checkpointing
2014-04-02 17:33:09 -07:00
vonrosen