I don't know why we've never seen this before, and now we suddenly have
a case of it, but this is the exception:
2018-07-23 17:47:01.123 SEVERE thread-2875088 org.archive.crawler.framework.CrawlJob.beansException() Failed to start bean 'scope'; nested exception is java.lang.IllegalStateException: java.nio.file.NoSuchFileException: /1/ait-h3-jobs/8144-20180723162745141/20180723174701/logs/scope.log.lck
org.springframework.context.ApplicationContextException: Failed to start bean 'scope'; nested exception is java.lang.IllegalStateException: java.nio.file.NoSuchFileException: /1/ait-h3-jobs/8144-20180723162745141/20180723174701/logs/scope.log.lck
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:169)
at org.springframework.context.support.DefaultLifecycleProcessor.access$1(DefaultLifecycleProcessor.java:154)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:335)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:143)
at org.springframework.context.support.DefaultLifecycleProcessor.start(DefaultLifecycleProcessor.java:89)
at org.springframework.context.support.AbstractApplicationContext.start(AbstractApplicationContext.java:1236)
at org.archive.spring.PathSharingContext.start(PathSharingContext.java:115)
at org.archive.crawler.framework.CrawlJob.startContext(CrawlJob.java:455)
at org.archive.crawler.framework.CrawlJob$1.run(CrawlJob.java:428)
Caused by: java.lang.IllegalStateException: java.nio.file.NoSuchFileException: /1/ait-h3-jobs/8144-20180723162745141/20180723174701/logs/scope.log.lck
at org.archive.crawler.reporting.CrawlerLoggerModule.setupSimpleLog(CrawlerLoggerModule.java:298)
at org.archive.modules.deciderules.DecideRuleSequence.start(DecideRuleSequence.java:174)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:166)
... 8 more
Caused by: java.nio.file.NoSuchFileException: /1/ait-h3-jobs/8144-20180723162745141/20180723174701/logs/scope.log.lck
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:335)
at java.util.logging.FileHandler.openFiles(FileHandler.java:478)
at java.util.logging.FileHandler.<init>(FileHandler.java:344)
at org.archive.io.GenerationFileHandler.<init>(GenerationFileHandler.java:63)
at org.archive.io.GenerationFileHandler.makeNew(GenerationFileHandler.java:158)
at org.archive.crawler.reporting.CrawlerLoggerModule.setupLogFile(CrawlerLoggerModule.java:275)
at org.archive.crawler.reporting.CrawlerLoggerModule.setupSimpleLog(CrawlerLoggerModule.java:296)
... 10 more
ContentDigestHistoryTest does not close BdbModule. It results in failure to delete bdb directory in following tests.
Added tearDown() method that closes BdbModule.
Since we were only enforcing the size limit after reading each
line a single very long line could cause us to to run out of
memory.
This change enforces the character limit on each read character
not just on line boundaries.
We also correct the count of processed characters in the warning
message, which was not counting newline and linefeed characters.
* origin/master:
Setup TravisCI
Set fetch status on curis when testing link extraction
No link extraction on URI not successfully downloaded
Fixes issue #158 : Noisy alerts about 401s without auth challenge (#159)
Make Content-Location header url INFERRED not REFFER hop type since Content-Location is not for redirection (#151)
fixes for kafka 0.9 (?)
upgrade to kafka 0.9
somewhat ugly fix to handle exceptions from the bean browser like java.lang.RuntimeException: not implemented at org.archive.modules.fetcher.BdbCookieStore$RestrictedCollectionWrappedList.get(BdbCookieStore.java:92)
* also fix new FormLoginProcessorTest.testFormLoginExtraInputs() in context of form-login-multipart changes
* master:
fix NullPointerException when using old HtmlFormCredential login
to make this branch only about multipart/form-data, undo the changes to handle more complicated forms with extra input fields; add unit test; store CrawlURI submit data in http-library agnostic way and remove httpcomponents-specific code from HTMLForm, refactor creation of post data accordingly
Be sure to return at least one of the candidates
Remove unneeded constant
Make sure to check form encoding type when addign form to curi data
More changes for https://webarchive.jira.com/browse/ARI-4656
More changes
Save enctype of form in submission curi
Add ability to submit forms with enctype="multipart/form-data"
If multiple username inputs in login form, favor the one with the word 'login' in it.
Noah Levitt