fd514b7bf6
Merge pull request #315 from ukwa/fix-oracle-doc-link
...
Use the Wayback Machine to repair a link to Oracle docs.
2020-03-05 01:40:39 +09:00
b4494a7ffb
Use the Wayback Machine to repair a link to Oracle docs.
2020-03-04 15:42:29 +00:00
348b5330bc
Utilize the d parameter
...
There are uses of this class that necessitate a lower false positive
rate than the value that was hard-coded. By making use of the parameter,
as indicated in the JavaDoc, the resulting delegate should meet the
demands of the caller.
2020-03-04 09:37:31 -05:00
7f80b4fd98
Merge pull request #281 from ukwa/upgrade-bdb-je
...
WIP: Upgrade BDB JE to version 7.5.11
2020-03-04 13:36:03 +00:00
bc6a15b41d
Use HTTPS to resolve dependencies in Maven Build where possible
2020-02-11 17:48:20 +09:00
b45f774066
Merge pull request #300 from hennekey/fix-299
...
Use Guice instead of custom implementation
2020-01-30 19:12:05 +09:00
fd9136dbfa
Merge pull request #304 from hennekey/remove-custom-base32
...
Replace custom Base32 encoding
2020-01-30 19:07:09 +09:00
54e05a7864
Correct encoding
...
The previous implementation appears to always have returned upper case,
was able to encode either case, and did not reutrn padding.
2020-01-28 15:12:54 -05:00
b5f95c5e06
Replace custom Base32 encoding
...
Guava is available so a custom implementation is unnecessary.
2020-01-24 12:46:37 -05:00
9927e1325f
Merge pull request #296 from hennekey/update-junit
...
Set JUnit version to latest
2020-01-23 16:44:59 +09:00
9cb9563da3
Increment the count only when the filter notes it
...
Otherwise this is a count of how many times this add method is called,
not how many times an element was noted as being actually added.
2020-01-22 17:15:24 -05:00
66723aea68
Remove version
...
Allow the parent POM to specify the version
2020-01-21 10:52:15 -05:00
080380b18a
Use Guice instead of custom implementation
...
This uses the avaialable code in Guice rather than a custom
implementation. It also provides a performance increase (as demonstrated
by the unit tests)
2020-01-15 18:13:20 -05:00
0fa87a36e8
Add default constructor to IdentityCacheWrapper
...
Kryo is spending a lot of time during serialization handling a
NoSuchMethodException. By adding this default constructor we can skip
doing all that work.
2020-01-15 13:39:27 -05:00
55308ba90a
Move IdentityCacheableWrapper
...
This is utilized only in tests so it belongs there
2020-01-15 13:39:27 -05:00
0fe8463e81
Remove AtomicInteger loop counter
...
This test isn't multithreaded so I see no need for this
2020-01-15 13:39:27 -05:00
611fa325df
Remove unused code
...
It reduces the clarity of the surrounding code
2020-01-15 13:39:27 -05:00
bb61aa0bcb
Fix unchecked warning
2020-01-15 13:39:27 -05:00
45cd46e030
Try upping to BDB JE 7.5.11
2019-08-11 12:42:03 +01:00
cefad96036
Merge pull request #276 from internetarchive/jdk11-upgrade-jetty
...
JDK11 support: upgrade to Jetty 9.4.19, Restlet 2.4.0 and drop JDK 7 support
2019-08-07 10:48:51 -07:00
a1f2c9513a
Merge pull request #273 from internetarchive/jdk11-remove-old-bdbcache
...
JDK11 support: remove unused class ObjectIdentityBdbCache and tests
2019-08-07 10:48:40 -07:00
0782fb0481
JDK11: upgrade engine to jetty 9.4.19 and restlet 2.4.0
...
Jetty 9.4.12+ is required for TLS to work correctly under JDK11
(due to SSL handshake failures). In order to upgrade
Jetty we also need to upgrade Restlet.
There is one intentional change in behaviour to simplify
upgrading. We remove a workaround for an old [webkit bug]
where the browser claimed to prefer application/xml. The
bug was fixed in 2011.
[webkit bug]: https://bugs.webkit.org/show_bug.cgi?id=27267
Summary of Jetty API changes:
- package names changed such as org.mortbay -> org.eclipse
- SocketConnector and SslSocketConnector merged to ServerConnector
- HashUserRealm split into UserStore and HashLoginService
- SecurityHandler -> ConstraintSecurityHandler
Summary of Restlet API changes:
- some classes have moved package (Request, Response, Router etc)
- ServerResource replaces Resource
- represent(), acceptRepresentation() renamed to get(), post()
- constructors were replaced by an init() method
- setModifiable() was removed
- getRequest().getEntityAsForm() -> new Form(entity)
- Guard -> ChallengeAuthenticator
Fixes #275
2019-08-06 15:16:37 +09:00
a5c2b46208
JDK11 support: remove unused class ObjectIdentityBdbCache and tests
...
In 2011 ObjectIdentityBdbCache was replaced by
ObjectIdentityBdbManualCache in order to "remove dependence on
GC/finalization/PhantomReference magic". It hasn't been used since.
As of JDK11 the ObjectIdentityBdbCache unit tests are failing so let's
remove it.
2019-08-02 17:10:13 +09:00
412885fd7d
JDK11 support: upgrade maven-surefire-plugin to 2.22.2
...
Under JDK11 our old version of it throws ClassNotFoundExceptions when
tests load some builtin classes like javax.transaction.xa.Xid.
Fixes #266
2019-08-02 16:43:20 +09:00
1bd8b713c6
quiet org.mortbay.log (jetty?) logging
...
There is already a clause for this in logging.properties, but it's using
log4j. It was dumping stack traces every time the client was dubious of
heritrix's self-signed certificate.
Why do we have so many identical log4j.xml's? 🤷♂️
2019-04-30 16:17:35 -07:00
b3961a2f96
[maven-release-plugin] prepare for next development iteration
2019-04-18 15:36:28 +01:00
c7c6141ee1
[maven-release-plugin] prepare release 3.4.0-20190418
2019-04-18 15:36:20 +01:00
ad9e74d8ad
[maven-release-plugin] prepare for next development iteration
2019-02-07 13:52:56 +00:00
83c7044b22
[maven-release-plugin] prepare release 3.4.0-20190207
2019-02-07 13:52:49 +00:00
317b19e7e2
[maven-release-plugin] prepare for next development iteration
2019-02-05 12:33:08 +00:00
9c7d0c299d
[maven-release-plugin] prepare release 3.4.0-20190205
2019-02-05 12:32:57 +00:00
ab34a54566
[maven-release-plugin] prepare for next development iteration
2019-02-05 12:29:33 +00:00
d1fc5cedb4
[maven-release-plugin] prepare release 3.4.0-20190205-2
2019-02-05 12:29:26 +00:00
096ba4e9c3
[maven-release-plugin] prepare for next development iteration
2019-02-05 12:15:26 +00:00
aa705bef2e
Correct spelling mistakes.
2018-11-04 19:49:37 +09:00
d85d2a3fe8
Move OneLineSimpleLayout into commons so all can use it.
2017-10-12 22:10:58 +01:00
ae8a0ef95f
update webarchive-commons dependency to incorporate fix to warc size calculation, and uncomment test that exercises the fix
2017-10-11 14:54:37 -07:00
3892c17b56
ughhh appease java 8 javadoc rules
2017-06-15 17:29:22 -07:00
d13bfb6a37
Remove unit tests which moved over from Heritrix3 to webarchive-commons
2017-03-09 10:26:32 -08:00
36ea1a9fb2
try to fix remaining serialization tests in oraclejdk8 by using ConcurrentSkipListMap instead of ConcurrentHashMap
2016-06-07 17:55:45 -05:00
99d87e8f23
hopefully fix serialization tests in oraclejdk8 by using TreeSet instead of HashSet in KeyedProperties.java
2016-06-07 14:53:11 -05:00
cbb5e3ab99
Simplify logic for urls with spaces, and make it better follow the pattern of UriUtils.isVeryLikelyUri(). Involves a subtle adjustment to the regex LIKELY_RELATIVE_URI_PATTERN to ensure the 2nd capturing group always gets the file extension. Also add a couple of tests of strings with spaces and file extensions that are not known good extensions.
2016-02-09 15:02:11 -08:00
088189ee8b
Fix up code based on pull requests comments. Add test for urls with spaces in them.
2016-01-28 15:11:49 -08:00
4518bc2da8
Make sure no urls with whitespace and not having good extension slip through
2016-01-20 14:29:42 -08:00
12a88d6d23
Allow spaces in urls extracted from JS.
2016-01-20 10:44:20 -08:00
23885557bb
Allow spaces in JavaScript urls, but only if they have a known good file extension
2016-01-19 18:07:05 -08:00
3b12a751e1
to make this branch only about multipart/form-data, undo the changes to handle more complicated forms with extra input fields; add unit test; store CrawlURI submit data in http-library agnostic way and remove httpcomponents-specific code from HTMLForm, refactor creation of post data accordingly
2016-01-14 12:17:19 -08:00
34cffd512a
Add ability to submit forms with enctype="multipart/form-data"
2016-01-14 12:05:38 -08:00
b7afc627ed
fix typo
2015-09-24 15:31:12 -07:00
9aabcb3272
add license header to new class
2015-09-23 12:07:38 -07:00
Alex Osborne