Jetty 9.4.12+ is required for TLS to work correctly under JDK11
(due to SSL handshake failures). In order to upgrade
Jetty we also need to upgrade Restlet.
There is one intentional change in behaviour to simplify
upgrading. We remove a workaround for an old [webkit bug]
where the browser claimed to prefer application/xml. The
bug was fixed in 2011.
[webkit bug]: https://bugs.webkit.org/show_bug.cgi?id=27267
Summary of Jetty API changes:
- package names changed such as org.mortbay -> org.eclipse
- SocketConnector and SslSocketConnector merged to ServerConnector
- HashUserRealm split into UserStore and HashLoginService
- SecurityHandler -> ConstraintSecurityHandler
Summary of Restlet API changes:
- some classes have moved package (Request, Response, Router etc)
- ServerResource replaces Resource
- represent(), acceptRepresentation() renamed to get(), post()
- constructors were replaced by an init() method
- setModifiable() was removed
- getRequest().getEntityAsForm() -> new Form(entity)
- Guard -> ChallengeAuthenticator
Fixes#275
tools.jar is no longer included in the JDK as of JDK 11 (per JEP 220) so
exclude it as a dependency. According to [HBASE-13963] it is only
required when compiling hbase itself and was unintentionally leaked
as a transitive dependency.
Fixes#265
[HBASE-13963]: https://issues.apache.org/jira/browse/HBASE-13963
There is already a clause for this in logging.properties, but it's using
log4j. It was dumping stack traces every time the client was dubious of
heritrix's self-signed certificate.
Why do we have so many identical log4j.xml's? 🤷♂️
... configure using rethinkdb url and segment id, instead of write url,
which means it can work if the segment gets reassigned and so forth
***backward incompatible change***
- make sure we "promote" segments (persist them to hdfs) after we're
entirely finished writing to theme; in other words, when we reach the
FINISH state, not while STOPPING
- shut down the promoter thread at crawl finish
plus a couple of tweaks and cleanups
realized that we're leaking zookeeper connections, and it might be
because the HConnection has been thread local, which probably means that
at shutdown only one of the possibly many connections is cleaned up
zookeeper by default allows 60 connections per client and it looks
likely that we start having trouble when we hit that limit
Catch exceptions and attempt to reset the hbase connection. The intent
is for this to fix the problem where the hbase connection (under the
hood the problem is the zookeeper connection) gets into a borked state,
and we never reset it.
