Commit Graph
59 Commits
Author SHA1 Message Date
Noah Levitt 35c65ccb5a Merge remote-tracking branch 'origin/master' into ait-qa
* origin/master:
  fix line ending and indentation issues
  AssignmentLevelSurtQueueAssignmentPolicy.java - Add support for forced queue assignment and parallel queues URIAuthorityBasedQueueAssignmentPolicy.java - Add interoperability between forced queue assignment and parallel queues QuotaEnforcer.java - Fix javadoc to match default behavior
  Override PUT so it doesn't change the file extension
  Use super.getVariants() rather than super.getVariants(GET)
  Remove testConcurrentLoad
  Mitigate random CookieStore.testConcurrentLoad test failures
  Add missing UUID import (interactive commit fail)
  Fix digest authentication
  Link to javadoc.io for more recent api docs
  Now jdk11 support is merged we can disallow test failures on it
  Fix restlet child component context warning
  Update statistics assertions as Jetty 9 produces different headers
  Fix 'No session data store configured' test error
  Drop support for JDK 7
  JDK11: upgrade engine to jetty 9.4.19 and restlet 2.4.0
  JDK11 support: upgrade jetty to 9.4.19 in modules tests
  Prevent newer surefire from invoking FetchHTTPTests directly
  JDK11 support: remove unused class ObjectIdentityBdbCache and tests
  JDK11 support: exclude tools.jar from hbase-client dependency
  JDK11 support: explicitly depend on JAXB
  JDK11 support: upgrade maven-surefire-plugin to 2.22.2
  Travis fixes
  Update README and add LICENSE.txt
2019-11-15 16:21:39 -08:00
Noah Levitt cefad96036 Merge pull request #276 from internetarchive/jdk11-upgrade-jetty
JDK11 support: upgrade to Jetty 9.4.19, Restlet 2.4.0 and drop JDK 7 support
2019-08-07 10:48:51 -07:00
Alex Osborne 9d1341c731 JDK11 support: upgrade jetty to 9.4.19 in modules tests
Fixes #268 ssl handshake_failure. Support for jdk11 was added in jetty
9.4.14 but we may as well bump to the latest stable version.

In Jetty 9 it appears the request is logged after the response is sent.
Thus it was racing with the assertions that check the client IP. So to
fix this rather stashing the 'lastRequest' we just make the server
echo the client's IP in a response header.

Some other minor tweaks were needed due to changes in Jetty behaviour:

- We stop checking the length of the raw response. It doesn't tell us
  anything and easily varies.

- Jetty now generates Set-Cookie with a space after the ;

- Jetty now lowercases the word "basic" in WWW-Authenticate header

- testLaxUrlEncoding(): Jetty now rejects bad paths with a 400 error so
  we disable the response checks. The actual request line is
  still checked which is the important thing.

Note: This patch does not affect the version of jetty used by the
Heritrix admin console. That will be tackled separately.
2019-08-05 14:35:09 +09:00
Alex Osborne d3d19e2248 Prevent newer surefire from invoking FetchHTTPTests directly
It seems older versions only matched *Test.java but the new version
is now incorrectly including it.
2019-08-02 17:57:09 +09:00
Alex Osborne 412885fd7d JDK11 support: upgrade maven-surefire-plugin to 2.22.2
Under JDK11 our old version of it throws ClassNotFoundExceptions when
tests load some builtin classes like javax.transaction.xa.Xid.

Fixes #266
2019-08-02 16:43:20 +09:00
Andrew Jackson b3961a2f96 [maven-release-plugin] prepare for next development iteration 2019-04-18 15:36:28 +01:00
Andrew Jackson c7c6141ee1 [maven-release-plugin] prepare release 3.4.0-20190418 2019-04-18 15:36:20 +01:00
Noah Levitt b12e34d340 Merge branch 'trough-dedup' into ait-qa
* trough-dedup:
  promote dirty segments at crawl finish
  trough dedup!
  Allow failed lookups to expire, for #234.
  [maven-release-plugin] prepare for next development iteration
  [maven-release-plugin] prepare release 3.4.0-20190207
  As @nlevitt suggestion, a further check.
  [maven-release-plugin] prepare for next development iteration
  [maven-release-plugin] prepare release 3.4.0-20190205
  [maven-release-plugin] prepare for next development iteration
  [maven-release-plugin] prepare release 3.4.0-20190205-2
  [maven-release-plugin] prepare for next development iteration
  Extend to set additional property.
  Use argument syntax.
  Skip tests during release process (covered by CI).
  Set consistent tag, and include contrib.
  Wrong repo spec.
  Add build profile for deployment to Maven Central.
  Avoid headings being treated as lists
  Clarification about APIs.
  Swapped original and link to make maintenance simpler.
  Tidy up markup and links.
  Add synchronized statements for internetarchive/heritrix3#221.
  Add checks to guard against server sending 304 in error, for #229.
  do not checkpoint if crawl job has not started
2019-03-14 15:17:39 -07:00
Andrew Jackson ad9e74d8ad [maven-release-plugin] prepare for next development iteration 2019-02-07 13:52:56 +00:00
Andrew Jackson 83c7044b22 [maven-release-plugin] prepare release 3.4.0-20190207 2019-02-07 13:52:49 +00:00
Andrew Jackson 317b19e7e2 [maven-release-plugin] prepare for next development iteration 2019-02-05 12:33:08 +00:00
Andrew Jackson 9c7d0c299d [maven-release-plugin] prepare release 3.4.0-20190205 2019-02-05 12:32:57 +00:00
Andrew Jackson ab34a54566 [maven-release-plugin] prepare for next development iteration 2019-02-05 12:29:33 +00:00
Andrew Jackson d1fc5cedb4 [maven-release-plugin] prepare release 3.4.0-20190205-2 2019-02-05 12:29:26 +00:00
Andrew Jackson 096ba4e9c3 [maven-release-plugin] prepare for next development iteration 2019-02-05 12:15:26 +00:00
Neil Minton 2ee5db7f3b Formating fixes in POM file. 2016-08-19 19:47:47 -07:00
Noah Levitt e5e5cd87e4 configure maven-compiler-plugin in root pom to avoid redundancy, inconsistency 2014-10-01 13:38:39 -07:00
Noah Levitt 3a429ffa44 Merge branch 'master' into hc43
Conflicts:
	commons/pom.xml
	commons/src/main/java/org/archive/io/RecordingInputStream.java
	commons/src/main/java/org/archive/io/RecordingOutputStream.java
	commons/src/main/java/org/archive/io/ReplayInputStream.java
	dist/pom.xml
	engine/pom.xml
	modules/pom.xml
	modules/src/main/java/org/archive/modules/extractor/ExtractorHTTP.java
	modules/src/main/java/org/archive/modules/recrawl/FetchHistoryProcessor.java
	pom.xml
2014-01-10 17:52:03 -08:00
Noah Levitt 749bd6a214 bump version to 3.3.0-SNAPSHOT after 3.2.0 release 2014-01-10 16:22:43 -08:00
Noah Levitt 32c083f720 bump version number to 3.2.0 for release 2014-01-10 12:36:45 -08:00
Noah Levitt 0ea10665f1 abbreviate maven repository clauses, reduce redundancy, and let default settings prevail, such as updatePolicy=daily, pointed out by Erik Hetzner 2013-12-15 11:53:18 -08:00
Noah Levitt 8e1125d062 bump version to 3.2.0-SNAPSHOT since there are some incompatible api changes since 3.1.1 (requested by kris sigurðsson) 2013-11-20 09:18:31 -08:00
Noah Levitt 1993c4388c Merge branch 'master' into hc43
Conflicts:
	.classpath
2013-03-20 14:28:13 -07:00
Noah Levitt 8fd361e518 update source/target version to 1.6 since we use 1.6 features, specifically @Override for methods only implementing an interface (not sure why maven wasn't enforcing this); also remove maven-antrun-plugin created timestamp.txt, doesn't seem to be used anywhere 2013-03-20 14:25:22 -07:00
Noah Levitt 12da824316 Merge branch 'master' into hc43 2013-03-18 23:32:51 -07:00
Noah Levitt fb758e53e3 Avoid pulling in hadoop-core.jar and its dependencies; add plugin versions to get rid of warnings 2013-02-22 15:43:15 -08:00
Noah Levitt a20824aea7 Merge branch 'master' into new-fetchhttp-only
Conflicts:
	.classpath
	engine/pom.xml
	modules/pom.xml
	modules/src/main/java/org/archive/modules/fetcher/FetchHTTP.java
	modules/src/test/java/org/archive/modules/fetcher/FetchHTTPTest.java
2012-11-30 17:04:28 -08:00
Noah Levitt b9cbbbd248 Bunch of unit tests for FetchHTTP
* FetchHTTPTest.java
    a bunch of new tests
* FetchHTTP.java
    make sslTrustLevel work correctly (bug found by test; even with fix, only respects value at startup, no overrides)
* modules/pom.xml, .classpath
    dependencies for testing
* engine/pom.xml
    update jetty to 6.1.26 since we require that for the new tests
2012-09-26 14:41:49 -07:00
Noah Levitt 77559bf81c **/pom.xml
bump version number because of api change
2012-09-13 17:47:26 -07:00
Noah Levitt 3f2295353a test dependencies belong in engine/ now too 2012-08-30 12:16:25 -07:00
Noah Levitt f502292eef commons-codec was already pulled in from elsewhere 2012-08-29 19:06:37 -07:00
Noah Levitt 731d382ae0 fetchhttp2 support for proxy auth, somewhat elaborately done, sharing principles and code with http "target" auth 2012-08-29 17:41:50 -07:00
Noah Levitt a2d65044f0 progress on http proxy test 2012-08-28 18:25:53 -07:00
Noah Levitt 373abbbe81 latest jetty 6.x; starting on basic-auth test 2012-06-29 17:19:46 -07:00
Noah Levitt 69d7b0a0cd make jetty server not require anything external; also make it work with maven 2012-06-29 13:43:54 -07:00
Noah Levitt 9fffa8fc53 **/pom.xml
bump version to 3.1.2-SNAPSHOT
2012-05-01 18:17:33 -07:00
Noah Levitt 79c40346fc **/pom.xml
* bump version number to 3.1.1 for release
2012-05-01 18:01:06 -07:00
nlevitt 5735839252 Post 3.1.0
* **/pom.xml
   switch version to "3.1.1-SNAPSHOT"
2011-10-21 19:00:37 +00:00
nlevitt 8c99fc947c Prep for 3.1.0 release
* **/pom.xml
    bump version-id to "3.1.0"
* README.txt
    refer to exactly 3.1.0 release notes
2011-10-21 17:08:04 +00:00
nlevitt 28aa2ffede * **/pom.xml
improve formatting (no substantive changes at all -- try "svn diff -x -ub -c7232")
2011-08-05 17:46:57 +00:00
nlevitt cb7a454d0c Post 3.1.0-RC1
* **/pom.xml
   switch version back to "3.1.0-SNAPSHOT"
2011-07-26 03:42:02 +00:00
nlevitt 347a25e417 Prep for 3.1.0-RC1 release
* pom.xml, */pom.xml
    bump version number to 3.1.0-RC1
2011-07-26 02:57:48 +00:00
gojomo 991a67e0e4 Post 3.1.0-beta
* **/pom.xml
    chnge version-identifier to "3.1.0-SNAPSHOT"
2011-04-15 22:28:52 +00:00
gojomo 52d00a9917 Prep for 3.1.0-beta
* **/pom.xml
    increment declared version
2011-04-15 18:09:43 +00:00
gojomo a757140cf6 [HER-1831] change version numbering convention to be less confusing
* **/pom.xml
    change 3.1.1-SNAPSHOT to 3.0.1-SNAPSHOT
2010-09-30 19:48:35 +00:00
gojomo 829a3f78b3 [HER-1794] Extra white spaces in pom.xml
* **/pom.xml
    remove leading blank lines
2010-07-02 00:51:19 +00:00
gojomo cb7df416d4 post-3.0.0 release build
* **/pom.xml
    bump version identifiers to "3.1.1-SNAPSHOT" (odd minor number meaning dev version, odd patch number indicating expected next patch number of 3.0.x release if before a 3.2.0 release)
2009-12-05 23:48:38 +00:00
gojomo 5661658b1b prep for 3.0.0 release
* **/pom.xml
    bump version-id to "3.0.0"
* README.txt
    refer to exactly 3.0.0 release notes
2009-12-05 09:28:19 +00:00
gojomo ed2061560e Improve Maven build robustness
* UURIFactoryTest.java
    specify stirng literal with escape-encoding, rather than as raw UTF-8
* **/pom.xml
    include 'project.build.sourceEncoding' property to suppress platform-encoding warnings
    (unfortunately, neither this nor similar 'maven.compile.encoding' property actually casues maven build to interpret Java source as UTF-8)
2009-12-02 01:34:40 +00:00
gojomo 79fd64a2b5 Post-RC1-build
* **/pom.xml
    revert version name to SNAPSHOT
2009-11-23 10:23:28 +00:00