lee/heritrix3
1
0
Fork 0
mirror of https://github.com/internetarchive/heritrix3.git synced 2026-09-26 15:46:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,510 Commits 38 Branches 33 Tags
2d862c3e2811c3fef0f21ba8511f6bbc4e248f6e
Commit Graph
124 Commits
Author SHA1 Message Date
Alex Osborne 19dd5c00de Switch ExtractorPDFContent to pdfbox 2024-02-21 14:48:58 +09:00
Alex Osborne 3700628cb2 Switch PDFParser and ExtractorPDF to pdfbox 2024-02-21 14:48:54 +09:00
Andy Jackson e83f4f9e02 Reset to SNAPSHOT 2022-07-28 10:56:59 +01:00
Andy Jackson 292d2998cb [maven-release-plugin] prepare release 3.4.0-20220727 2022-07-27 21:36:19 +01:00
Lauren Ko a0f0ba45e5 Fix files included by assembly 2021-12-03 16:05:59 -06:00
Lauren Ko 256f8742e9 Fix group id is too big 2021-12-03 12:20:32 -06:00
Andrew Jackson 024a7772b5 [maven-release-plugin] prepare for next development iteration 2021-09-23 22:08:33 +01:00
Andrew Jackson fedff1e2c7 [maven-release-plugin] prepare release 3.4.0-20210923 2021-09-23 22:08:30 +01:00
Andrew Jackson ddb695abad [maven-release-plugin] prepare for next development iteration 2021-08-03 09:56:26 +01:00
Andrew Jackson ba44cce7f2 [maven-release-plugin] prepare release 3.4.0-20210803 2021-08-03 09:56:20 +01:00
Alex Osborne 0dfb119df2 Upgrade maven-assembly-plugin to 3.3.0 to fix file permissions 
The old default version of maven-assembly-plugin generates packages
containing dangerous world writable files.

Fixes #413
 2021-07-19 15:46:36 +09:00
Andrew Jackson 0e93e22ba9 [maven-release-plugin] prepare for next development iteration 2021-06-21 15:42:52 +01:00
Andrew Jackson 846f2bc2cc [maven-release-plugin] prepare release 3.4.0-20210621 2021-06-21 15:42:46 +01:00
Alex Osborne 5d0a5b2ec4 Remove dependency on mg4j 
This eliminates one of the jars that Heritrix depends on that isn't in
Maven Central.

There were only two usages. The first usage was a method in a
CrawlerJournal which as far as I can tell is unused and can be removed.

The second usage was an unnecessary use of MutableString as a line
buffer when saving cookies to a file. Since there doesn't seem any
particular reason buffer per-line here let's just use a BufferedWriter
instead on the whole file which is also slightly simpler and more
efficient.
 2021-06-20 23:30:41 +09:00
Andrew Jackson a5beb7d6e9 [maven-release-plugin] prepare for next development iteration 2021-06-18 09:41:30 +01:00
Andrew Jackson 30598f8402 [maven-release-plugin] prepare release 3.4.0-20210618 2021-06-18 09:41:24 +01:00
Andrew Jackson 3423cab6a6 [maven-release-plugin] prepare for next development iteration 2021-06-17 14:39:00 +01:00
Andrew Jackson 7368ded9c2 [maven-release-plugin] prepare release 3.4.0-20210617 2021-06-17 14:38:54 +01:00
Andrew Jackson d3dfb34f83 [maven-release-plugin] prepare for next development iteration 2021-05-27 10:03:10 +01:00
Andrew Jackson 9dcbb240a6 [maven-release-plugin] prepare release 3.4.0-20210527 2021-05-27 10:03:04 +01:00
Andrew Jackson e215b89963 [maven-release-plugin] prepare for next development iteration 2020-05-18 14:08:12 +01:00
Andrew Jackson ef38dc08c3 [maven-release-plugin] prepare release 3.4.0-20200518 2020-05-18 14:08:04 +01:00
Andrew Jackson ff05de44b6 [maven-release-plugin] prepare for next development iteration 2020-03-04 23:05:41 +00:00
Andrew Jackson a1bdcb1be7 [maven-release-plugin] prepare release 3.4.0-20200304 2020-03-04 23:05:33 +00:00
Noah Levitt cefad96036 Merge pull request #276 from internetarchive/jdk11-upgrade-jetty 
JDK11 support: upgrade to Jetty 9.4.19, Restlet 2.4.0 and drop JDK 7 support
 2019-08-07 10:48:51 -07:00
Alex Osborne 0782fb0481 JDK11: upgrade engine to jetty 9.4.19 and restlet 2.4.0 
Jetty 9.4.12+ is required for TLS to work correctly under JDK11
(due to SSL handshake failures). In order to upgrade
Jetty we also need to upgrade Restlet.

There is one intentional change in behaviour to simplify
upgrading. We remove a workaround for an old [webkit bug]
where the browser claimed to prefer application/xml. The
bug was fixed in 2011.

[webkit bug]: https://bugs.webkit.org/show_bug.cgi?id=27267

Summary of Jetty API changes:
- package names changed such as org.mortbay -> org.eclipse
- SocketConnector and SslSocketConnector merged to ServerConnector
- HashUserRealm split into UserStore and HashLoginService
- SecurityHandler -> ConstraintSecurityHandler

Summary of Restlet API changes:
- some classes have moved package (Request, Response, Router etc)
- ServerResource replaces Resource
- represent(), acceptRepresentation() renamed to get(), post()
- constructors were replaced by an init() method
- setModifiable() was removed
- getRequest().getEntityAsForm() -> new Form(entity)
- Guard -> ChallengeAuthenticator

Fixes #275
 2019-08-06 15:16:37 +09:00
Alex Osborne 412885fd7d JDK11 support: upgrade maven-surefire-plugin to 2.22.2 
Under JDK11 our old version of it throws ClassNotFoundExceptions when
tests load some builtin classes like javax.transaction.xa.Xid.

Fixes #266
 2019-08-02 16:43:20 +09:00
Noah Levitt 1bd8b713c6 quiet org.mortbay.log (jetty?) logging 
There is already a clause for this in logging.properties, but it's using
log4j. It was dumping stack traces every time the client was dubious of
heritrix's self-signed certificate.

Why do we have so many identical log4j.xml's? 🤷‍♂️
 2019-04-30 16:17:35 -07:00
Andrew Jackson b3961a2f96 [maven-release-plugin] prepare for next development iteration 2019-04-18 15:36:28 +01:00
Andrew Jackson c7c6141ee1 [maven-release-plugin] prepare release 3.4.0-20190418 2019-04-18 15:36:20 +01:00
Andrew Jackson ad9e74d8ad [maven-release-plugin] prepare for next development iteration 2019-02-07 13:52:56 +00:00
Andrew Jackson 83c7044b22 [maven-release-plugin] prepare release 3.4.0-20190207 2019-02-07 13:52:49 +00:00
Andrew Jackson 317b19e7e2 [maven-release-plugin] prepare for next development iteration 2019-02-05 12:33:08 +00:00
Andrew Jackson 9c7d0c299d [maven-release-plugin] prepare release 3.4.0-20190205 2019-02-05 12:32:57 +00:00
Andrew Jackson ab34a54566 [maven-release-plugin] prepare for next development iteration 2019-02-05 12:29:33 +00:00
Andrew Jackson d1fc5cedb4 [maven-release-plugin] prepare release 3.4.0-20190205-2 2019-02-05 12:29:26 +00:00
Andrew Jackson 096ba4e9c3 [maven-release-plugin] prepare for next development iteration 2019-02-05 12:15:26 +00:00
Andrew Jackson 021aeeaa20 Swapped original and link to make maintenance simpler. 2019-02-05 10:25:54 +00:00
Andrew Jackson b08db899dd Tidy up markup and links. 2019-02-05 10:23:01 +00:00
Maurice Schleußinger 48908ee5b4 Fix link to User Guide 2018-07-04 22:53:43 +09:00
Kristinn Sigurðsson 05811705ed Point at GitHub wiki for latest release info 2018-05-29 09:42:24 +00:00
Noah Levitt 3892c17b56 ughhh appease java 8 javadoc rules 2017-06-15 17:29:22 -07:00
Noah Levitt 8009275075 make sure log4j is configured when running unit tests, to avoid log4j error messages polluting captured output and causing tests to fail, like so: https://builds.archive.org/job/Heritrix-3/org.archive.heritrix$heritrix-commons/587/testReport/org.archive.io.arc/ARCWriterTest/testLengthTooLongCompressed/ 2015-02-23 11:36:29 -08:00
Noah Levitt e5e5cd87e4 configure maven-compiler-plugin in root pom to avoid redundancy, inconsistency 2014-10-01 13:38:39 -07:00
Noah Levitt 6bdfec7e7f Merge branch 'master' of github.com:internetarchive/heritrix3 2014-01-10 16:22:51 -08:00
Noah Levitt 749bd6a214 bump version to 3.3.0-SNAPSHOT after 3.2.0 release 2014-01-10 16:22:43 -08:00
Noah Levitt 563c5b034d rename dist/README.md to dist/README.txt so that maven bundles it in the distribution 2014-01-10 14:13:32 -08:00
Noah Levitt 0467304f76 update readme for 3.2.0 release 2014-01-10 13:15:48 -08:00
Noah Levitt 32c083f720 bump version number to 3.2.0 for release 2014-01-10 12:36:45 -08:00
Noah Levitt 2388a35a06 useProjectArtifact=false to avoid [WARNING] Cannot include project artifact: ... it doesn't have an associated file or directory 2013-12-15 11:54:36 -08:00