Andrew Jackson
846f2bc2cc
[maven-release-plugin] prepare release 3.4.0-20210621
2021-06-21 15:42:46 +01:00
Andrew Jackson
33a01a3bb6
Merge branch 'master' of github.com:internetarchive/heritrix3
2021-06-21 14:28:38 +01:00
Andrew Jackson
6d0ae8f916
Remove unnecessary Maven setup that breaks cross-JDK builds.
2021-06-21 14:18:52 +01:00
Andrew Jackson
c75e710183
Update Changlog.
2021-06-21 14:18:06 +01:00
Andy Jackson
31e54ccdb0
Merge pull request #402 from internetarchive/remove-mg4j
...
Remove dependency on mg4j
2021-06-20 23:08:24 +01:00
Alex Osborne
b917a32eab
Merge pull request #401 from kris-sigur/shutdown
...
Graceful UI shutdown
2021-06-20 23:47:44 +09:00
Alex Osborne
5d0a5b2ec4
Remove dependency on mg4j
...
This eliminates one of the jars that Heritrix depends on that isn't in
Maven Central.
There were only two usages. The first usage was a method in a
CrawlerJournal which as far as I can tell is unused and can be removed.
The second usage was an unnecessary use of MutableString as a line
buffer when saving cookies to a file. Since there doesn't seem any
particular reason buffer per-line here let's just use a BufferedWriter
instead on the whole file which is also slightly simpler and more
efficient.
2021-06-20 23:30:41 +09:00
Kristinn Sigurdsson
37913cbb2e
Graceful UI shutdown
2021-06-18 10:56:20 +00:00
Andrew Jackson
a34e40a9f8
Update changelog.
2021-06-18 10:25:54 +01:00
Andy Jackson
50de3eea08
Merge pull request #400 from kris-sigur/robots-sitemap
...
Remove unnecessary fiddling with VIA path in ExtractorRobotsTxt
2021-06-18 10:14:11 +01:00
Andrew Jackson
07ada5f838
Update changelog.
2021-06-18 09:57:51 +01:00
Kristinn Sigurdsson
627befe07e
Remove unnecessary fiddling with VIA path
2021-06-18 08:42:05 +00:00
Andrew Jackson
a5beb7d6e9
[maven-release-plugin] prepare for next development iteration
2021-06-18 09:41:30 +01:00
Andrew Jackson
30598f8402
[maven-release-plugin] prepare release 3.4.0-20210618
2021-06-18 09:41:24 +01:00
Andrew Jackson
22b07e151a
Merge branch 'master' of github.com:internetarchive/heritrix3
2021-06-18 09:37:37 +01:00
Andrew Jackson
768c6ef458
Update changelog.
2021-06-18 09:37:13 +01:00
Andy Jackson
b60ae999cc
Merge pull request #399 from ukwa/master
...
Switch to properties that enforce Java 8 compatibility.
2021-06-17 21:23:39 +01:00
Andrew Jackson
2007ddb005
Switch to properties that enforce Java 8 compatibility.
2021-06-17 21:22:04 +01:00
Andrew Jackson
18e8779398
Updated changelog.
2021-06-17 16:42:23 +01:00
Andrew Jackson
3423cab6a6
[maven-release-plugin] prepare for next development iteration
2021-06-17 14:39:00 +01:00
Andrew Jackson
7368ded9c2
[maven-release-plugin] prepare release 3.4.0-20210617
2021-06-17 14:38:54 +01:00
Andrew Jackson
a5164dbcea
Update changelog.
2021-06-17 14:34:22 +01:00
Andy Jackson
4b82d6060c
Merge pull request #398 from kris-sigur/sitemap-tweak
...
Annotate nested sitemap links as sitemaps
2021-06-17 10:10:58 +01:00
Kristinn Sigurdsson
81bef22270
Mark nested sitemap links as sitemaps
2021-06-16 13:07:18 +00:00
Andrew Jackson
d0221642c3
Update changelog.
2021-06-14 11:04:33 +01:00
Andrew Jackson
09c8d5696b
Use HTTPS version for Oracle Maven repo.
2021-06-14 11:03:32 +01:00
Andy Jackson
169b3cc8ab
Merge pull request #393 from ukwa/checkpoint-success-stats-392
...
Only update last checkpoint stats if the checkpoint completed, for #392 .
2021-05-28 14:20:06 +01:00
Andy Jackson
699f107ccd
Merge pull request #394 from ukwa/update-ampq-client
...
Update AMPQ client library to address security warning.
2021-05-28 13:24:23 +01:00
Andy Jackson
921e349939
Update AMPQ client library to address security warning.
2021-05-28 12:47:59 +01:00
Andy Jackson
5caa83d327
Only update last checkpoint stats if the checkpoint completed, for #392 .
2021-05-28 12:42:17 +01:00
Andy Jackson
773b4a074d
Merge pull request #391 from ukwa/master
...
Sync changelog with release.
2021-05-27 10:27:40 +01:00
Andy Jackson
a87937caf9
Sync changelog with release.
2021-05-27 10:26:50 +01:00
Andrew Jackson
d3dfb34f83
[maven-release-plugin] prepare for next development iteration
2021-05-27 10:03:10 +01:00
Andrew Jackson
9dcbb240a6
[maven-release-plugin] prepare release 3.4.0-20210527
2021-05-27 10:03:04 +01:00
Andy Jackson
cf0f4ef144
Merge pull request #390 from ukwa/master
...
Update changelog.
2021-05-27 09:46:41 +01:00
Andy Jackson
9df4a0dc05
Update changelog.
2021-05-27 09:45:47 +01:00
Andy Jackson
3b4c442683
Merge pull request #389 from ukwa/update-dependencies-2021-05-26
...
Update dependencies 2021 05 26
2021-05-26 21:56:21 +01:00
Andy Jackson
669cf1ed2b
Merge branch 'master' into update-dependencies
2021-05-26 13:40:15 +01:00
Andy Jackson
a2c65f4618
Update dependencies for security issues, including changes to tests.
2021-05-26 13:39:30 +01:00
Andy Jackson
d175040e33
Merge pull request #386 from ukwa/master
...
Bring changlog up to date
2021-05-20 22:50:43 +01:00
Andrew Jackson
400677a46a
Merge branch 'master' of github.com:internetarchive/heritrix3
2021-05-20 22:49:05 +01:00
Andrew Jackson
7c72b8d584
Updated changelog.
2021-05-20 22:47:56 +01:00
Andy Jackson
d9fe123b2f
Merge pull request #384 from ukwa/configure-bdb-je-cleaner-threads
...
Allow tuning of BDB-JE evictor and cleaner threads.
2021-05-20 22:24:42 +01:00
Andy Jackson
276ced0de3
Allow tuning of BDB-JE evictor and cleaner threads.
2021-05-20 22:22:00 +01:00
Andy Jackson
c0dbfb2a0a
Merge pull request #383 from ukwa/upgrade-dnsjava
...
Update to latest version of dnsjava, for #344
2021-05-20 21:05:59 +01:00
Andy Jackson
d7869de8d9
Merge pull request #262 from kris-sigur/sitemaps
...
Support for extracting URLs in sitemaps
2021-05-20 21:05:13 +01:00
Andy Jackson
396467c535
Merge branch 'master' into sitemaps
2021-05-20 21:03:54 +01:00
Andy Jackson
c1bcdd9f9e
Merge pull request #341 from internetarchive/noplaylist-ydl
...
youtube-dl --no-playlist
2021-05-20 21:01:39 +01:00
Andy Jackson
b2a07b290f
Merge pull request #348 from internetarchive/fixes-leaky-file-handles
...
Fixes leaky file handles
2021-05-20 21:00:31 +01:00
Andy Jackson
c7b1dfa198
Merge pull request #368 from webdev4422/patch-1
...
Fix misspell in comments
2021-05-20 20:58:41 +01:00
Andrew Jackson