Alex Osborne
bb0cb1cea0
Bump spring from 6.2.7 to 6.2.9
2025-07-21 17:00:48 +09:00
Alex Osborne
2388db4229
Bump commons-lang from 2.6 to 3.18.0
2025-07-21 16:55:31 +09:00
Alex Osborne
b41a95109c
Bump jetty from 12.0.17 to 12.0.22
2025-07-21 16:15:21 +09:00
Alex Osborne
fbc7489e61
Bump commons-io from 2.19.0 to 2.20.0
2025-07-21 16:12:59 +09:00
Alex Osborne
329bf1c741
Bump jsch from 2.27.0 to 2.27.2
2025-07-21 16:11:08 +09:00
Alex Osborne
53f154574d
Bump crawler-commons from 1.4 to 1.5
2025-07-21 16:10:15 +09:00
Alex Osborne
e121b71c52
Bump junit-jupiter from 5.13.2 to 5.13.3
2025-07-21 16:07:15 +09:00
Alex Osborne
8eaaa27ae7
Bump codemirror@search to 6.5.11 and codemirror@state to 6.37.2
2025-07-21 16:06:32 +09:00
Alex Osborne
c45544a47b
Bump restlet from 2.6.0-rc1 to 2.6.0
2025-07-21 16:03:24 +09:00
Alex Osborne
85ceab905c
BrowserProcessor: create scratchDir on start
...
This fixes an exception on first launch.
2025-07-21 11:55:37 +09:00
Alex Osborne
d7f5748fe0
Merge pull request #661 from internetarchive/seeds-report-redirect-extra-codes
...
Seeds report: record redirect URI for statuses 303, 307 and 308
2025-07-15 14:54:12 +09:00
Alex Osborne
839a8f4604
Bump webarchive-commons from 2.0.1 to 2.0.2
2025-07-15 11:16:48 +09:00
Alex Osborne
db63c7ec85
Seeds report: record redirect URI for statuses 303, 307 and 308
...
These can also redirect, not just 301 and 302.
Fixes #564
2025-06-30 22:24:29 +09:00
Alex Osborne
acf41d598d
FetchHTTP2: Use HTTP/1.1 when the server doesn't support ALPN
...
This fixes `IOException: frame_size_error/invalid_frame_length`.
2025-06-20 18:52:32 +09:00
Alex Osborne
8bfde99af3
FetchHTTP2: Handle getRemoteSocketAddress() returning null
2025-06-20 17:27:32 +09:00
Alex Osborne
b20d36f607
Add RELEASING.md
2025-06-13 00:44:40 +09:00
Alex Osborne
18026bf66b
Switch base image from deprecated openjdk to eclipse-temurin
2025-06-12 23:55:07 +09:00
Alex Osborne
475f2d6b2d
Bump Dockefile version to 3.10.0
2025-06-12 23:22:10 +09:00
Alex Osborne
01378ddbbf
[maven-release-plugin] prepare for next development iteration
2025-06-12 22:01:16 +09:00
Alex Osborne
ecc46a38c3
[maven-release-plugin] prepare release 3.10.0
2025-06-12 22:01:10 +09:00
Alex Osborne
5431a55820
Migrate from OSSRH to the Maven Central Portal
...
OSSRH reaches end of life June 30.
2025-06-12 21:55:52 +09:00
Alex Osborne
45342f85ca
Update CHANGELOG.md for 3.10.0
2025-06-12 21:48:22 +09:00
Alex Osborne
eacba1b276
Bump groovy from 4.0.26 to 4.0.27
2025-06-12 21:46:05 +09:00
Alex Osborne
6c172fb60f
Merge pull request #660 from internetarchive/dependabot/maven/contrib/org.apache.kafka-kafka-clients-3.9.1
...
Bump org.apache.kafka:kafka-clients from 3.9.0 to 3.9.1 in /contrib
2025-06-12 21:41:32 +09:00
dependabot[bot]
fcbc2d6931
Bump org.apache.kafka:kafka-clients from 3.9.0 to 3.9.1 in /contrib
...
Bumps org.apache.kafka:kafka-clients from 3.9.0 to 3.9.1.
---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
dependency-version: 3.9.1
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com >
2025-06-12 12:38:13 +00:00
Alex Osborne
9b5900b528
FetchHTTP2: Avoid silently decoding gzip
...
Fixes `Recorder getContentReplayInputStream
WARNING: gzip problem; using raw entity instead`.
2025-06-12 21:36:14 +09:00
Alex Osborne
24da104e38
Add link to #657 in CHANGELOG.md
2025-06-12 10:57:53 +09:00
Alex Osborne
789bd265ca
Add CHANGELOG.md entry for #659
2025-06-12 10:56:41 +09:00
Alex Osborne
5601453a45
Merge pull request #659 from internetarchive/adam/handle-bdb-shutdown-interrupts
...
fix: handle bdb shutdown interrupts
2025-06-12 10:49:37 +09:00
Adam Miller
1ab9b4a206
fix: avoid bdb environment corruption when terminate is called during shutdown
2025-06-11 17:44:52 -07:00
Adam Miller
0494d63ea2
fix: Avoid a NullPointerException when terminate is called again during shutdown
2025-06-11 17:44:52 -07:00
Alex Osborne
61af1d83c4
MitmProxy: Handle URLs generated by browsers but disallowed by java.net.URI
2025-06-11 20:38:54 +09:00
Alex Osborne
8a15156fa1
BrowserProcessor: Catch WebDriver timeouts and exceptions
2025-06-11 17:17:02 +09:00
Alex Osborne
e0bae9a1b5
FetchHTTP2: Fallback to async DNS resolver
2025-06-11 16:55:11 +09:00
Alex Osborne
9a1bc7f287
Merge pull request #657 from internetarchive/fetchhttp2-http-proxy
...
FetchHTTP2: Add HTTP proxy support
2025-06-11 11:54:30 +09:00
Alex Osborne
c3f9afe64b
BrowserProcessor: use FetchHTTP2's configured proxy as an upstream proxy
2025-06-10 09:01:02 +09:00
Alex Osborne
cc50af72be
FetchHTTP2: Add HTTP proxy support
2025-06-09 22:28:49 +09:00
Alex Osborne
020ea34cb6
Bean reference: Add record builders under WARCWriterChainProcessor
2025-06-09 15:42:13 +09:00
Alex Osborne
e68e4975c9
Rename to BrowserProcessor and -Behavior
...
After I added them to the Bean reference they seemed out of place with the DecideRules and RecordBuilders.
We had two classes called 'Browser' which is potentially confusing. It's also probably good to more clearly differentiate the ExtractLinks browser behavior from the Extractor processors.
2025-06-09 15:35:25 +09:00
Alex Osborne
cf447f3bd9
Add Browser processor and behaviors to bean reference
2025-06-09 15:19:09 +09:00
Alex Osborne
8df957d77a
Add CHANGELOG.md entry for Browser processor
2025-06-09 14:42:31 +09:00
Alex Osborne
1aa6b82d32
Remove stray footnote reference
2025-06-09 11:49:07 +09:00
Alex Osborne
faf251359d
WbmPersistLoadProcessorTest: Remove dependency on easymock and re-enable test
...
Instead of mocking the HttpClient let's just run a test web server. This means we can drop easymock entirely as it was only being used for this one disabled test.
2025-06-09 10:17:16 +09:00
Alex Osborne
6782fc065d
Update dependencies
...
- **junit-jupiter**: 5.12.2 → 5.13.1
- **spring**: 6.2.6 → 6.2.7
- **codemirror__language**: 6.11.0 → 6.11.1
- **codemirror__view**: 6.36.5 → 6.37.1
2025-06-09 10:17:16 +09:00
Alex Osborne
52bbd8091b
Merge pull request #653 from internetarchive/bidi
...
Add Browser processor using WebDriver BiDi
2025-06-09 09:36:06 +09:00
Alex Osborne
7e15d2fda1
Merge pull request #656 from internetarchive/robotstxt-wildcard
...
Support * and $ wildcards in robots.txt
2025-06-09 09:34:16 +09:00
Alex Osborne
c7b7ee1842
Support * and $ wildcards in robots.txt
2025-06-07 10:37:11 +09:00
Alex Osborne
1161d87217
Browser: Add processor report
2025-06-05 10:48:15 +09:00
Alex Osborne
2210cb7262
Browser: Handle navigation abort from downloads starting
2025-06-04 17:44:42 +09:00
Alex Osborne
5131ea4a80
Browser: Disable downloads in Firefox and Chrome
...
This hopefully will stop us filling up ~/Downloads with random junk.
2025-06-04 17:44:42 +09:00
Alex Osborne