c50f4dc461
Bump org.apache.maven.plugins:maven-resources-plugin from 2.5 to 3.3.1 ( #674 )
2025-10-27 01:17:23 +00:00
b988940eff
[maven-release-plugin] prepare for next development iteration
2025-09-22 13:57:47 +09:00
e3bdd7220e
[maven-release-plugin] prepare release 3.11.0
2025-09-22 13:57:41 +09:00
1ed64142ea
Bump jsch from 2.27.2 to 2.27.3
2025-09-22 13:46:42 +09:00
94c0daeec5
Bump spring from 6.2.10 to 6.2.11
2025-09-22 13:45:11 +09:00
20dcc89244
[maven-release-plugin] prepare for next development iteration
2025-08-29 17:26:11 +09:00
7a2c676dc9
[maven-release-plugin] prepare release 3.10.2
2025-08-29 17:26:05 +09:00
31539e2d62
Use Groovy BOM to ensure consistent versions
2025-08-28 15:35:14 +09:00
3cde816e95
Use jetty BOM to ensure consistent versions
2025-08-28 15:27:10 +09:00
148477b4d3
Bump commons-net from 3.11.1 to 3.12.0
2025-08-20 17:06:31 +09:00
cbd72abfb7
Bump commons-codec from 1.18.0 to 1.19.0
2025-08-20 17:06:12 +09:00
ab1c9bef5a
Bump commons-cli from 1.9.0 to 1.10.0
2025-08-20 17:05:52 +09:00
362a40f3db
Bump spring from 6.2.9 to 6.2.10
2025-08-20 17:04:20 +09:00
ea67b59e51
[maven-release-plugin] prepare for next development iteration
2025-07-21 17:15:17 +09:00
1cf6762c7f
[maven-release-plugin] prepare release 3.10.1
2025-07-21 17:15:12 +09:00
bb0cb1cea0
Bump spring from 6.2.7 to 6.2.9
2025-07-21 17:00:48 +09:00
2388db4229
Bump commons-lang from 2.6 to 3.18.0
2025-07-21 16:55:31 +09:00
fbc7489e61
Bump commons-io from 2.19.0 to 2.20.0
2025-07-21 16:12:59 +09:00
329bf1c741
Bump jsch from 2.27.0 to 2.27.2
2025-07-21 16:11:08 +09:00
839a8f4604
Bump webarchive-commons from 2.0.1 to 2.0.2
2025-07-15 11:16:48 +09:00
01378ddbbf
[maven-release-plugin] prepare for next development iteration
2025-06-12 22:01:16 +09:00
ecc46a38c3
[maven-release-plugin] prepare release 3.10.0
2025-06-12 22:01:10 +09:00
1ab9b4a206
fix: avoid bdb environment corruption when terminate is called during shutdown
2025-06-11 17:44:52 -07:00
61af1d83c4
MitmProxy: Handle URLs generated by browsers but disallowed by java.net.URI
2025-06-11 20:38:54 +09:00
8a15156fa1
BrowserProcessor: Catch WebDriver timeouts and exceptions
2025-06-11 17:17:02 +09:00
c3f9afe64b
BrowserProcessor: use FetchHTTP2's configured proxy as an upstream proxy
2025-06-10 09:01:02 +09:00
6782fc065d
Update dependencies
...
- **junit-jupiter**: 5.12.2 → 5.13.1
- **spring**: 6.2.6 → 6.2.7
- **codemirror__language**: 6.11.0 → 6.11.1
- **codemirror__view**: 6.36.5 → 6.37.1
2025-06-09 10:17:16 +09:00
2210cb7262
Browser: Handle navigation abort from downloads starting
2025-06-04 17:44:42 +09:00
5131ea4a80
Browser: Disable downloads in Firefox and Chrome
...
This hopefully will stop us filling up ~/Downloads with random junk.
2025-06-04 17:44:42 +09:00
722552feef
Add Browser processor using WebDriver BiDi
...
The Browser processor can load a fetched page in a local web browser, record any requests the browser makes and run behaviors that interact with the page such as scrolling down and extracting links.
This differs from my previous attempt (ExtractorChrome) in a few ways:
- Uses the new WebDriver BiDi standard instead of the Chrome Devtools Protocol. The new protocol is mostly browser-agnostic, more consistent and hopefully more stable.
- Uses a MITM proxy instead of CDP request interception for recording sub-resources. That's partly because BiDi is still missing some key interception APIs. Even so in practice I found the proxy method loads pages faster and more reliably, likely because responses can be streamed incrementally, which helps a lot for large resources or server-sent events.
- Even when HTTP/2 is unavailable, the new FetchHTTP2 module does connection pooling which makes loading browser requests a lot faster. The original FetchHTTP opened a new connection for every request.
- The Browser processor can be configured with a list of behavior beans making it more customizable and extensible.
Obvious areas for future development:
- More Behavior beans: take screenshots, saveg the rendered DOM, run Browsertrix-compatible behavior scripts
- Support for remote WebDrivers (e.g. Selenium Server or cloud services)
2025-06-04 17:44:42 +09:00
567e27181d
Upgrade webarchive-commons from 1.3.0 to 2.0.1 (removes httpclient 3)
2025-05-21 17:39:21 +09:00
1827d93fe6
[maven-release-plugin] prepare for next development iteration
2025-05-13 12:25:24 +09:00
dd40970f15
[maven-release-plugin] prepare release 3.9.0
2025-05-13 12:25:16 +09:00
f5732e1a98
Bump jsch from 0.2.26 to 2.27.0
2025-05-09 18:19:21 +09:00
eb1205f6d5
Upgrade all tests to junit 5
2025-05-09 00:07:18 +09:00
bd8d6fb2c5
Bump commons-io from 2.18.0 to 2.19.0
2025-05-08 17:41:39 +09:00
d25a0b96ca
Bump spring from 6.2.5 to 6.2.6
2025-05-08 17:41:14 +09:00
e7fe566919
fix: autoregister Histotable and HighestUriPrecedenceProvider
2025-04-22 17:29:52 -07:00
75377c5a3b
fix: FieldSerializer wasn't passing unit tests. Switch to JavaSerializer
2025-04-21 18:16:47 -07:00
c7784f18e9
feat: Add serializer for Histotable used by HighestUriPrecedenceProvider
2025-04-21 17:20:38 -07:00
e9b60404a6
[maven-release-plugin] prepare for next development iteration
2025-04-01 17:43:31 +09:00
2323e2251e
[maven-release-plugin] prepare release 3.8.0
2025-04-01 17:43:25 +09:00
1a605cf9a6
Remove unused utility class JavaLiterals
...
Apache Commons StringEscapeUtils which we use for other things provides
this anyway.
2025-04-01 16:11:10 +09:00
60f512b4b0
Fix compiler warning: deprecated item is not annotated with @Deprecated
2025-03-31 15:12:21 +09:00
f6b083ecfb
Remove usages of deprecated Integer and Long constructors
2025-03-31 15:04:00 +09:00
1288e78c54
Bump spring from 6.1.16 to 6.2.5
2025-03-31 14:58:13 +09:00
b9bfd36d28
Bump dnsjava from 3.6.2 to 3.6.3
2025-03-31 14:53:01 +09:00
4688d8cda2
Bump commons-codec from 1.17.2 to 1.18.0
2025-03-31 14:52:42 +09:00
5ecfa7248c
Bump jsch from 0.2.22 to 0.2.24
2025-03-31 14:52:25 +09:00
b02590cca0
Use AtomicInteger serializer rather than change accessibility
2025-03-14 09:14:26 -07:00
dependabot[bot]