lee/heritrix3
1
0 Fork 0
mirror of https://github.com/internetarchive/heritrix3.git synced 2026-10-11 23:01:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,045 Commits 41 Branches 34 Tags
a09f38570da5fbc69ca856175cf52002e85feda8
Commit Graph
926 Commits
Author SHA1 Message Date
Alex Osborne a09f38570d Only write heritrix_dmesg.log when requested (#779) 
Only write heritrix_dmesg.log when requested

Heritrix.java always created heritrix_dmesg.log in $HERITRIX_HOME, even
though only the bin/heritrix script reads it, and only when it starts
Heritrix in the background. This got in the way of making
$HERITRIX_HOME read-only for Docker and systemd setups.

The startup log is now written only when the heritrix.dmesg system
property gives its path. bin/heritrix sets this property in background
mode and keeps the file in the same place as before.

Also fix a typo in bin/heritrix ($startmessage) that stopped it
removing a heritrix_dmesg.log left over from a previous start.

Set -Dheritrix.dmesg in heritrix.cmd too.
 2026-10-06 11:29:51 +09:00
Alex Osborne 0ca9e5dc2a Merge pull request #778 from internetarchive/ato/botblock-detector2 
BotBlockDetector: add additional services and enable by default
 2026-10-06 11:20:02 +09:00
Alex Osborne 838523d33a BotBlockDetector: enable in default profiles 2026-10-02 09:24:44 +09:00
dependabot[bot] c36460a0f3 Bump the minor-and-patch group with 13 updates 
Bumps the minor-and-patch group with 13 updates:

| Package | From | To |
| --- | --- | --- |
| [org.eclipse.jetty:jetty-bom](https://github.com/jetty/jetty.project) | `12.0.38` | `12.0.39` |
| [org.eclipse.jetty.ee10:jetty-ee10-bom](https://github.com/jetty/jetty.project) | `12.0.38` | `12.0.39` |
| [org.slf4j:slf4j-bom](https://github.com/qos-ch/slf4j) | `2.0.18` | `2.0.20` |
| [org.apache.maven.plugins:maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) | `3.15.0` | `3.16.0` |
| [org.apache.maven.plugins:maven-surefire-plugin](https://github.com/apache/maven-surefire) | `3.5.6` | `3.6.0` |
| [org.codehaus.mojo:versions-maven-plugin](https://github.com/mojohaus/versions) | `2.21.0` | `2.22.0` |
| [org.webjars:webjars-locator-lite](https://github.com/webjars/webjars-locator-lite) | `1.1.4` | `1.1.5` |
| org.webjars.npm:codemirror__commands | `6.10.4` | `6.11.0` |
| org.webjars.npm:codemirror__state | `6.7.1` | `6.7.4` |
| org.webjars.npm:codemirror__view | `6.43.7` | `6.43.11` |
| org.webjars.npm:marijn__find-cluster-break | `1.0.3` | `1.0.4` |
| [com.rabbitmq:amqp-client](https://github.com/rabbitmq/rabbitmq-java-client) | `5.35.0` | `5.36.0` |
| [at.yawk.lz4:lz4-java](https://github.com/yawkat/lz4-java) | `1.11.2` | `1.12.0` |


Updates `org.eclipse.jetty:jetty-bom` from 12.0.38 to 12.0.39
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.38...jetty-12.0.39)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.38 to 12.0.39
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.38...jetty-12.0.39)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.38 to 12.0.39
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.38...jetty-12.0.39)

Updates `org.slf4j:slf4j-bom` from 2.0.18 to 2.0.20
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](https://github.com/qos-ch/slf4j/compare/v_2.0.18...v_2.0.20)

Updates `org.apache.maven.plugins:maven-compiler-plugin` from 3.15.0 to 3.16.0
- [Release notes](https://github.com/apache/maven-compiler-plugin/releases)
- [Commits](https://github.com/apache/maven-compiler-plugin/compare/maven-compiler-plugin-3.15.0...maven-compiler-plugin-3.16.0)

Updates `org.apache.maven.plugins:maven-surefire-plugin` from 3.5.6 to 3.6.0
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](https://github.com/apache/maven-surefire/compare/surefire-3.5.6...surefire-3.6.0)

Updates `org.codehaus.mojo:versions-maven-plugin` from 2.21.0 to 2.22.0
- [Release notes](https://github.com/mojohaus/versions/releases)
- [Changelog](https://github.com/mojohaus/versions/blob/master/ReleaseNotes.md)
- [Commits](https://github.com/mojohaus/versions/compare/2.21.0...2.22.0)

Updates `org.webjars:webjars-locator-lite` from 1.1.4 to 1.1.5
- [Release notes](https://github.com/webjars/webjars-locator-lite/releases)
- [Commits](https://github.com/webjars/webjars-locator-lite/compare/webjars-locator-lite-1.1.4...webjars-locator-lite-1.1.5)

Updates `org.webjars.npm:codemirror__commands` from 6.10.4 to 6.11.0

Updates `org.webjars.npm:codemirror__state` from 6.7.1 to 6.7.4

Updates `org.webjars.npm:codemirror__view` from 6.43.7 to 6.43.11

Updates `org.webjars.npm:marijn__find-cluster-break` from 1.0.3 to 1.0.4

Updates `com.rabbitmq:amqp-client` from 5.35.0 to 5.36.0
- [Release notes](https://github.com/rabbitmq/rabbitmq-java-client/releases)
- [Commits](https://github.com/rabbitmq/rabbitmq-java-client/compare/v5.35.0...v5.36.0)

Updates `at.yawk.lz4:lz4-java` from 1.11.2 to 1.12.0
- [Release notes](https://github.com/yawkat/lz4-java/releases)
- [Changelog](https://github.com/yawkat/lz4-java/blob/main/CHANGES.md)
- [Commits](https://github.com/yawkat/lz4-java/compare/v1.11.2...v1.12.0)

---
updated-dependencies:
- dependency-name: org.eclipse.jetty:jetty-bom
  dependency-version: 12.0.39
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.39
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.39
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.slf4j:slf4j-bom
  dependency-version: 2.0.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-compiler-plugin
  dependency-version: 3.16.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-version: 3.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.codehaus.mojo:versions-maven-plugin
  dependency-version: 2.22.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars:webjars-locator-lite
  dependency-version: 1.1.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__commands
  dependency-version: 6.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__state
  dependency-version: 6.7.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__view
  dependency-version: 6.43.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:marijn__find-cluster-break
  dependency-version: 1.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.rabbitmq:amqp-client
  dependency-version: 5.36.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: at.yawk.lz4:lz4-java
  dependency-version: 1.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-10-02 08:51:34 +09:00
Alex Osborne f6a14ac1f6 Implement invoker commands for older browsers like Firefox ESR 
Fixes #773
 2026-10-01 19:05:53 +09:00
Alex Osborne 0d3582a136 [maven-release-plugin] prepare for next development iteration 2026-09-10 17:29:57 +09:00
Alex Osborne 48f9c37f83 [maven-release-plugin] prepare release 3.17.1 2026-09-10 17:29:50 +09:00
Leslie Bellony b0beb39518 Call tallySourceStats only if the URI is effectively crawled 2026-09-09 17:12:29 +02:00
Leslie Bellony 7b43a91e38 Include Heritrix status code in extended SourceTags report 2026-09-08 11:13:33 +02:00
dependabot[bot] f261b5beec Bump the minor-and-patch group with 2 updates 
Bumps the minor-and-patch group with 2 updates: [org.apache.groovy:groovy-bom](https://github.com/apache/groovy) and org.freemarker:freemarker.


Updates `org.apache.groovy:groovy-bom` from 5.1.0 to 5.1.1
- [Commits](https://github.com/apache/groovy/commits)

Updates `org.freemarker:freemarker` from 2.3.34 to 2.3.35

---
updated-dependencies:
- dependency-name: org.apache.groovy:groovy-bom
  dependency-version: 5.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.freemarker:freemarker
  dependency-version: 2.3.35
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-09-02 10:17:33 +09:00
Alex Osborne b5f32616c8 Fix javadoc warnings 2026-08-25 11:02:15 +09:00
Alex Osborne 527b202752 [maven-release-plugin] prepare for next development iteration 2026-08-25 10:17:24 +09:00
Alex Osborne 6eeb779923 [maven-release-plugin] prepare release 3.17.0 2026-08-25 10:17:12 +09:00
dependabot[bot] 1ece1be25c Bump the minor-and-patch group with 14 updates 
Bumps the minor-and-patch group with 14 updates:

| Package | From | To |
| --- | --- | --- |
| [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit-framework) | `6.1.2` | `6.1.3` |
| [org.apache.groovy:groovy-bom](https://github.com/apache/groovy) | `5.0.8` | `5.1.0` |
| [com.fasterxml.jackson:jackson-bom](https://github.com/FasterXML/jackson-bom) | `2.22.1` | `2.22.2` |
| [org.eclipse.jetty:jetty-bom](https://github.com/jetty/jetty.project) | `12.0.37` | `12.0.38` |
| [org.eclipse.jetty.ee10:jetty-ee10-bom](https://github.com/jetty/jetty.project) | `12.0.37` | `12.0.38` |
| [org.springframework:spring-core](https://github.com/spring-projects/spring-framework) | `7.0.8` | `7.0.9` |
| [org.springframework:spring-beans](https://github.com/spring-projects/spring-framework) | `7.0.8` | `7.0.9` |
| [org.springframework:spring-context](https://github.com/spring-projects/spring-framework) | `7.0.8` | `7.0.9` |
| [org.springframework:spring-expression](https://github.com/spring-projects/spring-framework) | `7.0.8` | `7.0.9` |
| [com.github.mwiede:jsch](https://github.com/mwiede/jsch) | `2.28.6` | `2.28.7` |
| org.webjars.npm:codemirror__commands | `6.10.2` | `6.10.4` |
| org.webjars.npm:codemirror__view | `6.43.6` | `6.43.7` |
| [com.rabbitmq:amqp-client](https://github.com/rabbitmq/rabbitmq-java-client) | `5.34.0` | `5.35.0` |
| [at.yawk.lz4:lz4-java](https://github.com/yawkat/lz4-java) | `1.11.1` | `1.11.2` |


Updates `org.junit.jupiter:junit-jupiter` from 6.1.2 to 6.1.3
- [Release notes](https://github.com/junit-team/junit-framework/releases)
- [Commits](https://github.com/junit-team/junit-framework/compare/r6.1.2...r6.1.3)

Updates `org.apache.groovy:groovy-bom` from 5.0.8 to 5.1.0
- [Commits](https://github.com/apache/groovy/commits)

Updates `com.fasterxml.jackson:jackson-bom` from 2.22.1 to 2.22.2
- [Commits](https://github.com/FasterXML/jackson-bom/compare/jackson-bom-2.22.1...jackson-bom-2.22.2)

Updates `org.eclipse.jetty:jetty-bom` from 12.0.37 to 12.0.38
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.37...jetty-12.0.38)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.37 to 12.0.38
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.37...jetty-12.0.38)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.37 to 12.0.38
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.37...jetty-12.0.38)

Updates `org.springframework:spring-core` from 7.0.8 to 7.0.9
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.8...v7.0.9)

Updates `org.springframework:spring-beans` from 7.0.8 to 7.0.9
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.8...v7.0.9)

Updates `org.springframework:spring-context` from 7.0.8 to 7.0.9
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.8...v7.0.9)

Updates `org.springframework:spring-expression` from 7.0.8 to 7.0.9
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.8...v7.0.9)

Updates `org.springframework:spring-beans` from 7.0.8 to 7.0.9
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.8...v7.0.9)

Updates `org.springframework:spring-context` from 7.0.8 to 7.0.9
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.8...v7.0.9)

Updates `org.springframework:spring-expression` from 7.0.8 to 7.0.9
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.8...v7.0.9)

Updates `com.github.mwiede:jsch` from 2.28.6 to 2.28.7
- [Release notes](https://github.com/mwiede/jsch/releases)
- [Changelog](https://github.com/mwiede/jsch/blob/master/ChangeLog)
- [Commits](https://github.com/mwiede/jsch/compare/jsch-2.28.6...jsch-2.28.7)

Updates `org.webjars.npm:codemirror__commands` from 6.10.2 to 6.10.4

Updates `org.webjars.npm:codemirror__view` from 6.43.6 to 6.43.7

Updates `com.rabbitmq:amqp-client` from 5.34.0 to 5.35.0
- [Release notes](https://github.com/rabbitmq/rabbitmq-java-client/releases)
- [Commits](https://github.com/rabbitmq/rabbitmq-java-client/compare/v5.34.0...v5.35.0)

Updates `at.yawk.lz4:lz4-java` from 1.11.1 to 1.11.2
- [Release notes](https://github.com/yawkat/lz4-java/releases)
- [Changelog](https://github.com/yawkat/lz4-java/blob/main/CHANGES.md)
- [Commits](https://github.com/yawkat/lz4-java/compare/v1.11.1...v1.11.2)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-version: 6.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.groovy:groovy-bom
  dependency-version: 5.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: com.fasterxml.jackson:jackson-bom
  dependency-version: 2.22.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty:jetty-bom
  dependency-version: 12.0.38
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.38
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.38
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-core
  dependency-version: 7.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-beans
  dependency-version: 7.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-context
  dependency-version: 7.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-expression
  dependency-version: 7.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-beans
  dependency-version: 7.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-context
  dependency-version: 7.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-expression
  dependency-version: 7.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.github.mwiede:jsch
  dependency-version: 2.28.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__commands
  dependency-version: 6.10.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__view
  dependency-version: 6.43.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.rabbitmq:amqp-client
  dependency-version: 5.35.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: at.yawk.lz4:lz4-java
  dependency-version: 1.11.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-08-25 00:40:49 +00:00
Leslie Bellony 78cc54eb1f Issue #753 verify if bdb is started before generating reports 2026-08-04 18:36:16 +02:00
Leslie Bellony 71bffa42e6 Issue #748 extend SourceTags report with status code 2026-08-03 14:25:53 +02:00
Leslie Bellony 58054b962d Issue #747 add heritrix response codes in ResponseCode report 2026-08-03 11:18:11 +02:00
dependabot[bot] b1b07fc477 Bump the minor-and-patch group with 21 updates 
Bumps the minor-and-patch group with 21 updates:

| Package | From | To |
| --- | --- | --- |
| [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit-framework) | `6.1.1` | `6.1.2` |
| [org.apache.groovy:groovy-bom](https://github.com/apache/groovy) | `5.0.6` | `5.0.8` |
| [com.fasterxml.jackson:jackson-bom](https://github.com/FasterXML/jackson-bom) | `2.22.0` | `2.22.1` |
| [org.eclipse.jetty:jetty-bom](https://github.com/jetty/jetty.project) | `12.0.36` | `12.0.37` |
| [org.eclipse.jetty.ee10:jetty-ee10-bom](https://github.com/jetty/jetty.project) | `12.0.36` | `12.0.37` |
| [commons-codec:commons-codec](https://github.com/apache/commons-codec) | `1.22.0` | `1.22.1` |
| [com.github.mwiede:jsch](https://github.com/mwiede/jsch) | `2.28.3` | `2.28.6` |
| [com.github.bbottema:java-socks-proxy-server](https://github.com/bbottema/java-socks-proxy-server) | `4.1.2` | `4.2.0` |
| org.apache.pdfbox:pdfbox | `3.0.7` | `3.0.8` |
| [org.webjars:webjars-locator-lite](https://github.com/webjars/webjars-locator-lite) | `1.1.3` | `1.1.4` |
| org.webjars.npm:codemirror__autocomplete | `6.20.1` | `6.20.3` |
| org.webjars.npm:codemirror__language | `6.12.2` | `6.12.4` |
| org.webjars.npm:codemirror__state | `6.5.4` | `6.7.1` |
| org.webjars.npm:codemirror__view | `6.39.16` | `6.43.6` |
| org.webjars.npm:codemirror__lint | `6.9.5` | `6.9.7` |
| org.webjars.npm:crelt | `1.0.6` | `1.0.7` |
| org.webjars.npm:marijn__find-cluster-break | `1.0.2` | `1.0.3` |
| [org.webjars.npm:lezer__common](https://github.com/lezer-parser/common) | `1.5.1` | `1.5.2` |
| org.webjars.npm:lezer__lr | `1.4.8` | `1.4.10` |
| [com.rabbitmq:amqp-client](https://github.com/rabbitmq/rabbitmq-java-client) | `5.33.0` | `5.34.0` |
| [at.yawk.lz4:lz4-java](https://github.com/yawkat/lz4-java) | `1.11.0` | `1.11.1` |


Updates `org.junit.jupiter:junit-jupiter` from 6.1.1 to 6.1.2
- [Release notes](https://github.com/junit-team/junit-framework/releases)
- [Commits](https://github.com/junit-team/junit-framework/compare/r6.1.1...r6.1.2)

Updates `org.apache.groovy:groovy-bom` from 5.0.6 to 5.0.8
- [Commits](https://github.com/apache/groovy/commits)

Updates `com.fasterxml.jackson:jackson-bom` from 2.22.0 to 2.22.1
- [Commits](https://github.com/FasterXML/jackson-bom/compare/jackson-bom-2.22.0...jackson-bom-2.22.1)

Updates `org.eclipse.jetty:jetty-bom` from 12.0.36 to 12.0.37
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.36...jetty-12.0.37)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.36 to 12.0.37
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.36...jetty-12.0.37)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.36 to 12.0.37
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.36...jetty-12.0.37)

Updates `commons-codec:commons-codec` from 1.22.0 to 1.22.1
- [Changelog](https://github.com/apache/commons-codec/blob/master/RELEASE-NOTES.txt)
- [Commits](https://github.com/apache/commons-codec/compare/rel/commons-codec-1.22.0...rel/commons-codec-1.22.1)

Updates `com.github.mwiede:jsch` from 2.28.3 to 2.28.6
- [Release notes](https://github.com/mwiede/jsch/releases)
- [Changelog](https://github.com/mwiede/jsch/blob/master/ChangeLog)
- [Commits](https://github.com/mwiede/jsch/compare/jsch-2.28.3...jsch-2.28.6)

Updates `com.github.bbottema:java-socks-proxy-server` from 4.1.2 to 4.2.0
- [Release notes](https://github.com/bbottema/java-socks-proxy-server/releases)
- [Changelog](https://github.com/bbottema/java-socks-proxy-server/blob/master/RELEASE.txt)
- [Commits](https://github.com/bbottema/java-socks-proxy-server/compare/4.1.2...4.2.0)

Updates `org.apache.pdfbox:pdfbox` from 3.0.7 to 3.0.8

Updates `org.webjars:webjars-locator-lite` from 1.1.3 to 1.1.4
- [Release notes](https://github.com/webjars/webjars-locator-lite/releases)
- [Commits](https://github.com/webjars/webjars-locator-lite/compare/webjars-locator-lite-1.1.3...webjars-locator-lite-1.1.4)

Updates `org.webjars.npm:codemirror__autocomplete` from 6.20.1 to 6.20.3

Updates `org.webjars.npm:codemirror__language` from 6.12.2 to 6.12.4

Updates `org.webjars.npm:codemirror__state` from 6.5.4 to 6.7.1

Updates `org.webjars.npm:codemirror__view` from 6.39.16 to 6.43.6

Updates `org.webjars.npm:codemirror__lint` from 6.9.5 to 6.9.7

Updates `org.webjars.npm:crelt` from 1.0.6 to 1.0.7

Updates `org.webjars.npm:marijn__find-cluster-break` from 1.0.2 to 1.0.3

Updates `org.webjars.npm:lezer__common` from 1.5.1 to 1.5.2
- [Changelog](https://github.com/lezer-parser/common/blob/main/CHANGELOG.md)
- [Commits](https://github.com/lezer-parser/common/compare/1.5.1...1.5.2)

Updates `org.webjars.npm:lezer__lr` from 1.4.8 to 1.4.10

Updates `com.rabbitmq:amqp-client` from 5.33.0 to 5.34.0
- [Release notes](https://github.com/rabbitmq/rabbitmq-java-client/releases)
- [Commits](https://github.com/rabbitmq/rabbitmq-java-client/compare/v5.33.0...v5.34.0)

Updates `at.yawk.lz4:lz4-java` from 1.11.0 to 1.11.1
- [Release notes](https://github.com/yawkat/lz4-java/releases)
- [Changelog](https://github.com/yawkat/lz4-java/blob/main/CHANGES.md)
- [Commits](https://github.com/yawkat/lz4-java/compare/v1.11.0...v1.11.1)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-version: 6.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.groovy:groovy-bom
  dependency-version: 5.0.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.fasterxml.jackson:jackson-bom
  dependency-version: 2.22.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty:jetty-bom
  dependency-version: 12.0.37
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.37
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.37
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: commons-codec:commons-codec
  dependency-version: 1.22.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.github.mwiede:jsch
  dependency-version: 2.28.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.github.bbottema:java-socks-proxy-server
  dependency-version: 4.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.apache.pdfbox:pdfbox
  dependency-version: 3.0.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars:webjars-locator-lite
  dependency-version: 1.1.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__autocomplete
  dependency-version: 6.20.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__language
  dependency-version: 6.12.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__state
  dependency-version: 6.7.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__view
  dependency-version: 6.43.6
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__lint
  dependency-version: 6.9.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:crelt
  dependency-version: 1.0.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:marijn__find-cluster-break
  dependency-version: 1.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:lezer__common
  dependency-version: 1.5.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:lezer__lr
  dependency-version: 1.4.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.rabbitmq:amqp-client
  dependency-version: 5.34.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: at.yawk.lz4:lz4-java
  dependency-version: 1.11.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-08-01 23:41:17 +00:00
Alex Osborne 97d8307420 BrowserProcessor: restart the browser automatically if it crashes 2026-07-10 16:12:52 +09:00
Alex Osborne d7676b1b67 BrowserProcessor: log failures continuing intercepted requests 2026-07-10 16:12:47 +09:00
Alex Osborne 1b107f06c1 BrowserProcessor: guard stop() against a partially failed start() 
If start() throws after the proxy is started but before the browser
launches, stop() could NPE on the null fields.
 2026-07-10 15:59:47 +09:00
Alex Osborne d03a43266a BrowserProcessor: catch tab close failures 
If closing the tab in the finally block throws, just log it rather than propagating to the toe thread.
 2026-07-10 15:59:46 +09:00
Alex Osborne e276d83f4b BrowserProcessor: avoid NPE when a WebDriverException has no message 2026-07-10 15:48:42 +09:00
Alex Osborne 789603c1b7 [maven-release-plugin] prepare for next development iteration 2026-07-03 10:20:27 +09:00
Alex Osborne 7af0bf56ea [maven-release-plugin] prepare release 3.16.0 2026-07-03 10:20:16 +09:00
Alex Osborne 6c7a54b5c5 [maven-release-plugin] prepare for next development iteration 2026-07-03 08:49:28 +09:00
Alex Osborne 9fdd5543af [maven-release-plugin] prepare release 3.16.0 2026-07-03 08:49:14 +09:00
Alex Osborne d3da2f9afe BrowserProcessor: handle result.isFailed() in SubresourceRecorder.onComplete() 2026-06-18 23:30:48 +09:00
Alex Osborne 2c9f167ae3 BrowserProcessor: handle recording truncation with length and timeout limits 2026-06-18 23:25:20 +09:00
Alex Osborne 4cb48ec5dc BrowserProcessor: fix content digest and recording limits for subresources 2026-06-18 23:18:23 +09:00
Alex Osborne ff6f98d38d BrowserProcessor: set User-Agent header 
Ensures requests made by the browser use the user-agent string configured for the page in the job config.
 2026-06-18 23:00:21 +09:00
Alex Osborne e156997dbd MitmProxy: Fix request recording 
This fixes a bug where POST requests would get recorded with a doubled header and a bunch of nulls at the end.

It seems that the onRequestContent() listener doesn't pass us the right buffer when proxying. The buffer seems to contain client-to-server data and the buffer's limit isn't set. So instead this records the request content by overriding newProxyToServerRequestContent.
 2026-06-17 16:56:24 +09:00
Alex Osborne 709ac00dd2 Add PaginationBehavior: repeatedly clicks next-page and extracts links 2026-06-11 13:34:55 +09:00
Alex Osborne 54b95f5e97 groovy profile: remove stray } 2026-06-10 18:23:12 +09:00
Alex Osborne bd34a7a0a9 Log the exception when creating new jobs fails 2026-06-10 17:43:57 +09:00
dependabot[bot] 3406c14632 Bump the minor-and-patch group with 11 updates 
Bumps the minor-and-patch group with 11 updates:

| Package | From | To |
| --- | --- | --- |
| [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit-framework) | `6.0.3` | `6.1.0` |
| [org.apache.groovy:groovy-bom](https://github.com/apache/groovy) | `5.0.5` | `5.0.6` |
| [com.fasterxml.jackson:jackson-bom](https://github.com/FasterXML/jackson-bom) | `2.21.3` | `2.22.0` |
| [org.eclipse.jetty:jetty-bom](https://github.com/jetty/jetty.project) | `12.0.34` | `12.0.36` |
| [org.eclipse.jetty.ee10:jetty-ee10-bom](https://github.com/jetty/jetty.project) | `12.0.34` | `12.0.36` |
| [org.slf4j:slf4j-bom](https://github.com/qos-ch/slf4j) | `2.0.17` | `2.0.18` |
| [org.apache.maven.plugins:maven-surefire-plugin](https://github.com/apache/maven-surefire) | `3.5.5` | `3.5.6` |
| [dnsjava:dnsjava](https://github.com/dnsjava/dnsjava) | `3.6.4` | `3.6.5` |
| org.glassfish.jaxb:jaxb-runtime | `4.0.7` | `4.0.9` |
| [com.rabbitmq:amqp-client](https://github.com/rabbitmq/rabbitmq-java-client) | `5.30.0` | `5.31.0` |
| org.apache.kafka:kafka-clients | `4.2.0` | `4.3.0` |


Updates `org.junit.jupiter:junit-jupiter` from 6.0.3 to 6.1.0
- [Release notes](https://github.com/junit-team/junit-framework/releases)
- [Commits](https://github.com/junit-team/junit-framework/compare/r6.0.3...r6.1.0)

Updates `org.apache.groovy:groovy-bom` from 5.0.5 to 5.0.6
- [Commits](https://github.com/apache/groovy/commits)

Updates `com.fasterxml.jackson:jackson-bom` from 2.21.3 to 2.22.0
- [Commits](https://github.com/FasterXML/jackson-bom/compare/jackson-bom-2.21.3...jackson-bom-2.22.0)

Updates `org.eclipse.jetty:jetty-bom` from 12.0.34 to 12.0.36
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.34...jetty-12.0.36)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.34 to 12.0.36
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.34...jetty-12.0.36)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.34 to 12.0.36
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.34...jetty-12.0.36)

Updates `org.slf4j:slf4j-bom` from 2.0.17 to 2.0.18
- [Release notes](https://github.com/qos-ch/slf4j/releases)
- [Commits](https://github.com/qos-ch/slf4j/compare/v_2.0.17...v_2.0.18)

Updates `org.apache.maven.plugins:maven-surefire-plugin` from 3.5.5 to 3.5.6
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](https://github.com/apache/maven-surefire/compare/surefire-3.5.5...surefire-3.5.6)

Updates `dnsjava:dnsjava` from 3.6.4 to 3.6.5
- [Release notes](https://github.com/dnsjava/dnsjava/releases)
- [Changelog](https://github.com/dnsjava/dnsjava/blob/master/Changelog)
- [Commits](https://github.com/dnsjava/dnsjava/commits)

Updates `org.glassfish.jaxb:jaxb-runtime` from 4.0.7 to 4.0.9

Updates `com.rabbitmq:amqp-client` from 5.30.0 to 5.31.0
- [Release notes](https://github.com/rabbitmq/rabbitmq-java-client/releases)
- [Commits](https://github.com/rabbitmq/rabbitmq-java-client/compare/v5.30.0...v5.31.0)

Updates `org.apache.kafka:kafka-clients` from 4.2.0 to 4.3.0

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.apache.groovy:groovy-bom
  dependency-version: 5.0.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.fasterxml.jackson:jackson-bom
  dependency-version: 2.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty:jetty-bom
  dependency-version: 12.0.36
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.36
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.36
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.slf4j:slf4j-bom
  dependency-version: 2.0.18
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-version: 3.5.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: dnsjava:dnsjava
  dependency-version: 3.6.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.glassfish.jaxb:jaxb-runtime
  dependency-version: 4.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.rabbitmq:amqp-client
  dependency-version: 5.31.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.apache.kafka:kafka-clients
  dependency-version: 4.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-06-03 16:56:20 +00:00
Alex Osborne e0b46789bc [maven-release-plugin] prepare for next development iteration 2026-05-26 11:51:31 +09:00
Alex Osborne bc28f4d1ae [maven-release-plugin] prepare release 3.15.0 2026-05-26 11:51:25 +09:00
Alex Osborne d3defaacd2 Merge pull request #732 from internetarchive/remove-foundation-js 
Remove obsolete JavaScript libraries
 2026-05-12 16:44:16 +09:00
Alex Osborne 56a1fc2198 Remove Internet Explorer compatibility shims 
There's no need for these anymore, IE has been sunset.
 2026-05-04 08:45:41 +09:00
Alex Osborne 05f5243207 Remove Foundation JS libraries 
Foundation 4 has been obsolete since 2013, and these libraries trigger security alerts. We're also not using them to do much, just the modal dialogs, alert close buttons and collapsing the top-bar on mobile. This replaces the modals with the native HTML dialog element and implements the trivial click event handlers needed for hiding and showing things.
 2026-05-04 08:45:40 +09:00
Alex Osborne 71ae9a013b Merge pull request #728 from internetarchive/groovy-config-editor 
Add Groovy mode to config editor
 2026-05-03 12:06:26 +09:00
Alex Osborne 82181f3d9e Add Groovy mode to config editor 
Includes bean and property name autocomplete, similar to the XML mode. Completion of short class names auto-inserts import statements as needed.
 2026-04-28 09:08:23 +09:00
Alex Osborne 9e43fde053 Recognise Groovy profiles during job discovery 2026-04-27 17:56:23 +09:00
Alex Osborne b1e13c550b Add profile selector to job creation form 
This means:

- Job profiles can be used without having to hunt for them among the regular jobs.
- The Groovy default profile can be selected as an alternative to Spring XML one.
- The default profiles can be customised by creating a job profile with the same name.
 2026-04-27 17:40:37 +09:00
Alex Osborne 55c1c20295 Remove httpProxyHost and httpProxyPort from fetcher properties after test execution 2026-04-25 21:28:44 +09:00
Alex Osborne 94aaebfca2 [maven-release-plugin] prepare for next development iteration 2026-04-06 17:11:14 +09:00
Alex Osborne 56f8aee836 [maven-release-plugin] prepare release 3.14.1 2026-04-06 17:11:04 +09:00
Alex Osborne 7490998843 Merge pull request #721 from simons-hub/fix/warn-on-precedence-clipping 
Log warning when URI precedence exceeds maximum 127
 2026-04-06 16:54:56 +09:00
Alex Osborne 1620dc06ce Merge pull request #713 from internetarchive/xxe 
Enable XXE protection when parsing crawl job XML
 2026-04-06 16:40:29 +09:00