lee/heritrix3
1
0
Fork 0
mirror of https://github.com/internetarchive/heritrix3.git synced 2026-08-24 23:46:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,907 Commits 40 Branches 31 Tags
777704a8bcbcdbb6dd52d0db073c2183db9c6323
Commit Graph
2907 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alex Osborne 777704a8bc Update CHANGELOG.md 2026-02-06 23:45:50 +09:00
Alex Osborne 1a2b108663 [maven-release-plugin] prepare for next development iteration 2026-02-06 23:34:10 +09:00
Alex Osborne 9ffead59c0 [maven-release-plugin] prepare release 3.14.0 3.14.0 2026-02-06 23:34:00 +09:00
Alex Osborne 04c2a22d60 Enable auto-publishing in Maven configuration 2026-02-06 23:27:20 +09:00
Alex Osborne e048c6cb86 Update CHANGELOG.md for 3.14.0 release 2026-02-06 23:26:49 +09:00
Alex Osborne d06dca0a68 Bump webarchive-commons from 3.0.2 to 3.0.3 2026-02-06 23:26:34 +09:00
Alex Osborne 274d2ebd5c Merge pull request #710 from internetarchive/ato/fetchhttp2-socks 
FetchHTTP2: Add SOCKS5 proxy support
 2026-02-06 23:05:13 +09:00
Alex Osborne d2562577b1 FetchHTTP2: Add SOCKS5 proxy support 2026-02-05 09:29:24 +09:00
Alex Osborne cb1f36b926 Merge pull request #709 from internetarchive/dependabot/maven/minor-and-patch-38ddffcac1 
Bump the minor-and-patch group with 25 updates
 2026-02-02 18:12:36 +09:00
Alex Osborne fe49a8ae58 Merge pull request #708 from mistydemeo/misty/yt-dlp_filter_-sr 
ExtractorYoutubeDL: skip YouTube super resolution videos
 2026-02-02 18:11:23 +09:00
dependabot[bot] 938c68dbd8 Bump the minor-and-patch group with 25 updates 
Bumps the minor-and-patch group with 25 updates:

| Package | From | To |
| --- | --- | --- |
| [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit-framework) | `6.0.1` | `6.0.2` |
| [org.apache.groovy:groovy-bom](https://github.com/apache/groovy) | `5.0.3` | `5.0.4` |
| [com.fasterxml.jackson:jackson-bom](https://github.com/FasterXML/jackson-bom) | `2.20.1` | `2.21.0` |
| [org.eclipse.jetty:jetty-bom](https://github.com/jetty/jetty.project) | `12.0.31` | `12.0.32` |
| [org.eclipse.jetty.ee10:jetty-ee10-bom](https://github.com/jetty/jetty.project) | `12.0.31` | `12.0.32` |
| [org.apache.maven.plugins:maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) | `3.14.1` | `3.15.0` |
| [org.codehaus.mojo:versions-maven-plugin](https://github.com/mojohaus/versions) | `2.20.1` | `2.21.0` |
| [org.sonatype.central:central-publishing-maven-plugin](https://github.com/sonatype/central-publishing-maven-plugin) | `0.9.0` | `0.10.0` |
| [commons-codec:commons-codec](https://github.com/apache/commons-codec) | `1.20.0` | `1.21.0` |
| [dnsjava:dnsjava](https://github.com/dnsjava/dnsjava) | `3.6.3` | `3.6.4` |
| [org.springframework:spring-core](https://github.com/spring-projects/spring-framework) | `7.0.2` | `7.0.3` |
| [org.springframework:spring-beans](https://github.com/spring-projects/spring-framework) | `7.0.2` | `7.0.3` |
| [org.springframework:spring-context](https://github.com/spring-projects/spring-framework) | `7.0.2` | `7.0.3` |
| [org.springframework:spring-expression](https://github.com/spring-projects/spring-framework) | `7.0.2` | `7.0.3` |
| [org.webjars.npm:codemirror__autocomplete](https://github.com/codemirror/autocomplete) | `6.18.6` | `6.20.0` |
| [org.webjars.npm:codemirror__commands](https://github.com/codemirror/commands) | `6.8.1` | `6.10.1` |
| [org.webjars.npm:codemirror__language](https://github.com/codemirror/language) | `6.11.3` | `6.12.1` |
| [org.webjars.npm:codemirror__state](https://github.com/codemirror/state) | `6.5.2` | `6.5.4` |
| [org.webjars.npm:codemirror__view](https://github.com/codemirror/view) | `6.38.1` | `6.39.11` |
| [org.webjars.npm:codemirror__lint](https://github.com/codemirror/lint) | `6.8.5` | `6.9.2` |
| [org.webjars.npm:lezer__common](https://github.com/lezer-parser/common) | `1.2.3` | `1.5.0` |
| [org.webjars.npm:lezer__lr](https://github.com/lezer-parser/lr) | `1.4.2` | `1.4.7` |
| [org.webjars.npm:lezer__highlight](https://github.com/lezer-parser/highlight) | `1.2.1` | `1.2.3` |
| [org.webjars.npm:style-mod](https://github.com/marijnh/style-mod) | `4.1.2` | `4.1.3` |
| [at.yawk.lz4:lz4-java](https://github.com/yawkat/lz4-java) | `1.10.2` | `1.10.3` |


Updates `org.junit.jupiter:junit-jupiter` from 6.0.1 to 6.0.2
- [Release notes](https://github.com/junit-team/junit-framework/releases)
- [Commits](https://github.com/junit-team/junit-framework/compare/r6.0.1...r6.0.2)

Updates `org.apache.groovy:groovy-bom` from 5.0.3 to 5.0.4
- [Commits](https://github.com/apache/groovy/commits)

Updates `com.fasterxml.jackson:jackson-bom` from 2.20.1 to 2.21.0
- [Commits](https://github.com/FasterXML/jackson-bom/compare/jackson-bom-2.20.1...jackson-bom-2.21.0)

Updates `org.eclipse.jetty:jetty-bom` from 12.0.31 to 12.0.32
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.31...jetty-12.0.32)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.31 to 12.0.32
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.31...jetty-12.0.32)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.31 to 12.0.32
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.31...jetty-12.0.32)

Updates `org.apache.maven.plugins:maven-compiler-plugin` from 3.14.1 to 3.15.0
- [Release notes](https://github.com/apache/maven-compiler-plugin/releases)
- [Commits](https://github.com/apache/maven-compiler-plugin/compare/maven-compiler-plugin-3.14.1...maven-compiler-plugin-3.15.0)

Updates `org.codehaus.mojo:versions-maven-plugin` from 2.20.1 to 2.21.0
- [Release notes](https://github.com/mojohaus/versions/releases)
- [Changelog](https://github.com/mojohaus/versions/blob/master/ReleaseNotes.md)
- [Commits](https://github.com/mojohaus/versions/compare/2.20.1...2.21.0)

Updates `org.sonatype.central:central-publishing-maven-plugin` from 0.9.0 to 0.10.0
- [Commits](https://github.com/sonatype/central-publishing-maven-plugin/commits)

Updates `commons-codec:commons-codec` from 1.20.0 to 1.21.0
- [Changelog](https://github.com/apache/commons-codec/blob/master/RELEASE-NOTES.txt)
- [Commits](https://github.com/apache/commons-codec/compare/rel/commons-codec-1.20.0...rel/commons-codec-1.21.0)

Updates `dnsjava:dnsjava` from 3.6.3 to 3.6.4
- [Release notes](https://github.com/dnsjava/dnsjava/releases)
- [Changelog](https://github.com/dnsjava/dnsjava/blob/master/Changelog)
- [Commits](https://github.com/dnsjava/dnsjava/compare/v3.6.3...v3.6.4)

Updates `org.springframework:spring-core` from 7.0.2 to 7.0.3
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.2...v7.0.3)

Updates `org.springframework:spring-beans` from 7.0.2 to 7.0.3
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.2...v7.0.3)

Updates `org.springframework:spring-context` from 7.0.2 to 7.0.3
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.2...v7.0.3)

Updates `org.springframework:spring-expression` from 7.0.2 to 7.0.3
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.2...v7.0.3)

Updates `org.springframework:spring-beans` from 7.0.2 to 7.0.3
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.2...v7.0.3)

Updates `org.springframework:spring-context` from 7.0.2 to 7.0.3
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.2...v7.0.3)

Updates `org.springframework:spring-expression` from 7.0.2 to 7.0.3
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.2...v7.0.3)

Updates `org.webjars.npm:codemirror__autocomplete` from 6.18.6 to 6.20.0
- [Changelog](https://github.com/codemirror/autocomplete/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/autocomplete/compare/6.18.6...6.20.0)

Updates `org.webjars.npm:codemirror__commands` from 6.8.1 to 6.10.1
- [Changelog](https://github.com/codemirror/commands/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/commands/compare/6.8.1...6.10.1)

Updates `org.webjars.npm:codemirror__language` from 6.11.3 to 6.12.1
- [Changelog](https://github.com/codemirror/language/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/language/compare/6.11.3...6.12.1)

Updates `org.webjars.npm:codemirror__state` from 6.5.2 to 6.5.4
- [Changelog](https://github.com/codemirror/state/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/state/compare/6.5.2...6.5.4)

Updates `org.webjars.npm:codemirror__view` from 6.38.1 to 6.39.11
- [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/view/compare/6.38.1...6.39.11)

Updates `org.webjars.npm:codemirror__lint` from 6.8.5 to 6.9.2
- [Changelog](https://github.com/codemirror/lint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/lint/compare/6.8.5...6.9.2)

Updates `org.webjars.npm:lezer__common` from 1.2.3 to 1.5.0
- [Changelog](https://github.com/lezer-parser/common/blob/main/CHANGELOG.md)
- [Commits](https://github.com/lezer-parser/common/compare/1.2.3...1.5.0)

Updates `org.webjars.npm:lezer__lr` from 1.4.2 to 1.4.7
- [Changelog](https://github.com/lezer-parser/lr/blob/main/CHANGELOG.md)
- [Commits](https://github.com/lezer-parser/lr/compare/1.4.2...1.4.7)

Updates `org.webjars.npm:lezer__highlight` from 1.2.1 to 1.2.3
- [Changelog](https://github.com/lezer-parser/highlight/blob/main/CHANGELOG.md)
- [Commits](https://github.com/lezer-parser/highlight/compare/1.2.1...1.2.3)

Updates `org.webjars.npm:style-mod` from 4.1.2 to 4.1.3
- [Commits](https://github.com/marijnh/style-mod/compare/4.1.2...4.1.3)

Updates `at.yawk.lz4:lz4-java` from 1.10.2 to 1.10.3
- [Release notes](https://github.com/yawkat/lz4-java/releases)
- [Changelog](https://github.com/yawkat/lz4-java/blob/main/CHANGES.md)
- [Commits](https://github.com/yawkat/lz4-java/compare/v1.10.2...v1.10.3)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-version: 6.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.groovy:groovy-bom
  dependency-version: 5.0.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.fasterxml.jackson:jackson-bom
  dependency-version: 2.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty:jetty-bom
  dependency-version: 12.0.32
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.32
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.32
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-compiler-plugin
  dependency-version: 3.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.codehaus.mojo:versions-maven-plugin
  dependency-version: 2.21.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.sonatype.central:central-publishing-maven-plugin
  dependency-version: 0.10.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: commons-codec:commons-codec
  dependency-version: 1.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: dnsjava:dnsjava
  dependency-version: 3.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-core
  dependency-version: 7.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-beans
  dependency-version: 7.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-context
  dependency-version: 7.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-expression
  dependency-version: 7.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-beans
  dependency-version: 7.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-context
  dependency-version: 7.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-expression
  dependency-version: 7.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__autocomplete
  dependency-version: 6.20.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__commands
  dependency-version: 6.10.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__language
  dependency-version: 6.12.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__state
  dependency-version: 6.5.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__view
  dependency-version: 6.39.11
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:codemirror__lint
  dependency-version: 6.9.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:lezer__common
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:lezer__lr
  dependency-version: 1.4.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:lezer__highlight
  dependency-version: 1.2.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.webjars.npm:style-mod
  dependency-version: 4.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: at.yawk.lz4:lz4-java
  dependency-version: 1.10.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-02-01 23:44:24 +00:00
Misty De Méo 3e19a24879 ExtractorYoutubeDL: configuration for skipping -sr 2026-01-30 13:19:47 -08:00
Misty De Méo 49d6c5d830 ExtractorYoutubeDL: skip YouTube super resolution videos 
YouTube has started upscaling older videos that are below a certain
resolution. yt-dlp identifies these via the `-sr` suffix in the
format ID. It's probably a good idea to avoid downloading these, and
just focus on the originals instead.

yt-dlp's `--dump-single-json` option ignores the `--format` argument,
so we can't use that to filter out super resolution videos. Instead,
we'll need to check every format we get while iterating and skip it
if it ends with `-sr`.

See: https://alexwlchan.net/til/2025/ignore-ai-scaled-videos/?ref=mastodon
 2026-01-30 11:51:52 -08:00
Alex Osborne 1ff35f6cb8 Merge pull request #707 from Querela/feature/job-deletion 
Add job deletion to API and UI
 2026-01-06 22:37:29 +09:00
Erik Körner c1bedebef1 Add job deletion to API and UI 2026-01-05 14:19:17 +01:00
Alex Osborne 5304ec0b00 Merge pull request #706 from internetarchive/dependabot/maven/minor-and-patch-eb130e3613 
Bump the minor-and-patch group with 12 updates
 2026-01-05 10:28:00 +09:00
dependabot[bot] c54fcc3aa7 Bump the minor-and-patch group with 12 updates 
Bumps the minor-and-patch group with 12 updates:

| Package | From | To |
| --- | --- | --- |
| [org.apache.groovy:groovy-bom](https://github.com/apache/groovy) | `5.0.2` | `5.0.3` |
| [org.eclipse.jetty:jetty-bom](https://github.com/jetty/jetty.project) | `12.0.30` | `12.0.31` |
| [org.eclipse.jetty.ee10:jetty-ee10-bom](https://github.com/jetty/jetty.project) | `12.0.30` | `12.0.31` |
| [org.apache.maven.plugins:maven-release-plugin](https://github.com/apache/maven-release) | `3.2.0` | `3.3.1` |
| [org.springframework:spring-core](https://github.com/spring-projects/spring-framework) | `7.0.1` | `7.0.2` |
| [org.springframework:spring-beans](https://github.com/spring-projects/spring-framework) | `7.0.1` | `7.0.2` |
| [org.springframework:spring-context](https://github.com/spring-projects/spring-framework) | `7.0.1` | `7.0.2` |
| [org.springframework:spring-expression](https://github.com/spring-projects/spring-framework) | `7.0.1` | `7.0.2` |
| [com.github.crawler-commons:crawler-commons](https://github.com/crawler-commons/crawler-commons) | `1.5` | `1.6` |
| [org.webjars:webjars-locator-lite](https://github.com/webjars/webjars-locator-lite) | `1.1.2` | `1.1.3` |
| [com.rabbitmq:amqp-client](https://github.com/rabbitmq/rabbitmq-java-client) | `5.27.1` | `5.28.0` |
| [at.yawk.lz4:lz4-java](https://github.com/yawkat/lz4-java) | `1.10.1` | `1.10.2` |


Updates `org.apache.groovy:groovy-bom` from 5.0.2 to 5.0.3
- [Commits](https://github.com/apache/groovy/commits)

Updates `org.eclipse.jetty:jetty-bom` from 12.0.30 to 12.0.31
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.30...jetty-12.0.31)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.30 to 12.0.31
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.30...jetty-12.0.31)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.30 to 12.0.31
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.30...jetty-12.0.31)

Updates `org.apache.maven.plugins:maven-release-plugin` from 3.2.0 to 3.3.1
- [Release notes](https://github.com/apache/maven-release/releases)
- [Commits](https://github.com/apache/maven-release/compare/maven-release-3.2.0...maven-release-3.3.1)

Updates `org.springframework:spring-core` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.1...v7.0.2)

Updates `org.springframework:spring-beans` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.1...v7.0.2)

Updates `org.springframework:spring-context` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.1...v7.0.2)

Updates `org.springframework:spring-expression` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.1...v7.0.2)

Updates `org.springframework:spring-beans` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.1...v7.0.2)

Updates `org.springframework:spring-context` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.1...v7.0.2)

Updates `org.springframework:spring-expression` from 7.0.1 to 7.0.2
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v7.0.1...v7.0.2)

Updates `com.github.crawler-commons:crawler-commons` from 1.5 to 1.6
- [Release notes](https://github.com/crawler-commons/crawler-commons/releases)
- [Changelog](https://github.com/crawler-commons/crawler-commons/blob/master/CHANGES.txt)
- [Commits](https://github.com/crawler-commons/crawler-commons/compare/crawler-commons-1.5...crawler-commons-1.6)

Updates `org.webjars:webjars-locator-lite` from 1.1.2 to 1.1.3
- [Release notes](https://github.com/webjars/webjars-locator-lite/releases)
- [Commits](https://github.com/webjars/webjars-locator-lite/compare/webjars-locator-lite-1.1.2...webjars-locator-lite-1.1.3)

Updates `com.rabbitmq:amqp-client` from 5.27.1 to 5.28.0
- [Release notes](https://github.com/rabbitmq/rabbitmq-java-client/releases)
- [Commits](https://github.com/rabbitmq/rabbitmq-java-client/compare/v5.27.1...v5.28.0)

Updates `at.yawk.lz4:lz4-java` from 1.10.1 to 1.10.2
- [Release notes](https://github.com/yawkat/lz4-java/releases)
- [Changelog](https://github.com/yawkat/lz4-java/blob/main/CHANGES.md)
- [Commits](https://github.com/yawkat/lz4-java/compare/v1.10.1...v1.10.2)

---
updated-dependencies:
- dependency-name: org.apache.groovy:groovy-bom
  dependency-version: 5.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty:jetty-bom
  dependency-version: 12.0.31
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.31
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.31
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-release-plugin
  dependency-version: 3.3.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-core
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-beans
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-context
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-expression
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-beans
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-context
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.springframework:spring-expression
  dependency-version: 7.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.github.crawler-commons:crawler-commons
  dependency-version: '1.6'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.webjars:webjars-locator-lite
  dependency-version: 1.1.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.rabbitmq:amqp-client
  dependency-version: 5.28.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: at.yawk.lz4:lz4-java
  dependency-version: 1.10.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-01-01 23:24:56 +00:00
Alex Osborne 807ff372a9 Merge pull request #704 from Querela/feature/styling-max-height-joblist-for-exit 
WebUI: Engine page: Limit height of "Exit Java" joblist, set auto-scroll behaviour on overflow
 2025-12-18 21:40:42 +09:00
Erik Körner 107a249a5d Limit height of Exit Java joblist, set auto-scroll behaviour on overflow 2025-12-18 12:26:08 +01:00
Alex Osborne 337aada218 Merge pull request #703 from Querela/feature/json-rest-api-support 
Feature/json rest api support
 2025-12-18 12:10:28 +09:00
Erik Körner 78b76345d0 Update other endpoints to use new JsonMarshaller, fix some edge cases 2025-12-17 23:25:37 +01:00
Erik Körner cf87b045b9 Add /beans endpoint JSON handling 2025-12-17 22:51:37 +01:00
Erik Körner 76d3847f35 Update documentation 2025-12-17 18:31:16 +01:00
Erik Körner 18b569736c Add org.restlet.ext.json dependency, add application/json REST API responses 2025-12-17 18:02:36 +01:00
Alex Osborne 3a1974528a [maven-release-plugin] prepare for next development iteration 2025-12-11 13:54:13 +09:00
Alex Osborne ec62a9b0ad [maven-release-plugin] prepare release 3.13.0 3.13.0 2025-12-11 13:54:07 +09:00
Alex Osborne 4091e6d6f7 Move jackson-bom import to the parent pom 
We're now using jackson in several submodules. This ensures we're using consistent versions everywhere.
 2025-12-11 13:46:31 +09:00
Alex Osborne 34f390af42 Add ExtractorJson to bean reference #701 2025-12-11 13:40:08 +09:00
Alex Osborne 6d0c400294 Update CHANGELOG.md for #700 and #701 2025-12-11 13:38:50 +09:00
Alex Osborne fbf0391d25 Merge pull request #701 from bnfleb/extractor 
Add ExtractorJson from BL
 2025-12-11 13:27:46 +09:00
Alex Osborne 4ea67a3820 Merge pull request #700 from bnfleb/bnf_2025 
Include sizeOnDisk on Job status
 2025-12-11 13:05:56 +09:00
Leslie Bellony 4d6735f7c7 Add ExtractorJson from BL (#698) 2025-12-09 17:33:18 +01:00
Leslie Bellony 8a7db14a51 Add sizeOnDisk on job status (#699) 2025-12-09 15:59:03 +01:00
Alex Osborne fae274a23b Update CHANGELOG.md 2025-12-09 17:51:24 +09:00
Alex Osborne b996bba36a Switch from org.lz4:lz4-java 1.8.1 to at.yawk.lz4:lz4-java 1.10.1 for CVE-2025-66566 2025-12-09 17:42:53 +09:00
Alex Osborne fd489d74f4 Update CHANGELOG for 3.13.0 release 2025-12-09 17:33:56 +09:00
Alex Osborne b3a37801dd Override lz4-java dependency to 1.8.1 for CVE-2025-12183 2025-12-09 17:33:56 +09:00
Alex Osborne 65233a5f56 Merge pull request #696 from internetarchive/dependabot/maven/spring.version-7.0.1 
Bump spring.version from 6.2.12 to 7.0.1
 2025-12-09 17:21:50 +09:00
Alex Osborne ed176b7d54 BeanDocProcessor: Unwrap IntelliJ ProcessingEnvironment 
It seems IntelliJ injects some kind of wrapper that causes Trees.instance() to fail. The error message suggests doing this as a workaround.
 2025-12-09 17:06:33 +09:00
Alex Osborne 4bcd5b3a68 Merge pull request #697 from bnfleb/bnf_2025 
Issue #688 ignore extracted strings longer than 2048 characters
 2025-12-09 16:58:20 +09:00
Leslie Bellony 771452ee83 when skipping too long value, move matcher pointer to the end of value 2025-12-08 17:01:03 +01:00
Leslie Bellony 305204af8b Issue #689 unit tests for image attributes 2025-12-04 17:10:43 +01:00
Leslie Bellony fb791e2b0c Issue #688 ignore extracted strings longer than 2048 characters 2025-12-04 14:54:06 +01:00
Alex Osborne df5920656c Merge pull request #695 from internetarchive/dependabot/maven/minor-and-patch-242d488f62 
Bump the minor-and-patch group with 14 updates
 2025-12-02 17:40:54 +09:00
Alex Osborne 105608c209 Merge pull request #694 from bnfleb/bnf_2025 
Current implementation of ClientFTP not working with FTP servers partial responses
 2025-12-02 11:41:34 +09:00
dependabot[bot] 54e0b50edc Bump spring.version from 6.2.12 to 7.0.1 
Bumps `spring.version` from 6.2.12 to 7.0.1.

Updates `org.springframework:spring-core` from 6.2.12 to 7.0.1
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v6.2.12...v7.0.1)

Updates `org.springframework:spring-beans` from 6.2.12 to 7.0.1
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v6.2.12...v7.0.1)

Updates `org.springframework:spring-context` from 6.2.12 to 7.0.1
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v6.2.12...v7.0.1)

Updates `org.springframework:spring-expression` from 6.2.12 to 7.0.1
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v6.2.12...v7.0.1)

---
updated-dependencies:
- dependency-name: org.springframework:spring-core
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
- dependency-name: org.springframework:spring-beans
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
- dependency-name: org.springframework:spring-context
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
- dependency-name: org.springframework:spring-expression
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-02 02:22:33 +00:00
dependabot[bot] cc04602074 Bump the minor-and-patch group with 14 updates 
Bumps the minor-and-patch group with 14 updates:

| Package | From | To |
| --- | --- | --- |
| [org.eclipse.jetty:jetty-bom](https://github.com/jetty/jetty.project) | `12.0.29` | `12.0.30` |
| [org.eclipse.jetty.ee10:jetty-ee10-bom](https://github.com/jetty/jetty.project) | `12.0.29` | `12.0.30` |
| [org.apache.maven.plugins:maven-source-plugin](https://github.com/apache/maven-source-plugin) | `3.3.1` | `3.4.0` |
| [org.codehaus.mojo:versions-maven-plugin](https://github.com/mojohaus/versions) | `2.19.1` | `2.20.1` |
| [org.apache.maven.plugins:maven-release-plugin](https://github.com/apache/maven-release) | `3.1.1` | `3.2.0` |
| [commons-io:commons-io](https://github.com/apache/commons-io) | `2.20.0` | `2.21.0` |
| [commons-codec:commons-codec](https://github.com/apache/commons-codec) | `1.19.0` | `1.20.0` |
| [commons-cli:commons-cli](https://github.com/apache/commons-cli) | `1.10.0` | `1.11.0` |
| [org.netpreserve.commons:webarchive-commons](https://github.com/iipc/webarchive-commons) | `3.0.1` | `3.0.2` |
| [com.github.mwiede:jsch](https://github.com/mwiede/jsch) | `2.27.5` | `2.27.7` |
| [org.apache.maven.plugins:maven-resources-plugin](https://github.com/apache/maven-resources-plugin) | `3.3.1` | `3.4.0` |
| [org.apache.maven.plugins:maven-assembly-plugin](https://github.com/apache/maven-assembly-plugin) | `3.7.1` | `3.8.0` |
| [com.rabbitmq:amqp-client](https://github.com/rabbitmq/rabbitmq-java-client) | `5.27.0` | `5.27.1` |
| org.apache.kafka:kafka-clients | `4.1.0` | `4.1.1` |


Updates `org.eclipse.jetty:jetty-bom` from 12.0.29 to 12.0.30
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.29...jetty-12.0.30)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.29 to 12.0.30
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.29...jetty-12.0.30)

Updates `org.eclipse.jetty.ee10:jetty-ee10-bom` from 12.0.29 to 12.0.30
- [Release notes](https://github.com/jetty/jetty.project/releases)
- [Commits](https://github.com/jetty/jetty.project/compare/jetty-12.0.29...jetty-12.0.30)

Updates `org.apache.maven.plugins:maven-source-plugin` from 3.3.1 to 3.4.0
- [Release notes](https://github.com/apache/maven-source-plugin/releases)
- [Commits](https://github.com/apache/maven-source-plugin/compare/maven-source-plugin-3.3.1...maven-source-plugin-3.4.0)

Updates `org.codehaus.mojo:versions-maven-plugin` from 2.19.1 to 2.20.1
- [Release notes](https://github.com/mojohaus/versions/releases)
- [Changelog](https://github.com/mojohaus/versions/blob/master/ReleaseNotes.md)
- [Commits](https://github.com/mojohaus/versions/compare/2.19.1...2.20.1)

Updates `org.apache.maven.plugins:maven-release-plugin` from 3.1.1 to 3.2.0
- [Release notes](https://github.com/apache/maven-release/releases)
- [Commits](https://github.com/apache/maven-release/compare/maven-release-3.1.1...maven-release-3.2.0)

Updates `commons-io:commons-io` from 2.20.0 to 2.21.0
- [Changelog](https://github.com/apache/commons-io/blob/master/RELEASE-NOTES.txt)
- [Commits](https://github.com/apache/commons-io/compare/rel/commons-io-2.20.0...rel/commons-io-2.21.0)

Updates `commons-codec:commons-codec` from 1.19.0 to 1.20.0
- [Changelog](https://github.com/apache/commons-codec/blob/master/RELEASE-NOTES.txt)
- [Commits](https://github.com/apache/commons-codec/compare/rel/commons-codec-1.19.0...rel/commons-codec-1.20.0)

Updates `commons-cli:commons-cli` from 1.10.0 to 1.11.0
- [Changelog](https://github.com/apache/commons-cli/blob/master/RELEASE-NOTES.txt)
- [Commits](https://github.com/apache/commons-cli/compare/rel/commons-cli-1.10.0...rel/commons-cli-1.11.0)

Updates `org.netpreserve.commons:webarchive-commons` from 3.0.1 to 3.0.2
- [Release notes](https://github.com/iipc/webarchive-commons/releases)
- [Changelog](https://github.com/iipc/webarchive-commons/blob/master/CHANGES.md)
- [Commits](https://github.com/iipc/webarchive-commons/compare/webarchive-commons-3.0.1...webarchive-commons-3.0.2)

Updates `com.github.mwiede:jsch` from 2.27.5 to 2.27.7
- [Release notes](https://github.com/mwiede/jsch/releases)
- [Changelog](https://github.com/mwiede/jsch/blob/master/ChangeLog)
- [Commits](https://github.com/mwiede/jsch/compare/jsch-2.27.5...jsch-2.27.7)

Updates `org.apache.maven.plugins:maven-resources-plugin` from 3.3.1 to 3.4.0
- [Release notes](https://github.com/apache/maven-resources-plugin/releases)
- [Commits](https://github.com/apache/maven-resources-plugin/compare/maven-resources-plugin-3.3.1...v3.4.0)

Updates `org.apache.maven.plugins:maven-assembly-plugin` from 3.7.1 to 3.8.0
- [Release notes](https://github.com/apache/maven-assembly-plugin/releases)
- [Commits](https://github.com/apache/maven-assembly-plugin/compare/maven-assembly-plugin-3.7.1...v3.8.0)

Updates `com.rabbitmq:amqp-client` from 5.27.0 to 5.27.1
- [Release notes](https://github.com/rabbitmq/rabbitmq-java-client/releases)
- [Commits](https://github.com/rabbitmq/rabbitmq-java-client/compare/v5.27.0...v5.27.1)

Updates `org.apache.kafka:kafka-clients` from 4.1.0 to 4.1.1

---
updated-dependencies:
- dependency-name: org.eclipse.jetty:jetty-bom
  dependency-version: 12.0.30
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.30
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.eclipse.jetty.ee10:jetty-ee10-bom
  dependency-version: 12.0.30
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-source-plugin
  dependency-version: 3.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.codehaus.mojo:versions-maven-plugin
  dependency-version: 2.20.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-release-plugin
  dependency-version: 3.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: commons-io:commons-io
  dependency-version: 2.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: commons-codec:commons-codec
  dependency-version: 1.20.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: commons-cli:commons-cli
  dependency-version: 1.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.netpreserve.commons:webarchive-commons
  dependency-version: 3.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: com.github.mwiede:jsch
  dependency-version: 2.27.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-resources-plugin
  dependency-version: 3.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: org.apache.maven.plugins:maven-assembly-plugin
  dependency-version: 3.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: com.rabbitmq:amqp-client
  dependency-version: 5.27.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: org.apache.kafka:kafka-clients
  dependency-version: 4.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-02 02:22:11 +00:00
Leslie Bellony c65bc1bcc0 Merge branch 'internetarchive:master' into bnf_2025 2025-12-01 10:21:11 +01:00
Alex Osborne da0fb468b2 Merge pull request #693 from manikyarathore/master_m2 
Fix: Add null checks and logging when deleting URIs from BdbMultipleWorkQueues (#690)
 2025-11-26 09:02:57 +09:00
Alex Osborne c46f3dd1e7 Merge pull request #691 from manikyarathore/master_manikya 
Fix: Normalize leading dot domains in cookie.txt loader so cookies are no longer ignored (#658)
 2025-11-26 08:58:19 +09:00