Commit Graph

Select branches

Hide Pull Requests

ARI-5439

adam/add-crawl-log-logging-to-extractoryoutubedl

adam/add_configurable_extractor_js_script_attribute_rules

adam/ait-add-crawl-log-logging-to-extractoryoutubedl

adam/fix_amqp_consumer_leak

adam/fix_configurable_js_extractor_strict_overlay

adam/fix_crawl_end_time_checkpoint_resumption

adam/knowledgable_extractorjs_extends_configurable_extractorjs

adam/restore-has-prerequisite-behavior

adam/strip_invalid_xml_chars_from_rest_api_output

adds-bdbmodule-max-file-size-option

adds-subqueue-support-forced-queue-assignment

adds-trough-dedup-performance-rework

adds-trough-local-dedup-write-cache

ait-qa

ait-ydl-updates

collectd-socket-stats-writer

fix-invoker-command-fallback

fix-yt-dlp-typo-too

fixes-WARCWriterChainProcessor-checkpoint-stats

fixes-extractor-multiple-regex-matcher-recycle

fixes-leaky-file-handles

fixes-prerequisite-status

fixes-trough-client-synchronization

fixes-trough-crawl-log-feed-synchronization

less_youtubedl

master

master-ait-contrib

master-it

meta-content-name

noplaylist-ydl

paged-log-viewer-search

restlet_npe

revert-367-fixes-trough-crawl-log-feed-synchronization

ydl-no-cache

yt-dl-format

yt-dl-format-medium-ish

yt-dlp

yt-dlp-20230706

yt-dlp-4-h3

yt-dlp-for-h3

#1

#1/merge

#10

#10/merge

#100

#101

#102

#103

#104

#105

#106

#107

#108

#109

#11

#11/merge

#110

#111

#112

#113

#114

#115

#116

#117

#118

#119

#12

#12/merge

#120

#121

#122

#123

#124

#125

#126

#127

#127/merge

#128

#129

#13

#13/merge

#130

#131

#132

#133

#134

#134/merge

#135

#136

#137

#138

#139

#14

#14/merge

#140

#140/merge

#141

#142

#143

#144

#145

#146

#147

#148

#149

#15

#15/merge

#150

#151

#152

#153

#153/merge

#154

#159

#16

#16/merge

#161

#163

#164

#165

#166

#168

#169

#17

#17/merge

#171

#174

#174/merge

#178

#179

#18

#180

#183

#183/merge

#184

#184/merge

#185

#186

#187

#188

#188/merge

#189

#19

#19/merge

#190

#192

#194

#196

#197

#2

#2/merge

#20

#20/merge

#205

#207

#209

#21

#21/merge

#210

#213

#215

#216

#217

#218

#219

#22

#22/merge

#220

#222

#223

#224

#226

#227

#23

#230

#231

#235

#238

#24

#240

#241

#242

#243

#244

#246

#248

#249

#25

#251

#253

#255

#256

#257

#26

#260

#262

#269

#27

#270

#271

#272

#273

#276

#28

#280

#281

#283

#285

#286

#287

#288

#289

#29

#290

#292

#292/merge

#294

#295

#296

#298

#3

#3/merge

#30

#300

#301

#302

#303

#304

#306

#308

#31

#310

#312

#313

#314

#315

#317

#32

#320

#323

#325

#326

#327

#328

#329

#33

#330

#331

#333

#334

#335

#336

#34

#341

#342

#345

#346

#347

#348

#349

#35

#352

#354

#355

#356

#357

#359

#36

#362

#364

#365

#366

#367

#368

#369

#37

#370

#373

#374

#375

#377

#379

#38

#381

#382

#383

#384

#385

#386

#388

#388/merge

#389

#39

#390

#391

#393

#394

#395

#397

#398

#399

#4

#4/merge

#40

#400

#401

#402

#403

#404

#405

#406

#407

#408

#409

#41

#410

#411

#412

#414

#415

#416

#418

#42

#421

#423

#424

#43

#431

#433

#434

#435

#44

#441

#442

#444

#445

#448

#45

#450

#452

#454

#456

#457

#459

#46

#460

#461

#465

#468

#47

#470

#475

#476

#478

#48

#480

#483

#484

#486

#488

#489

#49

#490

#491

#492

#493

#5

#5/merge

#50

#503

#51

#52

#53

#534

#535

#536

#537

#538

#539

#54

#542

#543

#544

#549

#55

#55/merge

#550

#551

#552

#554

#555

#56

#565

#566

#567

#57

#572

#573

#574

#575

#576

#577

#578

#579

#58

#580

#581

#582

#583

#586

#589

#59

#590

#591

#592

#593

#594

#595

#597

#598

#599

#6

#6/merge

#60

#600

#601

#602

#603

#605

#606

#607

#608

#609

#61

#610

#611

#612

#614

#615

#616

#617

#618

#619

#62

#621

#622

#623

#624

#625

#626

#627

#628

#629

#63

#630

#631

#632

#638

#639

#64

#640

#642

#643

#644

#645

#646

#647

#648

#649

#65

#650

#651

#652

#653

#654

#655

#656

#657

#659

#66

#660

#661

#663

#664

#665

#667

#668

#669

#67

#670

#671

#672

#673

#674

#675

#676

#677

#678

#679

#68

#680

#681

#682

#683

#684

#685

#686

#69

#691

#692

#693

#694

#695

#696

#697

#7

#7/merge

#70

#700

#701

#703

#704

#705

#706

#707

#708

#709

#71

#710

#712

#713

#719

#72

#721

#722

#723

#725

#728

#73

#730

#731

#732

#733

#734

#735

#736

#737

#738

#739

#74

#740

#741

#742

#744

#745

#746

#75

#750

#751

#755

#756

#757

#758

#759

#76

#760

#761

#761/merge

#762

#763

#764

#765

#766

#767

#77

#770

#771

#772

#774

#775

#776

#777

#778

#779

#78

#780

#781

#781/merge

#782

#783

#784

#785

#786

#787

#787/merge

#79

#8

#8/merge

#80

#81

#82

#83

#84

#85

#85/merge

#86

#87

#88

#89

#9

#9/merge

#90

#91

#92

#93

#94

#95

#96

#97

#98

#98/merge

#99

3.0.0

3.1.1

3.10.0

3.10.1

3.10.2

3.11.0

3.12.0

3.13.0

3.14.0

3.14.1

3.15.0

3.16.0

3.17.0

3.17.1

3.18.0

3.2.0

3.4.0-20190205

3.4.0-20190207

3.4.0-20190418

3.4.0-20200304

3.4.0-20200518

3.4.0-20210527

3.4.0-20210617

3.4.0-20210618

3.4.0-20210621

3.4.0-20210803

3.4.0-20210923

3.4.0-20220727

3.4.0-20240909

3.5.0

3.6.0

3.7.0

3.8.0

3.9.0

Mono Color

• 01378ddbbf [maven-release-plugin] prepare for next development iteration Alex Osborne 2025-06-12 22:01:16 +09:00
• ecc46a38c3 [maven-release-plugin] prepare release 3.10.0 3.10.0 Alex Osborne 2025-06-12 22:01:10 +09:00
• 5431a55820 Migrate from OSSRH to the Maven Central Portal Alex Osborne 2025-06-12 21:55:52 +09:00
• 45342f85ca Update CHANGELOG.md for 3.10.0 Alex Osborne 2025-06-12 21:48:22 +09:00
• eacba1b276 Bump groovy from 4.0.26 to 4.0.27 Alex Osborne 2025-06-12 21:46:05 +09:00
• 6c172fb60f Merge pull request #660 from internetarchive/dependabot/maven/contrib/org.apache.kafka-kafka-clients-3.9.1 Alex Osborne 2025-06-12 21:41:32 +09:00
• fcbc2d6931 Bump org.apache.kafka:kafka-clients from 3.9.0 to 3.9.1 in /contrib dependabot[bot] 2025-06-12 12:38:13 +00:00
• 9b5900b528 FetchHTTP2: Avoid silently decoding gzip Alex Osborne 2025-06-12 21:35:16 +09:00
• 24da104e38 Add link to #657 in CHANGELOG.md Alex Osborne 2025-06-12 10:57:53 +09:00
• 789bd265ca Add CHANGELOG.md entry for #659 Alex Osborne 2025-06-12 10:56:41 +09:00
• 5601453a45 Merge pull request #659 from internetarchive/adam/handle-bdb-shutdown-interrupts Alex Osborne 2025-06-12 10:49:37 +09:00
• 1ab9b4a206 fix: avoid bdb environment corruption when terminate is called during shutdown Adam Miller 2025-06-11 17:40:25 -07:00
• 0494d63ea2 fix: Avoid a NullPointerException when terminate is called again during shutdown Adam Miller 2025-06-11 17:39:29 -07:00
• 61af1d83c4 MitmProxy: Handle URLs generated by browsers but disallowed by java.net.URI Alex Osborne 2025-06-11 20:38:54 +09:00
• 8a15156fa1 BrowserProcessor: Catch WebDriver timeouts and exceptions Alex Osborne 2025-06-11 17:17:02 +09:00
• e0bae9a1b5 FetchHTTP2: Fallback to async DNS resolver Alex Osborne 2025-06-11 16:55:11 +09:00
• 9a1bc7f287 Merge pull request #657 from internetarchive/fetchhttp2-http-proxy Alex Osborne 2025-06-11 11:54:30 +09:00
• c3f9afe64b BrowserProcessor: use FetchHTTP2's configured proxy as an upstream proxy Alex Osborne 2025-06-10 09:01:02 +09:00
• cc50af72be FetchHTTP2: Add HTTP proxy support Alex Osborne 2025-06-09 09:25:50 +09:00
• 020ea34cb6 Bean reference: Add record builders under WARCWriterChainProcessor Alex Osborne 2025-06-09 15:42:13 +09:00
• e68e4975c9 Rename to BrowserProcessor and -Behavior Alex Osborne 2025-06-09 15:35:25 +09:00
• cf447f3bd9 Add Browser processor and behaviors to bean reference Alex Osborne 2025-06-09 15:19:09 +09:00
• 8df957d77a Add CHANGELOG.md entry for Browser processor Alex Osborne 2025-06-09 14:42:31 +09:00
• 1aa6b82d32 Remove stray footnote reference Alex Osborne 2025-06-09 11:49:07 +09:00
• faf251359d WbmPersistLoadProcessorTest: Remove dependency on easymock and re-enable test Alex Osborne 2025-06-09 10:17:08 +09:00
• 6782fc065d Update dependencies Alex Osborne 2025-06-09 09:50:01 +09:00
• 52bbd8091b Merge pull request #653 from internetarchive/bidi Alex Osborne 2025-06-09 09:36:06 +09:00
• 7e15d2fda1 Merge pull request #656 from internetarchive/robotstxt-wildcard Alex Osborne 2025-06-09 09:34:16 +09:00
• c7b7ee1842 Support * and $ wildcards in robots.txt Alex Osborne 2025-06-06 18:55:21 +09:00
• 1161d87217 Browser: Add processor report Alex Osborne 2025-06-05 10:48:15 +09:00
• 2210cb7262 Browser: Handle navigation abort from downloads starting Alex Osborne 2025-06-04 15:59:20 +09:00
• 5131ea4a80 Browser: Disable downloads in Firefox and Chrome Alex Osborne 2025-06-04 15:38:39 +09:00
• 722552feef Add Browser processor using WebDriver BiDi Alex Osborne 2025-05-05 15:07:43 +09:00
• 497d795005 Merge pull request #654 from internetarchive/web-auth-basic Alex Osborne 2025-06-04 11:03:57 +09:00
• 721c78ec8b Merge pull request #655 from Valadaress/master Alex Osborne 2025-05-27 10:14:43 +09:00
• c73cb60dba Adjust dependencies apt-get for Dockerfile and Dockerfile.contrib Valadaress 2025-05-26 20:07:21 -03:00
• 4dc990e1ff Update Docker compose to newest version Valadaress 2025-05-26 19:49:08 -03:00
• eaf72b5ce2 Add --web-auth basic command-line option Alex Osborne 2025-05-25 00:29:14 +09:00
• 0f0db370ee UI: Disable Jetty graceful shutdown for faster restarts Alex Osborne 2025-05-24 22:46:42 +09:00
• 679fde42c6 SelfTestBase: Use a dynamic port (port 0) for Heritrix web port Alex Osborne 2025-05-22 15:38:45 +09:00
• 4695f8a153 Merge pull request #652 from internetarchive/webarchive-commons-2.0.1 Alex Osborne 2025-05-22 15:19:49 +09:00
• 567e27181d Upgrade webarchive-commons from 1.3.0 to 2.0.1 (removes httpclient 3) Alex Osborne 2025-05-21 17:39:21 +09:00
• 58e1ac529b Merge pull request #651 from internetarchive/codemirror6 Alex Osborne 2025-05-20 19:05:36 +09:00
• 33446c7a2b Upgrade to CodeMirror 6 Alex Osborne 2025-05-16 18:34:30 +09:00
• 51347f7698 docs: Use javalang17 fork Alex Osborne 2025-05-13 14:23:08 +09:00
• 7db91b53c6 docs: HTTP/2: correct code quotes instead of italics Alex Osborne 2025-05-13 14:19:35 +09:00
• e1ac68f533 docs: Correct bean-example path for FetchHTTP2 Alex Osborne 2025-05-13 14:16:36 +09:00
• fee534c3db docs: Get version from pom.xml Alex Osborne 2025-05-13 14:15:20 +09:00
• 81ac46be45 docs: Catch parse errors and continue generating Alex Osborne 2025-05-13 14:09:09 +09:00
• 1827d93fe6 [maven-release-plugin] prepare for next development iteration Alex Osborne 2025-05-13 12:25:24 +09:00
• dd40970f15 [maven-release-plugin] prepare release 3.9.0 3.9.0 Alex Osborne 2025-05-13 12:25:16 +09:00
• d60161b6be Merge pull request #650 from internetarchive/deps-3.9 Alex Osborne 2025-05-09 22:05:42 +09:00
• e09106cc15 Document HTTP/2 and HTTP/3 support Alex Osborne 2025-05-09 20:40:51 +09:00
• 4a2cdfaa81 FetchHTTP2: Print the jetty-quiche-native warning earlier at build time Alex Osborne 2025-05-09 20:40:33 +09:00
• 8692bdf655 Add WARC-Protocol to request records too when using HTTP/2 or HTTP/3 Alex Osborne 2025-05-09 19:54:22 +09:00
• 1e502fba10 Limit permissions on github actions Alex Osborne 2025-05-09 18:52:32 +09:00
• 8e177f778b Update CHANGELOG for 3.9.0 Alex Osborne 2025-05-09 18:52:21 +09:00
• f5732e1a98 Bump jsch from 0.2.26 to 2.27.0 Alex Osborne 2025-05-09 18:19:21 +09:00
• eb1205f6d5 Upgrade all tests to junit 5 Alex Osborne 2025-05-09 00:07:18 +09:00
• bd8d6fb2c5 Bump commons-io from 2.18.0 to 2.19.0 Alex Osborne 2025-05-08 17:41:39 +09:00
• d25a0b96ca Bump spring from 6.2.5 to 6.2.6 Alex Osborne 2025-05-08 17:41:14 +09:00
• 083a60b04c Bump gson from 2.12.1 to 2.13.1 Alex Osborne 2025-05-08 17:40:10 +09:00
• d5e91eb1f3 Bump pdfbox from 3.0.4 to 3.0.5 Alex Osborne 2025-05-08 17:36:12 +09:00
• 227d51f467 Bump jetty from 12.0.16 to 12.0.17 Alex Osborne 2025-05-08 17:31:49 +09:00
• 4465c1b9f6 Bump restlet from 2.6.0-m2 to 2.6.0-rc1 Alex Osborne 2025-05-08 17:31:30 +09:00
• 1e3b0f6c39 Add /target, __pycache__, adhoc.keystore, heritrix_dmesg.log to .gitignore Alex Osborne 2025-05-08 17:27:01 +09:00
• 9807b93db3 Merge pull request #649 from internetarchive/http2 Alex Osborne 2025-05-08 17:25:08 +09:00
• fcfef68aff FetchHTTP2: A new fetch module for HTTP/2 and HTTP/3 Alex Osborne 2025-04-28 21:16:08 +09:00
• bc9b21e2ee Merge pull request #648 from internetarchive/jetty-12 Alex Osborne 2025-04-29 12:47:25 +09:00
• db16b60960 Fix 'anypath' URL generation and avoid unnecessary usages of it Alex Osborne 2025-04-28 17:09:33 +09:00
• 0f5358ffef Exclude jetty-quiche-native from restlet Alex Osborne 2025-04-28 16:29:22 +09:00
• bc259728a0 Update to Jetty 12.0.16 and Restlet 2.6.0-M2 Alex Osborne 2024-11-29 10:15:40 +09:00
• d73e2f6b87 Update CHANGELOG.md for #647 Alex Osborne 2025-04-24 11:02:17 +09:00
• 91cd4b815c Merge pull request #647 from adam-miller/add-histotable-serializer-for-HighestUriPrecedenceProvider Alex Osborne 2025-04-24 10:58:03 +09:00
• 02c4c8ee09 feat: add unit test for HighestUriPrecedenceProvider Kryo serialization Adam Miller 2025-04-23 16:25:58 -07:00
• fdada4c5ec chore: formatting Adam Miller 2025-04-23 15:20:40 -07:00
• cb98bfc522 chore: fix line endings Adam Miller 2025-04-22 17:33:32 -07:00
• e7fe566919 fix: autoregister Histotable and HighestUriPrecedenceProvider Adam Miller 2025-04-22 17:29:52 -07:00
• 75377c5a3b fix: FieldSerializer wasn't passing unit tests. Switch to JavaSerializer Adam Miller 2025-04-21 18:16:47 -07:00
• c7784f18e9 feat: Add serializer for Histotable used by HighestUriPrecedenceProvider Adam Miller 2025-04-21 17:20:38 -07:00
• e9b60404a6 [maven-release-plugin] prepare for next development iteration Alex Osborne 2025-04-01 17:43:31 +09:00
• 2323e2251e [maven-release-plugin] prepare release 3.8.0 3.8.0 Alex Osborne 2025-04-01 17:43:25 +09:00
• edb3f3aa00 Merge pull request #645 from internetarchive/deps-3.8 Alex Osborne 2025-04-01 17:41:02 +09:00
• fb9b286d95 Update CHANGELOG for 3.8.0 Alex Osborne 2025-04-01 17:38:06 +09:00
• 575a2653b9 Fix warning: assert statement with side effects Alex Osborne 2025-04-01 16:27:13 +09:00
• b8dbe8bb2d Remove unused utility class LogUtils Alex Osborne 2025-04-01 16:25:44 +09:00
• ed72112d9c Fix missing try-with-resources warning Alex Osborne 2025-04-01 16:19:58 +09:00
• 1a605cf9a6 Remove unused utility class JavaLiterals Alex Osborne 2025-04-01 16:11:10 +09:00
• 13052a1465 Fix more broken references in javadoc Alex Osborne 2025-04-01 16:06:10 +09:00
• f956b2ec1f Fix broken references in javadoc Alex Osborne 2025-04-01 15:59:38 +09:00
• 7f76f5abd1 FetchHTTPTest: Disable testHostHeaderDefaultPort due to random failures Alex Osborne 2025-04-01 15:37:21 +09:00
• b17e3fcab3 Bump beanshell from 2.0b5 to 2.0b6 Alex Osborne 2025-04-01 15:27:22 +09:00
• 7ff1078d21 Fix warning about implicit narrowing conversion for contentSinceCheck Alex Osborne 2025-03-31 15:31:04 +09:00
• f17000e14f Fix warning: Building a command line with string concatenation Alex Osborne 2025-03-31 15:17:28 +09:00
• 60f512b4b0 Fix compiler warning: deprecated item is not annotated with @Deprecated Alex Osborne 2025-03-31 15:12:21 +09:00
• f6b083ecfb Remove usages of deprecated Integer and Long constructors Alex Osborne 2025-03-31 15:04:00 +09:00
• 1288e78c54 Bump spring from 6.1.16 to 6.2.5 Alex Osborne 2025-03-31 14:58:13 +09:00
• b9bfd36d28 Bump dnsjava from 3.6.2 to 3.6.3 Alex Osborne 2025-03-31 14:53:01 +09:00
• 4688d8cda2 Bump commons-codec from 1.17.2 to 1.18.0 Alex Osborne 2025-03-31 14:52:42 +09:00
• 5ecfa7248c Bump jsch from 0.2.22 to 0.2.24 Alex Osborne 2025-03-31 14:52:25 +09:00