Alex Osborne
71b3065172
addPresentableNestedNames() recursively walks the properties of beans calling getters. Likely unintentionally this includes calling getClass() and recursing into the reflection API. When running on JDK 11 the reflection API has some methods (e.g. in java.lang.module) that return instances of Optional. With the newer version of Spring's BeanWrapperImpl encounters an Optional it attempts to unwrap it and throws IllegalArgumentException if it is empty. Let's fix this in two ways. Firstly let's avoid walking the reflection API entirely as its irrelevant for the purposes of bean browsing by not inspecting the properties of instances of java.lang.Class. Secondly in case any Heritrix beans start using Optional in future lets also handle the empty case by also not inspecting, the same as we do for null. Fixes #376 Reported-By: Lauren Ko <lauren.ko@unt.edu>
Heritrix
Introduction
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word for heiress (woman who inherits). Since our crawler seeks to collect and preserve the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt.
Crawl Operators!
Heritrix is designed to respect the robots.txt exclusion directives† and META nofollow tags. Please consider the
load your crawl will place on seed sites and set politeness policies accordingly. Also, always identify your crawl with contact information in the User-Agent so sites that may be adversely affected by your crawl can contact you or adapt their server behavior accordingly.
† The newer wildcard extension to robots.txt is not yet supported.
Getting Started
Developer Documentation
- Developer Manual
- REST API documentation
- JavaDoc: engine, modules, commons, contrib
Latest Releases
Information about releases can be found here.
License
Heritrix is free software; you can redistribute it and/or modify it under the terms of the Apache License, Version 2.0
Some individual source code files are subject to or offered under other licenses. See the included LICENSE.txt file for more information.
Heritrix is distributed with the libraries it depends upon. The libraries can be found under the lib directory in the release distribution, and are used under the terms of their respective licenses, which are included alongside the libraries in the lib directory.