mirror of
https://github.com/internetarchive/heritrix3.git
synced 2026-07-19 22:17:13 +00:00
269a763860
* CrawlServer, Robotstxt
move autoregister details to Robotstxt, RobotsDirectives
* RobotsDirectives
use ReferenceFieldSerializer so that repeated instances of RobotsDirectives in same context are replaced with backrefs
* RobotstxtTest
(from Kenji) unit test for instance multiplication