mirror of
https://github.com/internetarchive/heritrix3.git
synced 2026-10-05 03:47:43 +00:00
The robots.txt that is used in this test actually lives in the root webapp. It has to be there to it shows at the root of our host.