Tidy up markup and links.

This commit is contained in:
Andrew Jackson
2019-02-05 10:23:01 +00:00
parent e8aba660d8
commit b08db899dd
+14 -8
View File
@@ -9,8 +9,9 @@ Readme for Heritrix
6. License
1. Introduction
1) Introduction
----------------
Heritrix is the Internet Archive's open-source, extensible, web-scale,
archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or
misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word
@@ -19,8 +20,9 @@ preserve the digital artifacts of our culture for the benefit of future
researchers and generations, this name seemed apt.
2. Crawl Operators!
2) Crawl Operators!
--------------------
Heritrix is designed to respect the robots.txt
<http://www.robotstxt.org/wc/robots.html> exclusion directives and META robots
tags <http://www.robotstxt.org/wc/exclusion.html#meta>. Please consider the
@@ -30,25 +32,29 @@ User-Agent so sites that may be adversely affected by your crawl can contact
you or adapt their server behavior accordingly.
3. Getting Started
3) Getting Started
-------------------
See the User Manual at <https://github.com/internetarchive/heritrix3/wiki/Heritrix%203.0%20and%203.1%20User%20Guide>
See the User Manual, available from <https://github.com/internetarchive/heritrix3/wiki
4. Developer Documentation
4) Developer Documentation
---------------------------
See <http://crawler.archive.org/articles/developer_manual/index.html>.
For API documentation, see <https://webarchive.jira.com/wiki/display/Heritrix/Heritrix+3.x+API+Guide>
For API documentation, see <https://heritrix.readthedocs.io/en/latest/api.html>
and <http://builds.archive.org/javadoc/heritrix-3.2.0/>
5. Latest Releases
5) Latest Releases
-------------------
Information about releases can be found at <https://github.com/internetarchive/heritrix3/wiki#latest-releases>
6. License
6) License
-----------
Heritrix is free software; you can redistribute it and/or modify it
under the terms of the Apache License, Version 2.0: