Noah Levitt 0144df024e Real fix for HER-2016 FetchHTTP tries every url twice without credentials before sending credentials: Remember WWW-Authenticate headers, aka auth challenges, with CrawlURI, and after successful auth, with the CrawlServer. It's necessary to do this to know what Authorization header to send without getting another 401 first. ...

* CoreAttributeConstants.java
    new constant A_HTTP_AUTH_CHALLENGES key for CrawlURI.getData() for http auth challenges
* CrawlURI.java
    add A_HTTP_AUTH_CHALLENGES to keys to remember across URI processing; convenience getter and setter for auth challenges
* Credential.java, HtmlFormCredential.java, HttpAuthenticationCredential.java
    add httpAuthChallenges argument to populate()
* HttpAuthenticationCredential.java
    populate() - process httpAuthChallenges to set up auth state of http client to send the right auth header
* CrawlServer.java
    cache httpAuthChallenges
* FetchHTTP.java
    getAuthScheme() - cache auth challenges with CrawlURI
    configureHttp() - do not set authentication preemptive - this is handled by the configuration done in HttpAuthenticationCredential.populate()
    promoteCredentials() - cache auth challenges with the CrawlServer

2012-08-16 12:17:01 -07:00

..

.settings

copy 'springy' branch to heritrix3

2009-05-11 22:56:36 +00:00

src

Real fix for HER-2016 FetchHTTP tries every url twice without credentials before sending credentials: Remember WWW-Authenticate headers, aka auth challenges, with CrawlURI, and after successful auth, with the CrawlServer. It's necessary to do this to know what Authorization header to send without getting another 401 first.

2012-08-16 12:17:01 -07:00

.classpath

copy 'springy' branch to heritrix3

2009-05-11 22:56:36 +00:00

.project

copy 'springy' branch to heritrix3

2009-05-11 22:56:36 +00:00

pom.xml

**/pom.xml

2012-05-01 18:17:33 -07:00