Consolidate HTTP Digest auth header parsing

[dmiller-nmap]

From a mailing list post and the todo/nmap.txt file:

Digest auth support was added to the http library in r29520 (http:https://seclists.org/nmap-dev/2012/q3/517)
but I've just noticed that it doesn't use the existing http.parse_www_authenticate function, but does its own parsing of the header. I'm afraid that the parsing in http.generic_request and sasl.DigestMD5.parseChallenge doesn't look robust. For example, the match "digest.-realm" could match a "realm" that's part of a quoted-string, or even a following auth challenge.

This code should use http.parse_www_authenticate, as it handles things like multiple authentication challenges separated by commas, and is generally more careful. See the http-auth script for an example of using it. http.generic_request should also check for a 401 status code after its first request, and instead of throwing an error, it should just return the page if it gets a 200 or something.

David Fifield

Confirmed that we still use the basic string match and sasl.DigestMD5.parseChallenge. I haven't looked into the 401 status code handling.