'hex-all' encoding was previously ignoring slashes.
This pull adds 'hex-noslashes' mode which carries forward the previous functionality, and replaces all existing references to 'hex-all' with 'hex-noslashes' It then adds a replacement 'hex-all' mode, which really encodes *ALL* characters.
This commit changes the internals of HTTP fingerprinting to store
a whole trove of data about the HTTP response using a hash. The
current API is backwards compatible and has been tested with a
number of modules that depend on HttpFingerprint being sent.
In addition, this change paves the way for advanced fingerprints
that take advantage of the HTTP body and other headers. This is
a requested addition documented across various module comments.
Finally, this commit completes the closed loop for OS identification
by connecting MSF to MDM to Recog and applying Recog databases for
HTTP Servers, HTTP Cookies, and HTTP Authentication headers to the
results of HTTP fingerprinting runs.
For example, with the appropriate version of MDM/Recog in place,
a http_version scan of Microsoft-IIS/7.0 server will update the
host.os_name field to 'Windows 2008'.
[SeeRM #823], where Stephen was asking for SSL compression for
Meterpreter -- this isn't that, but it's at least now possible for other
Metasploit functionality.
In that order. #1476 was an attempt to salvage the functionality, but
sinn3r found some more bugs. So, undoing that, and undoing #1444 as
well.
First, do no harm. It's obvious we cannot be making sweeping changes in
libraries like this without a minimum of testing available. #1478 starts
to address that, by the way.
FixRM #7752
This function should not remove the trailing slash, because you may
end up getting a different HTTP response. The new function also
allows multiple URIs as argument, and will just merge & normalize
them together. [SeeRM #7733]
This also fixes an issue with report_vuln() from exploited hosts not
setting the service correctly. This introduces a fail_reason method
to the exploit base class, which attempts to determine why an exploit
did not work (closed port, unreachable host, missing page, etc). There
is still quite a bit of work to do around this to finish it up.
The purpose of re-raising an error from a library method like this is to
tell the user in no uncertain terms what all actually went wrong with the
module. This fix will cause a somewhat more pleasant error message than
the default message. Here's the raise from URI:
```
[-] Auxiliary failed: URI::InvalidURIError bad URI(is not URI?): what%ever
[-] Call stack:
[-] /home/todb/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/uri/common.rb:156:in `split'
[-] /home/todb/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/uri/common.rb:174:in `parse'
[-] /home/todb/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/uri/common.rb:626:in `parse'
[-] /home/todb/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/uri/common.rb:724:in `URI'
[-] /home/todb/git/rapid7/metasploit-framework/lib/msf/core/exploit/http/client.rb:535:in `target_uri'
[-] /home/todb/.msf4/modules/auxiliary/test_uri.rb:20:in `run'
[*] Auxiliary module execution completed
```
And here's the new, Metasploit-specific one:
```
[-] Invalid URI: "what%ever"
[-] Auxiliary failed: Msf::OptionValidateError The following options failed to validate: URIPATH.
[*] Auxiliary module execution completed
```
The user can now tell easily what's wrong with the module configuration,
and doesn't have to parse through a stack trace that leads down into
the Ruby stdlib.
James Lee