This is Microsoft's NT/LanManager connection oriented authentication protocol - sometimes referred to as NT Challenge/Response (NTCR). This protocol is commonly encountered on networks that use Microsoft products for firewalls, intranet servers and web proxy servers. This is also the authentication protocol used for [SMB] connections, which is what [Windows] and [Samba] use for file and printer sharing. The basis of the scheme is a challenge-response protocol where the client sends the server a request that contains information about the NT domain and the client hostname. The server then replies with a random nonce value and the client then creates an authentication response using the user name, user's password and the nonce value. This is similar to the HTTP Digest authentication scheme in that the password is not sent across the network, but the client can demonstrate to the server that it knows the user's password.

Well - that's the theory. In fact there are a number of weaknesses in the NTLM protocol. In fact the password is used twice to generate an NT password hash and a LM password hash. The LM hash takes the user's password, capitalises it, and sets the length to 14 characters by padding with nul characters. This value is then used as a DES key to encrypt a magic value ("\x4b\x47\x53\x21\x40\x23\x24\x25") thus providing a 16-byte value to be sent to the server.

The NT password hash uses the [MD4] message-digest (RFC 1320) to hash the [Unicode] version of the user's password.

[Pat Thoyts] is currently working on supporting this scheme within Tcl as part of the [autoproxy] package. [MD4] hashing is now part of [tcllib], as is [DES].
----
[CAU] wrote - There's a simple way to get around NTLM using basic authentication with [Jos Decoster]'s http::geturl modification at [HTTPS] (Note: check for repeated lines in the code!) and a python-based authentication hack - ntlmaps[http://ntlmaps.sourceforge.net/].

A big advantage of ntlmaps for me, is it can make http requests behave like they were issued from a web-browser, which is an unexpected bonus when trying to tunnel through firewalls!

If anyone knows of a pure Tcl NTLM hack, I'd be very interested.

[PT] 29-Apr-2005: There is NTLM client code in the [SASL] module in [tcllib] now.
----
[MJ] 8-Feb-2007: It is possible to use the NTLM client code [tcllib] to use NTLM proxies with sockets (proxy CONNECT request) and HTTP GETs (proxy GET request). Implementing the proxy authentication for http is quite messy at the moment because the [http] package doesn't support keepalives. If keepalives are supported the authentication scheme would be:

 * send GET request to proxy
 * proxy responds with a 407 (authentication required)
 * use a keepalive connection to authenticate
 * resend the GET request

As an example of the steps required to authenticate with an NTLM proxy look at the following code. The code is not robust and only tested on a single proxy server. Moreover it will fail in plenty of scenarios (e.g. 301 moved responses) Another issue I encountered is that the proxy can either expect the first authentication message from the client in a Proxy-Authorization: NTLM ... or in a Proxy-Authorization: Negotiate ... header. Also it is currently necessary to specify the correct authentication domain even though the server can supply the correct domain in the challenge response. This will not be needed if the 1653431 patch is applied to the NTLM package in [tcllib]. Summarizing the code below is a proof of concept and it is not intended to provide a robust http client implementation.

The code can be quite easily adapted to send a CONNECT request to the proxy and return the socket after authentication completes. This allows normal socket traffic as far as allowed by the proxy (usually only SSL encrypted traffic) to pass through the proxy.

 package require SASL::NTLM
 package require base64

 set url url
 regexp {.*//(.*?)/} $url -> host
 set proxy proxyserver
 set port port
 set user username
 set pass password
 set file filename ; # file to store the page

 proc Callback {context command args} {
    switch -exact -- $command {
        username    { return $::user }
        password { return $::pass }
        realm    { return "" }
        hostname { return [info host] }
        default  { return -code error unxpected }
    }
 }


 proc main {{mech NTLM}} {
  set challenge ""
  set s [socket $::proxy $::port]
  fconfigure $s -blocking 1 -buffering line -translation crlf
  set ctx [SASL::new -mechanism $mech -callback Callback]
  while {1} {
    set more_steps [SASL::step $ctx $challenge]
    set response [SASL::response $ctx]
    puts "----"
    puts "Sending NTLM message to proxy"
    puts "NTLM key: [base64::encode -wrapchar {} $response]"
    puts $s "GET $::url HTTP/1.1\nHost: $::host\nProxy-Authorization: NTLM [base64::encode -wrapchar {} $response]\n"
    puts "----"
    if {!$more_steps} {break}
    puts "Handling proxy reply"

    set response {}
    while {[set line [gets $s]] ne "" } {
      append response $line\r\n
      # break if headers done
      if {$line == ""} {
        break
      }
    }
    set length {}
    regexp {Content-Length: ([0-9]*)} $response -> length
    puts "response length: $length"
    # puts $response
    if {$length eq {} } {
      set body [read $s]
    } else {
      set body [read $s [expr $length-1] ]
    }
    # puts $body
    regexp {Proxy-Authenticate: NTLM (.*)\r\n} $response -> challenge
    puts "Server challenge: $challenge"
    set challenge [base64::decode $challenge]
   }
   puts "Handshake completed"
   puts "----"
   SASL::cleanup $ctx
   set response {}
   while {[set line [gets $s]] ne "" } {
    append response $line\r\n
    # break if headers done
    if {$line == ""} {
      break
     }
   }
   set length {}
   regexp {Content-Length: ([0-9]*)} $response -> length
   puts "response length: $length"
   puts $response
   fconfigure $s -translation binary -buffering full -blocking 1
   # if length is not specified, server will close the connection after this page
   if {$length eq {} } {
    set body [read $s]
   } else {
     set body [read $s $length ]
   }
   close $s
   set f [open $::file w]
   fconfigure $f -translation binary
   puts $f $body
   close $f
 }

 main
----
[MJ] - Note that Internet Explorer use NTLM negotiation and the SPPI API to create NTLM tokens based on your current Windows (domain) logon credentials. This will allow proxy authorization without providing credentials. What needs to be done for this is:

* Create a wrapper around some of the SPPI API calls
* Use the API calls to generate the base64 encoded keys.

For an implementation for [Ruby] see http://rubyforge.org/projects/rubysspi/
  
----
[[ [Category Internet] ]]