tls

tls , originally by Matt Newman, is a portable extension that provides the power of OpenSSL to Tcl programs. This extension can be used to utilize SSL encryption on top of any valid Tcl Channel - not just sockets.

Attributes

website
https://core.tcl.tk/tcltls/
website (old)
sourceforge .net
website (old)
sensus .org
current version
1.7.16
release time
2018-02-07

DA 2018-04-07: I just spent about 4 hours searching and trying to install TLS on Ubuntu with no success. I am an engineer with years of Tcl experience on windows and pretty solid grounding in Linux. I am a big user and fan of Tcl, but its no wonder folks don't flock to Tcl. I will revert to having my Tcl script write out a PHP script to make my https call.

dbohdan 2018-04-07: DA, if you give more information, someone here may help you get it working. Were you trying to build TclTLS yourself? If not, did sudo apt install tcl-tls not work for you?

DA 2018-04-10: Yes, 'sudo apt install tcl-tls' was one of the attempts I tried. But I am unable to do so on a shared hosting account. Thanks for your help, I will post more in a few hours...

ABU 1-apr-2019: TLS 1.7.16 multiplatform binaries.
At tls 1.7.16 you can download a multi-platform package wiht pre-built binaries for the most popular platforms ( Win x32/x64, Linux x32/x64, MacOS). Package is quite large (9.5 MB) but you are free to remove the dll/so/dylib for those platforms you are not interested in.

Using TLS in 2016

  • Starting with TLS 1.7 SSLv2 and SSLv3 are disabled by default
  • TLSv1, TLSv1.1, and TLSv1.2 are enabled by default
  • (tls 1.7.11) There is -autoservername true option which will perform the needed "-servername $host" work done in tls::socket, getting rid of the proc below, now you can do: http::register https 443 [list ::tls::socket -autoservername true]
  • Building under Windows see BAWT Build Automation With Tcl / Batteries included [1 ]

Using TLS in 2015

Since heartbleed and poodle and such fancifully-named vulnerabilities, TLS needs carefully chosen options to work with servers using today's best practices. The ultimate configuration seems to be:

  • SSLv2 and SSLv3 disabled
  • TLSv1 enabled
  • SNI enabled with the -servername option (requires tls 1.6.4)

Turning on SNI doesn't seem to be possible with a simple tls::init (or http::register with options), so a proc like below is needed:

package require tls 1.6.7
proc tls:socket args {
    set opts [lrange $args 0 end-2]
    set host [lindex $args end-1]
    set port [lindex $args end]

    # From version 1.6.7, it shouldn't be necessary to specify any ciphers.  If
    # that's not true, please report your experience on this page.
    # [APN] I think you misunderstand the full motivation for -ssl3 false. If you leave
    # it out, you will afaik land up talking SSL3 if the server does not mind SSL3.
    # That's something you generally don't want. Unless 1.6.7 has completely disabled
    # SSL3 support which I don't think it has.
    ::tls::socket -servername $host {*}$opts $host $port

    #If that doesn't work, try this
    ::tls::socket -ssl3 false -ssl2 false -tls1 true -servername $host {*}$opts $host $port

    # And if that doesn't work, try this
    # ::tls::socket -ssl3 false -ssl2 false -tls1 true -tls1.2 false -tls1.1 false \
    #    -servername $host {*}$opts $host $port
    
}

# using the http package also needs this:
package require http
http::register https 443 tls:socket

aspect is kinda-sure this is probably right based on the fact that it worked here . Anyone who knows better is invited to add more information here.

Response

[I]n Tcl, adding SSL support is usually about changing socket to tls::socket :-) Even on already opened channels this is mostly a matter of using tls::import — you don't have to change anything in the software itself. This is a great thing.
-- Wojciech Kocjan

See Also

   [https] : Information of tls with http protocol.
SSL Tunnel
a script to tunnel through web proxies using TLS. Demonstrates TLS's ability to convert an already existing connection into a secured connection.
TWAPI
which provides a tls_socket as well as certificate management command on Windows. See TLS server using TWAPI for a server-side example.
A Server Template
Provides an example of optional TLS support.
autoproxy
autoproxy features the command ::autoproxy::tunnel_connect, which uses tls to tunnel through proxies.

Development

https://core.tcl.tk/tcltls/dir?ci=trunk

Documentation

https://core.tcl.tk/tcltls/wiki/Documentation

APN: I've written a short blog entry describing how to build OpenSSL and the TLS extension on Windows using Visual C++.

Installation

[What's a complete "bill of materials" for a correct installation?]

Description

OpenSSL extension for Tcl, utilizing any standard Tcl bi-directional channel (not just sockets). Requires Tcl 8.2 or newer, Trf core patch. Runs under Win32 and Unix.

Windows, Solaris and Linux binaries available on SourceForge tls page.

SSL

PS 2015-01-04: Use at least version 1.6.3.3 to ensure compatibility with servers which have disabled SSLv3 and TLS 1.0. Earlier versions (1.6.1 for example) will return empty result when trying to connect to these servers.

HolgerJ This is true for instance for IMAPS connections to Strato. Actually, after installing 1.6.4 (manually, since Canonical hasn't managed to update the 1.6 version of tls-tcl in Ubuntu 14.4 LTS), not even any of the ::tls::init settings mentioned below (disabling older versions) are necessary; the negotiation with the server works ok. With version 1.6, connections have failed since about three months ago. The connection was established, but the first read gave an error. 2015-06-10

By now tls 1.6.7 is current, but even ActiveState's Tcl for Windows contains only 1.6.5. Luckily, there is help. Just call the teapot.exe to install the latest version C:\> teapot install tls 1.6.7 and you are done. Under Linux, it's not too difficult to compile the version from Sourceforge yourself. 2016-02-24


After Heartbleed , many SSL servers have been configured to drop the conversation with a client that offers to speak older versions of the protocol. So far, the workaround for TLS is to explicitly disable older protocols:

::tls::init -tls 1 -ssl3 0 -ssl0 0

In TLS On Windows , comp.lang.tcl, 2015-03-25, there was a report that it was necessary to also disable tls-1.1 and tls-1.2 for a particular server:

::tls::init -tls1 1 -tls1.2 0 -tls1.1 0 -ssl3 0 -ssl2 0

WHD: In point of fact, tls 1.6.4 rejects both of the preceding commands; the "-tls" and "-ssl" options are unknown.

APN The above line was copied wrong (now fixed). If you read the original carefully, you will see it is not -tls 1, it should be -tls1 1 and there is no -ssl 0.

It was also reported that when it fails to negotiate a connect, TLS can hang, perhaps because it is iteratively retrying after a protocol reset.

APN The hang is not iteratively trying. It is stuck in an infinite loop. A bug was logged and is fixed in 1.6.6 (and possibly in 1.6.5 as well).


PT 2003-07-22:

Playing with rmax's A Server Template I decided to add SSL support to the server. See the wiki page for the minimal changes required to support this. The main issue is creating a pair of certificates to use. I'm using openssl under windows and I'm being my own certificate authority. It might be possible to shortcut some of this.

  • Create a suitable openssl configuration file
Copy the distributed openssl.cnf and edit the Distinguished Name stuff to suit you.
  • Generate a 1024-bit private key for your server
       openssl genrsa -out server-private.pem 1024
  • Generate a self-signed public key certificate
       openssl req -new -x509 -key server-private.pem -out server-public.pem -days 365 -config openssl.cfg
Fill in the various questions or accept the defaults.

You now have a certificate pair suitable for use in the A Server Template example server.

You can test this with the following client script:

package require tls 1.4
set s [::tls::socket localhost 443]
fconfigure $s -blocking 0 -buffering line
puts $s "help"
while {[gets $s line] >= 0} { puts $line }
puts $s "bye"
while {[gets $s line] >= 0} { puts $line }
close $s

As you can see, there isn't much required on the client side — and yet all the communications will now be encrypted using the server's public key. With different options you can force the client to use a specific key and so on.

Alternatively, it turns out you can use the fine tkcon as a client for this. This works under windows as well. Open up a tkcon session and do the following

tkcon master package require tls
tkcon master tls::socket your.server.net 443

Then go to the Console -> Attach To -> Socket menu and the opened channel will be listed. Select the channel (sockXXX) and your tkcon session is now connected to the remote server using SSL. Try help.

SimpleCA

Joris Ballet has written a quite good Certification Authority in Tcl/Tk called SimpleCA, which greatly simplifies running a small CA. It ought to be integrated into tclhttpd.

jmn: In the interests of least licensing hassles, it ought not to be integrated with tclhttpd. GPL fans can do their own private integration and leave the rest of us free to plugin alternative CA solutions.

CMCc: Reasonable point, however modifications to SimpleCA to make it integrable with tclhttpd wouldn't impact tclhttpd's license.

20040701: Upon reflection ... what does it matter if a pure-tcl script is licensed GPL? The distribution is source, so how does the requirement to distribute source make any difference at all? qv GPL Scripts.

Asynchronous Connection

PT 2004-12-23: To connect asynchronously using TLS connections, you must first use socket -async to create the connection and then upgrade the link using tls::import. The following demonstrates this:

package require tls

proc Write {chan} {
   fileevent $chan writable {}
   tls::import $chan
   fconfigure $chan -buffering none -encoding binary -blocking 1
   tls::handshake $chan
   puts "[tls::status $chan]"
   fconfigure $chan -buffering none -encoding binary -blocking 0
}

proc Read {chan} {
   if {[eof $chan]} {
       fileevent $chan readable {}
       puts "Close $chan"
       return
   }
   set d [read $chan]
   puts "$chan [string length $d] '$d'"
}

proc Connect {host port} {
   set s [socket -async $host $port]
   fconfigure $s -blocking 0 -buffering none -encoding binary
   fileevent $s writable [list Write $s]
   fileevent $s readable [list Read $s]
   return $s
}

Problem: Vwait

Why doesn't the following work? (I've presented it here as 3 lines of input to tkcon, so you can see the output easily on a Windows system and to avoid timing issues.)

package require tls
proc w {f} {fconfigure $f -blocking 0;fileevent $f readable {set ::event 0};vwait ::event}
set f [tls::socket -async pop.gmail.com 995]; w $f; puts [read $f]

That hangs.

But if I manually open the socket and do a read, it completes (albeit very slowly). Here's what it looks like:

Main console display active (Tcl8.4.10 / Tk8.4.10)
1 % package require tls
1.5.0
2 % set f [tls::socket -async pop.gmail.com 995]
sock180
3 % read $f
+OK Gpop ready for requests from 129.6.33.161 27pf4267101wrl

4 % 

PT You are sitting in the vwait and never send anything. You can probably get that to work using tls::handshake. Usually the tls negotiation is only done once you send something. Calling tls::handshake can force it along. IIRC you want to be blocking and binary for the tls negotiation and then switch to non-blocking and line-mode for the POP protocol. Of course this has all been done and debugged in the relevant tcllib package so why not save yourself some grief. However here is a demo script that connects and the immediately quits.

package require tls
variable quit 0
proc Read {chan} {
    if {[eof $chan]} { fileevent $chan readable {}; puts "Closed"; set ::forever 1; return }
    puts [read $chan]
    variable quit ; if {!$quit} { puts $chan QUIT; set quit 0 }
}
proc Write {chan} {
    fileevent $chan writable {}
    tls::handshake $chan
    fconfigure $chan -buffering line -blocking 0 -translation crlf
}
set sock [tls::socket -async pop.gmail.com 995]
fconfigure $sock -encoding binary -buffering none -blocking 1
fileevent $sock writable [list Write $sock]
fileevent $sock readable [list Read $sock]
vwait ::forever

LWS: A possible problem with the original code, too, is the use of tls::socket. There seems to be some difference between its defaults and the built-in socket command. I struggled with this issue while trying to get some IMAP4 code running (with binary translation and in blocking mode). The fix is apparently to use the built-in socket command and then immediately invoke tls::import on the channel. Seemed to fix my problem, anyway.

Using tls for Authenticating to a Service

DKF:

Scenario: You've got a service (implemented in Tcl) and you only want the right users to access it.

Outline Solution: You use SSL sockets in client-authenticated mode (servers are always authenticated in the SSL protocol) so that each end proves to the other who it is.

Prerequisite: You've already set up a little CA and have issued the server and (each) client with a signed keypair. (Note that, for this sort of operation, I'd advise not trusting the usual commerce Website CAs; they're only of use precisely when you don't know who is going to connect ahead of time.)

Server Code:

package require tls
set acceptableDNs {
   CN=Jimbob,O=Example,C=US
}
proc tls::password args {
   return "theServerCertificatePassword"
}
# Assume we're on port 3000 for demonstration's sake
set srv [tls::socket -server acceptCmd -require 1 -cafile caPublic.pem -certfile server.pem 3000]
proc acceptCmd {chan addr port} {
   # Note that when we get here, the client has *already* authenticated
   # The checks here are *authorization* checks...
   global acceptableDNs
   set status [tls::status $chan]
   set whoIsCalling [dict get $status subject]
   if {$whoIsCalling ni $acceptableDNs} {
      # don't want to talk to them, even though they've authenticated
      close $chan
      return
   }
   # Now set up usual socket fileevents and fconfigures...
}

Client Code:

package require tls
proc tls::password args {
   return "1, 2, 3, 4, 5? That's the code for my luggage!"
}
set chan [tls::socket -cafile caPublic.pem -certfile client.pem server.site.net 3000]
# Now use as normal

That should be enough, but it's not tested code so you'll want to double-check for yourself. And (of course) report back here if it works. Note that doing the equivalent with username/password is actually similarly complex at the Tcl-scripting level.

Standard Channels

ProlyX 2007-05-28:

Is there any way to get TLS working for std channels? using a server (ftp like) with inetd but i cant get it working with TLS even with chan create to simulate an bidirectional channel for stdin and stdout.

DKF: I believe you can do it by using a reflected channel (see chan create) to make a fake bidirectional channel from stdin/stdout, and then tls::importing that channel.

Writing a -command Callback Handler

DKF: Melissa Schrumpf posted this in TLS Verification , comp.lang.tcl, 2007-05-24. It shows how to go about building your own procedure that is suitable for use with the -command option, as well as demonstrating the use of tls::init.

So, since my search for example code for a -command callback proc was more or less fruitless (I believe I found one example that always returned 1), I will now post mine here, with an explanation inline:

tls::init -certfile $files(my_cert) \
           -keyfile $files(my_key) \
           -ssl2 1 -ssl3 1 -tls1 0 -require 0 -request 1 \
           -cafile $files(ca)

# This is largely based on ::tls::callback

proc tlsCbfn {option args} {
   switch -- $option {

     error { return 1 }

     verify {
        foreach {chan depth cert rc err} $args {;}

        array set c $cert

        if {![info exists ::AUTH($chan,authcode)]} {
           if {$rc == 1} {
              set ::AUTH($chan,authcode) 1
              set ::AUTH($chan,autherr) {} 
           } else {
              set ::AUTH($chan,authcode) $rc
              set ::AUTH($chan,autherr) $err
           }
        } else {
           if {$rc != 1} {
              set ::AUTH($chan,authcode) $rc
              set ::AUTH($chan,autherr) $err
           }
        }

        # The information I'm interested in is whether or not the
        # cert validated.  I include the error message in case the
        # application wants to take different actions on different
        # errors (for example, accept an expired cert with a warning,
        # but reject one for which the chain does not validate.

        # TLS does not verify that the peer certificate is for
        # the host to whom we are connected:
        if {(($depth == 0) && ($::AUTH($chan,authcode) == 1))} {
           set subl [split $c(subject) /]
           foreach item $subl {
              set iteml [split $item =]
              if {[lindex $iteml 0] eq {CN}} {
                 set certcn [lindex $iteml 1]
                 set certhost [lindex [split $certcn .] 0]
                 set peerinfo [fconfigure $chan -peername]
                 set peercn [lindex $peerinfo 1]
                 set peerhost [lindex [split $peercn .] 0]
                 if {$peercn eq $peerhost} {
                    # need full cn
                    set mycn [lindex [fconfigure $chan -sockname] 1]
                    set mydomainl [lrange [split $mycn .] 1 end]
                    set peercnl [concat $peercn $mydomainl]
                    set peercn [join $peercnl .]
                 }

                 # on some networks -peername host will only
                 # be the hostname, not the full CN.
                 # whether it is the "right" thing to do
                 # to accept these connections is left as an
                 # exercise for the reader.  I decided to allow
                 # it here.  But then, I'm doing this on an intranet.
                 # I doubt I'd allow it in the wild.

                 if {$certcn != $peercn} {
                    set ::AUTH($chan,authcode) 0
                    set ::AUTH($chan,autherr) {CN does not match host}
                 }
              }
           }
        }

        # always accept, even if rc is not 1
        # application connection handler will determine what to do
        return 1
     }

     info {
        # For tracing
        # upvar #0 ::tls::$chan cb
        # set cb($major) $minor

        # NOTE: I have no idea what this is supposed to do.
        # It was in ::tls::callback, but if left in, it causes even
        # a valid certificate chain to fail to connect.  Removing it
        # does not seem to inhibit TLS's ability to detect:
        #    self-signed certificates
        #    expired certificates
        #    certificates that are not yet valid
        #    certificates signed by a different CA
        # therefore, out it goes.

        return 1
     }

     default  {
        return -code error "bad option \"$option\":\
           must be one of error, info, or verify"
     }
  }
}

# It's a good idea to unset any ::AUTH($chan,*) entries when they
# are no longer needed -- either after the connection has been
# established, and the authcode and autherr have been checked, or
# in the callback for cleanup/closing the socket.  Not doing so
# is bad bad bad, and should only be done in short-lived applications
# that exit after one connection. 

I hope that helps someone in the future.


LV: Just a note about configuring tls. You need to make certain that the "bitness" of tcl, tls, and openssl all match. That is to say, they all three need to be built as 32 bit apps, or all three as 64 bit apps.

For some reason, I also found myself taking several rounds to get the value of --ssl-dir correct. I don't know why but some output from the system, or readme, or something had me thinking that I needed to give configure the path to the directory where the .h was, when just as the ./configure —help says, it wants the root directory (the place you told openssl to install things).

Alas, for some reason, I _still_ can't get things to create a library. After all the struggles to get the right bitness, the right pathnames, etc. I end up with the following on SPARC Solaris 8.

/usr/ccs/bin/ld -G -z text -o libtls1.6.so tls.o tlsIO.o tlsBIO.o tlsX509.o fixstrtod.o -R /projects/intranet/ssl32/lib -L/projects/intranet/ssl32/lib -lssl -lcrypto -lsocket -lnsl -L/vol/tclsrcsol/tcl85/tcl8.5.3/unix -ltclstub8.5

Text relocation remains                         referenced
    against symbol                  offset      in file
<unknown>                           0x4         /projects/intranet/ssl32/lib/libssl.a(ssl_ciph.o)
<unknown>                           0x2c        /projects/intranet/ssl32/lib/libssl.a(ssl_ciph.o)
<unknown>                           0x54        /projects/intranet/ssl32/lib/libssl.a(ssl_ciph.o)

When I google for this type of error, I see recommendations to include the "-z text" which, in fact, is there...


MG: If you attempt to tls::import a socket which isn't SSL-enabled, you get a Tk error popup which you can't catch (as it happens in a background callback). The only way to prevent that is to give a -command callback, which allows you to change the "error" behaviour. However, beware - if you do that, the -require option to tls::import is effectively ignored, which causes self-signed certificates to fail where previously they had succeeded. Took me a while to find out where/why that happened, and how to get around it: you need to either specify -request 0 (to not even ask for a certificate in the first place), or make sure your callback proc always returns 1 for "verify" subcommands.


dkf - 2014-04-11 14:45:53: Are the builds of this floating around susceptible to the Heartbleed attack? Or are they so old it doesn't matter? APN The 1.6.4-6 series at least should not be susceptible though it is still best to disable ssl3 as described in this page.


APN: The TLS handshake/negotiation itself seems to be blocking no matter whether the socket is marked blocking/non-blocking. I wonder if this is expected behaviour or am I doing something wrong.


HaO 2015-03-02: APN published his build recipe for TLS on Windows with VC6++: [2 ]. Thanks, thats magic !


HaO 2015-07-06: Andreas reported, that tls suffers of Bug 336441ed59 on windows, which is fixed in TCL 8.5.16 and TCL 8.6.2. The issue is a randomly stalling connect process in 1 of 500 tries.


Use of CA certificates

Alexandru 2015-07-07: This is a summary of a post on comp.lang.tcl: https://groups.google.com/d/msg/comp.lang.tcl/Lr2HbXKLeO8/RdsomVy2WisJ

If I want my Tcl application to communicate with my web server through TLS (much like a web browser), I only need to use the -cafile option and ignore the -certfile option. I need this in order to avoid man-in-the-middle attacks

So I issue:

package require http
package require tls
::http::register https 443 [list ::tls::socket -request 1 -require 1 -ssl2 0 -ssl3 0 -tls1 1 -cafile CA.crt] 

where CA.crt is a bundle of root certificates of my Certificate Authority (CA). http://www.google.com/url?q=http%3A%2F%2Fcurl.haxx.se%2Fca%2Fcacert.pem&sa=D&sntz=1&usg=AFQjCNE0xlPAnyfNa_tlCkt4guaHpEz3kw provided a link to a Bundle of public CA Root Certificates, that works with most of the Certificate Authorities. Hier is the link: Ashok

In my case the CA is VeriSign and at the begining I had a problem communicating with my web server. When I input

::http::geturl https://www.mydomain.de

I get the error:

error reading "sock560": operation not supported on socket

Since the TLS package does not provide much insight about this error I followed Ashok's advice and tried to connect through openssl instead of Tcl's tls package:

openssl s_client -connect www.mydomain.de:443 -CAfile CA.crt

That helped a lot, because openssl has much more details in the output. The output was:

CONNECTED(000001B0) depth=2 C = US, O = "VeriSign, Inc.", OU = VeriSign Trust Network, OU = "(c) 2006 VeriSign, Inc. - For authorized use only", CN = VeriSign Class 3 Public Primary Certification Authority - G5 verify error:num=20:unable to get local issuer certificate verify return:0 --- Certificate chain

 0 s:/C=DE/ST=Baden-W\xC3\xBCrttemberg/L=Stuttgart/...
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
 1 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
 2 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
   i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority

The verify error:num=20:unable to get local issuer certificate is issued when a certificate in the chain is missing. So I have searched in the CA file and identified the missing certificate, which I could copy from the Firefox browser and paste into the CA file.

Then the connection through the Tcl TLS package worked like a charm:

::http::register https 443 [list ::tls::socket -request 1 -require 1 -ssl2 0 -ssl3 0 -tls1 1 -cafile CA.crt]
::http::geturl https://www.mydomain.de

How to copy a certificate from the Firefox browser: Choose from the main menu Tool>Options>Advanced>Certificates>View Certificates. Select the missing certificate of your Certificate Authority and click on Export. You can also select multiple certificates and export them. Then open the exported files and simply copy paste the text to the CA.crt.


Building on Debian 9 (Stretch)

apt install tcl8.6 tcl tcl8.6-dev tcl-dev build-essential libssl-dev pkg-config;
wget -qO- https://core.tcl.tk/tcltls/uv/tcltls-1.7.16.tar.gz | tar xz;
cd tcltls-1.7.16; ./configure; make; make install;