thread

thread an extension distributed with Tcl, brings native thread capabilities to Tcl.

Disambiguation

A distinct meaning of "thread" occurs in discussions of bytecode, especially when Forth is nearby.

Attributes

What
Thread extension
Where
http://tcl.sf.net/
Where
https://core.tcl.tk/thread/timeline?y=ci
Description
This Tcl extension, with a thread-enabled core, allows script level access to run Tcl scripts within threads. Currently at version 2.7.0.
Updated
2013-01-02
Contact
See web site

Associated Modules

ttrace
Replicate interpreter state in a multithreading application.
tpool
Pools of worker threads.
tsv
Share data between threads.

See Also

Threads
general information about threads
Tcl and threads
general information about threads and Tcl
asynchronous threads, by CmcC
An OO wrapper to make asynchronous threading over the Thread package a little easier.
two threads run synchronously
an example
Thread worker example
by dgp
thread: -eventmark and send -async
How these two features can interact in surprising ways.
daerth
Threaded pipelined backpressure-mediated dearth stations. Stations operate in multi or balanced mode, the latter being an alternative to tpool.
ttrace
Replicate interpreter state across threads.

Critique

dgp in the Tcl chatroom, 2013-04-04:

10:41 <@ijchain> <dgp> tpool::get has ability to capture the error state of a 
                                 worker thread in the pool.
10:43 <@ijchain> <dgp> it does look like [tpool::wait] is the only way to wait 
                                 for worker thread completion.  That's unfortunate.
10:45 <@ijchain> <dgp> I imagine you can work around the design defect.
10:45 <@ijchain> <dgp> But now I've got reasons not to use tpool.  Thanks for 
                                 that.

Documentation

Tcl Threading Model
an overview.
Multi-Threaded Tcl Scripts
A sample chapter from Practical Programming in Tcl and Tk.
thread man page
tpool man page
tsv man page
ttrace man page

Community

tcl-threads mailing list
at SourceForge.

Tools

dqkit
is a convenient deployment. snichols: I've had problems with dqkit and Tcl threads. On a Win32 system in order to find packages in a child thread I had to put the package files outside of the starkit. Because of that its almost easier to just use a thread enabled Tcl interpreter instead of dqkit if the package files have to exist outside of the starkit.

commands

package require Thread

thread::create ?-joinable? ?-preserved? ?script?
thread::preserve ?id?
thread::release ?-wait? ?id?
thread::id
thread::errorproc ?procname?
thread::unwind
thread::exit
thread::names
thread::exists id
thread::send ?-async? ?-head? id script ?varname?
thread::broadcast id script
thread::wait
thread::eval ?-lock mutex? arg ?arg ...?
thread::join id
thread::configure id ?option? ?value? ?...?
thread::transfer id channel
thread::detach channel
thread::attach channel
thread::mutex ...
thread::rwmutex ...
thread::cond ...

Description

Some resources are shared between threads. The effect on other threads must be taken into account when manipulating them. Such resources include:

environment variables
exit
Exits the process, regardless of which thread executed it. The value passed to exit only propagates as the process exit code if exit is called from the main thread.
pwd
cd changes the current directory for all threads.
stdio
stdin, stdout, and stderr are between all threads. But note that some platforms, namely Windows, do not provide these channels to processes. On those platforms, the standard channels can not be assumed to be shared among all threads. Corollary: In Windows, puts string output within threads do not show up in the wish shell. This is quity tricky, since novices could believe that the first threads created do not work. It must be noted especially that Tcl error output (always to stderr) does not appear in the shell as well. Hint: during basic thread debugging, please use plain tclsh instead of wish. Once debugged, one can use wish as well.
thread::errorproc
This is a global setting, common to all threads.

ttrace sub-package

The 2.6 release adds a sub-package called Ttrace. This allows for easy propagation of interpreter related resources (procedures, namespaces, objects (for XOTcl)). Usage is trivial: yust wrap the code you want to replicate within ttrace::eval like this:

package req Thread
package req Ttrace

for {set i 0} {$i < 4} {incr i} {
        set tid($i) [thread::create -preserved]
}

ttrace::eval {
    proc foo args {
       puts foo
    }
    proc bar args {
       puts bar
    }
}

and execute this from any thread created by the thread extension. This will replicate the definitions of 'foo' and 'bar' to all existing threads. In the above example, all 4 threads will be seeded. One very important design feature of Ttrace is that resources are propagated in a lazy fashion. That is, nothing is actually done in other threads until the resource gets referenced for the first time. This is accomplished by overloading the Tcl unknown command. So, when the unknown triggers, it tries to locate the resource definition by doing a lookup in the Ttrace private (in-memory) database first. On hit, the definition is loaded on-the-fly in the current interpreter. On miss, the unknown processing is delegated down-the-road. This way, thread startup and memory consumption are minimized. Thanks to Vince Darley for his great command trace framework, which is used as the base for the Trace package. For more information see 'man ttrace'.

Threads and the event loop

Silas 2006-12-17: By default threads that are not the main thread don't have an event loop (Am I wrong?). Do add it, insert a [vwait forever] or [vwait your_variable] inside your thread::create block.
AKM 2006-10-26 I thought the advice was to use thread::wait so that the thread can be terminated with thread::release?

Examples

Sharing a common logger thread example
An example by Zoran Vasiljevic of a logger thread and worker threads on c.l.t. (add google groups reference here):

Example: A Cookbook Approach

Marty Backe 2002-02-21:

How to use the Thread2.1 package

#! /bin/sh
#\
exec tclsh "$0" "[email protected]"

package require Thread 2.1

#
# Start a thread. If you will be running many long running commands
# at the same time, just create multiple threads identical to this one, and
# keep track of the threadID's.
#
set threadID [thread::create {
    #
    # From here to the 'thread::wait' statement, define the procedure(s)
    # that will be called from your main program (which, btw, is thread #1)
    #
    # In this case, I've defined a simple procedure that executes the command
    # passed into this thread. This command could be an external program or
    # long running tcl command.
    #
    # The result is then sent back to the main program (thread 1) via a call
    # to a procedure in thread 1. In this example, it's a procedure defined
    # at the global level.
    #
    # The 'thread::wait' is required to keep this thread alive indefinitely.
    #
    proc runCommand {ID command} {
        set result [eval $command]
        eval [subst {thread::send -async $ID \
            {::printResult [list $result]}}]
    }
    thread::wait
}] 

#
# Here is the procedure that gets called from the thread when the thread
# has completed its work.
#
proc printResult result {
    puts $result
    exit
}

proc passTheTime {} {
    puts [clock format [clock seconds]]
    after 1000 passTheTime
}

#
# Here we define a sample command, and pass it into the previously started
# thread.
#
set commandString "exec du -sk /usr/local"
eval [subst {thread::send -async $threadID \
    {runCommand 1 [list $commandString ]}}]

#
# In this example, lets just pass the time until the thread is complete
#
passTheTime
vwait forever

AW: The above example also provides a nice way to get results from a worker thread back to your main program: have the worker thread send them back to the main thread in the same way you send work to the worker threads. Also, note that there is an equivalent way of making a command with replaced params, instead of eval [subst ...]:

thread::send -async $threadID \
    [list runCommand 1 [list $commandString ]]

Also note that the main thread is not 1 on windows, so you'll need to pass the main thread ID to the worker threads.

To Do

update ftp://ftp.tcl.tk/pub/tcl/thread/
as of 2013-05-28, the latest version there is 2.6.7rc1

Gotcha: tsv::lappend

PYK 2015-03-27: Unlike lappend, which can leverage Tcl's copy-on-write semantics, tsv::lappend must copy the list in order to return it. This can be slow, and is almost never what you want in a threaded program. The short answer is ''always use ::tsv::lpush instead of tsv::lappend. Other commands that don't return the entire list are also ok: ::tsv::set ::tsv::linsert, ::tsv::lpush ::tsv::lpop, and ::tsv::keylset

In the future, perhaps ::tsv::lappend should take a hint from ::tsv::linsert and not return the list.

Debugging

AW: Debugging threads is non-obvious at first. This does work under tclsh, but not under wish (the puts from inside the thread is not displayed in the console):

package require Thread
catch {console show}
set ::gThread [thread::create {thread::wait}]
puts [list {created thread} $::gThread]
proc test {} {
    puts {test starting}
    thread::send -async $::gThread {puts [clock seconds]}
    after 2000 test
    puts {test ending}
}

test
puts {started first test}

#only needed for tclsh, to keep the interpreter alive and keep the event loop running
vwait forever

The reason is described here [1 ]. Replacing the thread command by:

thread::send -async $::gThread {
    tk_messageBox -message [clock seconds] -title t -type ok
}

Also does not work under wish (messagebox is never shown). The reason for that becomes obvious when you implement a thread error proc:

proc ThreadError {thread_id errorInfo} {
    puts [list {Error in thread $thread_id. Error:} $errorInfo]
}  
thread::errorproc ThreadError

There is no tk in the thread's tcl interpreter. (This is probably due to tk not being thread safe?)

So finally, this works in both tclsh and wish:

set ::a 0

package require Thread
catch {console show}

set ::gThread [thread::create {thread::wait} ]
puts "created thread $::gThread"
proc test {} {
    puts [list {test starting} $::a]
    thread::send -async $::gThread { return [clock seconds] } ::a
    after 2000 test
    puts [list {test ending} $::a]
}

test
puts {started first test}

#only needed for tclsh, to keep the interpreter alive and keep the event loop running
vwait forever

Alternatively, you can use the technique under 'A Cookbook Approach' above to send a command back to your main thread.

Misc

One curiousity--even annoyance--of threads prior to the 2.6 release is that, while threads can communicate variables and other resources, they do not have an easy way to share proc definitions. The main alternatives are to:

  • re-source in definitions from the file system; or
  • stuff a proc definition in a variable, send the variable, and eval

2.6 will provide [ttrace::eval], as in

ttrace::eval source myfile.tcl

Commands proper? For the foreseeable future, each thread must load its own.


Punana 2009-05-05: I tried using the thread callback in wish, on no matter what I did, the thread was never able to find the callback proc in the main thread. So, eventually I found a work around, Instead of using "thread::errorproc ThreadError" for an error reporting method, I manually generate an error inside the thread, and use the errorproc callback to do what I wanted in the first place.


Ro 2011-09-16: Anyone know where to find a threaded build of tclkit for OSX? 8.5 preferably... Roy Keene's tclkit-8.5.9-macosx-i686 from http://rkeene.org/projects/info/wiki/Tclkits isn't threaded.


AK 2011-09-16 11:54:21: ActiveState's basekits are threaded for OS X, Windows in all versions, and threaded for all platforms for 8.5+.


MHo See processing dirs with tpool.