tpool — Part of the Tcl threading extension implementing pools of worker threads
paulwal 2020-05-30 In trying to answer MHo's question below, I dug into the code a bit (threadPoolCmd.c) and did some experimenting. I believe I've found the magic formula to make -maxworkers function correctly.
First of all, you can't use the -nowait option. It breaks the -maxworkers functionality. I experience no waiting anyways with or without -nowait, so I'm not sure what the point of this option is.
Secondly, you need a slight pause between posting jobs. I haven't quite wrapped my head around why that is, but through testing this seems to be the case. In the code below, I add a one second pause between jobs, and each job takes 30 seconds. After 30 seconds, you should see a job complete every one second. In testing, I was able to reduce the pause down to 1 millisecond.
package require Thread
set pool1 [tpool::create -minworkers 5 -maxworkers 20 -idletime 40]
puts start:[clock seconds]\n----
for {set i 1} {$i <= 20} {incr i} {
tpool::post $pool1 {after 30000; puts [clock seconds]}
after 1000
}Result:
start:1590816680 ---- 1590816710 1590816711 1590816712 1590816713 1590816714 1590816715 1590816716 1590816717 1590816718 1590816719 1590816720 1590816721 1590816722 1590816723 1590816724 1590816725 1590816726 1590816727 1590816728 1590816729
MHo 2017-07-31 There's something that I don't understand:
Why ist the runtime of
set pool1 [tpool::create -minworkers 10 -maxworkers 20 -idletime 10]
for {set i 1} {$i <= 20} {incr i} {
lappend jobs [tpool::post $pool1 {after 10000}
}
while {[llength $jobs]} {
set ready [tpool::wait $pool1 $jobs jobs]
}double the time of
set pool1 [tpool::create -minworkers 20 -maxworkers 20 -idletime 10]
for {set i 1} {$i <= 20} {incr i} {
lappend jobs [tpool::post $pool1 {after 10000}
}
while {[llength $jobs]} {
set ready [tpool::wait $pool1 $jobs jobs]
}The first call runs ~20,xs, the second one, as expected, ~10,xs.
The documentation states, the only difference is the time at which the workers are created - beforehand, or "on demand" when a post is done. As there are enough -maxworkers defined, what is limiting the operation then...?
I also asked this on that page: Ask, and it shall be given # 12
I've looked at several examples that I've found here and there, and it seems that almost everyone sets -min == -max..... There must be something misterious around this values. I can also specify -minworkers 40 -maxworkers 20, no problem. 40 Workers are created.... Ok, this last fact becomes clear after looking at the c-source:
if (minw > maxw) {
maxw = minw;
}Even more mystery....it works as expected with this little variation:
set pool1 [tpool::create -minworkers 1 -idletime 10]
for {set i 1} {$i <= 20} {incr i} {
lappend jobs [tpool::post $pool1 {after 10000}
puts $jobs; # <--------- mystery
}
while {[llength $jobs]} {
set ready [tpool::wait $pool1 $jobs jobs]
}So, I think the answer to myself is for now:
Posting jobs one after another without doing something between each iteration blocks the start of new threads. Btw, it's not clear, what something exactly is, and why it helps....