Expect Hints and Tips

Expect Hints and Tips describes tricks of the trade when using Expect.

See Also

Stubsification of Expect
Secure expect

Sending Special Characters

This is how you send ctrl+C through expect: send \003

Unambiguous Commands

Because of the potential for name conflicts in the global namespace, [send] command is also available as [exp_send], a longer name which is less likely to conflict with other package commands. Likewise, various other Expect commands are also available via longer names.

When working with an extended Tcl, such as Tk, it's often necessary to use the longer names, since, for example, Tk has its own send command

If Expect had been written for ten-year-old Tcl, rather than being the first extension ever discovered, it of course would rely on namespaces to avoid name conflicts."

Proper Use of exp_close and exp_wait

D. J. Hagberg rightly writes, "Here's the BIG RULE for Expect: Every exp_spawn requires a call to exp_close *and* exp_wait."

RJ 2007-06-21: Yeah, I don't know. I've only ever ran across a single issue that required an exp_close/exp_wait to deal with it. That was a script that was testing how many processes I could spawn before running into memory problems on a Solaris station. I ran out of ptys first, and that hung the Xserver and dumped a huge core file. But I've never needed to close/wait - the script exits and the processes go away, unless you loop a spawn command too many times. I've never seen that in a real-use script, aside from multixterm.

PYK 2014-07-16: exp_wait is more often needed than exp_close, but neither is strictly necessary if a script is finished and happy to let the spawned program receive an EOF signal when the script exits. The most common gotcha is that the user doesn't foresee the Expect script ending and the spawned program getting prematurely killed. In this case, exp_wait is usually what's wanted, and if the spawned program isn't going to end on its own accord, maybe also exp_close. Another way to make sure the spawned program runs to completion is to expect the special pattern eof, perhaps after setting $timeout to -1. This is probably far more common than using exp_wait or exp_close.

Expect Variables Set in Local Scope

Expect commands have the interesting feature that they look for Expect such as $timeout first in the local scope, and then in the global scope. However, when an Expect command is called from within a procedure, any Expect variables that command sets are set in the local scope. For example, if spawn is called within a procedure, $spawnid will be set in that procedure. There are various ways to work under these circumstances, but the most common is to add, e.g. global spawnid to the procedure that calls spawn

Alternatives to Expect

When programmatic access to some facility is available, it is often preferable to driving a command-line program with Expect:

ssh
ftp
from tcllib
http
from tcllib
smtp
from tcllib

Note: FTP sends passwords in clear text, so be sure to first build a secure tunnel and use FTP over that. See CL's notes on FTP automation . Likewise, make sure not to send e-mail passwords as plain text, either.

Configuring

CL recently mentioned on comp.lang.tcl that expect has a $HOME/.expectrc file that can be used to predefine procs, variables, etc. that one uses regularly.

Terminal Setting

$TERM must be set correctly in the environment. In one case the terminal was screen , but $TERM was set to xterm, causing unexpected results. Setting it to screen solved the problem.

Debugging

To turn on messages from Expect that help explain what it is doing:

exp_internal 1

Embedding in a Shell Script

PYK 2016-08-02:

When embedding an Expect script in a shell script, for a more legible script that also correctly handles values that contain whitespace, pass arguments on the command line instead of interpolating them into the Expect script. For example, instead of this:

#! /usr/bin/env sh

expect <<EOF
    expect $value1 {
        ... $value2 ...
    }
EOF

Do this:

#! /usr/bin/env sh

expect - "$value1" "$value2" <<'EOF'
    lassign $argv value1 value2
    expect $value1 {
        ... $value2 ...
    }
EOF

Misc

[CL needs to explain inband_ftp, Expect philosophy, and vivid cat-based example. Also, how to handle sensitive information (passwords) embedded in plain-text Expect scripts.] <=== RJ 2004-08-08: To whomever authored this request, see Secure expect.

Cameron Laird's Critique

CL's personal view: Expect's old. Expect disappoints and frustrates some newcomers, because it's done well in the style of 1992-1994. With PHP, for example, if you have a question or a need for explanation, you're likely to find http://www.php.net/ rewarding. That's not how Expect is managed, at least not now (mid-2002). Expect's high-level aim is that, "It's in The Book" accurately answers all questions. The book is really quite marvelous as both introduction and reference, but its style might strike some readers as archaic and inaccessible, at least until they shift to accept the approach that was the best in computing about a decade ago.

Ro 2004-05-30: This is is really very interesting Cameron. I just got a copy of the Expect book by my dear old dad, and I'm about to start it. What is this 'older approach'?


There are things that Expect can do that few other things can. I'm using it to jump off Cisco boxes and grab config, stuff like that. I have to agree to a point about the outdated writing style. It's almost easier to learn myself through trial and error, than to understand and help guides.


IBSatisfied 2014-09-18 01:45:14:

Presently, I am involved in the management of ~~3000 customer-facing solaris/linux, servers that are distributed throughout the country. I use expect daily to automate tasks that I otherwise could not. Just night, I updated a profile on all servers in under 60 minutes start to finish. I can create/delete users, update/modify crontabs, install packages, etc. Anything you can do on the command-line, I can accomplish with expect.

Recently, we had an issue that required a telnet from each production server to 7 unique IPs on each of 5 unique ports through 3 separate interfaces. Success/Failure had to be logged per test. It took me ~~ 4 hours to get everything ready and tested. It took another 18 hours to complete over 250000 telnets all of which were logged. All of the individual logs were transferred to a common server for review. I found the needle in the haystack and I did it in my sleep!

Expect has saved me literally thousands of hours over the past 15 years. It can be confusing at times, but only because you don’t know the language. You do need a solid understanding of TCL but everything else you need to know to have a solid start in expect is in Don’s book. The secret: RTM!

You take your shinny ruby and I’ll take expect, and I’ll be in Scotland before ye! Many thanks Don Libes!