2025-06-27:
I am trying to learn the basics of Expect. I got the pdf version of the Expect book.
Most examples I have seen start by handling the login process. I wanted to try it. So, I found an ftp site and I wrote a script. I can login to it from a Windows shell and I think I have the patterns right. But the script is failing for some reason.
This is what I have at the moment. Any suggestions to make it work?
if 0 {you may need to download psftp: https://www.puttygen.com/psftp} set u demo set p password set id [spawn psftp.exe test.rebex.net -P 22] expect -i $id "*login *:" { exp_send -i $id "${u}\r" } \ "timeout" { puts "login timout" } \ "default" { puts "login no match" } expect -i $id "*assword:" { exp_send -i $id "${p}\r" } \ "timeout" { puts "password timeout" } \ "default" { puts "password no match" } expect -i $id "*psftp>*" { puts "OK"}
TWu - 2025-06-30 06:54:02
Hi saito, can You provide the error message or wrong output, please? I've not used Expect for now. But IMHO it maybe switch to a sftp or ftps connection and so need another or more port(s) to be spawned? Try to capture it with a tool like WireShark to see if this is the case. Do You open every time a new spawn connection? Then You loose the old one. Good success and best regards.
saito - 2025-06-30
Thanks TWu. I found out you can log the interaction to a file. Based on that, it seems like I was able to get past the login and the password stages but then it gets stuck again. I have added various patterns at the end but nothing matches and it times out.
I'd really appreciate it if you have any suggestions to make it work.
Here is the current code I have:
set u demo set p password set id [spawn psftp.exe test.rebex.net -P 22] expect -i $id "login *: " { exp_send -i $id "${u}\r" } \ "timeout" { puts "login timout" } \ "default" { puts "login no match" } expect -i $id "*assword: " { exp_send -i $id "${p}\r\n" } \ "timeout" { puts "password timeout" } \ "default" { puts "password no match" } ;# added this line but it made no difference expect -i $id "*\n" { exp_send -i $id "\r\n" } expect -i $id "*psftp> " { puts "GOT IN" } \ "timeout" { puts "session timeout" } \ "default" { puts "session no match" }
And this is the log data:
expect: does "" (spawn_id exp2) match glob pattern "login *: "? no login as: expect: does "login as: " (spawn_id exp2) match glob pattern "login *: "? yes expect: set expect_out(0,string) "login as: " expect: set expect_out(spawn_id) "exp2" expect: set expect_out(buffer) "login as: " send: sending "demo\r" to { exp2 } expect: does "" (spawn_id exp2) match glob pattern "*assword: "? no | Password: expect: does "| Password: " (spawn_id exp2) match glob pattern "*assword: "? yes expect: set expect_out(0,string) "| Password: " expect: set expect_out(spawn_id) "exp2" expect: set expect_out(buffer) "| Password: " send: sending "password\r\n" to { exp2 } expect: does "" (spawn_id exp2) match glob pattern "*\n"? no expect: does "\n" (spawn_id exp2) match glob pattern "*\n"? yes expect: set expect_out(0,string) "\n" expect: set expect_out(spawn_id) "exp2" expect: set expect_out(buffer) "\n" send: sending "\r\n" to { exp2 } expect: does "" (spawn_id exp2) match glob pattern "*psftp> "? no expect: timed out
TWu - 2025-07-01 10:10:00
Do You have to configure the expect channel and/or ftp channel to a system specific line-ending or encoding (Unix/Windows crosing)? Onetime You send without and ontime with a newline character. Can You directly after login send any requests like "get" or "mode"? When the timeout arrive, directly or after some minutes?
saito - 2025-07-01
It times out right after the password entry. It doesn't respond with the expected prompt and quits within a few seconds.
I added the newline character and it allowed me to match the password prompt and enter it. At this point, it is all trial-and-error to see what leads to progress. I have tried to channel configuration like so, without any difference. I am starting to suspect if perhaps Expect has changed in how it handles the spawned processes. I will try sending get or mode and see what happens.
fconfigure $id -blocking 0 -buffering none
Jeff Smith 2025-06-30 : for me the easiest way to get an expect script to work is to use autoexpect. It’s available at https://core.tcl-lang.org/expect/file?name=example/autoexpect&ci=tip
saito 2025-06-30:
Thanks! I will give autoexpect a try and see how it goes.
saito - 2025-07-01
Follow-up on autoexpect: does this assume Expect is an application? I only have it as a package. The autoexpect script is written as a Unix shell script. I tried to modify it to run in a tcl shell. While this modification seems to load OK, it doesn't actually run.
Jeff Smith 2025-07-02 : I have only run autoexpect by installing Expect from the package manager of a Debian/Ubuntu Linux system using
$ sudo apt-get install expect It installs expect and autoexpect $ which expect /usr/bin/expect $ which autoexpect /usr/bin/autoexpect Test expect $ expect expect1.1> expect1.1> exit Test autoexpect $ autoexpect autoexpect started, file is script.exp $ $ exit exit autoexpect done, file is script.exp $
See the man page for how to use autoexpect at https://www.tcl-lang.org/man/expect5.31/autoexpect.1.html
saito - 2025-07-02
I logged into to an Ubuntu installation. It did not have expect but following the commands above, both expect and autoexpect were installed successfully.
I didn't have psftp but apparently there is an Ubuntu package called "putty" and "putty-tools" that install it. Unfortunately the Ubuntu did not like installing either of these, it resulted in error messages. Jeff Smith, could you please check to see if you have psftp? Would you be able to install it via apt?
Jeff Smith 2025-07-03 : Below is what I did to install "putty" on Ubuntu. You could also use "sftp" which I think is installed by default on Ubuntu.
$ which sftp
/usr/bin/sftp
$ sudo apt install putty putty-tools Install putty and check. $ which putty /usr/bin/putty $ putty --version PuTTY: Release 0.81 Build platform: 64-bit Unix (GTK + X11) Compiler: gcc 13.2.0 Compiled against GTK version 3.24.41 Source commit: a8601a72a918dfc2a8e8536a77139d7f37700044 $ which psftp /usr/bin/psftp $ psftp test.rebex.net -P 22 The host key is not cached for this server: test.rebex.net (port 22) You have no guarantee that the server is the computer you think it is. The server's ssh-ed25519 key fingerprint is: ssh-ed25519 255 SHA256:d7Te2DHmvBNSWJNBWik2KbDTjmWtYHe2bvXTMM9lVg4 If you trust this host, enter "y" to add the key to PSFTP's cache and carry on connecting. If you want to carry on connecting just once, without adding the key to the cache, enter "n". If you do not trust this host, press Return to abandon the connection. Store key in cache? (y/n, Return cancels connection, i for more info) y login as: *** OR using sftp''' using login: demo Password : password *** $ sftp -P 22 [email protected] The authenticity of host 'test.rebex.net (194.108.117.16)' can't be established. ED25519 key fingerprint is SHA256:d7Te2DHmvBNSWJNBWik2KbDTjmWtYHe2bvXTMM9lVg4. This key is not known by any other names. Are you sure you want to continue connecting (yes/no/[fingerprint])? y Please type 'yes', 'no' or the fingerprint: yes Warning: Permanently added 'test.rebex.net' (ED25519) to the list of known hosts. Welcome to test.rebex.net! See https://test.rebex.net/ for more information. ([email protected]) Password: stfp> So to use autoexpect to generate the expect script, see below and type your commands interactively. $ autoexpect psftp test.rebex.net -P 22 autoexpect started, file is script.exp login as: demo Pre-authentication banner message from server: | Welcome to test.rebex.net! See https://test.rebex.net/ for more information. End of banner message from server Keyboard-interactive authentication prompts from server: | Password: End of keyboard-interactive prompts from server Remote working directory is / psftp> ls Listing directory / drwx------ 2 demo users 0 Mar 31 2023 . drwx------ 2 demo users 0 Mar 31 2023 .. drwx------ 2 demo users 0 Mar 31 2023 pub -rw------- 1 demo users 379 Sep 19 2023 readme.txt psftp> exit autoexpect done, file is script.exp $ more script.exp #!/usr/bin/expect -f # # This Expect script was generated by autoexpect on Thu Jul 3 14:36:06 2025 # Expect and autoexpect were both written by Don Libes, NIST. # # Note that autoexpect does not guarantee a working script. It # necessarily has to guess about certain things. Two reasons a script # might fail are: # # 1) timing - A surprising number of programs (rn, ksh, zsh, telnet, # etc.) and devices discard or ignore keystrokes that arrive "too # quickly" after prompts. If you find your new script hanging up at # one spot, try adding a short sleep just before the previous send. # Setting "force_conservative" to 1 (see below) makes Expect do this # automatically - pausing briefly before sending each character. This # pacifies every program I know of. The -c flag makes the script do # this in the first place. The -C flag allows you to define a # character to toggle this mode off and on. set force_conservative 0 ;# set to 1 to force conservative mode even if ;# script wasn't run conservatively originally if {$force_conservative} { set send_slow {1 .1} proc send {ignore arg} { sleep .1 exp_send -s -- $arg } } # # 2) differing output - Some programs produce different output each time # they run. The "date" command is an obvious example. Another is # ftp, if it produces throughput statistics at the end of a file # transfer. If this causes a problem, delete these patterns or replace # them with wildcards. An alternative is to use the -p flag (for # "prompt") which makes Expect only look for the last line of output # (i.e., the prompt). The -P flag allows you to define a character to # toggle this mode off and on. # # Read the man page for more info. # # -Don set timeout -1 spawn psftp test.rebex.net -P 22 match_max 100000 expect -exact "login as: " send -- "demo\r" expect -exact "demo\r Pre-authentication banner message from server:\r | Welcome to test.rebex.net! See https://test.rebex.net/ for more information.\r End of banner message from server\r Keyboard-interactive authentication prompts from server:\r | Password: " send -- "password\r" expect -exact "\r End of keyboard-interactive prompts from server\r Remote working directory is /\r psftp> " send -- "ls\r" expect -exact "ls\r Listing directory /\r drwx------ 2 demo users 0 Mar 31 2023 .\r drwx------ 2 demo users 0 Mar 31 2023 ..\r drwx------ 2 demo users 0 Mar 31 2023 pub\r -rw------- 1 demo users 379 Sep 19 2023 readme.txt\r psftp> " send -- "exit\r" expect eof
saito - 2025-07-03
Thank you very much, Jeff Smith. I appreciate it. I don't see how the generated script is different from what I have, apart from skip-matching over the non-essential verbiage. Even the exact match version from your output hangs after the password. So I have come to suspect my expect version. It is on Windows and it is just too old at v5.43.2. I will try to locate a newer Windows version.
TWu - 2025-07-04 06:50:00
The documentation state the following two pieces, which may be helpful (I hope):
"exp_continue [-continue_timer]
The command exp_continue allows expect itself to continue executing rather than returning as it normally would. By default exp_continue resets the timeout timer. The -continue_timer flag prevents the timer from being restarted. See expect for more information."
and
"exp_internal [-f file] [-info] value
This causes further commands to send diagnostic information internal to Expect to stderr if value is nonzero. This output is disabled if value is 0. The diagnostic information includes every character received, and every attempt made to match the current output against the patterns.
If the optional file is supplied, all normal and debugging output is written to that file (regardless of the value of value). Any previous diagnostic output file is closed.
The -info flag causes exp_internal to return a description of the most recent non-info arguments given."
Jeff Smith 2025-07-04 : Check out the wiki page Expect for Windows. It doesn’t look like people have had much success with it.
Thinking out loud, maybe you could get the Expect script working under Windows Subsystem for Linux (WSL), usually Ubuntu. Then I think you can exec from a Windows Tcl script but run the Expect script in WSL by using
exec wsl expect-script.exp
The above is untested!
saito - 2025-07-05
Thank you Jeff Smith and TWu for the nice feedback and encouragement. I appreciate it.
So remembering that Activestate had it at one time, I looked into it. From the info on their new builds and their community forums, I was able to glean the following key points:
So Expect seems to be a no go, unfortunately. The second point above nicely describes the challenges explained in this page. Expect simply doesn't have the handle to the spawned process. Without that handle, no communication takes place and Expect times out regardless of what patterns/commands you specify.