Most programmers don't know what Expect can do for them

This is the theme of "Expect exceeds expectations" (see the (archived) developerworks article at: http://web.archive.org/web/20070820001408/http://www.ibm.com/developerworks/library/l-sc1/?n-l-4112 , formerly here: [L1 ]): even most active Expect programmers are aware of only a fraction of its capabilities.

First, they don't realize that while writing Expect scripts, they have access to all of Tcl, and therefore can do everything Tcl can. Moreover, they're not aware that Tcl is a quite capable general-purpose programming language.

They also are not aware of the relationship between Tk and Expect, so don't realize how easy it is to add a GUI to existing Expect scripts.

Expect extends Tcl in three principal ways:

  • with pseudo-terminal (pty) management
  • with debugging facilities
  • with a command language for "challenge"-response dialogues.

In broad terms, few programmers realize the elegance of integration of pty management into Tcl's framework of event-oriented channels. [Much more explanation.]


interact is one of Expect's wonders.


RJ: An example of what Expect can do for programmers is multixterm, which comes with the package. Using multixterm, a programmer can test portability, by simply porting the application, logging into each server simultaneously using multixterm, then running the test app from the provided "broadcast" window. Anything typed into this window is sent to all open windows (one for each server in this case). If it fails on the ported-to server(s), it's simply a matter of comparing the environments of the working and not working machines.

At my company, I have adapted multixterm to allow simultaneous management and maintenance on routers and other network devices in our network. This has since been added to an application used in our Network Operations Center that provides automated login to these devices without the need for storing account ids and passwords in the clear in files or scripts. Passwords are stored in a secure database, accessed via a secure website, downloaded using ssl to the application and encrypted using DES into the user's subdirectory, and decrypted as needed by the application. Passwords can then be changed globally, routinely, without notification to users. In a global NOC, some devices require access from regional servers to pass Access List restrictions. This is also handled by the application. All of the automated logins are performed using Expect commands. All the administrivia (e.g., password handling, application flow) is done with Tcl.


clemp 2017: As a newbie, the title looked interesting since I'm one of the people who don't know what Expect can do for me. I was hoping to find out by reading this page but, unfortunately, I still don't know. Is Expect still useful in a world of GUI programs as opposed to command line programs?

APN Primarily working on Windows myself, I can understand thinking the world has moved to GUI programs :-) but don't think that is the case. A lot of embedded systems like industrial controllers, network devices etc. still run off a command line terminal-based interface. This is where I *think* (don't use Expect myself) Expect shines and is widely used.

 Can Expect be called a superset or extension of Tcl? What does that mean?

It was asserted in the article and an earlier version of this page that "Expect is a proper superset of Tcl", which caused some confusion. Some editing for length, clarification, spelling, etc has been applied to the discussion.

See also https://en.wikipedia.org/wiki/Subset , which defines both subset and superset.

---

Expect is a superset of Tcl? That's shocking news to me. Then how come we program in Tcl instead of Expect? Such play on words can have a mind-boggling effect on newbies.

Luciano, can we do this in clt? I don't understand whether you have an other-than-rhetorical question.

No, my dear IP number gumshoe. The wording has confused me indeed. I always thought that Expect was a Tcl application, or a Tcl subset in other words. Just tell us what "superset" means here, edit (delete) my clueless questioning and we're good. ;-)


Let's say that C has a set of features {structs, int, functions, pointers, arrays, recursion}. We can say that C++ have a larger set of features {structs, int, functions, pointers, recursion, classes, references, templates} or, that C++ has the features {{features of C}, classes, templates, references} which will make C's set of features a subset of C++ set of features or that C is a subset of C++ with regards to features.

In the same way we can say Tcl is a subset of XOTcl, or snit, or Expect, etc... that XOTcl offers Tcl and more.


RJ 2004-03-31: Allow me to suggest a compromise. In most of the Expect documentation I have, (e.g., BOOK Exploring Expect) Expect is referred to as an extension of TCL. It was written by Don Libes at NIST - in TCL - after John Ousterhout announced TCL. It has since been adapted as an extension to other languages.


CL is, as often, insensitive to all the emotional content here. For him, there's no cant or artifice with "superset" [L2 ]; mathematicians have used it routinely for at least the last century.

And there is a point to its employment here. It appears that hundreds of thousands of systems administrators are aware of Expect as an administrative tool or utility. A minority of that number appear to recognize that Expect is based on Tcl. A very small portion of the total understand that Expect is a proper extension of Tcl, that is, Expect does all the things Tcl does, and it also does more. I run into many, MANY Expect developers who sincerely don't realize that Expect can interpret individual Tcl commands, let alone whole scripts. My choice of "superset" is an attempt to cast about for different words, ones that might get through to naive audiences.


I agree, Cameron. Superset, subset, extension. To me it's all one language. And I'm still constantly amazed at what it can do.


So this business of superset vs extension all has to do with history. In the beginning there was Expect, it was a souped up Tcl interpreter. Therefore it was a supserset of tclsh. Don't know the history of when expectTk came into being, but it was a souped up Tk interpreter... and therefore a superset of wish. Somewhat later Tcl added package management. Sometime around then, the functionality (commands) of Expect were separated out into a loadable package, as was Tk, so starting with a tclsh you could:

package require Tk
package require Expect

and get the same effect as running expectTk. So now you can consider Expect a package. So everyone's right except those who said Expect was a subset of Tcl. The least it ever was was a superset of a snapshot in time of tclsh.

 Usability notice regarding developerWorks article discussion forum

Usability notice: The referenced developerWorks article contains what appear to be hyperlinks to a discussion forum. They are not hyperlinks. They are javascript directives. So, turn on your javascript, and forget about any "open in a new tab" option you may be used to when something looks like a link.

Usability followup: Once you navigate the maze above and actually reach the discussion forum, it is completely unusable. Do not waste your time. Just read the article, then go home.