How to make a Tcl application

Arjen Markus (27 november 2003) This page is intended to hold (the draft version of) a Tcl user/programmer guide. I feel something like that is needed, as it will concentrate information that is now scattered all over the place in one single document - information that is needed by unexperienced programmers and is tough to find, due to the scattering.


TV Sound more like mentioned below: example use of tcl for common computer use tools. The idea of the find is interesting because, like under the 30 year old unix find, you quickly get ideas of how to use the results to fire another command with. Valid example for the purpose.

But as for tcl/tk, I don't understand the fuzz about valid examples: I learned both Tcl and Tk from the manual page and by excellent enough enough examples from the original writer of the language, prof at berkeley at the time I seem to remember, J. Ousterhout, which I feel are still completely fine for the purpose, though maybe lacking a youth-appealing flashyness factor of current day computer power. I can start them from a windows menu just fine as an application, though I agree one must know a bit about argument passing and starting a process.


Idea for a tutorial/user guide

  • Use the script to find files
  • Describe in detail how to do it: why use procedures?
  • Describe options to implement it:
       - by running the command on each file in turn
       - by having it return a list of files
  • Describe in detail how to deploy the script!
  • Second example: polynomials (ah maths!)
      - Similar set-up

Subjects/aspects - just a reminder:

  • GUI for finding files
  • #! magic
  • associations under Windows
  • tclkit and console
  • MacOS X: how to work on that platform?
  • difference between application and library - info script trick
  • Start menu under Windows
  • Typical installation under UNIX
  • Environment variables
  • Version numbers: Tcl, own application
  • package require Tk
  • version number and patchlevel Tcl/Tk
  • wrapping in general

How to make a Tcl application

1. Introduction

Books on programming languages generally focus on the syntax and semantics of the language. They seldom explain in detail how to make an actual application, that is, a program or set of programs that you can give to a user and that he or she can start using then. Partly the reason is that many programmers already have some knowledge of how to do that (but how did they learn that in the first place?). Partly, especially for compiled languages (C, Fortran, Java, ...), the reason is that this is very platform-dependent. You can consult the manual for your compiler to find out what the command is for compiling a single source file, what options it supports. Such things are generally found in the programmer's guide.

The situation is different for an interpreted language like Tcl. As there is no compiler in the sense of a separate program that you need to run, the source code for your Tcl program is all that is needed: this is the application.

Well, if that were the full story, this guide could stop here. Of course it is not as simple as that. So you give your code to the unexpecting user. What then? How is he or she supposed to know how to start it? Or for that matter, if you know the basics of the language, but you do not have much (or any) experience with it, how do you as a programmer start?

This guide intends to help out: I will describe a few small applications from the beginning (the idea) to the end (the actual, deliverable application). By describing them in detail, I hope you will learn the following:

  • Basic programming skills in Tcl (if you do not have them already)
  • How to design a (small) application
  • How to organise your source code (use of main code, procedures, what to do with several source files, how script libraries work)
  • How to create a deliverable application (for various platforms)

2. Experimenting via the "shells"

Tcl/Tk comes with two standard interpreter programs, tclsh and wish and though you can make your own programs using the Tcl/Tk libraries, we will concentrate on these two:

  • tclsh is a program that executes Tcl scripts and can be used interactively to run Tcl commands
  • wish is a very similar program but it offers the Tk package right from the start. When run interactively, it sets up the main window for you, called "."

With a term from UNIX these programs are called "shells". So from now on we will use that term, whenever convenient.

(Note: while it is fairly easy to make your own Tcl/Tk shell or to make a program that uses the Tcl/Tk libraries, this is not covered in this user guide. We refer to TIP #66 and ... for more details)

If you just start tclsh, you get a so-called prompt (% by default). You can type any commands you like at the prompt:

   % puts "Hello, world!"
   Hello, world!

These commands are executed and the result is printed. This is a great environment for testing commands that you are not certain about. Say, you want to experiment with [string map] (which can replace "substrings" by other strings in a larger string - easier to use than the regular expressions):

   % # Convert the lowercase vowels into the uppercase equivalents
   % set line "abcdefg"
   % string map {a A e E i I o O u U} $line
   AbcdEfg

If you want to start your scripts (in an interactive environment), use the [source] command:

   % source myscript.tcl

and the script will be loaded and run.

A more direct way is to give the name of the script file as a command-line argument (for Windows users: open a DOS-box first):

Under Windows:

   d:\> tclsh myscript.tcl

Under UNIX/Linux:

   /users/me> tclsh myscript.tcl

Under MacOSX:

   ????

An alternative to the standard shells that is well worth getting acquainted with, is ''tkcon". This is a script that can help with package management, debugging and so on (see ...)

3. Finding files

The first program we are going to write (or perhaps develop is a better term, as we are going to write several versions, from simple to sophisticated), deals with the following question:

Given a directory, find all the files in that directory and in any directories below that, that fulfill some condition

(the condition could be: the file name ends in ".tcl", the file is older than 10 days, the file is larger than 100 kB, etc.)

Further requirements:

  • It should run under Windows, UNIX/Linux, MacOSX (well, all common platforms supported by Tcl)
  • The condition must be given via the command-line (as one of a set of possibilities)
  • We do not need a GUI (yet), we just want a list of names printed or actions performed
  • It should be possible to just say:
   findfiles "*.tcl" older 10

     (that is: it can be run as any ordinary system command, dir or ls for example)

What Tcl commands do we need? At least:

  • glob - this gets us a list of files matching some pattern ("*.tcl" for instance)
  • file - this allows us to get all kinds of properties of the files (like the time of creation) and to manipulate file names, should that be necessary
  • cd - this allows us to change directory (we can "descend" the directory tree)

(and then of course things like if and puts, but these are so basic, they belong to the language. Strictly speaking they are not a part of the definition of Tcl! But let us not get too side-tracked here)

A very first version:

  • We use two procedures, listFiles and handleDirs
  • The listFiles procedure simply lists all the files that match the pattern
  • The handleDirs procedure descends the directory tree, so we can inspect all directories underneath the current

Here is the code:

   proc listFiles { pattern } { 
      #
      # Go through the list of files - but make sure
      # via -nocomplain that the glob comand will accept 
      # an empty list
      #
      foreach fname [glob -nocomplain $pattern] {
         if { [file isfile $fname] } { 
            puts $fname
         }
      }
   }

   proc handleDirs { pattern } { 
      #
      # First handle the regular files
      #
      listFiles $pattern

      #
      # Now go through the list of files and directories:
      # If we encounter a directory, change the current 
      # directory and call this procedure recursively
      #
      foreach dname [glob -nocomplain *] {
         if { [file isdirectory $dname] } { 
            puts "Directory: $dname"
            cd $dname
            handleDirs $pattern
            cd ..
         }
      }
   }

   #
   # Get the program going ...
   #
   handleDirs "*.tcl"

There you go: a recursive directory listing. Note how we carefully "bracket" the call to handleDirs within change-directory commands. This ensures that we will return to the original directory when all directories have been sought. Such a depth-first search is often the easiest way to search a tree.

Notes on glob: The [glob] command will complain with an error if there are no files or directories that match the pattern. This can be annoying at times - so it is probably wise to use the flag "-nocomplain" all the time. On the other hand, I have had applications that switched between directories a lot and then the default behaviour of [glob] can be useful as a warning mechanism.

Recent versions of Tcl, at least from Tcl 8.4 on, offer the possibility to preselect the type of files to return: a regular file or a directory. This makes the checks via [file isfile] or [file isdirectory] unnecessary. The above code, however, is useable in older versions too.

So far, so good. We can list the files whose names patch a pattern. But how to select files that are older than 10 days? We will interpret this as meaning: files that have not been modified in the last ten days (as UNIX/Linux file systems as well as the modern versions of Windows keep track of the creation date, the date of last modification and the date of last access, we need to be a bit more precise).

The command [file mtime] returns the date and time we need: when the file was last modified. If that date/time is more than ten days ago, the file matches our criteria:

   if { [file mtime $fname] < $ten_days_ago } { 
      ...
   }

We could naively build this criterium into the code above, as an extra condition for the if-statement but that means we need to change the program each time we come up with a different criterium.

We can, however, extend the argument list to listFiles and handleDirs, one being the name of the procedure to run for a particular file, the other the number of days:

   proc listFiles { pattern matchProc days } { 
      ...
   }

It would be even nicer to allow an arbitrary number of arguments - then we do not have to worry about criteria like "older than 10 days but newer than 100" (unusual but not impossible). In Tcl you can do that in at least three completely different ways:

  • Store all arguments for the matching procedure in a list and pass that along.
  • The last argument can have the special name args. args absorbs any argument left over from the others:
   proc listFiles { pattern matchProc args } {
      #
      # Check the extra arguments ...
      puts "Extra: $args"
      ...
   }

   listFiles "*.tcl" mymatch 1 2 3 4

   ===> Extra: 1 2 3 4
  • By using the interp alias command you can hide fixed arguments:
   proc matchInterval {minage maxage filename} {
      #
      # Check the hidden arguments ...
      puts "Hidden: $minage maxage"
      ...
      ...
   }

   interp alias {} mymatch {} matchInterval 10 100

   listFiles *.tcl mymatch

   ===> Hidden: 10 100

The first method is perhaps more straightforward: we want a library of matching procedures that we or our customer can easily extend. So, the less details regarding the matching procedures, the better. (The second method looks more flexible, but because we need to pass the list in args around from procedure to procedure, we need to take care it does not turn into a list of lists ... By starting with a list from the beginning, there is no need to worry).

Now let us design a protocol for our matching procedures. All such procedures will be called as if they had the definition:

   proc matchProc {fname arglist} {
      ...
   }

It is important to realise that we do not have a "compile-time" protocol here - we simply guarantee that the procedures will be called this way: the first argument is the file name, any others are grouped in a list.

We explicitly will handle the last argument as a list, even if our procedure needs only one argument - if the user accidentally uses two or more, a run-time error would occur.

Here are a few examples (86400 is the number of seconds in a day, 1024 the number of bytes in a kilobyte):

   proc older {fname arglist} {
      set mtime   [file mtime $fname]
      set now     [clock seconds]
      set seconds [expr {[lindex $arglist 0] * 86400}]
      expr { ($mtime+$seconds) < $now }
   }

   proc larger {fname arglist} {
      set size     [file size $fname]
      set minbytes [expr {[lindex $arglist 0] * 1024}]
      expr { $size > $minbytes }
   }

   proc older_larger {fname arglist} {
      expr {[older  $fname [lindex $arglist 0]] &&
            [larger $fname [lindex $arglist 1]]}
   }

It is convenient to store these procedures in a different file than the source code for the general procedures: that way we can not accidentally change the general code when we add a new matching procedure or update an existing one. And it is less confusing.

All this does mean our original code should be changed:

  • The procedures handleDirs and listFiles must accept the name of a matching procedure and its arguments.
  • That name and the arguments come from the command-line (see our requirements).
  • We need to source the library of matching procedures

For the latter we assume they are stored in a file "matchproc.tcl" in the same directory as the main code. So, here is an implmentation of all the above:

   proc listFiles { pattern matchProc arglist } {
      #
      # Go through the list of files - but make sure
      # via -nocomplain that the glob comand will accept
      # an empty list
      #
      foreach fname [glob -nocomplain $pattern] {
         if { [file isfile $fname] } {
            if { [$matchProc $fname $arglist] } {
               puts $fname
            }
         }
      }
   }

   proc handleDirs { pattern matchProc arglist } {
      #
      # First handle the regular files
      #
      listFiles $pattern $matchProc $arglist

      #
      # Now go through the list of files and directories:
      # If we encounter a directory, change the current
      # directory and call this procedure recursively
      #
      foreach dname [glob -nocomplain *] {
         if { [file isdirectory $dname] } {
            puts "Directory: $dname"
            cd $dname
            handleDirs $pattern $matchProc $arglist
            cd ..
         }
      }
   }

   # default matching procedure - always 1
   #
   proc _defaultMatch_ {fname arglist} {
      return 1
   }

   # main code --
   #    Make it all happen
   #
   #
   # Source the library of matching procedures - we use
   # [info script] to get the directory where this script lives
   #
   set dirname [file dirname [info script]]
   source [file join $dirname "matchproc.tcl"]

   #
   # Get the program going:
   # - The first command-line argument is the pattern
   # - The second is the matching procedure (if none, then use the
   #   default)
   # - The rest is any arguments needed
   #
   # To get them, pick the global variable argv apart ...
   #
   if { [llength $argv] < 1 } {
      set pattern "*"
   } else {
      set pattern [lindex $argv 0]
   }
   if { [llength $argv] < 2 } {
      set matchProc _defaultMatch_
   } else {
      set matchProc [lindex $argv 1]
   }
   set arglist [lrange $argv 2 end]

   handleDirs $pattern $matchProc $arglist

You can call it like this:

   % tclsh findfiles.tcl "*.txt" older 100

and get the list of all files that are older than 100 days.

The above script is fairly straightforward and as such, not completely error-proof:

  • There are no measures against a wrong name of the matching procedure or a wrong number of arguments for it
  • There is no protection against errors accessing files
  • There is no safeguard against infinite loops that may arise due to circular links (directories or files that are only references to other directories or files)

All such problems can be taken care of and in fact have been taken care of: in the extensive library of Tcl utilities, called Tcllib (see also chapter 9). The utilities that search through a directory structure in much the same way as we just described can be found in the fileutil module of that library.

4. Adding a GUI

5. A library example

See: How to make a Tcl application - part two

6. A library in object-oriented style

See: How to make a Tcl application - part two

7. Distributing your application

See: How to make a Tcl application - part four

8. Platform issues

See: How to make a Tcl application - part three

9. How not to reinvent the wheel

P.M.

References


Bob Clark - cover some of the tcl library please, and for readers expecting object orientation (presumably most of them), a treatment of snit (and/or others) would be very useful. At least point out libraries like http and ftp as a wealth of wheels that don't need reinvention.

AM Noted!


IDGNov28/03 - Introduce tkcon? It's so much nicer to use than wish console.

---

ECS 2003-11-28 - As an alternative to glob, file and cd use fileutil::find from Tcllib.

AM Surely I will need to stress that there are many useful utilities out there. But people will need to see how to make such utilities too. This just seemed a familiar sort of problem :)


[ Category Tutorial