Version 5 of Higher level C API

Updated 2002-09-01 12:52:59

Higher level C API

Here I present an idea I have had to improve the C API for Tcl.

The basic idea is to allow the API to work at a higher level of abstractions, like a super getops() in GNU C and then allow the information about the command to be query by other (i.e. reflected).

In most cases when writing a Tcl command in C/C++ there are four different things to be managed, the registration of the command with the interpreter, the arguments, the return value.

The return value

The return value is either normal or an error return. Most of the error returns are related to the arguments of the command and typically only one error is related to the actual operation.

Non error returns only have type to worry about. The types are int, double, buffer, and list. The buffer itself may be a string, a byte array, the name of a variable, the name of a array, name of a ?proc?, a snippet of code, or a handle of some kind.

Error returns usually have a string explaining what happened.

The arguments

Arguments have meanings, operations, and values in Tcl.

The values are typical int, double, buffer, and list. But a buffer can be a string, a byte string, name of a list, name of a array, name of a ?proc?, a snippet of code, or some other type of handle (like a file handle).

The operations on a argument may be considered open-ended, but typically are ?subst?, ?expr?, ?join?, ?split?, and ?eval?. Before a value is received from an argument sometimes one or more of these may have to be done to the argument

The last and most complicated part of an argument is its meaning. The natural way to give meaning is just by position. The C/C++ interface in Tcl was (pre-8.0) just an array of string and now (post-8.0) is an array of Tcl_Object(s) so all the arguments are accessed by an index.

Tcl has always encourage people to make as expressive commands as needed. This has made the language very expression but also very complex.

Example: regexp ?switches? exp string ?matchVar? ?subMatchVar subMatchVar ...?

To understand the possible ways commands are given procedure you have to understand idioms about how variables are given meanings, other idioms about has operations are done on arguments, and still other idioms about advance argument types.

Idioms on how arguments are given meaning

1. Position idiom

    The nth argument has meaning x.

    Variations
    1.1. Simple positional idiom
    1.2. Simple positional idiom with default arguments

2. Subcommand idiom

   The name of the command and the first argument together make up the whole name of the command to execute.

   Variations
   2.1. Complete subcommand idiom.
      The first argument is always a subcommand.
   2.2. Incomplete subcommand idiom
      The first argument is not always a subcommand.
      Example: 
      after cancel script script script ...
      after ms ?script script script ...?

3. Switch idiom.

   Has one value for existence and another for non-existence.  In Tcl always of the form ?-switchName?.

   3.1. unlimited in position switch idiom
   3.2. limited to end position switch idiom
   3.3. limited to a certain place in argument list switch idiom
   3.4. limited to a certain place in argument list and ends with ??? switch idiom.

4. Named value idiom

   After a certain key the value of the key is the next argument.  In Tcl always of the form ?-key value?.

   4.1. unlimited in position named value idiom
   4.2. limited to end position named value idiom
   4.3. limited to a certain place in argument list named value idiom
   4.4. limited to a certain place in argument list and ends with ???named value idiom.

5. Variable number of arguments idiom.

   The actual number of arguments is unlimited (or at least unspecified).

   5.1. Variable number of arguments of same meaning idiom
      Example: proc sum {args} { set ret 0 ; foreach n $args { incr ret $n } ; return n }

   5.2. Variable number of arguments each a part of an expression idiom
      Example: expr 3 + 4 

That is 14 idioms you might see. To make things worst people will usually use several of these idioms in one command so the possible combinations of idioms can be huge. This makes Tcl very expressive but complex.

Both switch and named value idioms have several other characteristics.

1) Required or not. 2) Radio button action (one of a group of options must be present). 3) Part of name only required 4) Default values

Idioms on how operations are performed on arguments.

1) Control structure idiom.

Idioms on complex values

1. Little language idiom.

   Sometime we make up little languages to compactly encode the values given to an argument.
   1.1. Examples: regular expressions, format statements, end-1?..

2. Limited values idiom

   2.1. For numbers this can be ranges or groups of values.
   2.2. For buffers (strings or byte arrays) this may be some complex structure.
   2.3. For handles this may be a particular type of handle.

Other idioms I have not seen in Tcl.

   1) Use of ??? or some other symbol instead of ?-? in switchs and name value pairs.
   2) Use of ?+? and ?-? to turn on or off switches.
   3) Parameter groups (i.e. ?@readParamsFromThisFile?).

Conclusion

So far Tcl does little to help implement any of these idioms, again and again people must code these by hand and the information then has to be recaptured in documentation.

What I suggest we do is somehow encode all this information about commands and add APIs so that it becomes very easy for people to write complex command lines in a uniformly short and easy way. This also means that the errors from wrong arguments will be much more uniform. Another feature is that people can reflect this information to help them build static syntax checkers, test routines, and give hits to the compiler on what it can optimize.

What is needed is.

   1) An API to describe commands.
   2) An API to present this information to the C implementation in an abstract way.
   3) An API that allows others to see our description and ?reflect? it.
   4) Algorithms to deal with all the possiblities.

Well I leave it up to you do you think this is worth pursuing?

Earl Johnson