The goal of this document is to teach you how to extend Tcl.  It covers the various methods of doing this, and the pros and cons of each method.

----

Please note that after appropriate parts of this document are
finalized, they should be incoroprated into TIP 66.
http://www.tcl.tk/cgi-bin/tct/tip/66.html

----
'''Reasons for extending Tcl''':
   * to expose an [API] function to Tcl
   * to increase the speed of a CPU intensive calculation

'''Methods of extending Tcl''':
   * [exec]utable module(s) (EM) -- a separate program controlled by the Tcl program.
   * Tcl C API
   * Tcl C API using C++
   * Tcl C API using [Objective C]
   * Tcl C API using [FORTRAN]
   * [Critcl]
   * [cpptcl]
   * [Hush]
   * [Itcl++]
   * [Joy]
   * [jWrap]
   * [Mktclapp]
   * [Object Tcl]
   * [SWIG]
   * [Tcl++]
   * [tclbind]
   * [Tclblend]
   * [tclobj]
   * [TclObjectCommand]
   * [tcl_object]
   * [xWizard]
   * Writing a [TIP] to modify Tcl itself
   
'''Overview of the methods''':
With executable modules (EM), exposure of the Tcl C [API] is not needed.  Building an executable module is generally easier than writing a C extension for Tcl.  The Tcl [exec] command can be used to start a module and send it initial data.  For most EM however the [open] command is more useful, because you can use it to create a two-way pipeline.  Debugging is easier with an EM, because Tcl isn't involved in the function of the module.  It's also reasonable to assume that an EM works with most scripting languages, and is thus not dependent on Tcl.

The Tcl C [API] is powerful and for something like a new data type for a tree, threads, or other things that can not be easily done with executable modules it's a good solution.

[Critcl] is a newcomer to the world of Tcl.  It compiles an extension on the fly based on C code embedded in a Tcl script.  It can't be used for all extension work, but for many things it's good.
[[todo findout more about how well it works]]

A variation on the methods employed by [Critcl] allows Fortran programmers to write extensions for Tcl - it is called Critclf (not very imaginative :), I admit) and is currently under development. Contact [AM] for more information.

[Mktclapp] has an [API] for writing extensions.  It can be used to generate a stand-alone application by generating C code, or a load'able shared object library.  It can also be used to convert a Tcl script or scripts into C code for compilation with a C compiler, so its use is beyond just extensions.

[SWIG] automates the task of generating an extension.  It can generate an interface for Tcl and other languages, which can then be compiled. [[How well does it work?  What is it good at doing?]

[xWizard] can generate template C/C++ code for Tcl extension. It is a GUI program written by pure Tcl/Tk so it works for cross-platform. More details in http://www.neatware.com/myrmecox/professional/wizard.html

----
'''Executable Modules (EM)'''

An EM is one of the easiest methods for extending Tcl.  It's possible to build an EM from other Tcl scripts, Perl scripts 
[[but why would anyone want to do that?]], 
or some other language 
[[surely this third option is the most common?]], 
but sometimes a program usable as an EM already exists and then one doesn't even have to create it. 
An EM typically communicates with a Tcl pipe created with [open].  Tcl supports binary data over two-way pipes, via [fconfigure] and -translation binary, so theoretically any data can be transferred.  It's also possible to use shared memory.

'''Pro:''' 
   1. Any language may be used for an EM.
   2. Debugging may be easier, because programs have an extra memory boundary.
   3. Testing may be easier, because the program can be tested alone (interactively in many cases).

'''Con:'''
   1. Performance may be an issue if lots of I/O is required.
   2. Shared state can be a problem.  A possible solution is shared memory.
   3. Complex/structured data can be a problem. Tcl handles serializing very well, but other languages are often weak in this area.

 /*
  * By George Peter Staplin
  * This is version 2 of a simple executable-module written in C.
  * This is simply a demonstration.
  */
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
 #include <errno.h>
 #include <limits.h>
 
 #define SEPFMT " \t\n"
 
 void get_input (char *buf, size_t s) {
  if (NULL == fgets (buf, s, stdin)) {
   if (feof (stdin)) {
    exit (EXIT_SUCCESS);
   } 
   exit (EXIT_FAILURE);
  }
 }
 
 int expect_int (void) {
  char *t = strtok (NULL, SEPFMT);
  char *end;
  long l;
 
  if (NULL == t) {
   fprintf (stderr, "premature end of token stream.\n");
   exit (EXIT_FAILURE);
  }
 
  l = strtol (t, &end, 10);
 
  if ((LONG_MIN == l && ERANGE == errno)
    || (end < (t + (strlen (t) - 1)))) {
   fprintf (stderr, "invalid integer '%s'\n", t);
   exit (EXIT_FAILURE);
  }
 
  return (int) l;
 }
 
 int main (int argc, char *argv[]) {
  char buf[1024];
  char *tok;
 
 again:
  get_input (buf, sizeof (buf));
 
  tok = strtok (buf, SEPFMT);
 
  if (NULL == tok || 1 != strlen (tok)) {
   fprintf (stderr, "invalid operator: '%s'\n", tok);
   exit (EXIT_FAILURE);
  }
 
  switch (tok[0]) {
   case '+':
    printf ("%d\n", expect_int () + expect_int ());
   break;
  
   case '-':
   {
    int a, b;
    a = expect_int ();
    b = expect_int ();
    printf ("%d\n", a - b);
   }
   break;
 
   default:
    fprintf (stderr, "unknown operator: '%c'\n", tok[0]);
    exit (EXIT_FAILURE);
  }
  goto again;
 
  return EXIT_FAILURE;
 }
   
To use the code above as a module we will use a two-way pipe.  Tcl uses the | character in the [open] command to create a pipe.  We will [open] the pipe for '''w+''' also known as read-and-write mode.  The [gets] command is used to retrieve the result.  The read command could also be used, although it would require a fixed-length format, non-blocking I/O, or a read of a single character until \n is reached.

  #!/bin/tclsh8.4
  
  set ::mathModule [open {|./modmath} w+]
    
  proc modMath {args} {
   puts -nonewline "$args is "
   puts $::mathModule $args
   flush $::mathModule
   return [gets $::mathModule]
  }
  
  proc main {} {
   puts [modMath + 200 300]
   puts [modMath - 5600 1243]
   puts Done
  }
  main

----
'''Tcl C API'''

The Tcl C API is quite powerful.  There are at least 2 approaches to extending Tcl via the C API.  A new tclsh-like shell can be created, or a loadable extension can be [load]ed into an interpreter.

Our first example will be a new interactive shell that performs simple addition.  This will teach you how to create a Tcl interpreter, initialize Tcl, and create a simple command for addition.  

'''Pro:'''
    1. The C API is powerful and some things can be done that are impossible to do with an EM.
    2. The C API can be used as a portable abstract layer for I/O, and other uses.
    3. Performance is generally excellent -- provided that you choose the right algorithms.

'''Con:'''
    1. [Tcl_Obj] management can be tricky for beginners. Bad Tcl_Obj reference counting practice can result in crashes in other parts of Tcl that can be difficult to diagnose.
    2. C is the only language binding for the Tcl API (although "C" in this case typically includes derivatives such as [C++]), unless you count some tools that use assembly language or things like [ffidl].
    3. The C API may change overtime, and tends to change more than the Tcl script-level interface.
    4. Depending on what part of the API you use or require (for an extension library), you may find that Tclkit doesn't work, because the tclInt.h functions used are not in the stubs table.


  #include <stdio.h>
  #include <stdlib.h>
  #include <tcl.h>
  
  int AddObjCmd (ClientData clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[]) {
    long n1;
    long n2;
    Tcl_Obj *res;
  
    /*The command name is objv[0] so 3 arguments are expected.*/
    if (3 != objc) {
      Tcl_WrongNumArgs (interp, 1, objv, "n1 n2");
      return TCL_ERROR;
    }
  
    if (TCL_OK != Tcl_GetLongFromObj (interp, objv[1], &n1)) {
      /*The error result should be set by Tcl_GetLongFromObj.*/
      return TCL_ERROR;
    }
  
    if (TCL_OK != Tcl_GetLongFromObj (interp, objv[2], &n2)) {
      return TCL_ERROR;
    }
    
    res = Tcl_NewLongObj (n1 + n2);
  
    Tcl_SetObjResult (interp, res);
  
    return TCL_OK;
  }
  
  int main (int argc, char *argv[]) {
    Tcl_Interp *interp;
  
  
    /*
     * This finds Tcl's library files and performs some initialization.
     */
    Tcl_FindExecutable (argv[0]);
  
    interp = Tcl_CreateInterp ();
   
    if (TCL_OK != Tcl_Init (interp)) {
      fprintf (stderr, "Tcl_Init error: %s\n", Tcl_GetStringResult (interp));
      exit (EXIT_FAILURE);
  
    }
  
    Tcl_CreateObjCommand (interp, "+", AddObjCmd, (ClientData) NULL, (Tcl_CmdDeleteProc *) NULL);
  
    while (1) {
      char cmd[1024];
    
      fgets (cmd, sizeof (cmd), stdin);
  
      if (TCL_OK != Tcl_Eval (interp, cmd)) {
	fprintf (stderr, "error: %s\n", Tcl_GetStringResult (interp));
	continue;
      }
  
      printf ("result is: %s\n", Tcl_GetStringResult (interp));
    }  
  
    return EXIT_SUCCESS;  
  }

Links:
   1. [SampleExtension] - the ''official'' example
   1. [BOOK Practical Programming in Tcl and Tk, Third edition] - Chapter 44 on the Tcl C API is available on-line [http://www.beedub.com/book/3rd/bookTOC.html].
   1. http://www.cs.man.ac.uk/%7Efellowsd/tcl/funcproc.c
   1. ftp://www.tcl.tk/pub/tcl/misc/example.tar.gz
   1. http://www.iki.fi/mjk/files/test.c.txt

----
[[explain how to build a load'able extension]
[[explain how to build an export list for a DLL]
[[explain how to link using MSVC++]
----
'''Tcl C API using C++'''

Using C++ with Tcl is similar to using it with C.  There are a few tricks to using methods within a class.  Within the class we create a wrapper which calls a static function within the class.

  #include <stdio.h>
  #include <stdlib.h>
  
  /*This is specifically for C++.*/
  extern "C" {
    #include <tcl.h>
  }
  
  /*Thanks to Kevin Kenny for his help with this.*/
  
  class Math {
    public:
    static int AddObjCmd (ClientData clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[]) {
      return (reinterpret_cast<Math*>(clientData))->AddMethod (interp, objc, objv);
    }
    int AddMethod (Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[]);
  };
  
  int Math::AddMethod (Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[]) {
    long n1;
    long n2;
    Tcl_Obj *res;
  
    /*The command name is objv[0] so 3 arguments are expected.*/
    if (3 != objc) {
      Tcl_WrongNumArgs (interp, 1, objv, "n1 n2");
      return TCL_ERROR;
    }
  
    if (TCL_OK != Tcl_GetLongFromObj (interp, objv[1], &n1)) {
      /*The error result should be set by Tcl_GetLongFromObj.*/
      return TCL_ERROR;
    }
  
    if (TCL_OK != Tcl_GetLongFromObj (interp, objv[2], &n2)) {
      return TCL_ERROR;
    }
    
    res = Tcl_NewLongObj (n1 + n2);
  
    Tcl_SetObjResult (interp, res);
  
    return TCL_OK;
  }
  
  int main (int argc, char *argv[]) {
    Tcl_Interp *interp;
  
  
    /*This finds Tcl's library files and performs some initialization.*/
    Tcl_FindExecutable (argv[0]);
  
    interp = Tcl_CreateInterp ();
  
  
    if (TCL_OK != Tcl_Init (interp)) {
      fprintf (stderr, "Tcl_Init error: %s\n", Tcl_GetStringResult (interp));
      exit (EXIT_FAILURE);
    }
  
    Math *inst = new Math;
  
    Tcl_CreateObjCommand (interp, "+", inst->AddObjCmd, (ClientData) inst, (Tcl_CmdDeleteProc *) NULL);
  
    while (1) {
      char cmd[1024];
    
      fgets (cmd, sizeof (cmd), stdin);
  
      if (TCL_OK != Tcl_Eval (interp, cmd)) {
	fprintf (stderr, "error: %s\n", Tcl_GetStringResult (interp));
	continue;
      }
  
      printf ("result is: %s\n", Tcl_GetStringResult (interp));
    }  
  
    return EXIT_SUCCESS;  
  }
  

[[perhaps someone more knowledgeable about C++ could improve this]]

[SLB] I'll just comment that the 'tricks' are dangerous. Compiling the above code with
Sun's compiler reports:

  "tclmain.cpp", line 61: Warning (Anachronism): Formal argument proc of type extern "C" int(*)(void*,Tcl_Interp*,int,Tcl_Obj*const*)
  in call to Tcl_CreateObjCommand(Tcl_Interp*, const char*, extern "C" int(*)(void*,Tcl_Interp*,int,Tcl_Obj*const*), void*,
   extern "C" void(*)(void*)) is being passed int(*)(void*,Tcl_Interp*,int,Tcl_Obj*const*).

I don't have a reference to the relavent clause in the C++ standard to hand but warnings about portability problems from
treating C and C++ function pointers as interchangeable are widespread.
See for example [http://mail.gnome.org/archives/gtk-list/1999-December/msg00424.html]

Links:
   1. ftp://ftp.forwiss.uni-passau.de/pub/Os/Unix/languages/tcl/extensions/tkmin/  [[Does this URL work, just very slowly, or is it no longer valid?]]
   1. http://www.uni-frankfurt.de/%7Efp/Tcl/tcl-c++.txt [[This URL doesn't appear to be working]]
   1. http://prdownloads.sourceforge.net/incrtcl/sampleItclExtension1.1.zip?download


----
'''Tcl C API using Objective-C'''

To expose an Objective-C class to Tcl we can create a wrapper in much the same was as was done with C++ earlier.  

Compile this code using something like:
gcc tcl_objc.m -lobjc -I/usr/pkg/include -L/usr/pkg/lib -ltcl84 -lm -lpthread -Wl,-rpath -Wl,/usr/pkg/lib


'''tcl_objc.h:'''
 /*
  * By George Peter Staplin
  * This is version 2, and much better than the previous code I put here...
  */
 #include <stdio.h>
 #include <stdlib.h>
 #include <tcl.h>
 #include <objc/Object.h>
 
 
 @interface Adder : Object {
  /* This is just a pointless instance variable. */
  int lastresult; 
 }
 - (int) add:(Tcl_Interp *)interp x:(Tcl_Obj *)x  y:(Tcl_Obj *)y;
 @end
 int AdderWrapper (ClientData, Tcl_Interp *, int, Tcl_Obj *CONST[]);
 
'''tcl_objc.m:'''
 #include "tcl_objc.h"
 
 @implementation Adder
 
 - (int)add:(Tcl_Interp *)interp x:(Tcl_Obj *)x y:(Tcl_Obj *)y {
  int a, b;
 
  if (TCL_OK != Tcl_GetIntFromObj (interp, x, &a))
   return TCL_ERROR;
  
  if (TCL_OK != Tcl_GetIntFromObj (interp, y, &b))
   return TCL_ERROR;
 
  Tcl_SetObjResult (interp, Tcl_NewIntObj (lastresult = a + b));
 
  return TCL_OK;
 }
 @end
 
 int AdderWrapper (
   ClientData cdata, 
   Tcl_Interp *interp,
   int objc, 
   Tcl_Obj *CONST objv[]) 
 {
  id inst = (id) cdata;
 
  if (TCL_OK != [inst add: interp x: objv[1] y: objv[2]]) {
   return TCL_ERROR;
  }
 
  return TCL_OK;
 }
 
 int main (int argc, char *argv[]) {
  Tcl_Interp *interp;
  id inst;
 
  Tcl_FindExecutable (argv[0]);
 
  interp = Tcl_CreateInterp ();
 
  if (TCL_OK != Tcl_Init (interp)) {
   fprintf (stderr, "Tcl_Init error: '%s'\n", Tcl_GetStringResult (interp));
   return EXIT_FAILURE;
  }
 
  inst = [Adder new];
 
  Tcl_CreateObjCommand (interp, "add", AdderWrapper, (ClientData) inst,
    (Tcl_CmdDeleteProc *) NULL);
 
  if (TCL_OK != Tcl_Eval (interp, "add 200 123")) {
   fprintf (stderr, "Tcl_Eval error: '%s'\n", Tcl_GetStringResult (interp));
   return EXIT_FAILURE;
  }
 
  printf ("Success: the result of add 200 123 is: %s\n", Tcl_GetStringResult (interp));
 
  return EXIT_SUCCESS;
 }
 
----
15jan03 [jcw] - Your "add" example above in Critcl:
  package require critcl
  critcl::cproc addup {long a long b} long {
    return a + b;
  }
  interp alias {} + {} addup
  puts "123 + 864 = [+ 123 864]"
If you have tclkit and the critcl starkit, do:
  $ critcl add.tcl 
  123 + 864 = 987
  $
If you want to turn this into a loadable package, use this code instead:
  package provide add 1.0
  package require critcl
  critcl::cproc addup {long a long b} long {
    return a + b;
  }
Then use the -pkg flag:
  $ critcl -pkg add.tcl 
  Source: add.tcl 
  Library: add.so
  Package: /home/jcw/lib/add
  $
The result is a self-contained auto-generated package:
  $ ls -lR lib/add/
  lib/add/:
  total 9
  drwxr-xr-x    2 jcw      users	 104 Jan 15 18:52 Linux-x86
  -rw-r--r--    1 jcw      users	1506 Jan 15 18:52 critcl.tcl
  -rw-r--r--    1 jcw      users	  64 Jan 15 18:52 pkgIndex.tcl

  lib/add/Linux-x86:
  total 12
  -rwxr-xr-x    1 jcw      users	4660 Jan 15 18:50 add.so
  -rw-r--r--    1 jcw      users	 157 Jan 15 18:52 critcl.tcl
  $

----
Somewhere, [RS] wrote some code that talks about embedding C code into your
Tcl scripts... [C code generators]

----
What about extending via mechanisms like [CORBA], remote RPC, etc.?  Are
those kinds of ways, where you are invoking code over a network, considered
''on topic'' here?

[George Peter Staplin] Any kind of [RPC] would be on topic.  On a separate page linked to from here, or within this page.  RPC is an interesting topic, and I look forward to hearing about it.  :)

[LV] [Inventory of IPC methods] covers a variety of means of communicating between two "executable modules". However, I don't personally think of that as being a method of "extending" Tcl.  "Extending Tcl", to me, means Tcl having new commands / APIs at its disposal.

----
[How to invoke a C function from Tcl]

----
!!!!!!
%| [Category Tutorial] |%
!!!!!!