Version 253 of What is Tcl

Up: Contents	Next: What is Tk

: The agility you get from having built a system that's fundamentally simple dominates all other kinds of agility ... We can make the same exact software we're making today, with dramatically simpler stuff. Dramatically simpler languages, tools, techniques, approaches. Like really radically simpler. -- Rich Hickey, Rails Conf 2012 Keynote: Simplicity Matters

What is Tcl?

Tcl, or Tool Command Language, is unique among programming languages. It provides a programming environment consisting of only three basic elements: A script is composed of commands, and a command in turn is composed of words. That's all there is to it. This abstraction is at once simple enough to get to work with in just a few minutes, even if one is new to programming, but rich enough, if one care's to delve more deeply, to satisfy even the most demanding software engineers. There is simply no other language that even comes close to catering so adeptly to users at both ends of the spectrum. This makes it the ideal language both for learning the art of programming, and for actually getting stuff done.

Tcl is an open-source dynamic programming language. For decades it has served as an essential component in systems in organizations ranging from NASA to Cisco Systems, is a must-know language in the fields of EDA, and powers companies such as FlightAware and COMPANY: F5 Networks. Along with its C library, it provides an ideal cross-platform development environment for any programming project.

Tcl is one of the only languages that is convenient for both the smallest and largest programming tasks, without ever having to ask, "Is it overkill to use Tcl here", or "If this code gets bigger, will Tcl scale?". Whenever a shell script might be used, Tcl could be used just as easily. On the other hand, entire web ecosystems and mission-critical control and testing systems are also written in Tcl. Tcl excels in all these roles due to the minimal syntax of the language, the unique programming paradigm exposed at the script level, and the careful engineering that has gone into the design of the Tcl internals.

Tcl is the least prescriptive of the scripting languages. Its syntax is described in just a dozen rules. There are no reserved words and there is no built-in syntax for control structures or conditionals. Instead, control and branching operations are, like everything else in a Tcl program, implemented as commands. For example, if is a command that in its simplest form takes a conditional expression to test, and a block of code to be executed if the condition is true. A Tcl script could replace if with its own implementation. As with Lisp, each program written in Tcl essentially becomes a domain-specific language for performing the tasks at hand. Tcl is to scripting what Forth is to stack machines : A starkly-minimal language that fits its domain like a glove.

Tcl has all the features needed to rapidly create useful programs in any field of application, on a wide variety of platforms, and supporting any character set. Whatever the platform, Tcl is probably already there.

In addition to being a programming language, Tcl is also a cross-platform C library. If you've been looking into glib , APR , or NSPR , or qtcore , you might end up using Tcl not for its scripting capability, but to provide the features you've been looking for in a truly portable C library.

Tcl is simple to learn, and fun to use.

Features

Tcl provides a wide range of features, everything necessary to write very large and comprehensive programs:

Cross-platform scripting: Tcl provides a cross-platform API and handles the details under the hood, making it easy to write a program once and run it, without modification, on Unix, OS X, Android, or Windows platforms.

Cross-platform C Tcl Portable Runtime Library: In contrast with Perl, Tcl's C library is intuitive and a pleasure to use. Many programmers choose Tcl as their scripting language, but others choose it for its C library, which abstracts away the platform-specific details of synchronous and asynchronous I/O operations, mutexes, memory allocation, string formatting, threading, Unicode, event loops and queues, event notification, dynamic strings, arbitrary precision arithmetic, hash tables, shared object loading, and regular expression. Tcl_Obj object types provide automatic type conversion, and are reference-counted for easy cleanup. Tcl's C library can be particularly advantageous to those interested in learning C, providing a fantastic platform to ease into it, particularly when combined with on-the-fly integration libraries such as Critcl.

Cross-platform, event-driven, non-blocking I/O: node.js has recently popularized this programming style, but this I/O model has been a fundamental component of the Tcl core for many years.

Text-based computing: Tcl's everything-is-a-string model offers a simple approach to programming. Tcl recognizes that a script is already text, and substrings in a script are used directly as values. The syntactical rules allow Tcl scripts to embrace code as data and assimilate the best traits of shell languages while scaling to larger projects much better. As a superior alternative to both shell scripts and other general-purpose scripting languages, Tcl is the sole occupant of a certain sweet spot. Perl tried, but came down with a morbid sigil -lness. Python didn't catch that disease, but it's too obese as an interacctive shell. Lisp came close, but its SEXP's are needlessly complex compared to Tcl lists, its special operators are incongruous compared to Tcl's consistent command syntax, and Lisp's syntactical provision for a list type departs from the text-based computing paradigm.

Explicit Scoping: A pleasant hybrid of dynamic and lexical scoping that maximizes the advantages of both scoping strategies while minimizing the downside. Procedure scope is entirely local, and variables in other scopes and namespace are explicitly linked in as needed, resulting in code that makes it clear what is being referenced.

Composability: Like other functional programming languages, Tcl is highly composable. Each command is a list, and thus can be manipulated in the language itself.

Dynamicity: Tcl is about as dynamic a language as there is. Don't want any conditionals in your language? Just delete the if, for and while commands. Want to modify the body of the current function mid-execution, or modify variables and run commands in other stack frames? Go right ahead. Using these features, you can easily add new control structure to the language. In Python it isn't possible to dynamically create variables in a local function scope, and the approach is don't do that. In Tcl it's easy to do, and the approach is, You're the expert. You know what you need to do.

Instrumentation: From trace to namespace unknown, Tcl provides a variety of built-in mechanisms to intercept both command execution and variable access. Whether you're after powerful debugging or exotic control flow, you'll find the features you need.

Data structures: Tcl provides built-in data structures such as lists, arrays, and dictionaries, and many others are also available. Internally, Tcl works with structured representations of these values, so performance is good despite their stringiness at the script level.

Control structures: Use built-in control structures such as if, while, and foreach, or write your own if it fits the bill.

TCP/IP Network programming: socket, http, SSL, TLS, PKI, CGI, and various other modules provide full network programming support.

Threading: Tcl's true native threading module is recognized as exceptionally-well-designed and easy-to-use. Contrast with Perl threads, which are officially documented as "implemented in a way that makes them easy to misuse. Few people know how to use them correctly or will be able to provide help". Also Contrast with Python, which is not fully thread-safe , Ruby which is also not particularly thread-safe , and Lua, which makes no attempt to expose operating-system threads.

Regular expressions: a hybrid DFA/NFA engine written by Henry Spencer that achieves better performance than the regular expression engines of Perl, Python, PHP, etc.,

Unicode: Support for Unicode has been fully-integrated into tcl, whether working with strings and files, or matching regular expressions.

Encoding conversions: Easily transform text from one encoding to another.

Binary data handling: binary scan and binary format make it easy to wrangle bits and work with binary data formats.

Numerical computing: Tcl provides integers of arbitrary size and floating-point numbers of the C double type.

Built-in event loop: Unlike Python and Ruby, where facilities for event-driven programming are bolted on (Twisted and EventMachine ) an event loop is built into Tcl, which is distributed with event-handling commands such as after, fileevent, vwait, and update.

Bridge the Value/Reference Divide: Because there's no such thing as a reference at the script level, Tcl is free to manage values internally. A Copy-On-Write strategy is built-in to the design of Tcl, so copying huge data structures at the script level is a memory-efficient operation. Tcl only makes real in-memory copies as necessary to ensure program data integrity in the case that one of the logical copies of the data structure is modified.

Cycle-free garbage collection: In most languages, cycle-free garbage collection is not used because, despite its advantages, it doesn't play well with references. In Tcl everything is a string and there are no explicit references at the script level, so storage can be and is collected precisely when it becomes unused in a script.

multiple interpreters: Within a single program, multiple interpreters can be created, each with its own complete interpreter state. Using multiple interpreters, programs can be structured in a manner reminiscent of obejct-oriented programming, but unique in its own right.

Safe interpreters: Make it possible to create a sandbox where users of an application or web service can run untrusted code.

Virtual file system: VFS functionality is fully integrated into Tcl's I/O subsystem. This is a unique feature among programming languages, allowing virtually anything to be expressed in terms of the world's most ubiquitous user interface: the filesystem hierarchy

Reflected channels: Apply the API for working with file streams to just about anything.

Stacked channels: Easily create "pipelines" of channels for performing transformations of channel contents.

Extensibility: In Tcl all control structures are implemented as commands, and all commands can be replaced by user-defined procedures, making Tcl perhaps the most flexible programming language out there. New commands can be written in C and loaded into a running interpreter via load

Embeddability: Tcl was designed from the start as an embeddable language for use wherever scripting was needed. It's one of the only mature languages that can be safely embedded in a multi-threaded application.

Inline C support: critcl makes it easy to rewrite performance-critical code segments in C, providing unmatched integration between the two languages.

OO: available with the core distribution as of 8.6, and via add-on packages for 8.5 and earlier

Built-in database: SQLIte was designed from the ground up to be used with tcl, and is distributed along with Tcl. SQLite is renowned for its ease-of-use and robustness, and there's nothing like using it via Tcl.

Coroutines: Tcl coroutines allow yield to be called from anywhere in the coroutine call stack. yieldto allows an arbitrary number of coroutines to pass control among themselves as desired. This is more powerful than other more limited coroutine implementations such as Python's generators.

add-on packages: There are a large number of libraries and extensions, including tcllib, graph plotting packages, database access.

GUI toolkit: Tk was designed for Tcl and works best with Tcl, It has been adopted far and wide by other languages, but there's nothing like using directly with Tcl. See What is Tk.

Unit testing: A built-in package for unit testing makes it easy to practice test-driven development. It's minimal, flexible, and eminently functional. It's also the foundation of the unit tests that are distributed with Tcl itself, so there are plenty of examples of both simple and sophisticated use.

Easy Deployment: starkits and starpacks make deploying a Tcl application as easy as downloading a single executable file.

Computer science in a box: Tcl source code is considered to be exemplary for those learning C, and it provides illustrations of many fundamental and general computer science concepts. It is meticulously written and the concepts come together in a way which is easy to experiment on from the scripting level and through the C API. If you're hardcore, you'll probably get a kick out of Tcl. If you're new to computer science, you couldn't choose a better platform to learn the ropes on.

Strengths

Domain-Specific Languages: The key thing that Tcl does better than any other is to provide the components and environment within which you can create your own Domain Specific Language suitable for the problem you are tackling. This strength will change the way you program. This isn't about lexing and parsing, but about creating little languages using the syntax of Tcl itself. It may take some time for the ramifications of this strength to sink in, but the investment will pay off a thousand-fold. If you master this feature, your code will become more expressive, more readable, and more concise. Tk, an example of this, is natural in Tcl, and not even feasible in other languages, which either include Tcl so that they can provide Tk, or provide direct and under-portable access to underlying system libraries.

Security: Tcl has security policies that let you decide what kind of operations are appropriate, including for user-provided code.

Rapid Development: Most programs are easy to write in Tcl because it works with numbers and strings and files and network connections, rather than with the bits and bytes and pointers of C, C++, and Java.

Code Reuse: Tcl's package system makes it easy to write code that can be reused. Many other people have made their code available for reuse.

Performance: Various components of Tcl have been recognized as being best-of-breed. Two examples are regular expressions and Tcl's internal hash table.

Multi-paradigm: Tcl comfortably integrates styles from various programming paradigms. The TclOO object system can be used for both class-based and prototype-based object-oriented programming. Various other object systems are also available, including stooop incr Tcl, XOTcl, snit. Functional programmers will also find much to like in Tcl. For an overview of the possibilities search for playing* .

Portable: Tcl runs on IBM mainframes and AS/400s, Windows NT, 95, 98, ME, XP, Windows/CE, OS/2, Novell Netware, Cray supercomputers, Digital's VMS, Tandem Guardian, HP MPE/ix, Mac OS, and all flavours of Unix, as well as dozens of smaller operating systems like Acorn RISCOS, Amiga and Atari ST.

Joe Mistachkin 2006-06-08: Based on my previous research (when I had access to the actual hardware), modern versions of Tcl have not been ported to HP MPE/ix. Please correct me if this is inaccurate.

Open Source: Anyone may, without charge, download and inspect the source code to Tcl. If you need to verify or certify Tcl internally, for instance for Year 2000 audits, you have immediate and full access to the internals of the language.

Internationalization and Localization support: Tcl's "mother tongue" is Unicode, so just about all living writing systems of the world can be processed, provided you have fonts. Converting Unicode from or to just about any encoding is also very simple. See i18n - writing for the world or Is Tcl Different!.

Version invariance: This one is important, but hard to explain. Tcl has been maintained in a way that promotes forward- and backward-compatibility, at all levels, and the specific Stubs technical mechanism makes it automatic that compliant extensions work across version boundaries. While explanations of these advantages make most audiences yawn, they turn out to be enormously beneficial to real-world, working programmers.

Embed: Tcl Trivially embedded

Deploy: Deployment via starkit/starpack is a big advantage

Extend: easy to extend with C/Fortran

Fun Factor: Remember that slinky you got for your birthday when you were 5? Tcl is the slinky.

Quotes

I stick with Tcl because it gets the job done faster and more maintainably than anything else I use... to quote Bryan Oakley, "I prefer to pick my tools by what they can do, not by how popular they are": Steve Landers, Tcl Chatroom, 2014-05-09

Tcl is not so much a programming language in the traditional sense. It is a set of primitives to pass data and control execution scope.: John Roll

And these things are more than just convenience. The simpler notation means that the task to be done isn't obscured by the mechanism of doing it. Since our limit on what we can program is mostly the limit of what we can understand, with the simplicity comes power. Abstraction wins.: Kevin Kenny, Tcl Chatroom, 2013-09-30

Duck typing tends to make type errors less damaging, and a good dynamic language makes test-first development more practicable.: Kevin Kenny, Tcl Chatroom, 2013-09-30

One aspect of EIAS that is worth consideration is how it has kept Tcl "pure" in some sense. Part of EIAS that is little mentioned is that Tcl's strings are immutable. This means that Tcl's value space is purely functional, in the Haskell sense. All side-effects are confined to the shadow world of commands and variables and other second-class entities. What this means is that Tcl now possesses some very powerful purely functional data-structures that are somewhat better than those available in other languages. For instance, I cannot think of another popular language that supplies O(1) purely functional dictionaries and lists (arrays) out of the box (or even in the library). Not to mention efficient unicode and binary strings: NEM, EIAS 2010-12-15

Who Uses Tcl

see Who Uses Tcl

Testimonials

"If this isn't a testimonial to Tcl's prowess, I don't know what is.": myvzw.com and Tcl

"IMHO the Tcl event loop is much easier to use than twisted, although the python twisted code has some really nice features (deferred values) and a good framework for writing specific things like network servers.": Michael Schlenker, Tcl Event Loop and Threading, comp.lang.tcl, 2007-03-26

Case Studies

Ulrich Albrecht: One more story, same implication: A couple of years ago I was working at the technical university of Aachen (Germany) on a robot simulation and path-planning system, including fast collision detection. The core functionality was included in a C++ class hierarchy describing robot parts and kinematics, with a lot of C libraries from other sources mixed in. The code was to perform on a real robot controller (a PPC-machine running vxworks), SGI Irix (MIPS), Sun Solaris (SPARC), Linux (Intel) and eventually MS Windows (Intel), so we could not include any platform specific stuff. With the exception of vxworks though - where the real thing was moving - we thought we should have a GUI with a virtualization and visualization of the robot and its environment. Tcl/Tk looked good, because it was the only package that was as portable, flexible and extensible as we needed. I wrote an OpenGL-widget for Tk and a Tcl-to-C++ command-wrapper for every C++ class I needed to construct and control at runtime, and ended up with a robotics-enabled wish I could feed all kinds of scenarios and interact with. I can really say that 1. portability was not a big issue, because the core C/C++-libraries were OS-independent and Tcl/Tk as well as OpenGL were properly ported, 2. performance wasn't either, because the critical parts were nice, dirty compiled C/C++, 3. usability was very good because of the robust interactive Tcl-interpreter we could use for free, 4. the Tk-part performed and looked a lot better and more OS-consistent than anything I've since seen in Java...

We even run this stuff in MS Internet Explorer (using the Tk-Plugin) for demonstration purposes.

Nearly forgot: Porting the code to MS Windows was not the problem - it was the makefiles that nearly drove us mad.

Heritage

See Tcl heritage

Tcl was originally developed as a reusable command language for experimental computer-aided design (CAD) tools. The interpreter was implemented as a C library that could be linked into any application. Because it is very easy to add new functions to the Tcl interpreter, it is an ideal embeddable scripting language. (See Tcl Chronology for more history.)

Tcl could be roughly described as a cross-breed between:

LISP/Scheme: but with fewer kinship,
C: but not as low-level and numerocentric,
and command line shells: with more powerful structuring and predictable handling of strange data.

Tcl is a homoiconic language whose design was motiviated by the idea that there would be as little core syntax as possible, with as many other needed facilities as possible implemented as commands. Fundamentally, Tcl is nothing but a script parser, a substitution engine, and a command dispatcher.

(See Tcl heritage for more ancestry.)

About Tcl Scripts

Tcl programs are sometimes called "scripts" because the programs do not need to be compiled into a binary form before running them, but the Tcl performs JIT compilation transparently, so Tcl does not suffer from the performance penalty associated with traditional interpreters.

You can even use other Tcl extensions like 'Expect' to control command line programs from a GUI.

Components and Subsystems

event loop

I/O

non-recursive engine (NRE): Tcl is stackless since version 8.6

Various Tcl Interpreters

For many people, their first acquaintance with Tcl is as a component of another software program. Examples include Expect, Cisco IOS.

tclsh: a stand-alone interpreter that's included with Tcl

wish: a Tcl interpreter with the Tk extension pre-loaded. It is part of Tk.

tclkit: an all-in-one-file combination of Tcl, Tk, Metakit, Itcl, tclvfs and a few other extensions. It is used for code in starkit format.

expect: a Tcl interpreter with the expect extension pre-loaded. It is created when building expect.

tcl and wishx: commands that are included with Tclx

itclsh: tclsh that pre-loads Itcl

tixwish: included with Tix

bltsh and bltwish: included with BLT

tcldomsh: included with TclDOM

xotclsh and xowish': included with XOTcl

As of Tcl 8.4, the only interpreters that you really need to think about are tclsh, tclkit, and (on some platforms) wish, as Tcl provides the ability to dynamically load any of the other extensions during runtime. In fact, many of the extensions have been dropping the building and installing of custom executables due to this.

Tcl is a weakly-typed dynamic language. In Tcl, everything is a string, but that fact just scratches the surface.

Tcl semantics allows code to be seen as data, as with Lisp (see code is data).

Discussion regarding this page

Tcl and Just in Time (JIT) ?

slebetman: Does Tcl really do JIT? When was it/will it be introduced? 8.5? 8.6?

NEM: No. Certainly not JIT in the sense of compiling to native code. It compiles to byte-code on-demand, which could be seen as "just in time", but JIT usually means native.

DKF: We've talked (usually at conferences after a few beers) about doing JIT, and the key problem is that it takes quite a lot of development effort to do and would (potentially, depending on details) need to be re-implemented for every architecture that Tcl supports. On the other hand, doing just x86 would handle a very large fraction of what's actually deployed...

FB: A plausible solution would be to use LLVM as an intermediary target instead of the current bytecode format. Then LLVM would JIT-compile to the native architecture using its available backends (it can also degrade gracefully to interpreter mode). But it likely involves a lot of work.

Sarnold: Actually that's what Jacl does by compiling commands to Java bytecode that the JVM can JIT-compile. A similar thing could be achieved using a .NET implementation.

What are the advantages of LLVM over JVM/.NET ? (apart from giving more performance and integration with C code, which are already known) Wouldn't it be overkill to write a JIT-compiling Tcl from scratch for LLVM, since both Tcl implementations exist for the JVM and .NET/Mono?

FB: I thought that Jacl was only an interpreter, but you're right since it now includes the TJC compiler. From what I know, the most interesting features of LLVM are global and runtime optimizations, and a runtime compiler architecture that can be used to provide Critcl-like features. Very interesting given the glue nature of Tcl. Starting from TJC to build a LLVM-based compiler might give a head start.