Version 9 of Why adding Tcl calls to a C/C++ application is a bad idea

Updated 2003-08-04 15:29:14

Here are some opinions on why adding Tcl (and possibly also Tk) calls to a C/C++ application is a bad idea (IMO -jcw).

Unfortunately, these questions reflect a deep misunderstanding of what scripting is about. Suppose one were to ask:

      I've built this great wooden desk, how do I make it adjustable?
 or:  I have a great room.  How do I add a nice outside view to it?

The trouble, of course, is that the order of questioning is reversed.

The same goes for scripting, the point of which is that a program has structure at various levels, and consist of different types of coding. With scripting, the insight is that at the higher levels, performance is far less relevant than structural flexibility and the ability to evolve applications as development progresses. On deeper levels, the opposite is true: algorithms are pretty well fleshed out, and one needs all the speed one can get.

Hence the concept of having a main application as a scripting environment, with specialized parts inside coded as extensions, usually in C or C++.


[The conclusion, then, is] Build in Tcl/Python/whatever. Fill in the inner pieces in C/C++.

When faced with the task of adding Tcl/Tk to a C/C++ program, the best advice one can give is: throw out your "main()", quickly define all the rest as one or more extensions, and get going again quickly with tclsh or wish or tclkit. If nothing else, the hours spent making this transformation will be the start of the most productive phase of your development work. Really.

You can make your C/C++ accessible from Tcl by wrapping it with a small amount of code which registers new commands with the Tcl interpreter and converts the parameters. You can use a tools like the Simplified Wrapper Interface Generator SWIG to write the interface code for you.

Amomg the big advantages of this approach are:

   Tremendously flexible application
   Clean separation between GUI/Functional code
   Excellent framework for tcltest regression tests

If you disagree, the next best advice I think I can give is: stay away from Tcl and Python and Perl, and have a look at scripting tools such as Lua, which still take the perspective that scripting can be slapped on to existing code. That may well be true, but the fact is that for languages such as Tcl and Python, most people have long given up on the perspective that scripting is something you slap on as an afterthought.

The reality is that scripting is not an add-on, but that C/C++ is. You can agree with it or fight it. Your choice...


I'll disagree and fight.. -JR

I agree with you from a certain point of view. Scripting a c/c++ app is a bad idea and approaching the problem from the wrong direction.

However, tcl can be considered an "extension language" as well as a "scripting language". It can be incredibly useful to give the user hooks , where they can call arbitrary bits of functionality at particular places in the code. Tcl is easy to do that with and its extensibility (as a scripting langage) makes it an excellent choice.

As an example, the AOLserver web server was designed with tcl at its heart from the beginning. It works as a C application, calling the appropriate bits of tcl at the right times.


I don't think we disagree: one can have a Tcl outer layer, yet do just about everything in C. Also, what you're saying is that callbacks from C/C++ into Tcl are a good thing. Again, I agree - it lines up well with the point that scripting is for gluing at the highest level. -jcw


I want to try a slightly different expression of the same conclusion, that emphasizes technical specifics. There are places for both embedding and extending, of course. Even apart from categorical theorems about the power of scripting, the "front-line" reality is that, as Jean-Claude wrote in otherwise private correspondence, "once people see what is possible, and then want more, they may hit a wall with event loops, the needs for threads... when a C app is in control, it'll severely hamper using Tcl and Tk in an event-driven manner." The Tcl shell knows a lot of good stuff. A conventional C-style main() is a step backwards.


When embedding, remember to run Tcl's event loop. It may involve threads under windows (TES), or melding Tcl's event loop into your X loop under unix. See unix/tclXtNotify.c in the source for what's involved. I see nothing wrong with embedding -- it's just the road less traveled. -DG

Yes, reading back, I admit that I was being a bit harsh. Part of this is perhaps just personal preference, but what I wanted to point out is that one can go a long way by having a generic application which implements a scripting environment, and then adding extensions to it so C can be used when performance or secrecy or linking to other libs is important. Until now, compiling also had a substantial advantage of being able to generate a single executable - but there are now more ways to do that, also for scripted apps. -jcw

Yes, yes. A generic shell is a good way to go. That shell doesn't have to be tclsh or wish. It can do all the functionality you need at the C level to set the environment you need. One of my hobby projects is a windows MDI frame application that just exports some C functions for getting and setting items and parts with Stubs and cranks-up Tcl. I plan to add plugins to it for application components which themselves will be extensions for this shell. Its really bizzare and kinda cool too -- DG


Something Else

(Gerald Lester on comp.lang.tcl)

You may want to consider a two process model where your current simulator communicates to the GUI via either sockets or pipes. If you go with the socket model, then the GUI client can be running on a different machine than the number crunchier.

Splitting into two processes lets each process concentrate on what it is designed for, either number crunching or GUI. I've seen several people who have gone the #1 or #2 route with simulators complain about GUI being not responsive due to the number crunchier running or the complications of making sure you are not starving the event loop.

---

While I'm not necessarily disagreeing with you, there is a place for using Tcl as an extension language.

I'm working on an app right now that uses Tcl to talk to an embedded system. As you'd imagine, it's working out great, expsecially for interactive development and automated testing of the embedded system itself, as the Tcl interface becomes (effectively) a shell on the embedded device. However, the final application must have (marketing requirement) an MFC-based GUI drive the device. So I've created a C++ class that provides the glues the Tcl component (in a starkit) to the C++ layer. The C++ layer is just glue, though, no real logic. All of the actual work is done in the Tcl interpreter that's a private member leaving Tcl as purely an implementation detail.

The endpoint is that all of the business logic is developed and tested in Tcl and a thin C++ binding (glue) is put over the top, and then a native Windows GUI is added. You windup with (almost) all of the development benefits of Tcl without the whining about the look and feel of a Tk GUI on a Windows platform. If a Windows-native GUI is a requirement, it's not a bad way to go.


Another rant on the same subject appears in a Python wrapper, under the title, "Extending vs. Embedding" [L1 ]

Also see ...


I find this discussion amusing, at best, since the entire PURPOSE of Tcl, as originally envisioned, was as a common embedded language. Dr. Ousterhout wanted a library that would be able to be embedded easily within an application. See ftp://tcl.activestate.com/pub/tcl/doc/tclUsenix90.ps , the first paper on Tcl.


Exactly. What a bunch of poppycock. There are applications that need to run *fast*, and occasionally want some scripting for configuration or other tasks. Sometimes it makes sense to make a lot of scriptable components, and other times it makes sense to make a big app that does one thing well, and can benefit from a bit of scripting on the side.


Having components which make use of embedded scripting is a great way to add extremely powerful capabilities very quickly. The results are often more compact, easier to test and maintain (as they can be run independently, OUTSIDE of the application), and seem like "magic" to the casual user. As long as performance is to specification, an embedded script component is arguably SUPERIOR.

For example, at work I provide a library which manages what is essentially a huge hierarchical database for the storage and analysis of IC design data. Part of the API provides a way to query the database with complex logical expressions, including such things as name wildcards, file modification dates, existence flags, etc. All of these things are internally handled by a Tcl interpreter and 'expr'. It would be a project in and of itself to actually code this in C++! Plus, I can use my Tcl code to access the database from a standalone Tcl script, allowing for all sorts of cool scripting possibilities.

So, I'll second the 'poppycock' comment above. _tje


Joe Mistachkin I think there are a lot of limitations with the "one true event loop" model of doing things. Additionally, if an application is designed in a sufficiently modular way, adding the ability to script the primitive operations of an application via Tcl should be quite easy. Indeed, that is one of Tcl's strong suits.


jcw - Popycock, hah! (whatever the term means...)

It looks like someone is missing the point. Mine was: scripting is so effective that just about any app will benefit by using it. To which people responded how cool it is to add Tcl to an application, and how John Ousterhout invented the whole thing to do just that in the first place.

Yep. Over a decade ago. With impressive foresight. And - as I understand it - it was Kevin Kenny who took the logical next step, in saying that the main app can now be a generic interpreter, with extensions to add functionality. Tcl/Tk as slap-on-goodie sounds like "horseless carriage" thinking to me. But by all means, please continue to work that way if you prefer - while you fire up your IDE's and link to libtcl.{a,so}, I'll be humming along coding little bits and pieces to explore, design, and test all the pieces of each application, in scripted form, with a few key (and sometimes elaborate / complex) pieces added in as C/C++ extensions. Not the other way around. Been there, done that - many years ago ([L2 ]).


category concept