Here are some opinions on why adding Tcl (and possibly also Tk) calls to a C/C++ application is a bad idea (IMO -jcw).
Unfortunately, these questions reflect a deep misunderstanding of what scripting is about. Suppose one were to ask:
I've built this great wooden desk, how do I make it adjustable? or: I have a great room. How do I add a nice outside view to it?
The trouble, of course, is that the order of questioning is reversed.
The same goes for scripting, the point of which is that a program has structure at various levels, and consist of different types of coding. With scripting, the insight is that at the higher levels, performance is far less relevant than structural flexibility and the ability to evolve applications as development progresses. On deeper levels, the opposite is true: algorithms are pretty well fleshed out, and one needs all the speed one can get.
Hence the concept of having a main application as a scripting environment, with specialized parts inside coded as extensions, usually in C or C++.
[The conclusion, then, is] Build in Tcl/Python/whatever. Fill in the inner pieces in C/C++.
When faced with the task of adding Tcl/Tk to a C/C++ program, the best advice one can give is: throw out your "main()", quickly define all the rest as one or more extensions, and get going again quickly with tclsh or wish or tclkit. If nothing else, the hours spent making this transformation will be the start of the most productive phase of your development work. Really.
You can make your C/C++ accessible from Tcl by wrapping it with a small amount of code which registers new commands with the Tcl interpreter and converts the parameters. You can use a tools like the Simplified Wrapper Interface Generator SWIG to write the interface code for you.
Among the big advantages of this approach are:
If you disagree, the next best advice I think I can give is: stay away from Tcl and Python and Perl, and have a look at scripting tools such as Lua, which still take the perspective that scripting can be slapped on to existing code. That may well be true, but the fact is that for languages such as Tcl and Python, most people have long given up on the perspective that scripting is something you slap on as an afterthought.
The reality is that scripting is not an add-on, but that C/C++ is. You can agree with it or fight it. Your choice...
I'll disagree and fight.. -JR
I agree with you from a certain point of view. Scripting a c/c++ app is a bad idea and approaching the problem from the wrong direction.
However, tcl can be considered an "extension language" as well as a "scripting language". It can be incredibly useful to give the user hooks , where they can call arbitrary bits of functionality at particular places in the code. Tcl is easy to do that with and its extensibility (as a scripting langage) makes it an excellent choice.
As an example, the AOLserver web server was designed with tcl at its heart from the beginning. It works as a C application, calling the appropriate bits of tcl at the right times.
I don't think we disagree: one can have a Tcl outer layer, yet do just about everything in C. Also, what you're saying is that callbacks from C/C++ into Tcl are a good thing. Again, I agree - it lines up well with the point that scripting is for gluing at the highest level. -jcw
I want to try a slightly different expression of the same conclusion, that emphasizes technical specifics. There are places for both embedding and extending, of course. Even apart from categorical theorems about the power of scripting, the "front-line" reality is that, as Jean-Claude wrote in otherwise private correspondence, "once people see what is possible, and then want more, they may hit a wall with event loops, the needs for threads... when a C app is in control, it'll severely hamper using Tcl and Tk in an event-driven manner." The Tcl shell knows a lot of good stuff. A conventional C-style main() is a step backwards.
When embedding, remember to run Tcl's event loop. It may involve threads under windows (TES), or melding Tcl's event loop into your X loop under unix. See unix/tclXtNotify.c in the source for what's involved. I see nothing wrong with embedding -- it's just the road less traveled. -DG
Yes, reading back, I admit that I was being a bit harsh. Part of this is perhaps just personal preference, but what I wanted to point out is that one can go a long way by having a generic application which implements a scripting environment, and then adding extensions to it so C can be used when performance or secrecy or linking to other libs is important. Until now, compiling also had a substantial advantage of being able to generate a single executable - but there are now more ways to do that, also for scripted apps. -jcw
Yes, yes. A generic shell is a good way to go. That shell doesn't have to be tclsh or wish. It can do all the functionality you need at the C level to set the environment you need. One of my hobby projects is a windows MDI frame application that just exports some C functions for getting and setting items and parts with Stubs and cranks-up Tcl. I plan to add plugins to it for application components which themselves will be extensions for this shell. It's really bizzare and kinda cool too -- DG
Something Else
(Gerald Lester on comp.lang.tcl)
You may want to consider a two process model where your current simulator communicates to the GUI via either sockets or pipes. If you go with the socket model, then the GUI client can be running on a different machine from the number cruncher.
Splitting into two processes lets each process concentrate on what it is designed for, either number crunching or GUI. I've seen several people who have gone the #1 or #2 route with simulators complain about GUI being not responsive due to the number cruncher running or the complications of making sure you are not starving the event loop.
If you use seperate processes, you might like the libraries at http://www.trade-ideas.com/home/phil/ These make it quick and easy to export valid Tcl lists from C++.
While I'm not necessarily disagreeing with you, there is a place for using Tcl as an extension language.
I'm working on an app right now that uses Tcl to talk to an embedded system. As you'd imagine, it's working out great, especially for interactive development and automated testing of the embedded system itself, as the Tcl interface becomes (effectively) a shell on the embedded device. However, the final application must have (marketing requirement) an MFC-based GUI drive the device. So I've created a C++ class that provides the glue for the Tcl component (in a starkit) to the C++ layer. The C++ layer is just glue, though, no real logic. All of the actual work is done in the Tcl interpreter that's a private member leaving Tcl as purely an implementation detail.
The endpoint is that all of the business logic is developed and tested in Tcl and a thin C++ binding (glue) is put over the top, and then a native Windows GUI is added. You wind up with (almost) all of the development benefits of Tcl without the whining about the look and feel of a Tk GUI on a Windows platform. If a Windows-native GUI is a requirement, it's not a bad way to go.
Another rant on the same subject appears in a Python wrapper, under the title, "Extending vs. Embedding" [L1 ]
Also see ...
I find this discussion amusing, at best, since the entire PURPOSE of Tcl, as originally envisioned, was as a common embedded language. Dr. Ousterhout wanted a library that would be able to be embedded easily within an application. See ftp://tcl.activestate.com/pub/tcl/doc/tclUsenix90.ps , the first paper on Tcl.
Exactly. What a bunch of poppycock. There are applications that need to run *fast*, and occasionally want some scripting for configuration or other tasks. Sometimes it makes sense to make a lot of scriptable components, and other times it makes sense to make a big app that does one thing well, and can benefit from a bit of scripting on the side.
Having components which make use of embedded scripting is a great way to add extremely powerful capabilities very quickly. The results are often more compact, easier to test and maintain (as they can be run independently, OUTSIDE of the application), and seem like "magic" to the casual user. As long as performance is to specification, an embedded script component is arguably SUPERIOR.
For example, at work I provide a library which manages what is essentially a huge hierarchical database for the storage and analysis of IC design data. Part of the API provides a way to query the database with complex logical expressions, including such things as name wildcards, file modification dates, existence flags, etc. All of these things are internally handled by a Tcl interpreter and 'expr'. It would be a project in and of itself to actually code this in C++! Plus, I can use my Tcl code to access the database from a standalone Tcl script, allowing for all sorts of cool scripting possibilities.
So, I'll second the 'poppycock' comment above. _tje
Joe Mistachkin I think there are a lot of limitations with the "one true event loop" model of doing things. Additionally, if an application is designed in a sufficiently modular way, adding the ability to script the primitive operations of an application via Tcl should be quite easy. Indeed, that is one of Tcl's strong suits.
jcw - Poppycock, hah! (whatever the term means...)
It looks like someone is missing the point. Mine was: scripting is so effective that just about any app will benefit by using it. To which people responded how cool it is to add Tcl to an application, and how John Ousterhout invented the whole thing to do just that in the first place.
Yep. Over a decade ago. With impressive foresight. And - as I understand it - it was Kevin Kenny who took the logical next step, in saying that the main app can now be a generic interpreter, with extensions to add functionality. Tcl/Tk as slap-on-goodie sounds like "horseless carriage" thinking to me. But by all means, please continue to work that way if you prefer - while you fire ideas to explore, design, and test all the pieces of each application, in scripted form, with a few key (and sometimes elaborate / complex) pieces added in as C/C++ extensions. Not the other way around. Been there, done that - many years ago ([L2 ]). [So many years ago, that the given link has been broken for a couple of years. It's on the Wayback machine at [L3 ], but jcw may want to update. --KBK]
The adepts responsible for Python's Twisted module (which, incidentally, shares many ideas with Tcl) have written their own rant on "Extending Vs. Embedding" [L4 ].
TV Apart from the metaphorical reason for the force in the discussion, I'd say the idea that it is old news to program more than pieces in C and use tcl for the rest is not clear to me, the reasoning that tcl is easier 'is' clear to me of course, but developments in software land for the major part have been inspired by more practical reasons than language preferences and having to use a compiler or not. The availability of libraries being one, but more strongly, getting a certain job done.
When I want to do image processing, I need a C compiler or assembler or something else very fast to do that. When I want 3D programming, even more so. When I want number crunching, or anything else which takes a lot of processing, idem.
The reason for C++ was added libraries, objects/messaging, for openGL graphics, for java applet making based on a portable virtual machine, etc.
The idea of scripting has to do with imperative progamming based on interpreted readable code, and suits me pretty fine, but interacting with all pieces of an operating system when it is being used for more than a few percent of its computation capabilities probably will let you want to use a compiler. And depending on the percentage of that in your program, and whether tcllib is much larger than the amount of memory your actual program uses, the one can load the other or vice versa.
I've noticed a lot of rubbish when informaticists have no contentwise or even technical reason for advocating a certain approach, really, and the reasoning behind I guess most developments in computerland is pretty contentwise. I wrote assemblers, network simulators, electronical circuit simulators, graphics programs and cores, sound programs (also database), web and browsing based programs, an object oriented 3D language, image and 3D processing network based scripts, distributed number crunching applications, I've been there, too, and I'm sure developments in software land are subjected mostly to quite understandable logic.
But of course, there is a lot of crap reasoning, like that dll's can learn a lesson or two from systems made over 25 years ago.
Oh, I almost forgot mentioning the probably largest world wide programming based phenomenon in programming: video applications, streams, networks, coding, big screen presentation, processing.
The big interest is often where the hard stuff comes together, unless one has the fortune of not needing that.
davidw Advocating the "one true way" is all well and good, but not everyone wants to program that way:
http://www.advogato.org/article/735.html#9
CMCc POKEY: Is it a gun that shoots swords, or a sword with a gun in it???? [L5 ]
There may not be One True Way, but there may be one way which is much faster/better/easier than another. I agree with jcw - in my experience, extending tcl is faster/better/easier than embedding.
The only case I can think of where it would even be arguably a good idea to embed rather than extend is when you have an application you don't want to rewrite and you want to add a command interpreter to perform some kind of processing in a bag-on-the-side, where scripting is an afterthought. If you don't want to examine and modify the application code or if the code doesn't lend itself to modularisation, then I can sort of understand why you'd want to embed.
I don't think there's any virtue in designing an application to embed tcl.
IL 07 Dec 2005 - Looking to the wiki to find out the best way embed tcl in a cpp, I'm not very shocked to see this debate. ;) I can understand the point of view that TCL was designed to have c modules grafted onto it, but why isn't embedded TCL viewed as a possibility by the people against it?
just my 2c.
davidw: I was thinking about this concept (which I mostly agree with, but certainly not for all things) in relation to Tcl, Python, and Ousterhout's paper on Tcl and scripting ( http://home.pacbell.net/ouster/scripting.html ).
One of the things Ousterhout says is
Which is something that one is more likely to need to do if the program is first and foremost in Tcl with bits of C as extensions...
Lars H: I disagree with the "complex algorithms or data structures" criterion for not choosing a scripting language. I find it much easier to handle complex data structures in Tcl than it is in more traditional programming languages -- mainly because lists are a part of the core language and can be arbitrarily nested, since that tends to be all that is needed. However if the quote is from before Tcl 8.0 then I do see a point with it, since complex data structures stored as strings do have a poorer asymptotic complexity than their counterpart with references.
(An off-topic discussion about references has been moved to a separate page.)
DKF: It all depends on the complexity of the datastructure. For something like a complex priority queue (useful for some kinds of schedulers), it might not be best to implement that in Tcl. (Instead, do it as a new datatype that is accessible from a Tcl API. ;^))
davidw: Sure, but I don't see that kind of thing being a problem in Python. Not fast like C of course, but if you can't do it in Tcl or if it's forcing the language too much, you lose the ability to prototype in the scripting language, and rewrite in C to optimize, which is a negative.
RS wonders why a "complex" priority queue is asked for... how about a simple one? Each queue element obviously needs two parts: a "what" and its priority (say as an integer, bigger=higher). Tcl lists are good for representing queues, and priority queues too, if you just sort by priority after inserting one or more elements, e.g.
set pqueue [lsort -index 1 -integer [lappend pqueue [list $what_to_do $how_important]] puts "highest prio issue is: [lindex $pqueue end 1]"
Isn't that all it takes for a priority queue (and it is simple)?
DKF: The classic way of organizing a prioqueue is to use a balanced ordered tree.
slebetman For me, the decision of weather to embed to extend really depends on what you want to use Tcl for. If you're writing a 3D first person shooter and want a scripting language to control configuration and AI then it's better to write in C/C++ and embed Tcl. In this case, the majority of your coding will be on the 3D engine so you don't really want to fuss with Tcl too much.
On the other hand, if your app does something like spider across web pages and email you some search results then it's better to write in Tcl and extend where necessary. In this case, the majority of your coding will be handling text and strings and network connections which Tcl is good at. You really only want to go down to C/C++ to optimise things.
AMG: I don't see any reason for the main loop to be anything other than Tcl. Never ever. (Enlighten me?)
Tcl provides a wondrous event system, yet for a long time I dodged it anyway (because it was slow and hard to program and nonportable and difficult to understand and unpopular and all those other oft-quoted reasons that are actually 100% untrue... except maybe the popularity bit) in favor of my own code. But... I found that the more I worked on my main loop the more it became an event loop, and the more mature and feature-complete my event loop the more it converged on Tcl! I never want to be without Tcl I/O and events again.
In the case of the 3D game, I'd implement the 3D stuff in C (or, preferably, use tclogl if the speed is acceptable) and provide script commands at whatever level I find is appropriate (e.g., [plot_screen] or [plot_model] or [plot_polygon] or...). All my game logic (rules, physics, AI, on-screen indicators, warning tones, etc.) would be pure script. If the game logic requires complex, CPU-intensive algorithms for, say, path finding or ray tracing, I'd implement those in C and expose them as commands (e.g. [a*_path $from $to] or [cast_ray $from $direction]), then use them in script.
The game state needs to be regularly translated into a visual for the user to see. That is, the screen must be updated whenever there's something worth updating. Modern 3D game displays move continually, even if for no good reason, so basically the screen must be updated in anticipation of/coincident with as many monitor refreshes as possible. Can this be a GUI event or a timed event? Well, tclogl's -displayproc seems to handle this internally (actually it looks like OpenGL does it), so just use that, and don't worry about how it's implemented; your program's "main" is still Tcl not C, so you're able to spend your time working on game mechanics instead of figuring out how to portably connect to the graphics system or how to read a file of any size without overflowing buffers.
Maybe I'm wrong--- I need more experience in this area before I can pretend to speak authoritatively--- but I do have my imagination!
slebetman To me, the main loop should be Tcl if Tcl is or can be a fundamental part of the program. But if 99% of your code is C/C++ it doesn't make sense to mangle you code just to have a Tcl world view. In most games, like Quake for example, the interpreter is embedded into the code instead of the other way round for exactly this reason. You don't want to be writing Tcl when you program in C. In such cases, the programmer never writes a line of Tcl code. It is the content developer, who supplies the physics, AI and game logic, who actually write Tcl code. Using Tcl simply means you don't have to debug a custom scripting language - not as a central point of development.
Also, 3D games especially always strive to push the speed limit. Moving too much code into Tcl defeats this purpose. Tcl is 50 times slower (that's 5000% slower) than C depending on which benchmark you ask (my feeling is, with real-time 3D games that's a close approximation). You don't really want a Tcl loop calling C commands for this. It's the "right tools for the right job" kind of thing.
Game logic on the other hand can run slow since a human player's reaction time is not as fast as the screen's refesh rate. It is for this that script interpreters are usually used in games. Another common use of interpreters is the parsing/loading of configuration files. Since neither requires the full capability of the Tcl main loop and both are usually not the responsibility of the programmer it does not make sense to force the programmer to write a single line of Tcl.
Another area where embedding has been successful is CAD. But with today's CPUs it can be argued that a lot of it can be done in Tcl with custom C extensions (TkGate is a good example). But try doing the same thing 5 years ago on 90Mhz CPU.
I think that by arguing which is better, we miss one of the great strengths of Tcl: you really do have the best of both worlds.
The ease of mixing C and Tcl means that I can write in C when I feel like it and write in the opposite of C whenever C becomes a pain in the butt. Likewise, I can choose the best approach to combine the two. Sometimes extension is impossible, or awkward, especially when you are working with someone else's codebase.
Recently I wrote a server and client that connect by a socket. The server is written from the ground up and needed a couple routines to do fast byte-level manipulation and encryption. That was a job for extension. The client was an Objective-C plugin for someone else's Cocoa application. That had to be accomplished by embedding. In both cases, making it work was easy as pie, and I never felt like I was using a wrench as a hammer or vice versa.
I think the Tcl philosophy is truly a radical idea, but concealed by its pragmatism. Everyone else is trying to develop the ultimate programming language (a tired old idea concealed, by its lofty goals and wide-eyed optimism.) But here this dude Ousterhout designed the ultimate half-a-language instead. It doesn't have to be all things to all people, just everything C has failed at---combined with excellent capabilities for merging with C. That includes both the friendly C API and the terseness and flexibility of the Tcl syntax itself.
Xetwnk - 2017-08-05 08:03:21
I have a great reason to (want to) use Tk from a C++ application.
An application I'm trying to develop needs "palette"-style (what used to be called PseudoColor, in the X Windowing System) behavior. In that mode of operation, you draw graphics with a "color number," call it "N". The number is what goes into the memory buffer, pixmap, bitmap, or whatever, that holds the image you're drawing. The RGB color that visibly appears on the screen - - call it "C" - - for color number N is looked up in a colormap, either by hardware or by video-subsystem software. And if you change the RGB value that's in the N'th cell of the colormap, everything already drawn in color N instantaneously changes to the new RGB color. I need that behavior in a particular program I'm developing that has to emulate old 8-bit, 256-color hardware that worked that way.
In the standard Windows paradigm, however, as far as I can tell, you can only draw graphics "directly" with the actual RGB color, C, that is to appear on the screen. With this direct-display color scheme, all pixels of "color C" are independent, and in order to change all of them instantaneously to another color you have to redraw them all. And that means you have to do some heavy bookkeeping. It's not impossible, but it's ugly. One way to do it is to maintain my own mapping between color numbers N and RGB colors C, maintain a transparent-backgrounded pixmap (or single-plane bitmap) for every color I intend to use, and every time I draw a pixel of (the RGB color C corresponding to) "color number N" on the display, also set that pixel in the N'th background pixmap/bitmap and clear that pixel in all the other colors' background pixmaps/bitmaps. Then, when I want to change the RGB color C that's associated with color number N, "all" I have to do is (somehow!) blit with the new color C, "through the N'th background pixmap/bitmap" to the screen. I'm not sure what it would take to do all that, but I'm pretty sure there's some way. Still, it's pretty tedious, resource-consuming, and makes all drawing operations take N+1 times as long as they should. Can you imagine doing full 256-color support? :-b
On the other hand, I have also done this sort of thing in Perl/Tk, where I can use a Canvas widget and draw, say, 10,000 red rectangles, but tag each of them with, say, the word 'weasel', and then, at any future time, simply tell the Canvas to set the color of all 'weasel'-tagged objects to some other color, say yellow. POOF! It's done, and in exactly the "single, simple operation" manner of the palette-style color paradigm I want. I guess I could implement the same sort of "tagging" mechanism in my own code, but again, it's gonna be tedious - - and in the application I'm actually developing, I'd be tracking every pixel in an image of at least 800 x 480 - - which isn't big by today's imaging standards but might be a lot of excessive bookkeeping, possibly with additional concerns about stacking orders, overlap, etc.
The obvious "easy way" to get the behavior I want, therefore, is to leverage the fact that Tk already does all that, from within my C++ main program. So where am I going wrong, exactly?