Why do I build itcl-ng (itcl-Next Generation)?
About September 2006 I (apw) started to think about the future of incr tcl / Itcl. I had heard that there are a lot of problems in Tcl core because of Itcl. So I was thinking about a new version. The first idea was to build a Tcl only version. The story of that approach can be seen at tclOO missing features for Itcl.
After all I have started for the 4th time using original Itcl source code (because of performance problems with the other projects) and replacing all the dispatching and the code directly working in Tcl core areas with tclOO and a small layer which can perhaps in the future be integrated into Tcl core.
Goals, ideas etc.
itcl-ng is now as itcl 4.0 a bundled package of Tcl 8.6
Progress:
apw 2007-09-08 Today I have finished the implementation of the very first version of itcl-ng which is passing Itcl test suite with 0 failed test cases and 400 succeeded ones. An almost up-to-date version is now at Sourceforge in the incr tcl repository [L1 ] with tag itcl-ng. Same is done for itk which is running about 98 of 100 tests successful.
apw 2007-09-10 I have added an experimental version of 3 new commands (with sub commands) to itcl:
they have the same functionality as the corresponding commands described in TIP#257 i.e.: ::oo::define objectOrClass filter filterName ? ... ? etc. Any comments on adding these commands?
apw 2007-09-15 Have been working for the last few days on implementing some widget and delegation functionality for itcl-ng. Have mostly stolen ideas from William Duquette and his snit implementation, because I think he has done a very good job there. First round of how the interfaces in itcl-ng will look like (for discussion):
The difference to snit is that these 3 commands are internally handled identical to ::itcl::class concerning methods, procs (== typemethods in snit), variables, commons (== typevariables in snit) (I hope I have understood snit at that point correctly). These 3 commands have additional functionality which cannot be used with ::itcl::class: The following commands are handled on the same level as method, variable, constructor etc. apw 2008-10-26 delegate proc has been changed to delegate typemethod
optionSpec: something like the X-resource stuff: {-font font Font}
optionOptions: ?-default <defaultValue>? ?-readonly? ?-cgetmethod <methodName>|-cgetmethodvar <varName>? ?-configuremethod <methodName>|-configuremethodvar <varName>? ?-validatemethod <methodName>|-validatemethodvar <varName>?
-cgetmethodvar etc. is the name of a variable to be used. The variable contains the method to be used. This allows to initialize or change the methods during runtime!
An option can have a protection specifier (public, protected, private). The protection is handled as is done for variables
widgetType: any of the following: frame labelframe toplevel ttk::frame ttk::labelframe ttk::toplevel
cgetmethod signature: method myGetOption {optionName} { ... body ...}
configuremethod signature: method myConfigureOption {optionName value} { ... body ...}
validatemethod signature: method myCheckBoolean {optionName value} { ... body ...}
a component creates a variable and a method with componentName as the name. The method is using the contents of the variable as a command and is executing it with the arguments passed in. A component can have a protection specifier (public, protected, private). The protection is handled as is done for variables/methods.
instead of a win parameter for all methods in snit there is a thiswin special variable, which returns the object's window path name like .btn without any :: parts.
itcl_hull is a special component which is created automatically and is used as the container for the widget command
itcl_options is a special variable (array) which is automatically filled with all of the objects options
3) these commands are now "officially" in the release of itcl-ng 4.0a1 (2008-11-16)
There will be some additional subcommands to itcl-ng's info command like info option etc. (To be described in detail).
apw 2007-09-17 A mostly complete implementation of the additional commands described directly above is now in the itcl-ng cvs directory. There is a sub package of itcl named itclWidget for it in the directory itcl.
the additional info sub commands will be:
and for the class like commands:
1) these commands are now implemented (2008-11-15)
2) these commands are now implemented (2008-11-16)
4) as of 2009-01-02 are now fully implemented, and there exist tests
apw 2007-10-15 I have added additional commands to allow easier implementation of ntkWidget. These are:
and on the class level of ::itcl::extendedclass
this is a variable with an automatically generated setter/getter method with the same name and you can have a callback function called when setting the variable using the setter method
An extended class behaves like a class but has as additional features all the option and delegate commands. The object of ::itcl::extendedclass can be modified by ::itcl::adddelegatedoption to add delegated options after the object has been created. Same for delegated methods. To make that complete, there is the ::itcl::setComponent command, which removes existing delegated options/methods which point to the component and sets the component to a new command, installing all the delegated options/methods which should be delegated to that component (again after the object of ::itcl::extendedclass has been created). So here we have some functionality which allows to dynamically change the object behaviour at runtime!
That functionality is used in ntkWidget to add grid/pack/place/scrollbar etc. functionality to an existing ntk window
5) implemented and tested by some examples, but no tests for testing them harder are written yet (2008-12-11)
Where can itcl-ng be found?
itcl-ng can be found here: [L2 ] or the source package releases at [L3 ]
LV To which problems do you refer when you say that you had heard "[...]that there are a lot of problems in the Tcl core[...]"?
I can think of one possibility to which you might be referring. A TIP was approved to include itcl in the tcl core distribution. However, to move forward on this, someone who knew itcl internals sufficiently was needed to work on the changes that would be necessary to get the code integrated into the build process (and, likewise, to separate itcl from itk in its current distribution). As far as I am aware, no one ever stepped forward to work on that.
Otherwise, I would have to assume that itcl requires some internal functionality from the tcl core. If _that_ is the case, then while having a new OO extension built on tclOO is useful, to help the tcl core situation, one has to guarantee that people don't continue to use the old version. I suppose that the stubs interface could change in a way that the old version of itcl wouldn't work with a new version of Tcl. That would force the situation. It could also generate a lot of bad feelings, unless the change were critical.
apw I am in close contact to the TCT members, and they told me about the places where there are some workarounds in Tcl core to make Itcl still running. I have addressed these problems and have now built a version which avoids these critical places in Tcl core. So the "I have heard" is already mostly solved. For additional interfaces to make Itcl more independent of Tcl core I am in contact with miguel and dkf and we are on the way to get a good solution some time in the (I hope) not so far future. So everything with itcl-ng is running well at the moment.
LV This sounds really good. So, is the intention to make this version of itcl (NG) a part of itcl.sf.net, or a part of the tcl core, or what?
DKF: It's now in the itcl CVS. Progress to the Tcl core... well, we'll see. Not for 8.5, but after that things are more up for grabs.
LV 2007 Oct 15 It would be interesting to read experiences from itcl developers who have tried out itcl-ng and gotten it to work successfully.
apw 2007-10-15 at the moment you need a patch for tclOO, so I assume not so many have tried yet. To my knowledge only GPS has it successfully running on Linux. Right now there is only Linux support, as I have no other machines available (besides my none existing knowledge of Win/OSX).
APN 2008-04-14 What is the current state of itcl-ng? Is it still being worked on? I'm looking for a 8.5 TclOO based object system, finding Snit a bit too slow for my specific uses (though it's fine for widgets and such).
escargo - The news group just had an announcement for TclOO 0.2, which included this snippet: "Many small features, mostly hidden out of sight, for support of Arnulf Wiedemann's (apw) itcl-ng." This would seem to imply progress for both TclOO and itcl-ng.
apw 2008-09-12 just a note to state that there is still work going on for itcl-ng. At the moment I am looking for a better model for command and variable resolving, to get better performance (the current performance is comparable to itcl 3.4, but I want to be faster :) ). I have some thoughts, which are not yet ready/finished. One is to speed up command resolution by using a this command like the my command in TclOO for calling class methods. That would be optional for a user, but would give better performance and maybe when using it avoid the need of command name resolvers for that case. Stay tuned.
LV So, did those last two itk test cases that were failing last year get resolved? apw 2008-10-18 yes.
LV Also, does anyone have experience with the iwidget set and the new implementation? apw as the scripting API did not change, I think all iwdgets should run out of the box (have not yet tested)
apw 2008-09-14 started adapting itcl-ng to the TclOO version integrated into Tcl core and to NRE (because TclOO is using it, so I also have to adapt to that). 2008-10-04 done.
apw 2008-10-04 adapting itcl-ng to the TclOO version integrated into Tcl core and to NRE is mostly done (including itk). 2008-10-06 done.
apw 2008-10-04 I have started to describe some of the internals of itcl on this page itcl internals to discuss the strategy (speedup) for command and variable lookup in Tcl core with respect to itcl-ng. One side effect of that is TIP #333: New Variable and Namespace Resolving Interface.
apw 2008-10-18 The first alpha release version (Itcl 4.0a0 and Itk 4.0a0) has been released on SourceForge today [L4 ]
apw 2008-10-19 I am now writing documentation and tests for some of the new commands mentioned above like ::itcl::widget, ::itcl::extendedclass and the delegate and options commands, so they can go as a first trial version into the next release.
apw 2008-10-25 The first 5 tests for ::itcl::extendedclass and 2 tests for ::itcl::widget are now running ok. There was a bigger part of code to fix and to debug, as the functionality for delegation and components was not yet implemented completely for some special functionality of the test cases. Hull handling for ::itcl::widget had to be implemented new, as it was not yet done. I hope in a few weeks to have a version ready, which can be released as an alpha release for ::itcl::extendedclass ::itcl::type and ::itcl::widget including delegation and option functionality mostly as described above.
apw 2008-10-26 Now also the first 10 tests for ::itcl::type are running, had to write a bigger part of code for the "corner cases" of the tests.
apw 2008-11-03 About 30 tests for ::itcl::type are running.
apw 2008-11-09 Making progress, now there are about 140 tests successfully running for ::itcl::type. a lot of these tests are also checking functionality for ::itcl::extendedclass and ::itcl::widget, which is internally identical
apw 2008-11-14 Currently 207 tests are running successfully for snit like functionality. There are still 3 failing, where I have to figure out why and there are 3 where the messages are different. most of the functionality of ::snit::type, ::snit::widget and ::snit:.widgetadaptor is now running ok. There are some differences: I am not supporting "multi part" method names, there is no %....% substitution (for "%AUTO%" the itcl equivalent "#auto" can be used). Hull is named as component "itcl_hull" and the options array is named "itcl_options".
Next the introspection functionality will be implemented (info .... commands).
apw 2008-11-15 first part of info commands is mostly done (54 tests for the info commands are successfully running, there are 5 failing test cases for the info commands). There are 2 commands still missing: widgetclass and hulltype (to be done). From the info commands the detailed ones are missing like "info method (with options)" or "info typevariable (with options)" or "info option (with options)" ...
Date for next alpha release is coming closer :) .
apw 2008-11-16 It has happened the next alpha release of itcl-ng is released (Itcl4.0a1, Itk4.0a1 and ItclWidget4.0a1) see on the Sourceforge Page [L5 ].
The new functionality ::itcl::type, ::itcl::widgetadaptor and ::itcl::widget is now officially available. It is mostly compatible with the equivalent snit commands ::snit::type, ::snit::widgetadaptor and ::snit::widget. There are some rudimentary man pages in the release, but you can also use the snit documentation for the commands and look here on the wiki in this document at the notes of 2008-11-14 for differences
There are nearly 300 tests available for the new functionality 284 of them pass without problems. The remaining ones are special cases, which should be no big problem (I will try to fix them soon). I hope the big number of tests helps that there are not too many problems left :) . Please try out, play with it and report errors/problems.
Just to mention it: ::itcl::type, ::itcl::widgetadaptor and ::itcl::widget allow inheritance as in normal itcl classes with the standard inherit command as well as delegation using the delegate command - one of the main features of snit -, also in combination. But be aware, that combination has not yet been tested much (but will be in the future). Using ::itcl::widget and ::itcl::widgetadaptor you can build megawidgets using Tk commands/"objects" as classes/objects without the need of using itk.
apw 2008-11-23 Itcl 4.0a1 seems to work ok, at least there where about 50 downloads until today and no problem report has been sent to me yet.
The first round of "memory leak" tasks is finished. All objects and classes are freeing the functions, variables and options now correctly (the data structures). I have written for that a check module, which can be activated using a compile time flag to check and register all Itcl_PreserveData and Itcl_ReleaseData calls and checking if they are paired. That is working correct for all tests of the test suites (about 700 tests). I had to adapt a few tests to clean up correctly.
apw 2008-11-30 The second round of "memory leak" finding tasks is done. I have used a refcount checking function and found a lot of places where I was not doing refcounting correct. Had some trouble to understand, that some strings are "literal" in Tcl and was wondering, that my refcounts where paired, but the objects were still there. Now I have understood. There are a few places remaining to be done. Next task will be checking direct allocating and freeing of C structures.
apw 2008-12-06 The next round of "memory leak" finding tasks is done. This time using the functionality in Tcl core for getting for example ckalloc without ckfree. Found again a lot of places. At the moment it looks like I can get all the itcl tests running without memory leakage. There are still some places to fix, but not so many any more.
apw 2008-12-11 Again 3 days of heavy searching and fixing memory leaks. And now beta 1 version of itcl-ng (itcl4.0b1) is released see: [L6 ]. This is also the version to be bundled with Tcl core 8.6 beta.
And a big WARNING: all the package names are now all lowercase (also because of discussion on tcl core mailing list):
apw 2009-01-24 there have been some smaller changes for enhancing ::itcl::extendedclass to be able to build an itcl-ng only version of itclWidgets.
apw 2009-01-24 itclwidget package is no longer needed. It is now finally fully integrated into itcl 4.0. There were only the last 10% of the itclwidget functionality, which were still in that package. That part had to be converted from C to a Tcl script, whith Tcl_Eval calls to call the Tcl procs from C. The problem to solve was to load Tk only when needed and to not have references in C code to Tk library.
LV - 2009-06-29 10:26:21
What is the status of itk/iwidgets 4.0 beta? Are these packages easily downloadable? If so, from where?
If someone wishes to download itk/iwidgets 4.0 beta to build against tcl/tk 8.6 beta, are there any special instructions needed for doing so?
Thanks!
apw - 2009-10-18
Sorry for my delayed answer, I have been working very very hard for a payed work project, so I did not even have time to look here.
itk 4.0b3 is available here [L7 ] or the source package releases at [L8 ]. If you check out the sources from there, there should be no problems compiling against tcl/tk 8.6 beta, as I am doing so too.
apw - 2010-03-19
There is a new beta release of itcl-ng available (mostly bug fixes). This will be integrated into tcl 8.6b2 (at least I hope so).
itcl 4.0b4 cvs sources are available here [L9 ] or the source package releases (itcl4.0b4 and itk4.0b4) at [L10 ].
LV so, is this new beta release different than what is already present in Tcl 8.6 beta?
apw yes
apw - 2012-01-12 The newest beta release is itcl4.0b8. Itcl source hosting has moved (some time ago) to core.tcl.tk at [L11 ] This version is bundled to tcl 8.6b3.
RZ - 2012-02-09 13:13:25
The code for ticket 36577626c340ad59615f0a0238d67872c009a8c9 should be:
package req Itcl
::itcl::class ::a {
constructor args {puts $args}
destructor {}
}
::a ::a::b apw - 2012-02-25: problem is fixed in fossil (easy to fix, hard to find :) )
RZ - 2012-02-25 20:51:35
It works now. Thank you.