Introspection

What sorts of introspection are there? What sorts should there be? This is the omnium gatherum of introspective issues in Tcl.

Please add to this if you can. Examples and discussions of implementation issues are particularly welcome!

DKF

LV What do Tcl developers mean when they use the word introspection?

File System (Basic): This sort of OS introspection is pretty much expected of every language in existence. Key commands are file and glob.

Packages/Extensions: This sort of Tcl introspection allows one to to ask questions of a package such as what version are you, what dependencies do you have, what namespaces/variables/commands/procedures/etc. do you define, etc.

Variables: Tcl has plenty of support for this. You can not only list the variables that exist (using several info subcommands) but you can also determine if they are arrays (various array subcommands) and even perform introspection on the activity affecting that variable (the trace command.)

LV However, I don't know that one can determine the meta-type of the variable. For instance, if a variable is a plain string, or represents the name of a font, widget, etc.

Commands: You can easily get a list of the currently defined commands (use info commands of course.) The one wart is that it is difficult to get a list of the commands that are auto-loadable. Mostly this doesn't matter, of course, but it is a little niggle. There are times when being able to get the commands available from an extension, or from tcl, without having to parse error msgs, would be useful. The same goes for getting the options that a command offers. And for that matter, in cases where a command takes a fixed set of values for a particular option, it would be useful to get this fixed set back as a list, so that it could be formatted for visual display and selection.

Procedures: There is masses of excellent support for procedure introspection (various info subcommands.) Some people have even adapted this to create sophisticated debugging tools. Example, Printing proc sequence

Stack: a program can look at what procedured have been called higher up (my caller, caller's caller, and so forth) with info level. One application is memoizing far more directly than is possible in some other languages.

Namespaces: Dead easy to introspect. GWM Eg from wish shell type:

  namespace children

to see the namespaces which already exist - I see "::tablelist ::dom ::auto_mkindex_parser ::xml ::critcl ::activestate ::pkg ::msgcat ::vfs ::tk ::tcl". If your program has created a namespace then that will also be in the list (eg as "::mynamespace"). I find it useful to have a list of namespaces at startup (::tk, ::tcl etc) then source my program and introspect into the namespaces not listed in the startup namespaces. Note that Itcl creates its classes as namespaces, so you can find itcl class names this way Itcl introspection.

Interpreters: ? Various interp subcommands, for aliases take a look at introspection on aliases.

Threads: ?

File System (Advanced): There is not currently a standard mechanism in the core for getting the remaining free space on any particular volume (TclX?) The volume listing support is patchy (file volumes works fine on Windows (Mac?) but only ever returns / on Unix systems - from some points of view this is correct, but it is a bit of a pain as you then are forced to grok the info out of /etc/m*tab...)

Processor Time: Tough to measure, since elapsed wall-clock time is definitely not the right thing to measure (different systems run at different speeds, naturally) so it may be better to measure the number of commands executed/evaluations of code chunks performed. (Byte-code would need to be adapted to keep these numbers correct, of course!) Note that info cmdcount offers part of the information you would want already.

Memory Use: The other biggie. The main problems stem from handling the case where you are transferring an object from one thread or interpreter to another. Do you then change the ownership of that object and consequently the memory used by the interpreters? What about objects that are shared between interps? Who should own sourced source code? You don't want a particular thread to take a hit or not simply because it dynamically loads a piece of Tcl before another thread does in the same process... (Self-generated stuff is easier.)

What have I missed? :^) - DKF

LV what about metadata introspection - what version of tcl is this, what packages can be loaded, etc.?

Just noting that my inspiration for the last two on the list is to enable the creation of monitors (written in Tcl) for safe-Tcl interpreters to stop a whole extra bunch of denial-of-service attacks. That they would be useful for other things is just a happy bonus...

DKF

escargo - The RC File includes a proc called lspackages that lists some of that package info.

As to what tcl version, then the globals tcl_version and tcl_patchLevel would seem to be the obvious answers.

Channels: ?
Aliases: See above, Interpreters.

OK, for channels, I suppose we want to know what channels are available, and whether they are opened for reading and/or writing (in addition to the information currently available with fconfigure, that is.) We do not want to have a mechanism for determining if a particular channel is ready for reading or writing, since we want to enforce the use of the event-based mechanism. Do we want to have introspection of the event system?

Being able to tell if a particular command is an alias (and what it is an alias to, if that is in the same interpreter) would be nice. In fact, it might be nice to have a mechanism that says what kind of command a particular name is referring to. The kinds that people ought to be able to ask about (for commands in their own interpreter) should include:

Procedure - you can get the info now by using [info procs $name], but that is messy.
Compiled - something like set that gets handled specially (and efficiently.)
Alias - the command in question is an alias for another command.
Defined - the fallback answer (if the command exists.) Aliases from other interpreters come under this heading, and it might be reasonable for the introspector to return whether the command was defined with Tcl_CreateCommand() or Tcl_CreateObjCommand() (important to know if you really want to avoid Tcl_Obj "string-foot" creation!)

Why not let programs introspect on aliases coming in from other interpreters? Well, this might allow a script running under SafeTcl to figure out exactly what commands were being monitored by its interpreter's master. This information is probably not useful in any respect - though I cannot see how it could be used to damage the integrity of the system, you cannot give a name for an interpreter that is not yourself or a child of yourself. In other words, there is no meaningful value that the introspection system could return!

DKF

'Open for R/W' looks good. Same for enforcing usage of fileevent.
Do we want to have introspection of the event system?. Well, I would like to have that sometimes, when trying to figure out what I did wrong in an event-driven system (like my SMTP/POP/... engines :-). This makes me notice another thing however: All introspection (interfaces) so far is/are polling-based, i.e. I have to ask for the information, explicitly. Especially with the event-system it is highly desirable to have a callback-based (notification) interface telling me automatically about changes in the system. It would make the task of debuggers (Tuba/TclPro) easier too if they had hooks telling them the relevant things without requiring them to overload a multitude of commands affecting the state.
Back to channels: fconfigure -type to get the type of the channel (file, pipe, socket, pty?...). And, if the Trf patch ever makes it into the kernel, a list of all channeltypes stacked upon each other.

What resources (X, Windows, Mac) does a particular widget or command use, what values can that resource hold?

What environment variables does a particular command or widget use?

Isn't the first Tk introspection, and not Tcl introspection? Not that it is a bad thing, but...?

The second is probably not always possible to tell without actually running the code (computed env-var names!)

DKF

LV It would probably be good to have a section - or another page - that covered Tk introspection as well. Tk has a lot of already built in introspection - widgets can tell what their current settings are (thought they can't tell you, I don't think, which values came from where). But having the ability to ask a widget what specific font is being used, and why, would really be useful.

Besides Bruce's package, what tools are available for introspection? Does anyone know of anyone interested in supporting Tkinspect?

Well - I've been doing that for a bit now. It's being done within the tkcon project in sourceforge. Look to http://sourceforge.net/projects/tkinspect The current version has support for inspection when used with windows and can cope with incr tcl and namespaces. There is also tixinspect PT

And there's also the new INSPECT app which supports channels and slave interpreters and has more support for editing Incr Tcl than Tkinspect CRC [It's also disable-ware - tkinspect is maintained by PT and has nearly all the same abilities - Sourceforge RJ]

What about registered signal handlers? Generally overlooked, but an important behavioural characteristic of programs. I've always wished that sighandlers had become a standard 'section' of man pages.

ATW

I've always thought that TCL's greatest introspection feature was most powerfully demonstrated by the rmt demo. Being able to determine the command set of another application at runtime is pretty useful introspection. A nice extension to rmt would be to actually have it query the other app and build a menu dynamically!

ATW, again :)

EE: If you liked the rmt demo, you'll love tkcon... building a menu of available commands, though? why? just do [lsort [info commands]], and you've got the complete ordered list.

escargo 4 Dec 2002 - I was losing some sleep over some questions having to do with introspection and GUI Building Tools. If I have a GUI Building Tool, and I update my implementation of Tcl/Tk (say from 8.3 to 8.4), is it possible for the tool to know about new commands (e.g., panedwindow, labelframe), so that it can, through introspection, automatically be able to create GUIs using them.

Similarly, what would user-created megawidgets need to do to be able to make themselves known as widgets by the GUI Building Tools so that they could automatically be known.

In a full object system with inheritence or interfaces or protocols, new objects that obey the right protocol could be automatically managed. In Tcl and Tk, if I develop a new geometry manager, how would a GUI builder know that it existed and how to communicate with it?

Certainly individual commands that obey cget and configure will respond with their options, but how would a GUI builder know that a command produces a container or a widget that can be put into a container? Or that it's something as exotic as a new geometry manager that puts things into containers?

I see this as a way of using introspection to keep GUI Building Tools from becoming obsolete, and to easily be extended with new widgets (or even new OO extensions).

Event Loop: How to tell if there is an event loop running? How to tell if a new vwait might not be conflicting with existing vwaits? (See the bottom of vwait wiki page.)

TV Seeing this page come up on the recent changes , which may not be the most efficient or desirable viewing mode, but then again, when neither the server, bandwidth provider or certain track on a harddisc mind, why not, I was remembered that I put an issue up on the tk wished page, which is to my knowledge unresolved, and in my opinion interesting:

How can one in modern lingo serialize a widget, neat and cheap, as compressed as possible, given lets say the characteristics of one machine?

In very early versions if Tk I'd just store the conf information for widgets, produce complimentary widget creation commands, and I wouldn't need smalltalk much. At some point, that went, at first not hopelessly, wrong: various conf/option fields per option, unsettable options, apart from the obvious quoting and list issues (which can be resolved).

Since, I have tried to automatically save and restore the configuration of widgets, as bwise can save any canvas, but I don't think I spent a rewarding session hacking something together with pleasing overall behaviour, where I of course want it to work in general, and somewhat gracefully and with as little wonderfull of wieldy well laid out, probably expensive, according to the decency rules code to catch all cases. Just foreach widget {get conf data differeing from default} stuff.

Anyone?

PostgreSQL introspection is done by system tables pg_*.

Classes: info class and info object offer a large number of options for looking into the internals of classes and objects, in the new OO system included in tcl8.6 core. From inside a method, constructor or destructor body of a class, self gives information about how it was called.

Coroutines: info coroutine provides the name of a running coroutine, when called from inside the coroutine.

However the above bring up a question about the naming convention of tcl's introspection commands. Consider the following cases, which all give information about a block of code, when called from inside it:

namespace current: namespace

self: TclOO objects

info coroutine: coroutines

info level [info level]: procs

The semantics of these commands can be expressed more or less by a self command (and subcommands). Maybe it wouldn't be a bad idea to consider adding a self command to the above, as it would provide consistent command naming for that kind of introspection, separating inside (self) from outside (info) introspection contexts.

dzach 24-2-2010