http://www.tcl.tk/man/tcl8.5/TclCmd/tm.htm Benefits: * simplified deployment of packages (with some necessary formalization of layout) * speed: no filesystem search in subdirectories of $auto_path directories (there are often very many such subdirectories) * speed: no need to read/parse pkgIndex.tcl * … Purpose: Discuss the purpose, and implementation, of [TIP]#189 [http://tip.tcl.tk/189], TIP#190 [http://tip.tcl.tk/190], and TIP#191 [http://tip.tcl.tk/191] . Note this is something totally different from the Tcl [Modules] package. ---- In brief, "modules" is a vastly simplified approach to software packaging from 8.5. A Tcl module is a package in a single file that can be [source]d. (DLLs may still be embedded in that). There is no [pkgIndex.tcl] file, the package name and version are expressed in the module filename itself, so that package ''foo'' in version 3.14 must be named foo-3.14.tm [RS] 2007-08-17 ---- [JMN] 2008-03-15 TIP 189 provides for user specific paths through the use of an environment variable $::env(TCLX.y_TM_PATH) It seems however that in Apache, (supposedly for 'portability reasons') environment variables may contain only letters, numbers and the underscore character. Apache translates other characters to the underscore. This means for example that TCL8.5_TM_PATH may appear as TCL8_5_TM_PATH for cgi scripts and so the tcl::tm::list is not adjusted. I hacked my way around this by adjusting tcl::tm::Defaults (in lib/tcl8.5/tm.tcl ) 2008-03-24 Note that Andreas Kupries fixed this ( bug 1914604 ), so versions 8.5.2 & later will accept both forms of the environment variable. ---- [LV] 2007 Aug 30 * Why were Tcl modules created when [starkit]s already existed? What is the advantage intended for modules over starkits? * What tools are included in the core Tcl for building Tcl modules and extracting the code inside of Tcl modules? * What does a developer have to take into consideration when developing code that would eventually be placed into a tcl module? According to http://tip.tcl.tk/190 , a simple tcl script only package, consisting of only one file, is already a tcl module (though there are, I believe, some requirements about the filepath and file name for tcl to locate it). According to the same web page, for a simple package consisting only of tcl scripts in the same directory, all one has to do is concatenate the files into a single file (following, of course, the same distribution file naming conventions). * What special debugging and programming techniques have developers found useful when develop code residing in a tcl module? ---- [LV] I've seen people ask whether [zip] files will be supported as tcl modules; I myself am hoping to see [starkit]s as tcl modules. ---- [jmn] 2005-09-13 ::tcl::tm::add can have unintuitive (IMO) results. tip189 clearly states that paths added last are 'looked at' first - but it doesn't make it clear that a package of the same version in a later path will 'override' the earlier entry in the 'package ifneeded' database. This may be of concern if you are trying to create a local cache of packages with a fallback to your main repository on a (slower) network share - or if you are trying to force use of a particular module file during testing (without hacking at versions). 'package forget' won't help here - in contrast to the standard auto_path system where a path inserted into the head of $auto_path WILL allow loading of this package in preference to the one found later in the list (assuming either auto_path wasn't scanned prior to the addition, or you've called 'package forget' after the addition) . Is this behaviour intended? I think it'd be really nice to have some way (that works consistently for both module & auto_path systems) to override existing entries in the 'package ifneeded' database - for just those packages in a particular folder. e.g a 'package scan ' command might add just the packages found in - replacing any previous entries in the database. Otherwise - some better introspection of the 'package ifneeded' database would be nice.. It's currently quite confusing to debug package loading issues so if you could do something like a 'package ifneeded *' and see what it holds - that might help. [Lars H]: Do you mean something like this? proc package_ifneeded_* {} { set res {} foreach pkg [package names] { set L {} foreach ver [package versions $pkg] { lappend L $ver [package ifneeded $pkg $ver] } lappend res $pkg $L } set res } Example: % join [lrange [package_ifneeded_*] 0 5] \n rcs 0.1 {source /Library/Tcl/tcllib1.8/rcs/rcs.tcl} logger 0.6.1 {source /Library/Tcl/tcllib1.8/log/logger.tcl} SOAP::Utils 1.0 {source /Library/Tcl/tclsoap1.6.7/utils.tcl} [jmn] Thanks yes - of course you're right.. the package command does provide enough basics to build your own view of the database as you require - I just think something similar to your snippet would be a nice inclusion to the standard distribution. While I should have known better and just built such a thing - I think for new users (or even just anyone a little unsure of how it all works) the package mechanisms have enough tricky subtleties that any extra introspection help that can be provided would be good. Anyway.. this is an aside to the main issue I'm having: that I can't get the 'package require' system to use a package earlier in the module path if the package versions are the same. The theory may go that if the package versions are the same it shouldn't matter - but there may be administrative reasons to need this to behave properly - such as repository access speed.. or maybe a c-coded version to take higher precedence than a tcl-only version. [MG] Perhaps an addition to the way [[package require]] works would help. For instance, if you could do package require -path [list "/path/to/thisfile" "/path/to/thatfile"] myPackage 2.0 to either have only those paths checked, or to have those checked first, followed by all the usual places, it might solve your problem. That would also allow you to [[package require]] a package which you know resides in a certain place, on its own, without having to add that path into your $auto_path (which is checked for ''all'' packages, slowing things down a little, as I understand it). Another simpler solution to the C vs. Tcl versions is to just add another level to the version number. For instance, call a C version of a package version 2.0.2 and the pure-Tcl (slower) version 2.0.1, so that a [[package require myPackage 2.0]] would give priority to the C one. [LV] So, does this mean that the order of [auto_path] doesn't control the order that package require searches for a package to load? It does seem that it would be useful if one could say package require /path/to/specific/version/of/extension and only that would be used. That way, someone could test a new version of a package without it going into the ''live'' environment with possible negative consequences. Or one could code the above if there appeared to be some sort of problem with a package, to determine for certain whether a particular directory was the one causing a problem. [Lars H], 15 Oct 2005: One way to make sure a particular instance of a package is loaded would be to first explicitly [source] its [pkgIndex.tcl] file, like so: set dir /path/to/specific/version/of/extension source [file join $dir pkgIndex.tcl] package require packageName exactVersion But mostly that is going the wrong way. A major reason [package] is useful is that it relieves the programmer of having to know exactly where packages are located, so rolling that information back into its API would mess things up again. ''[escargo] 14 Oct 2005'' - Comparing the [package] mechanism to the executable search in [Unix] and [Linux], maybe we need something like a '''which'''[http://www.ss64.com/bash/which.html] or '''whereis'''[http://www.ss64.com/bash/whereis.html] command (as subcommands of package). In a way this is a kind of [introspection] of the execution environment. ---- [tje] Okay, I'm going nuts here... I made a simple module 'foo-1.0.tm' under directory 'a', like this: /_path_/ a/ foo-1.0.tm I then started my tcl shell (8.5a4, a tclkit) and issued this command: ::tcl::tm::add /_path_/a It shows up at the head of the list returned by ::tcl::tm::list now. So, what's the proper way to source this module without giving the absolute path? I've tried sourcing foo, foo-1, foo-1.0, foo.tm, foo-1.tm, and foo-1.0.tm, all to no avail. '''ANSWER:''' package require foo 1 ---- [RT] 12Jan2007 Would be nice having a wiki page that truely does explain modules but it's not this page. [LV] If you feel that explanations are missing here, please add questions so that the people who understand tcl modules can provide that information here. ---- So I went to TIP #189 which does seem to explain pretty well. Then my attention was caught by this RE ([[:alpha:]_][:[:alnum:]_]*)-([[:digit:]].*)\.tm which describes legal module file names. Thing is, it permits a colon in the name. Colons are totally illegal in DOS/Windows file names. I think this is a nasty booby trap for anyone wanting to bring a package from *nix world to Windows. Is a colon really worth giving up portability of module file names? - ''[RS]: TIP 189 says that the string :: in a Tcl module name is replaced with the platform's path separator, so module '''foo::bar 1.0''' will on Unix/Linux resolve to the pathname foo/bar-1.0.tm '' Not only that, but notice after the - that it specifies one digit followed by any character whatsoever. That's likely to present various unplesant surprises... - ''[RS]: Note the other condition that the version string must be acceptable to [package vcompare], which limits them to digits and periods.'' ---- See also * [Distribution Mechanisms] * [tm] * [Tcl Module creation script] * [Another Tcl module maker] * [tarpack] for an experiment in tar archives that are also sourceable Tcl Modules. * [Tcl Software Libraries] ---- [[[tb]]] 2008-08-06 - Hi, just came across this page. To me the concept of starkits fullfills any needs for modularization. If I ''require'' the packages of a starkit in the kit's main.tcl, then I have the complete module functionality with that kit. I simply source it and have the containing packages loaded, ready for usage. Allthough, having that lot of different semantics for starkits might be confusing? Maybe we should distinguish between: * modkit - A module - package are required, when the kit is sourced * libkit - A Library - making packages available * appkit - A [tclkit] application * starpack - An executable Otherwise it would be nice to have ''require from '' and remove the need for module kits. Just my thoughts :) [Ro]: I think breaking it down into those terms fractures the term starkit too much. Honestly though, I'm with you, starkits fulfill my modularization needs; I don't plan on using Tcl modules. Hell usually I don't even bother and just copy the necessary library into my application, for most of my apps disk space isn't that limited, and I just don't have the energy or time to debug library mismatches across applications. DLL Hell is over for me. That way I have a single app, with all it's necessary libraries, the exact interpreter binary that runs it, all together in one directory. I don't get bitten by interpreter or library bugs, changes or whatever. If it works today, it'll work tomorrow, the app is not in flux. ---- !!!!!! %| [Category Deployment] | [Category Repository] | [Category Distribution] |% !!!!!!