pkg_mkIndex pitfalls

GWM Jul 07 - including pkg_mkIndex in your tcl source can cause problems. I have a project where I am loading a number of packages in other tcl files, so I call pkg_mkindex. However if you include the main.tcl file in the pkg_mkindex, you get an infinite loop. Here is a very simple tcl file which goes into an infinite loop:

console show
update
puts "Making index"
pkg_mkIndex [pwd]
puts "Made index"

Note that NO packages are loaded, nothing useful occurs and we never reach the Made Index statement. My cure is to place all the packages in a subdirectory (here called plugins), when

pkg_mkIndex -verbose [file join [pwd] plugins] *.tcl

works fine. An alternative source which prevents the recursion (as 'info script' is a relative path after the top level) is:

console show
update
if {[file dirname [info script]] != "."} {
    puts "Making index"
    pkg_mkIndex [pwd]
    puts "Made index"
}

If a script in the library directory fails for any reason when it is sourced, then any subsequent package provide statements in that script will not be reflected in the pkgIndex.tcl file. Probably the most common cause for failure is that a script calls package require for a dependent package which cannot be loaded. For example, package scripts which package require Tk will fail (on Unix) if the $DISPLAY is not set.

pkg_mkIndex will fail if the required package defines namespaces or procedures which the file being indexed later uses at top-level. For example: if foo.tcl contains:

namespace eval foo { ... }
proc defineFoo {name args} { ... }
package provide foo 1.0

and bar.tcl contains:

package require foo 1.0
defineFoo barFoo ...
package provide bar 1.0

then the call to defineFoo will fail and the autoindexer will abort. If bar.tcl tries to use the namespace foo at top-level it will also fail.

To avoid this problem, you should call package provide before doing anything else.


DGP rebuts Just my opinion, but I think your package file should serve the package user first, and consider the indexer's needs less important. The user wants to know that when package provide foo returns a version number, that that version has been fully loaded. That implies that package provide foo $version should come last.

    package require IDependonThis 1
    package require AndThisOneToo 3.3
    package require YepAnotherOne 2.3.1
    namespace eval myPackage {
        # Still holding back that official release...
        variable version 0.9.9.9
        namespace export foo bar baz

        # Brilliant code goes here

        package provide [namespace tail [namespace current]] $version
    }

It is a very good idea for package authors to include prebuilt pkgIndex.tcl files in the distribution instead of having end-users run pkg_mkIndex as part of the installation process. There are too many things that can go wrong. The canonical TEA build structure (as of May 2002) uses pkg_mkIndex to build the package index as part of the installation process. Don't do that.

More often than not, pkg_mkIndex is overkill anyway: for simple packages it's just as easy to create a pkgIndex.tcl (or pkgIndex.tcl.in) by hand.

The biggest problem with "doing it by hand" is maintenance of a version number--and this is a serious practical nuisance. Several interesting proposals for automation are floating around. As of the weeks leading up to the 2002 Conference, though, packaging guru DGP says, "If you want to follow my example instead of advice, place a comment where the version number is defined reminding you of the other places that must be kept in sync."


pkg_mkIndex does not do dependency checking. It explicitly avoids the issue.