pkgDeps

Difference between version 20 and 21 - Previous - Next
<<toc>>

**Introduction**

[DDG] 2020-02-25: Below is a small utility script which writes all `package require` calls to stdout. This is useful to check if making a release of your application which packages are required for your application. It shows recursively all packages on the terminal. I use this to create standalone files which contains all embedded modules before the actual application script using the script `mk_tm.tcl` at the bottom of the Wiki page [Another Tcl module maker]. So this helps in creating single file Tcl applications where the application is the last file and all modules files are added before.

Here is the code:

======
#!/usr/bin/env tclsh
# file pkgDeps.tcl

rename package package.orig

proc package {args} {
    set subcmd [lindex $args 0]
    if {$subcmd eq "require"} {
        puts stderr "package require $args"
    }
    package.orig {*}$args
}

if {[info exists argv0] && $argv0 eq [info script]} {
    if {[llength $argv] == 0} {
        puts "Usage: [info script] app.tcl ?other args?"
        exit 0
    }
    set ::argv0 [lindex $argv 0]
    set ::argv [lrange $argv 1 end]
    source $argv0 
}
======

**Example: standalone mkdoc**

Example call for the package [mkdoc::mkdoc]:

======
$ ./pkgDeps.tcl mkdoc.tcl
package require require Tcl 8.4
package require require Markdown
package require require Tcl 8.4
package require require Tcl 8.2
package require require Tcl
package require require Tcl
package require require textutil
package require require Tcl 8.2
package require require textutil::string
package require require Tcl 8.2
package require require textutil::repeat
package require require Tcl 8.2
package require require textutil::adjust
package require require Tcl 8.2
package require require textutil::repeat
package require require textutil::string
package require require textutil::split
package require require Tcl 8.2
package require require textutil::tabify
package require require Tcl 8.2
package require require textutil::repeat
package require require textutil::trim
package require require Tcl 8.2
...
======

Now my Makefile code to create the standalone application with all required packages in one file:

======
# Makefile
app:
        tclsh ../tools/mk_tm.tcl mkdoc 0.4 \
                any-script ../libs/textutil/string.tcl \
                any-script ../libs/textutil/repeat.tcl \
                any-script ../libs/textutil/adjust.tcl \
                any-script ../libs/textutil/expander.tcl \
                any-script ../libs/textutil/split.tcl \
                any-script ../libs/textutil/tabify.tcl \
                any-script ../libs/textutil/trim.tcl \
                any-script ../libs/textutil/textutil.tcl \
                any-script ../libs/markdown/markdown.tcl \
                any-script mkdoc.tcl
======

Using a [tclkit] interpreter, which does not(!) use the Tcl library on the current machine, I can check if the order of the files in the output follows the inverse calling order in `package require`. Means that because "Markdown" needs "textutils" the latter must be added first.

The Makefile can be further simplified as we don't have any binary extensions. So we can just use as well cat instead of the `mk_tm.tcl` script:

======
# Makefile
VERSION=0.4
mkdoc-module:
        cat ../libs/textutil/string.tcl \
                ../libs/textutil/repeat.tcl \
                ../libs/textutil/adjust.tcl \
                ../libs/textutil/expander.tcl \
                ../libs/textutil/split.tcl \
                ../libs/textutil/tabify.tcl \
                ../libs/textutil/trim.tcl \
                ../libs/textutil/textutil.tcl \
                ../libs/markdown/markdown.tcl \
                mkdoc.tcl > mkdoc-$(VERSION).tm
        echo "#!/usr/bin/env tclsh" > mkdoc-$(VERSION).app
        cat mkdoc-$(VERSION).tm >> mkdoc-$(VERSION).app
======

**Tracing the source command**

Let's extend the script a little bit further by tracing as well the source command.

**Tool: pkgDeps.tcl**

======
#!/usr/bin/env tclsh
# file pkgDeps.tcl

rename package package.orig
rename source source.orig

proc package {args} {
    set subcmd [lindex $args 0]
    if {$subcmd eq "require"} {
        puts stderr "package require $args"
    }
    package.orig {*}$args
}

proc source {args} {
    set file [lindex $args end]
    # be sure do show only source commands from not standard Tcl/Tk files
    if {[file tail $file] ne "pkgIndex.tcl"} {
        puts stderr "source $args"
    }
    uplevel 1 [list source.orig {*}$args]
}

if {[info exists argv0] && $argv0 eq [info script]} {
    if {[llength $argv] == 0} {
        puts "Usage: [info script] app.tcl ?other args?"
        exit 0
    }
    set ::argv0 [lindex $argv 0]
    set ::argv [lrange $argv 1 end]
    source $argv0 
}
======

**Tool: autoPather.tcl**

We now see as well which files are sourced.  Please note that all the pkgIndex.tcl sourcings are not displayed. As said before I prefer for testing during making standalone applications the Tcl script with a tclkit interpreter. The advantage using this interpreter is, that it ignores the system available Tcl libraries as well as the `TCLLIBPATH` variable. This at the same time a problem in testing. I often solved this by adding temporarily an manipulation of the `auto_path` to my scripts and remove it later. Sometimes you might however forget this removal. Similar to the approach above however there is better solution. Let's write another small dispatcher script which manipulates the `auto_path` variable.

====== #!/usr/bin/env tclsh
# file: autoPather.tcl
if {[info exists argv0] && $argv0 eq [info script]} {
    if {[llength $argv] == 0} {
        puts "Usage: [info script] pathname app.tcl ?other args?"
        exit 0
    }
    lappend auto_path [lindex $argv 0]
    set ::argv0 [lindex $argv 1]
    set ::argv [lrange $argv 2 end]
    source $argv0 
}
======

This small script accepts a pathname to be added to the `auto_path` variable and then calls the actual application script updating the necessary variables `$argv0` and `$argv`.

So you can call if in libs are your local library files for instance with:

======
 tclkit autoPather.tcl ../libs myapp.tcl arg1 arg2
======

That's nice already, but even better we can combine it with our script `pkgDeps.tcl`

======
 tclkit autoPather.tcl ../libs pkgDeps.tcl myapp.tcl arg1 arg2
======

Now we see which packages are required and which files are sourced. To see only the files which are used within ../libs we can as well filter the `stderr` channel.

======
tclkit autoPather.tcl ../libs pkgDeps.tcl myapp.tcl arg1 arg2 2>&1 >/dev/null | grep '../libs'
======

As my programm needs the [snit] package we see for instance the lines:

======
source ../libs/snit/snit2.tcl
source ../libs/snit/main2.tcl
source ../libs/snit/validate.tcl
======

**Example: standalone hyperhelp**

Now let's do a sample standalone application for the [hyperhelp] library/application.

======
tclkit-8.6.10-min autoPather.tcl ../libs pkgDeps.tcl hyperhelp.tcl hyperhelp-docu.txt 2>&1 >/dev/null | grep '../libs'
source ../libs/dgtools/shistory-0.2.tm
source ../libs/snit/snit2.tcl
source ../libs/snit/main2.tcl
source ../libs/snit/validate.tcl
source ../libs/dgw/dgwutils-0.2.tm
======

This lists our required files.

Our final application script is basically just merge of all the files listed above and finally our application script. So we must disable all source calls within the files listed above. So create our Makefile task:

======
app-hyperhelp:
        # so let's strip of source commands from non-pkgIndex.tcl files
        grep -vE '\s*source' ../libs/snit/snit2.tcl > snit2.tmp
        # strip comments and documentation from helperfiles
        grep -vE "^\s*#'?" ../dgtools/shistory.tcl > shistory.tmp
        grep -vE "^\s*#'?" dgwutils.tcl > dgwutils.tmp
        cat snit2.tmp \
                ../libs/snit/main2.tcl \
                ../libs/snit/validate.tcl \
                shistory.tmp \
                dgwutils.tmp \
                hyperhelp.tcl > hyperhelp-0.8.1.tm
        echo "#!/usr/bin/env tclsh" > hyperhelp-0.8.1.app
        cat hyperhelp-0.8.1.tm >> hyperhelp-0.8.1.app
        # clean up files
        -rm *.tmp
======
This produces a standalone file containing the [snit] library as well as code from the `shistory.tcl` and the `dgwutils.tcl` file. At the end our actual hyperhelp code is added:

======

 [bariuke dgw]$ ls -lth hyperhelp-0.8.1.app 
 -rwxr-xr-x. 1 grth grth 237K Feb 28 07:23 hyperhelp-0.8.1.app

======

To test we repeat our tclkit command but without the `autoPather.tcl` script:

======
 tclkit hyperhelp-0.8.1.app hyperhelp-docu.txt 
======

This would as well work on a machine using the standard `tclsh`/`wish` interpreter where only the standard libraries are installed. So this will as well work:

======
 chmod 755 hyperhelp-0.8.1.app
 cp hyperhelp-0.8.1.app ~/bin/hyperhelp
 hyperhelp hyperhelp-docu.txt 
======

The resulting application can be downloaded from here: https://chiselapp.com/user/dgroth/repository/tclcode/wiki?name=releases
**TODO's**

   * can we do compression of the app-file?
   * example with more complex packages
   * example with embedded binary files, images etc. 

---
**Discussion**

Please discuss here:

[bll] - 2020-02-25: I actually have a script that traverses all of my modules, creates a dependency
list, runs a topological sort on it (using 'tsort'), then rewrites each module's list of 
'package require' statements in dependency order.   It warns of missing dependencies and unused 
dependencies.   It's by no means perfect, but generally works well enough
for occasional use.  It doesn't handle classes and instantiation very well,
there's some hard-coded data in there to handle some of my classes and other oddities.
It's also rather dependent on my style of fully qualifying my procedure calls.
I use '::uiutils::initUI' for example, I don't use namespace imports.

[DDG] - 2020-02-25: Yes, my approach above certainly only is useful for small apps with a few dependencies. For larger applications your approach, wrapping into [starkit]s, [freewrap], [thatch] etc. might be more useful. However I felt that in many cases this simple sample script above might be a good starter for small console applications.

[DDG] - 2020-02-28: Extending `pkgDeps.tcl` with source, adding `autoPather.tcl` and tutorial for packing [hyperhelp], replacing `mk_tm.tcl` with simple `cat` command. 

<<categories>> Application | Deployment | Dev. Tools