&| '''What''': | ''Tcl-Compiler'' |& &| '''Intro''': | http://nhi1.selfhost.co/wiki/Tcl-Compiler |& &| '''Description''': | The ''Tcl-Compiler'' compiles ''Tcl-Scripts'' into native ''C-Code'' |& &| '''Updated''': | 26 feb 2021 (reopen this project) |& &| '''Contact''': | mailto:aotto1968@t-online.de (Andreas Otto) |& ** Links ** * [http://nhi1.selfhost.co/Token_Stream_Technology %|%What is the Token Stream Technology ?] * [http://sourceforge.net/projects/tclcompiler %|%source-code at sourceforge.net] * [http://nhi1.selfhost.co %|%Compiler Factory] * [http://nhi1.selfhost.co/Tcl-Compiler_-_PERFORMANCE_TEST %|%POWER-Tcl performance figures] ** HOWTO PAGES ** * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_PROJECT.html %|% HOWTO setup a project] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_COMPILE.html %|% HOWTO compile a TCL package into a SHARD LIBRARY] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_INLINE_C.html %|% HOWTO write INLINE C code] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_MIC_TCL_AND_C.html %|% HOWTO mix TCL and C] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_MAKE.html %|% HOWTO use the make tool] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_SOURCE.html %|% HOWTO source] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_REPOSITORY.html %|% HOWTO use a repository] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_PLUGIN.html %|% HOWTO plugin] * [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_VPATH.html %|% HOWTO VPATH] ** Updates ** *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 09 June 2021 -> only small updates ] *** 1. spend 2 days on simple CORE-DUMP, later figure out that 'package unload Tcl' was the problem 1. add new POWER-Tcl package code (old code was 20 years old :-) -> now much smarter Start to rename parts of the software, the POWER-Tcl-!!PACKAGE!! is now called LIBRARY 1. a LIBRARY has usually a `project.cct` file, a file system structure and one or more files to compile or to install. 1. a LIBRARY is the distribution format of POWER-tcl code and binaries. 1. a LIBRARY is installed into the POWER-tcl runtime with `cct pkg --add ...` (the `pkg` may be changed later into `lib`) *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 01 June 2021 -> the BUILD tool get more power, today the TEST power ... ] *** ======none Compiler3> cct build --test ALL BUILD MSG : start in directory -> COMPILER3_SOURCE/. BUILD MSG : Tests began at -> Tue Jun 01 22:59:09 CEST 2021 BUILD MSG : Tcl 8.3.4 tests running in interp -> COMPILER_HOME:./exe/linuxi386/bin/cctsh8.3 BUILD MSG : CctTclsh - test running in working dir -> ./cmp/CctTclsh/tests Only sourcing test files that match: *.test tclsh.test ++++ tclsh-1 PASSED ++++ tclsh-1-b PASSED ++++ tclsh-2 PASSED ++++ tclsh-2-b PASSED ++++ tclsh-3 PASSED ++++ tclsh-3-b PASSED ++++ tclsh-3-c PASSED ++++ tclsh-4 PASSED ++++ tclsh-5 PASSED BUILD MSG : Md5sum - test running in working dir -> ./cmp/Md5sum/tests Only sourcing test files that match: *.test md5sum.test ++++ md5sum-1-1 PASSED ++++ md5sum-1-2 PASSED ++++ md5sum-1-3 PASSED ++++ md5sum-1-4 PASSED ++++ md5sum-2-1 PASSED ++++ md5sum-2-2 PASSED ++++ md5sum-2-3 PASSED ++++ md5sum-2-4 PASSED ++++ md5sum-3-1 PASSED ++++ md5sum-3-2 PASSED ++++ md5sum-3-3 PASSED ++++ md5sum-3-3 PASSED BUILD MSG : Blowfish - test running in working dir -> ./cmp/Blowfish/tests Only sourcing test files that match: *.test blowfish.test ++++ blowfish-1-1 PASSED ++++ blowfish-1-2 PASSED ++++ blowfish-1-3 PASSED ++++ blowfish-1-4 PASSED ++++ blowfish-1-5 PASSED ++++ blowfish-1-6 PASSED ++++ blowfish-1-7-1 PASSED ++++ blowfish-1-7-2 PASSED BUILD MSG : Tests ended at -> Tue Jun 01 22:59:10 CEST 2021 TEST RESULTS : Library Total Skipped Passed Failed : ---------------- -------- -------- -------- -------- : CctTclsh 9 0 9 0 : Md5sum 12 0 12 0 : Blowfish 8 0 8 0 ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 31 Mai 2021 -> my today problem, something eat's a byte ... ] *** ======tcl test blowfish-1-7-1 {IN stdin, OUT stdout, io in ascii and string} { exec cct blowfish --dec --key 3456 \ --str [exec echo "HelloWorld" | cct blowfish --enc --in stdin --key 3456 ] } {HelloWorld} test blowfish-1-7-2 {IN stdin, OUT stdout, io in binary and string} { exec cct blowfish --dec --binary --key 3456 \ --str [exec echo "HelloWorld" | cct blowfish --enc --binary --in stdin --key 3456 ] } {HelloWorld} ====== hmmmm ======none Compiler3> cct build --test Blowfish -- -match "blowfish-1-7-*" Tcl 8.3.4 tests running in interp: .../Compiler3.BUILD/lib/JavaKiller/exe/linuxi386/bin/cctsh8.3 Tests running in working dir: .../Compiler3/cmp/Blowfish/tests Only running tests that match: blowfish-1-7-* Only sourcing test files that match: *.test Tests began at Mon May 31 21:07:38 CEST 2021 blowfish.test ++++ blowfish-1-7-1 PASSED ==== blowfish-1-7-2 IN stdin, OUT stdout, io in binary and string FAILED ---- Result was: HelloWor& ---- Result should have been: HelloWorld ==== blowfish-1-7-2 FAILED Tests ended at Mon May 31 21:07:38 CEST 2021 TestDriver.tcl: Total 8 Passed 1 Skipped 6 Failed 1 Sourced 1 Test Files. Files with failing tests: blowfish.test ====== The problem is the `fconfigure`, if I delete it it works '''but''' other tests fail. ''good'' the default ''ascii'' feature (convert all binary in hex) works, the ''--binary'' option is more an addon and there seems to be a '''old''' tcl-bug. ======tcl set OUT [ ::FileLib::Open $VALUE(--out) w ] if { $VALUE(--binary) } { fconfigure $OUT -encoding binary -translation binary if { $VALUE(--enc) } { set ret [ ::crypt::bf_encrypt_bin $STR $KEY ] } else { set ret [ ::crypt::bf_decrypt_bin $STR $KEY ] } puts -nonewline $OUT $ret } else { if { $VALUE(--enc) } { set ret [ ::crypt::bf_encrypt $STR $KEY ] } else { set ret [ ::crypt::bf_decrypt $STR $KEY ] } puts $OUT $ret } flush $OUT ::FileLib::Close $OUT ====== It seemst to be ''no'' problem with the ''tcl'' `exec` because the `bash` show the problem too ======none Compiler3> cct blowfish --dec --binary --key 3456 --str "$(echo "HelloWorld" | cct blowfish --enc --binary --in stdin --key 3456)" | xxd 00000000: 4865 6c6c 6f57 6f72 26 HelloWor& ====== and more simple... ======none Compiler3> cct blowfish --dec --binary --key 3456 --str "$(cct blowfish --enc --binary --str "HelloWorld" --key 3456)" | xxd 00000000: 4865 6c6c 6f57 6f72 26 HelloWor& ====== now final simple… the ''--enc'' seems to be ok. ======none Compiler3> cct blowfish --enc --str "HelloWorld" --key 3456 a3302be41886d310c68c Compiler3> cct blowfish --enc --str "HelloWorld" --key 3456 --binary | xxd 00000000: a330 2be4 1886 d310 c68c .0+....... ====== I thing the problem is not the ''--binary'', because the ''binary-only-test' is always ok ======none Compiler3> echo $(cct blowfish --enc --str "HelloWorld" --key 3456 --binary | cct blowfish --dec --key 3456 --binary) HelloWorld ====== The problem is the ''binary to --str'', it seems that ''tcl'' always expect ''utf'' as string and do some ''conversion''. If I delete the `fconfigure` than tcl is doing ''output (binary) to utf'' and this fit into the ''--str''. But without `fconfigure` the binary only test fail because the ''output (binary) to utf'' not fit the expected binary. every bug create a new feature, and this feature is the ''--binary-to-utf'' option ======none Compiler3> echo $(cct blowfish --dec --binary --key 3456 --str "$(cct blowfish --enc --binary-to-utf --str "HelloWorld" --key 3456)") HelloWorld ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 30 Mai 2021 -> nice update for `cct tclsh` ... ] *** '''example :''' Simple ''tcl'' one-liner -> but the background is important. ======none cat /some/file | cct tclsh '::FileLib::Read stdin' ====== The `cct tclsh` eval the argument `::FileLib::Read stdin` as script and call `tclsh-unknown` if something is missed. The `tclsh-unknown` uses the following evaluation test order : 1. test if it is a file (script) or a ''power-tcl'' special-build-in 1. test if it is a command from a package which is '''not''' already loaded 1. test if it is a command from a auto_index database (auto_load) 1. test if it is a ''OS'' command (auto_execok) 1. final call ''original'' `unknown` to handle default lookup like (auto_load) Important is that the command `::FileLib::Read` is in a package '''not''' loaded by default. The `cct tclsh` replace the default `unknown` to a `tclsh-unknown` which is able to load missing package first. !!!!!! By default the ''package-name'' in ''power-tcl'' is the same name as the ''first'' namespace, in this case ''FileLib''. !!!!!! After the package is loaded successful than the `::FileLib::Read stdin` read the data from `stdin` and return the data as large string. The `cct tclsh` trimright the final '\n' from output and print the result to `stdout`. some examples from the default plugin test, called with: `cct tclsh --test` ======tcl set cct [info nameofexecutable] set echo [auto_execok echo] test tclsh-1 {eval string from commandline, return result} { exec $cct tclsh {set otto 78bb} } {78bb} test tclsh-1-b {eval string from commandline as args, return result} { exec $cct tclsh set otto 88oo } {88oo} test tclsh-2 {eval file from argument, return stdout from file} { makeFile {puts 12345} tclsh-2.txt exec $cct tclsh tclsh-2.txt } {12345} test tclsh-2-b {eval file from argument with additional argument} { makeFile {puts fff666-$argc-$argv} tclsh-2-b.txt exec $cct tclsh tclsh-2-b.txt xxx yyy zzz } {fff666-3-xxx yyy zzz} # attention expect SCRIPT not DATA from stdin test tclsh-3 {eval script from stdin, return stdout from script} { exec $echo {puts abcd} | $cct tclsh --eval-stdin } {abcd} test tclsh-3-b {eval script from stdin, return result from script} { exec $echo {set otto abcdxyz} | $cct tclsh --eval-stdin } {abcdxyz} test tclsh-3-c {eval script from stdin, return stdout AND result from script} { exec $echo {puts yyt ; set otto 444} | $cct tclsh --eval-stdin } {yyt 444} test tclsh-4 {autoload package, read text from stdin and return result from ::FileLib::Read} { exec $echo ghij | $cct tclsh {::FileLib::Read stdin} } {ghij} test tclsh-5 {exec external command} { exec $cct tclsh {echo pp23} } {pp23} ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 29 Mai 2021 -> testing, testing, testing ... ] *** last 12 days was more testing and bug fixing, not much code added. ======none Compiler3> git log --format= --numstat --after="mai 17" | awk '{add+=$1; del+=$2} END {print "add=",add," : ","del=",del}' add= 9688 : del= 15356 ====== Today I found the first '''real''' bug in old ''Compiler'' source, this happen in : ======tcl catch { uplevel #0 { ... } } ====== if `{ ... }` return with an error. The problem was that ''#0'' set the `varFramePtr = NULL` and was NOT reset. ======c int Ot_UplevelObjCmd(interp, level, refv) Tcl_Interp *interp; /* Current interpreter. */ int level; /* the call level */ Ot_X_Cmd_Proc *refv; /* Dynamic pointer Structure */ { register Interp *iPtr = (Interp *) interp; int result; CallFrame *savedVarFramePtr, *framePtr; Tcl_ResetResult(interp); iPtr->cmdCount++; /* * Find the level to use for executing the command. */ result = Ot_TclGetFrame(interp, level, &framePtr); if (result == TCL_ERROR) { return TCL_ERROR; } /* * Modify the interpreter state to execute in the given frame. */ savedVarFramePtr = iPtr->varFramePtr; iPtr->varFramePtr = framePtr; // iPtr->varFramePtr=NULL if level=0 /* * Execute the residual arguments as a command. */ result = (*refv)(interp); if (result == TCL_ERROR) { // old bug: 'return TCL_ERROR;' -> return does NOT set iPtr->varFramePtr back to savedVarFramePtr result = TCL_ERROR; } /* * Restore the variable frame, and return. */ iPtr->varFramePtr = savedVarFramePtr; return result; } ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 27 Mai 2021 -> test add missing features ] *** some small updates are ongoing… **** add plugin ''view'' as general purpose document viewer **** ======none > cct view --mlist mime : viewer : args : regexp : description ----- : ---------- : ----- : --------------- : ----------------- html : firefox : : .*[.]html? : html files pdf : firefox : : .*[.]pdf : pdf files info : info : -f : .*[.]info : info files man : man : -l : .*[.][1-9n] : man files plain : less : : [^.]+ : plain files ====== **** add plugin ''man'' as more special document viewer for man pages **** ======none Compiler3> cct man tcl tcldesjr.n tcldocstrip.n tcllib_dns.n tcllib_fifo.n tcllib_ip.n tcllib_msgcat.n tcllib_random.n tcllib_try.n tcllib_zero.n tcl_parse.n tclreadline tcltest.n tcldes.n tcllib_coroutine.n tcllib_fifo2.n tcllib_interp.n tcllib_memchan.n tcllib_null.n tcllib_string.n tcllib_variable.n tcllib_zlib.n tcl_prefix.n tclsh.1 ====== possible good news about ''tcl'', able to add ''type'' definition without break ''old'' code. ======tcl .../Compiler3> cct arg.test list element in braces followed by "arg1" instead of space while executing "proc otto { {L}arg1 {S}arg2 } { puts hallo }" (file "arg.test" line 16) ====== also possible ======tcl declare {I}counter {W}bigcounter {{I}L}ListOfInterger {A}keyBook ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 26 Mai 2021 -> add new technology. ] *** Every day new code, Every day a new feature. On top of the already existing ''power-tcl'' license code I add something new, I don't even have a good name for it. possible: power-tcl-guard The ''power-tcl-guard'' technology is something to control HOW and WHERE a software is used. It is more like an extension of the already available ''operating-system-security'' but full under control of the customer and '''not''' under control of ''something or somebody else''. The ''power-tcl-guard'' technology is the ''first'' technology in duty on application startup (ref ''power-tcl-boot'') and provide the possibility to '''stop''' or to '''log''' the software usage. What is possible ? 1. allow execution of the software ONLY if a special condition is met 2. protocol every start of a software to a logging server 3. really grantee in-house security by force, stolen software is useless 4. no user interaction required. 5. probably more... **** morning problem **** I locked myself out. ======none ../Compiler3/src/CctRt> OtBuild.sh test LICENCE ERROR (Power-Tcl): licence file not available please order a new licence with the public key : >>> linuxi386/Power-Tcl#d7c316233e44c4a853348ef6194cc473 <<< please contact : xxx@yyy more info at : http://nhi1.selfhost.co/Tcl-Compiler ====== ok, got it ======none > cat tmp.sig | cct key --action decrypt LICENCE OK (Power-Tcl): >>> Power-Tcl.linuxi386#3ad1d6510298718a559f403a7bf09ec1 <<< VALID: Fri May 28 20:56:28 CEST 2021 ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 25 Mai 2021 -> I really like my error messages ] *** This is nothing special just a file, mentioned in the `project.cct`, is missing. I really like the information in the message, the stack and the layout. ======none > cct make c CONFIG ERROR [::Config::Is::File] : PROJECT FILE_TCL -> 'internal.tcl' file does not exists STACK >>> ::Config::Is::File {PROJECT FILE_TCL} internal.tcl ... ::Config::Is::Test {PROJECT FILE_TCL} internal.tcl {} ... ::Config::Values::Read FILE_TCL internal.tcl {} ... ::Config::Interp::Source .../Compiler3/cmp/Key/project.cct ... ::Config::Values::Source .../Compiler3/cmp/Key/project.cct ... ::Config::read .../Compiler3/cmp/Key/project.cct 1 ... ::Config::Main @project.cct -cache yes ... ::IConfig::ProjectL * @project.cct ... ::CctMake::Make ... ::CctMake::Main ... ::Main ... file: Cct/Main.tcl ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 24 Mai 2021 -> more updates on "Key" ] *** In the past the '''Key''' feature was only the license-key but now the '''Key''' feature expanded. 1. The '''Key''' feature is a mixture of ''tcl'' and ''C'' code compiled into a power-tcl-binary. 1. The '''Key''' feature is '''always''' available as FIRST power-tcl code. 1. A lot of real '''basic''' features are moving into the '''Key''' feature to be available very early and without a `package require ...`. **** The ''boot-order'' of power-tcl with ''boot'' feature enabled **** %|function |decription | old | new |% &|Cct_Main |called from `main`, parse arguments | `generic/tclMain.c` | `generic/cctMain.c` |& &|Tcl_AppInit |init tcl & packages | `unix/tclAppInit.c` | `unix/cctBootInit.c` |& &|Key |''key'' check & ''power-tcl'' init | - | `cmp/Key/KeyCct` |& &|Tcl_Init | call for every '''new''' interp | - | - |& &|InitScript | setup tcl environment | `generic/tclInitScript.h` | `cmp/InitScript` |& &|TclInit_init | setup tcl, former `init.tcl` | `lib/tcl8.3` | `cmp/TclInit` |& &|PATTERN_boot_Init | init `package ineeded` & `auto_index` | - | `cct build --pattern PATTERN --boot .../cct8.3.4/unix` |& *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 22 Mai 2021 -> BOOT loader technology expanded ] *** The ''boot-loader'' now link into the ''TCL'' `Makefile.in` with just a few lines at the end: ======none # # add support for 'boot' # > cct build --pattern CCT --boot-dir ... # TCL_BUILD_LIB_SPEC = @TCL_BUILD_LIB_SPEC@ TCL_LD_SEARCH_FLAGS = @TCL_LD_SEARCH_FLAGS@ include export/boot.make # DO NOT DELETE THIS LINE -- make depend depends on it. ====== What it does? The '''POWER-Tcl-boot-loader''' technology does the following thing : 1. the ''pattern'' define the tcl packages be part of the bootloader 2. compile the tcl packages into power-tcl-binary-archive (libPackageX.Y.a) 3. compile a new tclsh with name ''pattern'' and ''static'' link the power-tcl-binary-archive into the shell 4. add all code to either `package require` or `auto_load` the code. 5. finally a single executable with name ''pattern'' (lowercase) is available with tcl and all ''pattern'' packages included The following ''makefile-targets'' are added +++ boot-binaries: extend ''binaries'' and add all ''pattern'' (multiple pattern are allowed) boot-install: extend ''install-binaries'' boot-clean: extend ''clean'' pattern-clean: extend ''boot-clean'' pattern-install: extend ''boot-install'' pattern: extend ''boot-binaries'' +++ The ''pattern'' binary is using the '''POWER-Tcl''' file `cctBootInit.c` as replacement for `tclAppInit.c`. typical ''Morning-Job'' ======none Compiler3> git commit -a -m "more on build and boot" [master b3d3189] more on build and boot 37 files changed, 1452 insertions(+), 938 deletions(-) create mode 120000 bin/cctsh create mode 100644 cmp/Build/db.tcl create mode 100644 cmp/Build/list.tcl create mode 100644 cmp/Build/make.tcl create mode 100644 cmp/Build/pkg.tcl create mode 100644 cmp/Build/prj.tcl rewrite cmp/Build/project.cct (72%) create mode 100644 src/CctRt/cct8.3.4/generic/cctMain.c rename src/CctRt/cct8.3.4/unix/{tclBootInit.c => cctBootInit.c} (99%) ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 19 Mai 2021 -> BOOT loader update ] *** The '''POWER-Tcl-boot-loader''' is close to finish, `cct` startup-time is close to '''ZERO'''. **** small example : startup '''POWER-Tcl''' with ''boot-loader'' **** task: start shell, initialize, setup the plugin-database and finally print help-screen 0.006 sec. The shell from the example is a shell with the basic '''POWER-Tcl''' features compiled in. HOWTO build the shell ? 1. create the boot-loader with: `cct build --pattern CCT --boot-dir ...` 2. using the build pattern: `CCT = KeyCct TclInit Cct StdLib` 3. the pattern is using the following `POWER-Tcl-projects` ... list below ... ======none > cct build --pattern CCT --list project package : projects ------------ : ---------------------------------------------------- StdLib : StdLib FileLib Cct : Cct Args CctCmd CctComplete KeyCct : Key TclInit : TclInit_auto TclInit_initScript TclInit_history msgcat1.1 TclInit_word opt0.4 tcltest1.0 TclInit_ldAout http1.0 http2.4 TclInit_parray TclInit_safe TclInit_package TclInit_init ====== One thing is important to know : 1. Every ''project'' is a tcl script(s) or a shared library or a static archive 2. if it is a `static archive` and compiled into the executable the binary code is added but '''NOT''' initialized 3. the initialize is done if a `package require...` is done '''OR''' the `auto_index` request a proc/command from the ''static archive'' to load 4. the startup-time is very ''low'' because ''only'' the code required is initialized One of the problems was to add the code from `.../lib/tcl8.3` into the binary. ======none > time ./cct -h usage: cct [master option ...] command ... master options : -m ARCH platform target [linuxi386] -d FLAG use debug mode [no] -v FLAG use verbose mode [no] -f FLAG use force mode [no] commands : blowfish blowfish encryption build build and install the distribution ... tkdemo start the tk demo view view plugin interface wish the POWER-Tk shell default options : --help FLAG print the help (-h) [no] --info FLAG get information about command [no] --edit FLAG open project and edit files [no] --test FLAG run tests in 'tests' subdirectory [no] > all remaining arguments are passed to tcltest --args LIST command-line arguments [NOT-SET] real 0m0,006s user 0m0,006s sys 0m0,000s ====== just to compare, the grep : ======none > time grep --help real 0m0,002s user 0m0,002s sys 0m0,001s ====== The '''POWER-Tcl''' build up the ''plugin-database'' for display the ''plugin-list'', the startup trace show this: ======none > strace ./cct -h |& grep access access("/etc/ld.so.preload", R_OK) = -1 ENOENT (Datei oder Verzeichnis nicht gefunden) access("/tcl8.3", R_OK) = -1 ENOENT (Datei oder Verzeichnis nicht gefunden) access(".../Compiler3.BUILD/lib/JavaKiller/var/SysIndex/linuxi386_cct_pkgIndex.tcl", F_OK) = 0 access(".../Compiler3/example/CctMan.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctTest.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/TclHttpd/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctKeyGen.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctFind.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/Build/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctDiff.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/CctIndex/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/PkgLib/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctProfile.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/BWidget/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctLabel/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/Crm/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctMail.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/TMakeLib/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/Bras/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/CctView/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CgiPoll.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctPerf.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctKey.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/Md5sum/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/Config/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CgiGbook.CVS/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/CctPlugin/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/Wish/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/Blowfish/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/Agent/Cct.cfg", R_OK) = 0 access(".../Compiler3/cmp/Tclsh/Cct.cfg", R_OK) = 0 access(".../Compiler3/example/TclIDE/Cct.cfg", R_OK) = 0 ====== It is important to '''notice''' that the entire `lib/tcl8.3` directory is gone, it's now total in '''`C`''' *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 18 Mai 2021] *** yesterday I add a small ''daily programmer helper'', section '''tclsh''' ======none > cct tclsh file dirname want/tcl want ====== To call a tcl shell use `cct tclsh`, but some things where ''expanded'' by default, this is good for shell scripts etc… The code is quite simple: ======tcl # main proc called by "cct tclsh ..." proc ::CctTclsh::Main {} { ::Args::CheckInit if { $SET(--args) } { ::StdLib::Push STDIN stdin STDOUT stdout STDERR stdout ERROR EXIT puts [ ::ITclSh::Exec $VALUE(--args) ] ::StdLib::Pop STDIN STDOUT STDERR ERROR } else { set ::tcl_interactive 1 if {[info exists ::env(TCL_RCFILE)]} { uplevel #0 source $::env(TCL_RCFILE) } } } # interface proc called for TCL code/files proc ::ITclSh::Exec { { Args {} } } { variable EXEC variable NOVERSION if {[llength $Args] == 1} { set Args [lindex $Args 0] } set cmd [lindex $Args 0] ::Env::Push $NOVERSION 1 if {[ file exists $cmd ] || "$cmd" == "package" || "$cmd" == "load" } { # real file -> do an exec set ret [ ::StdLib::Exec [ concat [list $EXEC] $Args ] ] } elseif {[llength [info commands $cmd]] != 0} { set ret [eval $Args] } elseif {[auto_load $cmd]} { set ret [eval $Args] } elseif { [set execL [auto_execok $cmd]] != "" } { set ret [ ::StdLib::Exec [concat $execL [lrange $Args 1 end]] ] } else { ::StdLib::ErrorN ITclSh "don't know how to exec: $Args" } ::Env::Pop $NOVERSION set ret } ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 17 Mai 2021] *** 1. last weekend I add a new technology called the '''Indexed-Binary-File-technology''' The ''indexed-binary-file'' is just a '''POWER-Tcl-compiled-library''' and in addition a `tclIndex` file with the definition for every ''proc'' in the binary. **** example using the `CctPlugin.tcl` file **** '''step-1,''' install the binary ======none > cct make c.is ## working on project 'CctPlugin' # making `c_exe.is' install : c_exe -> ./CctPlugin.tpkg/arch/linuxi386/lib/pkg # making `c.is' install : c.is -> OK ====== '''step2,''' show the index ======none > ls -al ./CctPlugin.tpkg/arch/linuxi386/lib/pkg -rwxr-xr-x 1 XXXXXX users 16032 17. Mai 12:23 libCctPlugin1.0.so -rw-r--r-- 1 XXXXXX users 528 17. Mai 12:23 libCctPlugin1_0.tclIndex > cat ./CctPlugin.tpkg/arch/linuxi386/lib/pkg/libCctPlugin1_0.tclIndex set auto_index(::CctPlugin::Main) [list load [file join $dir libCctPlugin1.0.so]] set auto_index(::CctPlugin::Args::Init) [list load [file join $dir libCctPlugin1.0.so]] set auto_index(::CctComplete::MySpecialComplete) [list load [file join $dir libCctPlugin1.0.so]] set auto_index(::CctPlugin::Args::Help) [list load [file join $dir libCctPlugin1.0.so]] set auto_index(::CctPlugin::ErrorN) [list load [file join $dir libCctPlugin1.0.so]] set auto_index(::CctPlugin::Args::Main) [list load [file join $dir libCctPlugin1.0.so]] ====== **** Ok, what is the goal ? **** 1. autoload of libraries, even if compiled into binary. ''traditional'' tcl only able to make `tclIndex` from pure tcl files. 2. currently a lot of work is done for the '''POWER-Tcl-boot-loader''' this mean to put '''POWER-Tcl''' wery early into duty, even befor all the ''package'' stuff is available. The technology is quite simple and '''only''' works for '''POWER-Tcl-compiled-binaries'''. The technology is using the repository of compiled ''tcl-code''. In the repository is already an index called `.IS_PROC-inputFileName` and this index already has all ''proc's'' defined in the '''POWER-Tcl-compiled-binary''' ======none > cat ~/.cct/repository/Master/CctPlugin_1_0/.IS_PROC-CctPlugin @X0-C13@ {::CctPlugin::Main {}} @X0-C8@ {::CctPlugin::Args::Init {}} @X0-C10@ {::CctComplete::MySpecialComplete S,s} @X0-C11@ {::CctPlugin::Args::Help {}} @X0-C6@ {::CctPlugin::ErrorN s,*} @X0-C12@ {::CctPlugin::Args::Main {}} ====== **** statistics **** ======none Compiler3> git log --format= --numstat | awk '{add+=$1; del+=$2} END {print "add=",add," : ","del=",del}' add= 2998894 : del= 533366 ====== **** nice usage screen-shot - distribution check with error **** ======none ==> cct -f pkg --check-dist |& less PKG CHECK MSG : check part 1 -> check package database PKG CHECK MSG : check part 2 -> find missing or broken files PKG CHECK MSG : check part 3 -> find unknown files PKG CHECK WARN : found non package directory '.../Compiler3.BUILD/lib/JavaKiller/bin' PKG CHECK WARN : found non package directory '.../Compiler3.BUILD/lib/JavaKiller/exe/linuxi386/man' PKG CHECK WARN : found non package directory '.../Compiler3.BUILD/lib/JavaKiller/exe/linuxi386/man/man3' PKG CHECK WARN : found non package file '.../Compiler3.BUILD/lib/JavaKiller/exe/linuxi386/lib/tclreadline3.0/tclIndex' PKG CHECK WARN : found non package directory '.../Compiler3.BUILD/lib/JavaKiller/exe/linuxi386/lib/tcl8.3/ccttest1.0' PKG CHECK WARN : found non package file '.../Compiler3.BUILD/lib/JavaKiller/exe/linuxi386/bin/tcl2meta0_8.3' PKG CHECK WARN : found non package file '.../Compiler3.BUILD/lib/JavaKiller/bin/cct' PKG CHECK MSG : check part 4 -> find unchecked files PKG CHECK MSG : check part 5 -> deep check PKG CHECK MSG : check package: T2mRt_3_0 PKG CHECK MSG : check package: CctRt_3_0 PKG CHECK MSG : check package: TclRl_3_0 PKG CHECK MSG : check package: Readline_1_0 PKG CHECK MSG : check package: PkgLib_2_0 PKG CHECK MSG : check package: MetaToC_3_0 PKG CHECK MetaToC_3_0 [::PkgLib::Check::Test-md5sum] : check failed for file: '.../Compiler3.BUILD/lib/JavaKiller/./template/Env/COMPILER.c' the existing md5sum 'ceb92f97bc9f84a3f12925a279b421dd' is not the original md5sum '356e74147f9839bb8a2b4a08339ace5f' please delete the package 'cct pkg --del MetaToC_3_0' and add the package 'cct pkg --add MetaToC_3_0' to rebuild the package database STACK >>> ::PkgLib::Check::Test-md5sum .../Compiler3.BUILD/lib/JavaKiller/./template/Env/COMPILER.c 356e74147f9839bb8a2b4a08339ace5f ... ::PkgLib::Check::Main MetaToC_3_0 ... ::PkgLib::Check::Dist ... ::PkgLib::Exec CHECKDIST 1 ... ::CctPkg::Main ... ::Main ... file: Cct/Main.tcl PKG CHECK MSG : check package: Config_2_0 PKG CHECK MSG : check package: CctPlugin_1_0 PKG CHECK MSG : check package: CctIndex_2_0 PKG CHECK MSG : check package: Base_2_0 PKG CHECK MSG : check package: Agent_2_0 ====== *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 15 Mai 2021] *** 1. [http://nhi1.selfhost.co/POWER-Tcl/group__EarylyBird.html %|% The '''Early-Bird-Dist''' now has a detailed package list] 2. I rename the core tcl executable into cctsh, cct and ccttest just to avoid unwanted interaction with traditional tcl 21. Unfortunately I CANNOT rename `libtcl8.3.so` to `libcct8.3.so` because every tcl extension still needs the traditional tcl name. 3. Lots of updates to the dist including message-layout, StdLib, FileLib etc. *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 10 Mai 2021] *** The last couple of days I add a new technology called '''Power-Tcl-Kit'''. The strategic goal is quite simple: "put everything under one roof" **** OLD **** ''PowerTcl'' was a directory-tree with a modified `tclsh` as starter and a filesystem filled with application specific files. Every application was a collection of ''Power-Tcl-projects'' installed into the ''Power-Tcl-runtime''. Everything together, the runtime, the application and supporting data files was the ''Power-Tcl-application''. **** NEW **** Now I add a new technology to install most of the files into '''ONE''' binary. This binary is called '''Power-Tcl-Kit''', a '''fat''' pure `C` binary. The goal was '''NOT''' to create a '''fat''' binary, the goal was to make this '''fat''' binary faster as the original '''Power-Tcl'''. The original '''Power-Tcl''' has a 10-1000 times faster startup as all the "traditional" languages developed/improved in the last ~20 Years, including all the scripting languages like perl, python, ruby, php etc but also the so called "high" languages like java, C#, go etc. The '''Power-Tcl-Kit''' goes further with a more aggressive optimization and a ''binary-internal-package-management''. Everything is still 100% `tcl` compatible. But every new technology also introduce a new problem. The biggest problem was the '''`software-in-duty`''' problem. Now there a 3 sources of application-code: 1. the '''traditional''' tcl-script. 2. the '''POWER-Tcl''' compiled tcl-script as shared-library. 3. NEW: the '''Power-Tcl-Kit''' code compiled into the '''fat''' binary. It takes some time to get all the package-management-tools used to this ''triple'' but now it seems to be stable. The '''Power-Tcl-Kit''' project is still not finished because now an avalanche of new opportunities arrive. I'm not only the developer of '''POWER-Tcl''' I'm also the developer of the NHI1, Managed-Object and Programming-Language-Micro-Kernel technology. All projects are based on the Token-Stream technology, which was developed in the 1990++ and probably some near time in future this projects will join :-) *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 05 Mai 2021] *** I'm not lazy - these are the stats from my '''POWER-Tcl-git''' repository since the project restarted. Usually I have between 500 and 5000 update lines per day :-) ======none Compiler3> git log --format= --numstat | awk '{add+=$1; del+=$2} END {print "add=",add," : ","del=",del}' add= 2981426 : del= 408284 ====== The above number is really everything, this is probably not a fair number as it also contains a lot of external references, but the code from the '''POWER-Tcl-Core''' is more like the lower limit. ======none Compiler3/cmp> git log --format= --numstat . | awk '{add+=$1; del+=$2} END {print "add=",add," : ","del=",del}' add= 92792 : del= 22722 ====== The last two days I update : 1. The ''project'' code now allow multiple ''project-files'' for a single ''project''. 2. The `autoconf` code to add more '''POWER-Tcl''' specific tests. 3. The `StdLib` to be more compact 4. And cleanup some definitions to be more clear to understand The ''project-definition-tree'' for a single ''project'' is now : 1. The ''build-in'' defaults defined in the `Config` package. 2. The ''toplevel'' project-file at `COMPILER3_SOURCE/project.cct` 3. Any project-file between the current directory and the toplevel directory 4. finally the `local` project file defined with `--project-file` option or `project.cct` as default. *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 04 Mai 2021] *** This days a new tool go into ''production'' and the '''POWER-Tcl''' got a lot of ''cleanup'' * the new tool is `cct label ...` this tool add a header to a file which need a ''legal'' information. * the `cct make *.exp...` or `cct export ...` was updated to create and export archive files * the `--edit` feature got an update to integrate into `screen` command-line multi window tool. * internal I update my web site to use '''ONLY''' `http` protocoll, `ssl` support was removed. To access '''POWER-Tcl''' use `http://nhi1.selfhost.co/POWER-Tcl` The `export` feature of '''POWER-Tcl''' is used to integrate into other software. * `exp.meta ` -> export intermediate `.meta` file(s) * `exp.tcl ` -> export compiled `.tcl` file(s) * `exp.c ` -> export `.c` file(s) * `exp.o ` -> export binary `.o` file(s), link with a single object file, use `int Ot_ProjectFileInit(interp)` for initialization of a ''single'' file, be aware that '''ONLY''' a single file is initialized and '''not''' the project/extension at all. * `exp.a ` -> export archive `.a` file(s) filled with all `.o` files of a project, add `int Project_Init(interp)` into the ''C-Code'' for initialization. * `exp.so` -> export shared `.so` file(s), use `load libProjectX.Y.so` to load this library into 'TCL'. The `label` feature is used to add text into a file with focus on different aspects, I already use this tool in the '''NHI1''' project but now a rewrite was done for '''POWER-Tcl''' : * legal: trademark/author/contact/disclamer etc - always choose the right legal header. * git: use `git log ...` to update file with commit information like date-time, hash, user etc. * code: based on file-type and local setup create and add static code into the file to avoid error-prone and booring code updates. *** [http://nhi1.selfhost.co/POWER-Tcl/ %|% 29 Apr 2021] *** This is a large release adding multiple improvements and features to '''POWER-Tcl''' : * automake, autoconf and make integration -> VPATH build. * `cct make` options for fast prototyping and debugging : --project-file, --test-file and -d. * a lot of internal code-updates and bug-fixes. *** 25 Apr 2021 *** This is a nice example of an "Morning-Error" but this is also a nice example of an ''error'' in a '''POWER-Tcl''' compiled file. As you see the ''STACK'' always works, even the ''errorInfo'' shows the stack with ''proc-names'' but without code. But one thing is missing, the good old ''__FILE__'' and ''__LINE___'' from the wonderfull ''C'' debugging. ''Tcl'' misses some really usefull debugging-infos from day one, and the most important miss is the ''line'' and ''file'' information. * I thing adding more debugging features is one point of the future in '''POWER-Tcl'''. ======none Compiler3> cct make --edit ENVIRONMENT ERROR [::Env::Check] : variable '' is not available (ERR08) -> please check 'Compiler' setup (ERR-07) STACK >>> ::Env::Check {} {} ... ::FileLib::Resolve {} ... ::PkgLib::COMPILER_HOME ... ::PkgLib::Db::INDEXFILE . ... file: pkg/libPkgLib2.0.so ====== *** 23 Apr 2021 *** Today a pretty nice technology was added: I call it '''POWER-starter''' In the past the '''POWER-Tcl''' was a normal ''tclsh'' with some enhancements, able to load and execute a ''POWER-Tcl-share-library''. But now something '''NEW''' was added… instead creating a ''shared-library'' I add a feature called '''objexp''' example: ''objexp'' on `StdLib` ======none > cd cmp/StdLib > cct make c.objexp ... compiles: tcl->c->o > find export/ -name "*.o" export/StdLib/COMPILER_STDLIB.o export/StdLib/StdLib.o export/FileLib/COMPILER_FILELIB.o export/FileLib/FileLib.o export/StackLib/COMPILER_STACKLIB.o export/StackLib/StackLib.o ====== and now I use the `"*.o"` files to create a special ''tclsh'' just called ''cct''. This ''cct'' is now the ''POWER-Tcl'' starter: OLD: cct (BASH script) -> set some environment -> exec `tclsh8.3 package Cct ...` (from POWER-Tcl) -> do the processing NEW: cct (BINARY with ''Cct'' and other stuff) -> do the processing This is a pretty quick start as there is NO single line of ''BASH'' or ''TCL'' script involved and everything is in pure C. *** 20 Apr 2021 *** 1. update [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_COMPILE.html %|% HOWTO compile] 1. update [http://nhi1.selfhost.co/POWER-Tcl/HOWTO_MAKE.html %|% HOWTO make] 1. add `TPACKAGE` project option to link a PROJECT with a installation PACKAGE 1. add `cct make --detail` to print out the project data: ======none .../Compiler3/example/CctPlugin.CVS> cct make --details PROJECT CctPlugin { DEBUGMODE 0 FILE_TCL CctPlugin.tcl { HEADER {} META2C { DEBUG 0 OPTIMIZATION 1 PRINT 0 SKIP {} } META2TCL { PRINT 0 } TCL2META { CMD {} PKG_REQUIRE Tcl PRINT 0 SAFE 0 TEST 0 } } INSTDIR_C .../Compiler3/example/CctPlugin.CVS/CctPlugin.tpkg/arch/linuxi386/lib/pkg INSTDIR_TCL .../Compiler3/example/CctPlugin.CVS/DIR-Tcl MAJOR 1 MINOR 0 STRIP 1 TPACKAGE CctPlugin.tpkg } ====== *** 19 Apr 2021 *** This is a nice overview about the ''POWER-Tcl-development-workplace'' packages : ======none .../Compiler3.CVS> cct build --list name tclV pkgV restriction pattern dir --------------- ----- ----- -------------- ------------------ ------------------------ Base 2.0 --pkg ALL EARLY RT ./cmp/Base StdLib 2.0 2.0 ALL EARLY RT ./cmp/StdLib External 2.0 2.0 ALL EARLY RT ./cmp/External CctIndex 2.0 2.0 ALL EARLY RT ./cmp/CctIndex Config 2.0 2.0 ALL EARLY RT ./cmp/Config PkgLib 2.0 2.0 ALL EARLY RT ./cmp/PkgLib TMakeLib 2.0 2.0 ALL CMP ./cmp/TMakeLib MetaToTcl 3.0 3.0 ALL CMP ./cmp/MetaToTcl MetaToC 3.0 3.0 ALL CMP ./cmp/MetaToC Wish 3.0 ALL TK ./cmp/Wish Agent 2.0 2.0 ALL CMP ./cmp/Agent System 2.0 2.0 ALL EARLY RT ./cmp/System InitScript 9.9 2.0 ALL RT ./cmp/InitScript Target 2.0 2.0 ALL CMP EARLY RT ./cmp/Target Md5sum 2.0 2.0 ALL EARLY RT ./cmp/Md5sum Bras 2.1 ALL CMP ./cmp/Bras Cct 2.0 2.0 ALL EARLY RT ./cmp/Cct CctView 1.0 1.0 ALL EARLY RT ./cmp/CctView CctRep 2.0 2.0 ALL CMP ./cmp/CctRep CctBuild 1.0 1.0 ALL ./cmp/Build TclToMeta 3.0 --pkg ALL CMP ./src/t2m8.3.4 CctRt 3.0 --pkg ALL EARLY RT ./src/cct8.3.4 CctPerf 2.0 2.0 ALL ./example/CctPerf.CVS CctTest 2.0 2.0 ALL ./example/CctTest.CVS CctDiff 2.0 2.0 ALL ./example/CctDiff.CVS CctFind 2.0 2.0 ALL ./example/CctFind.CVS CctPlugin 2.0 2.0 ALL ./example/CctPlugin.CVS CctMan 2.0 2.0 ALL ./example/CctMan.CVS BWidget 1.31 ALL ./example/BWidget TclIDE 2.0 2.0 ALL ./example/TclIDE TclHttpd 2.0 ALL ./example/TclHttpd TclLib 2.0 ALL ./example/TclLib Blowfish 1.0 1.0 ALL ./example/Blowfish Cgi 1.0 1.0 ALL CGI ./example/Cgi.CVS CgiGbook 2.0 ALL CGI ./example/CgiGbook.CVS CgiFormmail 1.0 2.0 ALL CGI ./example/CgiFormmail.CVS CgiPoll 1.0 1.0 ALL CGI ./example/CgiPoll.CVS CgiKey 1.0 1.0 ALL CGI ./example/CgiKey.CVS CgiUserkey 1.0 1.0 ALL CGI ./example/CgiUserkey.CVS CgiHidden 1.0 1.0 ALL CGI ./example/CgiHidden.CVS CctMail 1.0 1.0 ALL CGI ./example/CctMail.CVS CctCrm 1.0 1.0 ALL ./example/Crm OtCC 2.0 --pkg ALL CC ./src OtTcl 8.34 --pkg ALL ./src OtTk 8.34 --pkg ALL TK ./src OtMetaKit 1.0 --pkg ALL CGI ./src Readline 1.0 --pkg ALL EARLY RT ./src TclRl 3.0 --pkg ALL EARLY RT ./src TcLib --make ALL ./cmp/TcLib TkLib --make ALL TK ./cmp/TkLib ====== *** 18 Apr 2021 *** This week a lot of code-cleanup and feature-adding was done. The new features are: +++ package lock `cct pkg --lock pkgname` (also --unlock) distribution upgrade `cct build --pkg upgrade` plugin testing: `cct md5sum --test ...` +++ **** HOWTO lock/unlock **** The ''package-lock'' is important because in '''POWER-Tcl''' every code exist '''TWO''' times : 1. once as ordinary ''TCL'' code 1. once as '''POWER-Tcl''' binary code The goal is that only '''ONE''' code is in duty. To solve this problem the ''binary'' can be ''locked'' and ''unlocked''. To ''lock'' mean just renaming the `binary` to `binary.lock` and rebuild the package index files. '''Example:''' edit ''plugin'' code without lock ======none .../Compiler3.CVS> cct plugin --edit CctPlugin ARGUMENT ERROR [::Args::Edit-Arg] : The package 'CctPlugin' uses the BINARY from the directory: > .../Compiler3.CVS/lib/JavaKiller/exe/linuxi386/lib/pkg To edit the package, the SOURCE must be available and the BINARY must be locked. > Use 'cct pkg --lock CctPlugin' to lock the BINARY. STACK >>> ::Args::Edit-Arg ... ::Args:: ... ::Args::Invoke2 --edit ... ::CctPlugin::Args::Main ... ::Args::CmdInvoke CctPlugin ... ::CctCmd::Invoke CctPlugin ... ::Args::Invoke2 plugin ... ::CctCmd::Args::Main ... ::Args::Main ... ::Main ... file: Cct/Main.tcl ====== After edit is finished one simple command '`cct build --target CctPlugin --pkg upd`' recompile and install the new package: ======none > cct build --target CctPlugin --pkg upd > CctPlugin ./example/CctPlugin.CVS BUILD MSG : ./example/CctPlugin.CVS -> cct make -s c.is BUILD MSG : ./example/CctPlugin.CVS -> cct pkg --dir COMPILER_HOME --noidx --upd CctPlugin package: CctPlugin_2_0 PKG UPD MSG: update package -> CctPlugin_2_0 PKG ADD MSG: work on package -> CctPlugin_2_0 PKG ADD MSG: setup arch -> linuxi386 PKG ADD MSG: check destination directory -> .../Compiler3.CVS/lib/JavaKiller PKG ADD MSG: check destination directory -> .../Compiler3.CVS/lib/JavaKiller/exe PKG ADD MSG: write package to destination directory -> .../Compiler3.CVS/lib/JavaKiller/var/pkg/install/CctPlugin_2_0 PKG ADD MSG: cleanup installation directory -> .../Compiler3.CVS/lib/JavaKiller/var/pkg/install/CctPlugin_2_0 PKG ADD MSG: cleanup installation directory -> .../Compiler3.CVS/lib/JavaKiller/var/pkg/arch/CctPlugin_2_0 PKG ADD MSG: write package finish -> CctPlugin_2_0 PKG ADD MSG: delete build/inst/arch directory -> done PKG UPD MSG: clean package -> CctPlugin_2_0 BUILD MSG : . -> cct index --dir COMPILER_HOME --sys package PKG INDEX : work in directory -> .../Compiler3.CVS/lib/JavaKiller/exe/linuxi386/lib/pkg PKG INDEX : create pkgIndex.tcl (MSG01) -> START PKG INDEX : create pkgIndex.tcl (MSG02) -> END BUILD MSG : . -> cct index --dir COMPILER_HOME --sys cct CCT INDEX : work in directory -> .../Compiler3.CVS/lib/JavaKiller/exe/linuxi386/lib/pkg CCT INDEX : create Cct.cfg (MSG01) -> START > found : CctPkg CctPlugin CctRep CCT INDEX : create Cct.cfg (MSG02) -> END BUILD MSG : . -> cct index --dir COMPILER_HOME --sys cgi CGI INDEX : work in directory -> .../Compiler3.CVS/lib/JavaKiller/exe/linuxi386/lib/pkg CGI INDEX : create Cgi.cnf (MSG01) -> START > found : CGI INDEX : no valid package found (WAR01) -> END BUILD MSG : . -> cct index --dir COMPILER_HOME --sys create SYS INDEX : work in directory -> .../Compiler3.CVS/lib/JavaKiller SYS INDEX : create arch_pkgIndext.tcl (MSG01) -> START SYS INDEX : create file -> linuxi386_pkgIndex.tcl add package : AgProc AgVar Agent Args Blowfish Cct CctBlowfish CctBuild CctBwdemo CctCc CctCc CctCheck CctCmd CctComplete CctCrm CctDiff CctExport CctFind CctGbook CctHttpd CctIde CctIndex CctInfo CctInit CctKey CctKeygen CctMail CctMake CctMan CctMd5sum CctPerf CctPkg CctPlugin CctPoll CctProfiler CctRep CctTclsh CctTest CctTkdemo CctView CctWish Cgi CgiChat_main CgiCmd CgiFormmail CgiGbook_add_db CgiGbook_add_form CgiGbook_list_form CgiHidden CgiKey CgiPoll CgiPoll_res CgiUserkey Config External FileLib Find IAgent IBras IFind IIndex IMan IMetaToC IMetaToTcl ITMakeLib ITclHttpd ITclIDE ITclIDEserver ITkDiff IView IndexCct IndexCgi IndexPkg IndexSys IndexTcl InitScript LibMetaToTcl Md5sum MetaToC MetaToTcl PerfLib PkgLib Poll_db Poll_read StackLib StdLib System TMakeLib Target TclIDE TclIDEclient TclIDEserver TestDriver TkCon TkConProc TkDiff base64 bras cmdline counter csv fileutil ftp ftpd graph html htmlparse http http::admin http::auth http::cgi http::config http::counter http::debug http::demo http::direct http::dirlist http::doc http::eval http::imagemap http::include http::ismaptcl http::ismaptk http::log http::logstd http::mail http::mtype http::opentrace httpd httpd__admin httpd__auth httpd__cgi httpd__config httpd__counter httpd__debug httpd__demo httpd__direct httpd__dirlist httpd__doc httpd__eval httpd__imagemap httpd__include httpd__ismaptcl httpd__ismaptk httpd__log httpd__logstd httpd__mail httpd__mtype httpd__opentrace httpd__passcheck httpd__prodebug httpd__redirect httpd__safecgio httpd__session httpd__snmp httpd__srvui httpd__status httpd__stdin httpd__telnet httpd__threadmgr httpd__upload httpd__url httpd__utils httpd__version javascript log math matrix md5 mime ncgi nntp pop3 profiler queue report sha1 smtp stack start_httpd start_httpdthread stats struct tclreadline textutil tree typedcmdline uri SYS INDEX : create arch_pkgIndext.tcl (MSG01) -> END ====== **** POWER-Tcl-Ide **** * A lot of improvement got the '''POWER-Tcl-Ide'''. The '''Ide''' is a total new development-concept, nothing equal was ever done. The goal is to have a full featured '''Ide''' but only as ''command-line-tool'' with ''auto-complete''. * YES - it is working The ''auto-complete'' feature always give a hint to the user what to-do and the user interact with a ''POWER-Tcl-distrubution'' as target. A typical ''IDE'' has a permanently running process, GUI, etc., but the POWER-Tcl-ide has no running process. The only thing that POWER-Tcl has is a single command called '''`cct`''' and a tremendous startup speed that exceeds anything known in modern programming. example: if the user hit the `Tab` button the `bash` start the autocomplete `cct complete ...` and the `cct` return everything possible what a user can do with the current ''command-line''. Because always a full new `cct` is started (no server or something else is used) the `cct` need the tremendous startup-speed to provide immediate response. *** 13 Apr 2021 *** Test the '''Early-Bird''' distribution, for example the package verification utility: ======none > /build/compiler/EarlyBird> cct pkg --check2 PKG CHECK MSG: check part 1 -> find unused install template PKG CHECK MSG: finish part 1 PKG CHECK MSG: check part 2 -> find unknown files PKG CHECK MSG: finish part 2 ====== In ''POWER-Tcl'' an application has '''always''' the TCL interpreter and all dependency packages included. <
> The goal is that an installed application has '''no''' external dependencies. This is possible because <
> TCL ist very small compared to JAVA or other "modern" software and TCL is FREE. To put everything together the ''POWER-Tcl'' has an package-manager `cct pkg ...` which is also part of '''every''' application. <
> To check if an application is ok the package-manager uses a package-database with an `md5sum+info` about every <
> file installed. The command `cct pkg --check2` checks the entire application. The package-database is also a part of '''every''' application. To build an application a new tool was added this week called `cct build ...` . This tool is part of the `POWER-Tcl-Developer-Workplace`.<
> To build the ''Early-Bird'' application I use : `cct build --dir /build/compiler --dist Early --make c.is --pkg add` <
> ======none usage: cct [master-option...] build [user-option ...] [--] args short: build and install the distribution master options : -m ARCH platform target [linuxi386] -d FLAG use debug mode [no] > simulate the build, do NOT exec -v FLAG use verbose mode [no] -f FLAG use force mode [no] > use 'force' for package installation user options : -i FLAG interactive mode [no] -e TARGET edit target [NOT-SET] --list FLAG list all targets [no] --proc FLAG check the procs : cct check -o... [no] --dir DIR set DISTDIR root [/my/dist] --make ACTION compile the files : [NOT-SET] > m.exp, m.rb, m.rm, tcl.all, tcl.exp, tcl.is > tcl.rb, tcl.rm, c.all, c.cmp, c.exp, c.is > c.rb, c.rm, c.wk, clean --prj ACTION work on 'project.cct' file : [check] > edit : vim 'project.cct' > save : tar -czvf SAFE/DATE/target.tgz .project.cct > check : cct check -p ... --pkg ACTION work on package : [upd] > add upd del edit incr lock unlock index > edit : edit VERSION & pkg.tcl > incr : increment MAJOR from VERSION > index : create package index --dist ACTION select distribution, set '--pattern' : [NOT-SET] > WinRT LinRT LinCgi LinComp WinComp Early Dev --pattern LIST select pattern (combine with '+-') : [NOT-SET] > ALL CC CGI CMP EARLY RT TK --target LIST specific target (disable '--pattern') : [NOT-SET] > Agent, BWidget, Base, Blowfish, Bras, Cct, CctBuild > CctCrm, CctDiff, CctFind, CctIndex, CctMail, CctMan, CctPerf > CctPlugin, CctRep, CctRt, CctTest, CctView, Cgi, CgiFormmail > CgiGbook, CgiHidden, CgiKey, CgiPoll, CgiUserkey, Config, External > InitScript, Interface, Md5Sum, MetaToC, MetaToTcl, OtCC, OtMetaKit > OtTcl, OtTk, PkgLib, Readline, StdLib, System, TMakeLib > Target, TcLib, TclHttpd, TclIDE, TclLib, TclRl, TclToMeta > TkCon, TkLib default options : --help FLAG print the help (-h) [no] --info FLAG get information about command [no] --edit FLAG open project and edit files [no] --args LIST command-line arguments [*] ====== *** 9 Apr 2021 *** * today I put back the famous `tclreadline` package. `tclreadline` gives you a `bash` like interactive shell like `TkCon` for the commandline. `tclreadline` is started with `cct tclsh` and no file as argument. Note: A typical UNIX command like `ls -al` is evaluated directly without `exec` etc Example: using `find` together with `tclreadline` empowerd '''TCL''' : ======none tclsh8.3 [~/Project/Compiler3.CVS] find . -name README.md -ls 442018264 8 -rw-r--r-- 1 dev1usr users 4459 Apr 7 21:48 ./README.md ====== The `tclreadline` package was part of a project called '''POWER-os''' (a subproject of '''POWER-Tcl''') which is an entire ''LINUX'' without `bash` but using '''TCL''' as system-programming-language. To install the `tclreadline` feature '''two''' ''POWER-Tcl-packages'' are involved: Readline.tpkg: Compile and install the `libreadline` into the ''POWER-Tcl-runtime'' TclRl.tpkg: Compile and install the `tclreadline` into the ''POWER-Tcl-runtime'' All the configuration compiling and installation is done by ''POWER-Tcl'' with just '''ONE''' command : +++ Readline.tpkg `cct pkg --add Readline` TclRl.tpkg `cct pkg --add TclRl` +++ ***7 Apr 2021*** * [http://nhi1.selfhost.co/POWER-Tcl %|% add PLUGIN code and DOCUMENTATION ] * Example: Add `CctPlugin` the reference ''plugin'' ***3 Apr 2021*** * [http://nhi1.selfhost.co/POWER-Tcl %|% Compile the ''TCL'' default library ] * Example: Add `http2.4` the ''TCL'' http protocol ***2 Apr 2021*** * [http://nhi1.selfhost.co/POWER-Tcl %|% Tasks done this week: 1. ''API-cleanup'', 2. ''testing'', 3. ''bug-fixing'' ] * Example: Add `Md5Sum` with ''TCL'' and inline ''C'' mixed code ***28 Mar 2021*** * [http://nhi1.selfhost.co/POWER-Tcl %|% Early-Bird-Release of '''POWER-Tcl V3'''] ***26 Mar 2021*** In the last few days I setup the tools for distribution: * switch documentation tool from ''wiki'' to ''doxygen'' * add GNU ''automake'' for setup * work on general ''code-cleanup'' * compile the libraries ***20 Mar 2021*** In the last few days I have been working on the following tasks: * Frontend library improvement. ** much less code with more functions * bash-readline-auto-complete-plugin ** the frontend command line tool ''cct'' generates the code for the autocomplete function of bash automatically. * ''package'' and ''project'' file improvements ** the ''POWER-Tcl-package'' is used to distribute ''code'' and the final application ** the ''POWER-Tcl-project'' is used to configure the ''build-process'' One thing that '''Tcl-Compiler''' does very well is locating BUGS (proof by myself) * If ''proc-names'' are changed or the number and type of arguments then '''Tcl-Compiler''' finds the broken references. * If ''proc'' is used without a ''package require'' etc.. * Broken code, unexecutable branch of code etc.. There is a good chance to have '''end''' of next week a working ''early-bird'' edition. * but with one throw-back (tcl-8.3.4) (the 20 years old '''tcl''') * this ''tcl'' has all modern features like namespace, objects etc but missing all the ''oo'' code ***15 Mar 2021*** The compiling of ''TCL'' comes closer, step by step..., * propably the largest technology-improvement for a scripting-language in 20 years. * More details at: [http://nhi1.selfhost.co/Tcl-Compiler_-_HOWTO_MAKE %|% HOWTO use the make tool] compiling ......... c_MRep.cr .../Compiler3/lib/JavaKiller/exe/linuxi386/bin/tclsh8.3 package MetaToC loading script Meta2C loading script LibMeta2Tcl loading script Agent Compiling Step 1. (Parse) Working on procedure ... <::Agent::MRep::Warning> Working on procedure ... <::Agent::MRep::PkgList> Working on procedure ... <::Agent::MRep::LFileList> Working on procedure ... <::Agent::MRep::Read> Working on procedure ... <::Agent::MRep::ReadArray> Working on procedure ... <::Agent::MRep::LReadArray> Working on procedure ... <::Agent::MRep::ReadList> Working on procedure ... <::Agent::MRep::LReadList> Working on procedure ... <::Agent::MRep::Write> Working on procedure ... <::Agent::MRep::Close> Working on procedure ... <::Agent::MRep::Delete> Working on procedure ... <::Agent::MRep::DeletePkg> Working on procedure ... <::Agent::MRep::ListWrite> Working on procedure ... <::Agent::MRep::ArrayWrite> Working on procedure ... <::Agent::MRep::List> Compiling Step 2. (Header) Compiling Step 3. (Include) Compiling Step 4. (Static) Compiling Step 5. (CmpLoc) Compiling Step 6. (Prototype) Compiling Step 7. (Template) Compiling Step 8. (Procs) Compiling Step 9. (Init) create object ..... c_MRep_linuxi386 /usr/bin/gcc -DVERSION=\"3.0\" -D BUILD_TclToMeta -D OTTARGET_linuxi386 -I.../Compiler3/lib/JavaKiller/exe/linuxi386/include -I../../C -O -fPIC -Wall -c ../../C/MRep.c -o MRep.o ***13 Mar 2021*** * get the ''make'' tool back to work. * today I had my first build :-) Compiler3/cmp/Bras> cct -t IBras make m compiling ......... m_IBras PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined PROTOTYPE not defined meta .............. m ***11 Mar 2021*** * work on documentation * improve the ''cct'' front-end API ***08 Mar 2021*** Today is a '''good''' day because today the `cct` tool is back. * The main problem was that after '''20 years''' of silence, many code references are broken and parts of the tool need to be revised to make it work again. * As I mentioned earlier, there are '''three''' Tcl-Compiler revisions and the ''current'' revision ('''3''') depends on part of the previous revision ('''2''') and so on. *** example: cct *** usage: cct [master option ...] command ... master options : -t STRING compiler package [NOT SET] -a FLAG all "*" compiler packages [no] -m STRING platform target [linuxi386] -d FLAG use debug mode [no] commands : blowfish blowfish encryption bwdemo start the BWidget demo cc c/c++ cross-compiler check checks various parts of the installation crm customer relationship diff the diff tool export export internal code find find/grep/change tool gbook manage guestbook cgi httpd the tcl web-server ide the "Compiler" IDE index create all kinds of indexes info info about various details init setup a new project mail send emails make interface to the make tool man interface to the man pages md5sum compute the md5 digest perf interface to the tcl "perf" tool pkg interface to the cct "package" tool poll manage poll cgi profiler plugin profiler rep interface to the repository tclsh the POWER-Tcl shell test interface to the tcl "test" tool tkdemo start the tk demo view view plugin interface wish the POWER-Tk shell default options : -h print the help --help print the help ***07 mar 2021*** ''first signs of life'', the current status of ''Tcl-Compiler'' is ongoing. I need some time to get "used-to" 20J old code using tcl-8.3.4 interpreter :-) Infact there are THREE ''Tcl-Compiler'' * '''Compiler1''' the initial one, very rudimentary * '''Compiler2''' the one which was ''sold'', GOOD but not PERFECT * '''Compiler3''' the current one, more PERFECT but UNFINISHED :-) **** status: I work on Compiler3 **** The ''Compiler3'' is more than just an compiler, it is more like an entire infrastructure including * package: management, signing, distribution * frontend: command-line, gui, web… * speed: compile, anaylyze All '''tcl''' features are available in '''one''' command -> '''cct'''. * the ''tclsh'' is available as ''cct tclsh'' The goal of the '''cct''' is to give the user a ''perfect'' environment for '''tcl''' without doing any ''setup mess''. **** example: PkgLib **** > cct -t TkCon pkg --add -h loading script Cct usage: cct [master option ...] pkg [user-option ...] master options : -t STRING compiler package [TkCon] -a FLAG all "*" compiler packages [no] -m STRING platform target [linuxi386] -d FLAG use debug mode [no] user-options : --add FLAG install the package [yes] --del FLAG remove the package [no] --upd FLAG update the package [no] --list FLAG short info about packages [no] --info FLAG long info about a package [no] --check FLAG check a given package [no] --check2 FLAG check installation [no] --init FLAG create a package from template [no] --rec FLAG recover package database [no] --clean FLAG cleanup COMPILER_HOME [no] --find FILE find package for file [NOT SET] --noidx FLAG don't create index files [no] -f FLAG force mode, don't ask [no] -d DIR installation base [DEFAULT] default options : -h print the help --help print the help additional informations : 1) the "-t" option provides the name of the package: --del, --list, --info, --check -> PACKAGE_MAJOR_MINOR --add --upd -> PACKAGE 2) the "--list" "--info" and "--check" option accept a pattern for "-t" as argument 3) the "--rec" option does not recover all the lost data : recover: PACKAGE, MAJOR, MINOR lost: all other data, but set to useful defaults 4) with the master "-d" option no directory cleanup's are done 5) "--noidx" is mostly used during mass package installation ***04 mar 2021*** * Add first steps to build the [http://nhi1.selfhost.co/wiki/index.php?title=Tcl-Compiler#TCL_COMPILER %|% Tcl-Compiler] * there is a '''$100.000 bounty''' to add ''Tcl-Compiler'' back to Tcl… that's fine -> [https://github.com/flightaware/Tcl-bounties %|% FlightAware's Tcl Bounty program] ***26 feb 2021*** '''Yesterday''' I had a look into the source-code of `tcl-compiler` and I was really surprised how '''beautiful''' the source-code is :-) * '''I''' ([Andreas Otto]) as the developer have complete forgot this code. There was a discussion that I don't publish '''all''' of the source-code * I can say the ''sourceforge'' should be complete! As I mentioned 20 years ago the `tcl-compiler` can be used as plain '''tcl''' script and you can compile the `tcl-compiler` itself as executable. > be aware that `tcl-compiler` uses a '''minimal''' modified `tcl` distribution with TWO features added/improved 1. a `p-unkown` tcl function called if a '''unknown''' function is used 2. a `v-unkown` tcl function called if a '''unknown''' variable is used the modified `tcl` is part of the code at ''sourceforge'' mfg ---- ** OLD stuff below ** At the [First European Tcl/Tk Users Meeting] in 2001, [Andreas Otto] presented a system he called simply, "Tcl Compiler". ** Attributes ** website: [http://sourceforge.net/projects/tclcompiler/%|%sourceforge.net] website (original): [http://web.archive.org/web/20010801231826/http://www.compiler-factory.com/index.html%|%compiler-factory.com via archive.org] website (even more original): [http://web.archive.org/web/20010419065804/http://home.t-online.de/home/aotto/compiler.html%|%t-online.de via archive.org] current version: 2.2 release time: 2001-10-16 ** Reading ** [https://groups.google.com/d/msg/comp.lang.tcl/TEgFxtAZm9s/BoQCw_grneEJ%|%Compiler 1.3 Release Announcement] ,[comp.lang.tcl] ,2001-04-17: [http://web.archive.org/web/20010914013838/http://www.compiler-factory.com/WhitePaper_E.html%|%What Is Token-Stream]: [http://web.archive.org/web/20011022004957/http://www.compiler-factory.com/Compiler-TUTORIAL_E.html%|%tutorial (dead)]: Anyone who has a copy of this please shout out! [http://web.archive.org/web/20011022004957/http://www.compiler-factory.com/Compiler-HOWTO_COMPILE_E.html%|%How to Compile (dead link)]: [PYK] would really like to see this document if anyone still has a copy of it. [http://web.archive.org/web/20011022004957/http://www.compiler-factory.com/Compiler-HOWTO_CROSS_COMPILE_E.html%|%How to Cross Compile]: ditto. [https://groups.google.com/forum/#!topic/comp.lang.tcl/OHzCNynzCio%|%TCL Compiler - Code update] ,[comp.lang.tcl] ,2000-03-01: [https://groups.google.com/forum/#!searchin/comp.lang.tcl/andreas$20otto/comp.lang.tcl/VBHdocjEcG0/wOGv9PQMNsQJ%|%Tcl Compiler] ,comp.lang.tcl ,2006-03-14: ** Resources ** [http://sourceforge.net/p/tclcompiler/code/3/tree/%|%repository]: ** See Also ** [GSoc Idea: Evaluate and Update Tcl Compiler]: ** Description ** [arjen] 2014-01-08 08:48:52: The SourceForge project is still available, That might be a good starting point, if anyone wants to revive this tool. [SEH]: I have been combing through the sourceforge code and (sparse) docs. It appears that the code base is not complete, the author held back a C runtime library that is necessary actually to run the compiled Tcl program. If someone has up-to-date contact information for the author, perhaps we could persuade him to make a full release. The project looks to be worth saving. [SEH] 2021-03-15: I'm following your progress with interest... I encourage you to continue. I look forward to trying the compiler. ---- '''[rasputnik] - 2021-04-05 08:26:07''' Does this all make any sense? Read the page about Power-Tcl but cant understand a thing, maybe i'm too stupid but there's no information what is the tool, what it does, how it works. And no source code, no examples. The only link to source code is to something 15 year old, but even that doesn't seem to provide any useful info about what it is. ---- '''[JAL] - 2021-06-08 23:48:45''' I'm also interested/curious. I do have a large application that could benefit greatly from >3x speedup, where our strategy at the moment is to gradually convert performance critical parts into C. If we didn't have to hand-code these in C, it would be great. I've had a look through the information I could find, but could not quite understand some basic points. Please correct what I say below. 1. Is this fully an "ahead of time" compiler? Being that it requires handlers for unknown variable and command, it suggests the need to be able to execute the program in order to compile it. 2. Is this a tracing / caching JIT type of approach? E.g. doing a stack analysis and examining how the command at each level is invoked, then performing aggressive inlining (or caching) and then optimization? 3. Examining the C code output, does this compiler always just translate down to TCL core calls? Or can it go deeper? 4. Is this just stripping away the TCL-level parsing and emitting the equivalent C code? (via tracing during execution)? I'd be willing to take a deeper look / play around / help if I could get a better idea of what's going on. ---- '''[aotto1968] 2021-06-09''': The basic Idea is quite simple: 1. every TCL command has an adequat C command, example: `info` -> `Tcl_InfoObjCmd` 2. the next step is adding optimization for special cases like `expr` or `uplevel` etc. 3. finally to put everything together to be a tcl extension able to load pure binary tcl package. Basically the new C extension has the '''same''' structure as the ''tcl'' code but in ''C'' plus additional optimization. ---- '''[JAL] 2021-06-14 22:28:37''' Thanks [AOtto1968] for the reply. So, if I'm understanding correctly (and please correct me), OttoCompiler will essentially "inline" the equivalent C code, possibly specializing it according to the call pattern (e.g. constant folding? eliminating branches?). Can it also perform math operations (or maybe even string ops) in an "unboxed" way (convert TCL->native , do the op, then native -> TCL for returning results) ? <> Development | Performance