HaO 2020-04-23: This is the log of the creation of a project to embedd a program written in TCL in a DLL.
I have a program written in TCL (plus some C components). A customer wants to embedd the program in its own application and requires a DLL for a Microsoft compiler.
Additional requirements:
I am a medium level programmer and totally new to this subject.
I want to start by all the people helped within the journey:
KitDLL is binary distribution of a dll with an embedded tcl & bundles.
KitCreator/KitDLL contains the following options:
All those options were added to the Starkit framework still containing many bug-fixes for TCL8.4/5 and support for weired platforms like Windows-CE.
The build system downloads many packages, applies provided patches, compiles them and packs all scripts in the C-VFS and statically binds all binary extensions.
I tried KitDLL binary from [L1 ]. When I download the binary distribution, copied the tclsh.exe beside the binary and started it, I got "<sys/stat.h> not compatible with VC". It is cross-compiled on Linux for Windows. I already tried MingW and those distributions and always ran into subtile bugs, like inexact calculations, wrong colors in Widgets etc. Thus, I decided to compile on my own and use a Microsoft compiler.
I am only interested in the C-VFS part and want to build on my own.
Embedding TCL consists of the following steps:
Internally, Tcl_Main() (in tcl8.6.10/generic/tclMain.c) will do for each created interpreter:
As Tcl_Main() does not return, it is not a suited procedure for this task. Its tasks must be done manually.
For a vfs, this procedure must be done for each created interpreter, to have the vfs installed in.
After initialization, the interpreter handle may be used to invoke commands in the interpreter using Tcl_Eval*() functions.
At end of life, Tcl_DeleteInterp() is called.
To build a static TCL lib, the makefile.vc is used. I decided to use the community edition of MS-VisualC2015. So, first choose in the start menu: "Visual Studio 2015 -> Windows Desktop Command -> VS2015 x86 Native Tools-Eingabeaufforderung". TCL8.6.10 source distribution is unzipped in: "C:\test\tcl8.6.10"
TCL is compiled with the following options:
> cd test\tcl8.6.10\win > nmake -f Makefile.vc release OPTS=static,staticpkg,msvcrt,nostubs,symbols > nmake -f Makefile.vc install OPTS=static,staticpkg,msvcrt,nostubs,symbols INSTALLDIR=c:\test\tcl8610
Staticly added packages are loaded in the interpreter in a different way. Mostly they are added to interpreter awareness by the C command on setup time:
Tcl_StaticPackage(interp,pkgName, initProc, safeInitProc (=NULL) )
and later loaded in the script by:
load "" pkgName
If a custom package should later be linked with the static tcl lib, the define "STATIC_BUILD" must be defined before the inclusion of "tcl.h". This does not apply if stubs are used (define USE_TCL_STUBS).
Download: I took the trunk from [L2 ] dated 2020-01-22. The folder "kitsh\buildsrc\kitsh-0.0" contains the kit programs. Change to this folder.
First, the file tree with the scripts to include is set-up in the folder "C:\test\starpack.vfs". I added the tcl lib folder (excluding most encodings and time zones and tests) and the file "boot.tcl" from the upper "kitsh.0-0"-folder.
A C-VFS file containing this file tree is created by (see aclocal.m4):
tclsh dir2c.tcl tcl c:/test/starpack.vfs --obsfucate > cvfs_data_tcl.c
I did not use "--obfuscate" at the beginning, which does not put the scripts in clear text into the DLL.
To compile this file, I used a MSVC2015 dll project with the following additional defines:
HAVE_STRING_H;HAVE_STDLIB_H;CVFS_MAKE_LOADABLE,STATIC_BUILD;UNICODE;_UNICODE
First I tried using MS-VC6. The dir2c outputs C99 code, not C89. So this patch is applied: [L3 ]. When I included TCL itself, I ran into memory issues. So I changed to MS VC2015.
Then I ran into a compiler limit (C1091), that string constants may not exceed 65535 bytes. A solution is an unsigned char byte array, with the limit of size_t (0x7CFFFFFF).
This lead to the following patch of dir2c.tcl: [L4 ].
Then, the file "cvfs_data_tcl.c" is successfully compiled.
My DLL main program with the 3 exported functions is as follows:
#define STRICT
#define DLL_BUILD
#include <windows.h>
#include <windowsx.h>
#include <tchar.h>
#include <string.h>
#include <stdlib.h>
#include <math.h>
#include <tcl.h>
#include "tclkit.h"
#define ERROR_MESSAGE_MAX 1024
// >>> local Prototypes
__declspec(dllexport) void my_release();
static void TclGetError();
// The pointer to the tcl interpreter
static Tcl_Interp *fg_interp = NULL;
static WCHAR fg_w_error_msg[ERROR_MESSAGE_MAX + 1];
// >>>>> DllMain
BOOL APIENTRY DllMain(HANDLE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved)
{
switch (ul_reason_for_call) {
case DLL_PROCESS_DETACH:
// >>> DLL is unloaded
my_release();
}
return TRUE;
}
__declspec(dllexport) WCHAR * my_init()
{
// >> Release eventual present interpreter
if (fg_interp != NULL)
my_release();
Tcl_FindExecutable(NULL);
fg_interp = Tcl_CreateInterp();
// >> Init Tcl Kit
Tclkit_Init();
// >> Source the init.tcl script
if (Tcl_Init(fg_interp) != TCL_OK) {
TclGetError();
return fg_w_error_msg;
}
return NULL;
}
void __declspec(dllexport) my_release()
{
if (fg_interp != NULL)
Tcl_DeleteInterp(fg_interp);
fg_interp = NULL;
}
// >>>>> my_cmd
__declspec(dllexport) WCHAR * my_cmd(WCHAR *pwCmd, int *pfError)
{
WCHAR * pwError;
Tcl_DString dCmd;
int Res;
if (fg_interp == NULL) {
pwError = my_init();
if (pwError != NULL)
*pfError = 1;
return pwError;
}
if (Tcl_InterpActive(fg_interp)) {
*pfError = 0;
return NULL;
}
// >> Call command
Tcl_DStringInit(&dCmd);
Tcl_WinTCharToUtf(pwCmd, -1, &dCmd);
Res = Tcl_Eval(fg_interp, Tcl_DStringValue(&dCmd));
Tcl_DStringFree(&dCmd);
TclGetError();
*pfError = (Res != TCL_OK);
return fg_w_error_msg;
}
// >>>>> TclGetError
static void TclGetError()
{
Tcl_DString dErrorMessage;
Tcl_DString dErrorMessageWide;
int ErrorLength;
Tcl_DStringInit(&dErrorMessage);
Tcl_DStringInit(&dErrorMessageWide);
Tcl_DStringGetResult(fg_interp, &dErrorMessage);
Tcl_WinUtfToTChar(Tcl_DStringValue(&dErrorMessage),
Tcl_DStringLength(&dErrorMessage), &dErrorMessageWide);
// Limit length to size of error buffer
ErrorLength = min(Tcl_DStringLength(&dErrorMessageWide), ERROR_MESSAGE_MAX * 2);
memcpy(fg_w_error_msg, Tcl_DStringValue(&dErrorMessageWide), ErrorLength);
// Be sure we have a wide end 0 somewhere.
fg_w_error_msg[ERROR_MESSAGE_MAX] = 0;
}The interpreter is initialized, commands may be executed and the interpreter may be released.
I have no idea how it works, but the C file tree is made available to the script.
The Tclkit initialization is announced with the prototype file "tclkit.h":
void _Tclkit_Init(void);
It is a matter of taste to rename this function and remove the leading "_" which indicates an internal function.
The initialization tcl file cvfs.tcl is prepared for embedding by:
tclsh stringify.tcl cvfs.tcl > cvfs.tcl.h
Now, the file kitint.c is ready for inclusion.
I personally have removed from the file:
So, the following defines are set:
_WIN32;KIT_STORAGE_CVFS;TCL_THREADS;TCLKIT_DLL
There is a magic to replace Tcl_Init() in the TCL lib by Tcl_InitRead() and _Tcl_Init() by _Tcl_InitReal() (see Makefile.kitdll.in). Then, a new Tcl_Init() function is defined as follows:
int Tcl_InitReal(Tcl_Interp *interp);
int Tcl_Init(Tcl_Interp *interp) {
_Tclkit_Init();
return(Tcl_InitReal(interp));
}To compile, the internal C headers are required. So include:
By the build system,aclocal.m4 creates "kitInit-libs.h" with content for each contained binary package (excluding build, kitsh and common). In addition, an init function "_Tclkit_GenericLib_Init" is created.
We create this file manually with empty contents:
static void _Tclkit_GenericLib_Init(void) {
}The rechan package is included. I don't know, if it is used by C-VFS. Just include the file "rechan.c".
The VFS package is included by default. I suppose, it is required for C-VFS.
Get the root branch from: [L5 ]
And add file vfs.c in folder generic to the project. Fix bug [L6 ].
Required defines: HAVE_SYS_STAT_H
Now add the following libraries to the project:
In a 2nd project targeting a console application, I have a test file like that:
#pragma warning(disable : 4201 4214 4514)
#define STRICT
#ifndef UNICODE
#define UNICODE
#define _UNICODE
#endif
#include <windows.h>
#include <tchar.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include "my_dll.h"
#define BUFLEN 256
int main(int argc, char* argv[])
{
WCHAR *pwErrorMsg;
pwErrorMsg = my_init();
if (pwErrorMsg != NULL) {
wprintf(L"%s", pwErrorMsg);
return 1;
}
for (;;) {
WCHAR pwIn[BUFLEN];
WCHAR *pwRes;
int fError;
printf("cmd, x to exit:");
fflush(stdout);
fgetws(pwIn, BUFLEN, stdin);
if (wcscmp(pwIn, L"x") == 0) {
my_release();
return 0;
}
wprintf(L"Input: %s", pwIn);
pwRes = my_cmd(pwIn, &fError);
if (fError) {
wprintf(L"Error: %s", pwRes);
}
else {
wprintf(L"%s", pwRes);
}
}
}Link it with the result of the dll project to find out, if something is missing. And run it ;-)
ToDo
ToDo