Yet another dll caller

NJG May 24, 2006

Please, visit Yet again dll caller, new developments are announced there.

APN Jan 21 2007, I was looking for the source for this and could not find it in the dll.zip download. Are there plans for the C source to be released?

NJG August 24, 2004

You may well ask why bother with developing a new dll caller. Especially that originally I had wanted to use it only for accessing the Windows 2000 API. The short answer: ffidl cannot handle output pointer arguments to functions (or I overlooked something) and twapi (at the present stage of development) has a way too short Windows2000 API repertoire. Moreover, I deemed easier to write a new extension than patch the ffidl source (which I had contemplated for some time). Although in its present form the extension (bearing the rather unimaginative name dll) is a dll caller it would be fairly easy to turn it into an so caller for the Linux environment.

The sources and the windows binary may be downloaded from here [L1 ]

The following piece of script retrieves the names and positions of the desktop icons as well as the index of their images.

The package has a tcl and a binary component and can be activated by sourcing the former.

 #Store dll.dll alongside dll_tcl.tcl
 source $PackageDirectory/dll_tcl.tcl

When loaded the package creates the ::dll namespace and defines a number of commands within it. Of these ::dll::load is used for loading dynamic link libraries.

The first argument is the name of the library to be loaded. The naming convention used in calling LoadLibrary applies [L2 ]. The command also creates a new namespace for the loaded library the name of which is either [file rootname [file tail »name«]]] (in case of a single argument) or the one given in the optional -> name specification.

 ::dll::load user32 -> u
 ::dll::load kernel32 -> k

In the library namespace the loader defines a tcl command that can be used for assigning new tcl commands to the functions in the library.

 ::u::cmd "int GetDesktopWindow()"
 ::u::cmd "int FindWindowA(char *, char *)"
 ::u::cmd "int FindWindowExA(int, int, char *, char *)"
 ::u::cmd "int GetWindowThreadProcessId(int, int *)"
 ::u::cmd "int SendMessageA(int, int, int, int)"
 ::k::cmd "int OpenProcess(int, int, int)"
 ::k::cmd "int CloseHandle(int)"
 ::k::cmd "int VirtualAllocEx(int, int, int, int, int)"
 ::k::cmd "int VirtualFreeEx(int, int, int, int)"

(The declaration of the corresponding API functions can be found starting at [L3 ].)

As can be seen above, argument specifications follow a C-ish syntax. Base types are: void, char, uchar, short, ushort, int, uint, int64, uint64, float, double, pointer specs are * and **. Return type should always be specified, no return value by void.

By default the tcl command is created in the namespace of the library and with the same name as the corresponding library function. This can be overridden by the optional -> name specification. The first of the two following lines creates the command WPM in the global namespace for the WriteProcessMemory function in the kernel32.dll library; the second creates ::k::RPM for ReadProcessMemory.

 ::k::cmd "int WriteProcessMemory(int, int, void *, int, int *)" -> ::WPM
 ::k::cmd "int ReadProcessMemory(int, int, void *, int, int *)" -> RPM

Now we start using the armoury.

 set w1 [::u::FindWindowA "Progman" "Program Manager"]
 set w2 [::u::FindWindowExA $w1 0 SHELLDLL_DefView ""]
 set w3 [::u::FindWindowExA $w2 0 SysListView32 ""]

We now have the handle to the listview of the desktop the elements of which correspond to the desktop icons. So with a message sent to the window we ask how many of them are there.

 #  LVM_GETITEMCOUNT = LVM_FIRST(=0x1000)+4 (=4100)
 set iNum [::u::SendMessageA $w3 4100 0 0]

This particular message contains only constants as parameters. When pointers to data structures are passed, however, they are interpreted in the address space of the target process. So we have to place the parameters there. As the preliminary step we acquire the process ID -- returned in the variable pid -- of the process running the desktop.

 set th [::u::GetWindowThreadProcessId $w3 pid]

This line also shows that a pointer parameter requires the name of the variable as a command argument. The pointer passed to the function points to the internal representation of the variable's object thus it can both source and receive values for the function. If the variable does not exist it is created. Instead of a variable name the integer value 0 can be used as a NULL pointer.

We proceed by getting a handle on the process and then allocating 1000 bytes of storage in its address space for our purposes.

 # PROCESS_VM_OPERATION(=0x0008) | PROCESS_VM_READ(=0x0010) | PROCESS_VM_WRITE(=0x0020) (=56)
 set hnd [::k::OpenProcess 56 1 $pid]
 # AlocationType: MEM_COMMIT (=0x1000)
 # Protect: PAGE_READWRITE (=0x04)
 set addr [::k::VirtualAllocEx $hnd 0 1000 4096 4]

In the sequel we will need some memory space in our address space as well.

 set buff1 [::dll::buffer 1000]
 set buff2 [::dll::buffer 1000]

The ::dll::buffer command creates a bytearray value (as e.g. binary format does) of all 0's. As a warm up we clear part of our memory allocated in the desktop process.

 WPM $hnd $addr buff2 1000 0

We next iterate over the elements of the desktop's listview. In each iteration we receive the corresponding icon parameters into a LISTVIEW structure [L4 ].

 set nStart 96
 for {set i 0} {$i < $iNum} {incr i} {
    set lvitem [binary format iiiiiiiiiiiii 3 $i 0 0 0 [expr $addr + $nStart] 128 0 0 0 0 0 0]
    WPM $hnd [expr $addr + 8] lvitem 52 0
    # LVM_GETITEMPOSITION = LVM_FIRST+16 (=4112)
    ::u::SendMessageA $w3 4112 $i $addr
    # LVM_GETITEMA = LVM_FIRST+5 (=4101)
    ::u::SendMessageA $w3 4101 $i [expr $addr + 8]

The next call illustrates another rule. When a function parameter refers to memory filled during the function call, the corresponding parameter specification must be void * and a bytearray of suitable size should be provided.

    ::k::RPM $hnd $addr buff1 200 0
    binary scan $buff1 ii posX posY
    binary scan [string range $buff1 36 39] i imageIdx
    set txt [string range $buff1 $nStart end]
    set txt [string range $txt 0 [expr [string first "\x00" $txt] - 1]]
    puts "$i -- $posX $posY\t: @$imageIdx\t> $txt" 
 }

At this point we bail out cleanly.

 # MEM_RELEASE (=0x8000)
 ::k::VirtualFreeEx $hnd $addr 0 32768
 ::k::CloseHandle $hnd

We are not finished yet however. The package provides four more commands:

• ::dll::memrd address ?length? - Copies length memory bytes from address into the returned bytearray value. If length is missing the first four octets at address are assumed to contain the length. Negative length specifies a string copy and the return value is an ordinary tcl string.
• ::dll::memwr name address - Copies the bytearray value of the variable name to memory starting at address.
• ::dll::ref name - Name should refer to a bytearray variable, the call returns the starting memory address of the array (suitable for ::dll::memrd without length specification). The reference count of the variable's object is incremented.
• ::dll::deref name - Name should refer to a bytearray variable. The reference count of the variable's object is decremented.

These commands are included for a couple of reasons:

• They are needed for constructing the necessary input/output structures for ** (pointer-to-pointer) parameter specifications
• When pointer is specified as the return type only the value of the pointer is returned. Further extraction of values should be done in tcl.

Finally

• ::dll::error is the error code returned by windows after the windows API function call.
• ::dll::alias is an array that has an element for each created command containing the default name that would have been used if -> name would not have been specified.
• ::dll::entry is an array that holds as bytearray values the internal specifications for the created commands (of not much use outside of debugging).
• <namespace>::handle holds the handle to the loaded dynamic library.
• <namespace>::alias gives the name of the dynmic library loaded for namespace.

Use it at your own risk. Works for me though.

Peter Newman 7 January 2005: Nice work NJG. Much easier to call the Windows API with this, than with any of the alternatives I've tried (Swig, Ffidl and critcl).

But may I suggest you add global type to the top of your start-up script. Otherwise, if the start-up script isn't sourced at the global level, the type array becomes local, and the script breaks. Cheers.

escargo 20 Mar 2006 - I was looking at using dll to interface to the current version of AutoIt. The help file provides a convenient list of DLL entries, but mapping what dll needs to what the document says is provided leaves me with a bit of a puzzle.

Here are some examples:

 ///////////////////////////////////////////////////////////////////////////////
 // Exported functions
 ///////////////////////////////////////////////////////////////////////////////
 
 AU3_API void WINAPI AU3_Init(void);
 AU3_API long AU3_error(void);
 
 AU3_API long WINAPI AU3_AutoItSetOption(const char *szOption, long nValue);
 
 AU3_API void WINAPI AU3_BlockInput(long nFlag);
 
 AU3_API void WINAPI AU3_CDTray(const char *szDrive, const char *szAction);
 AU3_API void WINAPI AU3_ClipGet(char *szClip, int nBufSize);

First, I presume the AU3_API and WINAPI symbols are just going to be ignored (to dll they don't signify anything).

Next, the docs for dll don't say anything about what to do about const in the parameter list, and I didn't see any examples.

Finally, where the prototype has (for example) "char *szDrive", is that where the <namespace>::cmd would use something like this: <namespace>::cmd "void AU3_CDTray(char *, char *)"

There's also an interesting issue with some of the commands taking optional arguments. It's not at all clear how to do that with dll.

Any light you could shed on this would be welcome; once I have a new interface to AutoIt done, I'll release it back to the wiki.

Maiki - 2011-05-08 15:50:57

<Somebody still has this .zip file? I cannot find it anywhere>

robin - 2011-07-11 13:50:11

I uploaded it to j.mp/noGMFM [L5 ] (still not a persistent storage...) It is a .zip archive containing .dll, .so and .c (thanks daapp for the link!)

I use this package with Tcl8.5.9 in a Starpack as "package dllcaller". I enclosed in the .zip my pkgIndex.tcl and .tm files, too. It can be used like this:

  package require dllcaller
  dllcaller::load user32
  user32::cmd "int SystemParametersInfoA(uint, uint, uint *, uint)" -> ::SystemParametersInfo
  puts [SystemParametersInfo 11 31 0 2]   ;# sets keyboard repeat rate

I optimized the original loader code for Starpacks, to avoid extracting the .dll to %TEMP% every time when a process starts.

As for the license, I assume the author intended this code to be in the public domain, otherwise he would have announced something about it here since 2006. (Actually, I know him but didn't met him since 1998 when I was working for him at cpluscsystems. He is now retired, I think this is why the company ceased. I take the opportunity to congratulate him for this nice and useful work!)

neb 2011-07-23 Works well, mostly. But on Windows with the following code:

package require dllcaller
::dll::load Kernel32.dll

set buff [::dll::buffer 1024]

::Kernel32::cmd "uint GetWindowsDirectoryW(char *, uint)"

set ret [::Kernel32::GetWindowsDirectoryW buff 12]

I get:

 alloc: invalid block: 00883888: ef ef 0

 This application has requested the Runtime to terminate it in an unusual way.
 Please contact the application's support team for more information.

I can't find anything I've done wrong, and it doesn't matter if I use the dll::buff command, or format banary; Any time I make a call uses a buffer, it blows up. I'm looking at recompiling the dll, but I'm not sure of the dependencies, etc; so it may or may not happen. If I try to compile against the current headers, a bunch of defs seem to be missing, etc.

Dada_Ba - 2015-08-20 14:00:52

I have found a copy of old version tcl_dll.tcl; the link is : http://pan.baidu.com/s/1i3H8ThF