The Tcl [Windows] API (TWAPI) extension project provides direct and high level access to Win32 API's. The TWAPI home page is at http://twapi.magicsplat.com/. '''Note: To make them easier to track, please log bugs at the [http://sourceforge.net/tracker/?group_id=90123&atid=592543%|%bug tracker%|%], not here.''' ---- <> ** News ** 2019-10-29 TWAPI 4.3.8[https://sourceforge.net/projects/twapi/files/Current%20Releases/Tcl%20Windows%20API/twapi-4.3.8/] released. ** Overview ** TWAPI implements commands in the following areas: * System functions including OS and CPU information, shutdown and message formatting * User and group management * COM client support * Security and resource access control * Window management * User input: generate key/mouse input and hotkeys * Basic sound playback functions * Windows services * Windows event log access * Windows event tracing * Process and thread management * Directory change monitoring * Lan Manager and file and print shares * Drive information, file system types etc. * Network configuration and statistics * Network connection monitoring and control * Named pipes * Clipboard access * Taskbar icons and notifications * Console mode functions * Window stations and desktops * Internationalization * Task scheduling * Shell functions * Windows Installer * Synchronization * Power management * Device I/O and management * Crypto API and certificates * SSL/TLS * Windows Performance Counters ** Support ** The [TWAPI] project page is at [http://sourceforge.net/projects/twapi/%|%sourceforge%|%]. * For reporting bugs, use the [http://sourceforge.net/p/twapi/bugs/%|%sourceforge bug tracker%|%] I do not always notice changes to this wiki page so please use the above links. **Obtaining sources** The source code is in a mercurial repository [http://sourceforge.net/p/twapi/code/ci/default/tree/] on sourceforge. ** TWAPI Alternatives ** [Ffidl] also provides access to the Win32 API along with other platforms. [Comparing TWAPI and Ffidl on Windows] compares it with [TWAPI]. ---- Any idea how TWAPI compares with http://zazu.maxwell.syr.edu/nt-tcl/ ? I would definitely favor TWAPI as it is much newer and in active development. I have used a few parts of the nt-tcl package and it worked fine for me at the time (4 years ago). So I would use TWAPI exclusively and fall back to nt-tcl if there was no alternative. [RT] 11Apr2005 ** General discussion ** <> ---- [escargo] 2005-09-21: Can TWAPI be used to inspect the Running Object Table ([ROT])? [APN]: No, I don't think so. There aren't any COM interfaces in TWAPI since I assumed you would be much better off using the TCOM extension for COM-related stuff. [escargo]: From what I read about [ROT], it identifies objects, not their [COM] interfaces. So you need to know what the objects are before you can use [tcom] on them. Since the ROT is part of system state, it seemed like TWAPI would an appropriate way to access it. (See the sample code at the link on the [ROT] page.) ---- [escargo] 2006-03-10: Can TWAPI be used to read and write to raw disk devices (unformatted disks without partitions)? [APN]: No. Looking at the SDK, it looks like a job for DeviceIOControl but seems non-trivial, particularly for testing. Possibly one could also just call WriteFile after opening the raw device \\.\PhysicalDriveN using CreateFile. I'm afraid of trying something like this on my single development machine and trashing it! ---- [Alastair Davies]: asks if there is an API to configure wireless networking? For example, I would like to obtain a list of available wireless networks, and to connect to one. I would not be surprised if this were quite straight-forward, in Windows XP at least. [APN]: You might want to look at WRAPI [http://sysnet.ucsd.edu/pawn/wrapi/] but you would have to write a Tcl binding for that. ---- [LV] 2006-06-05: Sure would be useful if this extension became a part of [ActiveTcl]! [APN] 2011-02-12: I believe TWAPI is now available via [teapot] though there might be a version skew. ---- [MHo]: It would be nice if there where some way to '''register an event source''' to avoid such stupid error messages within eventvwr.exe. [APN] Yes it would be nice but I'm not sure how. AFAIK, to register an event source requires a resource in a DLL or other file. So the messages would have to be "bound" to the resource. You could do this with a resource compiler but then you would not need TWAPI. Or am I missing something? [MHo] That's right. Perhaps the necessary minimal resources/definitions could be build right into twapi.dll. I don't know exactly how to do event source registration by myself (with ms api's this is not uncommon.... smile ;-). No, it can't be built into twapi. Let's say your application wants to log a message "Danger, Danger! Resource do-hickey running low!". That message has to be in some resource, but obviously it can't be in twapi.dll which knows nothing about the application errors. The only way would be to create a resource file on the fly on the target system and that's just too much work. [MHo] Hm.... The actually ''message'' itself sure comes from the caller at runtime; what's missing is the registration of twapi as a valid event source... sure twapi cannot conatin application specific things... As an example, me as a tcl programmer can write generic messages to the eventlog through means of [Winserv] or [tclsvc]...[APN] It's not enough to register twapi.dll as a valid event source. It has to actually contain a valid resource string for the message logged as well. Just registering it will not make the message go away. [neb] 2010-09-01: Here's a proc that I wrote for that. It won't register your library, but it will shanghai somebody else's. appmgmts.dll should exist on any XP-class machine, and event 401 has a message string of "%1"; which means it will just replace the whole thing with your message. This proc uses twapi, and checks for a registration of $src. If it doesn't exist, it rigsters it to the app managements string library. Upshot is that you no longer get the disclaimer. HTH ====== proc logevent {msg {type information} {eventid 401} {src "My Default Source Here"}} { package require registry package require twapi if [catch {set h [registry keys HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\Eventlog\\Application $src]}] { puts "Big registry troubles! Eventlog Applications appear to be missing!" error } elseif {![string length $h]} { puts "$src not found. Writing it." registry set HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\Eventlog\\Application\\$src EventMessageFile %SYSTEMROOT%\\system32\\appmgmts.dll expand_sz registry set HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\Eventlog\\Application\\$src TypesSupported 31 dword } if !$eventid {set eventid 401} #::twapi::eventlog_log $msg -source $src -type $type -category $eventid set h [::twapi::eventlog_open -source $src -write] ::twapi::eventlog_write $h $eventid -type $type -category 0 -loguser -params [list $msg] ::twapi::eventlog_close $h } ====== ---- [male] 2008-10-01: Here one sample of manipulating a Windows message box using TWAPI ... [countdownMsgBox] ---- [MHo] 2009-02-20: I noticed that '''get_global_group_members''' does not give back members if that group only contains other groups (that is, no "direct members"). So which way I could perform a recursive resolution of all computers downwards from a given OU? That's the same behaviour as with NET GROUP, indeed. With some complicated combinations of DSQUERY and DSGET I'm not satisfied after hours and thought tcl and twapi could help... [APN] Do the ADSI COM interfaces not provide any help here? ---- [no-ids] 2010-04-21: I see that there is a new function called impersonate. Do you have an example of exactly what it does or how to use it? Background: Under certain conditions, I would like to launch my script under a different user than the one logged in. Would it help in this case? [APN] No. The impersonate_user command impersonates the user in the *current* thread which is not what you want to do (which is create a new process under a different user). You could use the CreateProcessAsUser call but unfortunately matters are not so simple. You need to additional create an appropriate context by loading appropriate registry hives etc. in the new process. That stuff is not encapsulated yet in TWAPI, mainly because exactly how to do it is somewhat murky and not well documented. ---- [ssing] - 2010-12-22 03:24:00: twapi miss the target when current window is lock or RDP session is in minimize mode. let me explain it more, I am creating a process via twapi, then a login window appear, I am setting that as a forground window and then sending the credential, problem is that when I run the test & lock the window(local system) or I am running the same test in RDP session & I minimize the RDP session, it will not find out the window with specific text ans unable to set the window as forground and exit with error "Invalid pointer or opaque value: ''." [APN] This is not a bug per se. What is unsaid in the documentation, because it is implicitly assumed to be known is that Windows does not allow processes in one desktop session to retrieve information or send messages to windows in another desktop session. That is a Windows security feature and AFAIK there is no way around it. If I misunderstand or if you have reason to believe it is possible, please let me know and I'll investigate further. The specific error you are getting is because you are passing an empty string (returned by find_windows when no window is found) to the set_foreground_window command. ---- [ssing] - 2010-12-22 06:09:23: Correct, But in the above problem I am not sending information from one desktop to other, both the above (RDP & local) are two diffferent scenario where I encounter this issue. The thing is I am running test on my local system & I locked the system it fails because it will not identified the login window but on the other hand if the system is unlock it will identify the login window and proceed further. RDP is different scenario, it's when I am runing my whole test from remote machine(provided whole env. is there), RDP session session is just to watch how test is going, If I have a watch on the session It will pass, say I minimize the RDP session window it will fail. The thing why I am saying this most of the record & play tool used for gui regression test have this feature as they keeps on running test ALAR. [APN] Sorry, still not sure exactly what the scenario is. For the moment, let us ignore RDP. For the login case, here is what I did - I set up a continuous loop that every second would do a find_windows using -text and then being that window to the foreground using set_foreground_window. I then locked the system (switch user screen) and logged back in. The loop was running while locked out and after logging in. So it all seems to work. I assume what I did is different from what you are doing. Can you provide a script (smaller the better) that demonstrates the problem, or provide more detail as to the exact scenario? When you way it does not identify the login window, which window are you referring to ? What text and other command options are you using to identify this window? ---- [ssing] - 2010-12-22 08:00:00: [gui login] sorry about recent changes, it was unintentionally. ---- [ssingh] 2010-12-23 03:22:29: It seems when the desktop is locked, "twapi::set_foreground_window $win" returns 0, and if the desktop is unlocked it returns 1. can we avoid this? [APN]: Sorry about the delayed response (holidays and other stuff). Regarding set_foreground_window - that command is a direct call to SetForegroundWindow Win32 API so it is what it is. Also, regarding the original issue, I wrote a small C program to enumerate all windows to verify what twapi is returning as the window list. The results are identical to what twapi returns with and without the login screen. So the only possibility is that either Windows is creating your app window "lazily" (meaning it is only created when it needs to be displayed which would be after the login screen is removed), or something else is going on with you app. ---- [MHo] 2011-02-11: A tip regarding services: every path specification should be an ''absolut path'', sind relative paths do not work with services... something I alway forget, running into trouble again and again... ---- [MHo] 2011-02-18: What´s this: (1) ====== C:\Dokumente und Einstellungen\Hoffmann>tclsh % package require twapi;namespace import twapi::* % comobj invalid command name "comobj" % twapi::comobj wrong # args: should be "twapi::comobj comid ..." % ====== [APN]: For reasons lost in the mists of time, you have to explicitly call twapi::import_commands. ====== package require twapi ; twapi::import_commands comobj .... ====== ---- [ssing]: Is there any functionality present in twapi like "psexec.exe" do. I mean can we start a remote process with the help of twapi. [APN]: psexec is an application/program. TWAPI is a library. I believe TWAPI contains all the Win32 functionality you would need to write a program like psexec but it is non-trivial. See the bottom of [http://www.windowsitpro.com/article/remote-computing/psexec] for an outline of how psexec is implemented. You would need to do the same things using TWAPI. ---- [MHo] 2011-12-11: Are there any plans to facilitate more console API functions like ''ReadConsoleOutput''? Otherwise it would be hard (or impossible) to use them as the basis for a console mode window manager (what I really need is an up-to-date CTk, but there's no). [APN] Not likely any time soon. I am about to release a new version, after which it's unlikely I will have much time to work on TWAPI. And what little time I have will be spent on other more important features. ---- [Googie] 2012-07-15: I'm trying to get access for my application to write to Windows registry under Windows 7. I'd like it to use gently an UAC in Windows. I guess that if I call "twapi::enable_privileges" I would get what I want. Problem is that it requires privileges list as an argument and there is no documentation about what are possible privileges. I also tried to locate any enumerated privileges on MSDN, but with no luck. Anyone has an idea where to find available privilege list? [APN]: For a privilege list, see http://msdn.microsoft.com/en-us/library/windows/desktop/aa375728(v=vs.85).aspx#privilege_constants. However, I don't think this would help in what you are trying to do. If you want to run a process with elevated privileges (UAC), it has to be created as such by its *parent* process. You can use the TWAPI shell_execute command for this: ====== twapi::shell_execute -path notepad.exe -verb runas ====== This will create notepad with elevated privileges after showing the UAC dialog. The original process still runs in unelevated mode. [Googie] 2012-07-16: I've looked up a little more and I found a pretty nice solution here (as an alternative to twapi dependency): [http://superuser.com/questions/106157/windows-vista-how-to-execute-a-exe-program-in-the-cmd-as-administrator] Might be useful if somebody doesn't want to add twapi dependency just for shell_execute ;) ---- [MHo] 2013-02-16: Using '''lookup_account_sid''' from within an "event sink" (monitoring callback of '''eventlog_monitor_start''') leads to skipping of records (seems that something isn't reentrant?). [APN] lookup_account_sid is a straight Windows call, should not be any reentrancy issues. Can you post the specific code snippet? And are the events being skipped existing events or new ones being logged? '''MHo: New event''' What Windows platform '''winXP SP3''' and twapi version '''The latest stable''' ? If the lookup_account_sid call is removed, are events read without skipping? '''Yes'''. * Note: The code was moved from here to [Tailing Windows Eventlogs Using TWAPI] Also, I don't fully understand this paragraph of the help: ====== -file EVENTLOGBACKUPFILE Specifies the name of a backed up event log file. This option may not be used with the -source or -file options. ====== What does this mean, ======none -file may not be used with -file ====== ? What can I specify for `-source`? If I specify '''Application''', I see Application-Events. If I specify '''System''', I see System-events. If I specify '''Security''', I still don't see security events. If I specify '''blablabla''', I see some events logged with source ''blablabla'', which makes no sense for me. Is there a simple method of trapping '''all''' events? Also, the interpretation of the '''eventID''' is a mistery for me... Excuse me, I know this is because of my bad understanding of Windows, not because of TWAPI, which provides a great layer of abstraction and simplification in using the various windows services. The windows API by itself is just too complicated and changes too fast....:-( Especially the event logs are getting on my nervs. Why not simply using text files, as the whole rest of the world, so a simple tail would help (the motivation for my prog...)? [APN]: Microsoft says "under some circumstances" the event signalling the callback may be missed as their notification code uses PulseEvent which is a deprecated Win32 call (funny how they advise applications not to use it and do so themselves). See http://sourceforge.net/p/snare/feature-requests/1/ and the MSDN documentation for NotifyChangeEventLog and PulseEvent. You have two alternatives - wait for the next event at which time you will get signalled, or put a poller in the background in addition to the monitoring. The same is true for file system monitoring as well. Why poll AND monitor ? Well, the monitoring will give you less delay in most cases allowing you to set the polling interval higher (at least 60seconds or more). Agreed the Windows event logging is more complicated but it does have some advantages. For one, it is language independent so the event log can be read in any language. The eventID is in effect a message id so depending on your language setting, you can see the event in your native language, something that is not possible with text logging. The eventlog_format_message will format the event into the appropriate language. Also since the eventId act as "atoms" for strings, the logs tend to be much smaller. The Windows event log also is a circular buffer so you don't have to roll over or clean up old logs etc. It is not possible in XP to monitor multiple event log sources without opening up multiple handles. In Vista and later you can do so with quite powerful event filtering capabilities. The twapi development version (4.0a16) has support for that but not fully tested yet (and of course the new api will only work on Vista and later). Oh, and the documentation is a typo - it should be "-file cannot be used with -source or -system". [MHo]: Is there something special in monitoring ''remote'' Systems? If I use '''-system''' '''''name''''' to open an event log from a remote computer and then do a '''eventlog_monitor_start''', the following error occurs: ======none Das Handle ist ungültig. (invalid handle) while executing "NotifyChangeEventLog $hevl $hevent" (procedure "eventlog_monitor_start" line 4) invoked from within "eventlog_monitor_start $hevl($ix) [list eventSink $hevl($ix) $logname]" ("foreach" body line 12) invoked from within ====== Unfortunately, the NotifyChangeEventLog Win32 API does not support remote handles. You will have to use WMI but that is non-trivial to set up and call (or at least I don't know the steps required to do it) and probably not reliable. If you are on Vista or later, you may be able to use evt_subscribe instead for notification of events on remote systems. [MHo]: Thanks. With MS eventvwr.exe it is possible to connect to remote logs, but those viewer also doesn't do realtime updates. Maybe I'll implement some kind of polling instead. ---- [DB]: I'd assumed `twapi::eventlog_count` would get me the current record number. Looks like you need to add that value to twapi::eventlog_oldest to get it. ---- [MHo]: Another monitoring problem. Just experimented with '''begin_filesystem_monitor''' and couldn't figure out an easy way to securely know when an operation in the monitored directory has finished. For example, if monitoring with '''-filename 1 -size 1''', one "added" and one "modified" event are fired immediately after the file appears in the directory. But if the file is big enough, it lasts some time until the copy is complete and one couldn't operate on the file until then. Maybe a kind of '''isOpen()''' would help, but I have no such function. Meanwhile I've found a combination, `-filename 1 -write 1`, that gives `added-modified-modified` for new files, and `modified-modified- modified` sequences for files which are overwritten, where the last modified signals completion. Together with a little logic this gives what I wanted. ---- [MHo] 2013-07-29: Can't start a service to handle PAUSE and CONTINUE signals. In the example code, the callback contains code to handle pause and continue signals. But '''twapi::run_as_service''' is called without '''-controls''', so I think only start and stop is handled by default. After adding '''-controls [[list pause continue stop]]''', the service never starts; I always get this error: '''Service error: can't use non-numeric string as operand of "|".''' regardless of the TWAPI version I tested. [APN] Please use '''-controls [[list pause_continue stop]]''' (that is, '''pause_continue''' is one token with an underscore, not two words. This is not documented clearly. ---- [MHo] 2013-07-30: begin_filesystem_monitor, as the documentation states, calls it's registered callback several times for each fs modification. This makes it nearly impossible to determine when a modification really is complete. I thought I've figured it out, but right now the system sends out one signal less than before (added-modified instead of added-modified-modified) although nothing changed in configuration. So, my program doesn't work anymore! [APN] As has been indicated elsewhere, Windows does not guarantee anything about notifications, neither the number, nor the type. It is therefore of use only under limited circumstances and with caveats. Also, even in theory it is not clear what is meant by a modification being complete. Is it when a complete log file is written ? Or a DB transaction is done ? Or a file is closed ? Or all handles to a file are closed ? The only way I would use the monitoring mechanism is as a quicker response for updating file system viewers etc. with some kind of a poller being the "fail safe" backup. [MHo]: Yes, I mostly agree. The question was to eleminate polling, keep cpu usage low and to avoid sleeps nevertheless in an app which monitores "queue"-like folders for incoming files to process further. The files may be large, so my program should only act on files which are fully copied and closed by the sender (could take several seconds). Perhaps I should only monitor the folder itself and use the change signal as a trigger for further testing. ---- [MHo] The pattern *.* for begin_filesystem_monitor suppresses files with have no extension. Should we use * instead? With cmd.exe, *.* and * are the same, whereas *. gives files without an extension. [APN] As the docs state, the pattern rules are identical to those for the [string match] command. So you need to use * (as in the Unix shells). ---- [MHo]: I use a '''set_console_control_handler''' to trap ctrl-c etc. This works fine if running from command line. If I start the console prog via task planner and then terminate through task planner, the control handler wouldn't get fired, so my program has no chance to clean up. [APN] A task manager kill (or any equivalent that uses ProcessTerminate) is no different from a Unix kill -9. There is no way to catch it. ---- [MHo]: Another observation regarding '''set_console_control_handler''': even with an event loop alive (''[vwait] var''), Control-C sometimes isn't trapped and the program is immediately terminated by the OS (at least with Windows XP). I noticed that when the event loop has not much to do (no callbacks), everything works fine, but when there are multiple event handlers active (even when they return relatively fast), there's a good chance that Control-C isn't noticed by the program. So, I didn't find a way to 100% protect a program from unwanted interruption or for a gracefull shutdown... Even worse, a new variant just notice with the same prog on my home pc (again XP): the program locks after pressing Control-C... Maybe I would better understand this behaviour If I'd know more about whats going on beding the scene. I'd thougt trapping is done by TWAPI "in background", resulting in a schedule of the registered event handler (if the event loop is alive, of course). Addition: Why does this minimal program not work as expected: ====== puts [package require twapi] ::twapi::import_commands proc trap args { puts <<<$args>>> return 1 } proc repeat {} { puts -nonewline stderr "." after 1000 [list repeat] } set_console_control_handler [list trap] after 10000 {generate_console_control_event ctrl-c} trap repeat puts "vwait start" vwait forever ====== The ctrl-c event isn't trapped, neither if entered via keyboard, nor via generate... [APN]: It is interference between `twapi::trap` and your own `trap`. Rename your `trap` to something else and it should work. This is a bug in twapi but for now you can work around it by renaming your trap handler. [MHo]: Thanks, I overlooked this... The demo program now 100% works as expected. But my production program (where I used the same logic but named the trap "trapClose"...), when there are many idle events serviced through vwait, still doesn't catch the ctrl-c event, just ends or hangs... [APN] Are you using [after] or [after idle] in your production program ? The ctrl-c handler is a separate thread which queues the event to the Tcl event queue. It then waits for 100ms for a response and calls the default handler if no response is received in that time. I could try increasing that timeout but first it would be good to know if you are continuously calling after idle in which case timer events will not get to run (I think). [MHo]: Yes, I use [after idle] to schedule my events. But as far as I know, idle events are only served ''after'' all other queued events are processed. E.g., callbacks from '''begin_filesystem_monitor''' are all serviced before idle events. Other handlers active in parallel are timer events (after 60000) and fileevents. [APN]: If this is a show-stopper for you, I can create a version with a configurable timeout to see if that helps. But first it would be good to verify that the timer is the problem. I tried the sample session below without any problem. Despite continuous idle scheduling, the ctrl-c handler still worked. Are you sure your handler is either not erroring out or is returning something other than "1" ? ====== % package require twapi 4.0b24 % proc x args {puts HANDLER; return 1} % twapi::set_console_control_handler x x % proc idler args {after idle idler} % idler after#0 % after 60000 exit after#1 % vwait forever HANDLER <- printed when Ctrl-C pressed HANDLER HANDLER HANDLER ====== [MHo]: I can confirm that your example works for me, too. I will try to extend this code until it now longer works... I think a configurable timeout would be great, nevertheless. ---- [MHo]: `eventlog_read xxx -seek yyy` gives error message (4.0b22): ====== package require twapi_eventlog set source "System" set hevl [twapi::eventlog_open -source $source] set recs [twapi::eventlog_count $hevl] set events [twapi::eventlog_read $hevl -seek [expr {$recs-10}]] # print out each record foreach eventrec $events { array set event $eventrec set timestamp [clock format $event(-timewritten) -format "%x %X"] set source $event(-source) set category [twapi::eventlog_format_category $eventrec -width -1] set message [twapi::eventlog_format_message $eventrec -width -1] puts "Time: $timestamp\nSource: $source\nCategory: $category\n$message\n" } ====== ======none D:\Home\Arbeit1\pgm\tcl\usr\Tst\logViewer\winlog>tclkitsh860.exe twapitest2.tcl Falscher Parameter. while executing "ReadEventLog $hevl $flags $offset" invoked from within "trap { set recs [ReadEventLog $hevl $flags $offset] } onerror {TWAPI_WIN32 38} { # EOF - no more set recs [list ] }" (procedure "twapi::eventlog_read" line 32) invoked from within "twapi::eventlog_read $hevl -seek [expr {$recs-10}]" invoked from within "set events [twapi::eventlog_read $hevl -seek [expr {$recs-10}]]" (file "twapitest2.tcl" line 9) ====== [APN]: The argument to -seek is not the index or position of a record in the event log. It is the record number (MS terminology) which might have been better named record id. It increments for each added record but does not necessarily start at 0/1. For what you are doing, you probably want to use `eventlog_oldest` to get the record number of the oldest record, then add `eventlog_count` and subtract 10 from that as the seek position. [MHo]: Yes, that does the trick. Thanks! ---- [MHo] 2013-11-12: Another observation with v4-beta: package require'ing twapi-modular from tcl 8.6: no problem. package require'ing twapi-modular from tcl 8.5 gives this error, e.g.: ======none couldn't load library "d:/var/temp/TCL63.tmp": this library or a dependent library could not be found in library path while executing "load D:/Home/Hoffmann/pgm/tcl/usr/Src/tclstdlib/tclstdlib.kit/lib/twapi-modular/twapi_service.dll twapi_service" ("uplevel" body line 1) invoked from within "uplevel #0 $loadcmd" (procedure "twapi::package_setup" line 47) invoked from within "twapi::package_setup D:/Home/Hoffmann/pgm/tcl/usr/Src/tclstdlib/tclstdlib.kit/lib/twapi-modular twapi_service 4.0b22 lo ad twapi_service {}" ("package ifneeded twapi_service 4.0b22" script) invoked from within "package require $pkt" ("foreach" body line 2) invoked from within "foreach pkt { base64 bgexec crypt1 elog execx2 Globx kill lock md5 mime rc4 readprof smtp socktest twapi_..." (file "test.tcl" line 3) ====== My test program is as simple as follows: ====== source [file join [file dir [info script]] tclstdlib.kit] foreach pkt { base64 bgexec crypt1 elog execx2 Globx kill lock md5 mime rc4 readprof smtp socktest twapi_service tdbc::odbc } { puts "$pkt -> [package require $pkt]" } puts [info loaded] puts [package names] ====== And the tclstdlib.kit contains the ususal structure: ====== \---lib +---base64 +---bgexec +---crypt1 +---elog +---execx2 +---globx +---lock +---md5 +---mime +---rc4 +---readprof +---socktest +---tclkill +---tdbc1.0.0 +---tdbc_odbc1.0.0 \---twapi-modular ====== Made the same observation with another starpack. [APN]: Do you have twapi_base.dll in your vfs ? All twapi modules need that. Although I do expect 8.6 to also fail if that's missing (unless it gets loaded from elsewhere on the 8.6 package path) [MHo] Yes, it's there... [MG]: Is it possible your TWAPI and Tcl 8.6 are 64-bit, but the 8.5 Tcl is 32-bit? I seem to recall getting that error when trying to load the wrong package that way before once. [MHo]: No. In the example shown above, only 32bit modules are involved. With Tclkitsh 8.6, it works, but it did not work with tclkitsh 8.5. In twapi-modular, all modules of the modular-distro are included. The tclkits are as follows: ====== 11.11.2013 11:59 726.519 tclkitsh-cli-8.5.13_32.exe 21.12.2012 17:43 841.274 tclkitsh.exe ====== The first one (8.5.13): ====== D:\Home\Arbeit1\pgm\tcl\usr\Src\tclstdlib>tclkitsh-cli-8.5.13_32.exe % info pa 8.5.13 % parray tcl_platform tcl_platform(byteOrder) = littleEndian tcl_platform(machine) = intel tcl_platform(os) = Windows NT tcl_platform(osVersion) = 5.1 tcl_platform(platform) = windows tcl_platform(pointerSize) = 4 tcl_platform(threaded) = 1 tcl_platform(user) = Arbeit1 tcl_platform(wordSize) = 4 % info loaded {{} registry} {{} dde} {{} zlib} {{} vfs} {{} rechan} {{} tclkitpath} {{} vlerq} % ====== The second one (8.6): ====== D:\Home\Arbeit1\pgm\tcl\usr\Src\tclstdlib>tclkitsh.exe % info pa 8.6.0 % parray tcl_platform tcl_platform(byteOrder) = littleEndian tcl_platform(machine) = intel tcl_platform(os) = Windows NT tcl_platform(osVersion) = 5.1 tcl_platform(pathSeparator) = ; tcl_platform(platform) = windows tcl_platform(pointerSize) = 4 tcl_platform(threaded) = 1 tcl_platform(user) = Arbeit1 tcl_platform(wordSize) = 4 % info loaded {{} registry} {{} dde} {{} vfs} {{} rechan} {{} tclkitpath} {{} Mk4tcl} {{} Itcl} ====== Hm, I see now, one is a Metakit-Version, the other vlerq. But is this relevant? [APN]: For now you can use the twapi-bin distribution. I verified that works with 8.5. With the twapi-modular distribution, the issue is that each module is dynamically linked with the twapi_base dll. The tclkit code copies the module out to a temp area and tries to load it. Windows cannot find the base dll and hence the error. This is the same issue that was discussed in (I think) clt w.r.t to Img or some other extension. Only workaround if you want to use the the modular distribution to reduce size is to copy the twapi_base.dll and twapi_eventlog.dll to the file system and explicitly load from there. I cannot explain how your 8.6 kit works. Possibly the code that implicitly copies the vfs dll to an external dir before loading has changed so that files are not renamed. That could be a 8.5/8.6 difference or a metakit/vlerq difference. [MHo]: I thought in the same direction. But it's irritating, as it works with 8.6... See [dllfix] and [Starkits with Binary Extensions]. ---- [MHo] 2014-03-18: I noticed that it isn't possible to feed the list of options fetched with '''get_service_configuration''' into '''set_service_configuration''': * List of config otions from get_service_configuration: ======none -dependencies {Tcpip Afd} -servicetype win32_own_process -starttype demand_start -errorcontrol normal -tagid 0 -command {D:\home\Hoffmann\bin\apache-tomcat-7.0.50\bin\tomcat7.exe //RS//Tomcat7} -loadordergroup {} -account LocalSystem -displ ayname {Apache Tomcat 7} -interactive 0 -description {Apache Tomcat 7.0.50 Server - http://tomcat.apache.org/} ====== * Error Message if this is applied to set_service_configuration: ======none Invalid option '-tagid'. Must be one of -displayname, -servicetype, -interactive, -starttype, -errorcontrol, -command, - loadordergroup, -dependencies, -account, -password, -system, -database ====== [APN]: TWAPI does not accept setting of -tagid because the underlying ChangeConfig Win32 call prototype marks it as output only. Having said that, I'm sure there will be other situations where the get/set are not symmetrical.win [MHo]: Thanks. The same happes with '''-description'''. Is it possible set the description programmatically, at all? Oh Microsoft... [APN]: The description is just stored in the registry ====== (tcl) 58 % registry get {HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\dxgkrnl} description Controls the underlying video driver stacks to provide fully-featured display capabilities. ====== And if I specify '''-servicetype win32_own_process''', I get a '''expected number but got "win32_own_process"'''-error... This is not the case if using it with '''create_service'''. [APN]: Looking into this, looks like it only happens when -interactive is also specified. Without -interaction being specified, -servicetype win32_own_process works. So you could potentially do the set config in two steps. Or wait for the next beta. So, here's my little tool for migrating a service from machine A to machine B for now: ====== # svcmig1.tcl 19.03.2014 Matthias Hoffmann # Migrating a service from Machine A to Machine B # http://wiki.tcl.tk/9886 lappend auto_path [file dirname [info script]] package require twapi if {$argc < 3} { puts stderr {args: servicename sourcesrv destsrv [password]} exit 1 } if {[catch {::twapi::get_service_configuration [lindex $argv 0] -all -system [lindex $argv 1]} cfg]} { puts stderr $cfg exit 2 } # Später als dictionary handhaben. array set c $cfg parray c # puts $cfg; exit if {[catch { ::twapi::create_service [lindex $argv 0] "dummy" -system [lindex $argv 2] -servicetype $c(-servicetype) unset c(-tagid); # not supported with 'set' unset c(-description); # not supported with 'set' unset c(-servicetype); # expected number but got "win32_own_process" with set_service_configuration set cfg [array get c] ::twapi::set_service_configuration [lindex $argv 0] -system [ lindex $argv 2] {*}$cfg -password [lindex $argv 3] } res]} { puts stderr $res exit 3 } if {[catch {::twapi::get_service_configuration [lindex $argv 0] -all -system [lindex $argv 2]} cfg]} { puts stderr $cfg exit 4 } puts "======================================================================" array set c $cfg parray c ====== ---- [MHo] 2014-11-25: feeding '''write_shortcut''' with the args of '''read_shortcut''' does not work as expected. Even for links to normal files, '''read_shortcut''' gives back funny values for '''-idl''', which I think is the reason that after writing them back the link is destroyed. After removing '''-idl''' from the dictionary before writing back the link, everything works (which is ok right now as I do not need -idl at all). Additionally, I noticed that the case of the given '''-path''' is changed (by TWAPI or OS). So reading it back and comparing with a given (new) -path only works when ignoring the case. See the following wrapper prog as a try to work around all this and to experiment with: ====== package require twapi twapi::import_commands proc writeLink {linkFile args} { if {[file exists $linkFile]} { set update 0 set curArgs [dict remove [read_shortcut $linkFile] -idl] ts $curArgs # mit dict merge haben wir keine Kontrolle über Änderungen, also zu Fuß foreach {k v} $args { puts "$k -> $v" if {[string equal $k "-path"]} { # der Pfad wird vom OS oder TWAPI irgendwie geändert (LW: uppercase), # daher zur Feststellung, ob sich etwas ändert, einen -nocase-Vergleich durchführen! set parm "-nocase" } else { set parm "" } if {[dict exists $curArgs $k] && [string compare $parm [dict get $curArgs $k] $v]} { ts "<<<[dict get $curArgs $k]>>>" ts "<<<$v>>>" dict set curArgs $k $v set update 1 } } } else { set update 1 set curArgs $args } if {$update} { ts "doUpdate" write_shortcut $linkFile {*}$curArgs } } # file delete [file join [get_shell_folder csidl_desktopdirectory] test0000.lnk] # writeLink [file join [get_shell_folder csidl_desktopdirectory] test0000.lnk] -path "d:\\temp\\2014-11-21_Buchungen in Projectile.pdf" writeLink [file join [get_shell_folder csidl_desktopdirectory] test0000.lnk] -path "d:\\temp\\HEK_neue_Systemumgebung.pdf" ====== [APN]: I've added a ticket https://sourceforge.net/p/twapi/bugs/138/ to track this. [APN]: Fixed for 4.1b16. However, the case changing as well as longname to shortname etc. is a Windows thing not in TWAPI's control. [MHo] 2015-02-09: Just noticed that in the path contained in `-path`'s value (oh lord...) backslashes have to be doubled, whereas in `-args`'s value this isn't the case... kind of confusing... [APN] Can you show an example? I don't see this (assuming you mean the write_shortcut command). Thanks [MHo] Yes, write_shortcut. Half of an an example is alreay above, here's another: ====== write_shortcut [file join [get_shell_folder csidl_desktopdirectory] {xyz.lnk}] \ -path c:\\programme\\HEK-Tools\\popsel.exe -args [file nativename $popselfile] ====== As I doubled the backslashes produced from `file nativename $popselfile`, they appear in the link and the link didn't work. But maybe there was some other fact I'm too blind for right now (as usual...). [APN]: The above double-backslash has nothing to do with the command. When you specify a "literal" Tcl will do backslash substitution and hence you need to double them. When the path is in a variable or returned by a command, this backslash subst does not happen. Try [puts] in the two cases to see the difference. [MHo]: Yes, I know this. -path expects Backslashes, as this value is passed 1:1 to windows, I think. And Backslashes in literals have to be doubled to escape them (would be better if one could use forward slashes here, but...). I thougt the same has to be done with this construct (I did a little simplification in the example above...): ====== -args "/n [file nativename $popselfile]" ====== This ''is'' a literal (at first glance), but I forgot the fact that Tcl does only ''one round of parsing'': after replacing the [[...]] with it's result, the string is ready as it is... Oh, shame on me and thanks for your patience. ---- [MHo]: Thanks for your wonderfull TWAPI. It provides a tcl'ish way to administer things on Windows! And thanks for your binary builds, especially that with TWAPI included. Unfortunally, the latest 8.6.4 interpreters are blamed again by our SYMANTEC security software as a risk. I downloaded them, and Symantec isolates them away. Maybe this will happen in production environments as well - not to think of...... [APN]: Mho, could you submit a false positive report at https://submit.symantec.com/false_positive/ I would do so myself but they ask for some details (like Symantec product and version) which I don't know. [MHo]: Symantec End Point Protection 12.1.4013.4013; VirusDefs: 150326016. Events-Details (Example, they're all the same): ====== Sicherheitsrisiko gefunden!WS.Reputation.1 in Datei: D:\home\Hoffmann\Downloads\tclkit-cli-8.6.4-x64.exe von: Auto-Protect-Scan. Aktion: Isolieren erfolgreich : Zugriff verweigert. Beschreibung der Aktion: Die Datei wurde erfolgreich isoliert. ====== Which means: "I don't know the file yet - it must be dangerous..." - What a f.... logic! [APN]: OK, I tried and I give up. Now they want me to fill in the field _Name of detection given by Symantec product_ Is there no way you can just override the quanrantine? Most AV will let you pull the files out of quarantine. [MHo]: I will see if we can send the false-positive-reports. At first, I have to explain to several people, what tcl an twap is all about... [MHo] 2015-03-30: Meanwhile I've submitted a report to Symantec, but only for one of the nine files isolated away, as an example. Let us see what will happen now... [MHo] 2015-03-31: Symantec responded very quick. Unfortunally, they only flagged only ''one'' download as okay. Actually, I didn't have the time to submit false positive reports aboput the eight other downloads... So I wait and hope that many people are try to download your tclkits, so that the reputation will automatically rise and grow above our threshold... ====== "...Upon further analysis and investigation we have verified your submission and as such this detection will be removed from our products...." ====== [APN]: Thanks. This whole situation is just absurd. There is no mechanism to get redress against this form of abuse. ---- [MHo]: Will there be new ''combined Helpfiles''? Or are there no changes at that level? [APN] I'm waiting for tcllib 1.17 to be released so I don't have to rebuild again. ---- [Mho]: I created a tclstdlib.kit with several included libraries, so that my tcl scripts can source it and can use the libraries they usually need. Now I included the newest twapi 4.1.27 in that starkit. The problem is: if I source the kit using an interpreter with an older TWAPI already build in, I get a version conflict. As I use the starkit from several interpreter/script combinations, I wonder if there is any way to work around this, and if it's possible to force the interpreter (with the build-in, older TWAPI) to use the external (newer) TWAPI out of the sourced starkit. This does not work for me even using ''package require -exact...''. Meanwhile I added the following at the top of the main file of the starkit, and this seems to work for every possible interpreter I use (perhaps an older build-in twapi is used, though, as I found no way to force the use of the newer version, that is, to "overwrite" a statically build-in twapi). ====== # tclstdlib.vfs/main.tcl 11.10.2013, 17.12.2014, 19.04.2015 # Massnahme gegen conflicting versions provided for package "twapi_base": 4.0.61, then 4.1.27 if {[lsearch -index 1 [info loaded] twapi*] >= 0} { # Interpreter mit eingebauten TWAPI catch {package require twapi} } package require starkit starkit::startup ====== ---- [MHo] 2015-04-21: Is there any secret in using '''itaskscheduler_get_tasks'''? No tasks are listed on my Windows 7-PC (whether local or remote). tested with TWAPI 4.1.27. Snippet: ====== set hSched [itaskscheduler_new] puts [itaskscheduler_get_tasks $hSched]; # should show something.... ====== Just tested with a tclkit with TWAPI 4.0.61 build-in on Windows 2008 R2. There it works... But still not on Windows 7. It only seems to work on a server. But the underlying APIs should be available on Windows 7 too? Hm, on Win2008, one task is listed, whereas many exist... Now I think ''there is some secret''... [APN]: I think the issue is that Vista and later have two task schedulers - 1.0 (XP compatible) and 2.0. The twapi task interfaces only support the former because the latter already has COM based scripting support that you can access with the twapi COM module. The itaskscheduler_get_tasks will only show tasks that were scheduled with the older API. For using the new 2.0 task scheduler with COM, see https://msdn.microsoft.com/en-us/library/windows/desktop/aa384006(v=vs.85).aspx but note that will not work with XP and Win 2003. You are seeing tasks on Win2k8 because they were probably created using the older API. [MHo] Thanks for the fast response. Ok, I have to dive into the horrors of COM...;-) But I'm sure the abstraction of your taskscheduler commands is a better one than using com... ---- [MHo] 2015-06-08: Are there any potentiell reasons why a call to '''begin_filesystem_monitor''' could fail? I have a production programs which ran unobtrusive for months, but since I changed from tcl 8.5 to 8.6 and twapi from 4.0.x to 4.1.x (ok, besides changes in *my* code of course) I see mysterious hangs. I added debug output to narrow down what happens, and today it seems that a call to '''begin_filesystem_monitor''' never returned. I don't know how to handle this case. I don't use catch and I don't get a run time error...: ====== : :putsLog "main(): Programmstart, arg=$argv, v=$_v" debugOut "main(): LogKonfig: File=[$::elog getLogFile] MaxSize=[$::elog getLogMaxSz] Buffering=[$::elog getLogBuffering]" <=== appears in the log set stopFile [file join [file dirname [info nameofexe]] newfileaction.stop!] set ids [list] set pCount 0 set aliveCount 0 set aliveOff 0 set endRequested 0 foreach {dir pattern action} $oplist { set id [begin_filesystem_monitor $dir [list cb $dir $action] -patterns $pattern {*}$monParms] lappend ids $id putsLog "main(): [list begin_filesystem_monitor $dir [list cb $dir $action] -patterns $pattern {*}$monParms]" <=== does not appear in the log } : : ====== Ok, could be that only the log entry is not flushed, etc., but I proved that the program indeed did'nt carried out further action... So, it looks to me that *maybe* the '''begin_file_system''' is the last statement the programm processed before "hanging".... [APN]: Never say never, but I doubt a change from 4.0.x to 4.1 would cause this as I didn't see anything directly related to file system monitoring in the twapi change log. Can you put 4.0.x back (keeping Tcl 8.6 and your new code) and see if the hang goes away? Is it possible that the patterns you are specifying might have changed causing the internal matching code to may be spin in an infinite loop due to a bug? Also, can you tell if the hang is actually a CPU spin (by looking at task manager). [MHo]: Thanks. The problem here is that the problem occures only very seldom, from time to time once a month or so ;-) Unfortunally, I don't remember the details, but the "hanging" prog didn't use cpu cycles, and there was no disk i/o. Very strange. The changes I made to my code are minor, the $dir, $action, $monParms didn't change etc. ---- How can I find examples of using the raw win32 api with TWAPI? [APN] You can look at the Tcl scripts that implement the high level commands since these make use of the raw api! Note that there is very little of the raw win32 api that TWAPI supports that is not already covered by the high level commands. [MHo] Thanks. Eventually I need AddFontResource(). [APN] That you should be able to call as ====== set nfonts [twapi::AddFontResource c:/path/to/font/resource/file] ====== Hm, I get "invalid command name twapi::addFontResouce"... But it seems to work with '''twapi::AddFontResourceEx'''. ---- [MHo] 2015-10-15: Just realized that the tclkits provided on the twapi site do not contain the thread package......are there any reasons for this...? [APN] No particular reason. It's not there because the line between what should be included and what should not is different for different folks. ---- [MHo] 2015-10-19: '''end_process''' returns 1 if specifying non-existant PIDs? '''get_process_info nnnn -pid''' returns given pid even if it doesn't exist: ====== % get_process_info 1144 -noexist -1 -pid; # this is an existent process -pid 1144 % get_process_info 1144999 -noexist -1 -pid; # this pid does not exist -pid 1144999 % ====== [APN] Both are by design. Semantics of end_process are to return 0 if a process with that PID exists when the call returns and 1 otherwise. This is more useful than generating an error, for example in the case where a process is ensuring all child processes are ended before it itself exits. Regarding -pid, that field is considered the "key" and always returns whatever is passed in. The -noexist or -noaccess options do not apply to it. [MHo] hm.... From The Help: end_process PID ?options? Terminates the process with id PID. This function will first try to gracefully end the process by sending all its toplevel windows a WM_CLOSE command. If the process does not have any toplevels or if it does not end by this means and the -force option is specified, it is forcibly terminated. The function returns 1 if the process ended and 0 otherwise. Note that if -wait has not been specified, or is too short, the function may return a value of 0 in the case where the process is in the process of terminating but has not fully exited by the time the command returns. * PID exists and kill succeed --> return 1 * PID exists and kill failed ---> return 0 * PID does not exist -----------> return ????? ---- [Mho] 2016-11-04: I noticed, that Tk programs do not handle WM_QUERYENDSESSION. I wonder if TWAPI can help in trapping logoff or shutdown events for GUI programs? Oh, I just looked into the Tk sources and found out that WM_SAVE_YOURSELF is synthesized in such cases. And it seems to work. ---- [MHo] 2017-07-03: Scenario: * Files are incoming with a name of ''file''.tmp * When complete, files are renamed to ''file'' * So '''begin_filesystemmonitor''' should not trigger for '''*.tmp''', but for any other file: '''-patterns {-*.tmp +*}''' * '''begin_filesystemmonitor''' indeed ingnores the .tmp-files * but it also ignores the renaming, so nothing is triggered... Options in effect are '''-filename 1 -write 1''' Oops - I'm sorry - just found the error: only gave the first part of the pattern.... now rename triggers a renamenew event... everythings works as excpected now... ---- [MHo] 2017-07-05: It would be usefull, if the pattern matching done by '''begin_filesystem_monitor''' would be available per API. I have a scenario where I must pick up accumulated files "manually" when re-starting an interrupted program, which normally picks up the files with begin_filesystem_monitor... For now, I'll try to re-implement the matching logic... First try: ====== # re-implementation of twapi::begin_filesystem_monitor´s -patterns-Matching: # # If the -patterns option is specified, then the value associated with the # option is a pattern filter that is used to match against the name # component being reported. SCRIPT will be invoked only if the pattern # filter matches. # # Each element of the pattern list is a string beginning with either + or - # followed by a pattern that is used for matching using the same rules as # for the Tcl string match -nocase command. Note that \ path separator need # to be escaped with another \ just as in the string match case. # Alternatively, you can use / as the path separator in the pattern. # # The name component being reported is matched against each element of the # pattern list. When a match occurs, further elements of the pattern list # are not matched. If the matching element is prefixed with a +, then # SCRIPT is invoked. If the matching element is prefixed with a -, then the # pattern filter is deemed to have not matched and SCRIPT is not invoked for # the notification. Even in this case, no further matching is attempted # against the remaining patterns. Pattern elements that do not begin with # +, or - are treated as having an implicit + prefix. # # If a non-empty pattern filter is specified but no element matches, the # pattern filter is treated as having not matched, and SCRIPT is not # invoked. Thus a pattern filter that only has - type elements will never # match anything. Note a pattern list where no element matches is different # from the case where -patterns is not specified or PATTERNLIST is empty in # which case the pattern filter is treated as matched. # # Note that the pattern matching is done only against the name component # being reported, not against the full path. # # Some examples of pattern matching are below (assume callback is your # SCRIPT procedure): begin_filesystem_monitor c:/temp callback -patterns # {*.exe} will only monitor files in c:\temp with .exe extensions. # begin_filesystem_monitor c:/temp callback -patterns {-*.foo +*.fo*} will # monitor files in c:\temp with an extension beginning with .fo except if # the extension is .foo. begin_filesystem_monitor c:/temp callback -patterns # {+*.fo* -*.foo} will monitor files in c:\temp with an extension beginning # with .fo including those with extension .foo. The second pattern -*.foo # has no effect as the matching stops as soon as a match is found. # proc tbfm_match {patterns singlefile} { if {[llength $patterns] == 0} { return 1; # pick the file if patterns empty } set f [file tail $singlefile] foreach pattern $patterns { set prefix [string range $pattern 0 0] if {$prefix eq "+" || $prefix eq "-"} { set pattern [string range $pattern 1 end] } else { set prefix "+" } if {[string match -nocase $pattern $f]} { return [expr {$prefix eq "+"}]; # no further examinations } } return 0 } # Some tests foreach {patterns testfile expRC} { {} shouldmatch 1 {-*.tmp +*} one 1 {-*.tmp +*} two.dat 1 {-*.tmp *} one 1 {-*.tmp *} two.dat 1 {-*.tmp *} no.tmp 0 -* nevermatch 0 * always 1 {-*.tmp -a*.* *} ok 1 {-*.tmp -a*.* *} a.notok 0 {-*.tmp -a*.* *} a 1 } { puts [format {%20s %20s %i %i} $patterns $testfile $expRC [tbfm_match $patterns $testfile]] } ====== Results: ====== >tclkitsh tbfm_match.tcl shouldmatch 1 1 -*.tmp +* one 1 1 -*.tmp +* two.dat 1 1 -*.tmp * one 1 1 -*.tmp * two.dat 1 1 -*.tmp * no.tmp 0 0 -* nevermatch 0 0 * always 1 1 -*.tmp -a*.* * ok 1 1 -*.tmp -a*.* * a.notok 0 0 -*.tmp -a*.* * a 1 1 ====== [APN] Although implemented a long while ago, from what I recall, my thinking was at the time that there can be any number of needs in terms of pattern matching and these can be met via implementing them in the callback itself. It's not clear to me that making -patterns customizable would be better, particularly because it needs to be fast. Or perhaps I misunderstood your request. [MHo] Thanks. Yes, every individual matching/filtering can be done in the callback. The machting itself is fine as it is. I just need a matching elsewhere which exactly behaves like your's, so the question was if your matching could be exposed to the script-level as something like '''twapi_base::fsm_match_oneFile''' which takes a list of patterns like begin_filesystem_monitor and a single filename. [MHo] Hm, it's not possible to directly retrieve the ''remotedesktop-services profilepath'' account attribute (in german: "Remotedesktopdienste-Benutzerprofil Profilpfad"), I fear... One way I found: ====== set objectTrans [comobj "NameTranslate"] $objectTrans -call init 3 "" $objectTrans -call set 3 "$yourDomain\\$theAccount" set ou [$objectTrans -call get 1]; # OU from SAMAccountname, or directly start with this if known set objUser [comobj_object "LDAP://$ou"] set tsProfPath [$objUser TerminalServicesProfilePath] $objUser -destroy ====== ----- Updated the version info at the very top. Question: is an update of the tclkits at https://sourceforge.net/projects/twapi/files/Tcl%20binaries/ to version 8.6.7 planned? ---- [MHo] 2018-01-26: It should be noted that only ''services'' always receive the '''shutdown'''- or '''logoff'''-events (if they registered themself for this). Console programs (via '''set_console_control_handler''') oder Tk-GUIs (via WM_SAVE_YOURSELF) sometimes aren't notified about shutdown (if running via taskplanner in system context, see https://stackoverflow.com/questions/24911125/does-scheduled-task-receive-wm-queryendsession-message). See https://docs.microsoft.com/en-us/windows/console/handlerroutine. Additional infos are at https://www.apriorit.com/dev-blog/413-win-api-shutdown-events. ---- [MHo] 2018-02-01: later windows versions have an (obscure;-) facility to control the ''order in which services are shutting down''.... see https://blogs.technet.microsoft.com/askperf/2008/02/04/ws2008-service-shutdown-and-crash-handling/. Some additional info from https://answers.microsoft.com/en-us/windows/forum/windows_xp-windows_programs/windows-services-shutdown-order/32547433-9c87-483a-a6d9-8a7131c2e57e?auth=1: "Service Control Handler Function looks interesting. Consider handling SERVICE_CONTROL_SHUTDOWN, and coordinating shutdown among your services in that fashion. Use the dwCheckPoint and dwWaitHint members of the SERVICE_STATUS struct to provide feedback to the SCM regarding service shutdown progress." [MHo] 2018-02-09: There also is an '''service_control_preshutdown'''-Event, but I don't know how to trap it. And a function called '''SetProcessShutdownParameters()''' to help determining the order in that processes are shutting down. Both of them, I fear, are not available via twapi... What I noticed is that simply catching shutdown in a service and trying to call remote functions to other machines eventually does not work (I think, other required services for this are alreay terminated then.). I get an error like this: ====== -code 1 -level 0 -errorstack {INNER {invokeStk1 OpenSCManager remotesys {} 0x00020000} CALL {get_service_status PatrolAgent -system remotesys} CALL {get_service_state PatrolAgent -system WB101SW0225} CALL onShutdown} -errorcode {TWAPI_WIN32 1722 {Der RPC-Server ist nicht verfügbar.}} -errorinfo {Der RPC-Server ist nicht verfügbar. while executing "OpenSCManager $opts(system) $opts(database) 0x00020000" (procedure "get_service_status" line 4) invoked from within "get_service_status $name {*}$args" (procedure "get_service_state" line 2) invoked from within "get_service_state $svc -system $otherSrv"} -errorline 1 ====== Windows does not respect service dependencies during shutdown. All Services receive the shutdown signal at the same time, normally. And then, it's simply to late for some actions, as part of the APIs are already gone....:-( Scheduled tasks are simply killed, so even waiting for an Win32_ComputerShutdownEvent doesn't work. Only programs started by users are notified, it seems. So, I am badly in need for '''preshutdown-notifications'''... ---- [AMG]: I am having trouble embedding TWAPI in a Starpack. When the DLL is loaded, it tries to source Tcl files which it expects to be located in the same directory as the DLL. However, a Starpack is not a typical TWAPI installation. The DLL is located in a temporary directory just long enough for Windows to load it, and there are no additional files in that directory. For now I think the solution is to use the twapi-bin distribution which I suspect incorporates the Tcl scripts directly into the DLL, but I would prefer an alternative that allows the Tcl scripts to be sourced from the Starpack VFS. I'm not sure why it is necessary that the DLL source the Tcl scripts; usually it's the other way around, with the DLL providing the low-level capability and the Tcl scripts providing the end interface. If the DLL actually depends on the Tcl scripts, I feel it is acceptable to have errors should the user manually load the DLL, bypassing the normal script initialization routine ([package require], etc.). ---- [MHo] 2018-06-19: In future releases, it would be great if TWAPI would support ANSI Escape Sequences (ENABLE_VIRTUAL_TERMINAL_INPUT 0x0200) with '''modify_console_output_mode''', see https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences and https://docs.microsoft.com/en-us/windows/console/setconsolemode. This is a feature missing for years in modern windows versions (ages ago, in MSDOS and MSDOS-based Windows versions, this function was available by loading the ANIS.SYS device driver...), but it's now back in Windows 10. In a cmd.exe shell it seems that ANSI processing is enabled by default now, but the [Console Text Editor in Pure Tcl 2] for example does not work as expected... [APN] Could you try ====== twapi::SetConsoleMode [twapi::get_console_handle stdout] 4 ====== and see if that does the needful? (4 -> ENABLE_VIRTUAL_TERMINAL_PROCESSING) [MHo] Hm, no this does not help there...maybe some other reason.... i will try and report further... [APN] Just to double check - the input and output mode changes are separate. The 4 refers to output processing. For input you would specify 0x0200 as you stated above. [MHo] Until yet I couldn't get it to work...just tried on windows 2016 server... [APN] Ok, I see that you also have to enable the PROCESSEDOUTPUT flag. So the correct call is ====== twapi::SetConsoleMode [twapi::get_console_handle stdout] 5 ====== (4 -> virtual terminal, 1 -> processed output) This worked for me to output color sequences. For example, ====== puts "\033\[0;32mTEST\033\[0m" ====== will output a green TEST. [MHo]: Aaaah - yes - this does the trick - many thanks! ---- [MHo] 2018-10-12: * Does a downloadable version of the html html help files of TWAPI 4.3.5 exist somewhere? * Will be ever an updated chm combined help file? * Will there be updated tclkits available some day with TWAPI 4.3.5 included...? Unfortunally, although I'm a "hardcore tcl user", I'm not able to help myself regarding the above topics.... [APN] Updated tclkits will be available concurrent with Tcl 8.6.9 release. Regarding the help files, the Microsoft HTML CHM compiler crashes on my system and I haven't been bothered to figure out why. Given that its last release was sometime around 1998, it's a miracle it's worked for so long. [MHo] 2019-01-15: * I get stucked with [SASL and TWAPI]. I wand to send SMTP-Mails via Exchange. The only option is NTLM. But I don't want to specify User and Password.... [APN] I commented on that page. ---- [MHo] 2019-07-05: '''Big Problem'''. I operate several Tcl programs which run without interruptions several day, weeks or months. In some of them, I recently noticed growing "private bytes" usage (via ProcMon). I tried to narrow down which construct is responsible for this phenomen, which I didn't notice on the OS before Windows 2016 and with a twapi-tclkit before 8.6.9, I think...... If you run simply run this loop, you will notice an ever growing memory consumption.... copying or deleting a large number of files pushes on the growth...: ====== package require twapi twapi::import_commands proc cb {h info} { lassign $info op obj after idle [list puts [format {%10s %s} $op $obj]] } begin_filesystem_monitor c:/ cb -patterns * -access 1 -attr 1 -create 1 -dirname 1 -filename 1 -secd 1 -size 1 -write 1 -subtree 1 vwait forever ====== My real programs depend on begin_filesystem_monitor, so what option do I have other then periodically (self-)restart the script (and/or interpreter)? 2019-07-29 [APN] Logged as a bug at https://sourceforge.net/p/twapi/bugs/179/. This should be fixed in 4.3.7. Please try and re-open the bug if you still see the same memory growth. ---- [MHo] 2020-06-02: I've problems enumerating Groups with get_global_groups. The command only return ~2500 groupnames, but thousands more exist. Is there any trick I'm missing? [APN] Did you use the -resume option? [MHo]: Yes I tested this as well, but the ''moredata'' return value says there is no more data (0).... [APN] Unfortunately I no longer have a domain environment to test this. Are you sure the account running twapi has access rights to those groups? Can you try running this program - https://nettools.net/netgroupenum/ - and see if you get all the groups. [MHo]: Unfortunally, like for many other URLs, the nettools-URL is blocked by our gateway security software.... So I did this: ====== >dsquery group -s hekintra.de -limit 0 > dsquerygroup.txt # gives 5150 entries >tclkitsh % packa re twapi; twapi::import_commands % set g [get_global_groups -system hekintra.de]; llength $g 2244 ====== [APN]: dsquery uses ADSI interfaces. I was trying to see if the issue was in TWAPI or a limitation of NetGroupEnum, thus I asked to try that specific tool. Difficult to make progress without a domain to test with. Can you try using TWAPI's COM commands (comobj) to get the groups with ADSI and see if that works better? [MHo] I will try this a soon as possible. Meanwhile I asked a coworker to do a NET GROUP on the domain controller, as I believe that this also uses the Win32 functions and not something newer like ADSI. NET GROUP lists 2252 Groups (a few more are added meanwhile).... now what's going on here...... I'll have to compare the results side by side to see what groups are missing. Could it really be true, that the older interfaces have a some coded limits...? I fear, inside windows, *everything* is possible..... I discussed further with my AD guy. Maybe groups of a certain (newer) types are only handled by newer APIs, like Exchange-Groups or so. Further investigations done: The output of twapi is almost identical with that of NET GROUP: TWAPI lists a groupname beginning with the (I think illegal) character §, while NET GROUP does not, and for another name vice versa (currently I'm seeing this group in the NET GROUP-Output which TWAPI does not include: õ13Abs.2_SGB_V). So, I think TWAPI works perfect! What I guess is that the following groups are not listed by NetGroupEnum&Co: * Groups with a different Pre-Win-2000-Groupname * Domain-local-groups Indeed, to see such groups, one have to use ADSI etc.: ====== package require twapi twapi::import_commands ################# ### AD-Suche als Coroutine # Portiert von: Windows Script Referenz, T.Weltner, S. 306 ### Parameter: # filter: LDAP-Suchfilter-Ausdruck # return: Liste von Attributen (durch , getrennt), die zurückgegeben werden sollen # ADsPath: Optionaler base (root) für Suche; falls fehlend := DefaultNamingKontext ### Achtung: Fehler werden hier aktuell nicht abgefangen, das ist Aufgabe des Aufrufers! ### Siehe auch: # - http://www.selfadsi.org/search.htm#Prepare # - http://www.selfadsi.de/ldap-filter.htm # - https://administrator.de/forum/active-directory-emailadressen-datei-ausgeben-74893.html # - https://www.faq-o-matic.net/2005/04/27/alle-mailadressen-anzeigen/ # - https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx # - https://www.w3schools.com/asp/ado_ref_recordset.asp # proc searchAD {filter return {ADsPath ""}} { # ADODB-Verbindung einrichten set connection [comobj "ADODB.Connection"] $connection -set Provider "ADsDSOObject" $connection Open # Abfrage formulieren set command [comobj "ADODB.Command"] $command -set ActiveConnection $connection $command -set Properties "Page Size" 5000; # 1000 $command -set Properties "Cache Results" True if {$ADsPath eq ""} { # ADsPath der Domäne ermitteln set rootDSE [comobj_object "LDAP://rootDSE"] set ADsPath [$rootDSE -call Get defaultNamingContext] } # Kommando konstruieren set query ";$filter;$return;subtree" # puts $query # Kommando ausführen $command -set CommandText $query set recordSet [$command Execute] # set ret [list] yield $recordSet; # oder yield if {[$recordSet RecordCount]} { while {![$recordSet EOF]} { # puts [$recordSet GetString]; # nicht alle Objekte haben offenbar GetString! # set d [dict create] set l [list] set o [$recordSet Fields] for {set i 0} {$i < [$o Count]} {incr i} { set item [$o item $i] # dict set d [$item Name] [$item Value] lappend l [$item Value] } # lappend ret $d # lappend ret $l # stattdessen: yield $l # if {![$recordSet EOF] && ![$recordSet BOF]} $recordSet MoveNext } } $recordSet Close return -code error "EOF" } if {![catch {coroutine getGrp searchAD \ {(&(objectCategory=group)(objectClass=group))} \ {sAMAccountName} \ {dc=hekintra,dc=de}}]} { while {![catch {getGrp} rec]} { puts $rec } } ====== Don't know why, but this output again differs from DSQUERY GET. Maybe the search spec I used isn't correct. But all this shows: The simple question "What Groups exist?" has ''no simple answer''... <> Foreign Interfaces | Package | Windows