OS-independent compiled extensions

jcw 2011-04-19 - With NaTcl illustrating that it's possible to run the same compiled code across different OS platforms, I'm wondering whether it'd be possible to do the same for Tcl extensions:

• a single build of a compiled C/C++ extension could be used under Windows, Mac OSX, and x86-based Linux
• you can't depend directly on OS-specific stuff, such as file system calls, but Tcl's stubs-based C API can provide alternatives (e.g. channels, usable from C as well as Tcl)
• the Tcl core already provides an OS-like context for scripts to all sorts of useful things, so it's only logical to re-use this de-facto system API for a lot more C extensions as well
• if special access is needed, run-time binding to shared libs could still be used to tie into OS-specific code, provided that the C code does a bit more setup at run-time
• complex extensions could be split in two part, an OS-independent part and a couple of OS-specific ones (more of these can then be added after the fact)
• the stubs mechanism is already extensible, allowing C extensions to efficiently expose call vectors for other C extensions to use (as with the Img library)
• it's not necessary to change the Tcl core for any of this, both loading and tying into Tcl more intimately could be done with a special-purpose "bridge" extension
• if extended with cross-OS support, Critcl could bring one-shot C/C++ extensibility to any one running Windows, Mac OSX, or Linux - i.e. compile once, run across OS'es

There are a range of hurdles, but for new developments such as NaTcl to become widespread, it may simply be unavoidable in the longer term.

The biggest hurdle may be how to implement a binary code loading mechanism which works across OS's (and which passes each of their code-signing requirements). And the next one is no doubt figuring out how to generate the binary code in this to-be-define binary extension format. IIRC, the Apache web server platform once did this sort of stuff.

RLE (2011-04-19) If memory serves, X.org already has an OS independent loading mechanism for loading compiled video drivers and other compiled extensions into the X server. Maybe you might want to look there for inspiration.

MS notes that at least Windows uses a different C calling convention from the others. This sounds like very problematic to me, I don't know how NaCl does it. Possibly by refusing to link in native libraries that are not part of the NaCl ecosystem?

RLE (2011-04-20) It would seem that such could be worked around by a small wrapper that switched calling conventions before jumping into the called function, then reversed the change upon return from the called function. Yes, a small performance loss for the side that contains the mismatch, but that loss might be worthwhile if a single compiled extension could work cross platform.