<<TOC>>
[FM] [TIP 764] aims to offer annotations to Tcl. Annotation are braced-prefix not followed by a space.
In the case of variables, annotations are taken as types. Thus, TIP 764 introduce a type system in Tcl.
** Presentation **
A type is defined with the command type def :
======tcl
type def ligne3 list(3).double; # 1 row of 3 float numbers
type def {-compact}matrix3 list(3).ligne3; # 3 rows
======
A typed var can be set with a list of arguments :
======tcl
# 3 rows of 3 numbers
set {matrix3}M {1 2 3} {4 5 6} {7 8 9}; # {1.0 2.0 3.0} {4.0 5.0 6.0} {7.0 8.0 9.0}
======We can access its parts through an array-like syntax, the index following a little language (taken from [a better way to do indexation]).
======
set M(1.1); # result : # 5.0 -> '.' is a nesting index operator
set M(0.0,1.1,2.2); # 1.0 5.0 9.0 = cells 0.0 1.1 2.2 -> ',' is a collecting index operatorset M((0,1).(1,2)); # {2.0 3.0} {5.0 6.0} = in rows 0,1 takes cols 1,2 -> '()' are used to group, and groups beeing multiplied by '.'.
set M(0:end.(0,1)); # {1.0 4.0 7.0} {2.0 5.0 8.0} = 2 first cols -> ':' notice a range
======
Let's play with a matrix type in Tcl. Let's define a matrix as 3 rows of 3 double number.
** Matrix example in Tcl **
======tcl
type def ligne3 list(3).double; # 1 row of 3 float numbers
type def {-compact}matrix3 list(3).ligne3; # 3 rows
======the option `{-compact}` here create objestructs whosere fields are arranged in array of cells.
Let's use this definition :
======tcl
# determinant
proc mat::tcl_det {M} {
expr {$M(0.0) * ($M(1.1) * $M(2.2) - $M(1.2) * $M(2.1))
- $M(0.1) * ($M(1.0) * $M(2.2) - $M(1.2) * $M(2.0))
+ $M(0.2) * ($M(1.0) * $M(2.1) - $M(1.1) * $M(2.0))}
}
# multiplication
proc mat::tcl_mul {A B} {
set {matrix3}C {0 0 0} {0 0 0} {0 0 0}
for {set i 0} {$i < 3} {incr i} {
for {set j 0} {$j < 3} {incr j} {
set C($i.$j) [expr {$A($i.0) * $B(0.$j) + $A($i.1) * $B(1.$j)
+ $A($i.2) * $B(2.$j)}]
}
}
return $C
}
======
But how does-it compare to usual Tcl values ? First introduce equivalent Tcl procedure
======tcl
# determinant
proc brut_det {M} {
lassign $M r0 r1 r2
lassign $r0 a b c; lassign $r1 d e f; lassign $r2 g h i
expr {$a*($e*$i-$f*$h) - $b*($d*$i-$f*$g) + $c*($d*$h-$e*$g)}
}
# multiplication
proc brut_mul {A B} {
set C {{0.0 0.0 0.0} {0.0 0.0 0.0} {0.0 0.0 0.0}}
foreach i [lseq 3] {
foreach [lseq 3] {
lset C $i $j [expr {[lindex $A $i 0] * [lindex $B 0 $j]
+ [lindex $A $i 1] * [lindex $B 1 $j]
+ [lindex $A $i 2] * [lindex $B 2 $j]}]
}
}
return $C
}
======
In the typeparse/exemples/matrices folder of the branch, there is a benchmark. Here are the result, with the following values :
======tcl
# typed vars :
set {matrix3}M {1 2 3} {4 5 6} {7 8 9}
set {matrix3}N {9 8 7} {6 5 4} {3 2 1}
# usual lists :
set L {{1 2 3} {4 5 6} {7 8 9}}
set K {{9 8 7} {6 5 4} {3 2 1}}
======
&| data | proc | timerate |&
&| typed|mat::tcl_det | 3.14 µs|&&| usual|brut_detcl | 0.38 µs |&
&| typed|mat::tcl_det | 30.12 µs|&&| usual|brut_tcmul | 8.34 µs |&
Using typed values is between 3.5 and 9 times slower than usual Tcl values. It is not a surprise : in Tcl, lists can never fail. The objects it contains are always valid. This is not the case of our matrices : they are always checked so that they can't contain anything that doesn't suit to their initial declaration. Illustration :
======tcl
# With typed var :
set R {{1 2 3} {4 a 6} {7 8 9}}set M $R; # complains : 'expected floating-point number but got "a"' -> error at the point where invalid data occurwas set.
set M; # {1.0 2.0 3.0} {4.0 5.0 6.0} {7.0 8.0 9.0} -> because of the error, M has not been set.
# With usual var :
set L $R; # {1 2 3} {4 a 6} {7 8 9} -> no error : it's a valid list.
brut_det $L; # cannot use non-numeric string "a" as left operand of "*" -> error during a posterior calculation, maybe a lot later
======
SoThus, here the type has givesn a garantee, but there is a price to pay : to be slower.
** Matrix example in C **
That said, as we know the shape of the data, we can easily export it to C. Tthere is a command `type` includes a subcommand `export` :
======tcl
type {c!}export matrix3; # create an header in c for the tcl type matrix3.
======
Here is the result :
======c
/* Produit par type {c!}export : little-endian, pointeurs de 8 octets. Ne pas modifier : regénérer. */
#include <stdint.h>
#include <stddef.h>
/* ligne3 */
typedef double ligne3[3];
_Static_assert(sizeof(ligne3) == 24, "ligne3: size");
/* matrix3 */
typedef ligne3 matrix3[3];
_Static_assert(sizeof(matrix3) == 72, "matrix3: size");
======WThis header can be included it directly in a c file :
======c
#include <tcl.h>
#include "../../tclAnnotatedApi.h" /* Waiting for C Stubs Annotation Api */
#include "mat_types.h" /* produit par type {c!}export */
*
* Cells of a value whose elements are of type named elemType
* (the name that Tcl gives : `type get` of one element) and of elemSize
* octets ; *cellsPtr = NULL if the value is not in cells.
*/
static int
GetCells(Tcl_Interp *interp, Tcl_Obj *obj, const char *elemType,
size_t elemSize, const void **cellsPtr, Tcl_Size *nPtr)
{ Tcl_Obj *t = Tcl_NewStringObj(elemType, -1); // the type name, as a string object, shared.
int rc;
Tcl_IncrRefCount(t);
rc = Tcl_AnnotatedGetCells(interp, obj, t, elemSize, cellsPtr, nPtr);
Tcl_DecrRefCount(t);
return rc;
}
/*
* A matrix3 in C. In Cells : a copie of 72 octets from the Cells
* (the pointer is available when the value desn't change). Else : with
* lists, checking the shape.
*/
static int
GetMatrix(Tcl_Interp *interp, Tcl_Obj *obj, matrix3 m)
{
const void *cells;
Tcl_Size n, nr, nc, i, j;
Tcl_Obj **rows, **cols;
/* matrix3 in Cells : 3 Cells per ligne (ligne3, 24 octets) */
if (GetCells(interp, obj, "ligne3", sizeof(ligne3), &cells, &n) != TCL_OK) {
return TCL_ERROR;
}
if (cells != NULL && n == 3) {
memcpy(m, cells, sizeof(matrix3));
return TCL_OK;
} /* Not in Cells, use List but check the shape of it */
if (Tcl_ListObjGetElements(interp, obj, &nr, &rows) != TCL_OK) {
return TCL_ERROR;
}
if (nr != 3) { /* there be 3 rows */
goto bad;
}
for (i = 0; i < 3; i++) {
if (Tcl_ListObjGetElements(interp, rows[i], &nc, &cols) != TCL_OK) {
return TCL_ERROR;
}
if (nc != 3) { /* there must be 3 cols */
goto bad;
}
for (j = 0; j < 3; j++) {
if (Tcl_GetDoubleFromObj(interp, cols[j], &m[i][j]) != TCL_OK) { /* it must be a number */
return TCL_ERROR;
}
}
}
return TCL_OK;
bad:
Tcl_SetObjResult(interp, Tcl_ObjPrintf("expected a 3x3 matrix but got "
"\"%.50s\"", Tcl_GetString(obj)));
return TCL_ERROR;
}
/* A new matrix3, in Cells, typed matrix3. */
static int
NewMatrix(Tcl_Interp *interp, const matrix3 m, Tcl_Obj **resPtr)
{ Tcl_Obj *t = Tcl_NewStringObj("matrix3", -1); // the type name, as a string object, shared.
int rc;
Tcl_IncrRefCount(t); rc = Tcl_AnnotatedCreateCells(interp, t, m, 3, sizeof(ligne3), resPtr); // copy the content of matrix m
Tcl_DecrRefCount(t);
return rc;
}
# determinant in C
static double
Det(const matrix3 m)
{
return m[0][0] * (m[1][1] * m[2][2] - m[1][2] * m[2][1])
- m[0][1] * (m[1][0] * m[2][2] - m[1][2] * m[2][0])
+ m[0][2] * (m[1][0] * m[2][1] - m[1][1] * m[2][0]);
}
# determinant Command for Tcl Interp
DetObjCmd(void *cd, Tcl_Interp *interp, Tcl_Size objc, Tcl_Obj *const objv[])
{
matrix3 m;
(void) cd;
if (objc != 2) {
Tcl_WrongNumArgs(interp, 1, objv, "matrix");
return TCL_ERROR;
}
if (GetMatrix(interp, objv[1], m) != TCL_OK) {
return TCL_ERROR;
}
Tcl_SetObjResult(interp, Tcl_NewDoubleObj(Det(m)));
return TCL_OK;
}
# Multiply two Matrices in C
static int MatMult(const matrix3 a, const matrix3 b, matrix3 c) {
int i, j;
for (i = 0; i < 3; i++) {
for (j = 0; j < 3; j++) {
c[i][j] = a[i][0] * b[0][j] + a[i][1] * b[1][j] + a[i][2] * b[2][j];
}
}
return 1;
}
# Command to Multiply two Matrices Command for Tcl Interp
static int
MulObjCmd(void *cd, Tcl_Interp *interp, Tcl_Size objc, Tcl_Obj *const objv[])
{
matrix3 a, b, c;
Tcl_Obj *res;
(void) cd;
if (objc != 3) {
Tcl_WrongNumArgs(interp, 1, objv, "matrixA matrixB");
return TCL_ERROR;
}
if (GetMatrix(interp, objv[1], a) != TCL_OK
|| GetMatrix(interp, objv[2], b) != TCL_OK) {
return TCL_ERROR;
}
MatMult(a,b,c);
if (NewMatrix(interp, (const double (*)[3]) c, &res) != TCL_OK) { return TCL_ERROR; }
Tcl_SetObjResult(interp, res);
return TCL_OK;
}
static int
MulObjCmd(void *cd, Tcl_Interp *interp, Tcl_Size objc, Tcl_Obj *const objv[])
{
matrix3 a, b, c;
Tcl_Obj *res;
int i, j;
(void) cd;
if (objc != 3) {
Tcl_WrongNumArgs(interp, 1, objv, "matrixA matrixB");
return TCL_ERROR;
}
if (GetMatrix(interp, objv[1], a) != TCL_OK
|| GetMatrix(interp, objv[2], b) != TCL_OK) {
return TCL_ERROR;
}
for (i = 0; i < 3; i++) {
for (j = 0; j < 3; j++) {
c[i][j] = a[i][0] * b[0][j] + a[i][1] * b[1][j] + a[i][2] * b[2][j];
}
}
if (NewMatrix(interp, (const double (*)[3]) c, &res) != TCL_OK) {
return TCL_ERROR;
}
Tcl_SetObjResult(interp, res);
return TCL_OK;
}
int
Mat_Init(Tcl_Interp *interp)
{
if (Tcl_InitStubs(interp, "9.0", 0) == NULL) {
return TCL_ERROR;
}
Tcl_CreateObjCommand2(interp, "::mat::det", DetObjCmd, NULL, NULL);
Tcl_CreateObjCommand2(interp, "::mat::mul", MulObjCmd, NULL, NULL);
return Tcl_PkgProvide(interp, "mat", "1.0");
}
======
Let's test it :
======tcl# on matrix3 typed var
timerate {mat::det $M}; # 0.326806 µs/# 3059918 # 3059918 #/sec 1000.000 net-ms# on usual list
timerate {mat::det $L} ; # expecting floating-point number but got "a"
set L {{1 2 3} {4 5 6} {7 8 9}}
timerate {mat::det $L}; # 0.371363 µs/# 2692783 # 2692785 #/sec 999.999 net-ms# on matrix3 typed vars
timerate {mat::mul $M $N}; # 1.444749 µs/# 692161 # 692161 #/sec 999.999 net-ms# on usual lists
timerate {mat::mul $L $K}; # 1.445566 µs/# 691770 # 691770 #/sec 999.999 net-ms
======
Here we can see that at the C level, typing a value as no cost anymore, even despite the garentee it offers at the Tcl level.
But these are little computation. In big computations, for instance wWhen computing 1000 determinants, using a batch of typed var is faster than tcl, because the compact matrix3 memory usage is a lot smaller. For the same reasons, when using a batch of typed vars, C is a lot faster than when using raw lists (more than 10 times) : list contains big Tcl_Obj, whereas matrices contains only 9 doubles. ?
In big computations, for instance when computing 1000 determinants, using a batch of typed var is faster than tcl lists, because 1° the compact matrix3 memory usage is a lot smaller, and 2° as we already know the shape of data, that is arranged in cells, C has just to copy it once. A list contains Tcl_Obj, whose each size can amount to 48 bytes. Thus, a matrix written as tcl list costs 432 bytes of memory, whereas a matrice in cells costs only 72 bytes, since it contains only 9 doubles. That is 6 times less memory costs than the usual matrix representation by lists. Here is an extract of measurement you can find in tip-764 branch under the folder [https://core.tcl-lang.org/tcl/dir?ci=tip-764_tcl90&name=typeparse/exemples/matrices%|%typeparse/exemples/matrices%|%] :
&|which case | time (µs) | compar. |&
&| 10000 déterminants, raw Tcl | 35542.07 | x 1 |&
&| 10000 déterminants, batch of Tcl matrices in Cells | 28169.97 | x 1.2 |&&| 10000 déterminants, list of Tcl matrices in Cells | 45237.67 | x 0.78 |&
&| 10000 déterminants, C on a batch of matrices in Cells | 716.12 | x 49.63 |&
&| 10000 déterminants, C on a raw list | 9208.06 | x 3.85 |&&| 10000 déterminants, C on a batch of matrices in Cells | 716.12 | x 49.63 |&
Every Tcl list must be read, then each of their elements must be accessed one by one, whereas a compact batch of typed matrices is just read once. C on raw list is only ''3.85'' faster than usual Tcl, whereas on cell-arranged batch of matrices C is ''49.53'' time faster.
This illustrate all the interset to be able to define precisely the shape of data through introduction of types in Tcl. Using type can help to be more fast.
** Conclusion **
Maybe typed variable can be very usefull when working at the C-level ? The garentee of var correctness has a cost, and typed variables will pay it : It will never be more efficient in speed than raw Tcl, that is optimised for speed, even with unknown values. But, if we could take advantage of the precision Type give, C can run securely, and then, we can get the better of the two worlds : the flexibility of Tcl, the garantee of type brought by TIP 764, and the speed of C.
Moreover, TIP 764 introduce a typology that allows to work with real linked list, trees, graphs and hyergraph. Example are given to read header of elf, png or gif files, and to work directly with network protocols.
<<discussion>>
<<categories>> TIP|types|C programming