Tcl_Obj is the C structure that implements every Tcl value. Tcl_Value would better describe its purpose, but that term was already taken by Tcl_CreateMathFunc() .
An extension that provides an alternative to Tcl_NewStringObj that returns a reference to an existing string object if it exists rather that creating a new duplicate string. The primary benefit of applying this mechanism to Tcl_Objs created from strings by an extension is performance - firstly because (in the case where an existing cached value is returned) no memory management is performed, the cached Tcl_Obj’s reference count is just incremented and the pointer returned. This can be a substantial win for an extension such as an XML parser that will spend much of its time creating Tcl_Objs for the same set of strings.
Tcl_Obj provides for both a string (modified utf-8) representation and an "internal" representation of the value, which is limited only by the constraints of the C language itself. Each Tcl_Obj carries information about the type of its internal representation, and how to perform a conversion from the string representation to the internal representation, and vice-versa. In this way, the string becomes the universal intermediate representation for conversions between types. To enhance performance, Tcl knows how to perform direct conversions between certain often-used types.
A Tcl_Obj is reference counted, and the allocator for it is very heavily tuned. the name "Tcl_Obj" is not related to object-oriented programming, but the more apt Tcl_Value was already used for handling user-defined expr functions - a now obsolete facility.
Despite the name "Tcl_Obj", this structure has nothing to do with object-oriented programming (that's what TclOO is for). Think of Tcl_Obj as a Tcl value with different clothes on. Depending on your needs, Tcl will provide you with the Tcl value dressed differently.
RS: thinks that the name is ok if one does not expect OO features, class membership etc. Objects have been there long before OO, and the name is certainly not under a monopoly (I'd object against that ;-). But the basic feature of Tcl_Obj's is that they have a string representation and possibly a problem-oriented one, but each can be regenerated from the other (also if you define your obj Obj types). If such type conversions occur frequently, this costs performance - the so-called shimmering occurs. E.g. see what happens to $i below:
for {set i 0} {$i < 10} {incr i} { #here we need the integer rep
puts [string length $i] ;#here the string rep..
puts [llength $i] ;# and here the list rep, so int rep goes away
}
CMcC: I've put together a summary page of Tcl_Objs current for 8.4, containing information culled from the source.
A Tcl_Obj is defined as a structure containing:
refCount (integer)
The number of references to this Tcl_Obj.
bytes (char *)
The Unicode string value, encoded in modified utf-8. Although each value is conceptually a string, actual generation of this value is delayed as long as possible to improve performance. Therefore, it often points to NULL. When it is not NULL, it points to memory allocated by Tcl_Alloc()y. For an empty string, objv[i]->bytes points to a static char in the Tcl library that holds a single NUL byte.
length (integer)
The length of the string representation in bytes (minus the extra byte for the terminating NUL).
typePtr (Tcl_ObjType *)
A pointer to the type of the object, a structure that provides the four fundamental operations which all Tcl_Obj instances implement.
internalRep (Tcl_ObjInternalRep)
A value for internal use the implementor of the object type. Whatever puts this to use must discipline itself to conform to what the interpretation string representation would be, even if the string representation hasn't been generated.
Each Tcl_ObjType structure contains the following four function pointers plus a name.
freeIntRepProc
Frees any the internal representation. NULL if nothing special is needed when the internal representation is cleaned up.
dupIntRepProc
Creates a new Tcl_Obj that is a copy of the current Tcl_Obj. If it is NULL, Tcl simply uses memcpy to copy whole internalRep structure.
updateStringProc
Updates the string representation from the internal representation. (Not sure what NULL means for this; IME that's not an especially good idea. DKF: It's OK provided you never ever set the bytes field to NULL.)
setFromAnyProc
Frees any existing internal representation, replacing it with a new internal representation for this type. Returns TCL_ERROR on failure. NULL indicates that objects of this type can't normally be created (typically because extra context is needed.)
Allocating a Tcl_Obj
DKF: You must not allocate a Tcl_Obj manually. Always call Tcl_NewObj (or one of its close relatives, such as Tcl_NewIntObj()) to do it for you. This is because Tcl uses a special memory management engine for them that is tuned to be extra efficient -- useful because Tcl uses these things a lot -- and that's only accessible through Tcl_NewObj (or some wholly internal APIs that aren't exposed outside the Tcl library).
Reference Counting
AMG: In C extension code for Tcl you have several options with respect to reference counting:
Manually invoke Tcl_IncrRefCount() on the Tcl_Objs you create. This protects them from being freed, but you're also responsible for calling Tcl_DecrRefCount() or else they'll leak.
Give your Tcl_Objs to something that increments their reference counts. For example, put them in a Tcl variable, list, or dict. Don't call Tcl_IncrRefCount() unless your code is also retaining pointers that you expect to be valid sometime in the future.
Don't fuss with reference counting because you're not the one creating the Tcl_Objs. This is the case when all you do is read arguments passed to your function which is implementing an extension command.
Don't call Tcl_IncrRefCount() because you like to live dangerously and "know" that you're only passing your Tcl_Objs to things that won't pull the rug out from under you. Call Tcl_DecrRefCount() when you're done, and the Tcl_Objs will be freed when their refcounts go negative just as surely as when they go zero.
There's no way Tcl can remotely zap your Tcl_Objs with nonpositive refcount unless you've passed your Tcl_Obj pointers to Tcl library functions. Tcl doesn't keep a list of Tcl_Objs in existence, so it can't sweep.
Discarding the Internal Cached Interpretation of a Value
Up until some point string length caused the internal cached interpretation of a value to be discarded, but this is no longer the case with more recent versions of Tcl:
% incr i
1
% ::tcl::unsupported::representation $i
value is a int with a refcount of 2, object pointer at 0x6000660e0, internal representation 0x1:0x600066320, string representation "1"
% string length $i
1
% ::tcl::unsupported::representation $i
value is a string with a refcount of 2, object pointer at 0x6000660e0, internal representation 0x6000ad6a0:0x600066320, string representation "1"
There's no guaranteed way to strip the internal representation from a Tcl_Obj, but a new Tcl_Obj can of course be created. To avoid any potential optimization to string range that might make this ineffective, use a slightly more complicated technique:
set var [string index $var 0][string range $var[set var {}] 1 end]
Threads
Data members of Tcl_Obj, particularly internalRep, can be mutated, so a Tcl_Obj should be exclusively owned by one thread. See Thread safety in tclZipfs.c .
Nested Tcl_Obj Structures
PYK 2018-05-12: Sometimes a Tcl_Obj is stored in the internal representation of another Tcl_Obj. This can lead to tricky issue such as this memory leak in foreach. The fix involved clearing the internal representation of the nested Tcl_Obj, but couldn't something else set the internal representation back to a problematic value again?