GSoC Idea: Common (De)Serialization Infrastructure


Areas	(De)serialization, RPC
Good if student knows	Tcl, C
Priority	Medium to Low
Difficulty	Medium to High
Benefits to the student	Learning about data serialization, the challenges therein.
Benefits to Tcl	Enhanced ability to exchange data with other languages
Mentor	AK, others
Related Projects & Ideas	GSoC Idea: Tcl Binding to Protocol Buffers GSoC Idea: Tcl Binding to MessagePack GSoC Idea: Updated Tcl bindings for ZeroMQ Tcl YAML STOMP

Project Description

We currently have three possibilities of handling the (de)serialization of Tcl data structures to other formats. The main constraint in the design is that most (all?) (de)serialization formats around are (strongly) typed, whereas Tcl's basic data structures are not.

The main ways of dealing with this impedance mismatch are:

Inbound map the serialized data to simple Tcl data structures, and ignore the type information. Outbound then requires constructor commands and/or additional data structures describing the data to generate and providing the necessary type information, implicitly or explicitly.
Inbound map the serialized data to simple Tcl data structures, but keep the type information somehow, for example as type tags. Outbound a generic converter can then directly serialize such a type-tagged structure without the need for helper structures.
Fill out: TclOO structures mimicking simple Tcl structures and other types, with accessor methods.

Each of these methods has their own advantages and disadvantages.

Examples of both (1) and (2) can be found in Tcl YAML , with (1) using separate type description data structures.

The pair of json and json::write packages of Tcllib are another example of (1), using constructor commands on the outbound side.

Regarding approach (3) I am not aware of existing examples.

This is what I am interested in here, with this idea. If that is a feasible approach, and if yes, what are its (dis)advantages ?

References

to be filled in.

Comments & Discussion

SEH -- FWIW, I use Metakit when I need to serialize data from a Tcl program into a type-aware external format. I have written a JSON exporter that automatically converts the contents of a Metakit view to a JSON string. The view is initialized using Metakit's simple schema format for specifying types. The JSON exporter uses Metakit's introspection features to determine the type of each field and construct the corresponding JSON string appropriately. Metakit has the additional advantage of supporting sub-views, so a hierarchically-structured JSON string can be constructed from a view and its subviews.

If a pure-Tcl solution is a must-have, maybe a Metakit-type approach could be used as design inspiration.

GSOC	Category Data Serialization Format