ton

ton is a data format for representing a ordered tree of typed values, and a system for working with such data.

Attributes

Description

There is a whole bunch of JSON parser implementations, half of them are C extensions and the other half written in pure Tcl.

ton is just another pure Tcl JSON parser(/producer) that came to life unintentionally. It stands out by two characteristics: It is faster then Tcllib's JSON parser (in Tcl-only mode), and it parses JSON from right to left.

Tcl Object Notation

ton parses JSON into an intermediate format called TON - Tcl Object Notation - that is crafted to make use of the data is code, Tool Data Language (TDL), Active File idiom, pattern, or whatever you want to name it.

TON maps JSON directly to a Tcl script, which produces the desired outcome.

The following JSON string:

{
  "Me": {
    "Name": "Georg Lehner",
    "Height": 178,
    "Pets": ["Kaiser", "Biba", "Fortunatus"],
    "Male": true
    }
}

represented in TON is:

o Me [o Name [s {Georg Lehner}] \
        Height [i 178] \
                Pets [a [s Kaiser] [s Biba] [s Fortunatus]] \
                Male [l true]]

o, s i (and d), a, and l are proc's which treat their argument(s) as JSON object, string, number, array or literal.

instead of using a single number representation we decided to split "numbers" into integers and doubles. It does not cost much when parsing and can be used to advantage when processing.

Usage

The ton implementation defines several of these proc sets in different namespaces, currently: ton::2list, ton::2dict, ton::a2dict and ton::json. The first three convert a TON string into a nested list, a dictionary in the same format as returned e.g. by the jimhttp JSON parser, and a dictionary with the same structure as returned by the Tcllib JSON parser. The last one, ton::2json provides proc's for turning TON into (unformatted) JSON. A ton::json2dict parser has been added which should return mostly the same results ash the Tcllib JSON parser, taking into account backslash substitution. It is slower than ton::2dict and uses subst for the task, which does not work 100% identical with JSON but should do for most of the cases.

The only automated way to generate TON with ton is currently the proc ton::json2ton which converts a JSON string into TON.

To get the same result from the above example JSON string as Tcllib's json parser (json::json2dict) you do the following:

% namespace eval ton::2dict [ton::json2ton $json]]
Me {Name {Georg Lehner} Height 178 Pets {Kaiser Biba Fortunatus} Male true}

Caveats

ton will parse the rightmost valid JSON construct in a string, and terminate. No check is done for extra characters.

Other then taking care of backslash escaped quotes \" on parsing, no processing for backslash escapes is done - unless you use ton::json2dict.

Numbers are only validated with Tcl's string is function. Hex or octal numbers in the JSON string are therefore admissible.

Security: TON should be executed in a safe slave interpreter to avoid arbitrary code execution with malicious crafted JSON or TON strings. I believe, that ton::json2ton does not generate dangerous TON, but this has not been scrutinized.