&| What        | ''ton'' - Tcl Object Notation, a fast JSON parser, amongst other things |&
&| Where       | http://at.magma-soft.at/darcs/ton |&
&|             | Github mirror: https://github.com/jorge-leon/ton (probably lagging behind original) |&
&| Description | TON is an Object Notation with (almost) the same characteristics as JSON, but executable. ''ton'' implements code which produces and consumes TON  |&
&| Contact:    | [LEG] |&
&| Updated     | 02/2019 |&
&| Status      | experimental |&


**What**

There is a whole bunch of [JSON] parser implementations, half of them are C extensions and the other half written in pure Tcl.

''ton'' is just another pure Tcl JSON parser(/producer) that came to life unintentionally.  It stands out by two characteristics:
It is http://at.magma-soft.at/sw/blog/posts/Reverse_JSON_parsing_with_TCL%|%faster%|% then Tcllib's JSON parser (in Tcl-only mode),
and it parses JSON from right to left.


**Tcl Object Notation**

''ton'' parses JSON into an intermediate format called TON - Tcl Object Notation - that is crafted to make use of the [data is code], Tool Data Language ([TDL]),
https://www.gidhome.com/ManHtml/HowTo/resource/tcl_fileformats.html%|%Active File%|% idiom, pattern, or whatever you want to name it.

TON maps JSON directly to a Tcl script, which produces the desired outcome.

The following JSON string:

======
{
  "Me": {
    "Name": "Georg Lehner",
    "Height": 178,
    "Pets": ["Kaiser", "Biba", "Fortunatus"],
    "Male": true
    }
}
======

represented in TON is:


======
o Me [o Name [s {Georg Lehner}] \
        Height [i 178] \
                Pets [a [s Kaiser] [s Biba] [s Fortunatus]] \
                Male [l true]]
======

`o`, `s` `i` (and `d`), `a`, and `l` are proc's which treat their argument(s) as JSON object, string, number, array or literal.

instead of using a single number representation we decided to split "numbers" into `i`ntegers and `d`oubles.  It does not cost much
when parsing and can be used to advantage when processing.


**How to use**

The ''ton'' implementation defines several of these proc sets in different namespaces, currently: `ton::2list`, `ton::2dict`,
`ton::a2dict` and `ton::json`. The first three convert a TON string into  a nested list, a dictionary in the same format as returned e.g. by the
[jimhttp] JSON parser, and a dictionary with the same structure as returned by the [Tcllib] JSON parser.  The last one, `ton::2json` 
provides proc's for turning TON into (unformatted) JSON. A `ton::json2dict` parser has been added which should return mostly the same results ash the [Tcllib] JSON parser, taking into account backslash substitution. It is slower than `ton::2dict` and uses [subst] for the task, which does not work 100% identical with JSON but should do for most of the cases.

The only automated way to generate TON with ''ton'' is currently the proc `ton::json2ton` which converts a JSON string into TON.

To get the same result from the above example JSON string as Tcllib's json parser (`json::json2dict`) you do the following:

======
% namespace eval ton::2dict [ton::json2ton $json]]
Me {Name {Georg Lehner} Height 178 Pets {Kaiser Biba Fortunatus} Male true}
======


**Caveats**

''ton'' will parse the rightmost valid JSON construct in a string, and terminate. No check is done for extra characters.

Other then taking care of backslash escaped quotes `\"` on parsing, no processing for backslash escapes is done.

Numbers are only validated with Tcl's `string is` function.  Hex or octal numbers in the JSON string are therefore
admissible.

'''Security''': TON should be executed in a save slave interpreter to avoid arbitrary code execution with
malicious crafted JSON or TON strings.  I believe, that `ton::json2ton` does not generate dangerous TON, but this
has not been scrutinized.