JSON is a data format for representing an ordered tree of typed values
The JSON site probably explains it best:
gchung 2012-11-06: Tcl's string is double does not conform to JSON specification. For testing valid numbers, you should use the following regexp:
proc is_valid_json_number {value} {
regexp -- {-?(?:[1-9][[:digit:]]*|0)(?:\.[[:digit:]]+)?(?:[eE][+-]?[[:digit:]]+)?} $value
}Using string is double allows non-valid numbers to slip through. E.g. "00", "0x0", "0.".
AMG: That regular expression may be fine for limiting the acceptable number forms, but also continue to use string is double to check for over/underflow.
OpenACS has its own take on handling JSON data, see [L2 ]. From the intro:
Utility ad_procs for Tcl <-> JSON conversion. This is based on the tcllib json package written by Andreas Kupries, and later rewritten to parse via regular expressions by Thomas Maeder. The tcllib version suffers from generating Tcl structures from JSON strings with no type (JSON array or object) information. The resulting structures can't be converted back to JSON strings, you have to munge them with type information first. And the code making use the Tcl structure also needs to know whether each field is an object or array. It also depends on the DICT package or Tcl 8.5. This rewrite doesn't depend on DICT, declares procs using ad_proc (so the API will be picked up by our API browser), and is symmetrical (you can convert from JSON to the Tcl representation and back again).
JMN 2009-01: The way I see it, a JSON generator is a little more useful than a JSON parser. The primary usecase I've had for JSON is pumping a chunk of structured data to the client where there is a JavaScript engine available to consume it.
Whilst in some scenarios JSON-formatted structured data might also be sent to the server where another language such as Tcl or PHP might need to parse it - it seems more common that the server-side language gets its data as simple values already split into the fields of a query string or form POST data.
TCV 2009-03-08: I've been solving this problem lately myself, the primary difficulty being how to encode the multiple types which are available in JSON in an ambiguously typed Tcl value. The solution for me is to make all values lists where the first element is the type name and the rest are the data for that type. Objects are tagged "obj" and then followed by alternating keys and values (the keys are strings so there's no reason to tag them, but values must be). Lists are tagged "list" and every subsequent element is an item in the list. Strings are "string", numbers are "num", and booleans are "bool". The special "null" when appearing alone is null. I have some preliminary code, not quite ready for release, on my website [L3 ].
slebetman 2009-04-01: The huddle package in tcllib does the tagged data thing quite well. And huddle has a built-in jsondump subcommand. I've been working on a different approach though. Having tagged data means I either have to jump through hoops to access the data in Tcl or rely on third party APIs (can't use regular list and dict commands for example). What I've done instead is to implement a JSON compiler. Given a specification of the data structure, and the tcl data, generate a JSON string:
# data is plain old tcl values
# spec is defined as follows:
# {string} - data is simply a string, "quote" it if it's not a number
# {list} - data is a tcl list of strings, convert to JSON arrays
# {list list} - data is a tcl list of lists
# {list dict} - data is a tcl list of dicts
# {dict} - data is a tcl dict of strings
# {dict xx list} - data is a tcl dict where the value of key xx is a tcl list
# {dict * list} - data is a tcl dict of lists
# etc..
proc compile_json {spec data} {
while [llength $spec] {
set type [lindex $spec 0]
set spec [lrange $spec 1 end]
switch -- $type {
dict {
lappend spec * string
set json {}
foreach {key val} $data {
foreach {keymatch valtype} $spec {
if {[string match $keymatch $key]} {
lappend json [subst {"$key":[
compile_json $valtype $val]}]
break
}
}
}
return "{[join $json ,]}"
}
list {
if {![llength $spec]} {
set spec string
} else {
set spec [lindex $spec 0]
}
set json {}
foreach {val} $data {
lappend json [compile_json $spec $val]
}
return "\[[join $json ,]\]"
}
string {
if {[string is double -strict $data]} {
return $data
} else {
return "\"$data\""
}
}
default {error "Invalid type"}
}
}
}
# Usage:
% compile_json {dict * list} {a {1 2 3} b {4 5}}
{"a":[1,2,3],"b":[4,5]}
# Data may be nested:
% compile_json {dict * {list {dict d list}}} {a {{c 1} {d {2 2 2} e 3}} b {{f 4 g 5}}}
{"a":[{"c":1},{"d":[2,2,2],"e":3}],"b":[{"f":4,"g":5}]}For the last example, the specification reads as:
{
dict <---------------- a dict
* {list <------- where all (*) values are lists
{dict <----- of dicts
d list <----- where the value of "d" is a list
} and all other values are strings (default)
}
}Steve Bennett 2010-10-13: I independently came up with something almost identical. My version correctly escapes string values using:
return \"[string map [list \\ \\\\ \" \\" \n \\n / \\/ \b \\b \r \\r \t \\t] $value]\"
Also, rather than automatically deciding whether to encode a string or a number, I use 'num' as a type. This also allows the special values false, true and null to be encoded as bare values rather than being quoted.
kanryu 2009-04-15: It's good way! I implement one for huddle as "huddle compile" :)
% huddle jsondump [huddle compile {dict * {list {dict d list}}} {a {{c 1} {d {2 2 2} e 3}} b {{f 4 g 5}}}] {} {}
{"a":[{"c":1},{"d":[2,2,2],"e":3}],"b":[{"f":4,"g":5}]}slebetman 2009-04-12: That's great kanryu. However I'd prefer this to be implemented as part of the json package rather than huddle. It would avoid having to do the conversion twice (dict->huddle->json). Could the maintainer of the json package in tcllib please steal my code and make a nice json::compile command? Pretty please..? ;-)
Some tests of json package using examples given on the JSON page.
if {[info script] eq $argv0} {
set examples {
{
"glossary": {
"title": "example glossary",
"GlossDiv": {
"title": "S",
"GlossList": [{
"ID": "SGML",
"SortAs": "SGML",
"GlossTerm": "Standard Generalized Markup Language",
"Acronym": "SGML",
"Abbrev": "ISO 8879:1986",
"GlossDef":
"A meta-markup language, used to create markup languages such as DocBook.",
"GlossSeeAlso": ["GML", "XML", "markup"]}]}}
}
{"menu": {
"id": "file",
"value": "File:",
"popup": {
"menuitem": [
{"value": "New", "onclick": "CreateNewDoc()"},
{"value": "Open", "onclick": "OpenDoc()"},
{"value": "Close", "onclick": "CloseDoc()"}
]
}
}
}
{"widget": {
"debug": "on",
"window": {
"title": "Sample Konfabulator Widget",
"name": "main_window",
"width": 500,
"height": 500},
"image": {
"src": "Images/Sun.png",
"name": "sun1",
"hOffset": 250,
"vOffset": 250,
"alignment": "center"},
"text": {
"data": "Click Here",
"size": 36,
"style": "bold",
"name": "text1",
"hOffset": 250,
"vOffset": 100,
"alignment": "center",
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
}
}
}
{"menu": {
"header": "SVG Viewer",
"items": [
{"id": "Open"},
{"id": "OpenNew", "label": "Open New"},
null,
{"id": "ZoomIn", "label": "Zoom In"},
{"id": "ZoomOut", "label": "Zoom Out"},
{"id": "OriginalView", "label": "Original View"},
null,
{"id": "Quality"},
{"id": "Pause"},
{"id": "Mute"},
null,
{"id": "Find", "label": "Find..."},
{"id": "FindAgain", "label": "Find Again"},
{"id": "Copy"},
{"id": "CopyAgain", "label": "Copy Again"},
{"id": "CopySVG", "label": "Copy SVG"},
{"id": "ViewSVG", "label": "View SVG"},
{"id": "ViewSource", "label": "View Source"},
{"id": "SaveAs", "label": "Save As"},
null,
{"id": "Help"},
{"id": "About", "label": "About Adobe CVG Viewer..."}]}
}
}
foreach example $examples {
puts "convert: $example"
puts [json::json2dict "\{$example\}"]
}
}RLH: I do a package require JSON on OSX and the library isn't there or my setup is broken?
JH: The package name is "json".
RLH: Is there any kind of "rule" about package names? I should have tried the lowercase json but didn't think of it. Just wondering.
RLH: I found it after doing a find. I am stupid sometimes.
HolgerJ 2008-08-14: when I do a package require json, I get a message "can't find package dict". I checked the Tcl version (Kubuntu Hardy 8.04) and found out it was 8.4.16; after installing 8.5.0 and calling the interpreter with tclsh8.5, it seems to work.
glennj: you can get the dict package for tcl 8.4: see the dict page
"tom.rmadilo" writes on comp.lang.tcl:
An example of http and JSON doing PHP page queries IP-geolocation
Antender 2010-12-25 13:51:59:
I'll post here my 1 line JSON parser which has such advantages:
Reformatted version by Antender and AMG. It's indented for readability, it combines many of the regsubs into a string map, and uses a single string range instead of two executions of string replace.
proc json2dict JSONtext {
string range [
string trim [
string trimleft [
string map {\t {} \n {} \r {} , { } : { } \[ \{ \] \}} $JSONtext
] {\uFEFF}
]
] 1 end-1
}Antender: Added dict2json. Have no support of JSON arrays due to Tcl's dynamical typing.
proc dict2json {dictionary} {
dict for {key value} $dictionary {
if {[string match {\[*\]} $value]} {
lappend Result "\"$key\":$value"
} elseif {![catch {dict size $value}]} {
lappend Result "\"$key\":\"[dict2json $value]\""
} else {
lappend Result "\"$key\":\"$value\""
}
}
return "\{[join $Result ",\n"]\}"
}dbohdan 2014-06-01: Unfortunately, because of how it uses string map the above implementation of json2dict will corrupt URLs stored in JSON values. A minimal example that demonstrates the problem:
eltclsh > set j {{"results": [{"artworkUrl512":"http://a1379.phobos.apple.com/long/path/mzl.lqtjexwh.png"}]}}
{"results": [{"artworkUrl512":"http://a1379.phobos.apple.com/long/path/mzl.lqtjexwh.png"}]}
eltclsh > puts [::json::json2dict $j]
results {{artworkUrl512 http://a1379.phobos.apple.com/long/path/mzl.lqtjexwh.png}}
eltclsh > puts [json2dict $j]
"results" {{"artworkUrl512" "http //a1379.phobos.apple.com/long/path/mzl.lqtjexwh.png"}}dbohdan 2014-06-01: I found the following function useful for accessing JSON data converted with ::json::json2dict.
proc jsonget {json args} {
foreach key $args {
if {[dict exists $json $key]} {
set json [dict get $json $key]
} elseif {[string is integer $key]} {
if {$key >= 0 && $key < [llength $json]} {
set json [lindex $json $key]
} else {
error "can't get item number $key from {$json}"
}
} else {
error "can't get \"$key\": no such key in {$json}"
}
}
return $json
}The motivation behind it is that right now ::json::json2dict converts JSON arrays into lists. As a result you need to chain calls to lindex and dict get to access values stored inside complexly structured JSON blobs. There is an issue on the Tcllib tracker to add the option -indexlists to ::json::json2dict that would make it index arrays into dicts of the form {0 data1 1 data2...} but it is not yet merged: https://core.tcl.tk/tcllib/tktview?name=2967134fff . With that in mind the above function allows you to say
jsonget [json::json2dict $j] results 0 artworkUrl512
instead of
dict get [lindex [dict get [json2dict $j] results] 0] artworkUrl512
dbohdan 2014-06-02: Fixed unnecessary exprs.
Here is a hack that I use to convert AWS IAM security policies from a more readable (to me) Tcl format to JSON. It relies on an upper-case / lower-case heuristic to differentiate between lists and dicts. This works most of the time in my use case. But please don't use this in your own code unless fully understand how wrong it is.
proc json_string {s} {
return \"[string map [list \\ \\\\ \" \\\"] $s]\"
}
proc _json {v} {
# Recursive JSON formatter...
if {[llength $v] == 1} {
return [json_string [lindex $v 0]]
} elseif {[regexp {^[A-Z]} [lindex $v 0]]} {
if {[lindex $v 0] eq "JSONDict:"} {
set v [lrange $v 1 end]
}
for {n v} in $v {lappend items "\"$n\": [_json $v]"}
return \{\n[join $items ,\n]\n\}
} else {
for v in $v {lappend items [_json $v]}
return "\[\n[join $items ,\n]\n\]"
}
}
proc json {dict} {
Format "dict" as a JSON string.
Note: Value lists begining with an upper-case letter are treated as
nested dictionaries. Other value lists are treated as plain lists.
To embed a nested dictionary with lower-case keys prepend "JSONDict:"
to the start of the nested dictionary.
e.g.
json {A 1 B 2 L {i j k}}
{"A": "1", "B": "2", "L": [ "i", "j", "k" ]}
} do {
# Generate JSON format...
set lines [_json $dict]
# Pretty indenting...
set indent ""
for l in [lines $lines] {
if {[regexp {[\}\]]} $l]} {
set indent [range $indent 0 end-4]
}
append result $indent$l\n
if {[regexp {[\{\[]} $l]} {
append indent " "
}
}
return $result
}example:
set policy [json [subst {
{
Effect Allow
Action s3:GetObject
Resource {
arn:aws:s3:::$instance.au.deploy/latest/lib-Linux-i686.tar.bz2
arn:aws:s3:::$instance.au.deploy/latest/$server_name.tar.bz2
}
} {
Effect Allow
Action s3:ListBucket
Resource arn:aws:s3:::*.com.au.deploy
} {
Effect Allow
Action sts:AssumeRole
Resource arn:aws:iam::*:role/$region-web-user-role
} {
Effect Allow
Action iam:GetRole
Resource arn:aws:iam::*:role/$region-web-user-role
}
}]]