typed-json

typed_json - JSON Parser with Type Preservation

Overview

A pure Tcl JSON parser that preserves complete type information, similar to tDOM's -json option but without requiring binary extensions.

The code has been moved to https://github.com/rocketship88/typed-json-tcl There are two versions of the code, one that is likely faster and uses dict stuctures, but cannont incorporate duplicate keys. A second version uses only list operations and thus can and that makes it more compatible with tDOM.

Design Philosophy

This parser is intended for simple JSON processing where a pure Tcl solution is desirable, or simply for educational use on using flask. It provides convenient utility functions for common operations and does not claim to be anywhere near as efficient as tDOM. For applications requiring high-performance JSON processing, complex document manipulation, or handling of very large JSON datasets, users should consider a full tDOM binary solution.

• Objects with duplicate keys follow "last wins" behavior, consistent with most JSON parsers
• The final occurrence of any duplicate key will be preserved in the result
• Type information is preserved to enable accurate round-trip conversion
• Utility functions assume unique keys for simple dict-style access

Acknowledgements

This typed JSON implementation draws inspiration from the design principles established in tDOM's JSON parser by Rolf Ade and the tDOM development team. tDOM's approach to preserving JSON type information during parsing has been influential in the Tcl community for solving the fundamental challenge of round-trip JSON conversion in Tcl's string-oriented environment.

We thank Rolf Ade for his continued maintenance and development of tDOM, and for pioneering type-preserving JSON parsing patterns that benefit the entire Tcl ecosystem.

tDOM project: http://tdom.org
tDOM repository: http://core.tcl.tk/tdom

Main Function

typed_json::json2dict

typed_json::json2dict jsonString ?options?

Parses a JSON string and returns a tDOM-style typed structure where each value carries its type information.

Options:

Returns: Typed structure in format {TYPE data}

Examples:

# Basic usage
set data [typed_json::json2dict {{"name": "Alice", "age": 30}}]

# With options
set data [typed_json::json2dict $json -strict yes -maxnesting 100]

# Parse root-level array
set data [typed_json::json2dict {[1, 2, "three"]}]
# Returns: {ARRAY {{NUMBER 1} {NUMBER 2} {STRING three}}}

Unicode handling

This function is called internally by the JSON parser but can also be used directly as a utility. When called by json2dict, the surrogate mode is determined by the -surrogate option (which auto-detects based on Tcl version if not specified). When called directly, the default is "attempt".

convertEscapes

typed_json::convertEscapes string ?surrogateMode?

Convert JSON escape sequences to actual characters. Handles Unicode escape sequences including surrogate pairs for emoji and other non-BMP characters.

Arguments:

Surrogate Modes:

Examples:

# Basic escape sequences
set result [typed_json::convertEscapes "Hello\nWorld\t\"test\""]
# Returns: Hello[newline]World[tab]"test"

# Unicode bullet point
set result [typed_json::convertEscapes "Bullet\u2022point"]
# Returns: "Bullet•point"

# Emoji via surrogate pair (Tcl 9.0+)
set result [typed_json::convertEscapes "Smile\uD83D\uDE00face"]
# Returns: "Smile[grinning emoji]face"

# With explicit surrogate mode for Tcl 8.6
set result [typed_json::convertEscapes "\uD83D\uDE00" "replace"]
# Returns: "\U01F600"

# Error handling for orphaned surrogate
catch {
    typed_json::convertEscapes "\uD83Dorphaned" "error"
} err
puts $err
# Outputs: "Orphaned high surrogate \uD83D - expected low surrogate to follow"

Notes:

• Surrogate pairs are only needed for characters above U+FFFF (emoji, some Asian characters, etc.)
• In Tcl 8.6, surrogate pairs cannot be properly rendered, use -surrogate error or -surrogate replace
• In Tcl 9.0+, surrogate pairs work correctly with attempt mode
• The parser automatically detects Tcl version and sets appropriate default

Basic Utility Functions

getValue

typed_json::getValue typedData

Extract the value from a typed data element.

Example:

set item {STRING hello}
puts [typed_json::getValue $item]  ;# Outputs: hello

getType

typed_json::getType typedData

Get the type of a typed data element.

Example:

set item {NUMBER 42}
puts [typed_json::getType $item]  ;# Outputs: NUMBER

isType

typed_json::isType typedData expectedType

Check if a value matches a specific type.

Example:

if {[typed_json::isType $item "STRING"]} {
    puts "It's a string!"
}

Navigation & Search Functions

getPath

typed_json::getPath typedStructure path

Navigate to a nested value using dot notation.

Example:

set email [typed_json::getPath $data "user.contacts.email"]
puts [typed_json::getValue $email]

findKey

typed_json::findKey typedStructure keyName

Find all occurrences of a key name (searches recursively).

Returns: List of {found keyName typedValue} tuples

Example:

set results [typed_json::findKey $data "email"]
foreach result $results {
    lassign $result status key value
    puts "Found: [typed_json::getValue $value]"
}

findByType

typed_json::findByType typedStructure targetType

Find all values of a specific type (STRING, NUMBER, OBJECT, ARRAY, TRUE, FALSE, NULL).

Returns: List of typed values matching the target type

Example:

# Find all strings in the structure
set allStrings [typed_json::findByType $data "STRING"]
foreach str $allStrings {
    puts [typed_json::getValue $str]
}

getAllKeys

typed_json::getAllKeys typedStructure ?prefix?

Get all key paths in the structure (recursively).

Returns: List of key paths using dot notation

Example:

set keys [typed_json::getAllKeys $data]
# Returns: {name age contacts contacts.email contacts.phone ...}

asPlainTcl

typed_json::asPlainTcl typedStructure

Convert typed structure back to plain Tcl dict/list (loses type information).

Example:

set plainDict [typed_json::asPlainTcl $typedData]
# Can now use with regular dict/list commands

asXml

typed_json::asXml typedStructure ?indent? ?ascii?

Convert typed structure to XML format.

Options:

Example:

# UTF-8 XML with Unicode characters
set xml [typed_json::asXml $data]

# ASCII-safe XML for maximum portability
set xml [typed_json::asXml $data "" yes]
# Unicode becomes: • for •, 
 for newlines

asJson

typed_json::asJson typedStructure ?indent? ?ascii?

Convert typed structure back to JSON format.

Options:

Example:

# UTF-8 JSON with Unicode characters
set json [typed_json::asJson $data]

# ASCII-safe JSON for maximum portability  
set json [typed_json::asJson $data "" yes]
# Unicode becomes: \u2022 for •

JSON Utilities (Optional)

The jsonutilities.tcl file provides path-based manipulation functions that work with the dict-based parser only. These utilities use dot notation for path navigation (e.g., "server.host.primary"). See below to use a different character for the dot.

Key Functions

• setObjectByPath - Modify or delete existing object values using path notation
• insertIntoArrayAtPath - Insert values into arrays at specific indices
• setJsonObjectByPath - Set object values using raw JSON text
• insertJsonIntoArrayAtPath - Insert JSON text into arrays

setObjectByPath typedStructure path newTypedValue ;# if newTypedValue is {} then object at path is deleted
setJsonObjectByPath typedStructure path jsonText

insertIntoArrayAtPath typedStructure path index newTypedValue
insertJsonIntoArrayAtPath typedStructure path index jsonText

JSON arrays are stored as lists in a typedStructure. The two insert into Array utilities, take an index parameter, which is in the same format as the tcl linsert command. It can be an integer, end, or end+-integer, and these are passed into an linsert to perform the action. Thus 0 inserts at the beginning, and end appends at the end and end-1 inserts before the last item.

Path Delimiter Configuration

The default path delimiter is ".". To change it to a single character:

namespace eval typed_json {set delimiter "/"}
# Now use paths like "server/host/primary"

Usage Options

You can use the utilities in several ways:

• Source the file: source jsonutilities.tcl
• Copy functions into your code for customization
• Append to your local copy of jsonparser.tcl (if creating a module)

Example Usage

# Start with this JSON configuration
set originalJson {
{
  "server": {
    "host": "localhost",
    "port": 8080,
    "users": ["alice", "bob"]
  },
  "database": {
    "host": "db.example.com",
    "settings": {"timeout": 30}
  }
}
}

# Load both parser and utilities
set no_tests 1
source jsonparser.tcl
source jsonutilities.tcl

# Parse the original JSON
set config [typed_json::json2dict $originalJson]

# Example 1: Modify existing object value using path notation
set config [typed_json::setObjectByPath $config "server.host" {STRING "production.server.com"}]

# Example 2: Set object value using raw JSON text
set config [typed_json::setJsonObjectByPath $config "database.settings" {"timeout": 60, "retries": 3}]

# Example 3: Insert new user into the array
set config [typed_json::insertIntoArrayAtPath $config "server.users" 0 {STRING "admin"}]

# Show the final result
puts [typed_json::asJson $config "  "]

After applying all three changes, the JSON becomes:

{
  "server": {
    "host": "production.server.com",
    "port": 8080,
    "users": [
      "admin",
      "alice", 
      "bob"
    ]
  },
  "database": {
    "host": "db.example.com",
    "settings": {
      "timeout": 60,
      "retries": 3
    }
  }
}

Working with Objects

# Parse JSON object
set data [typed_json::json2dict {{"user": {"name": "Alice", "age": 30}}}]

# Extract the OBJECT wrapper
lassign $data rootType rootData
# rootType = "OBJECT"
# rootData = {user {OBJECT {name {STRING Alice} age {NUMBER 30}}}}

# Access nested object
set userObj [dict get $rootData "user"]
lassign $userObj userType userData
# userType = "OBJECT"

# Get specific value
set nameValue [dict get $userData "name"]
puts [typed_json::getValue $nameValue]  ;# Outputs: Alice

Working with Arrays

# Parse JSON array
set data [typed_json::json2dict {["apple", 42, true]}]

# Extract the ARRAY wrapper
lassign $data rootType arrayContents
# rootType = "ARRAY"
# arrayContents = {{STRING apple} {NUMBER 42} TRUE}

# Iterate through array
foreach item $arrayContents {
    puts "[typed_json::getType $item]: [typed_json::getValue $item]"
}

Type Reference

JSON Types → Parser Types:

Comments Support

The parser supports JSON5-style comments by default:

{
  // Single-line comment
  "key": "value",
  /* Multi-line
     comment */
  "array": [1, 2, 3]
}

Use -strict yes to disable comments and enforce strict JSON compliance.

Error Handling

The parser provides clear error messages:

TCLLIB JSON Parser

HaO 2026-02-02: The json parser in TCLLib is only usable for non-array JSONs, as the information of an array is not given back. This version does not have this draw-back. It would be great to replace the TCLLib version by this one in the long run. In addition, tdom compatibility is just GREAT!

See Also

flask a mini-flex/lex proc, tDOM, json

Keywords

json, parser, flask, type preservation, pure tcl

gold 02/3/2026. Added categories, so can find message in Wiki.