Purpose of this page: To collate our knowledge about the facilities provided by Tcl to work with binary data, for example to talk to other applications using a binary protocol for exchanging information and commands.

-----

The main facility is the [binary] command with its subcommands to dissect (scan) and join (format) binary data into/from standard tcl values (strings, integers, lists, et cetera).

To exchange the binary information with other applications all of the facilities of the I/O system are at our fingertips and ready to be used. But note:

   * When writing data with [puts] do not forget to use the option -nonewline or else ''puts'' will write an additional end-of-line character after the data you actually wanted to send out.
   * Speaking of end-of-line characters I should note that another common error when creating a channel destined for exchange of binary information is forgetting to use '''[[[fconfigure] channel -translation binary]]'''. This command reconfigures the channel to leave the characters \n and \r untouched. Without this Tcl will treat them as end-of-line characters and mangle them during input and output.
   * In most cases, binary data also needs to be input and output to and from channels in a raw form.  Translating from (presumed) UTF-8 to your system's character set can be a disaster.  You therefore almost certainly need '''[[[fconfigure] $channel -encoding binary]]'''.  '''[DKF] sez:''' Note that setting the ''-translation'' to binary also sets the ''-encoding'' to binary, so you can usually ignore this one.
   * When reading binary information from a channel only [read] should be used. Avoid [gets]! The latter command will try to recognize end-of-line characters no matter what the channel is configured too. You can never be sure that such a character will not crop up in the middle of your packet.
   * When spawning an application which returns binary data via stdout do '''not''' use [exec], but the [[open "|..."]] idiom as only the latter allows you to change the pipe channel to binary. [exec] hides the pipe channel and may use the wrong encoding and translation settings when reading the information from the external application.


----

On news:comp.lang.tcl, [Mac Cody] and [Jeff David] write:

Mac Cody wrote:
 > Here is a simple example that
 > first writes binary data to a file and then reads back the
 > binary data:
 > 
 > set outBinData [binary format s2Sa6B8 {100 -2} 100 foobar 01000001]
 > puts "Format done: $outBinData"
 > set fp [open binfile w]

Important safety tip.  When dealing with binary files you
should always do:

 fconfigure $fp -translation binary

I got bit hard on this one once when my \x0a
and \x0d bytes got translated.

 > puts -nonewline $fp $outBinData
 > close $fp
 > set fp [open binfile r]

 fconfigure $fp -translation binary

 > set inBinData [read $fp]
 > close $fp
 > binary scan $inBinData s2Sa6B8 val1 val2 val3 val4
 > puts "Scan done: $val1 $val2 $val3 $val4"
 > 
 Jeff David

----
A post to comp.lang.tcl asks how best to embed binary data into a Tcl script.
[kennykb] has this summary of the answer:

   * for the occasional non-printing character embedded in a string, use \xNN.
   * for binary data embedded in a script and simply processed as a unit, use [base64] (examples include image files, files for foreign applications that you just want to write from the script, and blocks of cyphertext).
   * for byte sequences where it's important to preserve transparency, use hexadecimal and [binary] format.

In particular, you should avoid typing binary data directly into strings.
While Tcl is able to handle binary data, there are places where you can run
into problems.  In particular, if you happen to have a Tcl script containing
the literal character for a control-Z, you will find, as of Tcl 8.4, that
you get a syntax error from Tcl.  This is because beginning with 8.4 Tcl
treats a literal control-Z as the 'end of file' indicator across all platforms.
See [source] (in particular the reference page) for more details.


----

See [Binary representation of numbers] and [Dump a file in hex and ASCII] for examples of usage.