Tcl Tutorial Lesson 6a

Computers and numbers

If you are new to programming, then the following discussion may contain some surprising information. But even if you are used to writing programs, computers can do unexpected things with numbers. The purpose of this lesson is to shed some light on some of the mysteries and quirks you can encounter. These mysteries exist independently of the programming language, though one programming language may be better at isolating you from them than another. The problem is that computers do not deal with numbers the way humans normally do.

 % expr {1/6}
 0
 % expr {1/6.0}
 0.16666666666666666

The difference is the decimal point in the second expression.

Tcl's strategy

Tcl uses a simple but efficient strategy to decide what kind of numbers to use for the computations:

• If you add, subtract, multiply and divide two integer numbers, then the result is an integer. If the result fits within the range you have the exact answer. If not, you end up with something that appears to be completely wrong. (Note: not too long ago, floating-point computations were much more time-consuming than integer computations. And most computers do not warn about integer results outside the range, because that is too time-consuming too: a computer typically uses lots of such operations, most of which do fit into the designated range.)
• If you add, subtract, multiply and divide an integer number and a floating-point number, then the integer number is first converted to a floating-point number with the same value and then the computation is done, resulting in a floating-point number.

Floating-point computations are quite complex, and the current(IEEE) standard prescribes what should happen in minute detail. One such detail is that results outside the proper ranges are reported. Tcl catches these and displays a warning:

 % # Compute 1.0e+300/1.0-300
 % puts [expr {1.0e300/1.0e-300}]
 floating-point value too large to represent

What are those mysteries and quirks?

Now some of the mysteries you can find yourself involved in. Run the following scripts:

#
# Division
#
puts "1/2 is [expr {1/2}]"
puts "-1/2 is [expr {-1/2}]"
puts "1/2 is [expr {1./2}]"
puts "1/3 is [expr {1./3}]"
puts "1/3 is [expr {double(1)/3}]"

The first two computations have the surprising result: 0 and -1. That is because the result is an integer number and the mathematically exact results 1/2 and -1/2 are not. If you interested in the details of how Tcl works, the outcome q is determined as follows:

a = q * b + r
0 <= |r| < |b|
r has the same sign as q

Here are some examples with floating-point numbers:

puts "1/2 is [expr {1./2}]"
puts "1/3 is [expr {1./3}]"

set a [expr {1.0/3.0}]
puts "3*(1/3) is [expr {3.0*$a}]"

set b [expr {10.0/3.0}]
puts "3*(10/3) is [expr {3.0*$b}]"

set c [expr {10.0/3.0}]
set d [expr {2.0/3.0}]
puts "(10.0/3.0) / (2.0/3.0) is [expr {$c/$d}]"

set e [expr {1.0/10.0}]
puts "1.2 / 0.1 is [expr {1.2/$e}]"

While many of the above computations give the result you would expect, note however the last decimals, the last two do not give exactly 5 and 12! This is because computers can only deal with numbers with a limited precision: floating-point numbers are not our mathematical real numbers.

Somewhat unexpectedly, 1/10 also gives problems. 1.2/0.1 results in 11.999999999999998, not 12. That is an example of a very nasty aspect of most computers and programming languages today: they do not work with ordinary decimal fractions, but with binary fractions. So, 0.5 can be represented exactly, but 0.1 can not.

Some practical consequences

• The fact that floating-point numbers are not ordinary decimal or real numbers and the actual way computers deal with floating-point numbers, has a number of consequences: Results obtained on one computer may not exactly match the results on another computer. Usually the differences are small, but if you have a lot of computations, they can add up!
• Whenever you convert from floating-point numbers to integer numbers, for instance when determining the labels for a graph (the range is 0 to 1.2 and you want a step size of 0.1), you need to be careful:

#
# The wrong way
#
set number [expr {int(1.2/0.1)}]  ;# Force an integer -
                                  ;# accidentally number = 11

for { set i 0 } { $i <= $number } { incr i } {
   set x [expr {$i*0.1}]
   ... create label $x
}


#
# A right way - note the limit
#
set x     0.0
set delta 0.1
while { $x < 1.2+0.5*$delta } {
   ... create label $x
   set x [expr {$x + $delta}]
}

• If you want to do financial computations, take care: there are specific standards for doing such computations that unfortunately depend on the country where they are used - the US standard is slightly different from the European standard.
• Transcendental functions, like sin() and exp() are not standardised at all. The outcome could differ in one or more decimals from one computer to the next. So, if you want to be absolutely certain that the mathematical number "pi" is a specific value, use that value and do not rely on formulae like these:

#
# Two different estimates of "pi"
#
set pi1 [expr {4.0*atan(1.0)}]
set pi2 [expr {6.0*asin(0.5)}]
puts [expr {$pi1-$pi2}]
-4.4408920985006262e-016