Version 4 of Determining precision of numbers

Updated 2003-05-29 19:13:50

You can do this by getting the log base 10 of the number. [expr log10($number)]

You can get the regexp of the number, and then get the string length of the digits to the right and left of the decimal point.

In my case, I had a variable called precision, which determined how many places beyond the decimal point were displayed.

So, for precision = 2, .007 would be displayed as .01

To determine if this rounding was occuring:

            set non_rounded_number $my_value
            set rounding_p [expr (-1 * log10($non_rounded_number)) > $precision]
            if {$rounding_p} {
                # we have rounding
            }

[ Category Mathematics ]