## Version 32 of Random Number

Updated 2007-07-10 10:51:19 by arjen

Many applications require random numbers.

This includes -

• Games of chance: You want simulated dice to behave as unpredictable as their real counterpart.
• Monte Carlo simulations: Algorithms are tested with random input.
• Cryptography: Random numbers make "one time pads" perfectly secure.

The deterministic nature of computers makes it impossible for computer software to come up with truly random numbers.

Hardware devices exist that produce random numbers from unpredictable, or unpredictably chaotic, physical effects such as athmospheric noise, radioactive decay, or even the movement of liquids in a common lava lamp. Some operating systems (e.g., Linux) derive reasonably random numbers from user behavior, such as the time between keyboard inputs (i.e., something like "nanoseconds between hitting two keys"), or mouse movement.

True Random Numbers downloads true random numbers from a free internet service.

Without access to such a device, the best a software can do is to use pseudo-random numbers (PRNs) which are produced by mathematical algorithms. The remainder of this page uses random number synonymous with pseudo-random number.

Because of their importance in various fields, there is a large body of literature on the subject. The canonical reference is Donald E. Knuth's The Art of Computer Programming, Volume 2: Seminumerical Algorithms.

Two of the most important properties that sequences of random numbers must, or should, satisfy are unpredictability and unbiased. The sequence of integer numbers {1, 2, 3, 4, ...} is perfectly biased but predictable. The sequence of daily outside temperature readings is unpredictable but biased, as the range of numbers is narrow, and the readings between two days are usually close. Because of such subtleties, the (probabilistic) proof that a sequence of numbers is reasonably random is very hard.

Tcl provides a pseudo-random number generator (called rand) as part of the expr command:

` expr rand()`

returns a random number between 0 and 1.

Tcl's random number generator highlights a weakness that applies to all pseudo-random number generators: it has to be initialized, or "seeded." The initial seed value fully determines the sequence of numbers that is generated. If the generator is seeded with the same value over and over, it will keep producing the same random numbers. Also, by design, it never produces the same number twice in a row -- unlike dice, which can come up with the same number many times in a row without being biased.

Tcl's random number generator is as bad as many, and worse than some, in that it only accepts a single 31 bit integer as a seed value. In other words, there are only 2^31 possible random number sequences, and they repeat after a period of 2^31 (in fact, numbers never repeat in less than this period). For applications that only care about the distribution of pseudo-random numbers across the value space, such as simulated dice, that can be considered good enough. In cryptography, such a random number generator is not considered cryptographically secure. An attacker could potentially make a brute-force attempt to recover a random number sequence by checking all 2^31 seed values.

Many applications, and Tcl's random number generator (when not seeded by the application), cheat by using something like the time of day (e.g., [clock seconds]) as a seed value. However, for cryptographical applications, that is about the worst thing that can be done. Because [clock seconds] returns a number in a very narrow range, so an attacker could make a good prediction of the values that need to be tested. Using a higher resolution timer, such as [clock clicks], improves the issue by a cryptographically insignificant margin only.

Therefore, we have a chicken and egg problem: in order to have unpredictable random numbers, a pseudo-random number generator needs to be initialized with a random seed.

We have two problems to solve, if we want to improve upon Tcl's pseudo-random number generator:

• First, we need a better algorithm; one that allows for seed values greater than 2^32.
• Second, we need a means of coming up with unpredictable seed values.

LES Maybe I am very naïve, but I really believe that a programmer can come up with a very good level of randomness. For example: if you prompt the user for six letters, assign an arbitrary (secret) numeric value to each letter, add the three first values and multiply the other three, then add and/or multiply everything by [clock seconds], then add or multiply it by [clock format [clock sec] -format %d], then apply some really arcane operation on the result so that only two or three digits are left, boy, isn't that random enough? Note the extreme idiosyncrasy of my formula. I can come up with ten other similarly bizarre formulas in five or ten minutes. I really don't believe that someone could crack something like that.

FPX Do not mistake obscurity for security. Your code above depends on the secrecy of the algorithm, and that is impossible to achieve. Any attacker can examine the code, so your assumption of "secret numeric values" does not hold. All the remaining transformations are reversible, and the current value of [clock seconds] is not hard to guess. So your numbers are definitely not secure. Whether they provide a reasonable source of randomness (remember, unpredictable and unbiased) should be subject of analysis. Do not assume randomness without in-depth analysis.

Two candidates for a better algorithm are

2004-Dec-10 drh: I recommend rc4. It is well-known and well studied. And it is easy to stir in new randomness obtain (for example) from %x and %y of mouse bindings or from the low-order bits of clock clicks taken after various events.

2004-Dec-13 rwm: A lot of systems just defer to the fips approved prng [L1 ] algorithms. The digital signature standard mentioned uses sha-1 as a mixing function.

KPV In trying to get a truly random number (for a seed, etc.) a good way to think is in terms of random bits. There are many traits of a computer or procedures the user can perform that can get you some degree of good randomness. If you combine enough of these, you have a good start. For example, [clock seconds] is guessable to within a few seconds thus supplying one or two bits of randomness. Other traits, such as process id, parent process id, disk free space, etc., and user tasks, such as typing or moving the mouse, all can supply few bits of good randomness. The trick is combining enough of these random bits can produce a good random number.

The scheme that FPX currently uses to compute a seed value goes as follows:

``` set seed [clock clicks]
# the user has to complete a dialog box here
append seed [clock clicks]
append seed [clock seconds] [pid]
append seed [winfo id .] [winfo geometry .] [winfo pointerxy .]
set hashedseed [binary format H* [sha1::sha1 \$seed]]```

The \$hashseed value is then used to initialize the PRNG. (Note that the implementations of the Mersenne Twister and ISAAC algorithms mentioned above accept an arbitrary-length binary string as seed value.)

I did not fully analyze the above code for randomness yet. Each contributing value should be examined for its randomness properties. I.e., the lower and upper bounds that an attacker (local vs. remote) can estimate on each value should be analyzed, or in other words, the number of "unpredictably random" bits that each value contributes to the seed. Ideally, the sum of random bits in the seed should be at least 128, a number that is clearly not achieved by the above code.

One idea that I was tossing around was to display a sketchpad canvas, and to ask the user to draw some random figures. The pixels' coordinates could then be used to produce a seed. However, I wanted the above to be non-intrusive.

Discussion? Better ideas?

Why the fuss about cryptographically secure random numbers?

Already mentioned above, random numbers allow for perfectly secure, unbreakable cryptography, using "one-time pads." In use for decades, first on paper, now on computers, a one-time pad contains a sequence of random numbers. Only two copies of a one-time pad exist, one is with the sender of a message, and one with the recipient.

In order to encrypt a message, the sender takes each letter of the message, and transposes it with one random number: The random number, between 0 and 25, is added to the ordinal of the letter, modulo 26, to produce one letter of the encrypted message.

The receiver uses the same pad, and the inverse transformation (by subtracting the random number, modulo 26), to decrypt the message.

This scheme is theoretically unbreakable, if the numbers are perfectly random, and if each one-time pad is used only once. If a pad is used multiple times, an attacker can leverage frequency analysis to determine the contents of the pad and gain access to the messages. If the numbers are not perfectly random, but produced by a pseudo-random number generator, decryption of the message is equivalent to breaking the generator's algorithm, parameters and seed. With the algorithm and parameters usually public, only the secrecy of the seed remains, reinforcing the need for unpredictable seed values.

The problem with one-time pads is the amount of logistics involved, as the pads must be produced, and then distributed to sender and recipient via secure channels. And whoever gets his hand on the pads can produce or read messages -- like the clerk who makes a copy before forwarding the pads.

2004/12/10 sheila My latest issue of Science News has an article on random number generation [L2 ] and a visualization of a good random number generator  Good PRNGs are used for other purposes than just cryptography, e.g. in high energy physics simulations.

AM (21 june 2007) I came across a relatively simple PRNG recommended by George Marsaglia, arguably an authority on the subject:

``` # marsaglia.tcl --
#     Implementation of a PRNG according to George Marsaglia
#
namespace eval ::PRNG {
variable mod [expr {wide(256)*wide(256)*wide(256)*wide(256)-5}]
variable fac [expr {wide(256)*wide(32)}]
variable x1 [expr {wide(\$mod*rand())}]
variable x2 [expr {wide(\$mod*rand())}]
variable x3 [expr {wide(\$mod*rand())}]

puts \$mod
}

proc ::PRNG::marsaglia {} {
variable mod
variable fac
variable x1
variable x2
variable x3

set xn [expr {(\$fac*(\$x3+\$x2+\$x1))%\$mod}]

set x1 \$x2
set x2 \$x3
set x3 \$xn

return [expr {\$xn/double(\$mod)}]
}

# main --
#     A small test
#
package require Tk
package require Plotchart

pack [canvas .c]
set p [::Plotchart::createXYPlot .c {0 1 0.2} {0 1 0.2}]
set x [::PRNG::marsaglia]
\$p dataconfig series -type symbol -symbol dot
for {set i 0} {\$i < 1000} {incr i} {
set y [::PRNG::marsaglia]
\$p plot series \$x \$y
set x \$y
}```

The implementation has a few disadvantages: using the builtin random number generator for seeding it means you only get 2**31 different series, but the run-length is about 2**92 for each series.

AM (22 june 2007) I am thinking about a small package/module to make Monte Carlo simulations easy to do.

Here is my first attempt at such simulations: A simple Monte Carlo simulation

And here is a first attempt at a package that could be useful:

``` # random.tcl --
#     Create procedures that return various types of pseudo-random
#     number generators (PRNGs)
#

# math::random --
#     Create the namespace
#
namespace eval ::math::random {
variable count 0
variable pi    [expr {4.0*atan(1.0)}]
}

# prng_Binomial --
#     Create a PRNG with a binomial distribution
#
# Arguments:
#     p         Probability that the outcome will be 1
#
# Result:
#     Name of a procedure that returns a binomially distributed
#     random number
#
proc ::math::random::prng_Binomial {p} {
variable count

incr count

set name ::math::random::PRNG_\$count
proc \$name {} [string map [list P \$p] {return [expr {rand()<P? 1 : 0}]}]

return \$name
}

# prng_Uniform --
#     Create a PRNG with a uniform distribution in a given range
#
# Arguments:
#     min       Minimum value
#     max       Maximum value
#
# Result:
#     Name of a procedure that returns a uniformly distributed
#     random number
#
proc ::math::random::prng_Uniform {min max} {
variable count

incr count

set name ::math::random::PRNG_\$count
set range [expr {\$max-\$min}]
proc \$name {} [string map [list MIN \$min RANGE \$range] {return [expr {MIN+RANGE*rand()}]}]

return \$name
}

# prng_Exponential --
#     Create a PRNG with an exponential distribution with given mean
#
# Arguments:
#     mean      Mean value
#
# Result:
#     Name of a procedure that returns a uniformly distributed
#     random number
#
proc ::math::random::prng_Uniform {min max} {
variable count

incr count

set name ::math::random::PRNG_\$count
proc \$name {} [string map [list MEAN \$mean] {return [expr {-MEAN*log(rand())}]}]

return \$name
}

# prng_Discrete --
#     Create a PRNG with a uniform but discrete distribution
#
# Arguments:
#     n         Outcome is an integer between 0 and n-1
#
# Result:
#     Name of a procedure that returns such a random number
#
proc ::math::random::prng_Discrete {n} {
variable count

incr count

set name ::math::random::PRNG_\$count
proc \$name {} [string map [list N \$n] {return [expr {int(N*rand())}]}]

return \$name
}

# prng_Normal --
#     Create a PRNG with a normal distribution
#
# Arguments:
#     mean      Mean of the distribution
#     stdev     Standard deviation of the distribution
#
# Result:
#     Name of a procedure that returns such a random number
#
# Note:
#     Use the Box-Mueller method to generate a normal random number
#
proc ::math::random::prng_Normal {mean stdev} {
variable count

incr count

set name ::math::random::PRNG_\$count
proc \$name {} [string map [list MEAN \$mean STDEV \$stdev] \
{
variable pi
set phi [expr {2.0*\$pi*rand()}]
return [expr {MEAN + STDEV*\$r}]
}]

return \$name
}

# prng_Disk --
#     Create a PRNG with a uniform distribution of points on a disk
#
# Arguments:
#
# Result:
#     Name of a procedure that returns the x- and y-coordinates of
#     such a random point
#
variable count

incr count

set name ::math::random::PRNG_\$count
{
variable pi
set phi [expr {2.0*\$pi*rand()}]
return [list \$x \$y]
}]

return \$name
}

# main --
#     Test code
#
set bin [::math::random::prng_Binomial 0.2]

set ones  0
set zeros 0
for { set i 0} {\$i < 100000} {incr i} {
if { [\$bin] } {
incr ones
} else {
incr zeros
}
}

puts "Binomial: \$ones - \$zeros"

set discrete [::math::random::prng_Discrete 10]

for { set i 0} {\$i < 100000} {incr i} {
set v [\$discrete]

if { [info exists count(\$v)] } {
incr count(\$v)
} else {
set count(\$v) 1
}
}

puts "Discrete:"
parray count

set rect [::math::random::prng_Rectangle 10 3]

puts "Rectangle:"
for { set i 0} {\$i < 10} {incr i} {
puts [\$rect]
}

set normal [::math::random::prng_Normal 0 1]

puts "Normal:"
for { set i 0} {\$i < 10} {incr i} {
puts [\$normal]
}```

The idea is that each random variable will be associated with its own command. That way there can be no mixup of distributions or parameters, as there is a command specific for that variable.

see where random numbers flop for a source example of the predictability of random numbers in tcl, even when properly srand()'d

Initially written by FPX.