[Cryptech Tech] FIPS 140-2 test program

[Fredrik Thulin]

On Monday, July 14, 2014 10:37:09 PM Stephan Mueller wrote:
...
> But in any case, the ent tool gives you a "smoke" test indicator whether
> the statistics look good -- in particular the Chi-Square test. The
> dieharder test suite is a more comprehensive testset.

In my mind I've repeatedly returned to this statement many times now, so I'd 
better ask. Excuse my ignorance, but how do I use the Chi-square test of `ent' 
to smoke test some supposedly random data I have?

This is with data from my test circuit (avalanche noise) and to be precise it 
is entropy (of unknown amount) rather than random data, but still:

> Entropy = 7.998093 bits per byte.
> 
> Optimum compression would reduce the size
> of this 120387525 byte file by 0 percent.
> 
> Chi square distribution for 120387525 samples is 319771.56, and randomly
> would exceed this value 0.01 percent of the times.
> 
> Arithmetic mean value of data bytes is 127.4996 (127.5 = random).
> Monte Carlo value for Pi is 3.127210742 (error 0.46 percent).
> Serial correlation coefficient is 0.000117 (totally uncorrelated = 0.0).

Here is output from four 10 MB files, of which I already determined two are 
really flawed because of a bug in my entropy extractor code:

> xaa: Chi square distribution for 10485760 samples is 24521.23, and randomly
> xaa: would exceed this value 0.01 percent of the times.
> 
> xab: Chi square distribution for 10485760 samples is 24089.28, and randomly
> xab: would exceed this value 0.01 percent of the times.
> 
> xai: Chi square distribution for 10485760 samples is 251996.10, and randomly
> xai: would exceed this value 0.01 percent of the times.
> 
> xaj: Chi square distribution for 10485760 samples is 254102.71, and randomly
> xaj: would exceed this value 0.01 percent of the times.

I guess you can tell which ones are flawed, but I can't (using the Chi square) 
;).

/Fredrik