[Cryptech Tech] FIPS 140-2 test program

[Stephan Mueller]

Am Dienstag, 29. Juli 2014, 19:47:47 schrieb Fredrik Thulin:

Hi Fredrik,

>On Monday, July 14, 2014 10:37:09 PM Stephan Mueller wrote:
>...
>
>> But in any case, the ent tool gives you a "smoke" test indicator
>> whether the statistics look good -- in particular the Chi-Square
>> test. The dieharder test suite is a more comprehensive testset.
>
>In my mind I've repeatedly returned to this statement many times now,
>so I'd better ask. Excuse my ignorance, but how do I use the
>Chi-square test of `ent' to smoke test some supposedly random data I
>have?

The test is NOT a smoke test for random data. It shows that the data 
most likely shows no pattern and thus looks like white noise.

Note, this test is a necessary, but not sufficient indication for 
randomness. Thus, when this smoke test fails, you can already forget any 
other analysis and have to solve a more fundamental problem. When the 
smoke test passes, you can continue with your analysis.

In any case, Chi-Square is very sensitive to even the slightest 
patterns, much better than monobit test, poker test and other easy 
tests.
>
>This is with data from my test circuit (avalanche noise) and to be
>precise it
>is entropy (of unknown amount) rather than random data, but still:
>> Entropy = 7.998093 bits per byte.
>> 
>> Optimum compression would reduce the size
>> of this 120387525 byte file by 0 percent.
>> 
>> Chi square distribution for 120387525 samples is 319771.56, and
>> randomly would exceed this value 0.01 percent of the times.
>> 
>> Arithmetic mean value of data bytes is 127.4996 (127.5 = random).
>> Monte Carlo value for Pi is 3.127210742 (error 0.46 percent).
>> Serial correlation coefficient is 0.000117 (totally uncorrelated =
>> 0.0).
>Here is output from four 10 MB files, of which I already determined two
>are
>really flawed because of a bug in my entropy extractor code:
>> xaa: Chi square distribution for 10485760 samples is 24521.23, and
>> randomly xaa: would exceed this value 0.01 percent of the times.
>> 
>> xab: Chi square distribution for 10485760 samples is 24089.28, and
>> randomly xab: would exceed this value 0.01 percent of the times.
>> 
>> xai: Chi square distribution for 10485760 samples is 251996.10, and
>> randomly xai: would exceed this value 0.01 percent of the times.
>> 
>> xaj: Chi square distribution for 10485760 samples is 254102.71, and
>> randomly xaj: would exceed this value 0.01 percent of the times.
>
>I guess you can tell which ones are flawed, but I can't (using the Chi
>square) ;).

Read the man page to get an easy answer: the percentage value must be 
between 10 and 90% to "pass" the Chi-Square.
>
>/Fredrik


Ciao
Stephan