[Cryptech Tech] FIPS 140-2 test program

[Stephan Mueller]

Am Dienstag, 15. Juli 2014, 10:01:18 schrieb Benedikt Stockebrand:

Hi Benedikt,

>Hi Joachim and list,
>
>Joachim Strömbergson <joachim at secworks.se> writes:
>> [rngtest/rng-tools]
>> Great info, thanks!
>
>sorry I didn't mention it any sooner, but I assumed this was generally
>known...
>
>> Yes, Dieharder is what I use too. And ent.
>
>...while I didn't know about ent myself.  I've just taken a look at the
>man page, but again I've got a bad gut feeling about it:
>
>While it talks about entropy for example, it doesn't clearly define the
>statistical model that its entropy measurement relates to.  This can
>be a documentation bug, but I wouldn't want to bet on it.

There is no model behind it. All they do is to apply:

- Shannon Entropy formula

- Chi-Square Test

- mean calculation

- Simulation

with the bitwise or bytewise representation of the binary data stream. 
The goal is to show whether the data stream comes close to white noise.
>
>> But we really need good test suites and test programs implementing
>> them (see another mail on my sorry efforts at collecting
>> requirements,
>> tests and code) - for the entropy sources as well as the stages
>> through the RNG chain and for the CSPRNG output.
>
>Watch out, testing the output of a CSPRNG is completely different than
>testing a HWRNG.  Probably one of the big mistakes in commercial HSMs
>is that they use, or may use, the CSPRNG proper to hide the inadequacy
>of the HWRNG used---wrapped up in some glossy "tamper proofing" so
>nobody finds out.

Of course you only test the raw noise source and not anything whitened. 
Especially when you have a cryptographic whitening function, all you 
would test is that the crypto function is good. And any current crypto 
functions per definition must show good statistics as otherwise the 
crypto function weak.
>
>That's why I consider it so important to provide a test interface
>between the HWRNG and the CSPRNG proper: If you can feed defined seed
>data to the CSPRNG, then the output becomes deterministic and you can
>test by comparing it against a precomputed reference.  Of course, that
>sort of interface is rather useful for an attacker, too, but without
>the interface auditing is impossible.
>
>> FIPS 140-2 is important because it is what is used for Cryptech-like
>> things and will be asked about. But we should not constrain us to
>> FIPS
>> 140-2 (and AIS31 etc for that matter). The more complete we can be
>> and
>> show good results to anybodys favourite test/test suite the better.
>> We
>> therefore need to know about them and use them all. imho.
>
>At some point I've tried to come up with some proper tests myself;
>turned out my statistics knowledge in that area isn't remotely up for
>the job, so I got myself some texbook on it (makes me feel like an
>undergrad again---25 years younger:-).
>
>But yes, from all I've seen we need some significantly broader test
>sets.  I haven't yet taken a look at AIS31, but I'm more than
>suspicious of the FIPS140-2 stuff, and for a whole number of reasons;
>it's a good starter, but nothing I eventually want to rely on.
>
>Once I've got the hardware side finally sorted out, I hope to give
>testing a serious go---not just some quick Ruby hacks.  Not that I'm
>too happy about testing my own stuff, but apparently for the last
>fourty years this is the part of the job where everybody made sure not
>to look too closely...
>
>
>Cheers,
>
>    Benedikt


Ciao
Stephan