[Cryptech Tech] FIPS 140-2 test program

[Benedikt Stockebrand]

Hi Stephan and list,

>>While it talks about entropy for example, it doesn't clearly define the
>>statistical model that its entropy measurement relates to.  This can
>>be a documentation bug, but I wouldn't want to bet on it.
>
> There is no model behind it. All they do is to apply:

To quote from SP 800-90B (Draft) I've just started to read (page 11,
first paragraph):

    Entropy is defined relative to one's knowledge of (the probability
    distribution on) $X$ prior to an observation, and reflects the
    uncertainty associated with predicting its value---the larger the
    entropy, the greater the uncertainty in predicting the value of an
    observation.

If there were no model, then all outputs were valid.  What makes one
output suspect and the other not?

> - Shannon Entropy formula

I'm not sure I really understand what they mean with that term, so I'm
not going to comment on this one right now.

> - Chi-Square Test
> - mean calculation
> - Simulation

These tests all take an entirely arbitrary aspect and check for that
aspect only.  Fair enough, but why are these tests of any value?  Why
are they "better" than a test "Is a quote from the works of Shakespeare
encoded in EBCDIC"?  They are arbitrary, and as such only useful if the
underlying model matches the source of the data.

With the chi square test for example: What is so special about an 8 bit
word size, statistically speaking?  Why not use a 65517 bit word size
instead?

The mean is somewhat similar; it interprets the bit stream as a sequence
of binary encoded numbers and then checks these.  Why exactly that?  And
should it be big or little endian, which may make a huge difference on
the result of the test?

The simulation thing in particular is on the border of silly; all it
does is that it applies a yes/no test with known output distribution to
the input and checks that the ratio of yes and no results is getting
close enough to the theoretically computed value.  There's really little
reason to waste CPU cycles on expensive trigonometric functions (if done
badly) or at least multiplications (if done somewhat more sensibly).
The only reason for this test is that it somehow resembles the dart
board method used in old day statistics text books because the authors
couldn't really come up with anything better.

> Of course you only test the raw noise source and not anything whitened. 

>From a practical point of view this usually makes sense.  But:

Why is that "of course"?  If I had reason to assume that the test data I
got was really from the works of Shakespeare but had been sent through
3DES with a key of 1234567890, why shouldn't I test it for that?  Again,
entropy is always relative to what you know, or assume, about the data
stream you want to test.

I know this sounds somewhat weird, but the key point is that all testing
is making assumptions on the output, and the only thing we can do about
is to try and make them explicit so we can match them with the nature of
the actual source.

> Especially when you have a cryptographic whitening function, all you 
> would test is that the crypto function is good. And any current crypto 
> functions per definition must show good statistics as otherwise the 
> crypto function weak.

That's exactly the point.  However: As soon as I know about (a) the
design of the CSPRNG and (b) any weakness in the crypto function, then I
can do some "targeted" testing on the underlying HWRNG.

Or put in another way: If I take output x from a HWRNG, run that through
some whitening function f, and return f(x) as whitened noise, then I
could still run my usual tests on x as long as I knew the inverse
function f^-1 of f, so mathematically speaking such a test is well
possible; it may be cryptographically infeasible if f if chosen
properly, but that's a tremendous difference: Cryptographic feasibility
is not a constant, but something that changes over time, and people with
more intimate knowledge of f may actually come to significantly
different results than those without.


Cheers,

    Benedikt

-- 
Benedikt Stockebrand,                   Stepladder IT Training+Consulting
Dipl.-Inform.                           http://www.stepladder-it.com/

          Business Grade IPv6 --- Consulting, Training, Projects

BIVBlog---Benedikt's IT Video Blog: http://www.stepladder-it.com/bivblog/