[Cryptech Tech] FIPS 140-2 test program

[Benedikt Stockebrand]

Hi Joachim and list,

Joachim Strömbergson <joachim at secworks.se> writes:

> [rngtest/rng-tools]
> Great info, thanks!

sorry I didn't mention it any sooner, but I assumed this was generally
known...

> Yes, Dieharder is what I use too. And ent.

...while I didn't know about ent myself.  I've just taken a look at the
man page, but again I've got a bad gut feeling about it:

While it talks about entropy for example, it doesn't clearly define the
statistical model that its entropy measurement relates to.  This can be
a documentation bug, but I wouldn't want to bet on it.

> But we really need good test suites and test programs implementing
> them (see another mail on my sorry efforts at collecting requirements,
> tests and code) - for the entropy sources as well as the stages
> through the RNG chain and for the CSPRNG output.

Watch out, testing the output of a CSPRNG is completely different than
testing a HWRNG.  Probably one of the big mistakes in commercial HSMs is
that they use, or may use, the CSPRNG proper to hide the inadequacy of
the HWRNG used---wrapped up in some glossy "tamper proofing" so nobody
finds out.

That's why I consider it so important to provide a test interface
between the HWRNG and the CSPRNG proper: If you can feed defined seed
data to the CSPRNG, then the output becomes deterministic and you can
test by comparing it against a precomputed reference.  Of course, that
sort of interface is rather useful for an attacker, too, but without the
interface auditing is impossible.

> FIPS 140-2 is important because it is what is used for Cryptech-like
> things and will be asked about. But we should not constrain us to FIPS
> 140-2 (and AIS31 etc for that matter). The more complete we can be and
> show good results to anybodys favourite test/test suite the better. We
> therefore need to know about them and use them all. imho.

At some point I've tried to come up with some proper tests myself;
turned out my statistics knowledge in that area isn't remotely up for
the job, so I got myself some texbook on it (makes me feel like an
undergrad again---25 years younger:-). 

But yes, from all I've seen we need some significantly broader test
sets.  I haven't yet taken a look at AIS31, but I'm more than suspicious
of the FIPS140-2 stuff, and for a whole number of reasons; it's a good
starter, but nothing I eventually want to rely on.

Once I've got the hardware side finally sorted out, I hope to give
testing a serious go---not just some quick Ruby hacks.  Not that I'm too
happy about testing my own stuff, but apparently for the last fourty
years this is the part of the job where everybody made sure not to look
too closely...


Cheers,

    Benedikt

-- 
Benedikt Stockebrand,                   Stepladder IT Training+Consulting
Dipl.-Inform.                           http://www.stepladder-it.com/

          Business Grade IPv6 --- Consulting, Training, Projects

BIVBlog---Benedikt's IT Video Blog: http://www.stepladder-it.com/bivblog/