[Cryptech Tech] Filesystem on keystore flash

[Rob Austein]

One of the work items that came out of the Berlin workshop is updating
the HSM's internal keystore API to handle (much) larger numbers of
keys, to be able to store additional data beyond just the basic key
data, and so forth.   In essence, this means implementing some kind of
filesystem on the keystore flash.

I've been kicking this around with Paul and Fredrik, and there seem to
be two main options that would make some kind of sense in this space:

* Some kind of well-understood simple file system for which we can get
  portable code off the shelf with acceptable license.  Eg:

  http://elm-chan.org/fsw/ff/00index_e.html

* PKCS #15, either written from scratch or by somehow adapting
  Cryptlib's implementation.

In either case, we need to deal with the erase and write behavior of
the underlying flash, which is not particularly friendly to any kind
of update-in-place operation.

We're currently leaning towards the above-cited FatFs package, with
some minor voodoo in our underlying driver.  Specifically: for each
sector that the filesystem code knows about, the driver allocates two
sectors (hence the informal name of "the 2x scheme" we've been using
to label this approach).  We don't try to update an existing sector,
instead we write the replacement to the companion sector.

This scheme requires two meta data fields per pair of sectors: a
serial number and a magic value.  General idea is that a sector is
only valid if it has the right magic value (a constant, 0xdeadbeef
would do, just can't be all zeros or all ones), so the magic value
field is the last thing written to a sector.  If both sectors in a
pair have the right magic value, the driver compares the serial fields
according to the rules in RFC 1982: the sector with the higher number
is the real one.

Obviously this scheme "wastes" a fair amount of flash space, but we
have plenty, and we like that this scheme seems simple enough that we
have some chance of getting it right.

Before we dive into this, though, we thought we should share the plan
with the list in case somebody has a significantly better idea.

Keep in mind that writing a filesystem is not really a major goal of
this project per se, this is just a tool we need.  That's the main
objection to PKCS #15: it's designed for exactly this kind of purpose,
but it does not look at all simple to implement, so we'd rather not
dive down that rabbit hole if we can avoid it.