Skip to content

Key erasure AES variant #299

Closed
Closed
Key erasure AES variant#299
Labels
F-new-extFunctionality: new, outside of RandP-lowPriority: Low

Description

@vks
vks
opened on Mar 13, 2018

I ported an implementation of a fast-key-erasure RNG suggested by djb to Rust: https://github.com/vks/aesrng

It is about two times faster than non-SIMD xoroshiro when generating 100 MiB of random data. Only fill_bytes is implemented for now.

Once stdsimd is stabilized, this might be of interest for Rand.

Activity

  1. dhardy commented on Mar 14, 2018

    @dhardy
    dhardy
    on Mar 14, 2018
    Member

    Your point is that it's possible to produce very fast cryptographic generators for bulk output by using CPU-specific features? Cool. But I don't much like comparing apples to pineapples.

    I suggest this would make a good external crate, but not be such a good inclusion with Rand itself since:

    • The speed comes from using SIMD instructions to produce large blocks of data. Many users of RNGs only want small amounts of random data at a time, so this is not a good general purpose RNG.
    • Fast key erasure is likely mainly of interest to specific users with very high security requirements? It appears a "sensible precaution" but one with high cost and little expected benefit; for most users isn't use of ReseedingRng going to be sufficient?
    • CPU-specific instructions mean your code only works on modern x86_64 CPUs; Rand is designed to be portable.
    • Low-level instructions are hard to review, so it would be better in its own crate so that individual projects can choose whether or not to use it IMO.

    To be fair, we should probably consider SIMD-optimised generators at some point, though I don't know whether it would be worth making them the default generators (for many applications where randomness is consumed in small quantities the performance enhancements may not be significant).

  2. dhardy
    added
    F-new-intFunctionality: new, within Rand
    P-lowPriority: Low
    F-new-extFunctionality: new, outside of Rand
    and removed
    F-new-intFunctionality: new, within Rand
    on Mar 14, 2018

  3. nagisa commented on Mar 14, 2018

    @nagisa
    nagisa
    on Mar 14, 2018
    Contributor

    To be fair, we should probably consider SIMD-optimised generators at some point, though I don't know whether it would be worth making them the default generators (for many applications where randomness is consumed in small quantities the performance enhancements may not be significant).

    Could always pick a SIMD specialisation on when user calls a fill_buffer with a larger buffer, and otherwise use the non-SIMD implementation with less throughput but possibly better latency characteristics.

  4. dhardy commented on Mar 14, 2018

    @dhardy
    dhardy
    on Mar 14, 2018
    Member

    Yes, but SIMD probably makes more sense with buffer-backed RNGs like the current cryptographic ones (generate a block at once).

  5. dhardy
    added
    X-enhancement
    T-RNG
    on Mar 14, 2018

  6. vks commented on Mar 19, 2018

    @vks
    vks
    on Mar 19, 2018
    ContributorAuthor

    @dhardy

    The speed comes from using SIMD instructions to produce large blocks of data. Many users of RNGs only want small amounts of random data at a time, so this is not a good general purpose RNG.

    Why? Our current StdRng is buffered as well, via BlockRng. If I use a similar buffer (128 bytes), I get slightly worse performance than StdRng on master (5.1 ns instead of 3.55 ns per u64). Initialization is a lot cheaper (31 ns instead of 7.6 us) and the key is erased every 128 bytes.

    Fast key erasure is likely mainly of interest to specific users with very high security requirements? It appears a "sensible precaution" but one with high cost and little expected benefit;

    It gives forward secrecy. This is a nice property to have, because it means that i.e. compromising a server does not compromise previously generated secrets (if they are not stored in some other form). In that regard it is similar to ReseedingRng, but more reliable. I'm not sure what you mean with high cost? The total cost is comparable to StdRng. Removing key erasure improves next_u64 performance by about 40 % (3.08 ns, making it faster than StdRng).

    CPU-specific instructions mean your code only works on modern x86_64 CPUs; Rand is designed to be portable.

    Rand can still be portable by falling back to something else on other platforms.

    Low-level instructions are hard to review, so it would be better in its own crate so that individual projects can choose whether or not to use it IMO.

    That is a fair point, it contains a bit of unsafe code. I hope this can be improved by using a safe abstraction like faster and maybe aesni.

  7. pitdicker commented on Mar 19, 2018

    @pitdicker
    pitdicker
    on Mar 19, 2018
    Contributor

    @vks Nice project!

    I tried to read up a bit on the design of this RNG, but I am not there yet 😄. Is my understanding right?

    The basic idea is that common RNGs for cryptography keep some of their generated values in memory, and this memory can be leaked (swapping to disk, inspecting process memory, etc.).
    This RNG by design has to overwrite a generated value when making the next one, so it is secure by default.
    It is very fast because it uses a hardware implementation of AES, and because it does not follow the clunky implementation requirements of NIST.

    I sure think it is neat. It may be good to wait a while before really using it to give it time to prove itself. And also good to take lessons from. One thing I had planned for the BlockRng wrapper was to erase used values.

    But I see no reason why this RNG has to be in Rand, at least not right now. We are moving towards no, or as few as possible, RNG implementations, not many.

  8. vks commented on Mar 21, 2018

    @vks
    vks
    on Mar 21, 2018
    ContributorAuthor

    This is certainly not something for the near future (the algorithm, the implementation and SIMD in Rust are quite immature). I was just wondering whether it might be a better default in the long run, and the point of this issue is to discuss the pros and cons.

    Is my understanding right?

    Yes, I think it is.

  9. pitdicker
    changed the title [-]Faster RNG for filling buffers[/-] [+]Key erasure AES variant[/+] on May 8, 2018

  10. pitdicker commented on May 8, 2018

    @pitdicker
    pitdicker
    on May 8, 2018
    Contributor

    Renamed the issue from "Faster RNG for filling buffers" to "Key erasure AES variant", as this is more about a new CSPRNG than about filling buffers.

  11. pitdicker commented on May 8, 2018

    @pitdicker
    pitdicker
    on May 8, 2018
    Contributor

    This article offers an improvement over using AES in counter mode, in that it offers backtracking resistance at lower cost or a better API.

    If at some point the internal state of the common variant is exposed, all previously generated values can be recovered. The NIST document has some requirements that the common variant should overwrite its state when the user is 'done with it', but that is not easy for an API.

    This variant will, after generating a buffer with a couple of blocks of values, also use one block to generate a new internal state (or seed). This overwrites the old key, making it impossible to recover previous rounds.

  12. dhardy commented on May 15, 2018

    @dhardy
    dhardy
    on May 15, 2018
    Member

    Another fast backtracking-resistant RNG: https://github.com/google/randen

  13. vks commented on Jun 25, 2018

    @vks
    vks
    on Jun 25, 2018
    ContributorAuthor

    Note that Randen provides a Rust implementation in the official repository.

  14. dhardy commented on Oct 5, 2018

    @dhardy
    dhardy
    on Oct 5, 2018
    Member

    The Randen paper has been published: https://arxiv.org/abs/1810.02227

  15. dhardy commented on Sep 13, 2019

    @dhardy
    dhardy
    on Sep 13, 2019
    Member

    We shouldn't forget about Randen as a possible alternative to ChaCha; it is fast, fairly simple code and has backtracking resistance. Right now I see two issues:

    1. Limited portability: it relies on a CPU-native AES instruction for performance, thus we should keep an alternative available for some platforms (like we do with HC-128 now).
    2. Trust: while the code looks reasonably straightforward and is over a year old, the algorithm is still fairly new in academic terms, apparently with little review.

    This would provide an alternative to #872, for some platforms.

  16. dhardy
    removed
    T-RNG
    on Jul 10, 2024
  17. dhardy
    removed
    X-enhancement
    on Jul 20, 2024

  18. dhardy commented on Aug 20, 2026

    @dhardy
    dhardy
    on Aug 20, 2026
    Member

    On key erasure: we can potentially add this without much code (see this and the mention in #1828).

    On Randen: Google's repo is now archived and the crate hasn't been updated in eight years; ChaCha works well for us.

  19. dhardy
    closed this as completedon Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    F-new-extFunctionality: new, outside of RandP-lowPriority: Low

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions