Exploring Open-Source RNG Implementations Worth Learning From — Following Up on the rand-in-std Discussion

After asking why rand isn't part of std and getting a clear answer — that std's stability guarantee is fundamentally incompatible with the continuous refinement that a quality RNG crate requires (the rand_chacha → chacha20 dependency swap in 0.10.0 is a perfect concrete example of this) — I went down a rabbit hole comparing what other ecosystems and researchers are doing.

Sharing my notes here in case it's useful, and genuinely curious what this community thinks.

---

What rand currently uses (brief recap)

These are genuinely good choices. The question I was exploring is not "is rand broken?" — it isn't — but rather: are there open-source implementations with design ideas worth learning from?

---

Implementations I found instructive

1. PCG-DXSM — Melissa O'Neill (github.com/imneme/pcg-cpp)

The original PCG used an XSL-RR output function. O'Neill later introduced the DXSM ("double xorshift multiply") variant to correct a rare statistical shortcoming in XSL-RR. It is now the default generator in NumPy and was adopted by Go's math/rand/v2.

Go's source code even comments on the choice explicitly:

// Numpy uses DXSM and O'Neill suggests doing the same. // See https://github.com/golang/go/issues/21835#issuecomment-739065688 const cheapMul = 0xda942042e4dd58b5 hi ^= hi >> 32 hi *= cheapMul hi ^= hi >> 48 hi *= (lo | 1) 

What I find interesting here is the multiple independent streams model. rand's Xoshiro does expose a jump function, but PCG's stream design feels more principled for parallel simulation work. Caveat: PCG's streams are mathematically correlated, which is a real limitation in some high-dimensional Monte Carlo scenarios — there's an active debate between O'Neill and Vigna on this.

---

2. SFC64 — Chris Doty-Humphrey (PractRand source)

This one is underappreciated. Rather than a linear counter (as in LCG-based designs), SFC64 uses a chaotic counter, which gives it a different and arguably stronger equidistribution guarantee without reaching for cryptographic machinery. It's also the generator embedded in PractRand — the test suite effectively trusts it more than any other.

It doesn't seem to have wide adoption outside PractRand itself, which makes me wonder if there's a reason the community has moved past it, or if it's just less marketed.

---

3. Java 17's java.util.random — LXM Family (OpenJDK)

Setting aside Java's historical java.util.Random baggage (a plain LCG — genuinely terrible), the JDK 17 redesign introduced something architecturally worth studying: a proper interface hierarchy for generators.

RandomGenerator ├── JumpableGenerator ├── LeapableGenerator ├── SplittableGenerator └── ArbitrarilyJumpableGenerator 

The idea is that "splittability" and "jumpability" are first-class capabilities that a generator explicitly declares, rather than optional features bolted on. The LXM generators (which combine an LCG with a Xoshiro/Xorshift mixer) implement these interfaces cleanly.

This seems like a genuinely interesting design question for rand's API surface: should SeedableRng or a new trait eventually model jumpable/splittable generators in a more structured way?

---

4. ChaCha8 in Go's math/rand/v2

rand uses ChaCha12; Go chose ChaCha8. Neither is provably insecure, but the reasoning each project documented for their choice is a useful read in security-margin decision-making. Go's rationale (in the math/rand/v2 proposal discussion) is explicit about the tradeoff: ChaCha8 is fast enough that there's no incentive to reach for a weaker non-crypto PRNG even for non-security work.

This is exactly the kind of decision that belongs in a crate that can be revised — not frozen in std — which brings it full circle to where this thread started.

---

The Testing Question

One thing that became clear through this: "better" is empirically testable. PractRand (testing >32TB of output across 1000+ streams) and BigCrush from TestU01 are the bar that serious implementations are held to. Research comparing Xoshiro, Philox, PCG, and MRG32k3a systematically across 1000+ streams shows that testing a single stream is not sufficient for a full evaluation.

If anyone has run rand's current generators through extended PractRand and published the results somewhere, I'd be very interested in a link. The only results I've found are informal.

---

Summary

---

Questions for the Community

1. Is there a reason SFC64 hasn't been considered as an alternative to Xoshiro256++ for SmallRng?
2. Has anyone done extended PractRand testing on rand's current generators at multi-TB scale?
3. Does the jumpable/splittable interface model from Java's LXM design seem worth exploring for a future rand trait design?

1 post - 1 participant

Read full topic