The thing is, if we want to do hardware signing, threshold_crypto gives no easy way to get the bytes in a standard way for use with that hardware.
threshold_crypto::SecretKey.reveal() gives the ‘standard’ big endian hex value that matches all other places. But it’s given as SecretKey(Fr(0x<the hex>)), so isn’t ergonomic to work with (have to strip the start/end, then parse the hex to bytes). Why not just give us the bytes directly with SecretKey.to_bytes()!
hex(bincode(SerdeSecret(SecretKey))) does not match SecretKey.reveal().
So we just need to be careful about how the secret is managed. There’s always some way to recover/convert the info so let’s not sweat it too much just yet!
I’m currently working on an EIP2333 tool that will complement the threshold_crypto_ui tool nicely. I’m pretty sure EIP2333 will gain rapid traction, it’s a great standard.
That would be amazing.
I’ve poked around a bit with the idea of SecretKey.to_bytes() and SecretKey.from_bytes() in #104 but the presence of SerdeSecret makes me feel those changes may not be so easy to get through. threshold_crypto seems very cautious about accidentally leaking secrets.
Hopefully these helper methods can be added somehow, I would love it
You confused me a bit with this part, so for hardware signing (hardware wallet right?) we wouldn’t need the secret key, perhaps not even the pk, what scenario is it you are imagining here?
Oh yes sorry this is confusing! I meant to say offline signing, or any non-maidsafe-based signing. In that case people need a standard ‘view’ of the secret key to be exported/imported to/from maidsafe/other tools.
If we show users hex(bincode(SerdeSecret(SecretKey))) the value would not be importable anywhere else (because they expect big endian but bincode gives little endian).
So a basic compatibility test might be
get a key from maidsafe software (not currently possible but will be in the future)
try to import that key somewhere else (maybe a piece of zcash software)
(optional) use zcash software and the maidsafe key to sign a message
(optional) bring the signature back into maidsafe for verification (and broadcast if desired)
If we can do this then we will have access to a large ecosystem of bls tools that will ‘just work’ with maidsafe software. If we can’t do this people may need to do some preprocessing of keys coming out of maidsafe software before being used elsewhere.
It’s like BIP39 / BIP32 but for bls keys instead of bitcoin keys. Very handy indeed.
I’ll be adding signing / verifying / encryption / decryption in a future release but be aware these operations will not currently be compatible with the threshold_crypto_ui tool (despite both being bls12-381 based).
There doesn’t seem to be much likelihood of threshold_crypto becoming compatible with other bls implementations such as ethereum, zcash, chia, algorand etc.
I agree it would be good to make it compatible. Not sure if anyone is working on this repository right now, though.
There are several people I could see being able to implement the necessary changes to become compatible, maybe with funding it might be possible to get them to submit pull requests, I’m not sure, but I also don’t think it’s necessarily needed or desired.
But, for now, I’ve made threshold_crypto compatible to the degree I feel necessary with the repo blsttc - "poanetwork/threshold_crypto using supranational/blst for sign+verify "
Of interest is commit 27c64dc - Use BLST for sign and verify
This allows us to keep the existing api of threshold_crypto (no code changes to sn_* code) but get the performance and standardization benefits of blst.
Why use blsttc?
faster, around 2x to 12x faster to sign and verify than threshold_crypto (see this bls performance post)
based on a standardized implementation, so the inputs and outputs can be used with many other libraries, hardware devices, tooling etc where threshold_crypto would not.
compiles with musl
Why not use blsttc?
uses unsafe a lot (73 instances in lib.rs), may or may not be a problem in real life.
The library deliberately abstains from dealing with memory management and multi-threading, with the rationale that these ultimately belong in language-specific bindings.
Another responsibility that is left to application is random number generation.
The essential point to note is that it’s the caller’s responsibility to ensure that public keys are group-checked with blst_p1_affine_in_g1
State of audit / security is not as clear as threshold_crypto
blst: “Formal verification of this library is planned and will utilize Cryptol and Coq to verify field, curve, and bulk signature operations.”
threshold_crypto: “An official security audit has been completed on threshold_crypto by Jean-Philippe Aumasson.”
I have no expectation of blsttc being used but I personally find it useful so maybe others will too.
I do think we need to consider that move. Using an unmaintained (even if it is audited) lib is not a good move. This would be a great contender for BGF for sure.
Sorry I didn’t really get to explaining this properly. Let me explain why I don’t think we need to do major low-level changes to threshold_crypto.
threshold_crypto is a really nice lib. It has a clean interface and is well understood by maidsafe devs. These are the main differences:
dkg via poly+commitments (can be added to other libs, the initial poly.rs code comment is very helpful, but currently these features are unique to threshold_crypto). SN uses dkg extensively which is a good reason to keep threshold_crypto at least as an interface. Other bls libraries have the required low level functions, but do not have any sort of higher level Poly or Commitment types.
bls encrypt+decrypt functionality is unique to threshold_crypto (eg blst does not offer these methods). Sure, SN doesn’t use it (instead using ed25519 or symmetric encryption), so this is only a minor point. If we can use a standard then great we should, but no other library offers these methods (eg bls in javascript api does not offer encrypt+decrypt) so I’m not sure if there is a standard for bls encrypt+decrypt.
other bls libraries all seem to offer public key and signature aggregation, which threshold_crypto does not. This feature isn’t needed for current network operations but might be very useful later on, I’m not sure yet.
So I feel we shouldn’t necessarily aim to ‘fix’ threshold_crypto, since it’d probably still be slower than blst which is heavily optimized. But we probably should aim to keep threshold_crypto api since it’s nicer than the alternatives and it’s already used extensively by maidsafe. I guess blsttc satisfies both these in a very simple way, standardizing and speeding up the most common operations (sign+verify) but keeping the public interface identical to before.
This is important for sure. Dealerless key generation is critical to these libs IMO.
The weirdness part? We do this ourselves and I would prefer to use a crypto lib for that as we don’t want to hire cryptographers for this. This is a failing of threshold_crypto but we have a solution for now.
Small update to the tool that allows different encodings for keys (hex/bytes) and messages (ascii/hex/bytes). Very handy to have a simple switch when using these values in dev tools or hardcoded in rust.
For example the chia test specifies the signed message as bytes [7,8,9] (can’t type those ascii chars easily from a keyboard), so it’s nice to be able to test that sort of stuff.
And after all this bigendian stuff, turns out filecoin uses littleendian bls keys…
paulmillr/noble-bls12-381 - Add a note on filecoin - “Filecoin uses little endian byte arrays for private keys - so ensure to reverse byte order if you’ll use it with FIL.”
v0.4.0 is a really exciting step and feels close to being more than just a prototype.
The big change is to move from threshold_crypto to blsttc library, which was done for two main reasons:
data serialization is done in a consistent / documented / tested way using to_bytes and from_bytes. This means bincode is not used any more (yay!).
signatures are now bbs+ signatures which are compatible with virtually every other user of bls12-381 (eg chia, eth2 etc).
The serialization thing is pretty huge imo. Being able to settle on a particular format for Poly and Commitment data (and Ciphertext etc) makes them more portable and usable beyond just this tool (although I don’t know anyone using these kind of keys yet). Poly is sorta equivalent to bip32 xprv and Commitment is sorta equivalent to bip32 xpub.
Prior to this we only had portable secret keys and public keys, so hopefully we can see Poly and Commitment style keys become more widely used.
Now we can start to really think seriously about key management. EIP2333 is another similar key management tool but is a little constrained since it doesn’t have the concept of master public key for deriving child keys (only a master secret, at least as far as I know). So I’m really pumped about Poly and Commitment hopefully becoming easier to use.
Shoutout to github user glitch003 for giving some tips on how to get blst working with wasm in this issue. It was incredibly helpful and good timing considering this was an unprompted issue he raised at just the right time.
One small detail is I tried adding a single-threaded feature flag to blst, which worked for tests, but when I tried to compile to wasm using that feature flag it didn’t seem to omit the threading. I’ll have to look further into that (any help is welcome), but in the meantime I’ll be using this single_thread branch in the tool, which I’ll do my best to keep up to date with blst master branch.
No changes to the UI but the logic underlying it has changed in some important ways:
encryption and decryption has changed in a non-backwards-compatible way. Old ciphertexts will not decrypt with the new tool (but will still work with archived copies of the tool). The difference is how the plain text message to be encrypted is hashed to a point on the curve. The old hash method was non-standard, the new hash method follows the standard IETF hash-to-curve. Even though I’ve never seen any other library implement bls-based encryption (only signing), it’s really great to have the underlying primitives using standard protocols.
the bls low-level library changed from using a very old version of the bls12_381 crate to the modern blstrs (which uses blst behind the scenes). This means it’s faster and follows standards such as IETF hash to curve.
This doesn’t require a custom single_thread branch any more since blst has automatically disabled threading for wasm compilation (see Cargo.toml L34).
In future versions it would be handy to add functionality to derive child keys, but for now it feels good just to be up to date and following recognized standards.
