Simple web-based tool for BLS keys

You use single file and single page interchangeably. Parcel really generates a single page application, but not everything is stitched together in a single file. Instead html + wasm + css + javascript + map files are kept separated.

(I understand you prefer a single file, it’s just to be sure we are on the same page, pun intended!)

Just to be clear: build of application depends on nodejs, but the generated application is a static site that doesn’t need it (in the same way that build of bls-standalone.html file depends on Python, but not the generated file itself).

3 Likes

We have opted for this as default when we need human readable if that helps.

3 Likes

Ah yes, thanks for pointing this out, my mistake. I mean always single file, that is my goal for the tool. As far as I know Parcel cannot create a single file with wasm included, have you tried to get it to do that?

To illustrate why I aim for this, open this link BLS - Threshold Crypto and do File > Save (or your browser equivalent). It saves multiple files, which makes attaching or signing or hashing or verifying signatures difficult (could compress into a single file but it’s more steps for the user).
When I double-click the saved html file (or drag it into the browser) the tool does not work.

In the repository GitHub - Thierry61/threshold_crypto_ui: UI for threshold_crypto rust library if I clone it then go to the docs directory and double click index.html (or drag it into the browser) the tool does not work. The wasm depends on a server to be able to load. So this is not a portable solution.

Hopefully these example use-cases illustrate why I go for a single file so strongly. If Parcel can be made to do that in a simpler way than the python script I’d be keen to know, since using a standardised process would be preferable.

@bochaco maybe can clarify, will this be ‘standard’ base32z see philzimmermann.com for spec and cryptii.com for demo?

It looks like safe-api uses multibase which is not compatible with ‘standard’ base32z since the base is encoded into the string.

eg
“testing” to base32z using cryptii gives “qt1zg7djp3uo”
"testing to base32z using multibase gives “h7dfqp4g15u8”

Multibase is ‘standard’ in so much as it’s widely available across many languages, so no worries here just trying to understand the intended direction that maidsafe will take.

If multibase is chosen, it seems a mistake to say things are ‘base32z’ encoded, since they are really ‘multibase32z’ encoded. What’s the plan around communicating this? It would be frustrating to users to go to, say, cryptii.com and fail to decode because the format is not really base32z.

10 Likes

I also applied Parcel bundler on your new version (source: GitHub - Thierry61/threshold_crypto_ui: UI for threshold_crypto rust library and result: BLS - Threshold Crypto).

Same conditions as previous version (multiple static files in docs directory).

2 Likes

With version 0.2.0 the main feature set for the tool is complete.

https://iancoleman.io/threshold_crypto_ui/

This adds the group decryption feature that allows m nodes to decrypt a message encrypted using the group public key. Less than m nodes means the message can’t be decrypted.

A nice-to-have was also added, compressed wasm gets the tool from 467K to 237K, about half the size.

Next is

  • allow choice of encoding, default to base32z pending clarification above about ‘standard’ vs ‘multibase’ encoding.
  • add a glossary and explanation of the underlying data types and structures.
  • improve error detection and reporting.
  • add more tests to cover a wider range of use cases and edge cases.
  • feature to combine multiple bivarpoly commitments into a master commitment (handy for confirming dkg outputs)
  • feature to convert a commitment to a public key (handy for deriving the group master public key in dkg for nodes that only store the master commitment)
  • revisit UI/UX after a couple of idle weeks to check how it seems to newcomers (feedback on this would be great).
15 Likes

Looking into msgpack serialization and multibase32z encoding a bit, just noting some trivia and observations, there is no certainty yet if this is what will end up being used.

Using
multibase = “0.6.0”
rmp-serde = “0.14.4”

Secret Key

The current way in safe-api is bincode serialization and hex encoding which gives 64 characters

eg 281dbd717e63ae1845c66851977ea64223e5740c9e3b2637b84f0f8ea33d1a0a

The (maybe) new way in safe-api might be rmp-serde serialization and multibase32z encoding which gives 61 characters

eg hbfgxfkhyqabept7r9u9whqqdrjqxte6c6dw1sge9noum1z8ohfrupcdnnwxg

This would have a public key of 78 characters (sort of like the equivalent of a bitcoin address):

hmo1hqgurec6yhuas8edpb6w7b3nwqc7nrr5gs5ptmmmyowms9ntbe6yqsd75t6my7yupa5kszdbg7

So a slight decrease in the total characters when using msgpack and multibase32z.

To give an equivalent point of comparison, a bitcoin WIF encoded private key (base58) is 52 characters.

// Serialize using msgpack (ie rmp-serde crate)
let secret_key_vec = rmp_serde::to_vec(&SerdeSecret(&secret_key)).unwrap();
// Encode using multibase32z (ie multibase crate)
let mb32z_bytes = encode(Base::Base32z, secret_key_vec).into_bytes();

msgpack-to-json works as expected. Using toolslick.com messagepack-to-json the msgpack bytes (see below) can be decoded into a set of u64 values which represents the underlying 256 bit secret key finite field. Cool!

converting from multibase32z above into raw msgpack bytes

[148, 207, 42, 184, 7, 96, 40, 108, 122, 79, 207, 244, 227, 156, 50, 37, 207, 138, 60, 207, 14, 146, 177, 145, 241, 66, 107, 149, 207, 14, 20, 147, 107, 6, 33, 81, 230]

then into json using toolslick

[
  13837213670986298635,
  12811103515824107464,
  18046157036817630463,
  2492927070659696894
]

Poly with threshold 2

The current way in safe-api would be bincode serialization and hex encoding which gives 208 characters (although safe-api does not use or expose Poly it’s a larger and more interesting bit of data to use than the secret key and imo is inevitable when multisig wallets are implemented).

eg 0300000000000000b09167a4c8915eeb5c5e98b29a5bde0a6fd369375fedbaef3ddce4ec6ba9932557ca2d774b63f0d886dd200d442011162c47d33de5dbb2e2d99e3823e38e623b0979bde07259ba7f687f1e766bde4c7f72f6294f821836ef675108d76eca4603

The (maybe) new way in safe-api might be rmp-serde serialization and multibase32z encoding which gives 182 characters.

eg h1gj3juhjrg54nxzhsh9h9nnqqd1e37stjz8ibwzkzrudkb4m37u6ep48uu9r9tcw3her854rghbm7esxm9o1y8gkdtog9u6phiu4os4dh41h6gyrg5mhzfijyqkc98ja37sugesaed8st1p1ppo83cwk395u67frz68473qxmrbs54fiancnk

msgpack to json for poly-with-threshold-2

[
  [
    [
      8917136771447855699,
      10181468347128250025,
      16032800147641816965,
      1259811719774385146
    ],
    [
      4155445286346159596,
      8104559542406594117,
      2603743878687488964,
      1442636379315754736
    ],
    [
      922117657886018339,
      14657046883067931162,
      18344042788901578676,
      568681510477232256
    ]
  ]
]

u64 numbers

This way of encoding secret key / poly is sorta interesting, it uses rust u64 as the ‘base’ number (I guess due to how ff crate represents finite fields behind the scenes, presumably 4 x u64 which gives 256 bits?)

The thing is if this json were to be called in javascript like JSON.parse(my_list_of_u64s) it would be incorrect (with no indication of error!), since javascript works on f64 (max 253 integer) as the largest integers.

Taking the first poly u64 value and comparing to one less, 8917136771447855699 === 8917136771447855698 is true in javascript.

Or consider JSON.stringify([8917136771447855699]) gives "[8917136771447856000]"

and

JSON.parse("[13837213670986298635]")
gives
[13837213670986300000]

This u64 thing is not a problem per se but it might be worth considering places where rust u64 is used it might be better converted to 8 bytes for maximum cross-language compatibility? Dunno, just dumping my brain on this, need to think about it more.

11 Likes

That follows the Unix philosophy to a tee. Did you happen to check the base32z vs multibase32z encodings when converted to 8 bit char strings via their respective libraries?

2 Likes

using the secret key hex above gives

hex: 64 chars
281dbd717e63ae1845c66851977ea64223e5740c9e3b2637b84f0f8ea33d1a0a

multibase32z: 52 chars
hky7ziazha7qdbnhc4nt179kcotdhi4y38t5ra55ouaxt4tu4gok

base32z (from cryptii.com): 52 chars
fyq54hm6cqzbotqgpbe3q9igeet6k7ycua71cp7ajh8a7e37defy

base32 (from cryptii.com): 56 chars
FAO324L6MOXBQROGNBIZO7VGIIR6K5AMTY5SMN5YJ4HY5IZ5DIFA====

edit: sorry to clarify this is using bincode not rmp-serde… I’m gonna leave further work/edits on this until tomorrow, too much fuzz in the head to do a proper job just now!

7 Likes

If this is right then we will have to fix it as it may mean simply a bug I presume, I’ll eventually try to dig in to see what’s going on there.

5 Likes

Once again adapted it to Parcel:

Note that I have removed pako.js because Parcel uses wasm file as is.

Generated file sizes grouped by extensions (in bytes):

Extension Size
.html 16606
.css 674
.wasm 280216
.js 226227
.map 513670

I consider that wasm file is not excessively large.

Biggest part is the set of map files. I could remove them (Parcel has an option for that) but I don’t think they are downloaded by default (only on demand when debugging on source code).

For reference the command to get file sizes is:
for i in .html .css .wasm .js .map; do printf "%-5s: %6s\n" $i $(du -cb $(echo "docs/*$i") | tail -1); done | grep -v total

7 Likes

Curiosity got the better of me, yes, looks like multibase crate 0.6.0 has a bug, and 0.8.0 works as expected

println!("{}", encode(Base::Base32z, "This is a test"));

In 0.6.0 gives
hbkgo4murbwzgedbrb4gkh5w
which in base32z needs first char removed so becomes
bkgo4murbwzgedbrb4gkh5w
which does not decode as base32z.

In 0.8.0 gives
hktwg1h3ypf31yajyqt1zg7y
which in base32z is
ktwg1h3ypf31yajyqt1zg7y
which decodes from base32z correctly.

safe-api Cargo.toml multibase should be updated from “~0.6.0” to “0.8.0”

safe-nd too… not sure how far this is spread. It’s used for xorurl so is kinda critical in that respect!!

Note that multibase::Base::Base32z in 0.6.0 is changed to multibase::Base::Base32Z in 0.8.0 (capital Z)

14 Likes

Excellent, thanks so much for figuring it out @mav , we should be able to fix this for next release then (https://github.com/maidsafe/safe-api/pull/614).

13 Likes

Amazing response time!

I looked at upgrading safe-nd but it uses multibase::Decodable which is no longer in 0.8.0

Not sure if there’s a need for cross-compatibility in the two repos but I won’t be able to get further into this today unfortunately.

ian@ian-laptop:~/code/maidsafe/safe-nd (farming) $  grep -r "multibase" src/
src/utils.rs:use multibase::{self, Base, Decodable};
src/utils.rs:/// Wrapper for z-Base-32 multibase::encode.
src/utils.rs:    multibase::encode(Base::Base32z, &serialised)
src/utils.rs:/// Wrapper for z-Base-32 multibase::decode.
src/utils.rs:        multibase::decode(encoded).map_err(|e| Error::FailedToParse(e.to_string()))?;
src/blob.rs:use multibase::Decodable;
src/map.rs:use multibase::Decodable;
src/keys/mod.rs:use multibase::Decodable;
src/sequence/metadata.rs:use multibase::Decodable;
src/identity/node.rs:use multibase::Decodable;
src/identity/mod.rs:use multibase::Decodable;
src/identity/client.rs:use multibase::Decodable;
7 Likes

I don’t think so, or at least not for fixing the xorurl encoding if that’s what you mean? The xorurl is fully managed on the safe-api side, there is no other repo/crate which is even aware of it, eventually we can discuss about getting it in lower layers but this is how it works at the moment, i.e. the safe-api translates a xorurl into a xorname back and forth.

3 Likes

Came across EIP2333 today, basically BIP32 for BLS keys

“for deriving BLS private keys from a single source of entropy … allows for a practically limitless number of keys to be derived for many different purposes while only requiring knowledge of a single ancestor key in the tree.”

https://eips.ethereum.org/EIPS/eip-2333

Mostly my interest here is whether there’s a standard way to represent a BLS keypair and how it feeds into the concepts of poly / commitment in the threshold_crypto library, and whether Safe might benefit from this standard representation (existing libraries etc).

I came across EIP2333 from chia.net - they have a bunch of tests for bls keys in bls-signatures test.cpp and also refer to RFC 5869: HMAC-based Extract-and-Expand Key Derivation Function (HKDF) describing a HKDF (ie a Hashed Message Authentication Code (HMAC)-based key derivation function, got that?!) used by EIP2333

7 Likes

I’m still trying to track down a solid test vector for bls12-381 serialization. Not having much luck so far.

I found this from chia mapping secret key to public key, however threshold_crypto does not generate a matching public key:

I also came across this issue from Feb 2019 with ethereum2.0 (who will also be using bls12-381): Cross-blockchain BLS12-381 standardisation · Issue #605 · ethereum/consensus-specs · GitHub
“This is a heads-up that the BLS spec is likely to change. Many blockchain projects are converging towards BLS12-381 (Zcash Sapling, Chia, Ethereum 2.0, Filecoin, Dfinity, Algorand, etc.) and there is a strong desire for everyone to standardise on the fine details.”
“I encourage anyone interested in this discussion to email Sergey Gorbunov (sgorbunov@uwaterloo.ca) who seems to be leading the effort.”
Might be worth someone at maidsafe getting in touch.

I’ve started comparing blst vs threshold_crypto, since blst has a lot of to_bytes and from_bytes methods that can be compared with threshold_crypto bincode serialization. It’s a shame threshold_crypto doesn’t have any to_bytes or from_bytes since those methods would be a lot more reliable than arbitrary serialization (in this case using bincode).


Also to extend on EIP2333 (ie BIP32-HDkeys-for-BLS) there’s also EIP2335 BLS12-381 Keystore which specifies “a JSON format for the storage and interchange of BLS12-381 private keys. … This specification is designed not only to be an Ethereum 2.0 standard, but one that is adopted by the wider community who have adopted the BLS12-381 signature standard. It is therefore important also to consider the needs of the wider industry along with those specific to Ethereum.”
I’m not sure if this is worth picking up and using but it’s worth knowing about this ongoing work around key management for bls12-381 keys. Whatever problem (real or not) these proposals are addressing we can be pretty sure to be running into similar issues ourselves.

edit: to clarify why I’m pursuing ‘other peoples’ standardization, one of the benefits is hardware wallets will be interoperable between these other cryptos and Safe (so we don’t have wonder how to do hardware wallets, they’ll just be there already).

11 Likes

A bit of a large edit so I’ll put it in a separate post

I found this test for bls keypairs from chia which maps secret key to public key, however threshold_crypto does not generate a matching public key however the secret key must be converted to little endian before being used in the web tool. See this issue comment in threshold_crypto for more info.

The SecretKey in that test in little endian form is
045690f6a8fb6fac9ce7c1171740e4e2e1f572036240e6a7a091c0dae33b354a
which gives the matching public key from the test when used in the BLS web tool.

The current sn_api implementation serializes secret keys using bincode which defaults to little endian; the code is fairly deeply nested but ends up at sn_data_types/utils.rs L28.

5 Likes

Version 0.3.0 is out, see https://iancoleman.io/threshold_crypto_ui/

The main change is to be compatible with all other bls12-381 tools / libraries (which use big endian rather than little endian, eg ethereum2, chianet, zcash, algorand, …).

This means the chianet test posted above now passes here as well. Any hardware wallet, fpga, dev tools etc developed for other bls-based networks will also work with this tool (and presumably Safe Network too).

It’s a bit of a tricky situation because of these (unfortunately quite technical) reasons:

  • sn_api does not show keys to people any more so it’s hard to say if this tool is now compatible or not. I’m sure sn_api will end up importing / exporting keys the same way as all other bls12-381 projects, so it will end up being compatible with this tool (and other bls12-381 tools such as hardware wallets etc).
  • secret_key, public_key, signatures are clearly displayed as big endian in all places I can find, but it’s not clear how to display commitments and polys (as in, I can’t find any other bls12-381 tests for how these would be represented).
  • bincode is fine for simple structs like SecretKey but for more complex structs like Commitment and Poly bincode is almost certainly not what will be commonly used, so this is likely to change in the future. But it’s not clear how it will change.
11 Likes

This is something in the TODO list for the CLI as well, right now as a temporary solution the keypair the CLI obtains when authorised with authd is stored at ~/.safe/cli/credentials but with a non-standard format (we just serialise our KeyPair struct), so I guess apart from what you suggest about importing/exporting keys, I guess this is a good standard format for storing the CLI credentials file too?

Or what would be a good format to store diff type of keys, e.g. BLS and Ed25519 so it includes this info?

6 Likes

Hey @mav! Really happy to see that you’re maintaining and bringing new feats to this really nice multi-purpose web-app there! As for the pain points listed:

secret_key, public_key, signatures are clearly displayed as big endian in all places I can find, but it’s not clear how to display commitments and polys

threshold_crypto folks seem to have it implemented a basic Debug for poly and commitments, it shouldn’t be a problem to look into this and raise a PR to the crate, will get this in my to-do list (:

bincode is fine for simple structs like SecretKey but for more complex structs like Commitment and Poly bincode is almost certainly not what will be commonly used

bincode was our choice for all serializations in SAFE Network(due to it’s compactness and speed) and we made use of the same for our bls_dkg crate as well. But as you said so, it’s not the final version and we are defo looking out for alternatives to bincode(msg_pack, cookie_factory, etc,.) passively that’ll be more robust for complex structs. So do keep an eye if out for changes in the futures.

Hope it makes sense. Thanks! :slight_smile:

4 Likes