Proposal for "wallet" data structure - please discuss

Autonomi Network Token (incl (e)MAID)
1

The purpose of defining a data-structure for a SAFEcoin “wallet” is to provide some consistency between wallet APPs and to allow some additional functionality for the network

What is provided by this data-structure

  • storage of addresses of the SAFEcoins that are sent to the ID.
  • storage of sub-SAFEcoin balance
  • storage of indexes (MD Address/Tag) to various wallet APPs specific data
  • coin addresses of SAFEcoins that are to be automatically used for PUT payments when needed
  • User’s default preferences for handling of wallet functions. Such things as
    • default display format for SAFEcoin/partcoin amounts
    • common bitcoin/altcoin addresses for wallet APPs to use.
    • default number of SAFEcoins to “preload” to pay for network services.

What is not provided by this datasturcture

  • definition of data being held by specific wallet Applications. This is handled by the wallet Applications themselves. But I strongly recommend that a common definition is developed so that the user can replace one of their wallet APPs and use the old APPs data/setup/settings. Note that it is expected that some/many users will have multiple wallet APPs depending on their circumstances

What needs to be worked out

  • Best method for securing public/private keys in the common area.
  • What ID to be used by the network for any “wallets” held by groups.

How this datastructure fits into the scheme of things.

  • SAFEcoins/partcoins are sent to an ID. IDs (eg public/private IDs) are sinply a number, address if you will, and the value in the owner field of a coin. When a user wants to receive coins/payment from another person then they will provide an ID for the coins to be sent to. This ID can be their public/private ID or any other ID they provide.
  • “Wallet” IDs are simply an ID. This ID can be the same as a public/private ID the user has or any other generated ID.
  • So “wallet” are addressed by using the ID as the address of a master MD object whose tag is a system tag and is owned by the account ID of the user and controlled by the network processes
  • An account can have many “wallets” the same as an account can have many public/private IDs
  • Wallet APPs maintain their own data structures that are owned by the wallet and it is hoped that all wallet APPs respect the users and never keep centralised data.

Current sizing of various objects/fields used in datastructure sizing calculations

ID                 128 bits (16 Bytes)
SAFEcoin Address    32 bits ( 4 Bytes)
partcoin balance    64 bits ( 8 Bytes)  Proposed
MD data storage    Approx, but less than 1 MByte

Proposed Data Structure

Proposal is to use MD objects to hold the information.

  • One MD for the “master Wallet MD” This MD holds the system and user central data & indexes. The tag type is a reserved system tag since it is a requirement that the address is available and cannot be taken by malicious entities.
  • Multiple MDs for Wallet APPs to hold their settings/preferences/keys/etc for the user. All owned by the wallet NOTE: It is expected that “good” wallet APPs will not store or send user dasta to any other storage area not owned by the wallet.
  • Multiple MDs holding the addresses of SAFEcoins that have the wallet ID as the owner. The current SAFEcoin proposal mentions that a network message is sent to the ID with the address of the coin sent to that ID. It would then be the responsibility of a wallet APP to receive these messages and populate these MDs with those addresses.

WALLET MASTER RECORD (MD)

 +-------------------------------------------+
 |                                           |
 | Account ID signature                      | 
 |   - Account ID is the owner of the        |
 |     "wallet" and "signs" any API request  |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Part-SAFEcoin Balance  u64                |
 | + security data if required               |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Addition fields for Group wallets         |
 |                                           |
 | Part-SAFEcoin Pseduo Balance   u64        |
 | Part-SAFEcoin Receipt Balance  u64        |
 | + security data if required               |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | SAFEcoin Count  u32                       |
 |   - Count of SAFEcoins "owned" by ID      |
 |   - Updated when coin received/sent       |
 |   - Total holdings is Count + Balance     |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Default wallet Preferences                |
 |    For example                            |
 |    - (part)coin display format            |
 |    - array of bit/altboin keys and        |
 |      security/password to unlock the keys |
 |      for APPs                             |
 |    - Etc                                  |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Array of Safecoin addresses for coins     |
 | that the network can use to pay for       |
 | network services (eg PUTs)                |
 |   - Last in - First out                   |
 |                                           |
 | Each array element                        |
 |   - U32  (address of coin)                |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Last coin address Index                   |
 |   - last valid element in the array       |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Array of MD addresses to MDs storing      |
 | the SAFEcoin addresses "owned" by the     |
 |  (wallet) ID.                             |
 |   - Last in - First out                   |
 |                                           |
 | Each array element                        |
 |   - XorName   MD address                  |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Last "coin address MD" address Index      |
 |   - last valid element in the array       |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Array of MD addresses to APP's master     |
 | Record (MD)                               |
 |                                           |
 | Each array element                        |
 |   - APP Name   (maybe 64 bytes)           |
 |   - XorName+tag   MD address              |
 |                                           |
 +-------------------------------------------+
 |                                           |
 +-------------------------------------------+
 |                                           |
 +-------------------------------------------+

SAFEcoin addresses records (MDs)

 +-------------------------------------------+
 |                                           |
 | Account ID                                |
 |   - Not needed but allows verification    |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Wallet ID                                 |
 |   - Not needed but allows verification    |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Last coin address Index                   |
 |   - last valid element in the array       |
 |                                           |
 +-------------------------------------------+
 |                                           |
 | Array of SAFEcoin Addresses               |
 |   - Last in - First out                   |
 |                                           |
 | Each Array element                        |
 |   - U32 Address of coin                   |
 |                                           |
 |                                           |
 +-------------------------------------------+

Sizing of data in the MDs

SAFEcoin addresses records (MDs)

There is almost the whole MD data area available (>995K) to store the ‘Array of SAFEcoin addresses’

Number of elements possible = approx 995K/4 = 248750 elements (SAFEcoin addresses)

WALLET MASTER RECORD (MD)

The “wallet” Master MD has 3 large arrays. Initially consider the array of SAFEcoin addresses MDs to allow the wallet to hold the addresses of the whole 4 billion coins, and allow for an extreme amount of “pre-loaded” SAFEcoins array. (say 10,000 coin addresses)

Array of MD addresses to MDs storing the SAFEcoin addresses “owned” by the (wallet) ID.

Max number of elements = 2^32 (4,294,967,296) / 248750 = approx 16,267 elements (MD addresses)
Max Bytes = 16,267 * 16 bytes = 260,272 bytes.

Array of “pre-loaded” SAFEcoin addresses

10,000 coin addresses * 4 = 40 KBytes

Array of MD addresses to APP’s master Record (MD)

Allow 100 KBytes in “wallet” master MD for incidental values (balance, prefs, etc)
Allow 261 Kbytes for coin address index array
Allow 40 KBytes for “pre-load” array
This leaves approx 600 Kbtes available and if allow 100 bytes per APP index then it is possible to have 6000 APPs “registered”

Obviously it is extremely unlikely that anyone could ever use the maximum of the arrays in the “wallet” master MD and it would be safe to have the arrays grow as they are added to and not have a set size for them. They are never going to exceed the available space in the MD.

Requirements for the network core code

There will need to be an API added or modified to provide functionally to create the “Wallet Master MD” which is owned by the Account The address of the MD is the ID value and thus the ID would have to be verified as being owned by the Account. The partcoin balance field of the MD would require its permissions so that it can only be modified by a network manager.

The “pre-load” coin addresses will be used by the APP(s) that upload data and/or network managers to automatically pay for resources when required. This is to allow the user to have an automatic payment for uploads and not be queried every time a coin needs to be “spent”. Spending a coin for resource payments will now be dividing one of the wallet’s coins to add one coin’s worth to the wallet’s partcoin balance.

Coin divisibility topic examines the APIs needed to handle the partcoin balance.

16 Likes
2

@neo is this your SafeCoin plan that you’ve been hinting at for months and months? If so then I’m very excited to read it!!

Let’s all dive into this! Very timely, with the SafeCoin Wallet CEP near on the horizon

3

@whiteoutmashups, nope not the division proposal, but just the MD structure needed to hold SAFEcoins and importantly the divided coin balance.

We need some sort of objects to store the addresses of the SAFEcoins we own and why not incorporate the divided coin balance in there too

3 Likes
4

Gotcha. Nevertheless, very glad to hear that all your ideas are on the table now!

SafeCoin, here we come!!

3 Likes
5

Some dummy questions/comments…

So the master wallet is created only when the account is created?

Why would you want this to be part of the master wallet instead of being something held by the wallet apps?

I’m getting that you are making it explicit about LIFO since they are just entries in the MD so you need their indexes to be consecutive. For 2 arrays is simple since one array grows from the lowest index upward, and the other one from the highest index downward, but you have 3 arrays so I’m not sure if you can handle them together in the same MD without wasting MD storage when one of them overlaps with the other, unless you perform some shifting operations which I would try to avoid.

1 Like
6

Its “Wallet Master MD” by reversing it you get your question. By not reversing it, which is what I propose then the Master MD is for the wallet

The idea is to have any number of wallets in an account. Create an ID then create the wallet master MD and that is a wallet. A wallet APP is assumed to be able to perform this for the user.

The process would be

  • If new ID required then create it and add it to the account info. Otherwise ask which ID the user wants to create a wallet for.
  • Do API call to create the “master MD”
  • If the users wishes then the APP “registers” itself in the “master MD” record by storing its name, address/tag of its data MD that holds the preferences etc. This might be an automatic action or a “tick box” specifies it.
  • Asks the user for their preferences.

It is assumed that “good” wallet Applications will perform the above.

This is for the wallet master record not “master wallet”

The reason for the defaults is for any wallet Application to know what the user default preferences are and allows a much smoother experience for the user rather than asking every time the user tries out a new wallet APP

If this is not desired then it can be removed. I proposed it from my experiences with writing code that is user centred rather than developer forced code.

My understanding is that the safe core code handles the changes in sizes of each field in the MD. Each of the arrays (and associated “count”) is a independent field in the MD. Thus for the APP growing arrays is not a problem. The APP simply writes out the new array.

Or it could be that if the 100 field limit is increased to very large then each element is a field. But this might be “processing expensive” on the core code. I would prefer that each array is a field and the APP works on the array when increasing/decreasing the number of elements

If we were to use the old SD style then it would be that you read the arrays into memory arrays, do the work and then write the SD back out. The growing arrays is no issue

The reason for LIFO is that there has to be some method of choosing which coin to remove. If it was random then there would be “holes” and there would have to be a flag to say which element is valid/invalid. By using LIFO in the master “coin address MDs” and “coin address arrays” then only one variable is needed for each array and that is the “last coin address MDs”/“last coin address” in the array. (or array size if you want).

It would be unusual for the user to wish to choose a particular coin to spend since they are all the same. But if that was to become the case then they could perhaps use a wallet APP that allowed that and the APP simply takes the coin address from the last element and places it into the element that the specific coin came from.

tl;dr

LIFO is to keep it simple and easy. But APPs can do more complex as long as its left in a form that is packed.

2 Likes
7

I got you now, so an array can be just a json string “[…,…,…,…]” value of a key/value pair entry in the MD.

2 Likes
8

Essentially. I didn’t try to define the exact format and leave that to further MD development or discussions here. If this gets traction then the RFC would need to have this specified if people are going to write APPs using it.

2 Likes