Storing blockchains on SafeNetwork

The bitcoin blockchain is now nearing ~320GB of data. Sometimes there are just not enough peers seeding fast enough so downloading a snapshot of the block data can make the synchronization process a lot quicker. Torrentz as in p2p networks really get their fast speed due to many individuals seeding chuncks of files. In the same way we can now store blockchain data on the safenetwork and leverage the download speed of many nodes sharing these files. Once you download the blocks for the given snapshot your node can validate every block.

I also wonder if we could take it even further and allow a full node (except the personal data like wallet.dat) to be stored as ImmutableData and then mount this safe://directory as a network storage device (read-only) so we now no longer need to download this whole chain to our harddrive.
It will only require to get bits of this data to query transactions and find the given balance (UTXOā€™s).

Mount safe://blockchain/btc with symbolic link to ~/safe_btc_blocks and where ~/private_bitcoin_files is a folder that contains you private files like wallet.dat (to seperate private and public files)

My goal would be to be able to do something like this (on linux):
$ ./bitcoin-qt -blocksdir=~/safe_btc_blocks -datadir=~/private_bitcoin_files

Does anybody know if this would be technically possible? This could bring full nodes to even mobile devices as they only need to query certain blocks and then have remaining data stored locally.

And since these blocks would be public ImmutableData many people can also verify them, maybe sign or (ā€˜likeā€™) it to show it is authentic and valid. And because it is Immutable people can also trust that it cannot change anymore and is therefor SAFE to use :wink:

18 Likes

Iā€™ve been thinking this myself. I think the tricky part will be adding blocks/transactions which I havenā€™t given any thought but will be soluble.

Iā€™m not sure what the best way will turn out to be. There are already Blockchain wallets that operate a bit like this. I think Electrum is one, Edge wallet might be another. Or there might be a way of using the particular features of Safe data types.

4 Likes

This would be awesome because with this network consensus you can now easily host many blockchains on SafeNetwork and cross-chain atomic swaps become quite easy.

4 Likes

It will be interesting to see how the regular bitcoin app works over the SNFS mount. I donā€™t know anything about the characteristics of what the app does at start up or how it scans the blockchain data for specific entries, etc. It may be fine or it may be dog slow.

I suspect that a client tweaked to work directly with the safe network may resolve some issues that may crop up though.

It will be interesting to do some tests when weā€™re up and running.

1 Like

There arenā€™t just the blocks, there is the block indexes ā€¦ Iā€™m no expert, but my guess is the indexes are specific to the wallet transactions ā€¦ but if indexes can be stored on SN for the specific user and if the BTC client itself can be WASMā€™d, then the whole shebang can be run from SN and secured by SN - portable BTC wallet with nothing to carry around.

Not sure if how WASMā€™d clients can communicate internal to SN though - would require some client adaptation to SN.

Iā€™m probably just dreaminā€™ here though. :wink:

1 Like

It just occurred to me that one nice thing about storing blocks as a public blob is that everyone who tries to do it will get the same content hash and thus address. So it will be very efficient to crowd source the actual data. Then there can be multiple independent indexes for however one wants to interact with it.

8 Likes

First off, I disagree with the premise that blocks cannot be served quickly by the Bitcoin network. Syncing the blockchain is also dependent on the hardware your node is running on, and that is the bottleneck in my experience.

I think using block data stored on SAFE however would be a good way of pruning your node. You would run your full node as usual. Then once blocks have reached a depth that you are comfortable with you could check that the same data exists on SAFE and then replace your local data with a symlink to the data on your mounted SAFE drive.

If you just blindly followed a chain stored on SAFE I donā€™t think you would be getting all of the benefits of a full node. You need to be connected to the Bitcoin network and receiving blocks from different peers to ensure that you are on the longest chain. Blindly following a chain on SAFE would be like eclipse attacking yourself (just trusting what one peer is telling you the chain is).

2 Likes

This sounds unbelievably exciting and could be an interesting marketing ā€˜hookā€™ for the roll-out comms to generate interest about possibilities and future potential.

6 Likes

Regarding to plugins, do you think this is similar to oracles? Like nodes on the linkchain that provide smart-contracts with certain data?

I think oracles could be the first step for the SafeNetwork to slowly move to a fully decentralized computational framework and Iā€™d say writing AWS Lambda like code would be the best option. Small simple functions that have a predefined computational timeout, the user pays upfront and depending how long it takes is returned the remaining unused credit.

By serving many of these small functions to different nodes you obscure the larger process so nodes are less likely to figure out sensitive data (the goal anyways is to allow nodes to do all these calculations in a encrypted manner so this would eventually not be an issue).
Small code also has benefit that many machines can easily take part in the job, the micro services also scale incredibly well horizontally.

Really something to look into, decentralized microservices/computation for SAFE.

4 Likes

I suggested cardano stake pools run on SAFE.

They are 90% ā€œdecentralizedā€ apparently but what they donā€™t tell you is many of the stake pool operators use AWSā€¦

Just my two cents on what else can be run on SAFE.

7 Likes

If one ran, say a bitcoin full node and wanted to prune, it would be trivial to choose to trust SAFEā€¦ imagine walking through the database, using a SAFE-ffi function to hash and get the public blob xorurl for each block. You could easily build your own index directly to the SAFE network. You could then compare it to a specific version of an appendable index on SAFE, and then permanently trust the index/version, and if you wanted to you could store/verify a UXTO set at that height as a public immutable blob that you could always trust. SAFE is an unbelievably efficient method to archive blockchains.

10 Likes

Presumably, the data oracles would be hosting could just be stored natively on safe network by an external process. This data could be of any size, large or small. Clearly, blockchains canā€™t store large data on chain due to scaling issues, but that isnā€™t an issue for safe network.

E.g. The Met Office could write weather data directly to safe network, signing the data for authenticity. Same for stock market prices by Reuters, etc. Any other app can then read this data natively from the network, including smart contracts.

Ofc, in the early days, the source of data may be a proxy user until the big boys get involved. Maybe use multi sig to give more confidence. Maybe even have a plugin which pulls the data and asks other nodes to confirm and counter sign it too.

Having immutable data integrated natively from the lowest level on the network gives lots of options here.

Edit: To add, what would be really cool is not having to depend on centralised sources of truth at all. Maybe a network of folks taking weather data or observing price data, the publishing it, etc, would be feasible

4 Likes

If it can be stored on the Safe network, can it be stored on the storj network?

Storj is operating right now, so it might function as a temporal solution until the safe network comes online.

1 Like

The blockchain is a database with indexing.

Yes it could be stored on Safe and Iā€™ve suggested that as blocks are confirmed they could be added to the copy on Safe and the signatures of the nodes confirming the block can be added so that any invalid block is identified and ignored by any Apps.

Now the fundamental issue is speed. While Safe is fast a node with the blockchain stored locally on its disk will be able to access the blockchain and do lookups needed to verify balances much much faster than a node trying to use the copy on Safe. Plugin or no plugin, the issue is with lag across the wire for each lookup. 100mSec instead of sub-mSec for each read/lookup

But where the copy on Safe is good is for people wanting to build/rebuild their node. The download of the whole blockchain or portion would be able to max out most consumer internet connections.

6 Likes

@Antifragile ,

While I usually like what you write, I stopped reading the single gigantic para above because it is too hard on my eyes and too hard to follow . . could you edit the post and add some para breaks?

5 Likes

Maybe enter key was broken

2 Likes

The enter key is easy enough and would help a lot. I rarely read your posts because of this too. Same with others here who donā€™t break up their writing.

4 Likes

Not at all. Until Iā€™ve read them how can I know their value to me, so I have to choose what I read beforehand, and as I have limited time Iā€™m more likely to read something thatā€™s easier to understand. I be often begin reading a long paragraph and the find myself skimming or giving up because itā€™s asking more than I want to put in. Not just talking about you here, but in general.

Iā€™m just offering feedback. If you want more people to read, inserting some newlines would I think help and isnā€™t hard to do.

The same goes for people who write succinctly - how I value and admire them - or too much. The latter is my issue, and because I want people to be able to read my posts I work constantly to be more succinct, but itā€™s hard as you may have noticed. :wink:

10 Likes

I think a text editor plugin can do the job :stuck_out_tongue_closed_eyes:

2 Likes

Every lite wallet using Safe Network would be quite safe and good way to promote Safe Network as guard of their fortune.

2 Likes