Ryyn (massive multi-device virtual FS)

In the tutorial, 1.1B the instruction is ryyn init and id is no longer used according to the program when run

Also ryyn says the format is ryyn <COMMAND> <SUBCOMMAND> [args] and is failing when I give it ryyn init secret-key password which is what the examples in the ryyn program show. I tried secret key with the leading 0x and without it.

.\ryyn info doesn’t work either and just throws up the help text

Do I have to start the daemon first?

3 Likes

I see the problem now, the tutorial is wrong (and the returned help text).

It’s showing the usage of the <id> wrong or not listing it at all. I’m sorry, that’s crappy.

It is:

ryyn <ID> <COMMAND> <SUBCOMMAND> [ARGS]

e.g.

.\ryyn.exe alice init 4773b75ed27b51b9b56f4b6578c34e23aa2c9bb691d93d7a13c5d62f88575411 onelittlefox

Yes, after init, daemon has to be started, so that’s 1.2 in the tutorial. It should be without confusion now, all the issues you described were because of where the <id> was expected.


I’m placing the correct (should be) tutorial here and will update the repo as well.

Click to expand:

Detailed Tutorial

Detailed Tutorial

Getting familiar in a test environment

Except for the initialization, the steps are the same for test and live environment.
Try this out locally with mock network etc. to feel comfortable with the operations.

1. Test/Live instance

Before starting the daemon, and before anything at all can be done, we need to setup the key store.

For a test network, any 32-byte hex will do.

The id is anything you want, but you need to include it in all cli calls.
Secret key is a 32 byte hex. For the main network this is your EVM hex.
You need a passphrase to encrypt the secret on file. There is no validation of passphrase entropy currently, so you may use anything, but not empty.

1.1 A) Initialize test instance

Use the terminal from the path where you keep ryyn.exe.

.\ryyn.exe <id> test-init <secret_key> <passphrase> <device_nr> <test_run_dir>

Device nr is e.g. the sequence nr of simulated devices; 0, 1 .. etc.
Test run dir is the custom path where you want the simulated network and the instance app dirs to reside.

For example:

.\ryyn.exe alice test-init deadbeef... pwd123 0 /my/path

After this we start the actual backend, a daemon that will receive all the requests from the CLI.

1.1 B) Initialize live instance

The pattern is:

.\ryyn.exe <id> init <secret_key> <passphrase>

e.g.

.\ryyn.exe alice init deadbeef... pwd123

The application data will be stored in e.g. C:\Users\<YourUser>\AppData\Local\ryyn\main.

And you will see e.g.:

Key saved to C:\Users\<YourUser>\AppData\Local\ryyn\main\key.enc

(Note: Only default path option is showed here.)

1.2 Start daemon:

In this call the passphrase is included as an env var, so that the daemon can decrypt your secret key.
You do not need to include it after this, as the daemon keeps the key in memory while it is running.
The id is included as well, this is used by the daemon to lookup the other details of your instance.

set "RYYN_PWD=pwd123" && .\ryyn_daemon.exe alice

The daemon is now running and the application is ready to use.

After having started the daemon, leave the terminal window open, this keeps the daemon alive. (You can run it in background as well, but for tutorial, do it like this.) You will be able to follow the internal workings of the daemon as it outputs logs to the terminal window.

2. Using the app

2.1 Display info

.\ryyn.exe alice info

You will see a table of vaults, mounts and devices.
At this stage, only your device will show.

2.2 Create vault from a path

Note:

You may want to start clean and simple to observe the behavior without noise, and for that reason use an empty folder to create a vault of.

.\ryyn.exe alice vault create C:\path\to\vault

This will also mount the vault (the returned value is the id of the mount).

Display info again

.\ryyn.exe alice info

This will now show a vault and a mount.

You can see in the daemon terminal window that the corresponding logs show there. Somthing like:

2025-08-26T15:35:31.007684Z  INFO ryyn_daemon: [DAEMON] logging initiated
2025-08-26T15:35:31.009730Z  INFO ryyn_daemon: [DAEMON] server started at port 54711
2025-08-26T15:35:31.479410Z  INFO ryyn_lib::engine::app: [SERVICE] started with 0 mounts
2025-08-26T15:35:33.575379Z  INFO ryyn_lib::engine::user_input: [INPUT] received: GetSystemInfo
2025-08-26T15:35:33.605584Z  INFO ryyn_lib::engine::user_input: [INPUT] received: CreateVault("F:\\path\\to\\mount")
2025-08-26T15:35:33.692928Z  INFO ryyn_lib::engine::app: [SERVICE] created vault d0210adb and mounted b471b24f

2.3 A new file

Drag and drop a file to the mount path. You can see in the daemon terminal window what happens.
Something like the following:

2025-08-26T15:40:31.852625Z  INFO ryyn_lib::engine::pipeline::delta: Chunking file of 20971520 bytes.
2025-08-26T15:40:32.288292Z  INFO ryyn_lib::engine::sync::push: [PUSH] pushing 1 events..
2025-08-26T15:40:32.288627Z  INFO ryyn_lib::engine::sync::push: -- event: 5cec9c15, FileCreated, ts: 1756222831794      
[delta::finish] file ef1c9a22 @ 5cec9c15
[delta::finish] recorded 15 changed chunks out of 15 in total. Size of changed chunks: 20971520, out of total: 20971520 bytes
2025-08-26T15:40:34.178594Z  INFO ryyn_lib::engine::pipeline::egress: [MANIFEST] 1 emitted
2025-08-26T15:40:34.294604Z  INFO ryyn_lib::engine::sync::push: [PUSH] pushing 1 events..
2025-08-26T15:40:34.295038Z  INFO ryyn_lib::engine::sync::push: -- event: e873f2b5, FileVersionCompleted, ts: 1756222834170

The emitted manifest, transferred between devices via the FileVersionCompleted event, is what allows us to restore the file. It is the full map to the data in the network, and how to rebuild it locally.

You can test to drag and drop files of various types and sizes.


Verify data sync

A) Test environment

Start two instances using:

<id> test-init <secret_key> <passphrase> <device_nr> <test_run_dir>

e.g.:

.\ryyn.exe alice test-init deadbeef... pwd123 0 /same/network/path
.\ryyn.exe bob test-init deadbeef... pwd123 1 /same/network/path

The secret key and the path needs to be the same for all instances of the same workspace.
The key is what lets them read the same data, and the path is the local mock network,
so they need to be on the same network.
The device nr needs to be unique for the key-path combination.

Open the drive dirs of the two instances in an Explorer window, e.g.

<test_run_dir>\devices\0\drive and <test_run_dir>\devices\1\drive

Create a vault on device 0:

e.g.

.\ryyn.exe alice vault create <test_run_dir>\devices\0\drive\<any-name>

Only the absolute path is taken for now, so use the specified path.

You will see a folder show up named <any-name>. That is the mounted vault.

Now keep an eye on the device 1 drive and drag and drop a file to the mount on drive 0.
The file should show up on device 1 under a folder with the same name.

See the two daemon terminal windows,

for device 0:

2025-08-26T15:40:32.288292Z  INFO ryyn_lib::engine::sync::push: [PUSH] pushing 1 events..
2025-08-26T15:40:32.288627Z  INFO ryyn_lib::engine::sync::push: -- event: 5cec9c15, FileCreated, ts: 1756222831794      
[delta::finish] file ef1c9a22 @ 5cec9c15
[delta::finish] recorded 15 changed chunks out of 15 in total. Size of changed chunks: 20971520, out of total: 20971520 bytes
2025-08-26T15:40:34.178594Z  INFO ryyn_lib::engine::pipeline::egress: [MANIFEST] 1 emitted
2025-08-26T15:40:34.294604Z  INFO ryyn_lib::engine::sync::push: [PUSH] pushing 1 events..
2025-08-26T15:40:34.295038Z  INFO ryyn_lib::engine::sync::push: -- event: e873f2b5, FileVersionCompleted, ts: 1756222834170

for device 1:

2025-08-26T15:40:32.541822Z  INFO ryyn_lib::engine::sync::pull: [PULL] num events pulled: 1
2025-08-26T15:40:32.542048Z  INFO ryyn_lib::engine::sync::pull: -- event: 5cec9c15, FileCreated, ts: 1756222831794      
2025-08-26T15:40:34.533994Z  INFO ryyn_lib::engine::sync::pull: [PULL] num events pulled: 1
2025-08-26T15:40:34.534375Z  INFO ryyn_lib::engine::sync::pull: -- event: e873f2b5, FileVersionCompleted, ts: 1756222834170
2025-08-26T15:40:34.627454Z  INFO ryyn_lib::engine::sync::apply: FileVersionCompleted fully applied.

The same file should now exist on both instances.

B) Live environment

Basically the same steps, just spin it up on two actual different devices.


5 Likes

Hi @oetyng. I’m looking to give this a spin too. Love the concept.

I’m making my way through the tutorial and have had 2 issues so far.

  1. This command didn’t work for me: set “RYYN_PWD=pwd123” && .\ryyn_daemon.exe alice

It didn’t like the ‘&&’, so I tried running the commands separately, but that didin’t give the env variable to ryyn, so I did this instead: $env:RYYN_PWD=“password”

That seemed to work and let me proceed.

  1. I’ve got the daemon running, and successfully pulled the info screen.

When trying to add a vault, I got:

error: Service error: Failed to create vault: Invalid bytes representation.

Not sure where to go from there, but hopefully it’s an easy issue to get past & I’ll then set it up on another computer and see if the sync works.

Looking forward to trying it out when I can get further along :slight_smile:

2 Likes

Hi @DavidMc0, good to see!

I have looked quickly, not seen anything obvious. I will have a deeper look tomorrow.

Can you confirm the release version used? (I’d recommend latest: v0.1.0-alpha.2)

My initial suspicion is the path passed as argument. Not sure exactly why atm, but that’s the input, and I know the encoding isn’t 100% there yet. Would you be able to paste it here?

Scratch that. The input from cli is String so the path should be valid UTF-8 from creation. Was thinking that something odd was sneaking in that way and tripping up SQLite type inference.

This code is running hundreds of times per day in e2e tests both locally and on ci, so it is an interesting case! (As you can tell, I couldn’t really let it be until tomorrow :grinning_face_with_smiling_eyes:)

1 Like

Thanks. I’ll try again with that version. I was using 0.1.0 relev 1.

The path provided for the vault was C:/ryyn1/test. Is that what you wanted me to paste?

1 Like

That’s my bad. I was afraid someone might pick that one up. relev is my test branch for the release scripts. The releases from it end up among the others until I clean them up.

Not sure if that would actually make any difference here though, as it is based on alpha from today. But use the alpha version anyway.

Yeah thanks for the path. It looks fine.

1 Like

Just a note to say I’ve tried with the new version (Alpha 2), and on 2 different computers, with 2 different wallets, and I’ve had the same error again (Invalid bytes representation).

The error looks a little different on the second:

ERROR src\engine\discovery\mod.rs:25: scan worspage: Invalid bytes representation."

Looking forward to trying it out when this issue can be resolved (either my end or yours… I could be doing something that messes things up, not sure!).

1 Like

Great, that was a good clue.
It’s not something you are doing. I think I have a pretty good idea now.

1 Like

I’ve added a fix, the new release is here: ryyn v0.1.0-alpha.3

5 Likes

The daemon keeps throwing an error

scan workspace: Record could not be found

src\engine\discovery\mod.rs:25

It will not let me create a vault.

.\ryyn alice info gives me info - no vaults (yet) and no mounts

Record could not be found is the Autonomi error when requesting a Scratchpad (in this case) that is not there.

Do you see that starting right after the vault create cmd?

Edit: At that place, this could only come from the polling of the workspace manifest, it’s fetched every 1.5s starting from when the daemon is started. At daemon start the manifest is either created in the network or an existing fetched, and then the polling starts.

So this would mean that either the manifest wasn’t successfully created at start, or fetching it is problematic for some reason.

1 Like

Yes, after the create command.

But now fails with no response to anything

Even the info command times out.

restarted computer

ran through with new ID and both init, & info time out

1 Like

Just throwing in the thought that transaction fees can be nasty too… High fee season and data not being created due to limits being exceeded did trick me once… (now the Blockchain fees are neglectable (again?) … But possibly that wasn’t the case when people ran into something…)

4 Likes

Thanks for this quick response!

With the new version unfortunately I’m now stuck on a ‘Scratchpad signature is not valid’ error. I’ve tried it on 2 computers with 2 wallets with the same error.

1 Like

Okay, thanks for the info. I’m getting home in a bit and will look that up.

3 Likes

This will take a bit more time because I’m setting up automated tests over live network. These issues are all on that path and do not get tested in the mock setup used in CI.

It’s not that much to update from the existing test harness, but I still have a few things more to do, and then after that there is rooting out to be done.

So, to explain a bit about that:
Two app instances running on the same device (requires init with custom app path) is to the system basically the same thing as them being on two different devices. All the app and autonomi layer comms, flows, wiring is identical. So by starting up multiple instances, one per emulated device and all in the same workspace (defined by the key used to initiate), a full live sync scenario can be setup on a single physical device, while still covering all the multi-device capabilities (given that we are now focusing on a single platform).
That is practical. So, just some tetris now to move things in place and then to the issues.

8 Likes

The issue is found, and a couple more of them. I may be able to get that into a new release tomorrow, otherwise it’s after the weekend.

8 Likes

New version coming in a moment.

4 Likes

Release v0.1.0-alpha.4

Link: release v0.1.0-alpha.4

Hopefully things will be smoother now @DavidMc0. Don’t hesitate to ask for help if anything - however tiny - is confusing, not working or what ever. I’m just happy to help.

Bug fixes

With this release live network e2e test scenarios with multiple devices syncing files were setup and used to get the full (tested) path consistently flowing well live.

Logs

There is a much more detailed flow of logs showing in the daemon terminal. Helps to understand what is going on.

Performance

Scratchpad creation has been very slow for me, but I have used the same key for all simulated devices, so that would slow things down (serialized on chain).
But even without that, scratchpads are used heavily in the application and it has made me consider allowing a user to pass in multiple evm keys for rotation on hot paths. Before that’s implemented though there will probably be some pre-creation of pads.
Also, there is a lot of optimization to be done in handling of files, chunks, restore etc.

Workspace decoupled from payment

An addition to the api was made, primarily for fast iteration on the live e2e:s, but a generally sound feature: optional decoupling workspace key from evm-key. This means that your workspace key is not derived from the evm-key, and so you can use any evm-key anytime, from any device, it’s not coupled to anything, it’s just for payment. (It requires an optional parameter though, so it’s not the default mode.)
This was convenient since I didn’t want to fill a new evm-key for every test iteration :grinning_face_with_smiling_eyes:, instead the tests just bumped the seed to generating a new workspace key (0, 1, 2 …).
It will also be useful when considering creating workspaces as a way to share with others. You share the workspace, they pay for their writes. Well, that’s a huge space for developing ideas and features, so may not look like that at all in the end.

The rest

Outside of that basically just technical details, fixes etc.

10 Likes

Fantastic work.

I managed to get this set up with a vault enabled on one computer :tada:

On my second computer I’m trying to set up, unfortunately I get an error when ‘creating system manifest pad..’ on the first run of the daemon.

I’ve tried a few times, rebooted etc, but keep getting the same. The secret key is the same as what I used on the first machine which worked successfully.

Here’s what the error looks like:

Error: 

creating system manifest pad.. 
Error: Payment failure occurred during scratchpad creation. 

Caused by: 
    0: Cost error: Network(InsufficientPeers { got_peers: 0, expected_peers: 7, peers: [] }) 
    1: Network error: InsufficientPeers { got_peers: 0, expected_peers: 7, peers: [] } 
    2: Received 0 closest peers, expected at least 7 
PS C:\ryyn0\ryyn-windows-0.1.0-alpha.4> 

The wallet is funded, and I did the exact same procedure, on the same network… any idea what the issue might be, or what I could try to resolve it? There are plenty of funds on the wallet.

Edit: I had the same problem on the 2nd machine when trying with different settings.

2 Likes