Ryyn (massive multi-device virtual FS)

Yes, I think what’s mentioned with the simplicity and familiarity, ability to fiddle with things quickly, are very desirable properties.

The whole ignore thing is a very interesting side of the backup system when looking closer at it. It’s essential in my mind, it’s the black to white, and as such it deserves very good design and ergonomics.

We’ll see what comes first and when.

I think I wouldn’t use only a new syntax, if that was all. If there are useful capabilities missing, that could make people’s life easier, there is a good reason to go all in with syntax + semantics and make it as good as reasonably could be (since not tied to legacy), because I think changing semantics of standard approach would be confusing. I’m thinking that thing should work as it works, with no surprises.

Fully possible. This is a whole little world to think about :slight_smile: It will be interesting when I come to that. Don’t hold back if you have ideas.

6 Likes

I’m so glad you took this on Edward. The reason I decided to piggy back on rclone was to avoid having to create a backup app - because I knew it was very hard to do well - and of course to get all those extras. Good luck with this, I can’t wait to be using it.

10 Likes

I do think a very simple ignore/include syntax will be great for non/less-technical people where you can just say include/exclude directories and ones like “downloads”, “documents” etc. Also wildcards for files like *.txt

This would be a set of rules to feed a more flexible system that more experienced people can directly setup.

One would assume that this would be possible to become an App that the general public can & will use and it would be good to have a simplistic rule set that can either be directly used or pre-processed into the more flexible rule set which could then be modified if one wants to. And of course the user could just modify the actual rule set

I realise this would be a todo item that gets done before a release for the general public.

5 Likes

Yes I agree @neo.

The main net release in August required for IF qualification can actually be very slimmed down and still qualify. So, when the time comes for that, it’s likely that just a minimum version is actually released for main net, while the more advanced versions are still worked on and available in github. I think that is wise, to make sure the released version is 100% stable.

And for that version, ignore files are unlikely.

But I think they have a fairly high priority after that. So, yeah, a todo item nonetheless!

7 Likes

Very much so. And wise to do of course.

I thought it was mid July (July 17th) or has this changed. I mention it in case you thought August and potentially miss the deadline.

2 Likes

Ah, that’s right, July. August was final decisions. :+1:

5 Likes

I mentioned elsewhere that I do a part-time tech support type job now since a few months - while being very new to the whole thing, there have already been several times where I’ve been attending to some small and often seemingly inexplicable issue with Google Workspace regarding accounts getting mixed up, or 2FA set up wrong, or integrating a new domain and running in to snags, or the labyrinth of help docs sending you in a loop, or phantom Workspace payments for an account that no-one remembers setting up, or…

I did not expect that the answer to all my problems would be - Ryyn, from @oetyng :smiley: that’s what I get for not having the time to frequent the forum these days, I’m missing all the fun. This looks totally amazing! I wholeheartedly concur that if you can get the simple things working straightforwardly and reliably, it’ll be a major piece of infrastructure. Incredible

12 Likes

Hey, that’s nice to hear :blush:

Yes, those sorts of issues always amaze me when they happen.

So, regarding solving problems. Sharing files is exactly what Ryyn does, and versions are a fundamental part of the design, it comes with the design really.

But when it comes to real time collaboration of a document, the technical solution is different. It can definitely be on the todo list, but I have not even begun to consider it yet. It’s not inherently more complicated than anything else here (I’ve already planned for immediate p2p sync as a side channel to the durable storage), just a question of stated scope and developer bandwidth.

It’s great to hear about what problems people see this solving, it’s valuable to have in mind in all parts of the development even if the specific feature isn’t currently being developed.

9 Likes

I’m learning from your comment here that somewhere Google Workspace must advertise and even include options for real time collaboration :grinning_face_with_smiling_eyes: If the business I’ve been helping out is anything to go by, the reality is much simpler.

What it’s used for - at least in this company, which is not a tech company by any means - is one person can open something on their computer, work on it, save it, and move on to something else. Then later another person can log in from their computer and work on the same document in the state the first person left it in, save it, and move on to something else.

So there has been a voice in my head going - how much are we paying Google Workspace (really Google Drive is all that’s being used, and Gmail too sort of, as far as I can tell) for this feature? And what exactly is the feature and what does it rely on - storing a file somewhere that two users can access? We’ve a gargantuan middle-man charging us a hefty fee (which they keep jacking up recently, btw) in order to - store files on a server for us?

So, maybe you see my point more clearly there. I think they literally just need to be able to store files somewhere with an absolute minimum amount of “tech-speak” - I maybe should have already been saying to myself: perhaps if we had some kind of massive array of internet disks that any citizen could easily access :thinking:

6 Likes

Hi @oetyng! I hope you had a nice midsommar!

How is it going with Ryyn at the moment?

1 Like

Hey Toivo! Thank you I hope you had as well! This part of summer is my favorite in general. But there have been quite a few rainy days as well, and when working on code those are the best because you feel like you’re not missing anything :joy:.

It’s good that you ask, it’s time for an update!

A lot has happened, a lot of work. And even though I switched from spiking all over the application (driving a development idea far, deep down, to evaluate its feasibility) to picking some of those cherries out and gluing it together with MVP-grade solutions, there have still been things that have needed more definitive clarity. For example, the format of data being stored to network.

So, for that I finished a hybrid solution between binary diff and content defined chunking yesterday, and am integrating that into upload jobs. It resulted in a much simpler solution than what I was working with before. Time is becoming short for the IF, and it was a few sunny days worth of work, but it cleared up so much that I felt it was really worth it.

The pipeline from initial scan of the folder and continuous changes in it to upload is more or less done with that integrated. Most other parts are in place or to be placed there with some glue.

I need to cut and paste some things from the spikes, do some more in the local storage part, the events, error handling and such things, and then a crucial MVP-grade feature is to restore a particular file/folder or clone a folder. That is what the app is in the end meant to do, so it needs to be in place.

So, that’s basically what should be in the MVP:

  1. Setup with your ETH key,
  2. select a folder to watch,
  3. see that all content is backed up,
  4. see that further changes are continuously backed up,
  5. and then finally be able to restore from the backup (in one way or another, maybe just restoring a file to previous version can do for this stage).

The time frame here isn’t ideal, it would have been better with more real world testing by everyone before final date, but it will have to do!

We’ll see how it goes :sweat_smile:

11 Likes

For me it’s August. A combination of dark evenings and warmth is such a rare magic in Finland.

But great update, good to hear about all the progress!

7 Likes

Okay. Here it is now, the very slimmed down MVP. Or maybe I should say demo, not sure. Ready for main net anyway.

Have a look while I work on getting some basic user docs: GitHub - oetyng/ryyn
If you are digging in to the code, note that there are plenty of out of date comments, ignore those and look at the actual code instead

This will most likely only work on windows for now.

12 Likes

Just wanting to mention that on hawaii it’s the 21st of July in this second =D

(and there’s even places where it’s 21st for another 5+ hours)

8 Likes

Basic docs in place: oetyng/ryyn

8 Likes

I’ve added a CI workflow and an automated release workflow (CD) for alpha (prerelease) and stable now.

  • CI: on every push to main or alpha, it runs run fmt, clippy, units and e2e tests and a release-mode build.
  • CD:
    • Alpha prereleases auto-increment only the -alpha.X tag (e.g. v0.1.0-alpha.1v0.1.0-alpha.2).
    • Stable releases bump the semver per conventional commits.

A bit unnecessary with the stable branch at this stage, but anyway. I think I’ll eventually use nightly and alpha.
There’s a (temporary) prerelease you can check out here:
github.com/oetyng/ryyn/releases/tag/v0.1.0

6 Likes

In case you want to migrate to Codeberg I’m making good progress on cross platform Rust build and release using their CI.

3 Likes

I saw that and I am curious! I will to look into it.

3 Likes

So first time user

I read the github readme and tutorial.

Hit a minor roadblock here

  • i downloaded the exe file as instructed
  • started following the tutorial
  • 1.1B is the first step and ran ryyn init id secret-key pass
    • error immediately comes up
    • error: RUN_ENV required
    • with missing var RYYN_ENV

What is this environment variable for and what is it supposed to be set to? Could not find it mentioned

I could try and set it to nothing, but without know what it is supposed to be that might cause issues later on.

help needed on this

6 Likes

Hi neo,

Ah, clear mistake on my part. This error is from using the tutorial with the first release, I should have made that clear in the tutorial.

Try with the latest: ryyn - v0.1.0-alpha.1

(The env var is not used anymore. It was to discern between live and different test setups, but there is a separate `test-init` cmd now instead.)

7 Likes