:IF: Ryyn – out of sight, there when it matters. Quiet data backup

[oetyng]

Hi everyone, it’s been a while

I’ve been building something over the past couple of months called Ryyn.
It’s an app that quietly keeps your files backed up and in sync across devices. You can access, edit, and restore any version of your data from anywhere. The goal is that you shouldn’t even notice it running, it just does its job in the background.

Getting something that “just works” without nagging or getting in the way has taken some effort, and it’s coming together nicely. No apparent obstacles left.

How it works

There’s both a GUI and a CLI. You choose the folders you care about, and Ryyn handles the rest. Add or remove folders anytime—it’ll catch up.
You can share data with specific people, or make it public if you want.

Storage is set up in some new ways to keep costs low, by using the network more efficiently.

What’s next

A first release is on the horizon, but not quite ready yet. Still a few things to adjust and connect.

Code

A lot of time has gone into making certain modules of the backend solid. Some parts are built but not wired in yet, and will be added as we go along.
If you’re curious about the technical side there’s an early version on GitHub.

[riddim]

MR Oetyng! glad to see you again! wasn’t sure you will come back after you became a rather silent visitor of the forum!

[Nigel]

As soon as voting works for me, you’ve got support coming! I’m supporting all the OG and tech focused friends here.

[aatonnomicc]

Welcome home @oetyng

[happybeing]

This is the most exciting project so far for me. Partly from the goals and partly because Edward.

Backup and sync will be game changers but have to be done well and I can’t imagine anyone I’d trust more with this kind of task.

@oetyng I’m on mobile so have only looked at the directory structure for Rryn so don’t know how you are storing things.

I’m wondering about compatibility with other apps, and could this also be the basis for an Autonomi file system?

I am using public and private Archives (versioned using the dweb:: history::History data type), and plan to provide an rclone backend based on that (as well as publishing and viewing websites which dweb already supports using the same format).

rclone also does sync and creates an easy way for people to migrate from numerous cloud services to Autonomi, so having this compatible with Rryn would be very useful.

I use the same format as is created using ant file upload but with the addition of versioning using the dweb History generic data type.

It would create a lot of options if different apps use a common underlying format for this, including allowing data to be accessed for different purposes without you touching the local device.

Is that likely to be possible with Rryn or have you gone your own way?

Welcome back and good luck!

[oetyng]

Thank you guys!

Hey Mark. I agree these are important things. I have been thinking about it because my format is definitely unique in some ways, very different from ant file. That fact is not something I like particularly, but I didn’t really see what I wanted out there so I went for that, went for finding what made most sense to me. But it’s better for everyone if there is a shared format, much better. So I’m definitely prepared to change things to reach that. The format of storage is fairly well abstracted away (hasn’t been full focus on that though, so it can be better).

I’ll describe the major differences briefly:

Changes to files are made in two parts, metadata and content.

Big difference 1:
Metadata is cashed in a local db and is straight forward to compare.
The actual change though is saved in network as an event that is a part of the event stream of the backed up folder. (This event stream is what makes it possible to restore it to any state it ever had.)

Big difference 2:
Comparing the content is made by calculating the binary diff (actually using the rsync diff algo for that, so there can be similarities). The previous version of the content is cached on fs. If not there when access requested it is downloaded. So, the diff of the contents is what is saved on every change. Getting the file from network means base_content + diff_0 + diff_1 + .. diff_n.

Big difference 3:
Very small file changes are buffered in shards (Scratchpads.. this magic data type, so far), which are compacted into chunks when reaching threshold size. The address to a specific piece of small content (whether base or diff) is an address to a chunk and an offset, from where to read the length prefix and get the full range of bytes of that piece. (This is significantly more efficient for storage, but access efficiency depends on how sparse and random the future reads are.)

Not sure what would be possible for us all within the IF timeframe. I think we could all look at what particular solutions there are and see what makes sense and help each other out. I’ve seen it mentioned in discord I’ve just been so busy lately so I haven’t been able to follow up on it. I am about to go through all other repos soon to sanity check things etc., and I will be able to say more after that I think.

---

@happybeing

could this also be the basis for an Autonomi file system?

I think yes, conceptually there are parts here that could be a basis for an Autonomi file system, and there is also previous work on this. I’d love to look into that sometime after Ryyn.
The efficiency is very tightly connected to the data format, and there is still much to work on there. And Scratchpad mutability plays a part there as well.

@happybeing

Welcome back and good luck!

Thanks!

[happybeing]

I’m not at all surprised there are big differences thanks for explaining them.

The ant ‘standard’ may well not be the best but I think early on it’s good to try and support it while we figure things out.

I started with my own format too, but switched to Archives once they became useable for compatibility with ant and other apps. So dweb publish writes private Archives now, for efficiency, but viewing websites it will accept public or private, and the dweb lib and REST API make it easy to work with both. It could be extended to accept other formats such as public data stored by Rryn if Rryn needs it’s own format. If so, it would be handy to have a Rryn lib

I’m so pleased you are building again.

[Josh]

Super technical question here ? Is Ryyn a acronym for something?

[Josh]

Why are you exploding heading me @riddim .. is it obvious to everyone but me?

[riddim]

nooooo - because you’re way ahead of me - I was still trying to figure out how to pronounce it xD

[Josh]

@riddim

still trying to figure out how to pronounce it xD

Maybe @oetyng can clarify that too… I say it like Rhine

[Astroman]

I was saying it like Saving Private Ryyn

[scottefc86]

He’s going to have to clarify it I was saying rin

[Josh]

Well, he is smart.. its keeping his thread in the spotlight

[Southside]

It obviously stands for

RsYnc Yer NFTs n stuff

and should be pronounced “What a Spiffing App” - or something…

[Josh]

@Southside

It obviously

it is still not obvious to me how to pronounce his username

[Southside]

@Josh

how to pronounce his username

Edward of the yeØ-Tængg Clan <—justlikethat – not like that, like thät

[riddim]

that Southside guy is so smart

[Southside]

If I was so smart I probably wouldn’t wholeheartedly agree with you…

But I do

[oetyng]

Guys, you hilarious
I have a pronunciation in my mind (for the nick as well). But with the above I’m not sure I should “lock it in”..

The acronym-question tough, @Josh, I think it could actually be “RsYnc Yer NFTs n stuff” (how did you figure that out @Southside?).