Update 26 January, 2023

[oetyng]

@tfa

I would say that node age increases logarithmically (as a function of the number of events).

Yes. Correct.
What that probably meant to say was the time passed for each increment is not linear, but exponential.
Exp and log are the inverses of each other so can be formulated in both ways. But that formulation there would need an update.

[19eddyjohn75]

Thx 4 the update Maidsafe devs

Incredible work super ants

Great to read about experimentation with tinier virtual machines and small nodes.

@maidsafe

For example, we’d want the most stable nodes (say, age 10+) to be responsible for primary data storage. These nodes look after the data and give it up to a client on request. They are not likely to churn any time soon.

I wonder if these stable nodes could potentially also tap into different storage medium for the same data? If my hardrive holding the data corrupts, could the node serve the same data stored on my dvd or usb or does a churn event change the data?

Keep hacking super ants

[happybeing]

@19eddyjohn75

does a churn event change the data?

Yes, but older nodes will churn less.

[19eddyjohn75]

@happybeing

Yes, but older nodes will churn less.

In that sense data is sand changing by the tides of churn, but wouldn’t static data (a rock) also be useful for quick network memory recalls/recovery? To make more sense myself, imagine our 50Gb testnet. If the nodes had six 8.5 GB dvd to tap into they could recover the 50GB testnet quicker probably?

[happybeing]

I’m not sure I understand exactly but I’ll answer yes, more centralised can be quicker.

[jlpell]

@maidsafe

New nodes are not reliable and can go offline rapidly, meaning lots of data movement and a headache for the elders who have to manage it.

Wouldn’t this be managed in a simple way if the allowed storage capacity of a node was a simple function of its age?

@maidsafe

Primary and secondary storage

We’re looking at concepts around a stable set of nodes (more on that to come down the line). But one idea this gives us is separating nodes into two storage tiers based upon age and (therefor) likelihood of churning, and thus giving them different duties.

Wouldn’t this distinction be a good way to introduce the concept of “archive nodes” vs. “standard nodes”? I’ll admit that “unstable”, “testing”, and “stable” is also a nice paradigm here, but then we also have “child, adult, elder”. It would seem appropriate to stick with the same terminology and stop changing the analogies here.

@maidsafe

For example, we’d want the most stable nodes (say, age 10+) to be responsible for primary data storage. These nodes look after the data and give it up to a client on request. They are not likely to churn any time soon.

The challenge here comes when a stable node does not grow it’s storage capacity over time, even though it has always maintained a good connection to the network. A 1TB to 10TB hdd is common today, next decade we are talking another one to two orders of magnitude in growth. Nodes should be expected to continuously offer more storage capacity to the network as they age, imo. Another way to think about it might be to not let them continue aging if they cannot increase their storage capacity to the amount required for the next age level.

[DeusNexus]

Great work MaidSafe!

[happybeing]

I’m not sure if you are, but don’t forget that we wish to favour universality and use of existing resources (ie people’s existing commodity devices). All required or favourable characteristics for nodes need to consider their impact on that, as well as factors such as performance and evolution of the network.

So for example, we might want to avoid criteria that drive replacement of otherwise functional hardware before time. By not setting unnecessarily strict availability, capacity or performance criteria which make it impractical for regular people to run nodes on existing devices.

Over time I can see this being addressed by having more roles for nodes, so both high performance and a wide variety of legacy hardware have their place, but early on I think the latter should be a fairly high priority for various reasons.

[DeusNexus]

Would it be a good idea to make a metric of overall system performance capabilities and use that to attribute node to certain tiers? E.g.:

Tier-1: High Performance, High Storage, High Through-put, Low Latency, Minimal Downtime

Tier-2: Good Performance, High Storage, Good Through-put, Low-Moderate Latency, Low Downtime

Tier-3: Moderate Performance, Medium Storage, Acceptable Through-put, Moderate Latency, Sporadic/Occasional Downtime

Tier-4: Moderate to Low Performance, Medium/Small Storage, Acceptable Through-put, Latency >5-10s, Occasional Downtime

Tier-5: Low Performance, Small Storage, Low Through-put, High Latency, Frequent Downtime.

Based on tier it would be wise to prioritize hot data on higher ones and something more of a data lake or archive on lower end nodes that come with slower access times.
You could also determine based on a certain metric of how often node goes down how many Tier-X nodes you need to get a overall consistent uptime if anyone of those at the same time go down.

I’m just thinking out loud, would seem beneficial to have prior knowledge of distribution to inductively get a more optimize network state than without. Solely using uptime imo is too little info, including storage capacity, throughput also stimulate nodes to get better and reward them for being better network contributors.

[Blindsite2k]

@maidsafe

Nodes outside of such a stable set, those that are still working on increasing their node age, hold extra copies of data (secondary storage). In doing so, they provide redundancy to support the stable set.

Their behaviour in handling this data is also used to evaluate their quality in the usual way. But since they only hold extra copies they do not need to be tracked so closely by the elders, and can fail without causing serious problems to the network or requiring mass data migration.

So younger nodes act as backups for older nodes that hold the primary copies. Okay so what happens if the primary nodes go offline and there aren’t enough younger nodes to pick up the slack? Is there a plan C in place to ensure data retention?

Also if nodes have to achieve certain qualifiers or events in order to achieve node age then a chart of some kind depicting these events might be very useful.

[peca]

@Blindsite2k

Okay so what happens if the primary nodes go offline and there aren’t enough younger nodes to pick up the slack?

I gues young nodes get promoted and network allows new nodes to join and become young nodes.

[jlpell]

@happybeing

By not setting unnecessarily strict availability, capacity or performance criteria which make it impractical for regular people to run nodes on existing devices.

Over time I can see this being addressed by having more roles for nodes, so both high performance and a wide variety of legacy hardware have their place, but early on I think the latter should be a fairly high priority for various reasons.

I think that’s why tying all the storage/performance requirements to node age helps accommodate for this. For example, legacy devices might never grow beyond a certain age because they don’t offer enough connection stability, or storage capacity. As technology improves, nodes that don’t improve/upgrade could potentially lose age. However, the young nodes would still store extra copies and help out to whatever extent they can.

[happybeing]

It’s more complicated, which is really my point.

For example, I think you may be taking too narrow a view of the ‘value’ of universality, of having as many participants as possible rather than rewarding an elite who, if you take a narrow view of performance, seem to contribute more.

This is a typical human trait. We can’t comprehend everything at once so we focus in and simplify until what we see is of manageable complexity. But in doing so we risk ignoring other important factors.

Perhaps this is why our civilisation has reached this point, where we seem to be in a death spiral.

[Southside]

you are a right cheerful bugger tonight , Mark

[jlpell]

@happybeing

For example, I think you may be taking too narrow a view of the ‘value’ of universality, of having as many participants as possible rather than rewarding an elite who, if you take a narrow view of performance, seem to contribute more.

No, I think you are missing the point I made. I have detailed it more in past comments.
I was pointing out that the node age mechanism can naturally accommodate your concerns to a rather fine level of discretization. This keeps things simple, rather than naming special classes of nodes like a "stable " subset, “unstable” subset, tiered groupings or what not.

[Dimitar]

Thank you for the heavy work team MaidSafe! I add the translations in the first post

Safe Network новини 🇧🇬 26.1.2023

Обновление Safe Network 🇷🇺 26 января 2023 г

Safe Network Entwickler Update 🇩🇪 26. Januar 2023

Actualización de Safe Network Dev 🇪🇸 26 de enero de 2023

Actualités du développement Safe 🇫🇷 26 janvier 2022

---

Privacy. Security. Freedom

[happybeing]

It’s a subtle point we’re differing on I think.

I’m not sure if that does address my concerns because I think it is inherently elitist - unless combined with leveling mechanisms. You don’t mention that, so I’m still tending to believe we have differing understanding of the potential value of universality.

I don’t know that I’m right. I have a hunch and it is strong in me, a sign which I’ve learned to trust.

I think we should be careful and thoughtful on this, and I’m concerned that few people will get it in the way that I’m sensing may be important. Partly because I don’t see the world in the individualistic way that many do. I think one may need a certain political or philosophical perspective to share or even see my concern. Consequently I think I’ll have a hard time making this case, even when (if) I can figure out whether I have a case at all!

I do suspect David will be on board though (or rather ahead of me on this). And I may just be mistaken.

[dirvine]

@happybeing

It’s a subtle point we’re differing on I think.

It is. Node age covers a lot, but mainly

1. Trust the node operates cleanly
2. Indicates the nodes reliability

As age grows then so do these 2 and that is great. However when nodes are younger we can say they have not earned much trust and they are likely unstable.

So IMO asking these nodes to do work that will mean their instability will cause the network to do a lot of work is really a bad choice.

A granular approach to age would allow more granular decisions etc. and the stable_set does that for older nodes (no age, just ordered by first seen and in that way totally granular with no upper limit).

Node age for younger nodes though is our sybil defence. What that means is we move them around and constantly measure them until they have invested so much to be in the network that we let them stabilise in the stable_set.

While they are growing they are moving and while they are moving they are causing data relocation and many more “node not found” errors etc. So considering them as unreliable and finding work that does not rely on their reliability allows the network to maintain sybil defence but also offer a much more stable and reliable network to users.

I hope that helps. (btw totally granular infants are a really hard problem and the sledgehammer age thing really does seem to work well there).

[happybeing]

That all makes sense and may well be what jlpell is thinking etc.

I can’t really articulate what is brewing in me atm, but I’d say the difference is a reductionist/utilitarian analysis versus something globally cohesive. A gestalt perhaps. For example, inclusivity and diversity improving meta characteristics that are hard to see when looking in the details etc.