Sorry but eternity don’t exist. I will die, you will die and the earth will die so your “one-time payment divided by eternity is zero” is a joke.
The “time-to-live, rental model” is hard to implement. You must create a lot of metadata, insert time in the network, control it, churn it and secure it. You create an attack vector, making the network more insecure, and, in the end you can spend more computational resources with this model that only storing old data.
Viv, in this post, show us the path, pay by recycling. That’s the way, the SAFE way.
With a timestamp on the data, it is trivial to implement. Forget the word rental: once a data chunk passes its expiry period, it is deleted. if you want it to survive then you refresh it by re-uploading.
Not entirely trivial, how for instance do we get around some of this (then realise these are all connected, so fix one may break another etc.)
Synchronize timestamps across a serverless network (no nts)
Have timestamps somehow updated
Store timestamps
Churn timestamps between nodes
What timestamp to use for massively shared data (i.e. the chunk of a popular file)
Decorate every chunk with owner identifiable field and then handle the issue where the decoration is larger than the chunk (mass inefficiency) and for data with say a million chunks etc. …
Have the users account tie users to chunks for deletion (securely of course)
None of that is hand waving, these are real issues and only the tip of the iceberg of edge effects.
Not to say this is not something discussed as well as retiring data when accounts die off etc. Understanding the complexity and side effects is important, particularly creating a link between id’s and data. It’s easy to look microscopically and say use a hash or sqrl obfuscation type techniques (ie.e. have users create specific key pairs per chunk and derive [seed] that from the hash of the chunk and their private key for example).
TL;DR Side effects and unwanted consequences (zero day exploits aboud) make much of these things non trivial. The level of thought is important and many trivial additions will kill security and efficiency quickly.
My statement “with a timestamp on the data” assumes a working timestamp system, and given that, then the expiring of data would be trivial. Your objections are saying that getting a working timestamp system is non-trivial if no time servers are allowed (number 1. is the parent of most of the rest). Actually, with no time servers, I’d say it is impossible, not just non-trivial.
So, given the Scylla and Charybdis* that SAFEnet would end up between - the economic impossibility of fulfilling the promise of pay-once-and-store-forever, and the impossibility of keeping time without a reliable clock, some assumption/premise must be given up.
You need to read what I said much closer and not defensively (the most important part of Engineering/physics etc. is search for truth not self satisfaction). Even with 1 working how do you store all those with every data and not expose users. All the rest of the issues stand and as I said this is only the tip of the iceberg of edge effects.
The point I as trying (and possibly failing) to make is seemingly trivial changes can have dramatic side effects at a distance, in security and also complexity that are non-obvious.
It’s scary but also what makes projects like this fascinating for curious minds, which you seem to have in abundance, so great. It’s just that you need to go further down the rabbit hole and realise trivial changes are seldom that and more so when a network, such as this or any other are nowhere near perfect in terms of primitives and structure at a very deep level. So simplicity of change as much as possible where possible is great, but needs checked many times for containment.
Hope that helps.
[EDIT, I should add there is a few answers to the problem you are predicting and some of them very nice indeed (if there is this problem that is), none ready for RFC even yet though]
Apart from the technical issues I dispute your premise because the lifetime of this network is finite. In fact it might only be a decade or two before the next leap, or crash into Mad Max, whichever occurs.
I don’t know if the network will be finite or immortal. I was simply taking the assumption behind “pay once and store forever” and following it through.
I don’t believe in the technological singularity, because after every past leap it seems obvious. Before Gutenberg, Watt’s steam engine, the telegraph, electricity, personal computers, etc, such developments would have seemed a scary break with the past, and afterwards, life went on, only a bit better than before.
And Mad Max would have some sort of network. If just one Linux distro DVD survives (remember, the GPL stipulates the inclusion of source code) then some bright twelve year old can reboot the Internet.
I am not so sure it is an economic issue. People rarely delete data, as there is little need to - it is cheap. It is the accessing of the data that is expensive (CPU, memory, network bandwidth, etc).
Data which is stored forever but never accessed is little burden nor reward. Data which is retrieved frequently is a burden, but farmers are rewarded. If it was the other way around, I would share your concerns, but it isn’t.
OK (and for @neo) where is the cache of the early days of Usenet? If it is in some offline archive somewhere, how can I access it? And anyone who has used the so-called Internet Archive can attest that it is very incomplete. The claim that the cheapness of storage will outpace the firehose of data is not supported by such past and current examples.
I look forward living in network with no time. I rather not see any dev putting their energy to build a time system into this network. We don’t need it. Time is a relative - an illusion for us to perceive. Time is a by-production from the gravitification, and 3D space.
Albert Einstein concluded in his later years that the past, present, and future all exist simultaneously.
Since there exists in this four dimensional structure [space-time] no longer any sections which represent "now"objectively, the concepts of happening and becoming are indeed not completely suspended, but yet complicated. It appears therefore more natural to think of physical reality as a four dimensional existence, instead of, as hitherto, the evolution of a three dimensional existence.
I explain how fourth dimensional spatial directions travel through a series of independent three dimensional block-like spaces, which in science we call states, but they can also be thought of simply as patterns. Hawking has already proposed that imaginary time can be found at right angles to ordinary time. I further explain that it is possible in an objective way to understand the universe to be like a book or a movie film. Each moment is a separate universe just like each frame of a movie or page of a book is separate. Yet those separate states simultaneously form the larger whole of the movie or the book. Seeing each moment as a continually existing place sheds light on why particles would then travel as a quantum wave, rather than linearly from point a to point b. This is explained better elsewhere, but if each moment of ordinary time is a solid, static, “block of now”, or field of space, then time each new moment is a distinctly different universe. What we call time is a spatial direction that travels through many static three dimensional universes.
In such a model, what we call time is created purely out of space. Special directions in space travel through each static three dimensional space, therein producing a new realm of space beyond three dimensions, which we call time. The interesting quality this produces, is how the
inhabitants of this fourth dimension of space travel a linear path from past to future, but the surrounding environment of each path is shifting from one pattern to the next. This sends particles from one position in four dimensional space to the next without moving linearly. As a result, each individual observer in the fourth dimension experiences a continuous linear time, even though everything in their immediate environment is moving sequentially from place to place. Hence each temporal environment of four dimensional space is constructed relative to each independent observer.
The Relationship between Gravity and Time
Gravity is said (by those who understand this better than I do) to be caused by a distortion or a warping of space. Gravity is not a force that keeps objects in orbit, but that space itself is bent by objects with massand that an object orbits because it is really following a straight line through bent space. The greater the mass, the more space is bent.
But if space is bent, it has to be bent in at least a fourth dimension, which is likely to be that space is bent in time. If mass is really a thickness in quantum time, then the faster an object moves, and the more apparent mass it has because of that motion, the stronger the gravitational field. Thus if gravity is bent space, as Einstein suggests, and space is bent in time, then as one becomes thicker in time the bending would increase proportional to the thickness. 10 times the thickness becomes 10 times the bending and 10 times the gravity.
However, if another object were moving at the same speed in parallel then it’s is moving forward through time at the same rate and has the same thickness in quantum time. Since both objects have the same quantum thickness, then they neither see any increase in either mass or gravity, but because of time dilation, the have near infinite speed. So from their point of reference, they see only an increase in kinetic energy. The amount of warping of space in time is less because the effect of the warping of space is perceived in terms of warping relative to the thickness of one’s quantum moment.
If the thickness of one’s own quantum moment becomes the yardstick for which one perceives the bending of space in time, the effects of mass and gravity are inversely proportional to one’s quantum thickness. If this is so, then observers moving at different speeds will measure mass and gravity differently based on their quantum time thickness. And, if both mass and gravity are relative to quantum time, that would preserve the laws of physics at all speeds so that no one could calculate their own speed through absolute space by measuring their own change in mass and gravity. Their rate of motion through time cancels out their increase in mass and gravity due to the increase in quantum time thickness. What this means is that there is a time component to both mass and gravity.
Suppose two space ships were traveling together at a speed so fast that their aging process slowed by a factor of two. They would be moving forward in time twice as fast as a nonmoving observer. Now suppose one of the space ships turned on a 100 watt light bulb. To the other ship it would appear to radiate 100 watts. But if the bulb is aging half as fast, then it is putting out energy at half that rate to a nonmoving observer and appears as if it were a 50 watt bulb, do to time dilation. Remember that power is the rate that energy is dissipated over time, so if you double time, you cut power in half. Power has a time component. Similarly, at those high speeds it would appear to a nonmoving observer that both mass and gravity increase by a factor of two. But to the moving observer both mass and gravity stay constant. Like the light bulb, it would appear that gravity has a time component in that if you change the time reference, then you change gravitational attraction.
Even if people rarely delete data, everyone would still have to replace their storage devices eventually. How will data storage be divided amongst the farmers? If a farmer had to replace a faulty drive on which they hosted a vault that had rarely accessed data and hence wasn’t very profitable, what would motivate them to continue running the same vault when they replaced their drive as opposed to starting a fresh one? Or will the network pass on that old data to other farmers and give them something new?
Considering I never implied there was, I am unsure what you are asking of me.
To try and answer some of that for you - the early days is still mostly in University archives, and I GUESS most NNTP servers don’t have it online. If you were referring to more recent times of binary files (90’s & 00’s) stored on NNTP/Usenet then that is obviously mostly not on any usenet server now. Although many repost the old stuff and with some servers never deleting now it is being restored. Still the whole of binary files (de-duplicated) of Usenet from the 90’s could likely be stored in some very large personal NASes now-a-days.
There are some Usenet/NNTP servers out there with text messages (original use of NNTP) from the start of usenet. Requires so little storage and they kept them. Some university NNTP servers have been running since the start and never deleted messages from the groups they keep, mainly edu groups.
cheapness is in the eyes of the beholder. I never claimed all storage (esp V.large) was cheap to buy/build. But history shows what was expensive in storage 5 years ago is cheap now
Tell that to the likes of the big NNTP server companies, tell that to the likes of Google that is storing ever so much data, tell that to the hoard of hosting companies that now provide unlimited storage ( eg dreamhosting a cheaper hosting co)
You want to store data there is an abundance of drives to store it on. And the technology around write once systems is increasing archive storage even faster now than in previous years. Of course we are talking of actual data. Then again NSA is hovering up even the packets traveling through the boarder routers of the USA (and some internal routers). So they are not just storing all the information, but the headers and meta data around the packets (time, router location etc)
Data is increasing at an amazing rate and so is storage, especially archive storage. I have been watching this for some 45 years now.
But granted if you wanted to generate garbage data then you could fill every thing up, but we don’t see that.
And granted that I can foresee another explosion in data generation rate that could exceed data storage increase rate, but that explosion hasn’t happened yet and until it does we work with what we got. A lot of information increases occur after there is storage space for it. The reason being that money will not be allocated to projects that require large data storage unless there is a way to store it.
Hence the reason for the new model of cycling your vault so that you don’t end up with a full vault that after a year has a high percentage of “stale” data. Spread the “love”, restart your vault every so often so that you spread the “stale” chunks amongst numerous vaults, and you get a new batch of chunks being stored (some stale, some fresh) over time.
Only those who want to run vaults 24x7x365 that are large enough will be considered archive vaults and these people are willing to run vaults that has only a small percentage of fresh chunks and high percentage of “stale” chunks waiting for that moment (that may never come) that someone somewhere wants that old piece of data. Maybe a granny accessing that old photo she took when she was 10 years younger to show her grandkids on their visit.
Some of the archive vaults have been suggested to be write once storage that is far cheaper and some of the recent advancements may see many PBytes per unit at a reasonable price
Data is chopped up and shared out, so everyone is likely to get a mix of data. Some will be dormant, other will be active.
As farmers can’t pick and choose what data they would like (new or churn), they will always have a mixture.
Ofc, the this will all be factored into storage cost. We won’t know for sure until the network has Safecoin, but I suspect long term storage of static data will not have a large bearing on that. I do think that short term storage of any content may be relatively high though, as it is going to be priced at roughly what it costs farmers over the data’s lifetime.
Without revealing who is storing what and a way to record time, it is hard to see how individuals can have a lower cost, variable term, economic model. It may come down to the cost of privacy and whether it can be bypassed for some data types.
Actually after the network has been running a while and noting that newly stored data is statistically going to be accessed more often than 1 year old data. (Just look at your own habits.) Then we can postulate that if I run a vault for 1 year non stop and it was filled after 2 months then that data will be accessed a lot less around the end of a year than for someone who has a vault that was running for 2 months only.
I think this contains assumptions that require deeper analysis @neo. For example, aren’t you assuming that am empty vault will receive a substantially higher amount of fresh data compared to receiving old stale data due to churn?
Fresh data will if course go to vaults with space, but those which are deemed more reliable will fill up quicker, so wiping a vault may have a cost in terms of “rank”, and in any case much of the new data it acquires will be old stale data moving around due to churn.
I’m not saying you are wrong, just unlikely to be right given that there are many factors involved and the algorithms are both undefined at this point, and going to evolve over time.
You would expect a random selection of fresh/older data. But statistically a certain amount will be new.
But if your vault is on continuously and has already filled up and its 10 months after filling then all its chunks are 10 months or more old. No testing for that, just logic.
This has been refuted a dozen times. That is the old method when vaults were expected to remain on. BUT now vaults are to be non-persistent (for security) then when you restart your vault it is given a new ID and its rank is flawless until it proves otherwise.
This makes me wonder how the network react when a new node connects with an ID that is closer then yours is for chunks that you are already storing in your vault. I would expect these chunks to be moved to the new closest ID. if that’s the case then there will always be some shuffling around of chunks as nodes come and go.
I’m also curious to know how the network react when a vault is filled up. Where will go the chunks that were suppose to be found in your vault?
given that there are many factors involved and the algorithms are both undefined at this point, and going to evolve over time.