Thanks, glad I could help. I’ll let others reply for the specific, I’m pretty out of the loop with the API and the details of the inner working.Check out the dev forum if you haven’t already.
I understand that. I expect that the Web will be used for quite a while for links discovery on Safe. But you can bet the people working on Safe can’t wait for the network to be live so they can start working on more cool features. Meanwhile, John will have to wait a bit more, sorry John.
Indexing was discussed a bit here. In my opinion it is not feasible to do what you are looking for on the base layer of SAFE network. Think about it…everything happens client side. The site would be unusable if the user has to download the whole content index of the site every time they visit it. Or even if they just have to do it the first time they visit and update the index on subsequent visits, I don’t see that working well. If you tried Augur in its early days, one of the main complaints from users was the initial sync time. I don’t know how big these indexes are for YouTube, but given the volume of submissions they have, it must be huge. An app of this scale might need to think of being a next layer on the network, like how lightning network is on Bitcoin. I posted here about an indexing network for Ethereum. It might be worth looking into how that works.
@warz, I was about to say the same, that solution would be completely unscalable.
I saw @drehb that you mention MD capacity in your post from way back.
I think my topics on indexed and searchable dbs can add some information. Like this post DataStore over AppendableData design - #20 by oetyng and following contents.
Or the OP in that topic.
There’s also a repo for the SAFE.DataStore that has rudimentary indexing and searching.
However… SQL-like querying of data will be available with SPARQL over the RDF-emulations that MaidSafe implement.
I am not well versed in the RDF world, but it would still seem to me that it maybe is not the silver bullet for storage (since basically no db type is, all dbs suck - at some stuff), and eventually we might want to some other kind of emulation/implementation as well.
So, as for building up indexes, that would be the first best bet anyway; available in the core, supports SQL like queries.
As long as the “website” remains decentralized, it must per definition be possible.
If you mean with “run” to moderate or control with certain exclusivity or privileges, I would not call it decentralized and you would be responsible.
I think that the base layer of SAFE network is sufficient. Perpetual immutable data storage and global access is sufficient, and for anonymity anonymous PUT and GET would be nice. And SAFE Network will be distributed and scalable unlike Bitcoin and Ethereum.
As long as information is discoverable, we should be fine without any kind of master index. People will build indexes, some may be automated, but there does not need to be a monopoly on indexing.
I think it is just a matter of innovation, and getting rid of old ways of thinking.
There are three independent things
A site does not have to contain an index of content, that could be provided by the user.
If I create a video app, I don’t have to provide any default index. The app could be an open source project and completely independent of any index.
I could set up a site with this code and hardcode it to a particular index in which case I’d be responsible.
If I instead set up a site where the user has to type in the address of an index. I have no responsibility.
The app may even support a list of indexes to search and a list of blacklists of content to skip, such as spam. The blacklists are essentially moderation, but anyone can make them and a user can basically choose which moderators, if any, they want.
The software is now independent from the data and the indexes of the data, so the question is then reduced to whether an index can kept without anyone being responsible for the content of the index.
I think it may be possible to achieve this kind of indexes with AppendableData.
I thought this was a good analogy. A safe app / site can be thought of as this. Web browsers typically don’t block web sites and aren’t required to.
Whether you’ve created a pastebin clone, photo bulletin board, youtube-like clone, etc - you didn’t necessarily create a moderation system / delete button. Just like your browser vendor may not have. Without further development labor and publishing, you have no way to comply with the request.
There was a news story and bit of controversy some time ago with Apple being at odds with the U.S. government because they refused to build/sign a tool to hack their own operating system (San Bernadino terrorist’s iphone). Apple made the legal / constitutional argument that the government has no right to compel labor / speech. Similarly, you could refuse to develop/publish “delete” mechanisms (e.g. block lists). If anyone asks you to take down content, you reply “the software doesn’t have a way for me to censor content uploaded by users”. If they ask you to build one, you can decline like Apple did, or name an outrageous price they are unlikely to pay (e.g. “Tell you what - I’ll do it for 1 trillion dollars. Payment in advance, no refunds.”). That may not work if the adversary is willing to apply pressure anyway (e.g. torture you).
You can also have the consolation that users are still able to use the original tool (which is incapable of censoring the content). Torturing you would have limited usefulness since your code is open source and undeletable by the SAFE network (users can run the last version you published prior to being compromised).
In the case of Apple, it was reported that the government eventually got in by hiring a hacker, and basically dropped their argument with Apple. The thing is, law can always change in unpredictable ways. People in government can even pass illegal laws (violate their country’s own constitution). The U.S. for some time treated encryption software like military munitions and technically still does (the first and second amendment did not stop the government from restricting encryption software).
That said, in the US, you can also make an “ex post facto” argument. For example, say you published a tool, and the government later passed legislation that made your tool “illegal” for whatever arbitrary reason. You are unlikely to be prosecuted (courts are supposed to throw out the case), since it wasn’t illegal at the time you did it.
A lot of this is theoretical and down the road, since there is no decentralized permissionless database implementation for the SAFE network (additional development of SAFE is required). However such a database app is possible today AFAIK with other projects. For example, with a smart contract on EOS or Etherum. So I was considering it from that angle - a relational database that anyone can write to (but not delete).