Hi all,
We are a startup team from RMIT University in Melbourne. Currently the team are building a matrimonial APP named Together2Life. We have already done some research on graphic database (for building relationship between people) and MaidSafe. We have found that if we build a database and put it on SAFENetwork, the data cannot be searched because it is all distributed, and we cannot reach the matchmaking function. So we think we are going to build a semi-distributed database. This means we’ll putting some basic information into centralised database, and put the other information into distributed database. This is our main focus at the moment, but we find it hard to find the solution, so is there anyone who can guide us on that please?
Well, at the current stage (and probably in the first publicly available version) safe doesn’t provide much more than some simple file-system-style interface. One key feature of that is that the “distribution” would be transparent and you’d just directly interact with the file in question. So, theoretically, you could just upload the actual file the database creates/uses into the network and use it from everyone you like. However, traditionally databases rely on not reading/writing the same file through independent processes and most indexing stuff would work in some form in memory.
If that file was available “publicly” on the network, technically any system that has the program that can understand that file and it know where to find that file, could do that “searching” for you. However, most databases are still not really capable of doing partial-indexes searches, so you’d always rely on that specific system/its power and load for how long any query would actually take.
One key idea of SafeNet is to prevent any centralised server dependency and never store stuff centralized. As that would mean if your server is down or compromised, you effect all users. On the other side, while “safenet” is distributed, this is totally transparent to the user of the API. You wouldn’t distribute your database through safenet, your database if uploaded into the system would be distributed by the network under it – outside of your “control”.
A better idea is to keep indexes in a partially search-able manner inside safenet and for searching you’d issue a “search request” that a “computing cloud” of specific programs would answer in a map-reduce manner for you. Thus not requiring any one server or system and run reliably without any centralised database. Neither a computing cloud, nor issuing such a “request” are currently available in Safenet, unfortunately .
As Safenet is very new and the concepts have only been explored theoretically is so far, there is still a lot of work to be done and experiments to go through, before this is fully available yet.
It is true that data can’t be searched for now (though not because the data is distributed), but search is a crucial feature, so it won’t be like this forever; see, for example, my idea, which is especially geared towards “matchmaking” (semantic hashes.) This, or similar ideas, will be implemented in the future.
As for now, maybe if you can find a way to partition your graph (e.g. as clusters around the more highly connected nodes), and then store the partitions in Structured Data blocks (which are mutable), then you could build a more-or-less searchable graph database even on top of the current architecture without having to resort to a split design with a centralized database. You’ll still need to have to have an application server because only the owner can update SD blocks.
An improved owner field for SD using ‘smart contracts’ or ‘m-of-n’ multisig would go a long way here. Allowing more complex rules for when SD changes are considered valid seems inevitable. Fairly hand-wavey but I feel SD could have a lot of additional benefits in the future and the current multisig proposal is merely ‘good enough for now’.
Eventually, we’ll need a way to search, filter, and aggregate vast numbers of tiny pieces of data; I can’t imagine any other way to build applications where anybody can contribute with the same rights. It isn’t acceptable that, for example, an application developer would need to “manually” aggregate (or countersign) the posts for a forum thread, as one of the major points about SAFE is avoiding single points of failure.
Hi, thanks for the very detailed reply, sorry we couldn’t reply earlier because we are not that technology savvy, so we need to communicate and digest this
So as far as we understand, unfortunately, the searching function cannot be achieved, what we are going to do now is trying to build a graph database (because it is better for building relationship and connection with people) and wait for the further development of SafeNet.
Thanks for all your replies, they really help us a lot and provide a new direction. We are now going to focus more on the graph database development and looking forward to the further development. If you could, please tag us when there is news on the searching function to keep us in the loop. Thank you very much!
Probably the best course of action for the time being! I’m a huge fan of Neo4j by the way (though I can’t say I used it in production; I was just messing around for the fun of it.)
I think one of the devs like 
.
because only the owner can update SD blocks.
