Hello all,
Being familiar with Unix (Linux) systems, I was wondering what the VFS would look like once I mounted it on my machine. Here’s what the Linux Filesystem Hierarchy looks like. I have a negligible (read: “absolutely no”) understanding of this topic so please bear with me. Let me start from the very beginning if I may:
After mounting the SAFE Network VFS onto /media/$USER/safenetwork and self-authenticating:
$ cd /media/$USER/safenetwork
$ ls
What do I see?
This will be the drive library I think you are talking about, in that case you will see a hard drive as far as the OS and applications are concerned. So like a hard drive is easiest way to imagine that.
Initially you will get an NFS API though, where you can have programs written against that to act like they were on a hard drive (think of mobile apps, where you rarely see a hard drive or finder window, but every app has files (like music apps) and they show you different representations of those files, but not look a traditional hard drive.
You will see a directory listing that you can browse just like any other directory on your computer, and in which you can save and load files as normal.
Precisely what the directories and files within that will be I don’t know, but we can speculate there will be some default directories such as “documents” (for private stuff) “public” (for sharing) or “www” for a default public website. I’m guessing, and not likely to be correct here, just trying to give you an idea. I think we’ll see this for real very soon… fingers crossed for this week 
At a high level, yes. At a lower level, it seems like the REST API is interfacing with something. But where is it?
Where is that directory retrieved from? I’m trying to figure out how the REST API pulls data from the data map. For example, what do safe_nfs::container::Container.get_name() and safe_nfs::rest::Blob.get_name() query to get their values? Is it the data atlas that is retrieved with the PIN on login?
The virtual file system is implemented in top of/using the REST API, as I think you are aware, but you won’t see the REST API through the filesystem. You will only see directories and files!
To understand how the two fit together you will need to study the library code, and I think you will find that there is not a simple correspondence between the data map and other structures (such as Containers), but I may be wrong.
Sorry I can’t be definite, maybe someone who has studied the libraries can provide more detail.
So given that what a user sees on their NFS are representations of files and directories, what is the hierarchy thereof?
Thinking it through at the most abstract level, there are public and private directories, and there are different personas.
Might the filesystem root start out with two directories: public/ and private/? Then perhaps they could each contain one subdirectory per persona?
EDIT: This is predicated on the fact that once a keypair is made public for the directory, any and all files underneath it can be accessed. Therefore this implements access control by structure. Is there is a way to implement access control by file permissions?
What happens when you access your data from an operating system that doesn’t understand the original permissions?
It won’t be able to access the files. So if file permissions were a possibility, they would have to be implemented on the safe: protocol layer. I am unaware of any such implementation. If that is infeasible to implement than would the access control necessarily have to come from the structure of the filesystem?
SAFE drive (if you dig into the old C++ code at least) attempts to map file metadata between operating systems. This is imperfect because it can’t be done exactly and is a bit of a nightmare in my experience, but the details are there if you want to dig into them. These may now of course be in the Rust repo too, I haven’t looked, but if you grab the C++ repo it is definitely in their (or in the associated specification).
I don’t think I could understand the code, but my question was rhetorical - mapping between various file systems is only partially translatable and NFS (why oh why they didn’t listen and replace that unfortunate acronym!) permissions couldn’t be properly mapped (don’t forget it’s not only the mapping, but also User ID’s on the system level may differ, eg josh may be User ID 501 on system1, but completely absent on system2).
I don’t see how that can be worked around so the current approach makes sense).
So the access control is still implemented by structure. Is it still the case that given the public keypair to retrieve and decrypt one folder, every other subdirectory of that folder is accessible as well?
Getting a little deeper into NFS, I am struggling to understand how the system takes data from the datamap and translates it into the file system. I understand that the datamap operates on XOR space, however that does not seem to be directly applicable to a filesystem hierarchy.
Does the datamap provide an absolute pathname in the array of file-hash pairs that it generates? If so, is that absolute pathname mutable?
Also, can I add an absolute pathname of a public file to my own datamap? Would that be subject to a PUT fee?
The NFS is built on top of the raw PUT/GET storage system (data map). There is a detailed spec somewhere of early proposal for the data structures it uses and how they are implemented using a Blob data model where the Blobs are objects that are PUT/GET to SAFE storage (I think) - or maybe as SD, which wasn’t “invented” then. From memory, blobs are used to provide:
Users of the NFS API were cautioned not to nest containers more than one level deep because there was no way to detect and handle update conflicts, which is why I say they are not exactly like directories.
So if you looked in the data map you would find refs to both Blobs and any files you’ve stored using NFS or any other SAFE API. But Blobs are effectively meta data that is used by the API to provide an NFS interface, and won’t make sense or show up directly in any end user UI. They’ll be like hidden system files.
If you dig out the old C++ NFS and Drive repos you may get more accurate info than my memory is providing, but maybe this helps a bit 
I hadn’t even looked at the Drive repo! I think that’s really what I’m looking for. The rust implementation on github is empty, and the C++ version I can’t seem to find. (EDIT: found at MaidSafe-Archive · GitHub)
So, if I understand the hierarchy, anything on the datamap will point to a globally unique Blob. That Blob will contain the metadata that tells the NFS where to put the files contained within that Blob onto my Drive (the representation that looks like a normal filesystem). If I wanted to change the absolute pathname, would I then have to create a new Blob that pointed to (symlinked) the existing Blob as the one having the actual content? That blob would then have different metadata as to where to store the returned files on my drive.
Or would this be better handled at the Drive level perhaps? I did see that:
Which makes sense, but if the file that I was linking to was someone else’s on a public directory, it wouldn’t be in my own datamap, unless I put it there. And if putting it there requires a Blob, I could just specify the existing Blob in your datamap. But if you want to assign that to a different place in the filesystem (a.k.a. give it different metadata) I must create a new Blob. Then that Blob would be a placeholder that pointed to the actual data in the original Blob. If so, that means that any symlinks require a PUT cost.
I’ve been bouncing around trying to see how this symlinking ability interacts among the datamap, blob/container NFS, and Drive levels. This may just be implemented at Drive level. I guess I’ll have to see what documentation I can dredge up, and ask around a bit.
P.S. @happybeing hands down the best datamap ↔ NFS explanation that I’ve read so far. Thanks!
You’re welcome, but take it with a pinch of salt because my knowledge is well below full understanding and I’ve made some hopefully intelligent interpretations, as well as relied on my memory.
BTW I don’t think it’s right to see everything in the data map as a Blob. I believe the data map is a list of all the files you have access to on SAFE, and if you use Drive to do stuff, some of them will be Blobs, others might be created by any Apps you have that use the SAFE API directly, etc. EDIT: strike this paragraph, I think you are probably correct, because all files are versioned, and the Blob is needed to hold the meta data as you stated. Sorry, as I said, memory is flaky Trust the code, not me!
Containers, is it synonymous to btrfs “pool?”
I don’t know about btrfs.
