The essence of SAFE apps

[janitor]

@dallyshalla

you can make some meta data for the segment of the SQL database you want; so that instead of pulling a whole MySQL DB to query, use the file Container, scan meta data of the files inside; and then download just the content of that file which could be the one line in the MySQL DB you wanted…

You are saying MaidSafe will pull blocks, not entire files. Is that how it works?

But anyway, let’ see how much that could save us:

• How much data from a DB index file do I need to use the index? All.
• How much data from a table do I need for a SELECT * type of query? All.
• How much data from a table do I need for other (e.g. DELETE) operations? In case of MySQL, I suppose all.
• For all other data, how much do I need to wait for every single non-cached IO operation to complete? Say I click on a menu item that generates 15 DB operations. How long do you estimate that could take? Would such an app be considered usable?
• Most of files that I have to GET (such as indexes) also have to be PUT (because they change). So making any use of an app would likely require several PUTs.

@dallyshalla

(Referenced another user’s post) Largest is 1Mb Smallest is 3k min 3 chunks per file Cannot be altered by the user

So each put’s don’t have to be as much as 1MB of cost;

• Are you sure all PUTs are priced on a per capacity basis? Or does MaidSafe just allocate less space while charging you the same?
• What is the minimum unit of update for files that are 1MB or larger? Is it a chunk, or a file? If it’s a file, then the entire file has to be PUT when it’s updated and the flexible chunk size would have no bearing on cost here.

@happybeing

The following is even more suspect because it is only true if every single block in the database is modified, which would be rare in many, if not most applications, because designers will already have tried to minimise this for performance reasons.

You need to download the DB and that’s a scenario that I mentioned on this page yesterday

Why do you have to focus on what I clearly said is the worst case scenario? And you called me a pessimist!
My baseline scenario (in this topic) has always been that each update to MySQL will generate at least 2 PUTs, one for the data and one for the transaction log.

Designers (I assume you’re talking about MaidSafe developers) either have this plan for v1.0 or they don’t. I already said that perhaps by v2.0 there may be a better way to handle this. So it looks like you’re either repeating my earlier claims, or you are speculating that this optimization will happen in MaidSafe v1.0.

@happybeing

So again you’ve just ignored what I said and restated the same points. This is a familiar pattern with your posts. Rather than address a point of criticism you’ve simply repeated yourself.

I thought I spotted the same pattern in your replies the other day, I just didn’t want to call you out on that. I’m referring to my questions from this comment here, when I asked you - because you’re in favor of “appropriate regulation” - whether you think that MaidSafe should be regulated to prevent it from turning into a monopoly, and if yes, how should that be done.

@happybeing

I am pretty sure based on discussions on the forum, that it will not be one PUT per transaction, which is why I’m asking you if you are sure about that and if so to back it up. If you aren’t please stop repeating it.

Let me elaborate:

As I understand it, every write to a file on SAFE Drive does not result in a PUT. Instead there is some caching that leads to write operations being batched up. I don’t recall how the batching works, what if any precautions there are against data loss, so it remains to be seen what kind of applications this would be suitable for.

You saw a generic discussion about how write buffering will work with regular, flat files.
You are assuming that each MySQL write is the same. It’s not. Writes are inter-dependent.

A write to one place in DB often means that a write in other place (such as the transaction log or some table or index) must happen. MaidSafe 1.0 would have to know which writes can be buffered, which cannot, and which should be grouped with others in some particular way.

That’s why I said I don’t need to know how MaidSafe v1.0 will work on the low level, because:

• I haven’t seen such claims on the forum that relate to RDBMS files
• From experience I know those things are hard to do even on a local filesystem (if one attempts to write one from scratch)
• For v1.0 I would consider plans to implement something like that a bad idea because when one deals with such a complex app, a best practice is to build a stable core and not some fancy database caching features

So regardless of that discussion about write buffering of plain files that you mentioned above, I am telling you that I don’t believe write buffering and coalescing applies to RDMBS until I see a statement to that effect.

If you write-buffer databases like that, it will work great, except when you get disconnected or your app crashes or your MaidSafe crashes. Then you will end up with a corrupt database.

I didn’t count a whole bunch of additional updates that happen on WordPress MySQL (e.g. everything generated by plugins) because I don’t consider WordPress a single client app, so it wouldn’t be fair and those aren’t strictly required, although we know that faced with a choice of running WP without a single plugin would mean most people wouldn’t run it at all.
Something like SQLite might be more appropriate for single-client apps in MaidSafe v1.0, but even in those cases, my baseline assumption (1 PUT per update) would probably hold for databases that are not fully downloaded to the client every time the app is executed (and uploaded back after clean shutdown, assuming the user accesses it from different clients and/or wants to use MaidSafe as their primary data store and not as a backup device).

I outlined all of these problems and mentioned some potential workarounds (such as keeping 3 copies of a DB on MaidSafe and deleting the last one every time the latest is error-free) earlier, so this comment is also mere repeating of what I had said before and I’m not sure how effective it will be. I guess it won’t.
But if there’s no definitive proof one way or another, we’ll have to conclude this after v1.0 is available.