Pre-Dev-Update Thread! Yay! :D

[whiteoutmashups]

@neo

So is corser the actor or the lawyers

Whenever Mr. Irvine types a long post, there’s always & exactly one typo. I’ve been noticing it for years.

(Mysecret hypothesis is that he’s a superhuman from the future trying hard to act like a normal human ;D)

[chrisfostertv]

@dirvine

Ok Here is where we are tonight (again)

Fantastic update…better than any Dev updates, really spells out the progress and it sounds awesome.

Incoming from China …poking the panda…wow.

[chrisfostertv]

@anon81773980

Maybe I’m not thinking right but this is my biggest hurdle to date

If your not thinking right, I’ve got no chance of understanding your ‘Godmode’ …those Redox guys didn’t really warm to it?

I like your idea of microkernels for OS and that’s what @dirvine has been playing with. Maybe you could ask him a direct question as to wether this URL mapping SAFE <->Redox is a viable method…it sounds intriguing.

[dirvine]

@chrisfostertv

URL mapping SAFE <->Redox is a viable method.

It makes a lot of sense. IPFS also use everything is a file, conversion to url is a seemingly obvious approach. Files & directories are kinda passee in many ways, If everything was a resource to be located, you can still have your root resource (rot dir) and a tree like structure (directories), but others can link to any of your branches (directories) or files, applications, devices etc. So it gets very interesting. Blur the lines between dirs/files/devices/locations/ownership etc.

It’s been a long time coming, but it just may develop like this and from that perspective redux is interesting, There is a bunch to do when we get live, for sure

[bluebird]

@dirvine

Tcp hole punching has been very successful.
Tried from China which worked perfectly well
Automatic nat_traversal seems very good
We are not using any more resource in Udp and variants now as the Tcp nat_traversal is much better than we thought.
Much of logging is now improved, info for user facing liogs and debug level for more information such as numbers of messages, amounts of data per vault. Should be interesting for people, but especially for devs.
Service discovery switched back on (no need for config files)

1. On that last point: How do the latest vaults initialize to the network if they have no config files?

2. Seeing the advances listed above, I just now did a new clone, build and run: The vault listens but doesn’t find anything, until I start a second instance on the same host and then they talk to each other.

3. I take it that the mock crust file is only for testing at compile time?

4. Seems to me that if you are running a network every day, and given service discovery is now enabled, then if I run the latest build at random times I should land on the test network. Or is there an extra step to getting that far?

EDIT: Extending question 4: I was thinking that the test networks run during the day (UTC) manually started and stopped by the devs at Troon. Or do they rock around the clock? Maybe my attempt to connect failed because there was no network at 0200 UTC.

[dirvine]

@bluebird

On that last point: How do the latest vaults initialize to the network if they have no config files?

You will need a config file or bootstrap.cache file to connect to the main net. Unless another node on your Lan is already connected to it.

@bluebird

Seeing the advances listed above, I just now did a new clone, build and run: The vault listens but doesn’t find anything, until I start a second instance on the same host and then they talk to each other.

This is expected, you are creating your own local network in this case.

@bluebird

I take it that the mock crust file is only for testing at compile time?

It is a fake crust if you like. It allows testing of tiny fake networks to confirm routing and vault code. So it’s a testing helper.

@bluebird

Seems to me that if you are running a network every day, and given service discovery is now enabled, then if I run the latest build at random times I should land on the test network. Or is there an extra step to getting that far?

You would still need a config file to connect to a running test network. We internally use different seed nodes etc. to prevent anyone connecting while we measure stuff at the moment. Ultimately we would have separate testnets altogether, Just very rapid deployments measure - code - deploy - measure etc. at the moment.

We run the test networks during the day and overnight for soak tests. In these we count messages/message types/ connections etc. Thsi is why we keep them to the side atm if other connected we would be thinking something went wrong and maybe waste some cycles working out what happened

Longer running testnets wont be long though and folk can connect to them for their own testing etc. That will be better for sure.

[johnlu]

@dirvine

We are not using any more resource in Udp and variants now as the Tcp nat_traversal is much better than we thought.

@dirvine Are you definitely leaving udp? Or are you postponing this task as it turned less important than other tasks after you realized tcp is behaving good enough?

And have a good day you all guys!!

[dirvine]

@johnlu

@dirvine Are you definitely leaving udp? Or are you postponing this task as it turned less important than other tasks after you realized tcp is behaving good enough?

For now we have really amazing tcp results. So we are not coding out the multi protocol paradigm, but we are putting any resources in crust into Tcp. We will remove Udt but maintain the generic approach there for future protocols. In specific what is happening in crust is

1. Use secure_Serialisation
2. Make interface use generics (so no passing pre serialised messages only to serialise again in crust)
3. Switch to epoll/non block tcp to free resources and increase efficiency (a big increase is expected there)

These will be part of crust roadmap and some more smaller parts. Essentially this will happen as we launch and beyond perhaps.

[BIGbtc]

@dirvine

Ok Here is where we are tonight (again)

Good news

Excellent update David. China is the catalyst.

[bluebird]

Thanks for the openness, it makes this project quite special.

Since I would be getting in the way if I managed to connect, I won’t go back and mine old configs for droplet IPs that might be reused in the current ad hoc networks. Promise! But I’m eager for the open test network.

EDIT: What I’ll do is run it on a dedicated server that I already have, and on my office machine, and see if i can establish a network that way.

[whiteoutmashups]

@dirvine

So we are not coding out the multi protocol paradigm, but we are putting any resources in crust into Tcp.

So Skype, YouTube, Netflix, etc all that won’t be possible on SAFE anymore? (at least the MaidSafe version)

Excuse me but I definitely have to call that out… I feel like half of the promises of the future internet are being cut off at the heels right there! I hope I’m misunderstanding

[polpolrene]

Why isn’t that possible anymore? If you have several IP-pipes into the SAFE Network, you can watch a videowebsite (let’s call it SAFETube) and you get the chunks in a parallel way over TCP. It might even be faster than Youtube itself that way, who knows.

[whiteoutmashups]

But I understood TCP and UDP were meant for completely different things, and UDP was meant for heavier traffic data like steaming video packets etc Unlike TCP.

I think this is an incredibly Crucial and important part of SAFE and it’s potential applications

We would have to completely rewrite how video etc is streamed online, starting from ground zero all over again? We honestly can’t use the UDP protocol that was made specifically for this kind of thing?

[abittner]

I am kind wondering as well why udp (udt) is being neglected here, but that might be implemented by the community itself or at a later point in time when the whole project or network matures. I do wonder though if it will ever happen once a transport (tcp) is well established. Btw, how is ipv6 support going, is that automagically included and supported with the current tcp layer or is tcp over ip4 and tcp over ipv6 to be regarded and to be coded independently?

[polpolrene]

@whiteoutmashups

But I understood TCP and UDP were meant for completely different things, and UDP was meant for heavier traffic data like steaming video packets etc Unlike TCP.

TCP is more reliable but has more overhead. UDP is faster but you might experience some dataloss. Not a real problem for a videocall, but a big problem if you bank over the internet . So both have their pros and cons.

What is the difference between UDP and TCP internet protocols?

[abittner]

Reminds me of when seemingly eons ago I came across maidsafe-dht via udt and became introduced to the overall project. Its been a different time …

[dirvine]

Yea we are not ignoring Udp transports. We will still have that. For udp rtp etc. all we need to is set up and secure the endpoints. So not a problem really.

Tcp is the network connectivity between nodes and if we can get everywhere (almost) with this then we will push forward fast with that. In terms of negotiated endpoints then adding plain udp is very simple, it does not need to multiplex or provide listener abstractions etc. We can provide endpoints easily for websockets and more. They do not need to provide node to node connectivity though. So we are all good

[whiteoutmashups]

@dirvine

So we are all good

OK I understood this part

Sorry if I was spouting off about something outside my area of experience

[Tim87]

@whiteoutmashups

Whenever Mr. Irvine types a long post, there’s always & exactly one typo. I’ve been noticing it for years.

(Mysecret hypothesis is that he’s a superhuman from the future trying hard to act like a normal human ;D)

Well this also leads us to the hypothesis that the Dvorak layout gains wider adoption in the future.

[wes]

Well this also leads us to the hypothesis that the Dvorak layout gains wider adoption in the future.

I’m hoping for colemak myself. It just makes the most sense.