Routing - Build and Play With maidsafe/routing library (Rust version)

Development
1

David was bragging on twitter yesterday that he had several terminals open, each running “routing -n” and that it was cool to watch them talking to each other. So here’s how to join him in maidsafe rust routing smugness…

You need latest rust compiler (nightly): install rust compiler (I used yesterday’s nightly 14-May-15, @DrVecctor reports beta not ok)
You need libsodium (1.0.3): install libsodium (see @culexevilman’s CRUST instructions to install libsodium on Linux and don’t miss the “sudo ldconfig”, or see @Tom_Carlson’s post installing libsodium on Windows below)

Now build maidsafe/routing from github:

git clone https://github.com/maidsafe/routing
cd routing
cargo test --release

I got a load of warnings and then a long wait before it completed (maybe 40 minutes, on an old but fairly powerful laptop) so be patient (unlike me).

cd target/release/examples
./routing -n

You should see something like:

e856e3..529ec0  -- listening on : [Tcp(V4(127.0.1.1:53037))]
Input command (stop)

If that works, open multiple terminals, change to the “routing/target/release/examples” directory and type ./routing -n in each one. You’ll see routing goodness like this:

Input command (stop)
 handle_new_connect_event peer_ep : Tcp(V4(127.0.0.1:52968))
e856e3..529ec0 received FindGroup 1767392237
e856e3..529ec0 received ConnectRequest 
RT (size : 1) added b37ee1..c9dc6e 
e856e3..529ec0 construct_connect_response_msg 
 handle_new_connect_event peer_ep : Tcp(V4(127.0.1.1:49069))
RT (size : 1) Marked connected peer_id : b37ee1..c9dc6e , peer_ep : Tcp(V4(127.0.1.1:49069))
e856e3..529ec0 received ConnectRequest 
 handle_new_connect_event peer_ep : Tcp(V4(127.0.0.1:52973))
e856e3..529ec0 received FindGroup 3602834904
e856e3..529ec0 received FindGroupResponse
e856e3..529ec0 received ConnectRequest

Then go brag about it :wink:

But there’s more (first see @BenMS comment about performance immediately below) and then (courtesty of @dirvine below) open a terminal with just ./routing (no parameters) and try put <key> <value> with a few combinations and then get <key to see them retrieved!

For example:

$ ./routing
Input command (stop, put <key> <value>, get <key>)
time wasted
Invalid Option
Input command (stop, put <key> <value>, get <key>)
put time spent       
putting data TestData( key: "time" , value: "wasted" ) to network with name as 9d3046..8c136c
Input command (stop, put <key> <value>, get <key>)
put safe SecureAccessForEveryone
putting data TestData( key: "safe" , value: "SecureAccessForEveryone" ) to network with name as faf82e..36d5d6
Input command (stop, put <key> <value>, get <key>)
put happy being
putting data TestData( key: "happy" , value: "being" ) to network with name as c00192..5f7daa
Input command (stop, put <key> <value>, get <key>)
get time 
getting data having key time from network using name as 9d3046..8c136c
Input command (stop, put <key> <value>, get <key>)
testing client received get_response with testdata TestData( key: "time" , value: "spent" )
get safe
getting data having key safe from network using name as faf82e..36d5d6
Input command (stop, put <key> <value>, get <key>)
testing client received get_response with testdata TestData( key: "safe" , value: "SecureAccessForEveryone" )
get happy
getting data having key happy from network using name as c00192..5f7daa
Input command (stop, put <key> <value>, get <key>)
testing client received get_response with testdata TestData( key: "happy" , value: "being" )
17 Likes
2

Pretty awesome right ! :slight_smile:

As you guys keep discovering parts pre-announcements, here are some cautionary notes being addressed. If you find other issues, please report. That helps us a lot !

You’ll notice that on linux-mac CRUST now starts listening on local address 127.0.0.1; this is a funny story as a double correction actually put a bug in UNIX based systems; Windows systems work correctly. This causes that just now only Windows systems are discoverable on the LAN and linux/osx isn’t. This is being worked on as we speak.

Also swarming messages is currently breaking outside of its close group and each message will swarm your whole network - not what we’d want right :smile:

Note that Sentinel is not plugged in and as such each node will act on all messages individually; As an upside RoutingTable is that robust it handles that really well ! Sentinel’s job is to accumulate such messages and first verify that a full group consistently required the same action before Sentinel passes this command on to the routing node.

7 Likes
3

After a few are running then type routing on it’s own and then put X Y several times where X and Y are your choice (i.e. apples 6, pears 4 etc.) and then get X should get you back Y :slight_smile: Early days first test of new routing. Lots to check but incredible to just code a large lib like this and see it going so quickly. [@happybeing neat, have updated OP with this]

8 Likes
4

Lol…no clue what anybody’s talking about, but all sounds like things going well, right? I’ll just have to patiently wait for the big green button version…though @happybeing 's glee at being able to do loads of stuff and I can’t is beginning to get on my nerves a bit now…lol :smiley:

10 Likes
5

I tried it out, and according to my experience it does not work with RUST beta but only with RUST Nightly, it complains that a certain action is not permitted in beta (but I don’t remember the wording, and I can’t find the place in the console any more…). I will start experimenting now! Thanks for the instructions! [@happybeing: OP updated, thanks]

3 Likes
6

I also had to update my Rust to nightly. Thanks for the hint.

I was a little stupid about installing libsodium on windows, so to document it somewhere, I did this. (for a 64 bit windows system)

[@happybeing added to the OP thanks]

4 Likes
7

If somebody plays with it, please report how many nodes on different machines you have been able to connect together and checked they could talk to one another in a pair-wise fashion. 1.5 years ago (!) when I last tested on lab machines at McGill, I started getting troubles once I would go over ~30 machines. It would be great if that version scales better from the start ;-).

3 Likes
8

Windows 7 64 bit

c:>routing
Input command (stop, put <key> <value>, get <key>)
put time spent
putting data TestData( key: "time" , value: "spent" ) to network with name as 9d3046..8c136c 
thread '<main>' panicked at 'called `Option::unwrap()` on a `None` value', C:/bot/slave/nightly-dist-rustc-win-64/build/src/libcore\option.rs:362

Hmm, that’s weird. I don’t even have a C:/bot directory on my computer.

Let’s try it with quotes…

C:\>routing
Input command (stop, put <key> <value>, get <key>)
put "time" "spent"
putting data TestData( key: "\"time\"" , value: "\"spent\"" ) to network with name as 0b39a2..88bfce
Input command (stop, put <key> <value>, get <key>)

OK, let’s try it again, with no quotes

put time spent
putting data TestData( key: "time" , value: "spent" ) to network with name as 9d3046..8c136c
Input command (stop, put <key> <value>, get <key>)
get time
getting data having key time from network using name as 9d3046..8c136c
Input command (stop, put <key> <value>, get <key>)
testing client received get_response with testdata TestData( key: "time" , value: "spent" )

OK, that worked fine as well. In fact, I can’t duplicate the original panic, but it happened, so I’m documenting here.

Aside from that, it’s working and I see the traffic echoed on 2 other terminal sessions.

3 Likes
9

Yesss!!! :sunrise_over_mountains: n

3 Likes
10

Got it working on ubuntu 14.04, rust nightly build :smiley:

2 Likes
11

Tried on both Windows 8 64 bits and Ubuntu 14.04 32 bits (separately) and it almost worked: the same messages appeared except this line :

Am I alone getting no message from the get response handler?

1 Like
12

I installed it on 3 physical machines and one VM and I’m getting different results.

routing -n seems to be working in general, on the same machine and on the same subnet, but not accross subnets. My WiFI and LAN are in two different subnets and seperated by iptables. Allowed from LAN to WIFI, but not the other way around, although I added some exceptions to that for my laptop. Should routing -n only work in the subnet of the machine that it is started on?

Furthermore I seem to have the same problem as @tfa. There is no response when trying to get the value of the key back.

In addition to that, on at least one machine routing has a a panic attack when trying to put a key

     ./routing 
Input command (stop, put <key> <value>, get <key>)
put time spent
putting data TestData( key: "time" , value: "spent" ) to network with name as 9d3046..8c136c
thread '<main>' panicked at 'called `Option::unwrap()` on a `None` value', /home/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-linux/build/src/libcore/option.rs:362
13

That’s correct, please remember this routing -n is a a test example for LAN only, bootstrap allows across WAN as well but requires some nodes to be started remotely to connect to, then it all meshes together.

As Ben said we know of an issue with beacon and Win/Linux interoperability which is being worked on right now.

This is also a very sneak preview of the next part we will hand over, but we have not handed it over yet as working with instructions. We fully expect we need to debug a little, this is basically code and run with no time for us to debug at all, amazing it’s so quickly at this stage (0 → running network in 5 weeks).

So remember this is where you are in the process, first build :slight_smile: Nowhere near ready to hand over just yet. I hope it will be in one more week though.

8 Likes
14

That’s absolutely fine David, I just wanted to report in with my experience and I’m patiently waiting for updates. (it’s not like I don’t have anything else to play around with :D) Happy to try it out in this early stage, so keep it up :slight_smile:

6 Likes
15

Reading the doc comments in routing/examples/routing.rs, I realized that we need to launch at least two nodes (in addition to the client):

//!      starting first node : routing -n
//!      starting later on nodes : routing -n
//!      starting a client : routing

Doing so, I correctly get the value back in the get response handler:

testing client received get_response with testdata TestData( key: "un" , value: "one" )
3 Likes
16

Yes, some might remember the Local Network Controller (LNC) application that we had in testnet 1 to bootstrap the initial network. Remember that the SAFE network is a fully secured DHT, with an integrated handling of the Public Key Infrastructure, so a node can only connect to the network if the network knows the public key of that node. You can see the chicken-and-the-egg problem here: for the first node there is no network, for the second the network would be that first node… The LNC before was an automated construction of special first nodes that all knew each other from the start. It required delicate construction, and it was a mess. We’ve done away with that.

So one thing we’ve done is that routing now allows a node to publish its keys in a ‘quarantined way’ but enough to allow it to connect, after which the keys are properly stored and known by the network (in the routing table). This feature allowed an important improvement: self discovery and automatic bootstrapping of the network: it’s called ‘beacon’ in CRUST. It allows a node to broadcast on UDP and any existing SAFE node will reply with the information the new node needs to connect to the network.

This beacon is an important complement to the only other way we have to bootstrap a vault into the network: a bootstrap list. So the bootstrap list of old is still present and you can start a vault with ‘routing -n 213.73.45.231:54667’ for example (don’t try, just a made up IP), where some IP:port is the location of a known node on the network.

A vault will keep a list of all the nodes it has met when it was connected. So if it gets disconnected and restarted (even though it will start with a new ID) it still has a list of endpoints (IP port and protocol) that it can try to reconnect to the network.

So beacon is a new feature that allows to automatically (and without a LNC tool) to bootstrap a network on a local network. To bridge beyond that LAN all you need to do is provide an external endpoint (port forward on your router if you are starting your own testnet) or simply connect to a single outside node you know off (MaidSafe will provide such endpoints for community testnets soon).

Another important feature of beacon is that it allows for easy configuration of your farming vaults. If you have a farming node on any device, within the LAN they can self-discover and allows you to set the wallet for farming from your client device, for example.

There are plans for extending beacon with gossip algorithms to allow automatic discovery beyond the LAN, but that is for later.

So yes ‘routing -n’ will only work on the same LAN as the node has no information as where to find another networked node (other then beacon self-discovery), but you can run ‘routing -n ip:port’ to give the node that information and go anywhere.

hope that helps

9 Likes
17

Are the ports chosen predictable for setting firewalls?.. atm it looks rather random pick of five digits ports.

18

The code itself provides the parameters to try listening on a desired port. I think this is not reflected in the routing example. If such a function is desired we should indeed provide it for a beta release version

by default we opt for a random port in a wide range indeed. As such we make sure the SAFE network is not easily identified by port numbers. For normal firewall settings and normal nodes (i.e. when you’re connecting to an existing network) it is coded to punch a hole in your firewall/router to allow the traffic to be established.

1 Like
19

Thanks.

If it’s stable for each instance and visible, perhaps it doesn’t matter but if it’s reset each time a system becomes available, that might be a nuisance.

If it’s useful that it is flexible and changes over time, then at least knowing the range used would help.

1 Like
20

and it even works wir ä’s and ö’s in key names ! :smiley: (at least with ubuntu 14.04)

marvellous! it’s like magic … you put data in your flower pot and get it out of your coffee mug when you need it :open_mouth:

3 Likes