Rust vs C++

Development
44

I think there is a balance to be struck.

I expressed that some of this could wait until after v1 too, but only as constructive input. I am sure we have all seen or worked on projects that aim too high and fail to deliver altogether. I don’t think for a second that safe net is on of them, but focus remains important.

If you leave the direction 100% to the product people, you get very little emphasis on technology - this can result in a bad, short sighted, product.

However, if you leave the direction 100% to developers, they can concentrate on trying to investigate and integrate every new technology. This may well benefit the product in the longer term, but it means having nothing usable in the shorter term.

As far as business goes, you need something between the two to be sustainable. Being first to market, but then being unable to move forward is bad. Getting to he market too late with great future potential is no good either, as your potential user base has already settled on something else.

The maidsafe team is smart and David Irvine has impressed me many times with his understanding and insight. That is why I am certain the correct balance is being sought and found. This doesn’t mean that we shouldn’t ask for assurance on this, but I don’t think there is cause to be negative .

1 Like
45

@Nick Thought I read it somewhere, that testnet 3 was to begin before the end of March. Thought that it was you saying it. If that is not correct then when is testnet 3 starting and when is SAFEcoin going to be testable?

@everyone OK guys I don’t want to derail this thread, it’s obviously a development thread. But as a non-developer with a financial stake in the MAIDSAFECOIN asset issued on mastercoin that is paying for the development, this development update read like an announcement of a delay until at least May, at which time it would be announced how long the “transposition” to Rust code was going to take.

This sort of thing happens a lot in academia, in the social sciences, where someone is working on a dissertation or research project, then world events happen that invalidate their theory or make it moot or irrelevant, and the author has to scrap the work and start over. The difference is that this is engineering, not social science.

Since this is a software engineering project, the engineer claiming to be able to build the system has, implicit in that claim, claimed that it can be done with currently available tools. So when the engineer comes and says there is a new tool that is going to be available in the future that will make it possible to build the project, it is cause for concern for those that funded the project. The concern being that it cannot and will not be built with the tools that were available when the project was funded.

Additionally there is a phenomenon of delays, and delayed engineering projects in general, whereby there is not a normal Gaussian bell curve distribution of delays around an average time delay for all projects. Instead, somewhat counter intuitively, The longer the delay, the more likely there will be an even longer delay, and the more likely the delay will be terminal, with the project never being completed.

This is why I am so concerned, being a significant funder of the project and “bag-holder” of maidsafecoin. The delay that has already occurred makes it seem more likely that further delays will occur, so I am hypersensitive to any news of delays, in whatever form it may take. This thread might be a case of me seeing a delay before the engineers on the project even realize it is about to happen, which is why I am immediately pointing that out, in the hopes that the delay can be averted.

I hope to see you both at Collision in May, it’s been too long. And I hope that testnet 3 is live then, with SAFEcoin testable by then, so that I can talk about how awesome maidsafe is and how it looks like maidsafe is going to work. Before I can do that, I want to be able to hire an engineer to build and test maidsafe with safecoin to make sure it sort of works and is somewhat secure, so I can breathe easy.

Conversely, if there are further delays, and the collision event is just a way to seek more money because the money from the crowdsale is running out, really I just don’t want to think about how much that will be bad. At the very least I would expect the coin price to fall to below the ipo price on poloniex. After all, each testnet was promised to take 6 weeks and we have been in testnet 2 for longer than 6 weeks.

46

I guess it’s about this tweet:

2 Likes
47

I feel you I really, it’s a hard road and we need to get there fast, and we are not fast enough right now. I fully appreciate that and am addressing it as best I can, honest guv :slight_smile:
Ah testnet3 will implement safecoin testing, but during not before it starts. We have had folk see the roadmap that way and it still seems to confuse. The phases implement all the points in the map, so its not implement the stuff then we are testent3 it is the other way round.

If we switch anything it wil be to increase productivity. That is what the vault refactor was, so basically stop, go backwards a week and reimplement (so say lost 3 weeks), but with the huge reduction in code then we move forward much faster (technically 50X but that will never be the case).

Conversely a switch to rust if it happened would phase in team wise, so testnet3 needs to be up, apps need developed and while the app team are handling that some of the core team can transpose and confirm rust (if we do this).

Then we go forward with much less code (from the language of the network) and a much safer implementation of at least some libraries that protect against many zero day exploits (Mozilla’s big push for rust internally is this point, they cannot cope with the security advisories for their 8 million line code base).

So you can be sure this does not affect progress for testnet3 and if anything will happen during then or beta if we deem it worthwhile. If it is as easy and fast whilst preventing exploits and increasing dev participation it is a big issue we must consider.

So swap over during the live network could be very tricky, when you imagine the nodes need to speak 100% the same language so we would need to nobble the serialisation to and stream that works in both, so say json, then we lose efficiency. There are issues but this is why we need to investigate. When I say we it means me right now, nobody else involved :slight_smile:

I worry as much as anyone about delays and this is why I search for ways to increase efficiency and productivity (I do not believe in doing the same thing over and over, if something seems slow or not good I change it if I can).

8 Likes
48

If it increases security, improves efficiency, and makes David happy dammit it’s what will make the boat float :stuck_out_tongue: and I’m all for it

And this is also coming from an “investor” but I think of it more as a donation to a project that aligns with my beliefs. So I’m not in a rush for a return on investment.

5 Likes
49

This does not change Jim, it can be built in ada c c++ rust spark python ruby … any turing complete language in fact. You really need to watch how you put across statements you get very close to making an accusation about honesty or integrity or perhaps intelligence and that’s thin ice, I try and make time for any investor and yes that costs considerable time and delays launch (way more than it would take me alone to transpose to rust), however I deem it important and therefor do it. I also take the occasional bath but don’t need to, so please do not give me the benefit of your vast experience with major Engineering projects and soundbites like oh in academia etc. that’s not helping really it’s satisfying some personal thing and desire to preach.

As I said would you continue with a horse and cart if a car all of a sudden came along when you are transporting goods about? or build houses with mud when cement happens, this is Engineering and that derives from the Latin for to use ingenuity so we/I will reach for anything that gets us to the dance early man. I guarantee that.

I have shed loads of projects to “play with” and don’t, I wish I had the time, but I don’t until we are fully launched so I am pretty determined to do that.

4 Likes
50

Are you involved in the StorJ project?

51

https://twitter.com/jim_lowry

Co-founder of Storj

52

@dirvine I am curious if you had considered LuaJIT or Go for this rewrite. Or more generally, if you would touch anything with a GC and/or a jitted dynamic language (none of the slow ones like python, ruby etc. of course).

Personally I would like to translate maidsafe to Lua, if I get around to, and here’s why:

  1. it will require orders of magnitude less effort than the C/C++ version
  2. it will result in probably the most readable code (important for security-focused apps)
  3. it might be just as speedy as the C version (see below)

I know these are wild claims to make in front of a C++ crowd, but allow me to argument these points:

Being a dynamic lang is the primary LOC reducer of course, but structuring the code into coroutines (light threads, fibers) and avoiding the async callback mess entirely is even a more important in terms of reducing LOC and making the code easier to read and reason about.

Lua reads like pseudo-code, in fact I would argue that bad Lua code is more readable than good C code.

LuaJIT has a reputation for speed being used in very demanding networking apps (see http://snabb.co/ – cloudflare uses that to create custom firewalls with)

Even if LuaJIT doesn’t manage to compile every hot loop in the code, and you need to offload some of that to C, it still has by far the easiest ffi of any language to do that job (because it can read C header files directly).

I suspect (this is wild speculation - please correct me here) that maidsafe is I/O-bound and not CPU-bound (barring the encryption-decryption bits, which could be offloaded to C/ASM/SIMD if necessary). If so, it might just be the case that maidsafe remains I/O-bound even if the language is slower-than-C (but not much slower).

Thoughts?

53

I am not a GC fan but you can include Lua easily in Rust as well. I think if we get the serialisation correct then it will allow many versions to be created. We did tests to offload to gpu etc. for crypto but it did not yeild results worth the effort (cpu does aes very fast and faster than getting it to the gpu).

I have played around with lua (use it for my WM Awesome) and really like it a lot. I think though it may be more suited to apps as it’s the close to the metal and efficiency we need in routing/vaults etc.

54

@dirvine thanks for answering

So it seems that maidsafe is actually not primarily I/O-bound and the CPU is more busy generally rather than idle? Just trying to make a rough sense of the runtime characteristics.

55

I would think more IO but we will see more as we profile on launch. Vaults tend to be very efficient so we will see.

56

If you’re on an older Atom or ARM CPU, especially ones with < 800Mhz RAM and no AES offload you’ll be CPU bound.

With >= 800Mhz RAM and AES offload it’ll be more i/o bound depending on your filing system and network config.

Niall

3 Likes
Parallella - Is this the future for Farming?
57

I guess we’re focusing on Common Lisp here. CL macros don’t do type interference. That’s definitely a plus for C++. I guess it’s less relevant for CL as it is a dynamic language (ignoring optional type declarations for speed). The need for templates in the first place is a non-issue. However the combination of having the full language available at compile time, plus having such a regular language syntax (basically just lists of things) makes for extremely flexible meta-programming with little conceptual overhead. Perhaps that doesn’t fit your definition of power.

Indeed, this is a blessing and a curse. Mostly a curse :smile:
Schemers frown on it and have these things called hygienic macros, which does away with this.

Defgeneric is a part of how CL does methods. Methods are basically decoupled from classes. I guess you could describe them as virtual functions over multiple arguments. So you can specialize methods based on the individual arguments. These don’t necessarily have to be instances.

And now that we’re talking about meta-programming. CL methods, classes and such are themselves instances of metaclasses which are user-modifiable. See ‘The Art of the Metaobject protocol’.

58

Mr. Lowry is apparently a competitor who has hedged his bet with an investment in SAFE. He probably does better financially if SAFE does not work out. If SAFE works OB and Storj look like they will be a in a loser- take-nothing position or approaching it. His conflict of interest and then posting negative remarks look desperate.

The mature Jobs probably had it right with his approach to investors- those with a “wheres my money” attitude need to STFU or remove themselves from the picture- recognizing of course that early investors are the best kind. At this point SAFE is such a light bulb that even if MaidSAFE doesn’t deliver its an idea whose time has come and its a massive success just with the birth of the concept. The interest of the Australian high school kids shows that. It will be done.

2 Likes
59

bottom line is we all know in our hearts that a programming language is as good as its respective community.

60

I am more concerned with efficiency and stability. We spend inordinate amounts of time with build systems/library handling (no modules)/test rigs/benchmarking/packaging and much more (memory management is a huge area to work with and get right, just look at non const shared pointers to non const objects (the default for many programmers to use) and try to make them safe for concurrent access).

We need to have a systems language and c++ is terrific for that, what rust offers is an updated system (languages are tools to me, like a spanner or hammer).

So if our tests show we can spend less time on all the associated work outside actually coding then that is a win (much faster to market with products) and at the same time produce libraries and binaries that are much less (significantly) likely to have memory access type errors then great.

So I am looking for increased development speed at a systems level with the output being highly reliable code and systems.

This arguably can be done in c c++ ada but the associated workload is not to be underestimated. We kinda know as we have done it all so far in c++ and spent the time to ensure it is very high quality and tested and run through sanitisers and valgrind etc. to ensure memory safety, it is a huge task with a decent sized code base. We need to confirm we have enough forward motion as we wish to add more functionality, c++ is very hard and very slow to get right, it needs considerable effort and expertise. Not a bad thing, but some help would be good.

So research and testing is important. I also have looked at go nim etc. as these are developing and every one has the same start problem, there is always initially near zero community members, that is natural.

So the community deserve and should expect us to deliver as efficiently and securely as possible and not wait for indeterminate times for us to get the code right. Imagine a fork of a c++ code base to rust, the fork may have significant forward momentum and rightly so. The core algorithms are what is important and getting those correct is where as close to 100% of Engineers time should be spent in my opinion. If rust helps and is valid then the consideration is very real, if not we have still done our job by checking.

All the core devs want to be happy is a launched solid system (not being launched is the most exhausting thing we encounter), there is zero time to play around with anything unless it may be significant. I hope this is something everyone gets ( I think you seem to, so thanks) as it is important.

None of the core devs have touched Rust as of yet, only me and I like to investigate and make sure all is well. So happy is something we would not even know, (“here is a language that may not help your CV right now but helps the project right now”, perhaps is not making folk happy) I have spent many weeks/months now on NaCi for instance as it appears significant for network systems crypto primitives. There is not a lot wrong in always keeping a weather eye on progress.

8 Likes
61

I know that Go is being used for Docker, which has become very high profile.

I totally hear what you are saying. We have similar conflicts in my day job, albeit with a very different development stack. It is a hard balance to strike and having to make compromises is never nice (but inevitable).

1 Like
62

Well when i figure out the real reason why RUST is being used i hope really badly its because of a big gain :smiley: i hate reading new languages despite how easy they are to read.

63

Testnet3 will be fully c++, so any gain has to be forward looking efficiency, nothing else :wink: