Building a Non-Dynamically Linked Executable of safe_vault

Development
1

I’ve made a bit of progress on this but I’m still not there.

Here’s what I want. I want to compile a safe_vault binary that does this:

$ ldd safe_vault
        not a dynamic executable

“ldd” is a utility to list the dynamic libraries (runtime dependencies) of a binary.

So the binary above could be run on any Linux computer, because it has the dependencies (static libraries) included in the compile image. That’s the ideal anyway.

That binary came from a release.

If I compile the source code accompanying that release I don’t get that result. Here’s the compile process:

$ export PKG_CONFIG_ALLOW_CROSS=1
$ cargo build --release --target=x86_64-unknown-linux-musl

Notes:

  1. Setting the environment variable is required in order to allow the musl option to run.
  2. The musl option is intended to result in the standard C libraries being statically linked into the compile image.

Here’s the resulting files:

$ cd target/x86_64-unknown-linux-musl/release
$ ls -la
    drwxr-xr-x 8 user user    4096 Jun 12 14:01 build                                         
    drwxr-xr-x 2 user user   12288 Jun 12 14:07 deps                                          
    drwxr-xr-x 2 user user    4096 Jun 12 14:01 examples                                      
    -rw-r--r-- 1 user user 1958938 Jun 12 14:08 libsafe_vault.rlib                            
    drwxr-xr-x 2 user user    4096 Jun 12 14:01 native                                        
    -rwxr-xr-x 1 user user 7140240 Jun 12 14:08 safe_vault

If I run ldd:

$ ldd safe_vault
        linux-vdso.so.1 (0x00007ffe8e7fc000)
        libsodium.so.18 => /usr/local/lib/libsodium.so.18 (0x00007f9085022000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9084e05000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9084a5a000)
        /lib/ld64.so.1 => /lib64/ld-linux-x86-64.so.2 (0x00007f9085288000)

Notes:

  1. Libsodium is installed on this computer, and the libsodium.a static library is in the right place:
$ cd /usr/local/lib
$ ls -la
    -rw-r--r--  1 root staff 3940482 Jun 12 03:03 libsodium.a
    -rwxr-xr-x  1 root staff     899 Jun 12 03:03 libsodium.la
    lrwxrwxrwx  1 root staff      19 Jun 12 02:43 libsodium.so -> libsodium.so.18.1.0
    lrwxrwxrwx  1 root staff      19 Jun 12 02:43 libsodium.so.18 -> libsodium.so.18.1.0
    -rwxr-xr-x  1 root staff 2386272 Jun 12 02:43 libsodium.so.18.1.0

Notes:

  1. I’ve read @happybeing 's article on cross-compiling to ARM.
  2. I have tried adding an “attribute” to the toml file, as described in the Rust documentation and all I get is that stupid rlib file, but no statically-linked safe_vault binary:
#[link(name = "sodium", kind = "static")]

Any comments? What am I overlooking?

6 Likes
2

Could you post either a diff or a link to a repo with your code changes? That’d be helpful in helping you!

1 Like
3

Thanks for your reply.

I took the link line out. So, I am working with the unmodified source, either the one accompanying the release I linked to above, or the latest from the maidsafe repo (they both give similar non-results).

So nothing is modified, currently. Whatever it is that is required to achieve the static image is in the compiler switches, environment or system libraries, and not the safe_vault code, and I give all that in the OP.

Stands to reason, anyway, that that would be the case, since the GPL requires the release of the source code along with the binaries. So I have stopped looking into the build.rs, for example, because if a special build.rs is required then it is already there!

EDIT: For completeness I should mention the linker:

$ cc --version
    cc (Debian 4.9.2-10) 4.9.2

and the compiler:

$ rustup show

    installed targets for active toolchain         

    --------------------------------------
    x86_64-unknown-linux-gnu
    x86_64-unknown-linux-musl

    active toolchain
    ----------------

    stable-x86_64-unknown-linux-gnu (default)
    rustc 1.9.0 (e4e8b6668 2016-05-18)
4

Also, I proved my toolchain by running the simple example here of building a portable binary using the musl target (scroll down to the heading, “Example: Building static binaries on Linux”). That means my linker and compiler do indeed work to link the static std library into a compile image for a hello world example.

5

I am not super familiar with static linking and the way that it is implemented in rust. However, I suspect that this is cause by the fact that sodium isn’t a rust library but an interface to a c-version and I don’t think it compile that into it per default – but that line (placed at the proper) location should make the rust compiler include that.

But it might be because libsodium isn’t directly linked in the vault but through a dependency, called sodiumoxide. Have you tried using sodiumoxide there?

Also, I was asking for the code because, as mentioned above, without changes rust won’t necessarily statically link all these c-libs, so you have to configure some of it and I wanted to make sure that configuration is correct.

6

We don’t actually control the linking of the C-library sodium ourselves (as @lightyear mentioned). It’s done in a simple build script of a nested crate of sodiumoxide’s.

You can see there that you have two ways of controlling the linking: either set an env var SODIUM_LIB_DIR to the path of the folder containing the C-library or let pkgconfig do the work.

If you choose option 1, you can set a further env var SODIUM_STATIC to any value, which should cause the static version of sodium to get linked in.

Note that by default, if the linker finds a shared and static version of sodium, it will prefer the shared. Rather than taking any chances, we generally recommend not giving the linker the option of a shared sodium lib, hence our libsodium build instructions contain the --enable-shared=no argument in the ./configure call to avoid this situation.

Ideally your machine should have no instances of libsodium.so (or associated symlinks) to avoid any doubt. However, if you export the env vars as mentioned above, then run cargo clean && cargo build -v you should see -l static=sodium in the build output for libsodium_sys which is a good sign :slight_smile:

ldd will obviously confirm whether this worked or not.

5 Likes
7

Thank you very much for that input. Actually, I had tried almost every one of those things, garnered from my research, but not in a systematic enough way.

Here’s where I’m at now:

I have libsodium statically linked into the target image:

I recompiled libsodium and sodiumoxide with the appropriate environment variables and command-line options set to make the resulting library static. If I then compile with “cargo build --release” (i.e., no musl target), then I get a safe_vault binary for which ldd shows a list of so’s that don’t include libsodium, and the binary is about 2MB bigger. So I’d say it’s in there! I just need to get all the other libraries statically linked in. lf I run “cargo clean” followed by “cargo build --release --target=x86_64-unknown-linux-musl” (which ran previously, although it produced a rlib file along with the safe_vault binary) then it fails with cc can’t link. Rerunning it with --verbose doesn’t give me any obvious clue as to what is happening.

8

I suspect either

  1. You have musl as an .so itself (most likely)

  2. You do not have libsodium compiled with musl as static lib

I think one or both may be the case (the gcc compiled libsodium will not work with a musl compilation)

Musl is great BTW it gets rid of so much nonsense and I feel more secure when not dynamically loading anything.

3 Likes
9

Thanks for the information.

I don’t think musl is present on a vanilla Debian Jessie installation, and I haven’t installed it. My only download of musl was in Rustup when I added it as a target:

    $ rustup target add x86_64-unknown-linux-musl

So I doubt that musl is an so.

However, it was gcc (in the build-essentials apt) that I used to build the libsodium library, so from what you say, that must be the issue.

I have now looked into this. I surmise that the rust musl target was used to build the safe_vault release because it statically links the std libraries into the compile image, thereby making it more portable across different flavors of Linux. Although in practice, so far, I have only see it produce a separate rlib file in the target folder along with the target image. My understanding is that the rlib file is a static library with extra metadata, and it remains for me to find out how to incorporate that into the target image, since that is evidently what the release contains (a static binary and no rlib file there!).

Back to libsodium: Should I use the musl-gcc wrapper to compile libsodium or a musl-native gcc. 1. I’ll do some more digging.

EDIT: I’ll start with the musl-gcc wrapper since it seems the least intrusive.

10

Perhaps build musl from source may be the easiest option (pass static flag) and then use this (with gcc wrapper) to build libsodium etc. rustc should pick this up if it is in the library path afaik and also the compiler (gcc wrapper) in the $PATH and you should be good.

1 Like
11

Thank you very much for that information.

As a result I have now built what appears to be a a correct and portable safe_vault:

  1. “ldd safe_vault” reports that it is not dynamically linked.

  2. It runs on the community1 seed vault as well as client vaults, without any problems.

I’m a bit concerned at the following:

  1. In addition to safe_vault (8.3MB) the build also produced libsafe_vault.rlib (1.9 MB) that doesn’t appear to have any purpose, since the safe_vault binary runs without it.

  2. I used the musl target to compile safe_vault:

     $ cargo build --release --target=x86_64-unknown-linux-musl

…yet it doesn’t use /usr/local/musl/lib/libc.a which is the big std library that is musl’s replacement for the C std library. I know this because I moved it just before running a safe_vault compile and it made no difference. So it seems that it was only used during the libsodium compile. Does that sound right? That the compile isn’t using an std library? I’m scratching my head over this because the musl and C std libraries are incompatible, so that latter could not have slipped in along with the musl compiled libsodium. Is there any way to check what libraries the safe_vault binary has in it?

EDIT: Answering my own question: Of course the musl rust target (x86_64-unknown-linux-musl) doesn’t use /usr/local/musl/lib/libc.a as demonstrated by the hello world example that I linked to which doesn’t require musl to be pre-installed. Therefore the musl rust target has the necessary libraries built-in. Pre-installing musl is required only for the libsodium build, for the musl-gcc wrapper to use. Both sources of musl are required in order to have compatible (i.e., all musl) libraries in the target image; the packager can’t add libraries created by different compilers into the same target image.

1 Like
12

That is fine and nothing for you to worry about. safe_vault is configured to be used as either a standalone app or as a lib by other apps and cargo builds both by default. You can just ignore/discard it without any consequences.

1 Like
13

Thanks, that makes sense, that the rlib is an archived, hence compressed, version of the executable. I’ll write up and post the complete steps for others who might be interested.

4 Likes
14

Here is the complete sequence, as promised! All steps below are well-tested. Please let me know of any omissions or errors.

Requirements


1) A vanilla, Debian-family Linux installation (otherwise you will need to adapt the instructions).

2) The Rustup toolchain manager, which makes the Rust compiler available. To install Rustup, follow the instructions here: https://www.rustup.rs

3) The Rustup musl target. To install:

        $ rustup target add x86_64-unknown-linux-musl

4) The GCC toolchain, and a few other tools. To install:

        $ sudo apt -y install build-essential tar libtool autoconf pkg-config git curl libfreetype6-dev libssl-dev 

5) (optional) For the following steps, I keep my programming projects together in one folder: ~/projects. You might consider configurng your terminal program to open there by default.


Steps for the Build
  1. Install the musl standard libraries and the musl-gcc wrapper, with the terminal commands below. The wrapper is a program that intercepts and swaps out library calls to the gcc compiler, allowing you to compile C source code to a binary that links to the musl standard libraries. We compile musl-gcc with dynamic links disabled because the default behavior of gcc is to prefer dynamic libraries to static libraries if they are available.

NOTE: I will omit the output of commands except where relevant.

    $ cd ~/projects

    $ git clone git://git.musl-libc.org/musl

    $ cd musl

    $ ./configure --disable-shared

    $ make

    $ sudo make install

Make sure that musl-gcc is on your PATH. To do this, put a symbolic link in a folder that is on the PATH (vary these paths if you have a non-Debian distro):

    $ sudo ln -s /usr/local/musl/bin/musl-gcc /usr/local/bin/musl-gcc

(optional) Test the wrapper as well as the gcc toolchain. Just copy/paste the text from “cat” to the second “EOF” inclusive into your terminal and press ENTER. Then run the following two commands.

    $ cat > hello.c <<EOF
    #include <stdio.h>
    int main()
    {
        printf("hello, world!\n");
        return 0;
    }
    EOF

    $ musl-gcc hello.c

    $ ./a.out

…which should produce the result:

    hello, world!

  1. Install the Sodium system library (libsodium):

     $ cd ~/projects

 $ git clone https://github.com/jedisct1/libsodium.git

 $ cd libsodium

 $ ./autogen.sh

 $ CC=musl-gcc ./configure --enable-shared=no    #run configure within the wrapper and with dynamic library creation disabled.

 $ make

 $ sudo make install

  2. Install Sodiumoxide, the Rust binding for Sodium;

     $ cd ~/projects

 $ git clone https://github.com/dnaq/sodiumoxide.git

 $ cd sodiumoxide

 $ export PKG_CONFIG_ALLOW_CROSS=1; export SODIUM_LIB_DIR=/usr/local/lib; export SODIUM_STATIC=static   #set some environment variables

 $ cargo build --release

  3. Build Safe_Vault:

     $ cd ~/projects

 $ git clone http://github.com/maidsafe/safe_vault

 $ cd safe_vault

 $ cargo build --release --target=x86_64-unknown-linux-musl

  4. Test Safe_Vault

     $ cd target/x86_64-unknown-linux-musl/release

 $ ls -la   #let's see what has been produced

 $ ./safe_vault --version 

 $ ldd safe_vault    #probe the binary for dynamic links

The final command should give the result:

    not a dynamic executable

You can now use the safe_vault binary, on any Linux computer, as a subsitute for the binary in the official release.

References:


[Musl Libc FAQ](https://www.musl-libc.org/faq.html "Musl Libc FAQ")

[Musl target tutorial on the Rust Blog](http://blog.rust-lang.org/2016/05/13/rustup.html "Musl target tutorial on the Rust Blog")

[This topic, on the Safenetwork Forum](https://forum.autonomi.community/t/building-a-non-dynamically-linked-executable-of-safe-vault/9811/13 "This topic, on the Safenetwork Forum")
8 Likes
15

As of commit #56fca57 you shouldn’t need to install libsodium. I just successfully tested your instructions (nice work btw) on a VM running Ubuntu 16.04 with the following changes:

  • Only ran sudo apt -y install build-essential git (rustup requires curl, so if that still needs installed then step 4 should probably come before step 2 of the requirements)
  • Omitted steps 2 and 3 of the build instructions, i.e. didn’t build or install libsodium/sodiumoxide.
4 Likes