BASH script to clone & update MAID repositories from github

Development
1

I MAID a little script. – NEW VERSION.

Also on git: Script for pulling, building and testing MaidSafe code · GitHub

EDIT: Script now logs everything (including errors). You have the option to build and test.
EDIT: updated to add cargo clean and “export RUST_TEST_THREADS=1” - thanks to @eggmunkee

Thanks to everyone who has contributed, especially @philip_rhoades

I’ve added bits from everywhere to make it more functional. If you use ‘roxterm’ the display will be via terminal tabs instead of new terminal windows - or you can set for logOnly and not have any new tabs or windows.

All logs are located in → installDir/logs//logfiles

logfiles are compiled into one file for either cloning or updates and broken down into separate files for building or testing of each library.

Instructions for use:

  1. open a blank text document, cut/paste code below.
  2. edit code set installDir
  3. set-up proxy if you have one
  4. save the file where-ever convenient e.g. ~/filename or /usr/local/bin/filename or /opt/filename
  5. open a terminal and go to the folder you saved the file and type sudo chmod +x filename
  6. run the program by going to the folder and typing → ./filename or just filename if saved in the $PATH
#!/bin/bash

######################################## - USER DEFINITIONS  
# if you use a proxy add it here otherwise leave blank
gitProxy=
# make sure there is a slash at the end of your installDir definition
installDir="/opt/safenet/src/"
# do you want to build and test? Leave blank if not
# show build and test output via "xterm" (windows), 
# "roxterm" (tabs), or "logOnly" --> note case
buildTest="xterm"

######################################## - PROGRAM DEFINITIONS  
safeRepo="https://github.com/maidsafe/"
libraries="crust routing maidsafe_client maidsafe_types maidsafe_vault maidsafe_nfs"
logDir="${installDir}logs/$(date +"%T")/"


######################################## - MAIN
if [ ${gitProxy} ]; then
    http_proxy=${gitProxy} && HTTP_PROXY=$http_proxy
    export http_proxy HTTP_PROXY
    git config --global http.proxy $HTTP_PROXY
fi

[ ! -d ${installDir} ] && mkdir ${installDir} && cd ${installDir}
[ ! -d ${installDir}logs/ ] && mkdir ${installDir}logs/
mkdir ${logDir}

for library in $libraries; do
    subFolder=${installDir}${library}
    if [ ! -d ${subFolder} ]; then
        git clone ${safeRepo}${1} |& tee -a ${logDir}cloning.log
        cd ${subFolder}
    else
        cd ${subFolder}
        git pull origin master |& tee -a ${logDir}updating.log
    fi
    cargo clean
    export RUST_TEST_THREADS=1
    [ ${buildTest} == "xterm" ] && xterm -hold -title ${library} -e "bash -c 'cargo build |& tee -a ${logDir}building-${library}.log; cargo test |& tee -a ${logDir}testing-${library}.log'" &
    [ ${buildTest} == "roxterm" ] && roxterm --tab -f -n ${library} -e "bash -c 'cargo build |& tee -a ${logDir}building-${library}.log; cargo test |& tee -a ${logDir}testing-${library}.log'" &
    [ ${buildTest} == "logOnly" ] && cargo build > ${logDir}building-${library}.log 2>&1; cargo test > ${logDir}testing-${library}.log 2>&1 &
done

exit 0
6 Likes
2

Wow, that is handy! Thanks!

3

That’s so much nicer than my script :smile:

I’d be interested to know if this always works once the directories exist, because I’ve change my script to delete and then re clone.

I think I was using “git update”, and it wasn’t always enough. I’m a git novice!

4

When I ran @TylerAbeoJordan’s script again my machine complained of the folders already existing…

5

That’s not the issue I had - I was experiencing a build issue that wasn’t fixed by a “git update”, so I forced a new “git clone” and that has always worked.

6

I was hoping Webhooks could be configured to pull updates automatically when changes occur…if any brains here could check it out?

7

It should always work! - it runs the command ‘git pull origin master’ from within the subfolder of each repository and this updates the changed files.

Very strange! Can you copy the exact error message for me?

If the subfolders exist it goes into the directory and does a ‘git pull origin master’ command which just updates changed files only. So saves bandwidth. I have fully tested it on my machine and is working, so maybe are machines are configured differently? – you might try changing the subfolder line:

from: subFolder={parentFolder}{1}

to: subFolder=“{parentFolder}{1}”

Depending on how BASH is configured, it may have be picky about little things like that. Any feedback on your errors is appreciated.

Wow, I’m not sure how to do that. I set this script to run every night - cron.

Cheers

9

Maybe we can wire this into @happybeing brain, so he doesn’t miss a thing :smile:

1 Like
10

Just another thought …

make sure the parentFolder ends with a ‘/’

e.g.: parentFolder=‘/home/MAIDSAFE/’

I should have put that into the instructions, will do so now.

This is probably the source of your error IMO, when I go through the logic without the slash, I get your error too.

11

I got tripped up for a while with the errors reported because I read this as a relative path rather than a full, absolute path:

I have also added “cd ${subFolder}; cargo build” inside the pull fn at the end as well as some other echos so I can better see what is going on.

Thanks for the script!

Phil.

12

I’ll try to remember to upload a script to pull, and build all projects when I get home, in case anyone is interested. Not all of the crates are building without errors at the moment, but I think it is working correctly.

13

@eggmunkee,

See here:

for this:

  • I should have also posted here at here at the same time I guess . .
14

Btw the libraries such as maidsafe_client which had maidsafe_ in their repo names have been renamed to safe_client …

This is a significant change with not just updates to the repo name but also related CI systems / crates.io and few other places. @Ross is currently working on this change and it should be complete soon. Just something to watch out and hopefully have updated in your scripts too for cloning in the future.

1 Like
15

@philip_rhoades Thanks, I will try that script out! I’m using LinuxMint 17.2 at home.