git and rsync folders

This article will take pieces from other posts I have made at http://www.whiteboardcoder.com/2014/08/using-find-sed-and-xargs-to-replace.html and http://www.whiteboardcoder.com/2014/08/gitignore-and-calculating-tools.html

But, this article will be a little more focused and quick to the point.

I have a folder which has seven subfolders.  I do not want to track any file with git, but I do want to have easy access to rsync each folder to other computers.

Each individual folder will have its own rsync script that can retrieve its contents remotely.   I will use git to make a repository that only contains scripts and the seven folders.

Create the repository

> git init

Update .git/config  (not per say necessary)

> vi .git/config

Here is my config file

[core]         repositoryformatversion = 0         filemode = true         bare = false         logallrefupdates = true         ignorecase = true [core]         packedGitLimit = 128m         packedGitWindowSize = 128m [pack]         deltaCacheSize = 128m         packSizeLimit = 128m         windowMemory = 128m

Create .gitignore file

For this git repository I only want to see my scripts, and the folders they are in.

> vi .gitignore

Place the following in it.

#Ignore Everything * # But not scripts or folders !*.sh !*/

This first ignores everything.  Then it adds two exceptions. Folders and anything ending in *.sh (my scripts)

Create Default script

Create a default script that will be copied to each directory.

> vi .rsync-not-git-DEFAULT.sh

And place the following in it (replacing the highlighted area with your own information)

#!/bin/bash #Remote location url=git.example.com #Check for an override name name="" if [ $2 ] then   name="$2@" fi #======================================= # #loc is remote directory path loc="/not-git/rsync/my_main_folder/" folder=("XXXXX") # #=============================================== flags="-avzr" if [ "$1" == 'push' ] then   echo "Push it"   #Need to make the directories   #for folder   ssh "$name"$url mkdir -p $loc${folder}   rsync $flags * "$name"$url:$loc$folder else   echo "Pull it"   rsync $flags "$name"$url:$loc$folder/* . fi

Create Copy Script

After this is done create a script that will copy this script into each folder, rename it and replace the XXXXX with is parent folder name.

> vi makescripts.sh

And place the following in it.

#!/bin/bash for folder in `find $PWD -maxdepth 1 -mindepth 1 -type d -not -path "*/.git"`; do   cp .rsync-not-git-DEFAULT.sh $folder/.rsync-not-git.sh   pushd .     cd $folder     basename $folder | xargs -I '{}' sed -i 's/XXXXX/{}/g' .rsync-not-git.sh   popd done

This will find all folders with a maxdepth of 1, ignore the .git folder, and place the .rsync-not-git.sh script in each folder.  It will then replace the XXXXX with the folder name.

Create Run All script

Create a run all script that will find all the .rsync-not-git.sh scripts and run them.  (to make one easy push)

> vi runallpush.sh

And place the following in it. (replace the highlighted with your username)

#!/bin/bash for folder in `find $PWD -iname ".rsync-not-git.sh" -exec dirname {} \;`; do   echo $folder   pushd .   cd $folder   bash ./.rsync-not-git.sh push patman   popd done

Rsync all the data

After you have set up the base directory on the server you will rsync to run the runallpush.sh script/

> ./runallpush.sh

Commit git

> git add . > git commit -m "initial commit"

Then on your git server init a bare repo that you can push your repository to.  In this example I am sudo'n to the git user (who does not have a normal shell, so you have to designate it)

> sudo su git -s /bin/bash > cd /git/repos > mkdir project.git > cd project.git > git --bare init

Finally push it up to a remote master server (adjust the url to your remote server and the path)

> git remote add origin git@example.com:/git/repos/project.git > git push origin master

Clone the repo and pull rsync

From another computer clone the repository

> git clone git@example.com:git/repos/project.git

After this head into one of the directories from the command line and pull your data (rsync) edit command for your username.

> ./.rsync-not-git.sh pull patman

I could have written on script to run all the scripts and pull (rsync) all the data down, but that is not my goal here.   My goal is to be able to download a folders contents, remotely, in a pinch.  Otherwise I do not want to take up the hard drive space (Or in my specific case my thumb drive's space)

References