Bash Website Mirror lftp

hozza · 2012-11-29 15:01:20

Hi #!ers!

I'm trying to make a bash script to mirror/sync a few of my websites. I'm new to bash but have some programming experience. This is what I've got currently...

Arg 1 is the site name (acts as local path too), arg 2 is remote path, arg 3 is lftp bookmark name.

How it works (I think):

Connects and starts a mirror, outputting actions to a log file, if that completes...
make an archive of the site, if that completes...
echo confirmation and send desktop notification

#!/bin/bash
ARCHIVE="$1_$(date +%Y-%m-%d_%H-%M).tar.gz"

if [ -d "/home/hozza/Sites/$1" ]; then
	{
		{
			lftp -e "mirror -e -n -p --parallel=10 --log=/home/hozza/Backups/logs/$1_$(date +%Y-%m-%d_%H-%M).txt $2/ /home/hozza/Sites/$1/; exit top" $3 &&
			cd /home/hozza/Sites/ &&
			tar -czf /home/hozza/Backups/$ARCHIVE $1 &&
			echo "Mirror Log: $1_$(date +%Y-%m-%d_%H-%M).txt" &&
			echo "Archive Created: $ARCHIVE" &&
			notify-send -i gnome-network-properties -t 0 "Mirror & Archive Complete" "$(date +%Y-%m-%d) $(date +%H:%M) \n $1"
		}
	} || notify-send -i error -t 0 "Mirror & Archive Error" "$(date +%Y-%m-%d) $(date +%H:%M) \n $1 \n Something failed! D:"
else
	notify-send -i error -t 0 "Mirror & Archive Error" "$(date +%Y-%m-%d) $(date +%H:%M) \n $1 \n Does not exist."
fi

Is this code any good? :S Be honest, I'm only going to lean bash by constructive criticism.

I would like the "Something Failed D:" error to be more useful, such as showing the output of lftp (e.g. Wrong Password etc...) but would also like lftp to output to the terminal as well as the error is poss?

xaos52 · 2012-11-29 16:13:18

Questions you should answer:
(1) If any of the required arguments are missing, does the script still run? Perhaps you want it to run with default arguments then? Or you want your script to report that required arguments are missing? Prhaps you need to validate your arguments and the number of arguments.
(2) If any command in a series of commands goes awry:
- can the script continue?
- do you give enough feedback to the user regarding which command went wrong and for what reason?
Usually this means testing each command for error status, then deciding if the script can continue or not, log a warning and continue or break off and show the user why.
You can use techniques such as:
lftp -e ... 2> tempfile
rc = $?
if [ $rc -ne 0 ]; then {
log a warning
display (and log) the rc (return code) and return message caught in tempfile
decide if the program can continue or break it off
}

hozza · 2012-12-01 02:39:12

Yes some argument validation would be nice, if any are missing then the script should not run.

If anything fails (except the notify-send command) then the script should stop and provide feedback via the notify-send command.

Can anyone help me understand how to do these? And could you explain your code example?

hozza · 2012-12-01 20:37:20

I gave up tiring to get my head around bash for the moment as I'm on a tight deadline. So I wrote the mirror program I wanted in a language I know, it's only temporary, I know PHP isn't meant for this kind of thing.

Here is is:

<?php
	//init
	parse_str(implode('&', array_slice($argv, 1)), $_GET);
	$error = array();

	$lPrefix 		 = '/home/hozza/Sites/';
	$archiveLocation = '/home/hozza/Backups/';

	$lDir = isset($_GET['l']) ? $_GET['l'] : $error[] = 'Local Directory Required';
	$rDir = isset($_GET['r']) ? $_GET['r'] : $error[] = 'Remote Directory Required';
	$bookmark = isset($_GET['b']) ? $_GET['b'] : $error[] = 'lftp Bookmark Required';

	//Check for args
	if(count($error) == 0) {
		$archiveName = end(explode('/', $lDir)) . '_' . date('Y-m-d_H-i');

		//Validate local dir
		if(is_dir($lPrefix . $lDir)) {
			
			//preserve Sublime Text 2 config SFTP files
			$ST2_SFTP_conf = false;
			if(file_exists($lPrefix . $lDir . '/sftp-config.json')) {
				$ST2_SFTP_conf = file_get_contents($lPrefix . $lDir . '/sftp-config.json');
				unlink($lPrefix . $lDir . '/sftp-config.json');
			}

			//Start mirror
			$lftOutput = explode("\n", shell_exec('lftp -e "mirror -e -p --parallel=10 --log=' . $archiveLocation . 'logs/' . $archiveName . '.txt ' . $rDir . '/ ' . $lPrefix . $lDir . '/; exit top" ' . $bookmark));
			
			//Tar regardless of lftp error or success
			$tarOutput = shell_exec('cd ' . $lPrefix . ' && tar -czf ' . $archiveLocation . $archiveName . '.tar.gz ' . $lDir);
			
			//Output completion or errors
			shell_exec('notify-send -i gnome-network-properties -t 0 "Mirror & Archive Complete" "' . $archiveName . '\n\n' . implode('\n', $lftOutput) . $tarOutput . '"');

			//put back ST2 SFTP conf
			if($ST2_SFTP_conf != false) file_put_contents($lPrefix . $lDir . '/sftp-config.json', $ST2_SFTP_conf);
		}
		else shell_exec('notify-send -i error -t 0 "Mirror & Archive Error" "' . date('Y-m-d') . ' ' . date('H-i') . '\n' . $lDir . ' \n Does not exist! D:"');
	}
	else shell_exec('notify-send -i error -t 0 "Mirror & Archive Error" "' . date('Y-m-d') . ' ' . date('H-i') . '\n' . implode('\n', $error) . '"');
?>

Same args etc as before! smile

~/Programs/mirror.php l=/local/path r=/remote/path b=mysite

Last edited by hozza (2012-12-01 20:46:03)

hozza · 2012-12-07 01:27:49

how would I get lftp to output to terminal?

johnraff · 2012-12-07 17:24:46

lftp is an excellent program, but you might also consider a look at wget for mirroring a site. It has some useful options.

CrunchBang Linux

SEARCH

#1 2012-11-29 15:01:20

Bash Website Mirror lftp

#2 2012-11-29 16:13:18

Re: Bash Website Mirror lftp

#3 2012-12-01 02:39:12

Re: Bash Website Mirror lftp

#4 2012-12-01 20:37:20

Re: Bash Website Mirror lftp

#5 2012-12-07 01:27:49

Re: Bash Website Mirror lftp

#6 2012-12-07 17:24:46

Re: Bash Website Mirror lftp

Board footer