SEARCH

Enter your search query in the box above ^, or use the forum search tool.

You are not logged in.

#1 2012-11-29 15:01:20

hozza
Member
From: /dev/null
Registered: 2012-11-20
Posts: 45
Website

Bash Website Mirror lftp

Hi #!ers!

I'm trying to make a bash script to mirror/sync a few of my websites. I'm new to bash but have some programming experience. This is what I've got currently...

Arg 1 is the site name (acts as local path too), arg 2 is remote path, arg 3 is lftp bookmark name.

How it works (I think):

  1. Connects and starts a mirror, outputting actions to a log file, if that completes...

  2. make an archive of the site, if that completes...

  3. echo confirmation and send desktop notification

#!/bin/bash
ARCHIVE="$1_$(date +%Y-%m-%d_%H-%M).tar.gz"

if [ -d "/home/hozza/Sites/$1" ]; then
	{
		{
			lftp -e "mirror -e -n -p --parallel=10 --log=/home/hozza/Backups/logs/$1_$(date +%Y-%m-%d_%H-%M).txt $2/ /home/hozza/Sites/$1/; exit top" $3 &&
			cd /home/hozza/Sites/ &&
			tar -czf /home/hozza/Backups/$ARCHIVE $1 &&
			echo "Mirror Log: $1_$(date +%Y-%m-%d_%H-%M).txt" &&
			echo "Archive Created: $ARCHIVE" &&
			notify-send -i gnome-network-properties -t 0 "Mirror & Archive Complete" "$(date +%Y-%m-%d) $(date +%H:%M) \n $1"
		}
	} || notify-send -i error -t 0 "Mirror & Archive Error" "$(date +%Y-%m-%d) $(date +%H:%M) \n $1 \n Something failed! D:"
else
	notify-send -i error -t 0 "Mirror & Archive Error" "$(date +%Y-%m-%d) $(date +%H:%M) \n $1 \n Does not exist."
fi

Is this code any good? :S Be honest, I'm only going to lean bash by constructive criticism.

I would like the "Something Failed D:" error to be more useful, such as showing the output of lftp (e.g. Wrong Password etc...) but would also like lftp to output to the terminal as well as the error is poss?


facere aut mori conatur

Offline

Be excellent to each other!

#2 2012-11-29 16:13:18

xaos52
The Good Doctor
From: Planet of the @s
Registered: 2011-06-24
Posts: 4,602

Re: Bash Website Mirror lftp

Questions you should answer:
(1) If any of the required arguments are missing, does the script still run? Perhaps you want it to run with default arguments then? Or you want your script to report that required arguments are missing? Prhaps you need to validate your arguments and the number of arguments.
(2) If any command in a series of commands goes awry:
- can  the script continue?
- do you give enough feedback to the user regarding which command went wrong and for what reason?
Usually this means testing each command for error status, then deciding if the script can continue or not, log a warning and continue or break off and show the user why.
You can use techniques such as:
lftp -e ... 2> tempfile
rc = $?
if [ $rc -ne 0 ]; then {
log a warning
display (and log) the rc (return code) and return message caught in tempfile
decide if  the program can continue or break it off
}

Offline

#3 2012-12-01 02:39:12

hozza
Member
From: /dev/null
Registered: 2012-11-20
Posts: 45
Website

Re: Bash Website Mirror lftp

Yes some argument validation would be nice, if any are missing then the script should not run.

If anything fails (except the notify-send command) then the script should stop and provide feedback via the notify-send command.

Can anyone help me understand how to do these? And could you explain your code example?


facere aut mori conatur

Offline

#4 2012-12-01 20:37:20

hozza
Member
From: /dev/null
Registered: 2012-11-20
Posts: 45
Website

Re: Bash Website Mirror lftp

I gave up tiring to get my head around bash for the moment as I'm on a tight deadline. So I wrote the mirror program I wanted in a language I know, it's only temporary, I know  PHP isn't meant for this kind of thing.

Here is is:

<?php
	//init
	parse_str(implode('&', array_slice($argv, 1)), $_GET);
	$error = array();

	$lPrefix 		 = '/home/hozza/Sites/';
	$archiveLocation = '/home/hozza/Backups/';

	$lDir = isset($_GET['l']) ? $_GET['l'] : $error[] = 'Local Directory Required';
	$rDir = isset($_GET['r']) ? $_GET['r'] : $error[] = 'Remote Directory Required';
	$bookmark = isset($_GET['b']) ? $_GET['b'] : $error[] = 'lftp Bookmark Required';

	//Check for args
	if(count($error) == 0) {
		$archiveName = end(explode('/', $lDir)) . '_' . date('Y-m-d_H-i');

		//Validate local dir
		if(is_dir($lPrefix . $lDir)) {
			
			//preserve Sublime Text 2 config SFTP files
			$ST2_SFTP_conf = false;
			if(file_exists($lPrefix . $lDir . '/sftp-config.json')) {
				$ST2_SFTP_conf = file_get_contents($lPrefix . $lDir . '/sftp-config.json');
				unlink($lPrefix . $lDir . '/sftp-config.json');
			}

			//Start mirror
			$lftOutput = explode("\n", shell_exec('lftp -e "mirror -e -p --parallel=10 --log=' . $archiveLocation . 'logs/' . $archiveName . '.txt ' . $rDir . '/ ' . $lPrefix . $lDir . '/; exit top" ' . $bookmark));
			
			//Tar regardless of lftp error or success
			$tarOutput = shell_exec('cd ' . $lPrefix . ' && tar -czf ' . $archiveLocation . $archiveName . '.tar.gz ' . $lDir);
			
			//Output completion or errors
			shell_exec('notify-send -i gnome-network-properties -t 0 "Mirror & Archive Complete" "' . $archiveName . '\n\n' . implode('\n', $lftOutput) . $tarOutput . '"');

			//put back ST2 SFTP conf
			if($ST2_SFTP_conf != false) file_put_contents($lPrefix . $lDir . '/sftp-config.json', $ST2_SFTP_conf);
		}
		else shell_exec('notify-send -i error -t 0 "Mirror & Archive Error" "' . date('Y-m-d') . ' ' . date('H-i') . '\n' . $lDir . ' \n Does not exist! D:"');
	}
	else shell_exec('notify-send -i error -t 0 "Mirror & Archive Error" "' . date('Y-m-d') . ' ' . date('H-i') . '\n' . implode('\n', $error) . '"');
?>

Same args etc as before! smile

~/Programs/mirror.php l=/local/path r=/remote/path b=mysite

Last edited by hozza (2012-12-01 20:46:03)


facere aut mori conatur

Offline

#5 2012-12-07 01:27:49

hozza
Member
From: /dev/null
Registered: 2012-11-20
Posts: 45
Website

Re: Bash Website Mirror lftp

how would I get lftp to output to terminal?


facere aut mori conatur

Offline

#6 2012-12-07 17:24:46

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 3,030
Website

Re: Bash Website Mirror lftp

lftp is an excellent program, but you might also consider a look at wget for mirroring a site. It has some useful options.


John
--------------------
( a boring Japan blog , idle twitterings  and GitStuff )
#! forum moderator

Offline

Board footer

Powered by FluxBB

Copyright © 2012 CrunchBang Linux.
Proudly powered by Debian. Hosted by Linode.
Debian is a registered trademark of Software in the Public Interest, Inc.

Debian Logo