SEARCH

Enter your search query in the box above ^, or use the forum search tool.

You are not logged in.

#1 2015-03-31 03:50:39

KrunchTime
#! Die Hard
From: not where I belong
Registered: 2012-03-02
Posts: 3,264

Apt History and Configuration File Backup

It's important to make backup copies of the apt history and configuration files.  Having copies of apt history files is useful for troubleshooting issues surrounding package installs/removals; grep is handy for this purpose.  I recommend making copies of configuration files, containing their default values, that you know you will be changing or prior to making an intial change.  After making a change to a configuraton file and verifying that it works, make a copy of it in case you need to refer/revert back to it.  Finally, also remember to backup your apt history and configuration file backups to an external device or online storage.

I keep the apt history backup files and configuration backup files under separate folders/directories.  Following are the files that I recommend making backup copies of:

/home/<user>/.conkyrc
/home/<user>/.config/openbox/autostart
/home/<user>/.config/openbox/menu.xml
/home/<user>/.config/openbox/rc.xml
/home/<user>/.config/tint2/tintrc
/etc/fstab
/etc/apt/preferences
/etc/apt/sources.list(1)
/var/log/apt/history.log(2)

(1) -- Some Debian-based distros do not use sources.list, but instead use separate source files under /etc/apt/sources.list.d/.  In that case, make backups of each file under /etc/apt/sources.list.d/.
(2) -- I edit the backup copy of this file and remove any entries from previous days or entries that are already contained in a previous backup file.

Following are examples of file backup names that I use.  In each filename below, the last two components are <computer>_<distro> just in case that's not clear.  Also, you'll note that some date values are appended with -01.  This is an incremental number, so if you have more than one backup the same day, the incremental number would indicate that; e.g. -01, -02, -03, etc.

20150330-01_apt_history.log_et2322_crunchbang11
.conkyrc_0default_20150330_et2322_crunchbang11
.conkyrc_20150330-01_et2322_crunchbang11
autostart_openbox_0default_20150330_ux303la_debian8
autostart_openbox_20150330-01_ux303la_debian8
menu.xml_openbox_0default_20150330_et2322_crunchbang11
menu.xml_openbox_20150330-01_et2322_crunchbang11
rc.xml_openbox_0default_20150330_ux303la_debian8
rc.xml_openbox_20150330-01_ux303la_debian8
tintrc_0default_20150330_et2322_crunchbang11
tintrc_20150330-01_et2322_crunchbang11
fstab_0default_20150330_ux303la_debian8
fstab_20150330-01_ux303la_debian8

preferences_0default_20150330_et2322_crunchbang11
preferences_20150330-01_et2322_crunchbang11
-- or --
preferences_apt_0default_20150330_et2322_crunchbang11
preferences_apt_20150330-01_et2322_crunchbang11

sources.list_0default_20150330_ux303la_debian8
sources.list_20150330-01_ux303la_debian8
-- or --
sources.list_apt_0default_20150330_ux303la_debian8
sources.list_apt_20150330-01_ux303la_debian8

Last edited by KrunchTime (2015-03-31 03:52:07)


Linux User #586672
Come and Die -- Kyle Idleman

Offline

Be excellent to each other!

#2 2015-03-31 03:51:42

KrunchTime
#! Die Hard
From: not where I belong
Registered: 2012-03-02
Posts: 3,264

Re: Apt History and Configuration File Backup

Place holder for change history.


Linux User #586672
Come and Die -- Kyle Idleman

Offline

#3 2015-03-31 03:54:31

tknomanzr
#! Die Hard
From: Heavener, OK
Registered: 2014-12-09
Posts: 777

Re: Apt History and Configuration File Backup

This looks like a good candidate for an rsync script. Would be simple enough to sync all of my configs over to my Crunchbang Waldorf box.

Offline

#4 2015-03-31 04:07:00

KrunchTime
#! Die Hard
From: not where I belong
Registered: 2012-03-02
Posts: 3,264

Re: Apt History and Configuration File Backup

Maybe...I never thought of that, although I haven't made the time to learn scripting yet.


Linux User #586672
Come and Die -- Kyle Idleman

Offline

#5 2015-03-31 06:45:21

Head_on_a_Stick
CatMod
From: A world of pure imagination
Registered: 2014-01-21
Posts: 4,797

Re: Apt History and Configuration File Backup

# rsync -aAXv $SOURCE $DESTINATION

https://wiki.archlinux.org/index.php/Fu … with_rsync
wink

Offline

#6 2015-03-31 14:22:55

Sector11
#!'er to BL'er
From: SR11 Cockpit
Registered: 2010-05-05
Posts: 15,667
Website

Re: Apt History and Configuration File Backup

Excellent idea KrunchTime.

I have a rsync script that does a backup of my complete /home to an external HDD but this would be really slick.
Mine sets the /home backup by the month and just does a continuous backup of my data partition to /M5

 31 Mar 15 | 11:17:51 ~
    $ s11bk
Syncing /home     to /media/disk/S11-Mar/
Syncing /media/5/ to /media/disk/M5/

1. Syncing Complete
2. Unmount /media/disk? (Y|n) 
/media/disk Unmounted - Unplug now.
 
 31 Mar 15 | 11:20:23 ~
    $ 

Tomorrow it will automatically set up /S11-Apr

Must check you doing something with your idea.  Will take me forever as I'm not bash inclined.  big_smile
luc basically wrote what I use today.

EDIT:  Maybe not, Thanks HoaS

Last edited by Sector11 (2015-03-31 14:29:36)


·  ↓   ↓   ↓   ↓   ↓   ↓  ·
BunsenLabs Forums now Open for Registration
·  ↑   ↑   ↑   ↑   ↑   ↑  · BL ModSquad

Offline

#7 2015-03-31 15:43:42

tknomanzr
#! Die Hard
From: Heavener, OK
Registered: 2014-12-09
Posts: 777

Re: Apt History and Configuration File Backup

Yeah I should have noted, I have some bash wrappers setup already to backup media. It would be simple enough to fix them up to backup configs. When I get the time, I will possibly post them here. Do note, they are merely wrapper to help my mind with the complexity of an rsync command.

Offline

#8 2015-03-31 20:58:33

KrunchTime
#! Die Hard
From: not where I belong
Registered: 2012-03-02
Posts: 3,264

Re: Apt History and Configuration File Backup

Sector11 wrote:

Excellent idea KrunchTime.

Uh...the scripting idea was tknomanzr's not mine.  My OP was more to give people help/heads up about the importance of backing up configuration and apt history files, which ones, and example file formats.

@HoaS:  I'm not a fan of automated backups.  I'm very good at remembering what files I've worked on and only backup files that I have worked on.  I rarely need an automated backup of parts of, or my whole, HDD, usually when changing the partition structure.  I will take a look at rsync as a tool for that purpose though, since I currently hand backup in those instances, which is somewhat time consuming. wink


Linux User #586672
Come and Die -- Kyle Idleman

Offline

#9 2015-03-31 21:01:08

Head_on_a_Stick
CatMod
From: A world of pure imagination
Registered: 2014-01-21
Posts: 4,797

Re: Apt History and Configuration File Backup

KrunchTime wrote:

@HoaS:  I'm not a fan of automated backups.

Neither am I.

I would never run a backup automatically -- it is far too important for that!

Offline

#10 2015-03-31 21:02:30

KrunchTime
#! Die Hard
From: not where I belong
Registered: 2012-03-02
Posts: 3,264

Re: Apt History and Configuration File Backup

^ Automated backups are good for servers and desktops for those who can't seem to understand the importance of making a backup of a changed file.


Linux User #586672
Come and Die -- Kyle Idleman

Offline

#11 2015-03-31 21:30:29

tknomanzr
#! Die Hard
From: Heavener, OK
Registered: 2014-12-09
Posts: 777

Re: Apt History and Configuration File Backup

The nice thing about rsync is that it can do file diffs to see what needs to be backed up. Then, if I want automation, I just schedule it as a cron job. If not, then I have a couple of simple wrapper scripts in ~/bin that are a tad more mnemonic and memory friendly to run manual backups. I backup music automatically, but need finer grained control over my movies, as I keep my kid's movies separated from mine. That way they are not accidentally subjected to zombie apocalypses, etc.

Backing up configs automatically makes a ton of sense to me, especially regarding how hard it is for me to track all the changes I make for each Live Build I do.

Offline

#12 2015-03-31 21:39:36

damo
#! gimpbanger
From: N51.5 W002.8 (mostly)
Registered: 2011-11-24
Posts: 5,434

Re: Apt History and Configuration File Backup

KrunchTime wrote:

.....  I will take a look at rsync as a tool for that purpose though, since I currently hand backup in those instances, which is somewhat time consuming. wink

As has been said already, rsync is the go-to tool for incremental backups - after the first run it is very fast because only the diffs are copied. It's is very simple to make a wrapper script, although there are probably about a million you can download from the interwebz smile


BunsenLabs Group on deviantArt
damo's gallery on deviantArt
Openbox themes
Forum Moderator smile

Offline

#13 2015-03-31 22:10:05

KrunchTime
#! Die Hard
From: not where I belong
Registered: 2012-03-02
Posts: 3,264

Re: Apt History and Configuration File Backup

Thank you for the additional info, damo.


Linux User #586672
Come and Die -- Kyle Idleman

Offline

#14 2015-03-31 23:34:48

damo
#! gimpbanger
From: N51.5 W002.8 (mostly)
Registered: 2011-11-24
Posts: 5,434

Re: Apt History and Configuration File Backup

KrunchTime wrote:

Thank you for the additional info, damo.

Check out grsync - it is a gui frontend for rsync, in the repos.


BunsenLabs Group on deviantArt
damo's gallery on deviantArt
Openbox themes
Forum Moderator smile

Offline

#15 2015-04-01 00:19:16

KrunchTime
#! Die Hard
From: not where I belong
Registered: 2012-03-02
Posts: 3,264

Re: Apt History and Configuration File Backup

^ Another thank you.


Linux User #586672
Come and Die -- Kyle Idleman

Offline

#16 2015-04-01 01:52:59

tknomanzr
#! Die Hard
From: Heavener, OK
Registered: 2014-12-09
Posts: 777

Re: Apt History and Configuration File Backup

This is just a simple tossed together example of stuff that I would consider important rsynced to my #! desktop over ssh. It would probably be better to make a list of files/directories stored in home of everything I wanted to backup.

#!/bin/bash
	#rsync configs over ssh to backup
	rsync -aAxv ~/.config/compton user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv ~/.config/openbox user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv ~/.config/tint2 user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv ~/.conkyrc user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv ~/conky_clock_rc user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv ~/bin user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv /etc/fstab user@192.168.1.2:~/backup/wtfbox/configs
    rsync -aAxv /etc/apt/preferences user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv /etc/apt/sources.list user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv /var/log/apt/history.log user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv /etc/asound.conf user@192.168.1.2:~/backup/wtfbox/configs
	rsync -aAxv /etc/polkit-1 user@192.168.1.2:~/backup/wtfbox/configs
	
exit 0

You would need ssh setup and the machine you are pushing backups too would need ssh-server installed.
Here's a walkthrough
Be sure to use just keys w/ no passwords.

Finally to automate, schedule it in crontab with crontab -e. The following would run the script every hour. Take out the mailto line if you don't want to deal with the spam in your local mail spool. I usually turn it on for a bit to be sure everything is working.

MAILTO=user@localhost

0 * * * * ~/bin/rsync_wtfbox

Offline

#17 2015-04-01 02:12:38

damo
#! gimpbanger
From: N51.5 W002.8 (mostly)
Registered: 2011-11-24
Posts: 5,434

Re: Apt History and Configuration File Backup

^ Couldn't you use an includes file, which has a list of the filepaths you want to rsync? Then use something like

rsync -av `cat /path/to/file` /destination/

There is also  "--recursive --files-from="


BunsenLabs Group on deviantArt
damo's gallery on deviantArt
Openbox themes
Forum Moderator smile

Offline

#18 2015-04-01 02:46:23

tknomanzr
#! Die Hard
From: Heavener, OK
Registered: 2014-12-09
Posts: 777

Re: Apt History and Configuration File Backup

Yeah I was thinking something like that. The above example is a bit unwieldy and I would like it to be simple enough to configure that it would keep stuff I have found to be a pain to set up backed up for me.

Offline

#19 2015-04-01 03:57:08

damo
#! gimpbanger
From: N51.5 W002.8 (mostly)
Registered: 2011-11-24
Posts: 5,434

Re: Apt History and Configuration File Backup

tknomanzr wrote:

Yeah I was thinking something like that. The above example is a bit unwieldy and I would like it to be simple enough to configure that it would keep stuff I have found to be a pain to set up backed up for me.

I remembered something from when I used grsync a while ago - you can save the rsync setup and use the command elsewhere. It saves all the experimenting with the syntax

A Random Blog wrote:

here: "What makes it really interesting, in my opinion, are session support and scripting assistance. The first feature lets you save in external, plain text files, all the parameters for a certain rsync procedure, and load them automatically later on (grsync also comes with a companion program that runs unattended, grsync-batch).

"Scripting assistance" means that you can use Grsync to quickly create the rsync command that is best for you, and then reuse it elsewhere! To do this, press Alt+R or select "Rsync command line" from the file menu, then copy and paste in a file the text that will appear in a pop-up window. That is a working rsync command that you can study or insert in scripts, to automatize as many operations as possible, even on computers where Grsync is not installed!"


BunsenLabs Group on deviantArt
damo's gallery on deviantArt
Openbox themes
Forum Moderator smile

Offline

#20 2015-04-02 00:54:10

tknomanzr
#! Die Hard
From: Heavener, OK
Registered: 2014-12-09
Posts: 777

Re: Apt History and Configuration File Backup

Ok. Worked on this some and got it working. Thanks for the tip @damo. I didn't quite get it to work that way but this did work, plus copied the directory hierarchy over nicely as well:

#!/bin/bash
	rsync -arAX --files-from=$HOME/.config/backup-conf / user@192.168.1.2:~/backup/wtfbox/configs
exit 0

then the file list at $HOME/.config/backup-conf

/home/tknomanzr/.config/compton/
/home/tknomanzr/.config/openbox/
/home/tknomanzr/.config/tint2/
/home/tknomanzr/.conkyrc
/home/tknomanzr/conky_clock_rc
/home/tknomanzr/bin/
/etc/fstab/
/etc/apt/apt.conf.d/
/etc/apt/sources.list
/var/log/apt/history.log
/etc/asound.conf
/etc/polkit-1/
/etc/systemd/

I can create any arbitrary list I want now. Also, if you want the file list to sit on the host as well, simply put a : in front of the --files-from=/path, like so:

--files-from=:/$HOME/path/to/file

This is setup to push files to the host.

Offline

#21 2015-04-02 03:20:44

johnraff
nullglob
From: Nagoya, Japan
Registered: 2009-01-07
Posts: 4,148
Website

Re: Apt History and Configuration File Backup

I just back up in big chunks:
everything in /etc
everything in /home
most of /data
That just about covers it, except for
/var/log/apt/history.log

/etc isn't really all that big, and if there are some huge files or directories in /home or /data (if you use a /data that is) that you don't want to back up, you can make an "excludes" file to keep them out. I find this exclusive approach a bit easier on the nerves - there's less chance of forgetting to back up something important.

So, fwiw, my rsync line:

sudo rsync -av --exclude-from=$HOME/.config/user/backup_excludes /etc /boot/grub /data /home /media/backup/Desktop-backup

/media/backup is the mounted hard disk.

And, again fwiw, a typical "excludes" file:

/data/.Trash*
/data/bigstuff
/data/john/downloads/transmission
/data/john/downloads/k3b
/data/john/Virtualbox_vms
/home/*/.mozilla/**/Cache
/home/*/.local/share/Trash
/home/*/.thumbnails
/home/*/.cache
/home/*/.gvfs
/home/john/.dropbox-dist*
/home/john/.dropbox/cache/*
/home/john/.wine

Storage space is so cheap these days it's easiest to just throw it all in. As damo said, once you've run it the first time, the following backups will be pretty fast.


John
--------------------
( a boring Japan blog , Japan Links, idle twitterings  and GitStuff )
#! forum moderator    BunsenLabs

Offline

#22 2015-05-19 12:38:04

userx-bw
#! Die Hard
From: stuck in an unknown reality
Registered: 2013-11-19
Posts: 735

Re: Apt History and Configuration File Backup

Head_on_a_Stick wrote:
KrunchTime wrote:

@HoaS:  I'm not a fan of automated backups.

Neither am I.

I would never run a backup automatically -- it is far too important for that!

is there not a way to automatically check when or if the file(s) where last modified? I think that there should be.

Then use that to determine if it should be backed up or not. using the date time stamp for when it was last opened and modified then saved. if current date time stamp is later or not equal to the last saved back up file then back it up with the different name / numbering scheme or just over write it that last back up.

One could (should)  even be able to check all of the other backed up files then program it to either give the OP or just automatically over write which ever one the OP wants to be over written,  if that be the case. So that it too keeps the number of backed up files to a limit.

EDIT: NM - - SPOKE TOO SOON big_smile

tknomanzr wrote:

The nice thing about rsync is that it can do file diffs to see what needs to be backed up.....

Last edited by userx-bw (2015-05-19 12:43:14)


"How can you learn how to fix it, if you don't break it first? :8
"the only way to get away with murder is - by killing time" swp 1997  8o
"A computer is only as smart as the person using it"
"Just plug it in and see if it blows up, if not then take it apart and figure out how it works."

Offline

Board footer

Powered by FluxBB

Copyright © 2012 CrunchBang Linux.
Proudly powered by Debian. Hosted by Linode.
Debian is a registered trademark of Software in the Public Interest, Inc.
Server: acrobat

Debian Logo