SEARCH

Enter your search query in the box above ^, or use the forum search tool.

You are not logged in.

#1 2014-05-08 14:54:01

rmcellig
#! Die Hard
From: Ottawa, Canada
Registered: 2012-11-15
Posts: 624
Website

rsync clarification [SOLVED]

I was wondering if this is normal behavior.

I am running rsync to sync a folder from one machine to another through ssh. It's working fine. When I view the destination folder showing hidden files, I see multiple occurrences of the same file being downloaded from the target folder. Each occurrence has a different and changing size. Is this how rsync works? The files are being synced but I was just wondering.

Here is an example of what they look like:

file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.5xvIcJ
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.HHnjqY
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.VKSj60
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.5R0RxC
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.iGa9Mb
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.w5R8lt
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.O2CwkM
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.fTh3m0
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.9g9WIT
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.hDTuEi
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.ivaPoP
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.5EWjfz
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.l2BgaN
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.b3M7zy
file:///media/mainserver/lps/B/.berlioz-les-nuits-dete-baker-angel-eminence-ae-34473.flac.aDVINL
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.wWTslw
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.EFDBgJ
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.jegBym
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.viQdFC
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.K5i5fi
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.GCUIN2
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.XLWJ1g
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.xB2l4L
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.wXO9di
file:///media/mainserver/lps/B/.berio-visage-sequenziaiii-circles-cinque-variazioni.flac.6MPZiD

This is the rsync code I am using:

rsync -avr  /media/masterlps/lps/B/ randy@192.168.2.13:/media/randy/mainserver/lps/B

Last edited by rmcellig (2014-05-08 20:51:54)


Cheers Randy
www.mcran.com - my web site
www.chuo.fm - My radio show Sundays  noon-2pm EST or 89.1 fM

Offline

Help fund CrunchBang, donate to the project!

#2 2014-05-08 16:20:49

twoion
Moderator
Registered: 2012-05-11
Posts: 1,648

Re: rsync clarification [SOLVED]

You are correct in your assumption. rsync is far from being a set of options around a native copying algorithm; it's a very interesting and sophisticated program that uses much trickery to do a fine job.

By default, rsync does not update files in place, but rather copies them over into different files and then moves into place when they're ready. This allows for example for ensuring that no incomplete copies remain if the transfer is interrupted. You can disable this with --inplace, but this brings a number of caveats with it (see the man page on --inplace). The fact that multiple temporary 'shadow' files per original file exist is due to rsync's splitting the original file up into blocks for checksumming.

I really recommend to read through rsync's man page, it's really interesting. Here is an outline of the rsync alogrithm and an explanation of the theory behind it.


Tannhäuser ~ {www,pkg,ddl}.bunsenlabs.org/{gitlog,repoidx}

Offline

#3 2014-05-08 16:31:16

rmcellig
#! Die Hard
From: Ottawa, Canada
Registered: 2012-11-15
Posts: 624
Website

Re: rsync clarification [SOLVED]

That is so cool. Thanks for the link and explanation.


Cheers Randy
www.mcran.com - my web site
www.chuo.fm - My radio show Sundays  noon-2pm EST or 89.1 fM

Offline

#4 2014-05-08 18:29:45

Sector11
#!'er to BL'er
From: SR11 Cockpit
Registered: 2010-05-05
Posts: 15,667
Website

Re: rsync clarification [SOLVED]

@ twoion +1 for the thanks on that link it explains why my 'backups' go as fast as they do.

I backup home and my data partition to an external HDD and at time the speed it happens makes me thing - No, it cant be done.

 08 May 14 | 15:03:04 ~
    $ s11bk
Syncing /home     to '/media/sector11/disk/S11-May/'
Syncing /media/5/ to /media/sector11/disk/M5/
Syncing complete - Close Terminal
 
 08 May 14 | 15:03:55 ~
    $ 

51 seconds - not bad: /home = 20GB  /media/5 = 215GB

Only time I find it 'slow' is the first time I run s11bk in a month as it has to create "S11-May", S11-Jun" etc.  But I never really knew WHY.  Thank you! 

@ rmcellig - if I may ... I don't think you need the "r" in your options: rsync -avr  /media/m.... as -a includes the -r option.

-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)

---
Please mark the thread [Solved] if your question/query has been answered satisfactorily.


·  ↓   ↓   ↓   ↓   ↓   ↓  ·
BunsenLabs Forums now Open for Registration
·  ↑   ↑   ↑   ↑   ↑   ↑  · BL ModSquad

Offline

#5 2014-05-09 09:46:55

twoion
Moderator
Registered: 2012-05-11
Posts: 1,648

Re: rsync clarification [SOLVED]

Sector11 wrote:

@ twoion +1 for the thanks on that link it explains why my 'backups' go as fast as they do.

It's an example of Good Code implementing a Good Algorithm. Don't look at its source because it'd be difficult to hold back the blissful tears.


Tannhäuser ~ {www,pkg,ddl}.bunsenlabs.org/{gitlog,repoidx}

Offline

#6 2014-05-09 13:00:04

Sector11
#!'er to BL'er
From: SR11 Cockpit
Registered: 2010-05-05
Posts: 15,667
Website

Re: rsync clarification [SOLVED]

twoion wrote:

It's an example of Good Code implementing a Good Algorithm. Don't look at its source because it'd be difficult to hold back the blissful tears.

Oh I won't look, I couldn't code my way out of a wet paper bag if my life depended on it.  I'd have no idea if it was "Good Code implementing a Good Algorithm" or just something that works  big_smile

So I'll take your word on it and the feeling I get when I use it.  wink


·  ↓   ↓   ↓   ↓   ↓   ↓  ·
BunsenLabs Forums now Open for Registration
·  ↑   ↑   ↑   ↑   ↑   ↑  · BL ModSquad

Offline

#7 2014-05-09 13:11:28

rmcellig
#! Die Hard
From: Ottawa, Canada
Registered: 2012-11-15
Posts: 624
Website

Re: rsync clarification [SOLVED]

I second that! smile


Cheers Randy
www.mcran.com - my web site
www.chuo.fm - My radio show Sundays  noon-2pm EST or 89.1 fM

Offline

Board footer

Powered by FluxBB

Copyright © 2012 CrunchBang Linux.
Proudly powered by Debian. Hosted by Linode.
Debian is a registered trademark of Software in the Public Interest, Inc.
Server: acrobat

Debian Logo