[ILUG] rsync over ssh problem

[kevin]

On Wed, 2010-08-18 at 11:04 +0100, Brian O'Mahony wrote:
> FYI I have been having this issue sync'ing data over an MPLS between Dublin & Bangalore on a nightly basis. It only happens on the first data transfer of the night, and things are quiet (ie its 5am BLR time, and 1am Irish time, so there should be no one in the office.)
> 
> I would be very interested if you find a solution. To be honest I have ignored this for a good few months (I just scheduled an extra data transfer a minute before all the rest kick off that I know will fail). 
> B
> 
> -----Original Message-----
> From: ilug-bounces at linux.ie [mailto:ilug-bounces at linux.ie] On Behalf Of kevin
> Sent: Wednesday, August 18, 2010 10:00 AM
> To: Irish Linux Users Group
> Subject: [ILUG] rsync over ssh problem
> 
> Part of our backup routine is a cron job that runs rsync over ssh to an
> offsite computer every night. Recently I have noticed that it fails
> about 1/2 of the time, not always and a failed rysnc one night might
> work fine the next night.
> 
> The error we get is:
> 
> Write failed: Broken pipe
> rsync: connection unexpectedly closed (13729658 bytes received so far)
> [receiver]
> rsync error: error in rsync protocol data stream (code 12) at io.c(601)
> [receiver=3.0.7]
> rsync: connection unexpectedly closed (783 bytes received so far)
> [generator]
> rsync error: unexplained error (code 255) at io.c(601) [generator=3.0.7]
> 
> I have the "KeepAlive Yes" option in the sshd_config file
> 
> rsync options used in the batch file are:
> 
> rsync -e ssh -avzxHS --numeric-ids --del --delete-excluded
> --exclude-from "/home/cybercolloids/exclusions.txt" --exclude ".gvfs"
> 
> Looking at the log file the only thing that looks odd is that the
> connection fails when transferring large image files - a few Mb
> 
> Any ideas?
> 
> Thanks
> 
> Kevin.
> 

An interesting point - I actually have a batch file that runs through a
series of rsync commands - one for each partition on the raid array.
However the first is the /home directory which is the one that changes
most often and is the most important.

I will also swap the commands over so that rsync /home is further down
the list and see if that helps.

Kevin.