script for determining space consumed by increments
Spicer, Kevin
Kevin.Spicer@bmrb.co.uk
Sun, 12 May 2002 01:25:40 +0100
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
------_=_NextPart_001_01C1F94B.8B7F6550
Content-Type: text/plain;
charset="iso-8859-1"
Thats very interesting. I noticed that my rdiff backup takes a very long
time even when there have been few changes. Unfortunately my OS (Solaris
2.6) doesn't have a noatime (or similar) option - it was introduced in
Solaris 2.7 and this isn't a compelling enough reason to upgrade a
production server! However I set about looking at the atimes in my source
directory and found that the atime of all files had been modified by
rdiff-backup on its last run. I can understand why this would be the case
for directories but not for files, surely rdiff-backup would only need to
examine files whose mtime is different to the remote copy? Also I noted
that the atimes have been copied accross to the mirror (or caused to be set
to the same time) even when there are no increments for that file [is the
the bug fixed below?].
Finally - an idea - if rdiff-backup is only interested in the mtime would
keeping a local cache of the mtimes of files on the mirror to speed up
operation when the destination directory is on a remote machine. I'm not
sure whether this would be worthwhile on small backups, but my directory
contains the best part of a million and a half files - so even a few
milliseconds on each file would save substantial time (my whole rdiff-backup
run takes about 10hours per night).
-----Original Message-----
From: Ben Escoto [mailto:bescoto@stanford.edu]
Sent: 11 May 2002 20:09
To: dean gaudet; rdiff-backup@keywest.Stanford.EDU
Subject: Re: script for determining space consumed by increments
>>>>> "DG" == dean gaudet <dean-list-rdiff-backup@arctic.org>
>>>>> wrote the following on Sat, 11 May 2002 11:27:22 -0700 (PDT)
DG> p.s. on linux, this script, and rdiff-backup will run faster if
DG> you mount your backup partition with the "noatime" mount option.
DG> i know other unixes have similar options, but i don't know the
DG> syntax.
Thanks, good to know and added to my /etc/fstab. There is also a bug
in the last released rdiff-backup where atimes are sometimes preserved
unnecessarily, so maybe the next version will be faster now that
that's fixed.
--
Ben Escoto
BMRB International
http://www.bmrb.co.uk +44 (0)20 8566 5000
____________________________________________________________
This message (and any attachment) is intended only for the recipient and may
contain confidential and/or privileged material. If you have received this
in error, please contact the sender and delete this message immediately.
Disclosure, copying or other action taken in respect of this email or in
reliance on it is prohibited. BMRB International Limited accepts no
liability in relation to any personal emails, or content of any email which
does not directly relate to our business.
------_=_NextPart_001_01C1F94B.8B7F6550
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
5.5.2653.12">
<TITLE>RE: script for determining space consumed by increments </TITLE>
</HEAD>
<BODY>
<BR>
<P><FONT SIZE=3D2>Thats very interesting. I noticed that my rdiff =
backup takes a very long time even when there have been few =
changes. Unfortunately my OS (Solaris 2.6) doesn't have a noatime =
(or similar) option - it was introduced in Solaris 2.7 and this isn't a =
compelling enough reason to upgrade a production server! However =
I set about looking at the atimes in my source directory and found that =
the atime of all files had been modified by rdiff-backup on its last =
run. I can understand why this would be the case for directories but =
not for files, surely rdiff-backup would only need to examine files =
whose mtime is different to the remote copy? Also I noted that =
the atimes have been copied accross to the mirror (or caused to be set =
to the same time) even when there are no increments for that file [is =
the the bug fixed below?].</FONT></P>
<P><FONT SIZE=3D2>Finally - an idea - if rdiff-backup is only =
interested in the mtime would keeping a local cache of the mtimes of =
files on the mirror to speed up operation when the destination =
directory is on a remote machine. I'm not sure whether this would =
be worthwhile on small backups, but my directory contains the best part =
of a million and a half files - so even a few milliseconds on =
each file would save substantial time (my whole rdiff-backup run takes =
about 10hours per night).</FONT></P>
<BR>
<P><FONT SIZE=3D2>-----Original Message-----</FONT>
<BR><FONT SIZE=3D2>From: Ben Escoto [<A =
HREF=3D"mailto:bescoto@stanford.edu">mailto:bescoto@stanford.edu</A>]</F=
ONT>
<BR><FONT SIZE=3D2>Sent: 11 May 2002 20:09</FONT>
<BR><FONT SIZE=3D2>To: dean gaudet; =
rdiff-backup@keywest.Stanford.EDU</FONT>
<BR><FONT SIZE=3D2>Subject: Re: script for determining space consumed =
by increments </FONT>
</P>
<BR>
<P><FONT SIZE=3D2>>>>>> "DG" =3D=3D dean =
gaudet <dean-list-rdiff-backup@arctic.org></FONT>
<BR><FONT SIZE=3D2>>>>>> wrote the following on Sat, 11 =
May 2002 11:27:22 -0700 (PDT)</FONT>
</P>
<P><FONT SIZE=3D2> DG> p.s. on linux, this script, and =
rdiff-backup will run faster if</FONT>
<BR><FONT SIZE=3D2> DG> you mount your backup partition with =
the "noatime" mount option.</FONT>
<BR><FONT SIZE=3D2> DG> i know other unixes have similar =
options, but i don't know the</FONT>
<BR><FONT SIZE=3D2> DG> syntax.</FONT>
</P>
<P><FONT SIZE=3D2>Thanks, good to know and added to my =
/etc/fstab. There is also a bug</FONT>
<BR><FONT SIZE=3D2>in the last released rdiff-backup where atimes are =
sometimes preserved</FONT>
<BR><FONT SIZE=3D2>unnecessarily, so maybe the next version will be =
faster now that</FONT>
<BR><FONT SIZE=3D2>that's fixed.</FONT>
</P>
<BR>
<P><FONT SIZE=3D2>--</FONT>
<BR><FONT SIZE=3D2>Ben Escoto</FONT>
</P>
<BR>
<P><FONT SIZE=3D2>BMRB International </FONT>
<BR><FONT SIZE=3D2><A HREF=3D"http://www.bmrb.co.uk" =
TARGET=3D"_blank">http://www.bmrb.co.uk</A> +44 (0)20 8566 5000 </FONT>
</P>
<P><FONT =
SIZE=3D2>____________________________________________________________ =
</FONT>
<BR><FONT SIZE=3D2>This message (and any attachment) is intended only =
for the recipient and may contain confidential and/or privileged =
material. If you have received this in error, please contact the sender =
and delete this message immediately. Disclosure, copying or other =
action taken in respect of this email or in reliance on it is =
prohibited. BMRB International Limited accepts no liability in relation =
to any personal emails, or content of any email which does not directly =
relate to our business.</FONT></P>
</BODY>
</HTML>
------_=_NextPart_001_01C1F94B.8B7F6550--