0.7.6 slower than 0.6.1
Spicer, Kevin
Kevin.Spicer@bmrb.co.uk
Sun, 16 Jun 2002 00:56:23 +0100
This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
------_=_NextPart_001_01C214C8.40D1BE5C
Content-Type: text/plain;
charset="iso-8859-1"
I agree, I'd like to upgrade all to sol 8. But unfortunately both are
production servers running mission critical third party software which the
vendors only support under 2.51 and 2.6. Whereas rdiff-backup is 'just'
there to help us get back up quickly should the worst happen at the remote
site (basically saves us the day it would take to get the backup tape to our
HO), so I'm not able to upgrade the OS.
Interesting you mention the atime stuff (it is 7 that adds this option BTW),
I noticed that 0.7.6 doesn't seem to change the atime (or is it changing it
then changing it back - could that be whats slowing it up?)
I'm not that convinced that its all filesystem related though, I've just had
a quick glance at top (rdiff-backup is running at present) the 'top' process
is python (around 80% CPU) cpu states are about 80% user and 5-10% iowait.
Bearing in mind that another machine is performing a remote backup to this
machines tape drive I can't really complain about the iowait stats. [This
is the destination machine, the source machine is much happier]
Undoubtably some of this is due to hardware limitations on the Ultra1.
Filenames are almost all short - usually 8.3 DOS like.
Thanks for your feedback
-----Original Message-----
From: dean gaudet [mailto:dean-list-rdiff-backup@arctic.org]
Sent: 15 June 2002 22:08
To: Spicer, Kevin
Cc: 'rdiff-backup@keywest.stanford.edu'
Subject: RE: 0.7.6 slower than 0.6.1
On Sat, 15 Jun 2002, Spicer, Kevin wrote:
> Its about 1.2 Million at the moment. but only a relatively small subset
of
> these change daily (62000 today). Obviously these kinds of things are
> largely OS and hardware dependent, but how does these figures sound in
> relation to the time its taking?
a non-trivial amount of your overhead on sol2.6 is going to be needless
atime modifications on the mirror server. if you can upgrade the mirror
to 7 or 8 (i forget which) you'll be able to mount the mirror with noatime
and eliminate these disk writes.
2.6 can also get into lots of trouble when you've got a lot of filenames
longer than 31 characters in length (the basename without the path). it
has in kernel caching for filenames up to 31 characters in length, but
when you get a bunch of files longer than that the kernel wastes lots of
i/o re-reading directory contents and such. i'm pretty sure this is fixed
in 7 as well.
(this last problem shows up a lot for folks using maildir -- the default
filenames are really long...)
if your mirror is a dedicated server doing not much else you could
probably upgrade it without affecting production services ...
as for changes between 0.6 and 0.7 that might affect this -- i haven't
looked, but the problem with long filenames is exacerbated by doing many
syscalls with paths in them. i know the strace of 0.7.x has a lot of
overkill in file operations. but i haven't looked at 0.6.x to see if any
more were added. ("truss" is the closest command to strace on solaris
2.6...)
there's cases where patterns like:
creat()
close()
chown()
can be replaced with
creat()
fchown()
close()
which is generally faster everywhere due to the lack of extra name lookup
(and more secure). dunno if rdiff-backup has any such opportunities for
improvement.
-dean
BMRB International
http://www.bmrb.co.uk +44 (0)20 8566 5000
____________________________________________________________
This message (and any attachment) is intended only for the recipient and may
contain confidential and/or privileged material. If you have received this
in error, please contact the sender and delete this message immediately.
Disclosure, copying or other action taken in respect of this email or in
reliance on it is prohibited. BMRB International Limited accepts no
liability in relation to any personal emails, or content of any email which
does not directly relate to our business.
------_=_NextPart_001_01C214C8.40D1BE5C
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
5.5.2653.12">
<TITLE>RE: 0.7.6 slower than 0.6.1 </TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=3D2>I agree, I'd like to upgrade all to sol 8. But =
unfortunately both are production servers running mission critical =
third party software which the vendors only support under 2.51 and =
2.6. Whereas rdiff-backup is 'just' there to help us get =
back up quickly should the worst happen at the remote site (basically =
saves us the day it would take to get the backup tape to our HO), so =
I'm not able to upgrade the OS.</FONT></P>
<P><FONT SIZE=3D2>Interesting you mention the atime stuff (it is 7 that =
adds this option BTW), I noticed that 0.7.6 doesn't seem to change the =
atime (or is it changing it then changing it back - could that be whats =
slowing it up?)</FONT></P>
<P><FONT SIZE=3D2>I'm not that convinced that its all filesystem =
related though, I've just had a quick glance at top (rdiff-backup is =
running at present) the 'top' process is python (around 80% CPU) =
cpu states are about 80% user and 5-10% iowait. Bearing in mind =
that another machine is performing a remote backup to this machines =
tape drive I can't really complain about the iowait stats. [This =
is the destination machine, the source machine is much happier] =
</FONT></P>
<P><FONT SIZE=3D2>Undoubtably some of this is due to hardware =
limitations on the Ultra1.</FONT>
</P>
<P><FONT SIZE=3D2>Filenames are almost all short - usually 8.3 DOS =
like.</FONT>
</P>
<P><FONT SIZE=3D2>Thanks for your feedback</FONT>
</P>
<BR>
<P><FONT SIZE=3D2>-----Original Message-----</FONT>
<BR><FONT SIZE=3D2>From: dean gaudet [<A =
HREF=3D"mailto:dean-list-rdiff-backup@arctic.org">mailto:dean-list-rdiff=
-backup@arctic.org</A>]</FONT>
<BR><FONT SIZE=3D2>Sent: 15 June 2002 22:08</FONT>
<BR><FONT SIZE=3D2>To: Spicer, Kevin</FONT>
<BR><FONT SIZE=3D2>Cc: 'rdiff-backup@keywest.stanford.edu'</FONT>
<BR><FONT SIZE=3D2>Subject: RE: 0.7.6 slower than 0.6.1 </FONT>
</P>
<BR>
<P><FONT SIZE=3D2>On Sat, 15 Jun 2002, Spicer, Kevin wrote:</FONT>
</P>
<P><FONT SIZE=3D2>> Its about 1.2 Million at the moment. but =
only a relatively small subset of</FONT>
<BR><FONT SIZE=3D2>> these change daily (62000 today). =
Obviously these kinds of things are</FONT>
<BR><FONT SIZE=3D2>> largely OS and hardware dependent, but how does =
these figures sound in</FONT>
<BR><FONT SIZE=3D2>> relation to the time its taking?</FONT>
</P>
<P><FONT SIZE=3D2>a non-trivial amount of your overhead on sol2.6 is =
going to be needless</FONT>
<BR><FONT SIZE=3D2>atime modifications on the mirror server. if =
you can upgrade the mirror</FONT>
<BR><FONT SIZE=3D2>to 7 or 8 (i forget which) you'll be able to mount =
the mirror with noatime</FONT>
<BR><FONT SIZE=3D2>and eliminate these disk writes.</FONT>
</P>
<P><FONT SIZE=3D2>2.6 can also get into lots of trouble when you've got =
a lot of filenames</FONT>
<BR><FONT SIZE=3D2>longer than 31 characters in length (the basename =
without the path). it</FONT>
<BR><FONT SIZE=3D2>has in kernel caching for filenames up to 31 =
characters in length, but</FONT>
<BR><FONT SIZE=3D2>when you get a bunch of files longer than that the =
kernel wastes lots of</FONT>
<BR><FONT SIZE=3D2>i/o re-reading directory contents and such. =
i'm pretty sure this is fixed</FONT>
<BR><FONT SIZE=3D2>in 7 as well.</FONT>
</P>
<P><FONT SIZE=3D2>(this last problem shows up a lot for folks using =
maildir -- the default</FONT>
<BR><FONT SIZE=3D2>filenames are really long...)</FONT>
</P>
<P><FONT SIZE=3D2>if your mirror is a dedicated server doing not much =
else you could</FONT>
<BR><FONT SIZE=3D2>probably upgrade it without affecting production =
services ...</FONT>
</P>
<P><FONT SIZE=3D2>as for changes between 0.6 and 0.7 that might affect =
this -- i haven't</FONT>
<BR><FONT SIZE=3D2>looked, but the problem with long filenames is =
exacerbated by doing many</FONT>
<BR><FONT SIZE=3D2>syscalls with paths in them. i know the strace =
of 0.7.x has a lot of</FONT>
<BR><FONT SIZE=3D2>overkill in file operations. but i haven't =
looked at 0.6.x to see if any</FONT>
<BR><FONT SIZE=3D2>more were added. ("truss" is the =
closest command to strace on solaris</FONT>
<BR><FONT SIZE=3D2>2.6...)</FONT>
</P>
<P><FONT SIZE=3D2>there's cases where patterns like:</FONT>
</P>
<P><FONT SIZE=3D2>creat()</FONT>
<BR><FONT SIZE=3D2>close()</FONT>
<BR><FONT SIZE=3D2>chown()</FONT>
</P>
<P><FONT SIZE=3D2>can be replaced with</FONT>
</P>
<P><FONT SIZE=3D2>creat()</FONT>
<BR><FONT SIZE=3D2>fchown()</FONT>
<BR><FONT SIZE=3D2>close()</FONT>
</P>
<P><FONT SIZE=3D2>which is generally faster everywhere due to the lack =
of extra name lookup</FONT>
<BR><FONT SIZE=3D2>(and more secure). dunno if rdiff-backup has =
any such opportunities for</FONT>
<BR><FONT SIZE=3D2>improvement.</FONT>
</P>
<P><FONT SIZE=3D2>-dean</FONT>
</P>
<BR>
<P><FONT SIZE=3D2>BMRB International </FONT>
<BR><FONT SIZE=3D2><A HREF=3D"http://www.bmrb.co.uk" =
TARGET=3D"_blank">http://www.bmrb.co.uk</A> +44 (0)20 8566 5000 </FONT>
</P>
<P><FONT =
SIZE=3D2>____________________________________________________________ =
</FONT>
<BR><FONT SIZE=3D2>This message (and any attachment) is intended only =
for the recipient and may contain confidential and/or privileged =
material. If you have received this in error, please contact the sender =
and delete this message immediately. Disclosure, copying or other =
action taken in respect of this email or in reliance on it is =
prohibited. BMRB International Limited accepts no liability in relation =
to any personal emails, or content of any email which does not directly =
relate to our business.</FONT></P>
</BODY>
</HTML>
------_=_NextPart_001_01C214C8.40D1BE5C--