0.7.6 slower than 0.6.1

[dean gaudet]

On Sat, 15 Jun 2002, Spicer, Kevin wrote:

> Its about 1.2 Million at the moment.  but only a relatively small subset of
> these change daily (62000 today).  Obviously these kinds of things are
> largely OS and hardware dependent, but how does these figures sound in
> relation to the time its taking?

a non-trivial amount of your overhead on sol2.6 is going to be needless
atime modifications on the mirror server.  if you can upgrade the mirror
to 7 or 8 (i forget which) you'll be able to mount the mirror with noatime
and eliminate these disk writes.

2.6 can also get into lots of trouble when you've got a lot of filenames
longer than 31 characters in length (the basename without the path).  it
has in kernel caching for filenames up to 31 characters in length, but
when you get a bunch of files longer than that the kernel wastes lots of
i/o re-reading directory contents and such.  i'm pretty sure this is fixed
in 7 as well.

(this last problem shows up a lot for folks using maildir -- the default
filenames are really long...)

if your mirror is a dedicated server doing not much else you could
probably upgrade it without affecting production services ...

as for changes between 0.6 and 0.7 that might affect this -- i haven't
looked, but the problem with long filenames is exacerbated by doing many
syscalls with paths in them.  i know the strace of 0.7.x has a lot of
overkill in file operations.  but i haven't looked at 0.6.x to see if any
more were added.  ("truss" is the closest command to strace on solaris
2.6...)

there's cases where patterns like:

creat()
close()
chown()

can be replaced with

creat()
fchown()
close()

which is generally faster everywhere due to the lack of extra name lookup
(and more secure).  dunno if rdiff-backup has any such opportunities for
improvement.

-dean