Memory leaks and Python 2.2c1 (fwd)

[mike wolman]

Hi Ben,

I am also having some memory problems when backing up files when
their size greater than the backup servers memory (256M), i have
just installed Python 2.2c1 but i am still having problems with these
large files a quick snip from top:

25381 root      19   0  205M 205M  1304 R    12.7 82.2   2:03 python

then about 10 secs later rdiff-backup died with:
Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 3760, in ?
    Globals.Main.Main()
  File "/usr/local/bin/rdiff-backup", line 3528, in Main
    self.take_action(rps)
  File "/usr/local/bin/rdiff-backup", line 3504, in take_action
    elif self.action == "backup": self.Backup(rps[0], rps[1])
  File "/usr/local/bin/rdiff-backup", line 3556, in Backup
    else: HighLevel.Mirror(rpin, rpout)
  File "/usr/local/bin/rdiff-backup", line 3096, in Mirror
    DestS.patch_and_finalize(dest_rpath, diffiter)
  File "/usr/local/bin/rdiff-backup", line 3221, in patch_and_finalize
    RORPIter.patchonce(None, dsrp, diff_rorp)
  File "/usr/local/bin/rdiff-backup", line 2392, in patchonce
    Rdiff.patch_fileobj(basisrp, diff_rorp.open("rb"))
  File "/usr/local/bin/rdiff-backup", line 1141, in patch_fileobj
    delta_rp.write_from_fileobj(delta_fileobj)
  File "/usr/local/bin/rdiff-backup", line 2141, in write_from_fileobj
    RPath.copyfileobj(fp, outfp)
  File "/usr/local/bin/rdiff-backup", line 1579, in copyfileobj
    inbuf = inputfp.read(blocksize)
  File "/usr/local/bin/rdiff-backup", line 702, in read
    self.bufferlist = [combined_buffer[real_len:]]
MemoryError
Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 3760, in ?
    Globals.Main.Main()
  File "/usr/local/bin/rdiff-backup", line 3528, in Main
    self.take_action(rps)
  File "/usr/local/bin/rdiff-backup", line 3503, in take_action
    PipeConnection(sys.stdin, sys.stdout).Server()
  File "/usr/local/bin/rdiff-backup", line 1458, in Server
    self.get_response(-1)
  File "/usr/local/bin/rdiff-backup", line 1425, in get_response
    try: req_num, object = self._get()
  File "/usr/local/bin/rdiff-backup", line 1348, in _get
    assert len(header_string) == 9, len(header_string)
AssertionError: 0


Mike.


On Mon, 17 Dec 2001, Jan Kirchhoff wrote:

> On Son, Dez 16, 2001 at 10:51:40 -0800, Ben Escoto wrote:
> >
> > I just noticed that Python 2.2c1 is out.  A number of memory leaks
> > were present in 2.2b2 but fixed in 2.2c1.  Perhaps anyone who found
> > rdiff-backup to use too much memory could upgrade to this version and
> > tell me if the problem persists?
> >
> >
> > --
> > Ben Escoto
>
> I've had the memory-problem with 2.2b2 and I'll test the new Version
> in the next few days, hopefully before christmas. But I'm pretty
> busy right now, so it might take a few days until i can give
> feedback.
>
> Jan
> _______________________________________________
> Rdiff-backup mailing list
> Rdiff-backup@keywest.Stanford.EDU
> http://keywest.Stanford.EDU/mailman/listinfo/rdiff-backup
>