Directory statistics questions...

[Jason Piterak]

Hi Ben,
 
 Some more ideas from a lazy admin...

  I'm trying to put some wrapper scripts around rdiff-backup to make life
simpler... One of the things I'd like my scripts to do is to parse the logs,
and update me if there are any errors during any of the backups (on several
machines...). 
  The backup server is mounting several windows desktops, which may or may
not be there (laptops, etc.)... as well as several Windows and Linux
servers. There are also remote locations to deal with, etc.
  Whet I'm trying to get is an overall report as to how the backups went
last night... This means errors, but also things like: 
 o Which machines were successfully backed up?
 o Which machines had errors?
 o How many files were backed up from each machine?
 o How many bytes data?
 o How long did it take?
 o How does this compare to yesterday?
 o How does this compare to an average of the last week?
 o What is the total size of diff files for the rdiff-repository? 
 
  The data that you've created for the repository and each directory is spot
on:
[jpiterak@sasher increments]$ cat
directory_statistics.2002-05-22T07\:34\:13-04\:00.data
TotalFiles 4356
TotalFileSize 133426034
ChangedFiles 25
ChangedFileSize 4096
IncrementFileSize 0

  But I've got some questions... The information in the directory ststistics
files is perfect, but they don't seem to work as I would expect:

  Let's say I create a new backup, then add two directories, backup again
and delete the two directories I added...
  
  What I get are two statistics files, which appear to show data for the
first two backups (not the current one...) - see below. This is useful...
but not as useful as current statistics!

  What would be most useful would be:
  o Statistics for the full current repository get written to the end of the
log/stdout. 
     and...
  o Current statistics files are written for each directory, even on the
first iteration of a backup repository (eg: the increments folder is created
with the first backup [rather than the second...], and directory_statistics
files are written there...)

  Also... Digging into the files themselves:
[jpiterak@sasher  downloads]$ cat
/home/jpiterak/doc-backup/rdiff-backup-data/increments/directory_statistics.
2002-05-22T08\:07\:52-04\:00.data
TotalFiles 4356
TotalFileSize 133426034
ChangedFiles 25
  ---> Hmmm... This is the first backup... Where does the changed # come
from?
ChangedFileSize 4096
  ---> Ditto...
IncrementFileSize 0

[jpiterak@sasher  downloads]$ cat
/home/jpiterak/doc-backup/rdiff-backup-data/increments/directory_statistics.
2002-05-22T08\:15\:28-04\:00.data
TotalFiles 4380
TotalFileSize 136914994
ChangedFiles 25
   ---> Nope, only 16 files changed, in 5 directories...
ChangedFileSize 3493056
   ---> This is about right.
IncrementFileSize 1474433


  Anyway, thanks for all the work! This is really shaping into a great
product.

--Jason



-------------------------
[jpiterak@sasher  downloads]$ ll
/home/jpiterak/doc-backup/rdiff-backup-data/increments
total 136
...
-rw-------    1 jpiterak jpiterak       97 May 22 08:16
directory_statistics.2002-05-22T08:07:52-04:00.data
-rw-------    1 jpiterak jpiterak      106 May 22 08:21
directory_statistics.2002-05-22T08:15:28-04:00.data
...

[jpiterak@sasher  downloads]$ cat
/home/jpiterak/doc-backup/rdiff-backup-data/increments/directory_statistics.
2002-05-22T08\:07\:52-04\:00.data
TotalFiles 4356
TotalFileSize 133426034
ChangedFiles 25
ChangedFileSize 4096
IncrementFileSize 0

[jpiterak@sasher downloads]$ cat
/home/jpiterak/doc-backup/rdiff-backup-data/increments/directory_statistics.
2002-05-22T08\:15\:28-04\:00.data
TotalFiles 4380
TotalFileSize 136914994
ChangedFiles 25
ChangedFileSize 3493056
IncrementFileSize 1474433
[jpiterak@sasher  downloads]$

[jpiterak@sasher  downloads]$ ll
/home/jpiterak/doc-backup/rdiff-backup-data/
total 7304
drwx------    3 jpiterak jpiterak     4096 May 22 08:21 ./
drwxr-xr-x   31 jpiterak jpiterak     4096 May 22 08:19 ../
-rw-------    1 jpiterak jpiterak  7454658 May 22 08:21 backup.log
-rw-------    1 jpiterak jpiterak        0 May 22 08:21
current_mirror.2002-05-22T08:20:10-04:00.data
drwx------   32 jpiterak jpiterak     4096 May 22 08:21 increments/
-rwxr-xr-x    1 jpiterak jpiterak        0 May 22 07:41
increments.2002-05-22T08:07:52-04:00.dir*
-rwxr-xr-x    1 jpiterak jpiterak        0 May 22 08:15
increments.2002-05-22T08:15:28-04:00.dir*

---
Jason Piterak
System Administrator
CIS Technical Services
33 Main St., Suite 302
Nashua, NH 03064
(603) 889-4684 - FAX (603) 889-0534