Differential backup compares not with last full backup.

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Differential backup compares not with last full backup.

Gi Dot
Hi all,

The other day I was running a full and differential backup back to back on a database server, and I was surprised to see the size of both backups are not that far off. The full backup size was about 37G, and the differential backup that runs after that was about 30G. Excerpt from the log:

  JobId:                  521
  Job:                    DatabaseBackup_Server1.2016-11-19_07.49.07_55
  Backup Level:           Full

  JobId:                  522
  Job:                    DatabaseBackup_Server1.2016-11-19_07.59.04_02
  Backup Level:           Differential, since=2016-10-23 23:00:02

Seems like the differential backup was done by comparing the differences with the full backup done back in October 23rd, instead of the last full backup done just before. Is this even correct? The full backup that it compares with was the last full job before I ran the full backup on Nov 19th (jobid 521).

# The full job:
*llist jobid=521
           jobid: 521
             job: DatabaseBackup_Server1.2016-11-19_07.49.07_55
            name: DatabaseBackup_Server1
     purgedfiles: 0
            type: B
           level: F
        clientid: 3
            name: srv1
       jobstatus: T
       schedtime: 2016-11-19 07:48:48
       starttime: 2016-11-19 07:49:11
         endtime: 2016-11-19 09:43:44
     realendtime: 2016-11-19 09:43:44
        jobtdate: 1,479,519,824
    volsessionid: 8
  volsessiontime: 1,479,370,241
        jobfiles: 3,525
       joberrors: 0
 jobmissingfiles: 0
          poolid: 9
        poolname: ThursdayPool
      priorjobid: 0
       filesetid: 10
         fileset: Full Db Set


 # The differential job
*llist jobid=522
           jobid: 522
             job: DatabaseBackup_Server1.2016-11-19_07.59.04_02
            name: DatabaseBackup_Server1
     purgedfiles: 0
            type: B
           level: D
        clientid: 3
            name: srv1
       jobstatus: T
       schedtime: 2016-11-19 07:58:43
       starttime: 2016-11-19 09:43:45
         endtime: 2016-11-19 11:18:29
     realendtime: 2016-11-19 11:18:29
        jobtdate: 1,479,525,509
    volsessionid: 9
  volsessiontime: 1,479,370,241
        jobfiles: 342
       joberrors: 0
 jobmissingfiles: 0
          poolid: 9
        poolname: ThursdayPool
      priorjobid: 0
       filesetid: 10
         fileset: Full Db Set

# The old full job that it compares with:
*llist jobid=482
           jobid: 482
             job: DatabaseBackup_Server1.2016-10-23_23.00.00_11
            name: DatabaseBackup_Server1
     purgedfiles: 0
            type: B
           level: F
        clientid: 3
            name: srv1
       jobstatus: T
       schedtime: 2016-10-23 23:00:00
       starttime: 2016-10-23 23:00:02
         endtime: 2016-10-24 00:58:01
     realendtime: 2016-10-24 00:58:01
        jobtdate: 1,477,241,881
    volsessionid: 3
  volsessiontime: 1,477,199,060
        jobfiles: 3,559
       joberrors: 0
 jobmissingfiles: 0
          poolid: 5
        poolname: SundayPool
      priorjobid: 0
       filesetid: 10
         fileset: Full Db Set


# List jobs
|   481 | DatabaseBackup_Server1    | 2016-10-23 14:56:29 | B    | I     |       39 |  8,945,914,864 | T         |
|   482 | DatabaseBackup_Server1    | 2016-10-23 23:00:02 | B    | F     |    3,559 | 37,463,094,208 | T         |
|   483 | DatabaseBackup_Server1    | 2016-10-24 23:00:03 | B    | D     |      163 | 11,821,637,264 | T         |
|   484 | DatabaseBackup_Server1    | 2016-10-25 23:00:03 | B    | D     |       53 | 15,810,028,688 | T

----- >8 ----------- some long list of fatal test jobs ------------------------ >8 --------------------
 
|   521 | DatabaseBackup_Server1    | 2016-11-19 07:49:11 | B    | F     |    3,525 | 37,050,151,168 | T         |
|   522 | DatabaseBackup_Server1    | 2016-11-19 09:43:45 | B    | D     |      342 | 30,131,011,200 | T         |
|   523 | DatabaseBackup_Server1    | 2016-11-19 22:00:03 | B    | D     |        0 |              0 | f         |           
|   524 | DatabaseBackup_Server1    | 2016-11-20 22:00:04 | B    | F     |    3,549 | 36,792,979,584 | T         |
|   525 | DatabaseBackup_Server1    | 2016-11-21 22:00:03 | B    | F     |    3,624 | 38,165,345,888 | T         |
|   526 | DatabaseBackup_Server1    | 2016-11-22 22:00:03 | B    | F     |    3,465 | 37,542,860,816 | T         |
+-------+---------------------------+---------------------+------+-------+----------+----------------+-----------+


I can't point out what could have led to this behaviour. Below are some parts of my bacula configuration. Let me know if I missed anything or if I need to provide more information.


# Pool configuration:
Pool {
  Name = ThursdayPool
  Pool Type = Backup
  Recycle = yes
  Recycle Current Volume = Yes
  AutoPrune = yes
  Volume Retention = 3 days
  Maximum Volume Jobs = 3
}


# Schedule configuration:
Schedule {
  Name = "DailyBackup"
  Run = Level=Full Pool=SundayPool Sunday at 22:00
  Run = Level=Full Pool=MondayPool Monday at 22:00
  Run = Level=Full Pool=TuesdayPool Tuesday at 22:00
  Run = Level=Full Pool=WednesdayPool Wednesday at 22:00
  Run = Level=Differential Pool=ThursdayPool Thursday-Saturday at 22:00
}

# Job configuration:
JobDefs {
  Name = "DefaultJob"
  Type = Backup
  Client = srv1
  Pool = Default
  Level = Full
  Storage = TapeDrive
  Messages = Standard
  Priority = 10
  RunBeforeJob = "/etc/bacula/recycleused.sh"     #update volstatus of Used volume to Recycle
  Write Bootstrap = "/var/lib/bacula/%c.bsr"
}


Job {
  Name = "DatabaseBackup_Server1"
  JobDefs = "DefaultJob"
  Client = srv1
  FileSet = "Full Db Set"
  Schedule = "DailyBackup"
  RunBeforeJob = "/usr/lib64/bacula/make_catalog_backup.pl MyCatalog"
  RunAfterJob  = "/usr/lib64/bacula/delete_catalog_backup"
  Enabled = yes
}


Thanks.

--



------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Differential backup compares not with last full backup.

Martin Simmons
>>>>> On Wed, 23 Nov 2016 04:51:57 +0800, Gi Dot said:
>
> Hi all,
>
> The other day I was running a full and differential backup back to back on
> a database server, and I was surprised to see the size of both backups are
> not that far off. The full backup size was about 37G, and the differential
> backup that runs after that was about 30G. Excerpt from the log:
>
>   JobId:                  521
>   Job:                    DatabaseBackup_Server1.2016-11-19_07.49.07_55
>   Backup Level:           Full
>
>   JobId:                  522
>   Job:                    DatabaseBackup_Server1.2016-11-19_07.59.04_02
>   Backup Level:           Differential, since=2016-10-23 23:00:02
>
> Seems like the differential backup was done by comparing the differences
> with the full backup done back in October 23rd, instead of the last full
> backup done just before. Is this even correct? The full backup that it
> compares with was the last full job before I ran the full backup on Nov
> 19th (jobid 521).

I think the problem is that the differential job started while the new full
job was running.  That caused it to choose the old full job for comparison,
even though it didn't back up any files until after the new full job finished
(presumably due to concurrency restrictions).

__Martin

------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Differential backup compares not with last full backup.

Gi Dot
True, I think so too.

On Fri, Nov 25, 2016 at 9:18 PM, Martin Simmons <[hidden email]> wrote:
>>>>> On Wed, 23 Nov 2016 04:51:57 +0800, Gi Dot said:
>
> Hi all,
>
> The other day I was running a full and differential backup back to back on
> a database server, and I was surprised to see the size of both backups are
> not that far off. The full backup size was about 37G, and the differential
> backup that runs after that was about 30G. Excerpt from the log:
>
>   JobId:                  521
>   Job:                    DatabaseBackup_Server1.2016-11-19_07.49.07_55
>   Backup Level:           Full
>
>   JobId:                  522
>   Job:                    DatabaseBackup_Server1.2016-11-19_07.59.04_02
>   Backup Level:           Differential, since=2016-10-23 23:00:02
>
> Seems like the differential backup was done by comparing the differences
> with the full backup done back in October 23rd, instead of the last full
> backup done just before. Is this even correct? The full backup that it
> compares with was the last full job before I ran the full backup on Nov
> 19th (jobid 521).

I think the problem is that the differential job started while the new full
job was running.  That caused it to choose the old full job for comparison,
even though it didn't back up any files until after the new full job finished
(presumably due to concurrency restrictions).

__Martin

------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------

_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users