Quantcast

Incremental backups stacking up behind long-running job

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Incremental backups stacking up behind long-running job

mailing
Hi guys,

I have a problem about the schedules of long-running jobs.

One of the directories ("archive) I need to backup regularly is about 6
TB and growing.
We therefore decided to go for a full backup once per quarter, a
differential one once per week and an incremental one every day.

The problem I now have is that I configured the job on Monday, so the
incremental job on Monday evening became a full backup which is running
a bit less than three days. Other jobs which would have been due later
on Monday evening and on Tuesday queued up - so far everything works as
expected, but now we are getting towards my problem:

One of the queued backups is the next incremental backup of "archive".
My expectation was that the incremental backup would run only some hours
after the full backup finishes, so the difference is really small and it
only takes some minutes and only requires a small amount of tape
storage. The problem now is that bacula does its check if there already
is a full backup of "archive" available when adding the job to the queue
and not when running it. Since the full backup has not been finished
yet, there is none and bacula turns the second incremental backup (and
probably the third one) into a full backup as well.

I'm currently running bacula 5.2.6, so my question is if anybody knows
a solution to this problem (apart from manually cancelling the queued
incremental jobs) or if an upgrade to bacula 7 might solve the problem.
The upgrade to 7.4 is planned for the future already.

I think, it would be helpful to have some kind of feature which
automatically cancels a job if there still is a previous run of it in
the queue to prevent incremental runs stacking up behind a long-running
job once per month/quarter/year. Is there anything like that in newer
versions of bacula already?

Thanks in advance!

Kind Regards,
Philipp Wagner


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Incremental backups stacking up behind long-running job

Thomas Lohman
> One of the queued backups is the next incremental backup of "archive".
> My expectation was that the incremental backup would run only some hours
> after the full backup finishes, so the difference is really small and it
> only takes some minutes and only requires a small amount of tape
> storage. The problem now is that bacula does its check if there already
> is a full backup of "archive" available when adding the job to the queue
> and not when running it. Since the full backup has not been finished
> yet, there is none and bacula turns the second incremental backup (and
> probably the third one) into a full backup as well.
>
> I'm currently running bacula 5.2.6, so my question is if anybody knows
> a solution to this problem (apart from manually cancelling the queued
> incremental jobs) or if an upgrade to bacula 7 might solve the problem.
> The upgrade to 7.4 is planned for the future already.

I believe that the problem that you're describing is the same one I had
a number of years ago when running 5.2.x.  I had fixed it and submitted
a patch I believe.  So my guess is that this should now be fixed and
should not be an issue in 7.4.x.

http://bugs.bacula.org/view.php?id=1882

In addition, there are options to cancel new jobs if there are already
running jobs, etc.  Please see the following job options

   Allow Duplicate Jobs = yes/no
   Cancel Lower Level Duplicates = yes/no
   Cancel Queued Duplicates = yes/no
   Cancel Running Duplicates = yes/no


--tom


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Incremental backups stacking up behind long-running job

Kern Sibbald
Hello,

I confirm that the Allow Duplicate Jobs patch is and has been for some
time in Bacula.

Best regards,
Kern

On 02/08/2017 01:36 PM, Thomas Lohman wrote:

>> One of the queued backups is the next incremental backup of "archive".
>> My expectation was that the incremental backup would run only some hours
>> after the full backup finishes, so the difference is really small and it
>> only takes some minutes and only requires a small amount of tape
>> storage. The problem now is that bacula does its check if there already
>> is a full backup of "archive" available when adding the job to the queue
>> and not when running it. Since the full backup has not been finished
>> yet, there is none and bacula turns the second incremental backup (and
>> probably the third one) into a full backup as well.
>>
>> I'm currently running bacula 5.2.6, so my question is if anybody knows
>> a solution to this problem (apart from manually cancelling the queued
>> incremental jobs) or if an upgrade to bacula 7 might solve the problem.
>> The upgrade to 7.4 is planned for the future already.
> I believe that the problem that you're describing is the same one I had
> a number of years ago when running 5.2.x.  I had fixed it and submitted
> a patch I believe.  So my guess is that this should now be fixed and
> should not be an issue in 7.4.x.
>
> http://bugs.bacula.org/view.php?id=1882
>
> In addition, there are options to cancel new jobs if there are already
> running jobs, etc.  Please see the following job options
>
>     Allow Duplicate Jobs = yes/no
>     Cancel Lower Level Duplicates = yes/no
>     Cancel Queued Duplicates = yes/no
>     Cancel Running Duplicates = yes/no
>
>
> --tom
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Bacula-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Incremental backups stacking up behind long-running job

mailing
Hello,

thanks for your fast response.
Question is answered, problem should be solved - perfect!

Best regards,
Philipp


Am 08.02.2017 um 14:05 schrieb Kern Sibbald:

> Hello,
>
> I confirm that the Allow Duplicate Jobs patch is and has been for some
> time in Bacula.
>
> Best regards,
> Kern
>
> On 02/08/2017 01:36 PM, Thomas Lohman wrote:
>>> One of the queued backups is the next incremental backup of "archive".
>>> My expectation was that the incremental backup would run only some hours
>>> after the full backup finishes, so the difference is really small and it
>>> only takes some minutes and only requires a small amount of tape
>>> storage. The problem now is that bacula does its check if there already
>>> is a full backup of "archive" available when adding the job to the queue
>>> and not when running it. Since the full backup has not been finished
>>> yet, there is none and bacula turns the second incremental backup (and
>>> probably the third one) into a full backup as well.
>>>
>>> I'm currently running bacula 5.2.6, so my question is if anybody knows
>>> a solution to this problem (apart from manually cancelling the queued
>>> incremental jobs) or if an upgrade to bacula 7 might solve the problem.
>>> The upgrade to 7.4 is planned for the future already.
>> I believe that the problem that you're describing is the same one I had
>> a number of years ago when running 5.2.x.  I had fixed it and submitted
>> a patch I believe.  So my guess is that this should now be fixed and
>> should not be an issue in 7.4.x.
>>
>> http://bugs.bacula.org/view.php?id=1882
>>
>> In addition, there are options to cancel new jobs if there are already
>> running jobs, etc.  Please see the following job options
>>
>>     Allow Duplicate Jobs = yes/no
>>     Cancel Lower Level Duplicates = yes/no
>>     Cancel Queued Duplicates = yes/no
>>     Cancel Running Duplicates = yes/no
>>
>>
>> --tom
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Bacula-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Bacula-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Loading...