Restore Job RunAfterJob

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Restore Job RunAfterJob

Lloyd Brown
I hope this isn't something obvious I've missed, but ... well ... I
missed it, at least so far.  I'll keep looking but my google-fu seems to
be missing something.

Is there an easy way to run an external script (eg. a "RunAfterJob" or
similar) associated with a *restore* job?  I'm trying to build an
automated test of restores, and I need a way to trigger an external
script, so it can md5/sha256 checksum the restored files, etc.

So far, I've got the actual restore working pretty well, using syntax
like this:

> echo 'restore client=backup-fd where=/path/to/restore/destination/
> restoreclient=backup-fd file=</path/to/file/list/to/restore select
> current yes' | bconsole

At this point, I can script something that can check the files
checksums, and optionally notify us if a restore fails.  I just need a
way to trigger that script to run after the restore job completes.  I
considered just doing a "echo 'wait' | bconsole" or similar, but there's
a possibility of other jobs still running, so I don't necessarily want
to wait for *those*.

Any thoughts/recommendations? I'm coming up empty at the moment.

Thanks,
Lloyd

--
Lloyd Brown
Systems Administrator
Fulton Supercomputing Lab
Brigham Young University
http://marylou.byu.edu


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Restore Job RunAfterJob

Josip Deanovic
On Monday 2017-01-09 09:37:48 Lloyd Brown wrote:

> I hope this isn't something obvious I've missed, but ... well ... I
> missed it, at least so far.  I'll keep looking but my google-fu seems to
> be missing something.
>
> Is there an easy way to run an external script (eg. a "RunAfterJob" or
> similar) associated with a *restore* job?  I'm trying to build an
> automated test of restores, and I need a way to trigger an external
> script, so it can md5/sha256 checksum the restored files, etc.
>
> So far, I've got the actual restore working pretty well, using syntax
>
> like this:
> > echo 'restore client=backup-fd where=/path/to/restore/destination/
> > restoreclient=backup-fd file=</path/to/file/list/to/restore select
> > current yes' | bconsole
>
> At this point, I can script something that can check the files
> checksums, and optionally notify us if a restore fails.  I just need a
> way to trigger that script to run after the restore job completes.  I
> considered just doing a "echo 'wait' | bconsole" or similar, but there's
> a possibility of other jobs still running, so I don't necessarily want
> to wait for *those*.
>
> Any thoughts/recommendations? I'm coming up empty at the moment.
>
> Thanks,
> Lloyd


Hi!

Maybe you could make use of Verify jobs in case you are restoring the
most recent backup and you have fairly recent Verification job run.


--
Josip Deanovic

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Restore Job RunAfterJob

Lloyd Brown
On 01/09/2017 09:46 AM, Josip Deanovic wrote:
Hi!
Maybe you could make use of Verify jobs in case you are restoring the
most recent backup and you have fairly recent Verification job run.

I considered a Verify job.  I haven't done them much, but the trouble is that my boss wants a full end-to-end restore test, not just verifying what's in the catalog.  So, when the description says "In general, verify jobs permit you to compare the contents of the catalog to the file system, or to what was backed up", I didn't dig much further.

Of course, there could certainly be a less-obvious way to apply verify jobs, that I'm not thinking of now.  I'll dig into it further.


-- 
Lloyd Brown
Systems Administrator
Fulton Supercomputing Lab
Brigham Young University
http://marylou.byu.edu

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Restore Job RunAfterJob

Josip Deanovic
On Monday 2017-01-09 09:56:54 Lloyd Brown wrote:

> On 01/09/2017 09:46 AM, Josip Deanovic wrote:
> > Hi!
> > Maybe you could make use of Verify jobs in case you are restoring the
> > most recent backup and you have fairly recent Verification job run.
>
> I considered a Verify job.  I haven't done them much, but the trouble is
> that my boss wants a full end-to-end restore test, not just verifying
> what's in the catalog.  So, when the description says "In general,
> *verify* jobs permit you to compare the contents of the catalog to the
> file system, or to what was backed up", I didn't dig much further.
>
> Of course, there could certainly be a less-obvious way to apply verify
> jobs, that I'm not thinking of now.  I'll dig into it further.

There are several aspects of performing job verification.
The one I would find interesting in your case is the ability to
compare the md5sum of a file on the disk with a value of a md5sum
stored in the catalog.

Anyway it could prove to be hard to achieve because it would require
additional modifications of the bacula configuration (special file set).

So your original question was about running an external script from
the restore job.

I have never needed it but I have tested few minutes ago and I can
confirm that it works as expected.
In my test I have configured a restore job and added a line such as this:
ClientRunAfterJob  = "/etc/bacula/scripts/bla"

It will run on the client after the restore job.



If you are restoring just some subdirectory and not the whole system,
the idea to use the job verification if the job and the file set are
configured correctly should still be doable.

This approach would make it possible to check your files and get the
report about the missing or modified files sent to your e-mail but
it depends on your particular needs.

The idea with the ClientRunAfterJob will work but you will still need
to get the checksums from the database and compare them with your
file list.


--
Josip Deanovic

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Restore Job RunAfterJob

Lloyd Brown
On 01/09/2017 01:11 PM, Josip Deanovic wrote:
>
> I have never needed it but I have tested few minutes ago and I can
> confirm that it works as expected.
> In my test I have configured a restore job and added a line such as this:
> ClientRunAfterJob  = "/etc/bacula/scripts/bla"
>
> It will run on the client after the restore job.
>

Josip,

Just to be clear, are you defining this in the bacula-dir.conf file, or
as a parameter to the "restore" command in the CLI?

Lloyd

--
Lloyd Brown
Systems Administrator
Fulton Supercomputing Lab
Brigham Young University
http://marylou.byu.edu


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Restore Job RunAfterJob

Josip Deanovic
On Tuesday 2017-01-10 08:51:15 Lloyd Brown wrote:

> On 01/09/2017 01:11 PM, Josip Deanovic wrote:
> > I have never needed it but I have tested few minutes ago and I can
> > confirm that it works as expected.
> > In my test I have configured a restore job and added a line such as
> > this: ClientRunAfterJob  = "/etc/bacula/scripts/bla"
> >
> > It will run on the client after the restore job.
>
> Josip,
>
> Just to be clear, are you defining this in the bacula-dir.conf file, or
> as a parameter to the "restore" command in the CLI?
>
> Lloyd

I have added the line to the restore job resource definition in the
bacula-dir.conf file.

ClientRunAfterJob will execute the defined shell script on the client
while RunAfterJob would execute the defined script on the backup server.


--
Josip Deanovic

------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Loading...