How to build tray-monitor

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

How to build tray-monitor

Jason Voorhees
Hello guys:

I've built Bacula 7.4.0 but I have no idea how to build the
tray-monitor application. I can't find any arguments in ./configure
--help to achieve it even when I found a tray-monitor directory in the
contents of the tarball.

Hope someone can give me some ideas.

Thanks in advance.

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to build tray-monitor

Kern Sibbald
Hello Jason,

cd <bacula-source>
./configure <your options>
make    [or "make -j9" goes faster]

[I assume you have done the above, but if not do it]

cd src/qt-console/tray-monitor
make

If I am not mistaken the binary will be in bacula-tray-monitor. You must
then create a configuration file for it that is valid
(tray-monitor.conf) and execute it.

I have never tried it, so I am not sure it will work.  Feedback would be
appreciated.

Best regards,
Kern

On 04/16/2016 01:06 AM, Jason Voorhees wrote:

> Hello guys:
>
> I've built Bacula 7.4.0 but I have no idea how to build the
> tray-monitor application. I can't find any arguments in ./configure
> --help to achieve it even when I found a tray-monitor directory in the
> contents of the tarball.
>
> Hope someone can give me some ideas.
>
> Thanks in advance.
>
> ------------------------------------------------------------------------------
> Find and fix application performance issues faster with Applications Manager
> Applications Manager provides deep performance insights into multiple tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> _______________________________________________
> Bacula-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to build tray-monitor

Jason Voorhees
Thank you Kern.

Yes, I have tried that but there's no Makefile in tray-monitor
directory so I can't run make under this path :( I'd like to build it
manually using gcc + arguments but I have no idea how to do it.

On Mon, Apr 18, 2016 at 4:04 AM, Kern Sibbald <[hidden email]> wrote:

> Hello Jason,
>
> cd <bacula-source>
> ./configure <your options>
> make    [or "make -j9" goes faster]
>
> [I assume you have done the above, but if not do it]
>
> cd src/qt-console/tray-monitor
> make
>
> If I am not mistaken the binary will be in bacula-tray-monitor. You must
> then create a configuration file for it that is valid (tray-monitor.conf)
> and execute it.
>
> I have never tried it, so I am not sure it will work.  Feedback would be
> appreciated.
>
> Best regards,
> Kern
>
>
> On 04/16/2016 01:06 AM, Jason Voorhees wrote:
>>
>> Hello guys:
>>
>> I've built Bacula 7.4.0 but I have no idea how to build the
>> tray-monitor application. I can't find any arguments in ./configure
>> --help to achieve it even when I found a tray-monitor directory in the
>> contents of the tarball.
>>
>> Hope someone can give me some ideas.
>>
>> Thanks in advance.
>>
>>
>> ------------------------------------------------------------------------------
>> Find and fix application performance issues faster with Applications
>> Manager
>> Applications Manager provides deep performance insights into multiple
>> tiers of
>> your business applications. It resolves application problems quickly and
>> reduces your MTTR. Get your free trial!
>> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
>> _______________________________________________
>> Bacula-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to build tray-monitor

Kern Sibbald
Hello Jason,

Yes, building the tray monitor is fairly complex, and perhaps in the next major version, it will be included in the automatic build because we will have some important new functionality via an enhanced "tray-monitor".

So here is a corrected set of instructions that will probably work:

cd <bacula-source>
./configure --enable-bat <your options>   [note: I have added --enable-bat]
make    [or "make -j9" goes faster]

[I assume you have done the above, but if not do it]

cd src/qt-console/tray-monitor
qmake                                     [note: I have added qmake]
make
I am not sure that the --enable-bat is really needed, but in any case, if you cannot build bat, you will not be able to build the tray monitor both of which require the correct version of Qt.  I use Ubuntu 12.04, which has the correct version of Qt dev library packages for compiling Bat. You need Qt version 4.8.4 or later, and I doubt that bat/tray-monitor will build with Qt >= 5.0.0

That is about all the help I can be.  Sorry.

Best regards,
Kern


On 04/19/2016 02:00 PM, Jason Voorhees wrote:
Thank you Kern.

Yes, I have tried that but there's no Makefile in tray-monitor
directory so I can't run make under this path :( I'd like to build it
manually using gcc + arguments but I have no idea how to do it.

On Mon, Apr 18, 2016 at 4:04 AM, Kern Sibbald [hidden email] wrote:
Hello Jason,

cd <bacula-source>
./configure <your options>
make    [or "make -j9" goes faster]

[I assume you have done the above, but if not do it]

cd src/qt-console/tray-monitor
make

If I am not mistaken the binary will be in bacula-tray-monitor. You must
then create a configuration file for it that is valid (tray-monitor.conf)
and execute it.

I have never tried it, so I am not sure it will work.  Feedback would be
appreciated.

Best regards,
Kern


On 04/16/2016 01:06 AM, Jason Voorhees wrote:
Hello guys:

I've built Bacula 7.4.0 but I have no idea how to build the
tray-monitor application. I can't find any arguments in ./configure
--help to achieve it even when I found a tray-monitor directory in the
contents of the tarball.

Hope someone can give me some ideas.

Thanks in advance.


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications
Manager
Applications Manager provides deep performance insights into multiple
tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


      

    


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to build tray-monitor

Eric Bollengier
In reply to this post by Jason Voorhees
Hello Jason,

The Makefile is probably generated with the "qmake" command, so, try

qmake

make

in the qt-console/tray-monitor directory.

Hope it helps

Best Regards,
Eric

On 04/19/2016 02:00 PM, Jason Voorhees wrote:

> Thank you Kern.
>
> Yes, I have tried that but there's no Makefile in tray-monitor
> directory so I can't run make under this path :( I'd like to build it
> manually using gcc + arguments but I have no idea how to do it.
>
> On Mon, Apr 18, 2016 at 4:04 AM, Kern Sibbald <[hidden email]> wrote:
>> Hello Jason,
>>
>> cd <bacula-source>
>> ./configure <your options>
>> make    [or "make -j9" goes faster]
>>
>> [I assume you have done the above, but if not do it]
>>
>> cd src/qt-console/tray-monitor
>> make
>>
>> If I am not mistaken the binary will be in bacula-tray-monitor. You must
>> then create a configuration file for it that is valid (tray-monitor.conf)
>> and execute it.
>>
>> I have never tried it, so I am not sure it will work.  Feedback would be
>> appreciated.
>>
>> Best regards,
>> Kern
>>
>>
>> On 04/16/2016 01:06 AM, Jason Voorhees wrote:
>>>
>>> Hello guys:
>>>
>>> I've built Bacula 7.4.0 but I have no idea how to build the
>>> tray-monitor application. I can't find any arguments in ./configure
>>> --help to achieve it even when I found a tray-monitor directory in the
>>> contents of the tarball.
>>>
>>> Hope someone can give me some ideas.
>>>
>>> Thanks in advance.
>>>
>>>

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to build tray-monitor

Jason Voorhees
Thank you Kern and Eric, now I was able to build tray-monitor using
qmake and make commands

On Tue, Apr 19, 2016 at 7:30 AM, Eric Bollengier
<[hidden email]> wrote:

> Hello Jason,
>
> The Makefile is probably generated with the "qmake" command, so, try
>
> qmake
>
> make
>
> in the qt-console/tray-monitor directory.
>
> Hope it helps
>
> Best Regards,
> Eric
>
> On 04/19/2016 02:00 PM, Jason Voorhees wrote:
>> Thank you Kern.
>>
>> Yes, I have tried that but there's no Makefile in tray-monitor
>> directory so I can't run make under this path :( I'd like to build it
>> manually using gcc + arguments but I have no idea how to do it.
>>
>> On Mon, Apr 18, 2016 at 4:04 AM, Kern Sibbald <[hidden email]> wrote:
>>> Hello Jason,
>>>
>>> cd <bacula-source>
>>> ./configure <your options>
>>> make    [or "make -j9" goes faster]
>>>
>>> [I assume you have done the above, but if not do it]
>>>
>>> cd src/qt-console/tray-monitor
>>> make
>>>
>>> If I am not mistaken the binary will be in bacula-tray-monitor. You must
>>> then create a configuration file for it that is valid (tray-monitor.conf)
>>> and execute it.
>>>
>>> I have never tried it, so I am not sure it will work.  Feedback would be
>>> appreciated.
>>>
>>> Best regards,
>>> Kern
>>>
>>>
>>> On 04/16/2016 01:06 AM, Jason Voorhees wrote:
>>>>
>>>> Hello guys:
>>>>
>>>> I've built Bacula 7.4.0 but I have no idea how to build the
>>>> tray-monitor application. I can't find any arguments in ./configure
>>>> --help to achieve it even when I found a tray-monitor directory in the
>>>> contents of the tarball.
>>>>
>>>> Hope someone can give me some ideas.
>>>>
>>>> Thanks in advance.
>>>>
>>>>
>
> ------------------------------------------------------------------------------
> Find and fix application performance issues faster with Applications Manager
> Applications Manager provides deep performance insights into multiple tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> _______________________________________________
> Bacula-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

How to recover from lost connection

Ian Douglas
In reply to this post by Kern Sibbald
hi All

Is there a way to recover from this situation?

I'm trying to make up a NAS drive with many TB of data. 4th attempt.

I don't know why it lost connection to the SD, box is up and running and has
not gone down:

ssh tapeserver
ian@tapeserver's password:
Last login: Thu Apr 21 08:18:40 2016 from trooper
[ian@tapeserver ~]$ uptime
 08:19:19 up 3 days, 15:41,  2 users,  load average: 0.00, 0.01, 0.22
[ian@tapeserver ~]$

Any advice gratefully received... :-)

Comment: now that I read it again, it appears that SD spooled almost a TB of
data, without writing any ... do I need a heartbeat?

It feels like I should be setting some sort of "chunk size" that specifies to
write a block say every 50 GB or so, but I don't see any options like that in
the manual. Perhaps I'm searching for the wrong words.

20-Apr 17:03 trooper-dir JobId 10: No prior Full backup Job record found.
20-Apr 17:03 trooper-dir JobId 10: No prior or suitable Full backup found in
catalog. Doing FULL backup.
20-Apr 17:03 trooper-dir JobId 10: Start Backup JobId 10, Job=Backup2Nas-to-
Tape.2016-04-20_17.03.47_06
20-Apr 17:04 trooper-dir JobId 10: Using Device "LTO-6" to write.
20-Apr 17:04 TapeServer JobId 10: Warning: Director wanted Volume
"NASFull-0003".
    Current Volume "NonNASIncDiff-0002" not acceptable because:
    1998 Volume "NonNASIncDiff-0002" catalog status is Append, not in Pool.
20-Apr 17:04 TapeServer JobId 10: Please mount append Volume "NASFull-0003" or
label a new one for:
    Job:          Backup2Nas-to-Tape.2016-04-20_17.03.47_06
    Storage:      "LTO-6" (/dev/nst0)
    Pool:         NASFullPool
    Media type:   LTO-6
20-Apr 17:09 TapeServer JobId 10: Wrote label to prelabeled Volume
"NASFull-0003" on tape device "LTO-6" (/dev/nst0)
20-Apr 17:09 TapeServer JobId 10: Spooling data ...
20-Apr 23:44 TapeServer JobId 10: Writing spooled data to Volume. Despooling
940,494,394,596 bytes ...
21-Apr 03:16 TapeServer JobId 10: Despooling elapsed time = 03:32:23, Transfer
rate = 73.80 M Bytes/second
21-Apr 07:39 trooper-fd JobId 10: Error: bsock.c:448 Write error sending 65540
bytes to Storage daemon:192.168.1.60:9103: ERR=Broken pipe
21-Apr 07:39 trooper-fd JobId 10: Fatal error: backup.c:853 Network send error
to SD. ERR=Broken pipe
21-Apr 07:40 trooper-dir JobId 10: Error: Director's connection to SD for this
Job was lost.
21-Apr 07:40 trooper-dir JobId 10: Error: Bacula trooper-dir 7.0.5 (28Jul14):
  Build OS:               x86_64-pc-linux-gnu gentoo
  JobId:                  10
  Job:                    Backup2Nas-to-Tape.2016-04-20_17.03.47_06
  Backup Level:           Full (upgraded from Incremental)
  Client:                 "nas2" 7.4.0 (16Jan16) amd64-portbld-
freebsd9.3,freebsd,9.3-RELEASE
  FileSet:                "Nas2Files" 2016-04-20 16:50:33
  Pool:                   "NASFullPool" (From Job FullPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "TapeServer" (From command line)
  Scheduled time:         20-Apr-2016 17:03:45
  Start time:             20-Apr-2016 17:04:15
  End time:               21-Apr-2016 07:40:08
  Elapsed time:           14 hours 35 mins 53 secs
  Priority:               10
  FD Files Written:       369,687
  SD Files Written:       0
  FD Bytes Written:       1,593,625,355,624 (1.593 TB)
  SD Bytes Written:       0 (0 B)
  Rate:                   30324.2 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               no
  Volume name(s):         NASFull-0003
  Volume Session Id:      17
  Volume Session Time:    1460903935
  Last Volume Bytes:      939,939,840,000 (939.9 GB)
  Non-fatal FD errors:    2
  SD Errors:              0
  FD termination status:  Error
  SD termination status:  Error
  Termination:            *** Backup Error ***

Thanks, Ian
--
[hidden email] http://www.zti.co.za
Zero 2 Infinity - The net.works
Phone +27-21-975-7273

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to recover from lost connection

Josh Fisher


On 4/21/2016 2:36 AM, Ian Douglas wrote:

> hi All
>
> Is there a way to recover from this situation?
>
> I'm trying to make up a NAS drive with many TB of data. 4th attempt.
>
> I don't know why it lost connection to the SD, box is up and running and has
> not gone down:
>
> ssh tapeserver
> ian@tapeserver's password:
> Last login: Thu Apr 21 08:18:40 2016 from trooper
> [ian@tapeserver ~]$ uptime
>   08:19:19 up 3 days, 15:41,  2 users,  load average: 0.00, 0.01, 0.22
> [ian@tapeserver ~]$
>
> Any advice gratefully received... :-)
>
> Comment: now that I read it again, it appears that SD spooled almost a TB of
> data, without writing any ... do I need a heartbeat?

Yes, most likely. In the 3 hours or so it took to despool to tape, the
SD->FD TCP connection was dropped for some reason. I am of the opinion
that not all interfaces interpret IEE 802.3az Energy-Efficient Ethernet
in the same way, and when one interface puts its transmitter into sleep
mode, the receiving interface sees it as a dropped connection. Setting a
heartbeat should work, if that is what is happening.

>
> It feels like I should be setting some sort of "chunk size" that specifies to
> write a block say every 50 GB or so, but I don't see any options like that in
> the manual. Perhaps I'm searching for the wrong words.
>
> 20-Apr 17:03 trooper-dir JobId 10: No prior Full backup Job record found.
> 20-Apr 17:03 trooper-dir JobId 10: No prior or suitable Full backup found in
> catalog. Doing FULL backup.
> 20-Apr 17:03 trooper-dir JobId 10: Start Backup JobId 10, Job=Backup2Nas-to-
> Tape.2016-04-20_17.03.47_06
> 20-Apr 17:04 trooper-dir JobId 10: Using Device "LTO-6" to write.
> 20-Apr 17:04 TapeServer JobId 10: Warning: Director wanted Volume
> "NASFull-0003".
>      Current Volume "NonNASIncDiff-0002" not acceptable because:
>      1998 Volume "NonNASIncDiff-0002" catalog status is Append, not in Pool.
> 20-Apr 17:04 TapeServer JobId 10: Please mount append Volume "NASFull-0003" or
> label a new one for:
>      Job:          Backup2Nas-to-Tape.2016-04-20_17.03.47_06
>      Storage:      "LTO-6" (/dev/nst0)
>      Pool:         NASFullPool
>      Media type:   LTO-6
> 20-Apr 17:09 TapeServer JobId 10: Wrote label to prelabeled Volume
> "NASFull-0003" on tape device "LTO-6" (/dev/nst0)
> 20-Apr 17:09 TapeServer JobId 10: Spooling data ...
> 20-Apr 23:44 TapeServer JobId 10: Writing spooled data to Volume. Despooling
> 940,494,394,596 bytes ...
> 21-Apr 03:16 TapeServer JobId 10: Despooling elapsed time = 03:32:23, Transfer
> rate = 73.80 M Bytes/second
> 21-Apr 07:39 trooper-fd JobId 10: Error: bsock.c:448 Write error sending 65540
> bytes to Storage daemon:192.168.1.60:9103: ERR=Broken pipe
> 21-Apr 07:39 trooper-fd JobId 10: Fatal error: backup.c:853 Network send error
> to SD. ERR=Broken pipe
> 21-Apr 07:40 trooper-dir JobId 10: Error: Director's connection to SD for this
> Job was lost.
> 21-Apr 07:40 trooper-dir JobId 10: Error: Bacula trooper-dir 7.0.5 (28Jul14):
>    Build OS:               x86_64-pc-linux-gnu gentoo
>    JobId:                  10
>    Job:                    Backup2Nas-to-Tape.2016-04-20_17.03.47_06
>    Backup Level:           Full (upgraded from Incremental)
>    Client:                 "nas2" 7.4.0 (16Jan16) amd64-portbld-
> freebsd9.3,freebsd,9.3-RELEASE
>    FileSet:                "Nas2Files" 2016-04-20 16:50:33
>    Pool:                   "NASFullPool" (From Job FullPool override)
>    Catalog:                "MyCatalog" (From Client resource)
>    Storage:                "TapeServer" (From command line)
>    Scheduled time:         20-Apr-2016 17:03:45
>    Start time:             20-Apr-2016 17:04:15
>    End time:               21-Apr-2016 07:40:08
>    Elapsed time:           14 hours 35 mins 53 secs
>    Priority:               10
>    FD Files Written:       369,687
>    SD Files Written:       0
>    FD Bytes Written:       1,593,625,355,624 (1.593 TB)
>    SD Bytes Written:       0 (0 B)
>    Rate:                   30324.2 KB/s
>    Software Compression:   None
>    VSS:                    no
>    Encryption:             no
>    Accurate:               no
>    Volume name(s):         NASFull-0003
>    Volume Session Id:      17
>    Volume Session Time:    1460903935
>    Last Volume Bytes:      939,939,840,000 (939.9 GB)
>    Non-fatal FD errors:    2
>    SD Errors:              0
>    FD termination status:  Error
>    SD termination status:  Error
>    Termination:            *** Backup Error ***
>
> Thanks, Ian


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to recover from lost connection

Ian Douglas
hi All

On Thursday 21 April 2016 07:21:08 Josh Fisher wrote:

> Yes, most likely. In the 3 hours or so it took to despool to tape, the
> SD->FD TCP connection was dropped for some reason. I am of the opinion
> that not all interfaces interpret IEE 802.3az Energy-Efficient Ethernet
> in the same way, and when one interface puts its transmitter into sleep
> mode, the receiving interface sees it as a dropped connection. Setting a
> heartbeat should work, if that is what is happening.

Okay thanks, will add that. I guess I will have to start the backup over, how
do I remove what's in the catalog so that I can reset the tape?

Is there a way to write to tape every 50GB or so? I see setting for volume
size but as I understand it (newbie and all) that that refers to the tape size
not the "chunk" size.

Thanks, Ian

--
[hidden email] http://www.zti.co.za
Zero 2 Infinity - The net.works
Phone +27-21-975-7273

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to recover from lost connection

Clark, Patti
Volume sizing is used for disk-based volumes – do not use it for tape media volumes.  

If you are using spooling (recommended) your spool size parameters are used to control the size of your job spool and the total spool space available.  The example below shows 50GB of job spool size and a total of 1TB of spool space.  You will need to try sizes that are appropriate for your environment and see what performs best for you.  The job spool size when reached will write to tape at that point.  It will continue with spooling and writing to tape media until the end of data has been reached.  Also for something like a NAS, it’s best to break the backups into more manageable chunks.  Don’t try to backup the entire NAS in one job.  And yes, use the heartbeatinterval feature.

  Maximum Spool Size = 1000GB;

  Maximum Job Spool Size = 50GB;


You don’t mention which version of Bacula that you are using.  If it is 7.4.x is has a resume command that will restart failed jobs roughly from the point that the backup failed.  The documentation will talk about this under Incomplete Jobs.  I have found the use of this feature to be great when it works, otherwise it will start from the beginning as a new job using the job schedule to determine the level of the backup.  Client connection drops are hit and miss on the resume working.

Patti

From: Ian Douglas <[hidden email]>
Organization: Zero 2 Infinity
Reply-To: "[hidden email]" <[hidden email]>
Date: Thursday, April 21, 2016 at 9:33 AM
To: "[hidden email]" <[hidden email]>
Cc: Josh Fisher <[hidden email]>
Subject: Re: [Bacula-users] How to recover from lost connection

hi All

On Thursday 21 April 2016 07:21:08 Josh Fisher wrote:

Yes, most likely. In the 3 hours or so it took to despool to tape, the
SD->FD TCP connection was dropped for some reason. I am of the opinion
that not all interfaces interpret IEE 802.3az Energy-Efficient Ethernet
in the same way, and when one interface puts its transmitter into sleep
mode, the receiving interface sees it as a dropped connection. Setting a
heartbeat should work, if that is what is happening.

Okay thanks, will add that. I guess I will have to start the backup over, how
do I remove what's in the catalog so that I can reset the tape?

Is there a way to write to tape every 50GB or so? I see setting for volume
size but as I understand it (newbie and all) that that refers to the tape size
not the "chunk" size.

Thanks, Ian

--
Zero 2 Infinity - The net.works
Phone +27-21-975-7273

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
_______________________________________________
Bacula-users mailing list



------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to recover from lost connection

Ian Douglas
hi All

On Thursday 21 April 2016 17:00:14 Clark, Patti wrote:

> If you are using spooling (recommended) your spool size parameters are used
> to control the size of your job spool and the total spool space available.
> The example below shows 50GB of job spool size and a total of 1TB of spool
> space.  You will need to try sizes that are appropriate for your
> environment and see what performs best for you.  The job spool size when
> reached will write to tape at that point.  It will continue with spooling
> and writing to tape media until the end of data has been reached.  Also for
> something like a NAS, it’s best to break the backups into more manageable
> chunks.  Don’t try to backup the entire NAS in one job.  And yes, use the
> heartbeatinterval feature.
 
>
>   Maximum Spool Size = 1000GB;
>
>   Maximum Job Spool Size = 50GB;

Okay your explanation is clearer to me than what is in the manual. I was going
to use Maximum File Size as a way to "chunk" it, even though I was dubious
about it.

Will implement your suggestions, thanks.
 
> You don’t mention which version of Bacula that you are using.  If it is
> 7.4.x is has a resume command that will restart failed jobs roughly from

Versions vary, Director and FD on this machine (Gentoo) are the same, 7.05 but
the NAS boxes are running FreeNas and I had to compile a version for them in
VirtualBox, don't think the FreeBSD releases are at that level yet. Also was
trying to back up a site on the net running CentOS 6, which had it's own
issues. Eventually gave up, could not get the remote FD and local SD to talk
to each other, I think the problem is somewhere between my firewall (IPFire)
and the SD (CentOS 7 on HP server). So now I rsync remote to here and backup
from here. Initial rsync took 20 hours so it's probably better that way.

It would be nice if the various OSs could get their act together and get the
versions in sync :-)

Thanks, Ian
--
[hidden email] http://www.zti.co.za
Zero 2 Infinity - The net.works
Phone +27-21-975-7273

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to recover from lost connection

Clark, Patti
I use stand-in clients that have the various NAS’s filesystems mounted with full read permissions.  I control the OS (RHEL or CentOS)  for stability and can use the EPEL packages for the client which are the latest version available to the community.  Is it the most efficient – maybe not, but it’s the most expedient.  Not only that, but if I replace the storage with something else, I can get always get the data there.  The NAS’s weakest link usually is the controller.  It doesn’t matter how HA the disks are if the controller goes bad.

Patti


From: Ian Douglas <[hidden email]>
Organization: Zero 2 Infinity
Reply-To: "[hidden email]" <[hidden email]>
Date: Thursday, April 21, 2016 at 1:37 PM
To: "Clark, Patricia A." <[hidden email]>
Cc: "[hidden email]" <[hidden email]>, Josh Fisher <[hidden email]>
Subject: Re: [Bacula-users] How to recover from lost connection

hi All

On Thursday 21 April 2016 17:00:14 Clark, Patti wrote:

If you are using spooling (recommended) your spool size parameters are used
to control the size of your job spool and the total spool space available.
The example below shows 50GB of job spool size and a total of 1TB of spool
space.  You will need to try sizes that are appropriate for your
environment and see what performs best for you.  The job spool size when
reached will write to tape at that point.  It will continue with spooling
and writing to tape media until the end of data has been reached.  Also for
something like a NAS, it’s best to break the backups into more manageable
chunks.  Don’t try to backup the entire NAS in one job.  And yes, use the
heartbeatinterval feature.
   Maximum Spool Size = 1000GB;
   Maximum Job Spool Size = 50GB;

Okay your explanation is clearer to me than what is in the manual. I was going
to use Maximum File Size as a way to "chunk" it, even though I was dubious
about it.

Will implement your suggestions, thanks.
You don’t mention which version of Bacula that you are using.  If it is
7.4.x is has a resume command that will restart failed jobs roughly from

Versions vary, Director and FD on this machine (Gentoo) are the same, 7.05 but
the NAS boxes are running FreeNas and I had to compile a version for them in
VirtualBox, don't think the FreeBSD releases are at that level yet. Also was
trying to back up a site on the net running CentOS 6, which had it's own
issues. Eventually gave up, could not get the remote FD and local SD to talk
to each other, I think the problem is somewhere between my firewall (IPFire)
and the SD (CentOS 7 on HP server). So now I rsync remote to here and backup
from here. Initial rsync took 20 hours so it's probably better that way.

It would be nice if the various OSs could get their act together and get the
versions in sync :-)

Thanks, Ian
--
Zero 2 Infinity - The net.works
Phone +27-21-975-7273



------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to recover from lost connection

Kern Sibbald
In reply to this post by Clark, Patti
Hello Patti,

Yes, the restart command is in its "infancy" -- that is it does not currently (on purpose) handle all cases -- each failure must have explicit code to permit a partial backup to be used.  This is to ensure as much integrity of the final backup as possible. 

If you or anyone else has some explicit cases where a connection dropped and Bacula restarts from the beginning rather than as close as possible to the failure, please send me the exact output from the original backup failure showing the connection failure.  That way I can track down the code path of the failure and make sure it permits restarts.

Best regards,
Kern

On 04/21/2016 07:00 PM, Clark, Patti wrote:
Volume sizing is used for disk-based volumes – do not use it for tape media volumes.  

If you are using spooling (recommended) your spool size parameters are used to control the size of your job spool and the total spool space available.  The example below shows 50GB of job spool size and a total of 1TB of spool space.  You will need to try sizes that are appropriate for your environment and see what performs best for you.  The job spool size when reached will write to tape at that point.  It will continue with spooling and writing to tape media until the end of data has been reached.  Also for something like a NAS, it’s best to break the backups into more manageable chunks.  Don’t try to backup the entire NAS in one job.  And yes, use the heartbeatinterval feature.

  Maximum Spool Size = 1000GB;

  Maximum Job Spool Size = 50GB;


You don’t mention which version of Bacula that you are using.  If it is 7.4.x is has a resume command that will restart failed jobs roughly from the point that the backup failed.  The documentation will talk about this under Incomplete Jobs.  I have found the use of this feature to be great when it works, otherwise it will start from the beginning as a new job using the job schedule to determine the level of the backup.  Client connection drops are hit and miss on the resume working.

Patti

From: Ian Douglas <[hidden email]>
Organization: Zero 2 Infinity
Reply-To: "[hidden email]" <[hidden email]>
Date: Thursday, April 21, 2016 at 9:33 AM
To: "[hidden email]" <[hidden email]>
Cc: Josh Fisher <[hidden email]>
Subject: Re: [Bacula-users] How to recover from lost connection

hi All

On Thursday 21 April 2016 07:21:08 Josh Fisher wrote:

Yes, most likely. In the 3 hours or so it took to despool to tape, the
SD->FD TCP connection was dropped for some reason. I am of the opinion
that not all interfaces interpret IEE 802.3az Energy-Efficient Ethernet
in the same way, and when one interface puts its transmitter into sleep
mode, the receiving interface sees it as a dropped connection. Setting a
heartbeat should work, if that is what is happening.

Okay thanks, will add that. I guess I will have to start the backup over, how
do I remove what's in the catalog so that I can reset the tape?

Is there a way to write to tape every 50GB or so? I see setting for volume
size but as I understand it (newbie and all) that that refers to the tape size
not the "chunk" size.

Thanks, Ian

--
Zero 2 Infinity - The net.works
Phone +27-21-975-7273

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
_______________________________________________
Bacula-users mailing list




------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z


_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: How to recover from lost connection

Kern Sibbald
In reply to this post by Ian Douglas
On 04/21/2016 07:37 PM, Ian Douglas wrote:

> hi All
>
> On Thursday 21 April 2016 17:00:14 Clark, Patti wrote:
>
>> If you are using spooling (recommended) your spool size parameters are used
>> to control the size of your job spool and the total spool space available.
>> The example below shows 50GB of job spool size and a total of 1TB of spool
>> space.  You will need to try sizes that are appropriate for your
>> environment and see what performs best for you.  The job spool size when
>> reached will write to tape at that point.  It will continue with spooling
>> and writing to tape media until the end of data has been reached.  Also for
>> something like a NAS, it’s best to break the backups into more manageable
>> chunks.  Don’t try to backup the entire NAS in one job.  And yes, use the
>> heartbeatinterval feature.
>  
>>    Maximum Spool Size = 1000GB;
>>
>>    Maximum Job Spool Size = 50GB;
> Okay your explanation is clearer to me than what is in the manual. I was going
> to use Maximum File Size as a way to "chunk" it, even though I was dubious
> about it.
>
> Will implement your suggestions, thanks.
>  
>> You don’t mention which version of Bacula that you are using.  If it is
>> 7.4.x is has a resume command that will restart failed jobs roughly from
> Versions vary, Director and FD on this machine (Gentoo) are the same, 7.05 but
> the NAS boxes are running FreeNas and I had to compile a version for them in
> VirtualBox, don't think the FreeBSD releases are at that level yet. Also was
> trying to back up a site on the net running CentOS 6, which had it's own
> issues. Eventually gave up, could not get the remote FD and local SD to talk
> to each other, I think the problem is somewhere between my firewall (IPFire)
> and the SD (CentOS 7 on HP server). So now I rsync remote to here and backup
> from here. Initial rsync took 20 hours so it's probably better that way.
>
> It would be nice if the various OSs could get their act together and get the
> versions in sync :-)

If all works out as I would like, the Bacula project will be able to
provide you with binaries for most popular distros thus reducing the
need to rely on distros binaries.  This is a long time wish of mine (I
cannot do it alone), and it appears to be much closer to realization.

Best regards,
Kern

>
> Thanks, Ian


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users