Regression testing - let's get more involved

classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Regression testing - let's get more involved

Dan Langille
Regression testing is important. With it, you can ensure that future releases work with your chosen combination of OS and DB.

Setting up regression testing is not difficult.  Automating it is also easy.  It can be done in any environment and DOES NOT require root privs. It can be run as whatever user you choose.

After a short absence, it appears that http://regress.bacula.org is back, although I'd like to see it available over https as well.

I am not sure the regression testing is configured to accept incoming tests yet.  From last night's logs:

###
The following tests FAILED:
          1 - disk:acl-xattr-test (Failed)
          6 - disk:backup-bacula-test (Failed)
         28 - disk:estimate-test (Failed)
         77 - disk:tls-test (Failed)
Performing coverage
Submit files (using http)
   Using HTTP submit method
   Drop site:http://regress.bacula.org/submit.php?project=bacula
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Build.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Configure.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Test.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Update.xml
   Using HTTP trigger method
   Trigger site: http://regress.bacula.org
   Dart server triggered...
   Errors occurred during submission.
End nightly-disk Thu Mar  9 04:06:33 UTC 2017
###

My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6 on FreeBSD 10.3.  I could do similar on FreeBSD 11 but ... time.

Anyone else interested in helping out the devs? This involves a little bit of initial work, but the long-term payback is great.  It is a simple way that anyone can contribute to the project.

-- 
Dan Langille - BSDCan / PGCon
[hidden email]



------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Wanderlei Huttel
Hello Dan

I would like to help if I can. But I don't have the minimal idea how to proceed

Best Regards

Wanderlei Hüttel

2017-03-09 11:12 GMT-03:00 Dan Langille <[hidden email]>:
Regression testing is important. With it, you can ensure that future releases work with your chosen combination of OS and DB.

Setting up regression testing is not difficult.  Automating it is also easy.  It can be done in any environment and DOES NOT require root privs. It can be run as whatever user you choose.

After a short absence, it appears that http://regress.bacula.org is back, although I'd like to see it available over https as well.

I am not sure the regression testing is configured to accept incoming tests yet.  From last night's logs:

###
The following tests FAILED:
          1 - disk:acl-xattr-test (Failed)
          6 - disk:backup-bacula-test (Failed)
         28 - disk:estimate-test (Failed)
         77 - disk:tls-test (Failed)
Performing coverage
Submit files (using http)
   Using HTTP submit method
   Drop site:http://regress.bacula.org/submit.php?project=bacula
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Build.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Configure.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Test.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Update.xml
   Using HTTP trigger method
   Trigger site: http://regress.bacula.org
   Dart server triggered...
   Errors occurred during submission.
End nightly-disk Thu Mar  9 04:06:33 UTC 2017
###

My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6 on FreeBSD 10.3.  I could do similar on FreeBSD 11 but ... time.

Anyone else interested in helping out the devs? This involves a little bit of initial work, but the long-term payback is great.  It is a simple way that anyone can contribute to the project.

-- 
Dan Langille - BSDCan / PGCon
[hidden email]



------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users



------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Dan Langille
On Mar 9, 2017, at 9:25 AM, Wanderlei Huttel <[hidden email]> wrote:

Hello Dan

I would like to help if I can. But I don't have the minimal idea how to proceed



------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Andrea Venturoli
In reply to this post by Dan Langille
On 03/09/17 15:12, Dan Langille wrote:

> Anyone else interested in helping out the devs?

Maybe... that depends on what would be required on my site (like how
much work is needed, which kind of system, etc...).

Could you elaborate or provide some pointers to some doc?

What should we do?
What kinds of test are covered?





> My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6
 > on FreeBSD 10.3.

I'm using FreeBSD 10.3 too, and only 10.3 with Postgres (apart from some
embedded Storage Daemon).
Does this mean I wouldn't be of any help?



  bye & Thanks
        av.

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Dan Langille
On Mar 9, 2017, at 9:37 AM, Andrea Venturoli <[hidden email]> wrote:

On 03/09/17 15:12, Dan Langille wrote:

Anyone else interested in helping out the devs?

Maybe... that depends on what would be required on my site (like how much work is needed, which kind of system, etc...).

Could you elaborate or provide some pointers to some doc?

What should we do?
What kinds of test are covered?


My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6
> on FreeBSD 10.3.

I'm using FreeBSD 10.3 too, and only 10.3 with Postgres (apart from some embedded Storage Daemon).
Does this mean I wouldn't be of any help?

No, I think overlapping tests are good. Please proceed.

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Andrea Venturoli
On 03/09/17 15:40, Dan Langille wrote:

> Have a read
> of http://www.bacula.org/7.0.x-manuals/en/developers/Bacula_Regression_Testing.html

Done.
I think I understand the big picture and answers to my doubt (along with
more questions :) will come when I'll try.



>> My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6
>> > on FreeBSD 10.3.
>>
>> I'm using FreeBSD 10.3 too, and only 10.3 with Postgres (apart from
>> some embedded Storage Daemon).
>> Does this mean I wouldn't be of any help?
>
> No, I think overlapping tests are good. Please proceed.

Would testing binaries from FreeBSD ports (not packages) be useful?
Or are you suggesting I'll try from source?



  bye
        av.

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Kern Sibbald
In reply to this post by Dan Langille
Hello Dan,

Thank you for taking the initiative on this.  Yes, there was a brief period when there was no project defined for regression testing (Dart) after the switch from the old server to the new one.  However, it is up and working.  The project is Bacula-7.4. 

I would especially appreciate it if someone could test on OpenBSD, where currently there is a seg fault in the new acl-xattr code.  If all goes well, I will shortly have a fix, but I don't have a system to test it on.

Best regards,
Kern

On 03/09/2017 03:12 PM, Dan Langille wrote:
Regression testing is important. With it, you can ensure that future releases work with your chosen combination of OS and DB.

Setting up regression testing is not difficult.  Automating it is also easy.  It can be done in any environment and DOES NOT require root privs. It can be run as whatever user you choose.

After a short absence, it appears that http://regress.bacula.org is back, although I'd like to see it available over https as well.

I am not sure the regression testing is configured to accept incoming tests yet.  From last night's logs:

###
The following tests FAILED:
          1 - disk:acl-xattr-test (Failed)
          6 - disk:backup-bacula-test (Failed)
         28 - disk:estimate-test (Failed)
         77 - disk:tls-test (Failed)
Performing coverage
Submit files (using http)
   Using HTTP submit method
   Drop site:http://regress.bacula.org/submit.php?project=bacula
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Build.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Configure.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Test.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Update.xml
   Using HTTP trigger method
   Trigger site: http://regress.bacula.org
   Dart server triggered...
   Errors occurred during submission.
End nightly-disk Thu Mar  9 04:06:33 UTC 2017
###

My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6 on FreeBSD 10.3.  I could do similar on FreeBSD 11 but ... time.

Anyone else interested in helping out the devs? This involves a little bit of initial work, but the long-term payback is great.  It is a simple way that anyone can contribute to the project.

-- 
Dan Langille - BSDCan / PGCon
[hidden email]




------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford


_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users



------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Kern Sibbald
In reply to this post by Andrea Venturoli
Hello Andrea,

Although it is possible to run regression tests on preinstalled
binaries, it is not yet common or documented, so today, the only
practical way is to be able to build from source.  Once you have all the
source dependencies satisfied, the whole process of pulling the source,
compiling/linking/installing, and testing is automatically handled by
the Dart scripts.

Best regards,
Kern

On 03/09/2017 04:59 PM, Andrea Venturoli wrote:

> On 03/09/17 15:40, Dan Langille wrote:
>
>> Have a read
>> of http://www.bacula.org/7.0.x-manuals/en/developers/Bacula_Regression_Testing.html
> Done.
> I think I understand the big picture and answers to my doubt (along with
> more questions :) will come when I'll try.
>
>
>
>>> My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6
>>>> on FreeBSD 10.3.
>>> I'm using FreeBSD 10.3 too, and only 10.3 with Postgres (apart from
>>> some embedded Storage Daemon).
>>> Does this mean I wouldn't be of any help?
>> No, I think overlapping tests are good. Please proceed.
> Would testing binaries from FreeBSD ports (not packages) be useful?
> Or are you suggesting I'll try from source?
>
>
>
>    bye
> av.
>
> ------------------------------------------------------------------------------
> Announcing the Oxford Dictionaries API! The API offers world-renowned
> dictionary content that is easy and intuitive to access. Sign up for an
> account today to start using our lexical data to power your apps and
> projects. Get started today and enter our developer competition.
> http://sdm.link/oxford
> _______________________________________________
> Bacula-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Dan Langille
In reply to this post by Kern Sibbald
On Mar 9, 2017, at 11:03 AM, Kern Sibbald <[hidden email]> wrote:

Hello Dan,

Thank you for taking the initiative on this.  Yes, there was a brief period when there was no project defined for regression testing (Dart) after the switch from the old server to the new one.  However, it is up and working.  The project is Bacula-7.4. 

I have jails set up just for this; I just need to upgrade the regress directory and enable the cron jobs.

I would especially appreciate it if someone could test on OpenBSD, where currently there is a seg fault in the new acl-xattr code.  If all goes well, I will shortly have a fix, but I don't have a system to test it on.

I got my first proof-of-concept working with the latest source.  Over the next week, I'll enable the other jails. I'm catching up on mail today.



Best regards,
Kern

On 03/09/2017 03:12 PM, Dan Langille wrote:
Regression testing is important. With it, you can ensure that future releases work with your chosen combination of OS and DB.

Setting up regression testing is not difficult.  Automating it is also easy.  It can be done in any environment and DOES NOT require root privs. It can be run as whatever user you choose.

After a short absence, it appears that http://regress.bacula.org is back, although I'd like to see it available over https as well.

I am not sure the regression testing is configured to accept incoming tests yet.  From last night's logs:

###
The following tests FAILED:
          1 - disk:acl-xattr-test (Failed)
          6 - disk:backup-bacula-test (Failed)
         28 - disk:estimate-test (Failed)
         77 - disk:tls-test (Failed)
Performing coverage
Submit files (using http)
   Using HTTP submit method
   Drop site:http://regress.bacula.org/submit.php?project=bacula
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Build.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Configure.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Test.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Update.xml
   Using HTTP trigger method
   Trigger site: http://regress.bacula.org
   Dart server triggered...
   Errors occurred during submission.
End nightly-disk Thu Mar  9 04:06:33 UTC 2017
###

My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6 on FreeBSD 10.3.  I could do similar on FreeBSD 11 but ... time.

Anyone else interested in helping out the devs? This involves a little bit of initial work, but the long-term payback is great.  It is a simple way that anyone can contribute to the project.

-- 
Dan Langille - BSDCan / PGCon
[hidden email]




------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford


_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

francisco javier funes nieto
I have some spare time and one test machine to do some regression tests. 

How can I start? 
J.

2017-03-12 16:53 GMT+01:00 Dan Langille <[hidden email]>:
On Mar 9, 2017, at 11:03 AM, Kern Sibbald <[hidden email]> wrote:

Hello Dan,

Thank you for taking the initiative on this.  Yes, there was a brief period when there was no project defined for regression testing (Dart) after the switch from the old server to the new one.  However, it is up and working.  The project is Bacula-7.4. 

I have jails set up just for this; I just need to upgrade the regress directory and enable the cron jobs.

I would especially appreciate it if someone could test on OpenBSD, where currently there is a seg fault in the new acl-xattr code.  If all goes well, I will shortly have a fix, but I don't have a system to test it on.

I got my first proof-of-concept working with the latest source.  Over the next week, I'll enable the other jails. I'm catching up on mail today.



Best regards,
Kern

On 03/09/2017 03:12 PM, Dan Langille wrote:
Regression testing is important. With it, you can ensure that future releases work with your chosen combination of OS and DB.

Setting up regression testing is not difficult.  Automating it is also easy.  It can be done in any environment and DOES NOT require root privs. It can be run as whatever user you choose.

After a short absence, it appears that http://regress.bacula.org is back, although I'd like to see it available over https as well.

I am not sure the regression testing is configured to accept incoming tests yet.  From last night's logs:

###
The following tests FAILED:
          1 - disk:acl-xattr-test (Failed)
          6 - disk:backup-bacula-test (Failed)
         28 - disk:estimate-test (Failed)
         77 - disk:tls-test (Failed)
Performing coverage
Submit files (using http)
   Using HTTP submit method
   Drop site:http://regress.bacula.org/submit.php?project=bacula
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Build.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Configure.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Test.xml
   Submission failed: Not a valid project.
   Uploaded: /usr/home/dan/src/Bacula7RegressionTesting-TRUNK/regress/Testing/20170308-2200/Update.xml
   Using HTTP trigger method
   Trigger site: http://regress.bacula.org
   Dart server triggered...
   Errors occurred during submission.
End nightly-disk Thu Mar  9 04:06:33 UTC 2017
###

My own regression testing covers MySQL 5.5 and 5.6 & PostgreSQL 9.2-9.6 on FreeBSD 10.3.  I could do similar on FreeBSD 11 but ... time.

Anyone else interested in helping out the devs? This involves a little bit of initial work, but the long-term payback is great.  It is a simple way that anyone can contribute to the project.

-- 
Dan Langille - BSDCan / PGCon
[hidden email]




------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford


_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users




--
_____________________________________________

Francisco Javier Funes Nieto [[hidden email]]
CANONIGOS
Servicios Informáticos para PYMES.
Cl. Cruz 2, 1º Oficina 7
Tlf: 958.536759 / 661134556
Fax: 958.521354
GRANADA - 18002

------------------------------------------------------------------------------
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Vanush "Misha" Paturyan
In reply to this post by Kern Sibbald
On Thu, Mar 09, 2017 at 05:03:48PM +0100, Kern Sibbald wrote:

> Hello Dan,
>
> Thank you for taking the initiative on this.  Yes, there was a brief period
> when there was no project defined for regression testing (Dart) after the
> switch from the old server to the new one.  However, it is up and working.  The
> project is Bacula-7.4. 
>
> I would especially appreciate it if someone could test on OpenBSD, where
> currently there is a seg fault in the new acl-xattr code.  If all goes well, I
> will shortly have a fix, but I don't have a system to test it on.
Hi Kern,

I've tried to run regression tests for Bacula 7.4.6 on OpenBSD 6.0,
but the "make setup" step (ran in the bacula/regress directory)
returns with the following error:

Compiling bnet.c
bnet.c: In function 'int set_socket_errno(int)':
bnet.c:632: error: 'EPROTO' was not declared in this scope
*** Error 1 in target 'bnet.lo'
*** Error 1 in build/src/lib (Makefile:184 'bnet.lo': @/home/misha/bacula/regress/build/libtool --silent --tag=CXX --mode=compile /usr/bin/g...)


  ====== Error in /home/misha/bacula/regress/build/src/lib ======


*** Error 1 in build (Makefile:152 'all')
Bad exit code giving up
*** Error 1 in /home/misha/bacula/regress (Makefile:36 'bacula')

Also, this might be completely unrelated, but at the very beginning of
the "make setup" command I can see grep complaining about "-m" flag.

I am new to any BSD, so I don't really know what to do next. Is this
the appropriate place to report this issue?

Cheers,

Misha

--
Vanush "Misha" Paturyan
Senior Technical Officer
Room 120
Computer Science Department
Eolas Bulding
Maynooth University
Maynooth

ext: 4539

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Dan Langille
In reply to this post by francisco javier funes nieto
On Mar 13, 2017, at 8:39 AM, Francisco Javier Funes Nieto <[hidden email]> wrote:

I have some spare time and one test machine to do some regression tests. 

How can I start? 



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Kern Sibbald
In reply to this post by Vanush "Misha" Paturyan
Hello,

If you try pulling the git repository again and rerunning the regression tests, they should now work.

Best regards,
Kern

On 03/14/2017 01:05 PM, Vanush Misha Paturyan wrote:
On Thu, Mar 09, 2017 at 05:03:48PM +0100, Kern Sibbald wrote:
Hello Dan,

Thank you for taking the initiative on this.  Yes, there was a brief period
when there was no project defined for regression testing (Dart) after the
switch from the old server to the new one.  However, it is up and working.  The
project is Bacula-7.4. 

I would especially appreciate it if someone could test on OpenBSD, where
currently there is a seg fault in the new acl-xattr code.  If all goes well, I
will shortly have a fix, but I don't have a system to test it on.
Hi Kern,

I've tried to run regression tests for Bacula 7.4.6 on OpenBSD 6.0,
but the "make setup" step (ran in the bacula/regress directory)
returns with the following error:

Compiling bnet.c
bnet.c: In function 'int set_socket_errno(int)':
bnet.c:632: error: 'EPROTO' was not declared in this scope
*** Error 1 in target 'bnet.lo'
*** Error 1 in build/src/lib (Makefile:184 'bnet.lo': @/home/misha/bacula/regress/build/libtool --silent --tag=CXX --mode=compile /usr/bin/g...)


  ====== Error in /home/misha/bacula/regress/build/src/lib ======


*** Error 1 in build (Makefile:152 'all')
Bad exit code giving up
*** Error 1 in /home/misha/bacula/regress (Makefile:36 'bacula')

Also, this might be completely unrelated, but at the very beginning of
the "make setup" command I can see grep complaining about "-m" flag.

I am new to any BSD, so I don't really know what to do next. Is this
the appropriate place to report this issue?

Cheers,

Misha



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Andrea Venturoli
In reply to this post by Dan Langille
On 03/09/17 15:40, Dan Langille wrote:

> Have a read of
> http://www.bacula.org/7.0.x-manuals/en/developers/Bacula_Regression_Testing.html 
> ..
> No, I think overlapping tests are good. Please proceed.

Hello.

I'm finally giving a shot at this, running it all inside a jail, but I'm
encountering some problems.



First of all, I'd like to use Postgres, but the server is running on
another box and the tests are trying to reach it via UNIX socket: how do
I tell them to connect to a remote address via TCP/IP?



Going over this error, I later get:
> echo "Doing: scripts/do_sed"
> Doing: scripts/do_sed
> scripts/do_sed
> ./scripts/get-os: 5: Bad file descriptor
> : error: cannot run /bin/csh scripts/config.sub
> ./scripts/get-os: 5: Bad file descriptor
> : error: cannot run /bin/csh scripts/config.sub

Any hint on this?



  bye & Thanks
        av.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Dan Langille
> On May 8, 2017, at 11:21 AM, Andrea Venturoli <[hidden email]> wrote:
>
> On 03/09/17 15:40, Dan Langille wrote:
>
>> Have a read of http://www.bacula.org/7.0.x-manuals/en/developers/Bacula_Regression_Testing.html ..
>> No, I think overlapping tests are good. Please proceed.
>
> Hello.
>
> I'm finally giving a shot at this, running it all inside a jail, but I'm encountering some problems.
>
>
>
> First of all, I'd like to use Postgres, but the server is running on another box and the tests are trying to reach it via UNIX socket: how do I tell them to connect to a remote address via TCP/IP?

I have something like this in my bacula-dir.conf for production.

Catalog {
  Name = MyCatalog
  dbname = bacula; dbaddress = slocum.int.unixathome.org; user = bacula; password = "TheRealPassword"
}

In the regression testing config file, I see this:

# DB parameters
db_name="regress"
db_user="regress"
db_password=""

I do not see a configuration parameter for dbaddress for the Catalog.

Is anyone running regression testing with a non-localhost database server?

> Going over this error, I later get:
>> echo "Doing: scripts/do_sed"
>> Doing: scripts/do_sed
>> scripts/do_sed
>> ./scripts/get-os: 5: Bad file descriptor
>> : error: cannot run /bin/csh scripts/config.sub
>> ./scripts/get-os: 5: Bad file descriptor
>> : error: cannot run /bin/csh scripts/config.sub
>
> Any hint on this?

I don't know why it's trying /bin/csh

What OS are you ... FreeBSD I'm guessing.  I run my bacula-dir in a jail.  What version of FreeBSD?

--
Dan Langille - BSDCan / PGCon
[hidden email]



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Kern Sibbald
In reply to this post by Andrea Venturoli
Hello,

In the next week or two, it will be *very* important for users to have
regression scripts running, because finding problems now will allow me
to fix them before the release.

To answer your question: normally you configure the Catalog interface
details in the Catalog {} resource of the Director, but most of the
bacula-dir.conf files that are used in regression testing, and there are
many of them, are "canned", which means I am not sure that you can
easily change the Catalog resource in each of these scripts.  You might
be able to do it with a simple sed command that adds what Dan
recommends.  You would point sed at all the *bacula-dir*.in files that
are in <regress>/scripts.

It might be easier to bring up a new local postgres on your test machine
or possibly a MySQL.

By the way, Bacula version 9.0.0 (currently labeled as version 7.9.0) is
almost ready for testing.  I hope to push it to the repo before the end
of next week (possibly sooner).  The biggest issue aside from testing on
multiple OSes is the documentation.

Best regards,

Kern


On 05/08/2017 05:21 PM, Andrea Venturoli wrote:

> On 03/09/17 15:40, Dan Langille wrote:
>
>> Have a read of
>> http://www.bacula.org/7.0.x-manuals/en/developers/Bacula_Regression_Testing.html
>> ..
>> No, I think overlapping tests are good. Please proceed.
> Hello.
>
> I'm finally giving a shot at this, running it all inside a jail, but I'm
> encountering some problems.
>
>
>
> First of all, I'd like to use Postgres, but the server is running on
> another box and the tests are trying to reach it via UNIX socket: how do
> I tell them to connect to a remote address via TCP/IP?
>
>
>
> Going over this error, I later get:
>> echo "Doing: scripts/do_sed"
>> Doing: scripts/do_sed
>> scripts/do_sed
>> ./scripts/get-os: 5: Bad file descriptor
>> : error: cannot run /bin/csh scripts/config.sub
>> ./scripts/get-os: 5: Bad file descriptor
>> : error: cannot run /bin/csh scripts/config.sub
> Any hint on this?
>
>
>
>    bye & Thanks
> av.
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Bacula-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Andrea Venturoli
On 05/08/17 18:51, Kern Sibbald wrote:

> Hello,
>
> In the next week or two, it will be *very* important for users to have
> regression scripts running, because finding problems now will allow me
> to fix them before the release.
>
> To answer your question: normally you configure the Catalog interface
> details in the Catalog {} resource of the Director, but most of the
> bacula-dir.conf files that are used in regression testing, and there are
> many of them, are "canned", which means I am not sure that you can
> easily change the Catalog resource in each of these scripts.  You might
> be able to do it with a simple sed command that adds what Dan
> recommends.  You would point sed at all the *bacula-dir*.in files that
> are in <regress>/scripts.
>
> It might be easier to bring up a new local postgres on your test machine
> or possibly a MySQL.

Thanks Dan and Kern for your answers.

AFAIK Postgres does not run in a jail (where I'm running the tests),
unless you set up your machine in such a way that the whole purpose of
jails is defeated, so this is a no-go.

I'd be interested in testing Postgres, since all my setups are based on
this; however, I'll choose sqlite over MySQL if I have to (i.e. if the
sed you suggest don't work).



  bye & Thanks
        av.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Andrea Venturoli
In reply to this post by Dan Langille
On 05/08/17 18:05, Dan Langille wrote:

>> Going over this error, I later get:
>>> echo "Doing: scripts/do_sed"
>>> Doing: scripts/do_sed
>>> scripts/do_sed
>>> ./scripts/get-os: 5: Bad file descriptor
>>> : error: cannot run /bin/csh scripts/config.sub
>>> ./scripts/get-os: 5: Bad file descriptor
>>> : error: cannot run /bin/csh scripts/config.sub
>>
>> Any hint on this?
>
> I don't know why it's trying /bin/csh
>
> What OS are you ... FreeBSD I'm guessing.  I run my bacula-dir in a jail.  What version of FreeBSD?
>

Yes, that's FreeBSD 10.3/amd64.

  bye & Thanks
        av.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Dan Langille
In reply to this post by Andrea Venturoli
> On May 8, 2017, at 1:04 PM, Andrea Venturoli <[hidden email]> wrote:
>
> On 05/08/17 18:51, Kern Sibbald wrote:
>> Hello,
>> In the next week or two, it will be *very* important for users to have regression scripts running, because finding problems now will allow me to fix them before the release.
>> To answer your question: normally you configure the Catalog interface details in the Catalog {} resource of the Director, but most of the bacula-dir.conf files that are used in regression testing, and there are many of them, are "canned", which means I am not sure that you can easily change the Catalog resource in each of these scripts.  You might be able to do it with a simple sed command that adds what Dan recommends.  You would point sed at all the *bacula-dir*.in files that are in <regress>/scripts.
>> It might be easier to bring up a new local postgres on your test machine or possibly a MySQL.
>
> Thanks Dan and Kern for your answers.
>
> AFAIK Postgres does not run in a jail (where I'm running the tests), unless you set up your machine in such a way that the whole purpose of jails is defeated, so this is a no-go.

I run PostgreSQL in my jails.  Every one of my Bacula regression tests run in a separate jail.

> I'd be interested in testing Postgres, since all my setups are based on this; however, I'll choose sqlite over MySQL if I have to (i.e. if the sed you suggest don't work).

I'd prefer PostgreSQL too...


--
Dan Langille - BSDCan / PGCon
[hidden email]




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Regression testing - let's get more involved

Andrea Venturoli
On 05/08/17 19:35, Dan Langille wrote:

> I run PostgreSQL in my jails.  Every one of my Bacula regression tests run in a separate jail.

The problem is that Postgres uses SYSV IPC.
In FreeBSD 10.3 you can enable SYSV IPC for jails, but that's a nice way
to let them escape into the base host.
So jailing will still be a way to "partition" your system for a cleaner
management, but it won't help any more WRT to security, so it's not
something I'm willing to do.

I believe FreeBSD 11 relaxed this limitation, allowing SYSV IPC per
jail, but I cannot upgrade right now.



So, for now, I'm testing with SQLite: most tests are failing, so I'm
sure I've done something wrong.
Investigating this will take precedence over switching to PostGres :)


  bye
        av.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
12