Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

Heitor Faria
Hello, Bacula Users,


Regards,
--
===========================================================================
Heitor Medrado de Faria  | Bacula do Brasil
• Não seja tarifado pelo tamanho dos seus backups, conheça o Bacula Enterprise: http://www.bacula.com.br/enterprise/
 Ministro treinamento e implementação in-company do Bacula Community: http://www.bacula.com.br/in-company/
(61) 98268-4220 | www.bacula.com.br 
============================================================================
Indicamos também as capacitações complementares:
 Shell básico e Programação em Shell com Julio Neves.
 Zabbix com Adail Host.
============================================================================

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

Dan Langille

On May 31, 2017, at 5:22 PM, Heitor Faria <[hidden email]> wrote:


Most of the post seems to deal with ddumbfs.  Why not leave that out entirely?

If we are not using ddumbfs, it sounds like the only requirements are:

Perhaps a ZFS-only post is worthwhile?

If I use ZFS, what filesystem attributes are required?

-- 
Dan Langille - BSDCan / PGCon
[hidden email]



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

Heitor Faria
Hello, Dan,

Thanks for the feedback.
Most of the post seems to deal with ddumbfs.  Why not leave that out entirely?
Just to confirm, you are aware that default Bacula Volumes are basically not deduplicable, right? The main point is the new Aligned volumes feature.
Ddumbfs FS was the one I had time to test for this laboratory. But of course, there are other options. I will try to do the same with ZFS.
If we are not using ddumbfs, it sounds like the only requirements are:

Perhaps a ZFS-only post is worthwhile?
Sure.
If I use ZFS, what filesystem attributes are required?
Hopefully we are going to find out. =)
-- 
Dan Langille - BSDCan / PGCon
[hidden email]
Regards,
--
===========================================================================
Heitor Medrado de Faria  | Bacula do Brasil
• Não seja tarifado pelo tamanho dos seus backups, conheça o Bacula Enterprise: http://www.bacula.com.br/enterprise/
 Ministro treinamento e implementação in-company do Bacula Community: http://www.bacula.com.br/in-company/
(61) 98268-4220 | www.bacula.com.br 
============================================================================
Indicamos também as capacitações complementares:
 Shell básico e Programação em Shell com Julio Neves.
 Zabbix com Adail Host.
============================================================================

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

Dan Langille
Please do not take this as negative feedback. Do not be discouraged. It is mean to help improve the post.

On Jun 1, 2017, at 7:50 AM, Heitor Faria <[hidden email]> wrote:

Hello, Dan,

Thanks for the feedback.
Most of the post seems to deal with ddumbfs.  Why not leave that out entirely?
Just to confirm, you are aware that default Bacula Volumes are basically not deduplicable, right?

Yes. I am.  I suggest mentioning that at the start of the post. Something like:

   Bacula take take advantage of filesystem deduplication with a specially configured Device.

Putting the prerequisites at the top of the post helps readers understand whether or not this article is for them.

The main point is the new Aligned volumes feature.

Agreed.  The post needs to concentrate on that feature, not ddumbfs.  At present, it makes me think the title should be:

    Using ddumbfs for Block-level Deduplication with Aligned Volumes

Ddumbfs FS was the one I had time to test for this laboratory. But of course, there are other options. I will try to do the same with ZFS.

I understand.

If we are not using ddumbfs, it sounds like the only requirements are:

Sorry, I did not complete that sentence. I should have deleted it.

Perhaps a ZFS-only post is worthwhile?
Sure.
If I use ZFS, what filesystem attributes are required?
Hopefully we are going to find out. =)


-- 
Dan Langille - BSDCan / PGCon
[hidden email]




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

Michael Watters
In reply to this post by Heitor Faria

The instructions for setting up ZFS on CentOS 7 do not work.  yum gives an error as shown below.

[root@bacula ~]# yum localinstall --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm
Loaded plugins: fastestmirror, versionlock
Cannot open: http://archive.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm. Skipping.
Nothing to do

I also highly suggest you do *not* disable GPG checks.  Signature verification is critical for ensuring that you do not end up installing invalid packages.  The proper commands to use should be like this:

yum install http://download.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm

gpg --quiet --with-fingerprint /etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux


This matches the instructions given at github.com/zfsonlinux/zfs/wiki/RHEL-%26-CentOS.


On 05/31/2017 05:22 PM, Heitor Faria wrote:
Hello, Bacula Users,


Regards,
--
===========================================================================
Heitor Medrado de Faria  | Bacula do Brasil
• Não seja tarifado pelo tamanho dos seus backups, conheça o Bacula Enterprise: http://www.bacula.com.br/enterprise/
 Ministro treinamento e implementação in-company do Bacula Community: http://www.bacula.com.br/in-company/
(61) 98268-4220 | www.bacula.com.br 
============================================================================
Indicamos também as capacitações complementares:
 Shell básico e Programação em Shell com Julio Neves.
 Zabbix com Adail Host.
============================================================================


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

Heitor Faria
Hello, Michael,

The instructions for setting up ZFS on CentOS 7 do not work.  yum gives an error as shown below.

[root@bacula ~]# yum localinstall --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm
Loaded plugins: fastestmirror, versionlock
Cannot open: http://archive.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm. Skipping.
Nothing to do

I also highly suggest you do *not* disable GPG checks.  Signature verification is critical for ensuring that you do not end up installing invalid packages.  The proper commands to use should be like this:

yum install http://download.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm

gpg --quiet --with-fingerprint /etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux

Thanks! I was just editing the post according to Dan suggestions and was about to test on CentOS environment.
Your way is safer and more accurate.

This matches the instructions given at github.com/zfsonlinux/zfs/wiki/RHEL-%26-CentOS.

Regards,
On 05/31/2017 05:22 PM, Heitor Faria wrote:
Hello, Bacula Users,


Regards,
--
===========================================================================
Heitor Medrado de Faria  | Bacula do Brasil
• Não seja tarifado pelo tamanho dos seus backups, conheça o Bacula Enterprise: http://www.bacula.com.br/enterprise/
 Ministro treinamento e implementação in-company do Bacula Community: http://www.bacula.com.br/in-company/
(61) 98268-4220 | www.bacula.com.br 
============================================================================
Indicamos também as capacitações complementares:
 Shell básico e Programação em Shell com Julio Neves.
 Zabbix com Adail Host.
============================================================================


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


_______________________________________________
Bacula-users mailing list
[hidden email]https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users

--
===========================================================================
Heitor Medrado de Faria  | Bacula do Brasil
• Não seja tarifado pelo tamanho dos seus backups, conheça o Bacula Enterprise: http://www.bacula.com.br/enterprise/
 Ministro treinamento e implementação in-company do Bacula Community: http://www.bacula.com.br/in-company/
(61) 98268-4220 | www.bacula.com.br 
============================================================================
Indicamos também as capacitações complementares:
 Shell básico e Programação em Shell com Julio Neves.
 Zabbix com Adail Host.
============================================================================

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|

Re: Block-level Deduplication with Aligned Volumes Tutorial (Bacula 7.9 – 9.0 and above)

Dimitri Maziuk
On 2017-06-01 09:02, Heitor Faria wrote:

> Thanks! I was just editing the post according to Dan suggestions and was
> about to test on CentOS environment.

Just point it to ZoL wiki on github and to Aaron Toponce's ZFS pages.
It's all well documented, you don't have to repeat that, especially if
you repeat stuff that's out of date or just plain incorrect.

(You don't have to have epel, kernel-devel, nor gpg really. Pool
creation command is wrong too.)

Dima


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users