Typical Transfer Rates

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Typical Transfer Rates

Slartibartfast-5
I have bacula 1.34.2 running as Director, Storage and Client on a Pent
IV 3Ghz running CentOS 3.4.

Did fill tests with an Overland LoaderXpress LTO2 drive.  The drive
maxed at 27MB/s transfer rate.

Same drive connected to a Sun v240 running Solaris 9 got 15MB/s.  Same
drive connected to a Sun v880 running Solaris 9 got 18MB/s.

I have done full backups of 747GB on the v240 at an average 10.2MB/s
onto one tape. ;)  Just completing 1.15TB on the v880 at an average  
11.5MB/s.

So the question is...

Am I missing something big on the Solaris platforms?  I can't believe
the 8proc v880 is getting smoked by a single proc Pent IV.

Any suggestions welcome.



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Typical Transfer Rates

Nicolai E M Plum
Slartibartfast writes:

> I have bacula 1.34.2 running as Director, Storage and Client on a Pent
> IV 3Ghz running CentOS 3.4.
>
> Did fill tests with an Overland LoaderXpress LTO2 drive.  The drive
> maxed at 27MB/s transfer rate.
>
> Same drive connected to a Sun v240 running Solaris 9 got 15MB/s.  Same
> drive connected to a Sun v880 running Solaris 9 got 18MB/s.
>
> I have done full backups of 747GB on the v240 at an average 10.2MB/s
> onto one tape. ;)  Just completing 1.15TB on the v880 at an average  
> 11.5MB/s.
>
> So the question is...
>
> Am I missing something big on the Solaris platforms?  I can't believe
> the 8proc v880 is getting smoked by a single proc Pent IV.
>
> Any suggestions welcome.

Do you have a lot of sparse files and are you doing sparse file processing?
If so, it seems (based on one of my systems) that you can end up CPU-limited
as the bacula-fd goes through reading a lot of zeros as fast as it can.

Other than that, do you have a lot of files on the Solaris machines,
particularly a lot of files in a single directory? Reading a lot of distinct
can noticeably slow down backups and I've noticed it on Solaris on
filesystems like webservers full of small image files or mail spools with
file-per-message filestores.

Nicolai


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Typical Transfer Rates

Slartibartfast-5
On Thu, 2005-05-12 at 05:19, Nicolai E M Plum wrote:

> Slartibartfast writes:
> > I have bacula 1.34.2 running as Director, Storage and Client on a Pent
> > IV 3Ghz running CentOS 3.4.
> >
> > Did fill tests with an Overland LoaderXpress LTO2 drive.  The drive
> > maxed at 27MB/s transfer rate.
> >
> > Same drive connected to a Sun v240 running Solaris 9 got 15MB/s.  Same
> > drive connected to a Sun v880 running Solaris 9 got 18MB/s.
> >
> > I have done full backups of 747GB on the v240 at an average 10.2MB/s
> > onto one tape. ;)  Just completing 1.15TB on the v880 at an average  
> > 11.5MB/s.
> >
> > So the question is...
> >
> > Am I missing something big on the Solaris platforms?  I can't believe
> > the 8proc v880 is getting smoked by a single proc Pent IV.
> >
> > Any suggestions welcome.
>
> Do you have a lot of sparse files and are you doing sparse file processing?
> If so, it seems (based on one of my systems) that you can end up CPU-limited
> as the bacula-fd goes through reading a lot of zeros as fast as it can.

Well the two backups are of Oracle database files which I've been told
contain lots of whitespace.  However I do not current have sparse file
checking on.

>
> Other than that, do you have a lot of files on the Solaris machines,
> particularly a lot of files in a single directory? Reading a lot of distinct
> can noticeably slow down backups and I've noticed it on Solaris on
> filesystems like webservers full of small image files or mail spools with
> file-per-message filestores.
The 747GB backup is of ~330 files, the 1.1TB backup was of 30000+
files.  I don't know the makeup of the individual directories though.

Ignoring the results of the two 'real' backups I'm wondering what would
contribute to such low Solaris performance during the btape fill tests.




-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Typical Transfer Rates

John Stoffel
In reply to this post by Slartibartfast-5

Slartibartfast> I have bacula 1.34.2 running as Director, Storage and
Slartibartfast> Client on a Pent IV 3Ghz running CentOS 3.4.

Slartibartfast> Did fill tests with an Overland LoaderXpress LTO2
Slartibartfast> drive.  The drive maxed at 27MB/s transfer rate.

How did you do your fill tests?  Just using 'dd' and a large input
file?  

Slartibartfast> Same drive connected to a Sun v240 running Solaris 9
Slartibartfast> got 15MB/s.  Same drive connected to a Sun v880
Slartibartfast> running Solaris 9 got 18MB/s.

How are the drives connected?  SCSI or FibreChannel?  Which
controllers are you using?  More details please...

Just on the offhand, I suspect that your using a SCSI connection from
your SUNs to the LTO2 drive, but what speed are they negotiating?
Check /var/adm/messages or 'dmesg' output.  It's almost as if you have
the wrong SCSI controllers on the SUN, and that's where the bottleneck
is.

Slartibartfast> I have done full backups of 747GB on the v240 at an
Slartibartfast> average 10.2MB/s onto one tape. ;) Just completing
Slartibartfast> 1.15TB on the v880 at an average 11.5MB/s.

If you were only backing up 330 files for the 750gb backup, then you
do have a bottleneck somewhere.  Is the data local to the SUNs or is
it being pulled over the network from another host?  What kind of
network do you have?  

10mb/s is about what a 100baseT network can do, the theorectical max
is 12.5mb/s.  

Slartibartfast> Am I missing something big on the Solaris platforms?
Slartibartfast> I can't believe the 8proc v880 is getting smoked by a
Slartibartfast> single proc Pent IV.

It's not, I had a V880 with six LTO2 tape drives and I could push data
at over 50mb/s to multiple drives at a time using Veritas NetBackup.
But the data was local to the system on a FibreChannel disk array, the
LTO2 were FibreChannel drives, etc.

When is comes to backups, the entire chain that the data needs to flow
through needs to be looked at, and just one choke point can make a big
difference.  

Slartibartfast> Any suggestions welcome.

Can you copy a large file to the V880 and just use:

  time dd if=/path/to/file bs=64k of=/dev/rmt/1cbn

and see how the time changes for different bs= values?  I seem to
remember that going past 1m (using gnu dd) didn't make alot of
difference, but I'd have to dig back into old email to find my
results.

Good luck, and give us some more info and I'm sure we'll be able to
help.

Jhon


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Bacula-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/bacula-users
Loading...