Use lzo instead of zlib compression.

[Mark Harvey]

A patch to change data compression used from the zlib package to lzo.

Why: I've see many reports where the use of lzo compression package results in lower CPU (in my mind, more CPU may mean faster throughput).

My testing within an Oracle VirtualBox guest Linux using 'top' showed vtltape daemon CPU usage down from 20 - 30% to 3 - 8%

Compiling with zlib is still required (so we can still read data from older format media).

Before I submit to 'github', I'm after anybody willing to test this patch and provide feedback.
** Any data written using this patch will not be readable without lzo support patch.. **

This patch is on top of mhvtl-1.1-1 (mhvtl-2011-12-24.tgz).

Note: You will need to install the lzo-devel package on your Linux distro before attempting to compile mhvtl with this patch.

0001-Swap-compression-from-zlib-to-lzo.patch

Cheers
Mark

[ap2010]

Hi Mark,

here a few results. I haven't really looked at CPU usage, but more on speed an compatibility.

Hardware: laptop with 1.4 GHz P4 running Fedora 12 and bacula.

When doing a mail backup from my humble nas server normally (zlib) I have the following results:

Size: 1.3 GB
Time: 10m35s
Data file size:  832840K
Data rate: 2089 KB/s

Doing the same backup with lzo compression:

Size: 1.3 GB
Time: 10m44s
Data file size: 1056M
Data rate: 2060 KB/s

Looking at the results I can assume that mhvtl probably is not the bottleneck in this as the times are quite similar. Looking at the data file size I can see that the compression is not as effective for lzo as zlib, but if the cpu load is considerably lower I can live with that. The longer time can easily be attributed to the fact that the written data file is larger (and that takes more time).

FYI I tried to restore a file using bacula which went fine. Although the compression on the drive was set to lzo, the old gzip created files can be read back without a problem.

Regards,

Albert

[ap2010]

Hmm,

something else must have been running on the laptop together with the backup.

Here is the latest result. Backup of nearly exactly the same data.

Size: 1.3 GB
Time: 8m20s
Data file size: 1100MB
Data rate: 2653 KB/s

Which is nearly 25% faster. Good work!

Albert

[Mark Harvey]

How much RAM in your test env ?

It could be that caching on 2nd run is skewing the results.

I've pushed new code to github and now the compression type is configurable at runtime.

vtlcmd <> compression lzo

vtlcmd <> compression zlib

Or setable via device.conf

Sent from my iPad

On Jan 15, 2012, at 3:11, "ap2010 [via MHVTL - Linux Virtual Tape Library - Community Forums]"<[hidden email]> wrote:

Hmm,

something else must have been running on the laptop together with the backup.

Here is the latest result. Backup of nearly exactly the same data.

Size: 1.3 GB
Time: 8m20s
Data file size: 1100MB
Data rate: 2653 KB/s

Which is nearly 25% faster. Good work!

Albert

---

If you reply to this email, your message will be added to the discussion below:

http://mhvtl-linux-virtual-tape-library-community-forums.966029.n3.nabble.com/Use-lzo-instead-of-zlib-compression-tp3639340p3659205.html

To start a new topic under MHVTL - Linux Virtual Tape Library - Community Forums, email [hidden email]
To unsubscribe from MHVTL - Linux Virtual Tape Library - Community Forums, click here.
NAML

[ap2010]

Hi Mark,

it has 1.5 GB of memory. In this case it wasn't that, I did the run after a reboot.

I hadn't found the vtlcmd option yet, thanks.

Albert

On 01/14/2012 11:38 PM, Mark Harvey [via MHVTL - Linux Virtual Tape Library - Community Forums] wrote:

How much RAM in your test env ?

It could be that caching on 2nd run is skewing the results.

I've pushed new code to github and now the compression type is configurable at runtime.

vtlcmd <> compression lzo

vtlcmd <> compression zlib

Or setable via device.conf

Sent from my iPad

On Jan 15, 2012, at 3:11, "ap2010 [via MHVTL - Linux Virtual Tape Library - Community Forums]"<[hidden email]> wrote:

Hmm,

something else must have been running on the laptop together with the backup.

Here is the latest result. Backup of nearly exactly the same data.

Size: 1.3 GB
Time: 8m20s
Data file size: 1100MB
Data rate: 2653 KB/s

Which is nearly 25% faster. Good work!

Albert

---

If you reply to this email, your message will be added to the discussion below:

http://mhvtl-linux-virtual-tape-library-community-forums.966029.n3.nabble.com/Use-lzo-instead-of-zlib-compression-tp3639340p3659205.html

To start a new topic under MHVTL - Linux Virtual Tape Library - Community Forums, email [hidden email]
To unsubscribe from MHVTL - Linux Virtual Tape Library - Community Forums, click here.
NAML

---

If you reply to this email, your message will be added to the discussion below:

http://mhvtl-linux-virtual-tape-library-community-forums.966029.n3.nabble.com/Use-lzo-instead-of-zlib-compression-tp3639340p3659776.html

To unsubscribe from Use lzo instead of zlib compression., click here.
NAML