I just brought up a “new” mail server with FreeBSD 13 (actually refurbished an old one with more memory, new disks, and a fresh OS load). I’ve always used lz4 compression on these servers in the past, and wanted to know if the new zstd compression would be faster or slower, and whether the compression ratios are better or worse.
From my research, mostly from https://github.com/openzfs/zfs/pull/9735, it sounded like zstd would be comparable to lz4 but not wildly faster. If I could get gzip-level compression at lz4-level speeds, I would be happy. So here goes.
The machine is a dual Xeon L5630 2.13 GHz with 72GB of RAM. There are two pools:
- zroot: 2x 1TB SSD mirror, NVMe SSD SLOG
- tank: 6x 2TB 2.5″ spinning rust, raidz2, NVMe SSD SLOG
The “test job” was to tar up the source directory from the SSD mirror and untar it on the target pool (spinning raidz2). The directory has 50 user mailboxes in dovecot’s mdbox format, 38GB of raw data. The hope is that by changing the compression settings of the target pool we should see the effect of those changes only.
Results of note:
- lz4 is the same speed as no-compression, with a 13% reduction in storage required.
- zstd uses 22% less space than lz4, with a 10% speed penalty
- zstd-fast-1 is a little faster than zstd, but slower than lz4, with the same compression as lz4
- zstd-10 is the same compression as zstd, but is a whopping 62% slower
- zstd comes very close to the goal of “gzip compression with lz4 speed”. The compression is as good as gzip, with speed only 10% slower.
Conclusions
Based on these tests which are very specific to this machine’s workload, the best choices are zstd with default settings, or lz4. The trade-off is between an extra 22% space efficiency vs the 10% speed penalty. In general with these machines, because of the number and type of disk bays they have, and email users’ ever-increasing INBOX bloat, they tend to run out of storage space before they have performance problems. So I think we’ll go with zstd.
I recently read that there is a new version of zstd with many efficiency improvements. Since FreeBSD is using OpenZFS upstream, I expect those improvements will probably be available in FreeBSD this year. If possible, I’ll re-run these tests then.