upstream tag v1.5.7-kernel, which is signed by upstream's signing key
EF8FE99528B52FFD.
Link: https://github.com/facebook/zstd/releases/tag/v1.5.7
Link: https://github.com/facebook/zstd/releases/tag/v1.5.7-kernel
Link: https://keyserver.ubuntu.com/pks/lookup?search=EF8FE99528B52FFD&fingerprint=on&op=index
Signed-off-by: Nick Terrell <terrelln@fb.com>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEmIwAqlFIzbQodPwyuzRpqaNEqPUFAmfjI5oACgkQuzRpqaNE
qPUZJA//foLgy1etgBSTaUbCCIFxOLguFjmH2qfs/0yGX1ekhlv5jXobyUmKhYVM
q0WR3G4lS1MNC40T9zNoKR0GfmZGyrCjOlGkwMEwdNYc+4y5sWujbckE+Xl/mgld
Gz1NEEentFNIeC5htnBX797PJldqawHl6OYax/+6gVZyeLPYfbNYtTGy30fQIvcz
vdIR/KCR2XzHn8+xah1zga5Ey/8LAXpgoYY9Pu3J3HWFRTV35laUe0nZ8EQ1mW3q
nGritp0453RFJgD1wHewp1CgJx9lAixPAMPZ5BCOqOxsCxyalbvefWc6u/cS3zJM
KEeKChyF6k5VqaW4A9jVeKq+HoGfngYjFJmELeKG4vK1d2UwMeDZRJ2IfkKej7xK
0awM0E0LO95H0mWEPhI3bmNbcfOLiJ4TIdWcr/sztF8Vv7fxKkK67Bwk4NTYmyPv
sgFZMEyw0eFYNf8/0j9FCATu7AgmbF3yes4vExuAy0cgZaiNxOxaAspRM2A8Tmdf
WWiAIsS6ZYwp6L1Gm6Rva+GRB15I3wevxOuEJ4kTsVVgvzgLQ+N6Fn5H5g2zgb1Q
hgRlJx6ivyRpoaJhbBB7tqNsK38lQ53i0DHQ21jkBHEPFmRzLnvC+D205Dz3tQK5
kwPAGOCbxoiQbqzhY4NOm75ZPxzy8OW7ygjow0HaX6fgv9Y9n9s=
=+maf
-----END PGP SIGNATURE-----
Merge tag 'zstd-linus-v6.15-rc1' of https://github.com/terrelln/linux
Pull zstd updates from Nick Terrell:
"Update zstd to the latest upstream release v1.5.7.
The two major motivations for updating Zstandard are to keep the code
up to date, and to expose API's needed by Intel for the QAT
compression accelerator.
Imported cleanly from the upstream tag v1.5.7-kernel, which is signed
by upstream's signing key EF8FE99528B52FFD"
Link: https://github.com/facebook/zstd/releases/tag/v1.5.7
Link: https://github.com/facebook/zstd/releases/tag/v1.5.7-kernel
Link: https://keyserver.ubuntu.com/pks/lookup?search=EF8FE99528B52FFD&fingerprint=on&op=index
* tag 'zstd-linus-v6.15-rc1' of https://github.com/terrelln/linux:
zstd: Import upstream v1.5.7
Due to pending percpu improvements in -next, GCC9 and GCC10 are
crashing during the build with:
lib/zstd/compress/huf_compress.c:1033:1: internal compiler error: Segmentation fault
1033 | {
| ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-9/README.Bugs> for instructions.
The DYNAMIC_BMI2 feature is a known-challenging feature of
the ZSTD library, with an existing GCC quirk turning it off
for GCC versions below 4.8.
Increase the DYNAMIC_BMI2 version cutoff to GCC 11.0 - GCC 10.5
is the last version known to crash.
Reported-by: Michael Kelley <mhklinux@outlook.com>
Debugged-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: https://lore.kernel.org/r/SN6PR02MB415723FBCD79365E8D72CA5FD4D82@SN6PR02MB4157.namprd02.prod.outlook.com
Cc: Linus Torvalds <torvalds@linux-foundation.org>
In addition to keeping the kernel's copy of zstd up to date, this update
was requested by Intel to expose upstream's APIs that allow QAT to accelerate
the LZ match finding stage of Zstd.
This patch is imported from the upstream tag v1.5.7-kernel [0], which is signed
with upstream's signing key EF8FE99528B52FFD [1]. It was imported from upstream
using this command:
export ZSTD=/path/to/repo/zstd/
export LINUX=/path/to/repo/linux/
cd "$ZSTD/contrib/linux-kernel"
git checkout v1.5.7-kernel
make import LINUX="$LINUX"
This patch has been tested on x86-64, and has been boot tested with
a zstd compressed kernel & initramfs on i386 and aarch64. I benchmarked
the patch on x86-64 with gcc-14.2.1 on an Intel i9-9900K by measruing the
performance of compressed filesystem reads and writes.
Component, Level, Size delta, C. time delta, D. time delta
Btrfs , 1, +0.00%, -6.1%, +1.4%
Btrfs , 3, +0.00%, -9.8%, +3.0%
Btrfs , 5, +0.00%, +1.7%, +1.4%
Btrfs , 7, +0.00%, -1.9%, +2.7%
Btrfs , 9, +0.00%, -3.4%, +3.7%
Btrfs , 15, +0.00%, -0.3%, +3.6%
SquashFS , 1, +0.00%, N/A, +1.9%
The major changes that impact the kernel use cases for each version are:
v1.5.7: https://github.com/facebook/zstd/releases/tag/v1.5.7
* Add zstd_compress_sequences_and_literals() for use by Intel's QAT driver
to implement Zstd compression acceleration in the kernel.
* Fix an underflow bug in 32-bit builds that can cause data corruption when
processing more than 4GB of data with a single `ZSTD_CCtx` object, when an
input crosses the 4GB boundry. I don't believe this impacts any current kernel
use cases, because the `ZSTD_CCtx` is typically reconstructed between
compressions.
* Levels 1-4 see 5-10% compression speed improvements for inputs smaller than
128KB.
v1.5.6: https://github.com/facebook/zstd/releases/tag/v1.5.6
* Improved compression ratio for the highest compression levels. I don't expect
these see much use however, due to their slow speeds.
v1.5.5: https://github.com/facebook/zstd/releases/tag/v1.5.5
* Fix a rare corruption bug that can trigger on levels 13 and above.
* Improve compression speed of levels 5-11 on incompressible data.
v1.5.4: https://github.com/facebook/zstd/releases/tag/v1.5.4
* Improve copmression speed of levels 5-11 on ARM.
* Improve dictionary compression speed.
Signed-off-by: Nick Terrell <terrelln@fb.com>
Updates the kernel's zstd library to v1.5.2, the latest zstd release.
The upstream tag it is updated to is `v1.5.2-kernel`, which contains
several cherry-picked commits on top of the v1.5.2 release which are
required for the kernel update. I will create this tag once the PR is
ready to merge, until then reference the temporary upstream branch
`v1.5.2-kernel-cherrypicks`.
I plan to submit this patch as part of the v6.2 merge window.
I've done basic build testing & testing on x86-64, i386, and aarch64.
I'm merging these patches into my `zstd-next` branch, which is pulled
into `linux-next` for further testing.
I've benchmarked BtrFS with zstd compression on a x86-64 machine, and
saw these results. Decompression speed is a small win across the board.
The lower compression levels 1-4 see both compression speed and
compression ratio wins. The higher compression levels see a small
compression speed loss and about neutral ratio. I expect the lower
compression levels to be used much more heavily than the high
compression levels, so this should be a net win.
Level CTime DTime Ratio
1 -2.95% -1.1% -0.7%
3 -3.5% -1.2% -0.5%
5 +3.7% -1.0% +0.0%
7 +3.2% -0.9% +0.0%
9 -4.3% -0.8% +0.1%
Signed-off-by: Nick Terrell <terrelln@fb.com>
