Arch Linux developers plan to use zstd in pacman

arch linux logo

Arch Linux developers have released recently through a statement on your intention to enable support for compression algorithm zstd (included since November 2017 in Linux kernel 4.14) in the pacman package manager.

Since Arch Linux developers performed the comparison of different compression algorithmsIn the end they opted to plan to use zstd instead of the default compression algorithm in devtools. The current compression method is "xz-cz-", which is single threaded and slow so the team wants to replace it with a faster algorithm.

Compared to the xz algorithm, using zstd will speed up packet compression and unpacking (as it offers large search window and fast entropy encoding stage, using Finite State Entrop), maintaining compression level. As a result, switching to zstd will increase the speed of package installation.

The zstd compression algorithm offers faster compression and decompression, while maintaining a compression ratio comparable to xz. This will speed up the installation of the package with pacman, without further hassle. They commented in the statement Arch Linux developers

Support for compressing packages using zstd that pose integrate later, it will appear in the version of Pac-man 5.2, but installing such packages will require a version of zarchd libarchive.

Pacman is the Linux Arch Linux package manager, it is able to resolve dependencies, and automatically download and install all the necessary packages. In theory, the user only needs to execute a single command to fully update the system.

Pacman uses tar-packed and gzipped or xz-compressed files for all packages, each containing compiled binaries. Packages are downloaded via FTP, you can also use HTTP and local files, depending on how each repository is configured. Complies with Linux Arch Build System (ABS) used to create packages from source code.

About Zstandard

zstandard (zstd) was designed to provide a compression ratio comparable to that of the DEFLATE algorithm, but faster, especially for decompression. It can be adjusted with compression levels ranging from negative 5 (fastest) to 22 (slowest compression speed, but best compression ratio).

The zstd package includes parallel implementations (multithreaded) compression and decompression. As of version 1.3.2, zstd optionally implements very long-range search and deduplication similar to rzip or lrzip.

Compression speed can vary by a factor of 20 or more between the fastest and slowest levels, while decompression is uniformly fast, varying by less than 20% between the fastest and slowest levels.

Zstd has a maximum compression level provides a compression ratio close to lzma, lzham and ppmx and works better than lza or bzip2. Zstandard reaches the current Pareto frontier, since it is decompresses faster than any other algorithm available currently with a similar or better compression ratio.

Dictionaries can have a huge impact on the compression ratio of small files, so the Zstandard algorithm can use a user-supplied compression dictionary. It also offers a training mode, capable of generating a dictionary from a set of samples.

In particular, a dictionary can be loaded to process large sets of files with redundancy between files, but not necessarily within each file, eg log files.

Therefore, before starting with the distribution of the compressed packages using zstd within Arch Linux channels lUsers must first install libarchive at least version 3.3.3-1 (a package with this version was prepared a year ago, so most likely the required version of libarchive is already installed).

Packages compressed by zstd will be shipped with the extension ".pkg.tar.zst".


Be the first to comment

Leave a Comment

Your email address will not be published. Required fields are marked with *

*

*

  1. Responsible for the data: AB Internet Networks 2008 SL
  2. Purpose of the data: Control SPAM, comment management.
  3. Legitimation: Your consent
  4. Communication of the data: The data will not be communicated to third parties except by legal obligation.
  5. Data storage: Database hosted by Occentus Networks (EU)
  6. Rights: At any time you can limit, recover and delete your information.