zpaq

ZPAQ: Worlds Highest Data Compression (Crushes .7 and ZIP in Performance.) Command Reference Examples.

We look at the Worlds most advanced data compression tool - zpaq which can outperform ZIP by 2:1 but can take days to compress data..

thinkmelt@protonmail.com

• 12 min read
ZPAQ: Worlds Highest Data Compression (Crushes .7 and ZIP in Performance.) Command Reference Examples.
  • ZPAQ is the worlds highest data compression - outperforming .7z compression which itself crushes your popular .ZIP compression by a large margin.
  • ZPAQ can take HOURS to compress a file. Where you have near infinite compute and no space to store anything.
  • ZPAQ is slow. Really slow like 341 hours / Terabyte using 12 cores of compute. So where you have days and days and want to compress everything down to a single thumb drive for funsies.
  • ZPAQfranz is listed at the end of this article for even more features, journaling etc..

ZPAQ Installation and Usage Guide (Linux)

ZPAQ is a free, open-source, command-line journaling archiver that provides exceptional compression ratios through deduplication, context-mixing algorithms, and incremental (append-only) updates. It is particularly suited for long-term backups, versioned archives, and data sets requiring maximum size reduction with rollback capability. The current official version is 7.15 (released August 2016), which remains the reference implementation conforming to the ZPAQ open standard format.

Installation on Linux

Preferred method (recommended for most users): Use the native package manager where available.

Debian, Ubuntu, and derivatives (includes zpaq 7.15):

sudo apt update
sudo apt install zpaq

Fedora, RHEL, CentOS, and derivatives:

sudo dnf install zpaq

Arch Linux: Available via AUR (original or zpaqfranz fork).

yay -S zpaq  # or zpaqfranz-git for the enhanced fork

Other distributions: Search for the zpaq package or use the generic binary from the official site.

Compressing War and Peace

  • We took War and Peace and compressed it at level 5. Here is how it came out.
└──╼ $zpaqfranz a wap.zpaq warandpeace030164mbp_djvu.txt -method 5
zpaqfranz v61.2d-JIT,-L4,(2025-04-02)
franz:-method                                   5 0
Creating wap.zpaq at offset 0 + 0
Add 2026-02-22 16:07:24         1          4.019.920 (   3.83 MB) 8T (0 dirs)
1 +added, 0 -removed.

0 + (4.019.920 -> 4.019.920 ->  780.414) = 780.414  @ 436.58 KB/s
Files added +1
8.992s (00:00:08,59.01MB) (all OK)
-rw-rw-r-- 1 c c  780414 Feb 22 16:07 wap.zpaq
-rw-rw-r-- 1 c c 4019920 Feb 22 16:05 warandpeace030164mbp_djvu.txt
┌─[c@parrot]─[~]
└──╼ $python -c "print (780414/4019920)"
0.19413669923779578

Compared to .7z which compressed to 25% of original and zip which compressed to 75% of original - ZIP isn't even in the same ball park as these tools!

-rw-rw-r-- 1 c c  995872 Feb 21 15:36 wnp.7z
-rw-rw-r-- 1 c c 3867598 Feb 21 15:35 wnp.txt
-rw-rw-r-- 1 c c 1324187 Feb 21 15:35 wnp.zip
┌─[c@parrot]─[~]
└──╼ $python -c "print(995872 / 1324187)"
0.7520629639167278
┌─[c@parrot]─[~]
└──╼ $python -c "print(995872 / 3867598)"
0.2574910836131366
┌─[c@parrot]─[~]

Speed - zpaq is slow it took 8.99s to compress 4019920 bytes,  we ran it again using 12 threads and that hardly sped it up:

└──╼ $zpaq a wap.zpaq warandpeace030164mbp_djvu.txt -method 5 -threads 12
zpaq v7.15 journaling archiver, compiled Jan  5 2021
Creating wap.zpaq at offset 0 + 0
Adding 4.019920 MB in 1 files -method 56 -threads 12 at 2026-02-22 21:14:50.
100.00% 0:00:00 + warandpeace030164mbp_djvu.txt 4019920
100.00% 0:00:00 [1..60] 4020168 -method 56,192,1
1 +added, 0 -removed.

0.000000 + (4.019920 -> 4.019920 -> 0.780388) = 0.780388 MB
8.856 seconds (all OK)
  • So it's slow, prepare accordingly, using 12 cores only processes at 850,000 bytes/second.  
  • That works out to about 20 minutes / GB.
  • That works out to 341 hours / terabyte.

Note on forks: The zpaqfranz fork is highly recommended for production use on Linux. It maintains full compatibility with the original ZPAQ format while adding enhanced encryption, paranoid verification, hardware acceleration, and improved multi-version management. Installation follows similar package methods (e.g., sudo apt install zpaqfranz on Debian 13+ or from https://github.com/fcorbelli/zpaqfranz). Commands are identical unless noted.

After installation, the binary is invoked as zpaq.

General Usage

Syntax

zpaq <command> <archive[.zpaq]> [files/directories]... [-options]...

Commands (abbreviations allowed):

  • add (a): Create or incrementally update the archive.
  • extract (x): Extract files or entire archive.
  • list (l): List contents and compare with disk.
  • test: Verify archive integrity.

The archive name defaults to .zpaq extension if omitted.

Directories are processed recursively.

Updates are incremental and deduplicated (only changed fragments are stored).

Highest compression is achieved with -method 5 (or custom ZPAQL configurations).

Encryption uses -key <password> (AES-256 CTR).

ZPAQ does not natively support direct stdin/stdout piping for archiving (use find | xargs or shell scripting for file selection). Output of list or test can be piped to other tools.

Key Options (selected; run zpaq for full list):

  • -method <0-5>: Compression level (0 = store, 5 = maximum).
  • -threads <N>: Number of parallel threads.
  • -force: Force overwrite or content comparison.
  • -until <date|version>: Limit to specific version/date.
  • -all: Show all versions.
  • -to <dir>: Extract/rename to target directory.
  • -not <pattern> / -only <pattern>: Exclude/include files.
  • -key <password>: Encrypt/decrypt.
  • -index <file>: For multi-part remote archives.

Important Notes

  • ZPAQ excels at deduplication across versions but is slower than 7z/RAR for one-time compression.
  • Always test extraction on non-critical data first.
  • For maximum compression on new archives, use -method 5.

25 Command Examples

Example 1: Basic creation of a new archive from a single file (maximum compression)

zpaq add backup.zpaq report.pdf -method 5
  • add: Creates or updates the archive.
  • backup.zpaq: Archive filename (creates if it does not exist).
  • report.pdf: Source file to archive.
  • -method 5: Highest compression level (slowest, best ratio).

Example 2: Incremental backup of an entire directory

zpaq add daily_backup.zpaq /home/user/projects
  • add: Incremental update mode.
  • daily_backup.zpaq: Archive name.
  • /home/user/projects: Directory (recursively included; only changed files added on subsequent runs).

Example 3: Add multiple files with fast compression

zpaq add archive.zpaq file1.txt file2.doc image.jpg -method 1
  • add: Archive creation/update.
  • archive.zpaq: Target archive.
  • file1.txt file2.doc image.jpg: List of source files.
  • -method 1: Fast LZ77 compression (default balance of speed and ratio).

Example 4: Create multi-part archive for large datasets

zpaq add "parts??.zpaq" /large_data_dir
  • add: Update mode.
  • "parts??.zpaq": Wildcard pattern for multi-part archives (creates parts01.zpaq, parts02.zpaq, etc.).
  • /large_data_dir: Source directory.

Example 5: Encrypted incremental backup

zpaq add secure_backup.zpaq /important_data -key MyStrongPass123
  • add: Incremental update.
  • secure_backup.zpaq: Archive.
  • /important_data: Source.
  • -key MyStrongPass123: AES-256 encryption password (prompts if omitted in some builds).

Example 6: Add with explicit thread count and block size

zpaq add fast_backup.zpaq /data -method 3 -threads 8
  • add: Update.
  • fast_backup.zpaq: Archive.
  • /data: Source.
  • -method 3: Good compression with BWT/context mix.
  • -threads 8: Use 8 parallel threads.

Example 7: Use with find and xargs for selective file addition (piping support)

find /docs -name "*.pdf" -print0 | xargs -0 zpaq add pdf_archive.zpaq
  • find ... -print0: Generates null-terminated list of PDF files.
  • xargs -0: Passes the list safely to zpaq (handles spaces/special characters).
  • zpaq add pdf_archive.zpaq: Adds matching files incrementally.

Example 8: Create remote-friendly archive with local index

zpaq add remote???.zpaq /big_dataset -index local_index.zpaq
  • add: Update.
  • remote???.zpaq: Multi-part pattern.
  • /big_dataset: Source.
  • -index local_index.zpaq: Creates/updates index without storing compressed data locally.

Example 9: Basic full extraction

zpaq extract backup.zpaq
  • extract: Extraction mode.
  • backup.zpaq: Archive to extract from (latest version only).

Example 10: Extract specific files to a target directory

zpaq extract archive.zpaq report.pdf images/ -to /restore/
  • extract: Extraction.
  • archive.zpaq: Archive.
  • report.pdf images/: Files/directories to extract.
  • -to /restore/: Destination directory (renames if needed).

Example 11: Extract a previous version by date

zpaq extract history.zpaq -until 2025-12-01
  • extract: Extraction.
  • history.zpaq: Archive.
  • -until 2025-12-01: Extracts state as of the specified date.

Example 12: Extract with force overwrite

zpaq extract current_backup.zpaq -force
  • extract: Extraction.
  • current_backup.zpaq: Archive.
  • -force: Overwrites existing files on disk.

Example 13: List archive contents

zpaq list backup.zpaq
  • list: Listing mode.
  • backup.zpaq: Archive to inspect (shows latest version).

Example 14: List all versions with detailed comparison

zpaq list archive.zpaq -all -force
  • list: Listing.
  • archive.zpaq: Archive.
  • -all: Shows every version (numbered directories).
  • -force: Compares file contents via hashes (ignores dates).

Example 15: List only matching files (piping output example)

zpaq list backup.zpaq -only "*.log" | grep -E "202[5-6]"
  • list ... -only "*.log": Lists only log files from archive.
  • | grep ...: Pipes output to filter by year (example of post-processing).

Example 16: Compare archive with current directory

zpaq list backup.zpaq /current/dir -not =
  • list: Comparison mode.
  • backup.zpaq: Archive reference.
  • /current/dir: Disk directory to compare against.
  • -not =: Shows only differences (excludes identical files).

Example 17: Test archive integrity

zpaq test large_archive.zpaq
  • test: Integrity verification mode.
  • large_archive.zpaq: Archive to validate (decompresses internally and checks hashes).

Example 18: Test with all versions

zpaq test backup.zpaq -all
  • test: Verification.
  • backup.zpaq: Archive.
  • -all: Tests every version.

Example 19: Repack archive with new compression

zpaq extract old.zpaq -test | zpaq add new.zpaq -method 5   # (simulated; use repack where supported)

(Note: Direct repack uses built-in logic; this illustrates piping simulation for extraction test before new add.)

Example 20: Add excluding certain patterns

zpaq add clean_backup.zpaq /project -not "*.tmp" -not "*.bak"
  • add: Update.
  • clean_backup.zpaq: Archive.
  • /project: Source.
  • -not "*.tmp" -not "*.bak": Excludes temporary and backup files.

Example 21: Extract with no attributes preserved

zpaq extract archive.zpaq -noattributes
  • extract: Extraction.
  • archive.zpaq: Archive.
  • -noattributes: Skips restoring permissions/attributes.

Example 22: Custom fragment size for deduplication tuning

zpaq add tuned.zpaq /data -fragment 7
  • add: Update.
  • tuned.zpaq: Archive.
  • /data: Source.
  • -fragment 7: Sets larger deduplication fragment size (642^7 to 81282^7 bytes).

Example 23: Summary mode for large archives (suppress file list)

zpaq add big_backup.zpaq /huge_dir -summary 10
  • add: Update.
  • big_backup.zpaq: Archive.
  • /huge_dir: Source.
  • -summary 10: Shows only top 10 largest files in progress report.

Example 24: Multi-part extraction

zpaq extract "parts??.zpaq"
  • extract: Extraction.
  • "parts??.zpaq": Wildcard for all parts (automatically joins).

Example 25: Advanced incremental backup with maximum options (enterprise context)

zpaq add final_backup.zpaq /critical_data -method 5 -threads 16 -key SecurePass2026 -force -not "*/temp/*"
  • add: Incremental update.
  • final_backup.zpaq: Archive.
  • /critical_data: Source directory.
  • -method 5: Maximum compression.
  • -threads 16: High parallelism.
  • -key SecurePass2026: Encryption.
  • -force: Force content checks.
  • -not "*/temp/*": Exclude temporary subdirectories.

For complete option reference, run zpaq or man zpaq. ZPAQ pairs excellently with PAR2 for added redundancy after archiving. If you require tailored examples for specific data types, sizes, or integration with scripts, provide additional details for further customization.

Manual compilation from official source (always up-to-date with the reference implementation):

Download the source package:

wget https://www.mattmahoney.net/zpaq/zpaq715.zip

Extract:

unzip zpaq715.zip

Compile (requires g++ and make):

make

Install system-wide (optional):

sudo make install

Verify:

zpaq

(Displays usage information.)

ZPAQFRANZ Installation and Usage Guide (Linux)

ZPAQFRANZ is an actively maintained, enhanced fork of the original ZPAQ 7.15 (released 2016). It retains full backward compatibility with the ZPAQ journaling format while adding hardware acceleration, multi-hash paranoid verification, ransomware-neutral append-only snapshots, rsync-friendly incremental transfers, -stdin/-stdout support, ZFS integration, and dozens of specialized commands for backup, disaster recovery, and data management. It excels at deduplicated, multiversioned archives with extreme compression and integrity guarantees, making it ideal for long-term backups, virtual machine images, cloud synchronization, and high-security environments.

The binary is invoked as zpaqfranz. Commands are typically single-letter abbreviations (e.g., a for add/append, x for extract), though some long-form aliases exist. Help is embedded: zpaqfranz h (short), zpaqfranz h h (full), or zpaqfranz h <topic>.

Installation on Linux

Recommended package-manager methods (preferred):

Debian 13+, Ubuntu 24.10+, and derivatives:

sudo apt update
sudo apt install zpaqfranz

Arch Linux and derivatives:

yay -S zpaqfranz-git   # or paru / aur helpers

OpenSUSE:

sudo zypper install zpaqfranz

Generic binary (any Linux distribution, 32/64-bit):
Download the latest static binary from http://www.francocorbelli.it/zpaqfranz/linux (version 60.7u as of February 2026) and place it in /usr/local/bin (make executable with chmod +x).

Compilation from source (for latest development version or custom builds):

Clone or download the repository:

git clone https://github.com/fcorbelli/zpaqfranz.git
cd zpaqfranz

Compile (single-file source since branch 51):

g++ -Dunix -O3 -march=native zpaqfranz.cpp -o zpaqfranz -pthread -static

Install system-wide (optional):

sudo mv zpaqfranz /usr/local/bin/

Verify:

zpaqfranz

(Displays short help and version.)

Note: For hardware-accelerated SHA on supported CPUs (AMD Ryzen, etc.), use the dedicated zpaqfranzhw variant where available, or compile with -DHWSHA2.

General Usage

Syntax

zpaqfranz <command> <archive[.zpaq]> [files/directories]... [-options]...

Main commands (abbreviations shown; full help via zpaqfranz h):

  • a / add: Create or incrementally append a new version (snapshot).
  • x / extract: Extract files or versions.
  • l / list: List contents or compare with disk.
  • i / info: Detailed archive statistics and version history.
  • t / test: Standard integrity test.
  • p / paranoid: Extreme multi-hash verification.
  • Additional specialized: backup, consolidatebackup, zfsadd, zfsbackup, etc.

Key options (selected; run zpaqfranz h franz for zpaqfranz-specific):

  • -method <0-5>: Compression level (0 = store + dedup only; 5 = maximum).
  • -fragment <N>: Deduplication fragment tuning (default 6; same as original ZPAQ).
  • -key <password>: AES-256 encryption.
  • -paranoid: Enables strongest multi-hash verification (SHA-2-256, SHA-3-256, BLAKE3, etc.).
  • -hw: Hardware acceleration for SHA (when using accelerated build).
  • -checksum: Forces default triple-hash (CRC32 + XXHASH64 + SHA-1).
  • -until <N|date>: Limit operation to a specific version or date.
  • -force: Overwrite or force content comparison.
  • -threads <N>: Parallel threads (inherited and optimized).
  • -stdin / -stdout: Pipe support (unique enhancement).
  • -to <dir>: Target directory for extraction.

Deduplication is always enabled in journaling mode (a command) and works identically to original ZPAQ (content-defined fragments, SHA-1 hashes, global cross-version deduplication). The -fragment N option remains fully supported.

Important notes

  • Archives are append-only single .zpaq files (or multipart with wildcards).
  • Each a operation creates a new immutable version/snapshot.
  • Extremely efficient for repeated backups (only changed fragments are stored).
  • Test all operations on non-critical data first.

25 Command Examples

Example 1: Basic creation of a new archive from a single file (maximum compression)

zpaqfranz a backup.zpaq report.pdf -method 5
  • a: Add/append mode (creates new version if archive exists).
  • backup.zpaq: Archive filename.
  • report.pdf: Source file.
  • -method 5: Highest compression level with deduplication.

Example 2: Incremental snapshot of an entire directory

zpaqfranz a daily_backup.zpaq /home/user/projects
  • a: Incremental add (only new/changed fragments stored).
  • daily_backup.zpaq: Archive.
  • /home/user/projects: Recursive directory source.

Example 3: Add multiple files with fast compression

zpaqfranz a archive.zpaq file1.txt file2.doc image.jpg -method 1
  • a: Add mode.
  • archive.zpaq: Target archive.
  • file1.txt file2.doc image.jpg: Source files.
  • -method 1: Fast LZ77-style compression.

Example 4: Multipart archive for very large datasets

zpaqfranz a "parts??.zpaq" /large_data_dir
  • a: Add mode.
  • "parts??.zpaq": Wildcard multipart pattern (creates parts01.zpaq, etc.).
  • /large_data_dir: Source directory.

Example 5: Encrypted incremental backup

zpaqfranz a secure_backup.zpaq /important_data -key MyStrongPass123
  • a: Incremental add.
  • secure_backup.zpaq: Archive.
  • /important_data: Source.
  • -key MyStrongPass123: AES-256 encryption.

Example 6: Add with maximum compression, hardware acceleration, and threads

zpaqfranz a fast_backup.zpaq /data -method 5 -hw -threads 16
  • a: Add mode.
  • fast_backup.zpaq: Archive.
  • /data: Source.
  • -method 5: Maximum compression.
  • -hw: Hardware SHA acceleration.
  • -threads 16: Parallel processing threads.

Example 7: Piping files via stdin for selective addition

find /docs -name "*.pdf" -print0 | zpaqfranz a pdf_archive.zpaq -stdin
  • find ... -print0: Null-terminated file list.
  • zpaqfranz a pdf_archive.zpaq -stdin: Reads file list from stdin (unique zpaqfranz enhancement).

Example 8: Archive information (detailed snapshot history)

zpaqfranz i backup.zpaq
  • i: Info command (shows versions, file counts, deduplicated sizes, statistics).

Example 9: Basic extraction of latest version

zpaqfranz x backup.zpaq
  • x: Extract mode.
  • backup.zpaq: Archive (extracts latest version to current directory).

Example 10: Extract specific files to target directory

zpaqfranz x archive.zpaq report.pdf images/ -to /restore/
  • x: Extract.
  • archive.zpaq: Archive.
  • report.pdf images/: Specific items.
  • -to /restore/: Destination directory.

Example 11: Extract previous version by number

zpaqfranz x history.zpaq -until 5
  • x: Extract.
  • history.zpaq: Archive.
  • -until 5: Limits to version 5 (snapshot rollback).

Example 12: List contents with version filtering

zpaqfranz l backup.zpaq -until 3 -find "*.log"
  • l: List mode.
  • backup.zpaq: Archive.
  • -until 3: Up to version 3.
  • -find "*.log": Wildcard filter.

Example 13: Standard integrity test

zpaqfranz t large_archive.zpaq
  • t: Test command (verifies hashes).
  • large_archive.zpaq: Archive to test.

Example 14: Paranoid-level verification (extreme integrity)

zpaqfranz p backup.zpaq -paranoid
  • p: Paranoid test command.
  • backup.zpaq: Archive.
  • -paranoid: Multi-algorithm hash verification (SHA-2/3, BLAKE3, etc.).

Example 15: Compare archive with current filesystem

zpaqfranz l backup.zpaq /current/dir -force
  • l: List/compare mode.
  • backup.zpaq: Reference archive.
  • /current/dir: Disk path.
  • -force: Content-based comparison (ignores timestamps).

Example 16: Piping stdout for extraction to another tool

zpaqfranz x archive.zpaq important.file -stdout | sha256sum
  • x ... -stdout: Extracts to stdout (zpaqfranz enhancement).
  • | sha256sum: Pipes for external verification.

Example 17: Add with deduplication fragment tuning

zpaqfranz a tuned.zpaq /data -fragment 5 -method 4
  • a: Add.
  • tuned.zpaq: Archive.
  • /data: Source.
  • -fragment 5: Finer fragments for better deduplication.
  • -method 4: High compression.

Example 18: Consolidated multipart backup

zpaqfranz consolidatebackup "parts??.zpaq" consolidated.zpaq
  • consolidatebackup: Specialized command to merge multipart into single archive.
  • "parts??.zpaq": Source multipart pattern.
  • consolidated.zpaq: Output file.

Example 19: Archive with forced checksums

zpaqfranz a checksummed.zpaq /project -checksum -method 3
  • a: Add.
  • checksummed.zpaq: Archive.
  • /project: Source.
  • -checksum: Enables default triple-hash verification.
  • -method 3: Balanced compression.

Example 20: Extract with force overwrite

zpaqfranz x current_backup.zpaq -force
  • x: Extract.
  • current_backup.zpaq: Archive.
  • -force: Overwrites existing files.

Example 21: Info on multipart archive

zpaqfranz i "multi???.zpaq"
  • i: Info.
  • "multi???.zpaq": Multipart wildcard.

Example 22: Add excluding patterns (via shell or options)

zpaqfranz a clean.zpaq /project -not "*.tmp"   # note: -not supported via extended help

(Use shell find for complex excludes if needed; core supports pattern filtering.)

Example 23: Hardware-accelerated paranoid test

zpaqfranz p secure.zpaq -paranoid -hw
  • p: Paranoid test.
  • secure.zpaq: Archive.
  • -paranoid: Extreme verification.
  • -hw: Hardware acceleration.

Example 24: Piped stdin creation from tar stream (hybrid workflow)

tar -c /data | zpaqfranz a hybrid.zpaq -stdin -method 5
  • tar -c /data: Creates stream.
  • zpaqfranz a ... -stdin: Imports via stdin.
  • -method 5: Maximum compression.

Example 25: Comprehensive enterprise snapshot with all enhancements

zpaqfranz a final_backup.zpaq /critical_data -method 5 -fragment 6 -key SecurePass2026 -hw -threads 32 -checksum -paranoid
  • a: Incremental snapshot.
  • final_backup.zpaq: Archive.
  • /critical_data: Source directory.
  • -method 5: Maximum compression.
  • -fragment 6: Default deduplication tuning.
  • -key SecurePass2026: Encryption.
  • -hw: Hardware acceleration.
  • -threads 32: High parallelism.
  • -checksum: Triple-hash.
  • -paranoid: Extreme verification on add.

For complete options and specialized commands (ZFS, robocopy-like, etc.), run zpaqfranz h h or consult the embedded help and wiki at https://github.com/fcorbelli/zpaqfranz/wiki. ZPAQFRANZ pairs exceptionally well with PAR2 for additional redundancy. If you require customized examples for specific workloads (e.g., ZFS snapshots, VM images, or scripting), provide further details for tailored sequences.

Linux Rocks Every Day