TOA Reed-Solomon Archiving! Command Reference. Make Archive Files That Repair!

Historically you backed up your files once - then ten years later when it's all long forgotten and you needed the files back they were corrupted! So many of us ended up paying a recovery service to attempt a repair! The smart archivists among us would make a double backup and if one failed they always had a second. What if there was a way to put forward error-correcting information into the file? We introduce toa - which is the newest hot project from the new Rust language. If you want to get into the meaty details on how Reed-Solomon mathematically can repair your information we recommend a good Wikipedia reference

TOA is a modern, open-source compression format that combines high compression ratios (comparable to xz using an LZMA2 variant) with built-in Reed-Solomon error correction. This makes it particularly suitable for long-term archival storage, where protection against bit rot or minor corruption is valuable. The format supports streaming operations and parallel processing, and includes BLAKE3 hashing for cryptographic integrity verification.  

• Installation guide (here)
• Because toa supports 1 file at a time - typically it is combined with tar.  Here are 25 command examples to help you boilerplate your archiving project!

Guide to Using TOA with tar for Backup and Restore of Directories

Because TOA is a stream-based file compressor (not an archiver), it is paired with tar via standard input/output pipes to create and restore directory backups. The resulting archives use the conventional .tar.toa extension. This combination provides the full fidelity of tar (permissions, ownership, timestamps, symlinks) with TOA’s superior compression and error-resilience features.

The examples below demonstrate practical backup and restore workflows. The first 20 examples focus on local operations. Examples 21–25 incorporate rsync for efficient, resumable transfer of the resulting .tar.toa archives to remote systems. All commands assume a modern Linux environment with TOA installed (via cargo install toa or pre-built binary). Use sudo when operating on system directories. Always verify archives and test restores in a non-production location.

Example 1: Basic backup of a directory (default settings)

tar -cf - /path/to/directory/ | toa > backup.tar.toa

tar -cf - creates an uncompressed tar stream to stdout; toa (default preset 6, standard ECC) compresses it to backup.tar.toa. This provides balanced compression with error correction for general-purpose backups.

Example 2: Backup with maximum compression

tar -cf - /path/to/directory/ | toa -9 > backup.tar.toa

tar -cf - produces the tar stream; toa -9 applies the highest compression preset. This minimizes archive size for storage-constrained environments while retaining full error correction.

Example 3: Backup with fastest compression

tar -cf - /path/to/directory/ | toa -0 > backup.tar.toa

tar -cf - creates the stream; toa -0 selects the ultra-fast preset. Suitable for time-critical backups where speed is prioritized over size.

Example 4: Backup with no error correction on payload

tar -cf - /path/to/directory/ | toa --ecc none > backup.tar.toa

tar -cf - generates the tar stream; toa --ecc none disables Reed-Solomon correction on data blocks (metadata remains protected). This achieves slightly better compression ratios when ECC overhead is unnecessary.

Example 5: Backup with paranoid error correction

tar -cf - /path/to/directory/ | toa --ecc paranoid > backup.tar.toa

tar -cf - creates the stream; toa --ecc paranoid applies stronger Reed-Solomon protection (12.5 % overhead). Ideal for single-copy archives on less reliable storage media.

Example 6: Backup with verbose tar output

tar -cvf - /path/to/directory/ | toa > backup.tar.toa

tar -cvf - creates the stream with verbose file listing; toa compresses it. The verbosity aids monitoring during large backups.

Example 7: Backup while excluding patterns

tar -cf - --exclude="*.tmp" --exclude="*.log" /path/to/directory/ | toa > backup.tar.toa

tar excludes matching files before streaming; toa compresses the result. This produces a cleaner, smaller archive by omitting temporary files.

Example 8: Backup with absolute paths

tar -cf - -P /path/to/directory/ | toa > backup.tar.toa

tar -P (absolute names) preserves full paths in the tar stream; toa compresses it. Useful for system restores requiring exact original locations.

Example 9: Backup with numeric owner/group

tar -cf - --numeric-owner /path/to/directory/ | toa > backup.tar.toa

tar --numeric-owner stores numeric IDs; toa compresses the stream. This ensures consistent restoration across systems with differing user databases.

Example 10: Incremental backup (tar snapshot + TOA)

tar -cf - -g snapshot.snar /path/to/directory/ | toa > backup.tar.toa

tar -g uses a snapshot file for incremental operation; toa compresses the resulting stream. Subsequent runs back up only changed files.

Example 11: List contents of a TOA-compressed tar archive

toa -d backup.tar.toa | tar -t

toa -d decompresses the stream to stdout; tar -t lists contents without extraction. This verifies the archive structure.

Example 12: Show TOA archive metadata

toa -l backup.tar.toa

toa -l displays TOA-specific metadata and block information, including compression preset, ECC level, and integrity hashes.

Example 13: Full restore to current directory

toa -d backup.tar.toa | tar -xf -

toa -d decompresses the archive stream to stdout; tar -xf - extracts it. Files are restored with original structure and metadata.

Example 14: Restore to a specific directory

toa -d backup.tar.toa | tar -xf - -C /target/restore/path/

toa -d streams the decompressed tar; tar -C changes to the target directory before extraction. This directs the restore safely.

Example 15: Selective restore of a single file

toa -d backup.tar.toa | tar -xf - -C /target/restore/path/ path/to/specific/file.txt

toa -d decompresses; tar extracts only the specified member. This enables targeted recovery.

Example 16: Backup multiple directories

tar -cf - /path/to/dir1/ /path/to/dir2/ | toa > backup.tar.toa

tar includes multiple top-level directories in the stream; toa compresses the combined archive.

Example 17: Backup following symbolic links

tar -cf - -h /path/to/directory/ | toa > backup.tar.toa

tar -h dereferences symlinks; toa compresses the expanded content. This includes target files rather than links.

Example 18: Backup with maximum threads for TOA

tar -cf - /path/to/directory/ | toa -9 --threads 0 > backup.tar.toa

tar -cf - streams the data; toa -9 --threads 0 uses all available CPU cores for fastest possible high-compression processing.

Example 19: Verify archive integrity after creation

toa --verify backup.tar.toa

toa --verify checks BLAKE3 hashes and Reed-Solomon integrity without full decompression. This confirms the archive is uncorrupted.

Example 20: Restore while keeping the TOA archive

toa -dk backup.tar.toa | tar -xf - -C /target/restore/path/

toa -dk decompresses while retaining the original .toa file; tar extracts the stream. Useful when the compressed archive must be preserved.

Example 21: High-compression backup followed by rsync transfer

tar -cf - /path/to/directory/ | toa -9 > backup.tar.toa && rsync -avz backup.tar.toa user@remotehost:/backups/

tar | toa -9 creates a maximally compressed archive; rsync -avz transfers it with archive mode, verbosity, and transit compression.

Example 22: Rsync transfer of a TOA-compressed tar archive with progress and resume

rsync -avzP --partial backup.tar.toa user@remotehost:/backups/

(Assumes prior creation.) -a preserves attributes, -v verbose, -z compresses transfer, -P enables progress and partial/resume support. This ensures reliable delivery of large .tar.toa files.

Example 23: Rsync directory to remote, then create TOA-compressed tar remotely

rsync -avz /path/to/directory/ user@remotehost:/remote/directory/ && ssh user@remotehost "tar -cf - /remote/directory/ | toa -9 > /remote/backup.tar.toa"

rsync transfers raw files; remote tar | toa -9 performs compression on the destination, minimizing local resource usage.

Example 24: Batch backup multiple directories with TOA, then rsync

for d in dir1 dir2 dir3; do tar -cf - "$d/" | toa -9 > "${d}.tar.toa"; done && rsync -avz *.tar.toa user@remotehost:/backups/

The loop creates individual high-compression archives; rsync -avz transfers all resulting files efficiently.

Example 25: High-compression backup, rsync with resume, and local cleanup

tar -cf - /path/to/directory/ | toa -9 > temp.tar.toa && rsync -avzP --partial temp.tar.toa user@remotehost:/backups/ && rm temp.tar.toa

tar | toa -9 creates the archive; rsync -avzP --partial transfers with resume capability; rm removes the local copy after successful transfer.

These 25 examples provide a complete, production-ready reference for leveraging TOA’s advanced compression and error-correction capabilities together with tar for robust directory backup and restore workflows. Adjust paths, ECC levels, thread counts, and remote credentials to suit your specific requirements. For extremely large datasets, consider TOA’s --block-count and --threads options to optimize performance. Periodic integrity checks with toa --verify are strongly recommended for long-term archives.