5 results for tags
compression x
-
Intel(R) Intelligent Storage Acceleration Library
ISA-L is a collection of optimized low-level functions targeting storage applications. ISA-L includes:Mon Jun 15 15:26:37 2020 - permalink -
Erasure codes - Fast block Reed-Solomon type erasure codes for any encode/decode matrix in GF(2^8).
CRC - Fast implementations of cyclic redundancy check. Six different polynomials supported.
iscsi32, ieee32, t10dif, ecma64, iso64, jones64.
Raid - calculate and operate on XOR and P+Q parity found in common RAID implementations.
Compression - Fast deflate-compatible data compression.
De-compression - Fast inflate-compatible data compression.
BSD 3-Clause "New" or "Revised" License.
According to https://github.com/catid/cm256, ISA-L uses a O(N^3) Gaussian elimination solver for decoding. The CM256 decoder solves the linear system using a fast O(N^2) LDU-decomposition algorithm from "Pivoting and Backward Stability of Fast Algorithms for Solving Cauchy Linear Equations" (T. Boros, T. Kailath, V. Olshevsky), which was hand-optimized for memory accesses.
See also https://github.com/catid/leopard for a fast O(N log(N)) RS erasure decoder.- https://github.com/intel/isa-l
-
Lzip - LZMA lossless data compressor
Lzip is a lossless data compressor with a user interface similar to the one of gzip or bzip2. Lzip can compress about as fast as gzip (lzip -0) or compress most files more than bzip2 (lzip -9). Decompression speed is intermediate between gzip and bzip2. Lzip is better than gzip and bzip2 from a data recovery perspective. Lzip has been designed, written and tested with great care to replace gzip and bzip2 as the standard general-purpose compressed format for unix-like systems.Mon Jan 13 13:50:35 2020 - permalink -
The lzip file format is designed for data sharing and long-term archiving, taking into account both data integrity and decoder availability:
- The lzip format provides very safe integrity checking and some data recovery means. The lziprecover program can repair bit flip errors (one of the most common forms of data corruption) in lzip files, and provides data recovery capabilities, including error-checked merging of damaged copies of a file.
- The lzip format is as simple as possible (but not simpler). The lzip manual provides the source code of a simple decompressor along with a detailed explanation of how it works, so that with the only help of the lzip manual it would be possible for a digital archaeologist to extract the data from a lzip file long after quantum computers eventually render LZMA obsolete.
- Additionally the lzip reference implementation is copylefted, which guarantees that it will remain free forever.
The suffix .lz is used for files compressed with lzip.
See https://lists.dyne.org/lurker/message/20181020.205455.fbab5607.en.html for negative claims about Lzip.- https://nongnu.org/lzip/lzip.html
-
Re: [DNG] How to unarchive an .xz
Overview of current (2018-10) compression algorithms.Mon Oct 22 13:55:56 2018 - permalink -- https://lists.dyne.org/lurker/message/20181020.205455.fbab5607.en.html
-
research!rsc: Zip Files All The Way Down
Self-referential LZ file, by no other than Russ Cox.Tue May 4 20:28:18 2010 - permalink -
See also https://research.swtch.com/nih, where Ken Thompsons code for a "trusting trust" attack is described in detail.
The unix v6 simulator at https://research.swtch.com/v6 does not work on Chrome or Brave under Android, but it does work on Firefox (Android 12).- http://research.swtch.com/2010/03/zip-files-all-way-down.html
-
LZMA Utils
Compression tool similar to gzip and bzip2, with better compression and speed than bzip2 at the cost of higher memory consumption (mostly for compression). From a GNU coreutils announcement.Mon Dec 3 17:05:42 2007 - permalink -- http://tukaani.org/lzma/