The compression of data is predicated on the existence of certain predictable or typical statistical characteristics in data, which must be identified by expert or inferred by algorithm. This information is called the source model, because it models the statistical source from which the data is presumed to be drawn. The technology the Inventors present, called “compression with model-free encoding,” is based on the surprising discovery that the source model, a key piece of information for any compression system, does not need to be known at the upstream encoder at all for efficient compression to take place. This recognition is counterintuitive and offers a substantially different way of performing compression than current methods. The technology therefore entails new algorithms for encoders, decoders and other system components, which may have application beyond data compression.
Compression with model-free encoding is completed by “blindly” decimating the original data in the upstream encoder to an agreed upon rate. The output of this process is a set of succinct but potentially non-unique specifications on the data: this is the compressed data stream. In the downstream decoder, these specifications are combined with the statistical characteristics of the data type, i.e. the source model, to recover a solution that agrees with the original data.