Tufa Labs has introduced tufazip, a self-learning AI transformer model that compresses one gigabyte of Wikipedia text into just 105.39 megabytes without losing any data. The method relies entirely on a neural network as its predictor, achieving the smallest result on the Large Text Compression Benchmark for any entry that does not mix a transformer with hand-built models. According to a research post, a dozen hybrid compressors currently rank higher on the benchmark, but none of them relies on a neural network alone.
Tufa Labs Neural Network Architecture
Researcher Tommy He built the full package, which includes a small, 4.33-megabyte pre-trained starter model stored at low precision to save space. This starter model is required to unpack the file and contains just two of the 12 layers of the full 163-million-parameter Transformer. Once the full model spins up in memory, it copies these two stages back and forth to fill all 12 layers, with each copy continuing to learn independently. This approach differs from traditional chatbots or cloud AI, functioning instead as a self-contained language model that learns on the fly.

The compression is lossless, meaning there is no risk of data loss. To ensure this, predictor updates must be fully deterministic. Training occurs in two stages. The starter, a 34-million-parameter model, is trained offline before any compression begins. The full model then learns online by predicting each token and training on it. The unpacking side repeats the same steps, rebuilding the model alongside the text. The total file size includes only the compressed text, the unpacking program, and the starter model, excluding the full trained model.
Hardware Requirements and Processing Time
While the compression ratio is impressive, the process is computationally expensive and currently serves research purposes rather than practical use. Running the 105.39-megabyte version requires two Nvidia B200 GPUs and two Intel Xeon Platinum 8570 CPUs. The system takes 631,548 seconds, or about 7.3 days, to compress the data and 645,748 seconds, or about 7.5 days, to decompress it. These times do not include the offline training of the starter model.

During the process, the GPUs peak at 43.37 gibibytes of video RAM. Total memory usage, including the GPUs, peaks at roughly 68,525 mebibytes during compression and 73,748 mebibytes during decompression. The first version of the model, which learned entirely online and resulted in a 105.98-megabyte file, took 647,595 seconds to compress and 630,136 seconds to decompress.
Optimization Trade-offs
The experiment highlights several counterintuitive findings about optimization. A smaller vocabulary of 4,000 words actually outperformed larger vocabularies of 8,000 and 16,000 words. Giving Wikipedia’s XML markup its own dedicated tokens worsened compression results. Model size also presents diminishing returns, as a 267-million-parameter model performed slightly worse than a 188-million-parameter model on this dataset. With a smaller 100-megabyte version of the file, gains leveled off at about 77 million parameters.

Increasing the training window from 15 to 35 million tokens reduced the payload by only 1.21 percent but increased compute costs by 40 percent. A run starting from a bigger six-layer offline model predicted better, resulting in a 99.17-megabyte payload. However, the larger weights made the total file size bigger than the online-only run. He frames this six-layer trade-off as the next research question, focusing on how much predictive quality a smaller model can keep.
Source: Tom’s Hardware

