A new tool named GziPT demonstrates that the standard gzip compression algorithm can be used to perform language modeling without the use of neural networks, training, or learned parameters. Based on the information theory principle that compression and prediction are equivalent, the tool generates text by finding the byte sequences that achieve the best compression ratio.
Product LaunchesGziPTNathan RS
Can gzip function as a language model? A new implementation uses beam search for text generation
The implementation, released by developer Nathan RS, uses the DEFLATE algorithm via Python's zlib library. To generate text, the model is "primed" with a corpus to fill its sliding window with context. When provided with a text prompt, the tool searches for continuations that compress more efficiently than random data.
To overcome the limitations of byte-level quantization—where small changes in a single byte might not significantly impact the compressed length—GziPT employs a beam search. By looking ahead at a specific span of bytes before committing to a choice, the tool can better identify the most probable continuations. Experimental results using the Tiny Shakespeare dataset show that while the output is not perfectly coherent, it demonstrates an unexpected ability to capture the patterns of the input text.
Sources
- Can gzip be a language model? （Hacker News Frontpage, 2026-09-22）
- GitHub