Niklas Gustafsson reportedly believed that a model capable of solving 10-digit addition tasks with fewer than 1,000 parameters could be achieved with even fewer parameters. He attempted to reduce the parameter count to the absolute limit while maintaining the Transformer architecture.
NewsNiklas GustafssonTransformer
Minimal Transformer for 10-digit Addition Claimed to be Implementable with Under 95 Parameters
This article is a translation. Read the Japanese original
As a result, he demonstrates that the count is 95 if all non-zero parameters are counted as dense matrices, and 36 if identity matrices are ignored. Furthermore, he claims that by reusing input embedding dimensions or using RoPE or Alibi for positional encoding, the count can be reduced to 28, 22, or in some cases, as few as 12.
He stated that he prioritized maintaining a natural structure such that the model could still be identified as "definitely a Transformer" when viewed as an ONNX file. He established his own criteria of not including zero tensors or identity matrices in the parameter count.
Additionally, while the Codex solution simplifies carry logic by reversing the order of digits, he evaluates forward zero-padding as being cleaner.
Source: Building a Minimal Transformer for 10-digit Addition (HN 73pt, 11 comments) (HN Search (backfill), 2026-03-01)