Researchers at the Georgia Institute of Technology have developed Transformer Explainer, an interactive web tool designed to visualize the internal mechanisms of the Transformer architecture. The tool allows users to explore how text is processed through various components such as embedding, tokenization, and the self-attention mechanism.
Product LaunchesGPT-2Transformer Explainer
Transformer Explainer provides interactive visual breakdown of the architecture
The explainer is powered by a live GPT-2 (small) model with 124 million parameters, which runs directly in the browser using ONNX Runtime. By interacting with the interface, users can observe how input text is converted into high-dimensional vectors and how multi-head self-attention calculates attention scores to capture long-range dependencies.
The tool provides a step-by-step walkthrough of the data flow, including the role of Query (Q), Key (K), and Value (V) vectors, the function of the Multi-Layer Perceptron (MLP) layer, and the impact of hyperparameters like temperature, top-k, and top-p during the sampling process. The interface was built using JavaScript, Svelte, and D3.js to provide real-time updates of numerical values based on user input.
Sources
- Transformers Explained Visually （Hacker News Frontpage, 2026-09-21）