LLM Visualizer is an interactive learning guide designed to help users understand the inner workings of the Transformer architecture by building the process from raw text to generated output from scratch.

This guide offers mathematical explanations, actual code execution, and hands-on experience with model training. Although the model being built is small, with approximately 550K parameters, it employs the same architecture as large-scale models such as GPT-3. A real PyTorch model runs in the backend, allowing users to observe how input text flows through the model pipeline and how weights are updated by gradients during training.

The learning process consists of 10 steps, providing conceptual explanations, mathematical foundations, and Python implementations at each stage. Additionally, after completing the steps, the guide is designed so that users can explore the formulas of modern architectures adopted by the latest models—such as LLaMA 3, DeepSeek-V3, and Claude—including RoPE, RMSNorm, GQA, MLA, and MoE.

Source: