A new open-source project named OpenArch has been released on GitHub, providing hand-written PyTorch implementations of various modern Large Language Model (LLM) architectures. The repository aims to serve as a learning resource by prioritizing code readability and structural clarity over performance optimization.

The implementations are based on original research papers, technical reports, and configuration files. Included architectures cover a wide range of models, such as the Llama, Qwen, DeepSeek, Gemma, and GPT-OSS families. For certain architectures like PaliGemma, the repository includes multimodal support.

Unlike production-level libraries that optimize for speed and sharding, OpenArch focuses on making architectural choices—such as attention mechanisms, normalization, and layer mixes—explicit and easy to compare. The project is licensed under Apache License 2.0.

Sources: