Janus is a single Go binary designed to run .gguf models on local hardware using either a CPU or a GPU via the Vulkan API. The tool exposes an OpenAI-compatible API and includes a built-in web UI, eliminating the need for Python, Docker, or Ollama, though Ollama can be used as a backend if preferred.
Janus: Single Go binary for running GGUF models via Vulkan
The application functions as both a runner and a router. It handles model inference, request routing, and tool execution while keeping all processes local. Users can interact with the model through the command line, scripts, or third-party clients like Cursor and Cline by pointing them to the Janus endpoint.
Janus is designed for portability, requiring only the binary and a model file (typically 2–8 GB). It supports various workflows, from simple chat interfaces to automated tasks via its API. For users requiring OCR capabilities, Tesseract can be integrated.
Sources
- Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia （Hacker News Frontpage, 2026-10-01）