Researchers have demonstrated that off-the-shelf Large Language Models (LLMs) can act as high-speed "System One" decision models—specialized models designed to output a chosen option alongside confidence values for all available options via a single forward pass. By reading the probability distribution of tokens, the method avoids the high latency and cost associated with generating full JSON objects or using reasoning models.
GLM-5.3-Flash performs on par with specialized decision models like Jev
In evaluations using GLM-5.3-Flash on the Privatemode platform, the approach achieved decision accuracy on par with TypeSafe's Jev across 28 text-based datasets. While Jev remains more cost-effective and slightly more accurate in some instances, the GLM-5.3-Flash setup provides a significant advantage in versatility: unlike Jev and Laya, it enables typed decisions on images, such as scanned invoices.
The method involves prompting the model to select a specific index from a list of options and reading the log probabilities at that position. While using the model's reasoning capabilities before answering increases accuracy, it also significantly increases costs and latency. For example, reasoning with GLM-5.3-Flash cost approximately EUR 350 per million decisions, compared to EUR 0.062 for the single-token decision approach.
On text datasets, the median gap in accuracy between GLM-5.3-Flash and Jev was 0.7 percentage points in Jev's favor, which was not found to be statistically significant. However, when the number of options increases, accuracy tends to drop for both systems. For instance, on the TREC dataset, accuracy fell from 91.2% to 79.6% when the number of options increased from 6 to 42.
Sources
- Turning GLM-5.3-Flash into a Jev-like decision model （Hacker News Frontpage, 2026-09-26）
- GitHub benchmark repository
1 more sourcesHide sources
- Calling the AI bluff: Adding "Do not guess" cut made-up fields from 71% to 20% （Hacker News Frontpage, 2026-09-27）