A researcher has demonstrated that small-scale language models can effectively perform complex domain-specific tasks, such as database query optimization, through targeted training. In a recent experiment, a 4B parameter model was trained to produce PostgreSQL query plans that were, on average, 1.81x faster than the default optimizer for join-heavy SQL workloads.
Small 4B Model Achieves 1.81x Speedup in PostgreSQL Query Optimization via RL
PLUS ULTRA by Amenoyomi
The training process utilized a two-stage approach. First, the model underwent supervised fine-tuning (SFT) using off-policy distillation. The researcher used trajectories from a frontier-scale teacher model, GPT-6 Astra, to teach the 4B student model the "language" of a specialized agentic harness designed for query optimization.
Following SFT, the model was further refined using reinforcement learning (RL). The researcher implemented a custom variant of Group Relative Policy Optimization (GRPO) to optimize the model's ability to select superior query plans. The reward mechanism was specifically anchored to the execution speedup of the candidate plan relative to the PostgreSQL default, while penalizing invalid plans or those that merely replicated the default behavior.
The results showed that even a tiny model can achieve significant efficiency gains in niche, high-value domains. By using an agentic multi-turn approach—allowing the model up to 15 candidate attempts per query—the final checkpoint achieved a 44.7% reduction in total workload latency for the tested join-heavy queries. The study highlights the potential for companies to use small, open-weights models for specialized, cost-effective optimization tasks by leveraging existing high-quality data from larger models.
PLUS ULTRAby Amenoyomi
While the performance gains are significant, the success of the model relies on addressing the structural limitations of how databases estimate query costs. PostgreSQL's default optimizer struggles with "join ordering," a task known to be NP-hard. To estimate the cost of different plans, the system relies on a "uniform distribution assumption," assuming that the frequency of a value in one table applies evenly across another. When real-world data deviates from this—such as when a small percentage of entries accounts for a majority of the related records—the cost model's estimates fail, causing the optimizer to select inefficient plans.
The model overcomes this by learning through two distinct stages. First, it undergoes supervised fine-tuning to master the "language" of the agentic harness, learning how to interface with the database via pg_hint_plan. Rather than rewriting the database engine, the model produces structured hints as SQL comments to steer the optimizer. Once the interface is mastered, reinforcement learning is used to optimize for a single verifiable axis: execution time.
The effectiveness of this reinforcement depends on the purity of the reward signal. Because execution times vary based on whether data is in the OS page cache or the database's shared_buffers, "phantom" speedups can occur, fooling the learning process. By increasing the shared_buffers to 2GB, the working set became fully resident in the database cache, eliminating this noise and allowing the model to accurately correlate its plan choices with actual performance gains.
Sources
- Training a 4B model to produce 81% faster query plans than Postgres （Hacker News Frontpage, 2026-09-16）
- empero-ai/Qwen3.8-4B-Distill