1. Summary
LLM-jp-4-VL 9B is a 9B-parameter vision-language model developed by LLM-jp. It combines the llm-jp-4-8b-thinking (8.6B) as the base LLM, the SigLIP 2 So400m (0.4B) as the vision encoder, and a 2-layer MLP projector. The architecture is inspired by InternVL3.0.
According to the Hugging Face Model Card, changes from the beta version include the replacement of training data with datasets that have no licensing issues for commercial use, and the addition of a feature to generate inference traces (analysis channel) before the final answer.
Evaluation results show that the model significantly outperforms the beta version in text-only tasks and chart understanding, such as CharXiv-Reasoning and HakushoBench. It is a model well-suited for Japanese document/chart analysis and inference tasks.
2. Bunrin Bench (BUNRIN LABO Original Test)
No data available (scheduled for implementation). For the methodology, refer to About Bunrin Bench.
3. Various Benchmarks
The following figures are reported in the Evaluation section of the Hugging Face Model Card. Evaluations were calculated as an average of 3 runs using the simple-evals-mm framework under the conditions of greedy decoding (temperature=0.0), max_new_tokens=32k, and reasoning effort=medium.
Benchmark Scores
| Benchmark | LLM-jp-4-VL-9B | LLM-jp-4-VL-9B-beta | InternVL3.5-8B | Qwen3.5-9B | Gemma-4-12B |
|---|---|---|---|---|---|
| Document Understanding | |||||
| AI2D (N=3088) | 80.1 | 76.2 | 83.4 | 90.8 | 79.9 |
| ChartQA (N=1250) | 64.9 | 70.5 | 75.0 | 86.4 | 72.6 |
| JGraphQA (N=196) | 86.7 | 87.8 | 85.9 | 98.8 | 89.5 |
| CharXiv-Reasoning (N=1000) | 40.7 | 33.2 | 39.9 | 69.1 | 55.8 |
| ChartQAPro (N=1948) | 30.8 | 27.8 | 33.5 | 58.9 | 53.9 |
| HakushoBench (N=2053) | 46.3 | 41.3 | 39.4 | 74.3 | 71.5 |
| DocVQA (N=5349) | 89.5 | 90.1 | 89.4 | 94.2 | 88.5 |
| InfoVQA (N=2065) | 67.4 | 69.6 | 68.9 | 88.2 | 81.5 |
| TextVQA (N=5000) | 88.4 | 87.7 | 83.9 | 87.7 | 73.8 |
| JDocQA (N=861) | 74.0 | 73.7 | 59.5 | 93.3 | 69.4 |
| CC-OCR-JA (N=145) | 70.3 | 67.6 | 51.5 | 79.8 | 47.8 |
| BusinessSlideVQA (N=216) | 60.6 | 62.2 | 57.4 | 84.1 | 69.3 |
| STEM | |||||
| MMMU (N=847) | 50.4 | 52.3 | 65.3 | 79.1 | 72.7 |
| JMMMU (N=1270) | 47.5 | 46.8 | 50.9 | 69.6 | 63.8 |
| MathVision (N=304) | 13.2 | 17.4 | 30.6 | 71.4 | 51.4 |
| Multi-Image | |||||
| BLINK (N=1901) | 47.3 | 46.2 | 54.7 | 71.3 | 58.4 |
| JA-Multi-Image-VQA (N=53) | 75.5 | 88.7 | 62.3 | 92.5 | 84.9 |
| 2D/3D Grounding | |||||
| CountBenchQA (N=491) | 79.2 | 82.9 | 78.4 | 93.5 | 72.0 |
| General VQA | |||||
| OK-VQA (N=5046) | 67.5 | 65.8 | 70.5 | 66.1 | 58.7 |
| RealWorldQA (N=765) | 70.5 | 71.2 | 64.4 | 79.0 | 64.3 |
| CVQA-JA (N=200) | 69.5 | 70.2 | 50.5 | 72.8 | 68.5 |
| MECHA-ja (N=1818) | 69.7 | 63.6 | 56.6 | 68.4 | 71.5 |
| Heron-Bench (N=88) | 63.6 | 63.6 | 50.4 | 61.7 | 40.5 |
| JA-VLM-Bench (N=49) | 65.3 | 70.1 | 46.9 | 72.1 | 51.7 |
| Text-only | |||||
| GPQA (N=198) | 40.9 | 29.3 | 48.0 | 82.8 | 73.4 |
| MMLU-Redux (N=5376) | 84.2 | 73.5 | 80.1 | 92.1 | 91.2 |
| SimpleQA (N=4326) | 6.1 | 5.8 | 3.3 | 8.7 | 4.3 |
| MATH (N=500) | 79.7 | 46.1 | 62.7 | 96.5 | 95.7 |
Domain Averages
| Category | LLM-jp-4-VL-9B | LLM-jp-4-VL-9B-beta | InternVL3.5-8B | Qwen3.5-9B | Gemma-4-12B |
|---|---|---|---|---|---|
| Document Understanding (12) | 66.7 | 65.6 | 64.0 | 83.8 | 71.1 |
| STEM (3) | 37.0 | 38.9 | 48.9 | 73.4 | 62.7 |
| Multi-Image (1) | 47.3 | 46.2 | 54.7 | 71.3 | 58.4 |
| 2D/3D Grounding (1) | 79.2 | 82.9 | 78.4 | 93.5 | 72.0 |
| General VQA (4) | 69.3 | 67.7 | 60.5 | 71.6 | 65.7 |
| Text-only (4) | 52.7 | 38.6 | 48.5 | 70.0 | 66.1 |
| All (25) | 61.0 | 58.3 | 59.3 | 78.3 | 68.0 |
| Ja-Avg (8) | 65.6 | 64.1 | 56.5 | 80.1 | 68.9 |
| En-Avg (17) | 58.9 | 55.6 | 60.7 | 77.4 | 67.5 |
(Source: Per-benchmark results / Domain-level averages tables from Hugging Face Model Card)
4. Official announcements
- Public information about the model (architecture, approximately 29.3M training samples, evaluation results, and usage code) is available on the Hugging Face Model Card. Related blog: LLM-jp Blog – llm-jp-4-vl-9b, code repository: github.com/llm-jp/llm-jp-4-vl
- 2026-07-28: MLX-VLM v0.6.8 added model support for llm-jp-4-vl (llmjpvl) (via PR #1738 by @takuyaomi). This enables usage in inference libraries for Apple Silicon.
5. Real-world performance (Community reception)
Insufficient information has been gathered yet.
6. Recommended parameters
The following values are described in the Evaluation settings and Usage sections of the Hugging Face Model Card.
| Parameter | Recommended Value | Notes |
|---|---|---|
| temperature | 0.0 (Greedy Decoding) | do_sample=False |
| max_new_tokens | 32,000 | Includes inference trace + final answer. Default in usage examples is 1,024 |
| reasoning_effort | "medium" | 3 levels: "low" (direct answer) / "medium" / "high" |
(Source: Evaluation settings and Usage sections from Hugging Face Model Card)
7. Sources
- Hugging Face Model Card (Official)
- v0.6.8 (MLX-VLM, 2026-07-28)
- Headline News for September 2, 2026 (GIGAZINE, 2026-09-02)