MLX Model Test Report

Generated 2026-02-26 23:25 · AFM MLX Backend · v0.9.5-4d58e3f

mlx-model-test.sh --models

Test Runs
30
Passed
29
Failed
1
Best tok/s
294.3
Fastest
mlx-community/lille-130m-instruct-8bit

Performance Ranking (by tokens/sec)

Click a row to jump to its full response below.

# Model / Config Status Temp Load (s) Tokens Gen (s) Tokens/sec Prompt
1 mlx-community/lille-130m-instruct-8bit OK 0.7 1.0 707 2.4
294.3
 Explain calculus concepts from limits through multivariable ...
2 mlx-community/exaone-4.0-1.2b-4bit OK 0.7 1.0 992 3.75
264.3
 Explain calculus concepts from limits through multivariable ...
3 mlx-community/LFM2-2.6B-4bit OK 0.7 1.0 2873 12.34
232.7
 Explain calculus concepts from limits through multivariable ...
4 mlx-community/LFM2-VL-3B-4bit OK 0.7 1.0 720 3.11
231.2
 Explain calculus concepts from limits through multivariable ...
5 mlx-community/Ling-mini-2.0-4bit OK 0.7 5.0 2904 13.8
210.5
 Explain calculus concepts from limits through multivariable ...
6 mlx-community/SmolLM3-3B-4bit OK 0.7 1.0 1708 9.97
171.3
 Explain calculus concepts from limits through multivariable ...
7 mlx-community/granite-4.0-h-tiny-4bit OK 0.7 2.0 957 7.16
133.8
 Explain calculus concepts from limits through multivariable ...
8 mlx-community/Qwen3-VL-4B-Instruct-4bit OK 0.7 2.0 4142 31.87
130.0
 Explain calculus concepts from limits through multivariable ...
9 mlx-community/gpt-oss-20b-MXFP4-Q4 OK 0.7 4.0 2535 19.74
128.4
 Explain calculus concepts from limits through multivariable ...
10 mlx-community/Qwen3.5-35B-A3B-4bit OK 0.7 7.0 4162 34.23
121.6
 Explain calculus concepts from limits through multivariable ...
11 mlx-community/Qwen3-30B-A3B-4bit OK 0.7 6.0 3729 34.64
107.7
 Explain calculus concepts from limits through multivariable ...
12 mlx-community/gemma-3-4b-it-8bit OK 0.7 2.0 1606 16.0
100.4
 Explain calculus concepts from limits through multivariable ...
13 mlx-community/Qwen3-VL-4B-Instruct-8bit OK 0.7 2.0 3372 35.39
95.3
 Explain calculus concepts from limits through multivariable ...
14 mlx-community/Qwen3.5-35B-A3B-8bit OK 0.7 12.0 5000 55.81
89.6
 Explain calculus concepts from limits through multivariable ...
15 mlx-community/gemma-3n-E2B-it-lm-4bit OK 0.7 2.0 2671 30.16
88.6
 Explain calculus concepts from limits through multivariable ...
16 mlx-community/mistralai_Ministral-3-14B-Instruct-2512-MLX-MXFP4 OK 0.7 3.0 3762 51.86
72.5
 Explain calculus concepts from limits through multivariable ...
17 mlx-community/Qwen3-Coder-Next-4bit OK 0.7 14.0 4808 66.34
72.5
 Explain calculus concepts from limits through multivariable ...
18 mlx-community/Apertus-8B-Instruct-2509-4bit OK 0.7 4.0 44 0.65
67.8
 Explain calculus concepts from limits through multivariable ...
19 mlx-community/GLM-4.7-Flash-4bit OK 0.7 6.0 3437 50.78
67.7
 Explain calculus concepts from limits through multivariable ...
20 mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit OK 0.7 6.0 3603 58.09
62.0
 Explain calculus concepts from limits through multivariable ...
21 mlx-community/MiniMax-M2.5-5bit OK 0.7 51.0 2408 55.21
43.6
 Explain calculus concepts from limits through multivariable ...
22 mlx-community/JoyAI-LLM-Flash-4bit-DWQ OK 0.7 10.0 5000 119.82
41.7
 Explain calculus concepts from limits through multivariable ...
23 mlx-community/Qwen3.5-397B-A17B-4bit OK 0.7 66.0 4620 115.79
39.9
 Explain calculus concepts from limits through multivariable ...
24 mlx-community/MiniMax-M2.5-6bit OK 0.7 67.0 5000 125.33
39.9
 Explain calculus concepts from limits through multivariable ...
25 mlx-community/functiongemma-270m-it-bf16 OK 0.7 2.0 42 1.19
35.3
 Explain calculus concepts from limits through multivariable ...
26 mlx-community/mistralai_Devstral-Small-2-24B-Instruct-2512-MLX-8Bit OK 0.7 8.0 3180 123.42
25.8
 Explain calculus concepts from limits through multivariable ...
27 mlx-community/Llama-3.3-70B-Instruct-4bit-DWQ OK 0.7 13.0 1107 71.71
15.4
 Explain calculus concepts from limits through multivariable ...
28 mlx-community/GLM-5-4bit OK 0.7 126.0 4187 329.92
12.7
 Explain calculus concepts from limits through multivariable ...
29 mlx-community/Kimi-K2.5-3bit OK 0.7 144.0 183 74.12
2.5
 Why is the sky blue? Be brief

Failed Runs

ModelErrorConfigLoad (s)
lmstudio-community/gemma-3n-E4B-it-MLX-4bit [- ] --.-% loading model | mem 0.17 GB | 0.2s [\ ] --.-% loading model | mem 0.23 GB | 0.4s [| ] --.-% loading model | mem 0.29 t=0.7 2

Full Responses

▶ mlx-community/lille-130m-instruct-8bit 707 tokens · 294.3 tok/s

▶ mlx-community/exaone-4.0-1.2b-4bit 992 tokens · 264.3 tok/s

▶ mlx-community/LFM2-2.6B-4bit 2873 tokens · 232.7 tok/s

▶ mlx-community/LFM2-VL-3B-4bit 720 tokens · 231.2 tok/s

▶ mlx-community/Ling-mini-2.0-4bit 2904 tokens · 210.5 tok/s

▶ mlx-community/SmolLM3-3B-4bit 1708 tokens · 171.3 tok/s

▶ mlx-community/granite-4.0-h-tiny-4bit 957 tokens · 133.8 tok/s

▶ mlx-community/Qwen3-VL-4B-Instruct-4bit 4142 tokens · 130.0 tok/s

▶ mlx-community/gpt-oss-20b-MXFP4-Q4 2535 tokens · 128.4 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit 4162 tokens · 121.6 tok/s

▶ mlx-community/Qwen3-30B-A3B-4bit 3729 tokens · 107.7 tok/s

▶ mlx-community/gemma-3-4b-it-8bit 1606 tokens · 100.4 tok/s

▶ mlx-community/Qwen3-VL-4B-Instruct-8bit 3372 tokens · 95.3 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-8bit 5000 tokens · 89.6 tok/s

▶ mlx-community/gemma-3n-E2B-it-lm-4bit 2671 tokens · 88.6 tok/s

▶ mlx-community/mistralai_Ministral-3-14B-Instruct-2512-MLX-MXFP4 3762 tokens · 72.5 tok/s

▶ mlx-community/Qwen3-Coder-Next-4bit 4808 tokens · 72.5 tok/s

▶ mlx-community/Apertus-8B-Instruct-2509-4bit 44 tokens · 67.8 tok/s

▶ mlx-community/GLM-4.7-Flash-4bit 3437 tokens · 67.7 tok/s

▶ mlx-community/NVIDIA-Nemotron-3-Nano-30B-A3B-4bit 3603 tokens · 62.0 tok/s

▶ mlx-community/MiniMax-M2.5-5bit 2408 tokens · 43.6 tok/s

▶ mlx-community/JoyAI-LLM-Flash-4bit-DWQ 5000 tokens · 41.7 tok/s

▶ mlx-community/Qwen3.5-397B-A17B-4bit 4620 tokens · 39.9 tok/s

▶ mlx-community/MiniMax-M2.5-6bit 5000 tokens · 39.9 tok/s

▶ mlx-community/functiongemma-270m-it-bf16 42 tokens · 35.3 tok/s

▶ mlx-community/mistralai_Devstral-Small-2-24B-Instruct-2512-MLX-8Bit 3180 tokens · 25.8 tok/s

▶ mlx-community/Llama-3.3-70B-Instruct-4bit-DWQ 1107 tokens · 15.4 tok/s

▶ mlx-community/GLM-5-4bit 4187 tokens · 12.7 tok/s

▶ mlx-community/Kimi-K2.5-3bit 183 tokens · 2.5 tok/s