MLX Model Test Report

Generated 2026-02-25 17:28 · AFM MLX Backend · v0.9.5

mlx-model-test.sh --prompts Scripts/test-stop-sequences.txt --smart claude,codex

Test Runs
34
Passed
34
Failed
0
Best tok/s
52.8
Fastest
mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase

Performance Ranking (by tokens/sec)

Click a row to jump to its full response below.

# Model / Config Status codexclaude Temp Load (s) Tokens Gen (s) Tokens/sec Prompt
1 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase stop-long-phrase OK 54 0.0 1.0 1119 21.18
52.8
Write a 3-paragraph essay about renewable energy. Start the ...
2 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-code-fence stop-code-fence OK 23 0.0 1.0 122 2.6
46.8
Write a Python function that computes factorial, inside a co...
3 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-double-newline stop-api-double-newline OK 23 0.0 1.0 695 15.35
45.3
Write a short paragraph about the ocean. Then write a second...
4 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-period stop-api-period OK 24 0.0 1.0 2041 45.45
44.9
Tell me about the sun in three sentences.
5 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered stop-system-numbered OK 44 0.0 1.0 1010 23.07
43.8
What are the main benefits of exercise?
6 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-no-match stop-guided-json-no-match
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]}'
OK 25 0.0 1.0 1150 26.88
42.8
Generate a person record for Bob aged 25.
7 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace stop-guided-json-brace
--guided-json '{"type":"object","properties":{"color":{"type":"string"},"hex":{"type":"string"}},"required":["color","hex"]}'
OK 32 0.0 1.0 819 19.27
42.5
Describe the color blue with its hex code.
8 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate stop-system-pirate OK 24 0.0 1.0 1113 26.24
42.4
Tell me about treasure hunting on the high seas.
9 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-no-match stop-json-object-no-match OK 25 0.0 1.0 473 11.3
41.9
Generate a JSON object with keys "fruit" and "color" for an ...
10 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate stop-immediate OK 54 0.0 1.0 111 2.7
41.1
What is the capital of Japan?
11 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-word stop-api-word OK 24 0.0 1.0 2579 63.01
40.9
Name 5 programming languages and briefly describe each one.
12 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match stop-no-match OK 24 0.0 1.0 134 3.34
40.1
What is 2+2? Answer briefly.
13 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma stop-guided-json-comma
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"},"city":{"type":"string"}},"required":["name","age","city"]}'
OK 25 0.0 1.0 628 15.76
39.9
Generate a person profile for someone named Alice who is 30 ...
14 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode stop-unicode OK 25 0.0 1.0 420 10.6
39.6
List 5 items about space using bullet points (•).
15 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens stop-low-max-tokens OK 2 0.0 1.0 100 2.61
38.3
List 10 mountains, numbered 1 through 10, one per line.
16 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi stop-cli-multi
--stop "```,END"
OK 24 0.0 1.0 284 7.45
38.1
Write a Python hello world in a code block, then write END.
17 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars stop-special-chars OK 23 0.0 1.0 525 13.95
37.6
List 3 facts about the moon. Use **bold** markdown for empha...
18 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-newline stop-api-newline OK 23 0.0 1.0 199 5.51
36.1
What is the capital of France? Answer in one sentence.
19 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value stop-guided-json-value
--guided-json '{"type":"object","properties":{"cities":{"type":"array","items":{"type":"string"}}},"required":["cities"]}'
OK 23 0.0 1.0 657 18.41
35.7
List 5 major world cities as a JSON array. Include Tokyo.
20 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key stop-json-object-key OK 25 0.0 1.0 711 19.95
35.6
Generate a JSON object with keys "name", "age", and "city" f...
21 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word stop-multi-word OK 33 0.0 1.0 585 17.2
34.0
Write a 5-step recipe for making tea. Label each step as "St...
22 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-streaming stop-streaming OK 24 0.0 1.0 627 19.78
31.7
List 10 planets or celestial objects, numbered 1 through 10,...
23 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge stop-cli-api-merge
--stop "5."
OK 24 0.0 1.0 253 8.0
31.6
List 10 countries, numbered 1 through 10, one per line.
24 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-single stop-api-single OK 34 0.0 1.0 435 13.81
31.5
List 10 colors, numbered 1 through 10, one per line.
25 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming stop-non-streaming
--no-streaming
OK 22 0.0 1.0 627 19.93
31.5
List 10 planets or celestial objects, numbered 1 through 10,...
26 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-multi stop-api-multi OK 24 0.0 2.0 471 15.17
31.1
List 10 animals, numbered 1 through 10, one per line.
27 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max stop-four-max OK 25 0.0 1.0 359 11.62
30.9
List 10 items, numbered 1 through 10. One per line.
28 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup stop-cli-api-dedup
--stop "3."
OK 24 0.0 1.0 334 10.98
30.4
List 10 cities, numbered 1 through 10, one per line.
29 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag stop-html-tag OK 24 0.0 1.0 249 8.3
30.0
Write an HTML unordered list of 5 fruits using <ul> and <li>...
30 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only stop-cli-only
--stop "3."
OK 33 0.0 1.0 196 6.59
29.7
List 10 fruits, numbered 1 through 10, one per line.
31 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp stop-high-temp OK 34 1.0 1.0 187 6.53
28.6
List 10 random words, numbered 1 through 10, one per line.
32 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 stop-seed-run1 OK 24 0.0 2.0 253 8.99
28.2
List 10 flowers, numbered 1 through 10, one per line.
33 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 stop-seed-run2 OK 23 0.0 1.0 253 8.99
28.2
List 10 flowers, numbered 1 through 10, one per line.
34 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-top-p stop-top-p OK 2 0.0 1.0 357 12.87
27.7
List 10 rivers, numbered 1 through 10, one per line.

AI Analysis (--smart)

codex Analysis · avg score: 2.4/5

claude Analysis · avg score: 3.8/5

Full Responses

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase 1119 tokens · 52.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-code-fence 122 tokens · 46.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-double-newline 695 tokens · 45.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-period 2041 tokens · 44.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered 1010 tokens · 43.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-no-match 1150 tokens · 42.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace 819 tokens · 42.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate 1113 tokens · 42.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-no-match 473 tokens · 41.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate 111 tokens · 41.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-word 2579 tokens · 40.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match 134 tokens · 40.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma 628 tokens · 39.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode 420 tokens · 39.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens 100 tokens · 38.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi 284 tokens · 38.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars 525 tokens · 37.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-newline 199 tokens · 36.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value 657 tokens · 35.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key 711 tokens · 35.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word 585 tokens · 34.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-streaming 627 tokens · 31.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge 253 tokens · 31.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-single 435 tokens · 31.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming 627 tokens · 31.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-multi 471 tokens · 31.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max 359 tokens · 30.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup 334 tokens · 30.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag 249 tokens · 30.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only 196 tokens · 29.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp 187 tokens · 28.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 253 tokens · 28.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 253 tokens · 28.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-top-p 357 tokens · 27.7 tok/s