MLX Model Test Report

Generated 2026-02-27 07:20 · AFM MLX Backend · v0.9.5-4d58e3f

mlx-model-test.sh --prompts Scripts/test-Qwen3.5-35B-A3B-4bit.txt --smart

Test Runs
132
Passed
132
Failed
0
Best tok/s
125.4
Fastest
mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple

Performance Ranking (by tokens/sec)

Click a row to jump to its full response below.

# Model / Config Status codex Temp Load (s) Tokens Gen (s) Tokens/sec Prompt
1 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple guided-json-simple
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]}'
OK 3 0.0 1.0 1373 10.95
125.4
Generate a person record.
2 mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode raw-mode
--raw
OK 2 0.7 1.0 1662 13.27
125.3
Think step by step: what is 17 * 23?
3 mlx-community/Qwen3.5-35B-A3B-4bit @ scientist scientist OK 2 0.3 1.0 2236 17.86
125.2
Tell me about quantum computing.
4 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested guided-json-nested
--guided-json '{"type":"object","properties":{"city":{"type":"string"},"population":{"type":"integer"},"landmarks":{"type":"array","items":{"type":"string"}}},"required":["city","population","landmarks"]}'
OK 3 0.0 1.0 937 7.48
125.2
Describe Tokyo as a structured record with at least 3 landma...
5 mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv small-kv
--max-kv-size 2048
OK 5 0.7 1.0 1231 9.86
124.9
Summarize the key ideas of machine learning in a few paragra...
6 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 seed-42-run1
--seed 42
OK 5 0.7 2.0 4096 32.82
124.8
Write a limerick about a cat.
7 mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty no-penalty
--presence-penalty 0.0
OK 5 0.8 2.0 4096 32.82
124.8
Write a long essay about the history of bread making across ...
8 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 prefill-small-256
--prefill-step-size 256
OK 5 0.0 1.0 3652 29.28
124.7
Given this architecture, what are the top 3 performance bott...
9 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default prefill-default OK 5 0.0 1.0 3652 29.29
124.7
Given this architecture, what are the top 3 performance bott...
10 mlx-community/Qwen3.5-35B-A3B-4bit @ math math OK 5 0.0 1.0 1830 14.68
124.7
A store sells notebooks for $3.50 each. If you buy 5 or more...
11 mlx-community/Qwen3.5-35B-A3B-4bit @ long-output long-output OK 5 0.7 1.0 1994 16.0
124.6
Write a detailed recipe for chocolate chip cookies with ingr...
12 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 seed-42-run2
--seed 42
OK 5 0.7 1.0 4096 32.88
124.6
Write a limerick about a cat.
13 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json response-format-json OK 4 0.0 1.0 1303 10.46
124.6
Return a JSON object with keys "language", "year_created", a...
14 mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 eli5 OK 2 0.7 1.0 1144 9.18
124.5
Tell me about quantum computing.
15 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 prefill-large-4096
--prefill-step-size 4096
OK 5 0.0 1.0 3652 29.34
124.5
Given this architecture, what are the top 3 performance bott...
16 mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized kv-quantized
--kv-bits 4
OK 5 0.7 1.0 1032 8.3
124.3
Summarize the key ideas of machine learning in a few paragra...
17 mlx-community/Qwen3.5-35B-A3B-4bit @ long-form long-form OK 5 0.7 1.0 3468 27.9
124.3
Write a detailed technical blog post explaining how Mixture-...
18 mlx-community/Qwen3.5-35B-A3B-4bit @ pirate pirate OK 2 0.7 1.0 1212 9.78
123.9
Tell me about quantum computing.
19 mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming no-streaming
--no-streaming
OK 5 0.7 1.0 4094 33.08
123.8
Write a short poem about the moon.
20 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 agent-cached-turn2
--enable-prefix-caching
OK 5 0.0 1.0 4096 33.12
123.7
Now add a --timeout flag to the CLI that sets a request time...
21 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens max-completion-tokens OK 5 0.0 1.0 1174 9.5
123.5
max_completion_tokens: 100
22 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens max-completion-tokens OK 4 0.0 1.0 2958 23.97
123.4
Describe the entire history of Rome.
23 mlx-community/Qwen3.5-35B-A3B-4bit @ min-p min-p
--min-p 0.05
OK 5 0.8 1.0 572 4.64
123.3
Explain why the sky is blue in exactly 3 bullet points.
24 mlx-community/Qwen3.5-35B-A3B-4bit @ pirate pirate OK 3 0.7 1.0 4096 33.26
123.2
What is the capital of Japan? Answer in one sentence.
25 mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift code-swift OK 5 0.0 1.0 4096 33.35
122.8
What is the capital of Japan? Answer in one sentence.
26 mlx-community/Qwen3.5-35B-A3B-4bit @ json-output json-output OK 5 0.0 1.0 384 3.13
122.8
Respond with a valid JSON object containing keys "name", "ag...
27 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml tool-call-xml
--tool-call-parser xmlFunction
OK 4 0.0 1.0 4096 33.36
122.8
What is the weather in Paris in celsius?
28 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi tool-call-multi OK 4 0.0 1.0 1169 9.52
122.7
What is the weather in London and what time is it in Tokyo?
29 mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded non-streaming-seeded
--no-streaming --seed 123
OK 5 0.7 1.0 2469 20.12
122.7
Write a 4-line poem about the ocean.
30 mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp high-temp OK 5 1.5 1.0 995 8.11
122.7
Explain why the sky is blue in exactly 3 bullet points.
31 mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw think-raw
--raw
OK 2 0.0 1.0 833 6.79
122.6
Solve step by step: If a train travels at 60 mph for 2.5 hou...
32 mlx-community/Qwen3.5-35B-A3B-4bit @ code-python code-python OK 5 0.0 1.0 4096 33.42
122.6
What is the capital of Japan? Answer in one sentence.
33 mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded streaming-seeded
--seed 123
OK 5 0.7 1.0 2469 20.15
122.5
Write a 4-line poem about the ocean.
34 mlx-community/Qwen3.5-35B-A3B-4bit @ top-k top-k
--top-k 30
OK 5 0.8 1.0 1270 10.37
122.5
Explain why the sky is blue in exactly 3 bullet points.
35 mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list numbered-list OK 5 0.0 1.0 489 4.0
122.3
List exactly 5 animals, one per line, numbered 1-5. No other...
36 mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift code-swift OK 2 0.0 1.0 4096 33.51
122.2
Write a Swift async function that fetches JSON from a URL us...
37 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role developer-role OK 5 0.0 1.0 4096 33.51
122.2
developer: You are a helpful coding assistant. Only respond ...
38 mlx-community/Qwen3.5-35B-A3B-4bit @ scientist scientist OK 5 0.3 1.0 4096 33.52
122.2
What is the capital of Japan? Answer in one sentence.
39 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto tool-call-auto OK 4 0.0 1.0 502 4.11
122.2
What is the weather in Tokyo?
40 mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 eli5 OK 5 0.7 1.0 4096 33.55
122.1
What is the capital of Japan? Answer in one sentence.
41 mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal think-normal OK 5 0.0 1.0 833 6.83
122.0
Solve step by step: If a train travels at 60 mph for 2.5 hou...
42 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex tool-call-complex OK 4 0.0 1.0 458 3.76
121.8
Search recursively for all Swift files under Sources/ with a...
43 mlx-community/Qwen3.5-35B-A3B-4bit @ default default OK 5 0.7 1.0 997 8.24
121.0
Explain why the sky is blue in exactly 3 bullet points.
44 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role developer-role OK 2 0.0 1.0 437 3.62
120.8
Write a Python function that reverses a string.
45 mlx-community/Qwen3.5-35B-A3B-4bit @ verbose verbose
--verbose
OK 5 0.7 1.0 381 3.16
120.7
Hello, how are you?
46 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none tool-call-none OK 5 0.0 1.0 502 4.16
120.7
What is the weather in Tokyo?
47 mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose very-verbose
--very-verbose
OK 5 0.7 1.0 347 2.89
120.2
Hello, how are you?
48 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 agent-cached-turn3
--enable-prefix-caching
OK 5 0.0 1.0 214 1.78
120.2
Write a unit test for the timeout feature you just described...
49 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema response-format-schema OK 4 0.0 1.0 1478 12.31
120.0
Generate a fictional person profile.
50 mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt long-prompt OK 5 0.0 1.0 831 6.93
120.0
Repeat after me exactly: The quick brown fox jumps over the ...
51 mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual multilingual OK 5 0.3 1.0 688 5.74
119.9
Translate "Hello, how are you?" into French, Spanish, Japane...
52 mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars special-chars OK 5 0.0 1.0 4028 33.61
119.8
Repeat these characters exactly: <tag> "quotes" 'apostrophes...
53 mlx-community/Qwen3.5-35B-A3B-4bit @ top-p top-p
--top-p 0.9
OK 5 0.8 1.0 794 6.64
119.5
Explain why the sky is blue in exactly 3 bullet points.
54 mlx-community/Qwen3.5-35B-A3B-4bit @ code-python code-python OK 2 0.0 1.0 267 2.25
118.7
Write a function that finds all prime numbers up to n using ...
55 mlx-community/Qwen3.5-35B-A3B-4bit @ greedy greedy OK 5 0.0 7.0 546 4.6
118.7
Explain why the sky is blue in exactly 3 bullet points.
56 mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format strict-format OK 5 0.0 1.0 334 2.82
118.5
Output EXACTLY 3 lines. Each line must contain exactly one w...
57 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml tool-call-xml
--tool-call-parser xmlFunction
OK 5 0.0 1.0 330 2.8
117.9
tools: [{"type":"function","function":{"name":"get_weather",...
58 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text response-format-text OK 5 0.0 2.0 370 3.14
117.7
Tell me the language Python, what year it was created, and w...
59 mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers combined-samplers
--top-k 50 --min-p 0.03 --top-p 0.95
OK 5 0.8 1.0 1057 9.04
116.9
Explain why the sky is blue in exactly 3 bullet points.
60 mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt minimal-prompt OK 5 0.7 1.0 235 2.01
116.7
Hi
61 mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs logprobs
--max-logprobs 5
OK 5 0.0 1.0 144 1.24
116.0
What is 1+1?
62 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi tool-call-multi OK 5 0.0 1.0 287 2.48
115.5
tools: [{"type":"function","function":{"name":"get_weather",...
63 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto tool-call-auto OK 5 0.0 1.0 330 2.86
115.2
tools: [{"type":"function","function":{"name":"get_weather",...
64 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex tool-call-complex OK 5 0.0 1.0 338 2.94
114.9
tools: [{"type":"function","function":{"name":"search_files"...
65 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi stop-multi OK 5 0.0 1.0 253 2.3
110.1
Write a Python hello world program in a code block, then wri...
66 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 agent-no-cache-turn3 OK 5 0.0 1.0 149 1.38
108.0
Write a unit test for the timeout feature you just described...
67 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline stop-newline OK 2 0.0 1.0 173 1.63
105.9
What is the capital of France? Answer in one sentence.
68 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested guided-json-nested
--guided-json '{"type":"object","properties":{"city":{"type":"string"},"population":{"type":"integer"},"landmarks":{"type":"array","items":{"type":"string"}}},"required":["city","population","landmarks"]}'
OK 2 0.0 1.0 269 2.54
105.7
What is the capital of Japan? Answer in one sentence.
69 mlx-community/Qwen3.5-35B-A3B-4bit @ json-output json-output OK 5 0.0 1.0 269 2.55
105.3
What is the capital of Japan? Answer in one sentence.
70 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple guided-json-simple
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]}'
OK 5 0.0 1.0 269 2.57
104.7
What is the capital of Japan? Answer in one sentence.
71 mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list numbered-list OK 5 0.0 1.0 269 2.58
104.1
What is the capital of Japan? Answer in one sentence.
72 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi stop-multi OK 2 0.0 1.0 269 2.59
103.8
What is the capital of Japan? Answer in one sentence.
73 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single stop-single OK 3 0.0 1.0 269 2.6
103.4
What is the capital of Japan? Answer in one sentence.
74 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json response-format-json OK 2 0.0 1.0 269 2.6
103.4
What is the capital of Japan? Answer in one sentence.
75 mlx-community/Qwen3.5-35B-A3B-4bit @ long-form long-form OK 5 0.7 1.0 266 2.58
103.1
What is the capital of Japan? Answer in one sentence.
76 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens max-completion-tokens OK 4 0.0 1.0 269 2.62
102.8
What is the capital of Japan? Answer in one sentence.
77 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto tool-call-auto OK 5 0.0 1.0 269 2.62
102.7
What is the capital of Japan? Answer in one sentence.
78 mlx-community/Qwen3.5-35B-A3B-4bit @ verbose verbose
--verbose
OK 5 0.7 1.0 275 2.68
102.7
What is the capital of Japan? Answer in one sentence.
79 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 prefill-large-4096
--prefill-step-size 4096
OK 5 0.0 1.0 284 2.77
102.6
What is the capital of Japan? Answer in one sentence.
80 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi tool-call-multi OK 2 0.0 1.0 269 2.62
102.6
What is the capital of Japan? Answer in one sentence.
81 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 agent-cached-turn1
--enable-prefix-caching
OK 5 0.0 1.0 64 0.62
102.5
Read the file Sources/MacLocalAPI/main.swift and explain wha...
82 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default prefill-default OK 5 0.0 1.0 284 2.77
102.5
What is the capital of Japan? Answer in one sentence.
83 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml tool-call-xml
--tool-call-parser xmlFunction
OK 3 0.0 1.0 269 2.63
102.3
What is the capital of Japan? Answer in one sentence.
84 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role developer-role OK 5 0.0 1.0 269 2.64
102.0
What is the capital of Japan? Answer in one sentence.
85 mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw think-raw
--raw
OK 5 0.0 1.0 269 2.64
102.0
What is the capital of Japan? Answer in one sentence.
86 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 prefill-small-256
--prefill-step-size 256
OK 5 0.0 1.0 284 2.79
101.8
What is the capital of Japan? Answer in one sentence.
87 mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming no-streaming
--no-streaming
OK 5 0.7 1.0 237 2.33
101.8
What is the capital of Japan? Answer in one sentence.
88 mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format strict-format OK 5 0.0 1.0 269 2.65
101.7
What is the capital of Japan? Answer in one sentence.
89 mlx-community/Qwen3.5-35B-A3B-4bit @ default default OK 5 0.7 1.0 332 3.27
101.6
What is the capital of Japan? Answer in one sentence.
90 mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal think-normal OK 5 0.0 1.0 269 2.65
101.6
What is the capital of Japan? Answer in one sentence.
91 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex tool-call-complex OK 3 0.0 1.0 269 2.66
101.2
What is the capital of Japan? Answer in one sentence.
92 mlx-community/Qwen3.5-35B-A3B-4bit @ short-output short-output OK 2 0.7 1.0 50 0.49
101.1
Describe the entire history of Rome.
93 mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt long-prompt OK 5 0.0 1.0 269 2.67
100.6
What is the capital of Japan? Answer in one sentence.
94 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 agent-no-cache-turn2 OK 5 0.0 1.0 103 1.03
100.3
Now add a --timeout flag to the CLI that sets a request time...
95 mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs logprobs
--max-logprobs 5
OK 3 0.0 1.0 269 2.68
100.2
What is the capital of Japan? Answer in one sentence.
96 mlx-community/Qwen3.5-35B-A3B-4bit @ long-output long-output OK 2 0.7 1.0 204 2.04
100.0
What is the capital of Japan? Answer in one sentence.
97 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 seed-42-run1
--seed 42
OK 5 0.7 2.0 214 2.14
100.0
What is the capital of Japan? Answer in one sentence.
98 mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars special-chars OK 5 0.0 1.0 269 2.7
99.8
What is the capital of Japan? Answer in one sentence.
99 mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty repetition-penalty
--repetition-penalty 1.2
OK 5 0.8 1.0 4096 41.37
99.0
Write a long essay about the history of bread making across ...
100 mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty with-penalty
--presence-penalty 1.5
OK 5 0.8 1.0 3893 39.33
99.0
Write a long essay about the history of bread making across ...
101 mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty no-penalty
--presence-penalty 0.0
OK 2 0.8 2.0 213 2.16
98.7
What is the capital of Japan? Answer in one sentence.
102 mlx-community/Qwen3.5-35B-A3B-4bit @ top-k top-k
--top-k 30
OK 5 0.8 1.0 311 3.16
98.5
What is the capital of Japan? Answer in one sentence.
103 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema response-format-schema OK 5 0.0 1.0 269 2.74
98.3
What is the capital of Japan? Answer in one sentence.
104 mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized kv-quantized
--kv-bits 4
OK 5 0.7 1.0 204 2.08
98.2
What is the capital of Japan? Answer in one sentence.
105 mlx-community/Qwen3.5-35B-A3B-4bit @ math math OK 2 0.0 1.0 269 2.74
98.2
What is the capital of Japan? Answer in one sentence.
106 mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose very-verbose
--very-verbose
OK 5 0.7 1.0 222 2.26
98.1
What is the capital of Japan? Answer in one sentence.
107 mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv small-kv
--max-kv-size 2048
OK 5 0.7 1.0 195 1.99
97.8
What is the capital of Japan? Answer in one sentence.
108 mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual multilingual OK 2 0.3 1.0 255 2.61
97.6
What is the capital of Japan? Answer in one sentence.
109 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 seed-42-run2
--seed 42
OK 2 0.7 1.0 214 2.2
97.4
What is the capital of Japan? Answer in one sentence.
110 mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded non-streaming-seeded
--no-streaming --seed 123
OK 4 0.7 1.0 205 2.11
97.3
What is the capital of Japan? Answer in one sentence.
111 mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded streaming-seeded
--seed 123
OK 3 0.7 1.0 205 2.11
97.3
What is the capital of Japan? Answer in one sentence.
112 mlx-community/Qwen3.5-35B-A3B-4bit @ greedy greedy OK 3 0.0 7.0 269 2.77
97.1
What is the capital of Japan? Answer in one sentence.
113 mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode raw-mode
--raw
OK 4 0.7 1.0 204 2.11
96.8
What is the capital of Japan? Answer in one sentence.
114 mlx-community/Qwen3.5-35B-A3B-4bit @ top-p top-p
--top-p 0.9
OK 5 0.8 1.0 254 2.63
96.8
What is the capital of Japan? Answer in one sentence.
115 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none tool-call-none OK 4 0.0 1.0 269 2.79
96.5
What is the capital of Japan? Answer in one sentence.
116 mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp high-temp OK 5 1.5 1.0 198 2.06
96.1
What is the capital of Japan? Answer in one sentence.
117 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text response-format-text OK 3 0.0 2.0 269 2.89
93.0
What is the capital of Japan? Answer in one sentence.
118 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline stop-newline OK 2 0.0 1.0 232 2.5
92.7
What is the capital of Japan? Answer in one sentence.
119 mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt minimal-prompt OK 3 0.7 1.0 199 2.16
92.1
What is the capital of Japan? Answer in one sentence.
120 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 agent-no-cache-turn1 OK 5 0.0 1.0 67 0.74
91.1
Read the file Sources/MacLocalAPI/main.swift and explain wha...
121 mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers combined-samplers
--top-k 50 --min-p 0.03 --top-p 0.95
OK 5 0.8 1.0 215 2.37
90.8
What is the capital of Japan? Answer in one sentence.
122 mlx-community/Qwen3.5-35B-A3B-4bit @ min-p min-p
--min-p 0.05
OK 5 0.8 1.0 206 2.29
89.8
What is the capital of Japan? Answer in one sentence.
123 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single stop-single OK 5 0.0 1.0 199 2.3
86.6
List 10 fruits, numbered 1-10, one per line.
124 mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty repetition-penalty
--repetition-penalty 1.2
OK 4 0.8 1.0 263 3.15
83.6
What is the capital of Japan? Answer in one sentence.
125 mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty with-penalty
--presence-penalty 1.5
OK 4 0.8 1.0 257 3.12
82.5
What is the capital of Japan? Answer in one sentence.
126 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 agent-cached-turn2
--enable-prefix-caching
OK 2 0.0 1.0 64 1.06
60.6
What is the capital of Japan? Answer in one sentence.
127 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 agent-no-cache-turn2 OK 2 0.0 1.0 64 1.06
60.4
What is the capital of Japan? Answer in one sentence.
128 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 agent-cached-turn1
--enable-prefix-caching
OK 2 0.0 1.0 64 1.07
59.6
What is the capital of Japan? Answer in one sentence.
129 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 agent-no-cache-turn3 OK 2 0.0 1.0 64 1.08
59.5
What is the capital of Japan? Answer in one sentence.
130 mlx-community/Qwen3.5-35B-A3B-4bit @ short-output short-output OK 2 0.7 1.0 50 0.87
57.3
What is the capital of Japan? Answer in one sentence.
131 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 agent-cached-turn3
--enable-prefix-caching
OK 2 0.0 1.0 64 1.24
51.7
What is the capital of Japan? Answer in one sentence.
132 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 agent-no-cache-turn1 OK 2 0.0 1.0 64 1.39
46.1
What is the capital of Japan? Answer in one sentence.

AI Analysis (--smart)

Quality, Anomalies & Recommendations · avg score: 4.1/5

Full Responses

mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple 1373 tokens · 125.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode 1662 tokens · 125.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ scientist 2236 tokens · 125.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested 937 tokens · 125.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv 1231 tokens · 124.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 4096 tokens · 124.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty 4096 tokens · 124.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 3652 tokens · 124.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default 3652 tokens · 124.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ math 1830 tokens · 124.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ long-output 1994 tokens · 124.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 4096 tokens · 124.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json 1303 tokens · 124.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 1144 tokens · 124.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 3652 tokens · 124.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized 1032 tokens · 124.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ long-form 3468 tokens · 124.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ pirate 1212 tokens · 123.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming 4094 tokens · 123.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 4096 tokens · 123.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens 1174 tokens · 123.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens 2958 tokens · 123.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ min-p 572 tokens · 123.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ pirate 4096 tokens · 123.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift 4096 tokens · 122.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ json-output 384 tokens · 122.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml 4096 tokens · 122.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi 1169 tokens · 122.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded 2469 tokens · 122.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp 995 tokens · 122.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw 833 tokens · 122.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ code-python 4096 tokens · 122.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded 2469 tokens · 122.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ top-k 1270 tokens · 122.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list 489 tokens · 122.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift 4096 tokens · 122.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role 4096 tokens · 122.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ scientist 4096 tokens · 122.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto 502 tokens · 122.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 4096 tokens · 122.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal 833 tokens · 122.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex 458 tokens · 121.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ default 997 tokens · 121.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role 437 tokens · 120.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ verbose 381 tokens · 120.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none 502 tokens · 120.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose 347 tokens · 120.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 214 tokens · 120.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema 1478 tokens · 120.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt 831 tokens · 120.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual 688 tokens · 119.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars 4028 tokens · 119.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ top-p 794 tokens · 119.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ code-python 267 tokens · 118.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ greedy 546 tokens · 118.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format 334 tokens · 118.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml 330 tokens · 117.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text 370 tokens · 117.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers 1057 tokens · 116.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt 235 tokens · 116.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs 144 tokens · 116.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi 287 tokens · 115.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto 330 tokens · 115.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex 338 tokens · 114.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi 253 tokens · 110.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 149 tokens · 108.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline 173 tokens · 105.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested 269 tokens · 105.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ json-output 269 tokens · 105.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple 269 tokens · 104.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list 269 tokens · 104.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi 269 tokens · 103.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single 269 tokens · 103.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json 269 tokens · 103.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ long-form 266 tokens · 103.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens 269 tokens · 102.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto 269 tokens · 102.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ verbose 275 tokens · 102.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 284 tokens · 102.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi 269 tokens · 102.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 64 tokens · 102.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default 284 tokens · 102.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml 269 tokens · 102.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role 269 tokens · 102.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw 269 tokens · 102.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 284 tokens · 101.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming 237 tokens · 101.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format 269 tokens · 101.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ default 332 tokens · 101.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal 269 tokens · 101.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex 269 tokens · 101.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ short-output 50 tokens · 101.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt 269 tokens · 100.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 103 tokens · 100.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs 269 tokens · 100.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ long-output 204 tokens · 100.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 214 tokens · 100.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars 269 tokens · 99.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty 4096 tokens · 99.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty 3893 tokens · 99.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty 213 tokens · 98.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ top-k 311 tokens · 98.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema 269 tokens · 98.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized 204 tokens · 98.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ math 269 tokens · 98.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose 222 tokens · 98.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv 195 tokens · 97.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual 255 tokens · 97.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 214 tokens · 97.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded 205 tokens · 97.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded 205 tokens · 97.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ greedy 269 tokens · 97.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode 204 tokens · 96.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ top-p 254 tokens · 96.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none 269 tokens · 96.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp 198 tokens · 96.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text 269 tokens · 93.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline 232 tokens · 92.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt 199 tokens · 92.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 67 tokens · 91.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers 215 tokens · 90.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ min-p 206 tokens · 89.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single 199 tokens · 86.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty 263 tokens · 83.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty 257 tokens · 82.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 64 tokens · 60.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 64 tokens · 60.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 64 tokens · 59.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 64 tokens · 59.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ short-output 50 tokens · 57.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 64 tokens · 51.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 64 tokens · 46.1 tok/s