| Model | Parser | Pass | Fail | Skip | Total | Status |
|---|---|---|---|---|---|---|
| mlx-community/gemma-4-e4b-it-4bit | auto | 340 | 30 | 0 | ❌ [details] |
| # | Test | Group | Coverage | Status | Duration | Details |
|---|---|---|---|---|---|---|
| 1 | collects timing and token stats from final chunk | UnitTest | unit smoke standard full | PASS | 0.0s | pass |
| 2 | finishReason is stop when stopped by sequence | UnitTest | unit smoke standard full | PASS | 0.0s | pass |
| 3 | finishReason is tool_calls when tool calls present | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 4 | collects logprobs from stream chunks | UnitTest | unit smoke standard full | PASS | 0.0s | pass |
| 5 | empty stream produces nil content | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 6 | finishReason is stop by default | UnitTest | unit smoke standard full | PASS | 0.0s | pass |
| 7 | buildChoiceLogprobs converts ResolvedLogprob to ChoiceLogprobs | UnitTest | unit smoke standard full | PASS | 0.0s | pass |
| 8 | think extraction with tool calls returns nil content and reasoning | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 9 | extracts think tags into reasoningContent | UnitTest | unit smoke standard full | PASS | 0.0s | pass |
| 10 | decodeJSONEscapes passes through plain text without escapes | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 11 | coerceStringValue returns nil for non-numeric string with integer type | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 12 | returns unchanged when tools is nil | UnitTest | unit smoke standard full | PASS | 0.0s | pass |
| 13 | buildChoiceLogprobs returns nil for empty input | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 14 | finishReason is length when max tokens reached | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 15 | multi-chunk think tags extracted correctly | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 16 | BatchInputLine handles messages with special characters | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 17 | JSONL with blank lines are skipped | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 18 | Multiple JSONL lines parse independently | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 19 | no think extraction when disabled | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 20 | getFile returns nil for nonexistent file | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 21 | coerceStringValue returns nil for non-numeric string with number type | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 22 | coerceStringValue returns nil for invalid JSON with array type | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 23 | leaves already-correct types alone (string stays string) | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 24 | NoSystemMessageGenerator includes name for tool messages | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 25 | user message has no name field in dict | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 26 | stream with only think tags and no visible content | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 27 | does not fill non-required missing params | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 28 | tool message without name omits name from dict | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 29 | markBatchInProgress transitions status and stores slot IDs | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 30 | createBatch returns ID with correct prefix and initial state | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 31 | BatchInputLine handles system + user messages | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 32 | storeFile creates file with correct metadata | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 33 | BatchInputLine handles optional body fields | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 34 | deleteFile returns false for nonexistent file | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 35 | markBatchFailed sets status and error | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 36 | getFileData returns raw data | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 37 | getFileData returns nil for nonexistent file | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 38 | coerces string to object | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 39 | decodeXMLEntities decodes all 5 standard entities | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 40 | getBatch returns nil for nonexistent batch | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 41 | decodeXMLEntities passes through bare & without entity suffix | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 42 | system messages filtered by NoSystemMessageGenerator | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 43 | handles multiline parameter values | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 44 | coerceStringValue coerces '0.0' to Int 0 for number type (whole number) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 45 | coerces string to negative integer | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 46 | parses JSON-in-XML format with 'arguments' key | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 47 | runtime parses adaptive xml JSON fallback | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 48 | regex fallback handles bare < in parameter value | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 49 | decodeXMLEntities handles double-encoded entities: &lt; → < | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 50 | text content before and after tool call is preserved | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 51 | JSON-in-XML with missing name returns no tool calls | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 52 | runtime emits placeholder then final replacement for XML tool call | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 53 | bash tool with JSON-escaped command gets decoded | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 54 | runtime salvages incomplete tool call at stream end | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 55 | tool message with name includes name in dict | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 56 | coerceStringValue handles negative numbers | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 57 | Multiple files have unique IDs | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 58 | markBatchCancelling and markBatchCancelled transitions | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 59 | write tool with real newlines is NOT affected by decodeJSONEscapes | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 60 | requiredParamNames returns empty for tool with no required array | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 61 | coerceStringValue returns nil for unknown schema type | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 62 | decodeJSONEscapes full edit oldString from real Qwen3-Coder output | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 63 | parses Mistral [TOOL_CALLS] func_name[ARGS]{...} format | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 64 | XMLParser path drops unclosed parameter (no bare < to trigger fallback) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 65 | deleteFile removes the file | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 66 | recordResult increments failed count for error results | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 67 | completed tool call parsing preserves remaining non-tool text | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 68 | handles multiple parameters with mixed string and JSON types | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 69 | returns unchanged when arguments JSON is invalid | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 70 | bash tool with special characters | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 71 | preserves JSON array parameter values | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 72 | parses bare JSON tool call without wrapper tags | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 73 | strips <think></think> tags from remaining text | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 74 | parses JSON-in-XML format with 'parameters' key | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 75 | decodeJSONEscapes handles only \" without \n | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 76 | BatchScheduler stop helper buffers partial stop across chunk boundaries | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 77 | bare JSON with 'parameters' key instead of 'arguments' | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 78 | toJinjaCompatible() result survives ToolSpec → Jinja Value round-trip | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 79 | EBNF grammar param_value uses permissive pv_char rule | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 80 | regex fallback salvages unclosed parameter (max_tokens truncation) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 81 | write file with Python containing comparison operators (OpenCode pattern) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 82 | does not coerce when property not in schema | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 83 | does not coerce non-string values | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 84 | completed tool call parsing resolves adaptive XML tool names against offered tools | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 85 | decodeJSONEscapes handles \t escapes | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 86 | edit tool with JSON-escaped oldString gets decoded (model pre-escaping bug) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 87 | JSON-in-XML with invalid JSON falls through | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 88 | handles mixed XML and JSON in separate <tool_call> blocks | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 89 | insert: partial edge split with remaining tokens | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 90 | truncateToOffset on empty cache is no-op | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 91 | KVCacheEntry stores tokens and layerStates | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 92 | toJinjaCompatible() strips null keys from dicts — Jinja accepts it | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 93 | Count tracks insertions, evictions, and invalidation | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 94 | parses basic XML tool call | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 95 | coerces string to integer | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 96 | toAny() result in ToolSpec — Jinja throws (proves the bug) | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 97 | JSON-in-XML with no arguments key still parses name | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 98 | toAny() wraps null as NSNull — Jinja throws on it | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 99 | StreamChunk defaults: only text, everything else nil | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 100 | mixed: write content has real newlines, edit oldString has escaped — both decoded correctly | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 101 | empty <tool_call></tool_call> block returns no calls | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 102 | EBNF grammar generates named required param rules for bash tool | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 103 | findPrefix: query shorter than cached edge | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 104 | normalizeToolCalls assigns sequential indices from startIndex | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 105 | EBNF grammar generates named required param rules for edit tool | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 106 | BatchResultLine with error encodes correctly | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 107 | BatchScheduler emits only completed tool calls from slot runtime events | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 108 | toJinjaCompatible() strips null elements from arrays | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 109 | BaseKVCache truncateToOffset is safe no-op on MambaCache | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 110 | KVCacheSimple and ArraysCache are batch-eligible | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 111 | findPrefix with debugLogging exercises all log branches | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 112 | MLXServiceError.noModelLoaded has descriptive message | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 113 | findPrefix: miss with matched == 0 and debugLogging | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 114 | RequestSlot appendContent accumulates text | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 115 | KVCacheEntry touch updates access time | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 116 | mixed coercion: some params coerced, some left alone | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 117 | getFile returns stored file metadata | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 118 | runtime preserves nulls in adaptive xml JSON fallback | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 119 | normalizeToolCall remaps keys before coercing values | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 120 | insert: empty tokens is no-op | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 121 | toAny() dict with null value — Jinja throws | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 122 | Eviction with debugLogging | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 123 | FileDeleteResponse defaults | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 124 | fills missing required boolean param with false | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 125 | findPrefix: deepest cached node wins over shallower | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 126 | MLXServiceError.serviceShuttingDown has descriptive message | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 127 | RotatingKVCache is NOT batch-eligible | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 128 | insert: new edge (no matching child) | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 129 | invalidateAll with debugLogging | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 130 | StreamChunk carries cached token count | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 131 | does NOT fill missing required string param (let client report missing) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 132 | decodeXMLEntities handles multiple < in Python code | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 133 | toJinjaCompatible() deeply nested nulls — Jinja accepts them | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 134 | MambaCache without leftPadding returns nil mask | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 135 | returns unchanged when tool name not found in tools list | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 136 | BatchListResponse encodes with has_more field | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 137 | BatchObject encodes with all fields | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 138 | coerceStringValue returns nil for invalid JSON with object type | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 139 | insert: partial edge split at exact split point | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 140 | StreamChunk carries timing info | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 141 | BatchScheduler stop helper emits stopped chunk on exact stop match | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 142 | recordResult increments completed count | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 143 | coerces string to array | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 144 | recordResult drops results for cancelling batch | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 145 | recordResult drops results for cancelled batch | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 146 | requiredParamNames preserves client schema order (not alphabetical) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 147 | BatchCreateRequest decodes with snake_case keys | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 148 | toJinjaCompatible() preserves all non-null types | NullableSchema | unit smoke standard full | PASS | 0.1s | pass |
| 149 | TopKProcessor is no-op when k <= 0 | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 150 | PresenceContext with empty tokens is no-op | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 151 | GrammarLogitProcessor with nil mask is passthrough | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 152 | Batched prefill cache creation matches expected pattern for mixed layers | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 153 | listBatches returns all batches | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 154 | EBNF grammar allows extra optional params after required ones | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 155 | parses Mistral [TOOL_CALLS] with multiple items in array | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 156 | BatchInputLine decodes from JSONL line | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 157 | insert with debugLogging exercises log branches | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 158 | getSlotIds returns empty for nonexistent batch | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 159 | recordResult still works for in_progress batch | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 160 | question tool: array of objects with typed fields (OpenCode pattern) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 161 | findPrefix: traverses nodes but no cached node found | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 162 | RequestSlot IDs are unique across instances | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 163 | Output JSONL file contains all results | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 164 | Batch auto-completes when all requests are done | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 165 | BatchScheduler emits delta chunks before completed tool call chunks | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 166 | BatchResultLine round-trips through JSON | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 167 | handles parameter values with special characters in Python code | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 168 | bare JSON that's not a tool call doesn't match | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 169 | Conversation fork: both branches survive and are findable | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 170 | fills missing required integer param with 0 | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 171 | preserves JSON object parameter values | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 172 | findPrefix: query longer than cached — partial prefix hit | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 173 | decodes XML entities in parameter values via XMLParser | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 174 | coerceStringValue returns nil for float string with integer type | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 175 | EBNF grammar handles tool with no required params | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 176 | coerceStringValue handles empty string for all types | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 177 | insert: multiple inserts grow the tree (multi-turn) | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 178 | RequestSlot initializes with unique ID and zero content | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 179 | decodeJSONEscapes unescapes literal \n and \" when no real newlines present | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 180 | FileObject encodes with snake_case keys | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 181 | MambaCache mask returns nil after state is set (non-prefill) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 182 | findPrefix: empty cache — miss | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 183 | BatchScheduler default max concurrent is 8 | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 184 | truncateToOffset is no-op when offset matches array size | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 185 | createSSMMask delegates to MambaCache.makeMask | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 186 | truncateToOffset physically shrinks arrays to offset | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 187 | BatchKVCacheSimple update stores KV for all sequences | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 188 | state getter returns raw refs after truncateToOffset (fast path) | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 189 | MinPProcessor works with 2D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 190 | createAttentionMask delegates to BatchKVCacheSimple.makeMask | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 191 | Touch via findPrefix prevents eviction | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 192 | BatchCompletionRequest decodes array of requests | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 193 | BatchScheduler stop helper does not stop while still inside think block | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 194 | findPrefix: no child for next token — miss at root | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 195 | BatchScheduler helper extraction keeps tool call streams isolated per event list | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 196 | invalidateAll clears everything | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 197 | handles duplicate parameters (keeps first non-empty) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 198 | findPrefix: exact match returns full prefix | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 199 | RadixNode isLeaf and hasCachedState | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 200 | decodeXMLEntities handles empty string | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 201 | JSON format switch mid-conversation (Qwen3-Coder-Next common error) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 202 | truncateToOffset after multiple trim cycles | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 203 | BatchScheduler emits incremental tool call deltas from slot runtime events | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 204 | insert: exact match update — node already has cache | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 205 | insert: exact match — node had no prior cache | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 206 | Multiple evictions on single insert (maxEntries=1) | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 207 | decodeXMLEntities passes through strings without entities | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 208 | preserves content outside tool calls | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 209 | findPrefix: partial edge match divergence mid-edge — returns cached state | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 210 | compactUpward merges single-child node after sibling eviction | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 211 | decodeXMLEntities handles entities at string boundaries | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 212 | findPrefix: miss with matched > 0 and debugLogging | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 213 | LRU eviction at capacity evicts oldest | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 214 | handles multiple tool calls | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 215 | decodeXMLEntities handles consecutive entities | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 216 | decodeJSONEscapes handles empty string | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 217 | decodeJSONEscapes preserves real backslashes (\\n stays as \n literal) | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 218 | decodeJSONEscapes preserves values that already have real newlines | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 219 | returns nil for content without <function=...> tag | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 220 | coerceStringValue coerces '0' to Int 0 for integer type | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 221 | coerces whole-number string to Int for 'number' type | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 222 | parses Mistral [TOOL_CALLS] JSON array format | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 223 | multiple parameters with same non-empty value keeps first | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 224 | returns empty array and original text when no tool calls found | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 225 | does NOT coerce 'yes'/'no'/'1'/'0' to boolean | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 226 | returns unchanged when tools is empty | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 227 | coerces string to number (float) | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 228 | skips empty parameter values | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 229 | coerces string to boolean true and false | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 230 | RequestSlot is thread-safe under concurrent writes | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 231 | coerces boolean case-insensitively (TRUE, True, FALSE) | XMLParsing | unit smoke standard full | PASS | 0.0s | pass |
| 232 | Cancel batch transitions to cancelled | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 233 | Cancel batch on non-in_progress returns 400 | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 234 | Cancel nonexistent batch returns 404 | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 235 | BatchStore handles concurrent batch creates safely | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 236 | BatchCompletionsController sets grammar downgrade header when strict tools present | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 237 | Rejects line with wrong method | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 238 | Rejects line with wrong url | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 239 | Accepts valid method and url | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 240 | BatchStore handles concurrent file operations safely | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 241 | TopKProcessor is no-op when k >= vocabSize | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 242 | GrammarLogitProcessor works with 1D logits | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 243 | MambaCache leftPadding creates correct SSM mask | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 244 | GrammarLogitProcessor works with 2D logits | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 245 | TopKProcessor works with 2D logits [B, vocabSize] | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 246 | BatchKVCacheSimple mask with uniform padding (same-length sequences) | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 247 | Extracted cache can be loaded into KVCacheSimple for radix save | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 248 | TopKProcessor works with 1D logits [vocabSize] | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 249 | RepetitionContext works with 2D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 250 | RepetitionContext works with 1D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 251 | PresenceContext works with 1D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 252 | CompositeLogitProcessor chains processors correctly with 2D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 253 | PresenceContext works with 2D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 254 | MinPProcessor works with 1D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 255 | RepetitionContext applies multiplicative penalty to negative logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 256 | BatchKVCacheSimple decode mask with padding (n=1) | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 257 | CompositeLogitProcessor chains processors correctly with 1D logits | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 258 | BatchKVCacheSimple mask with asymmetric padding (different-length sequences) | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 259 | MambaCache batched state can be sliced per-sequence | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 260 | Full extraction round-trip: batch → individual → KVCacheSimple | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 261 | Separate processor instances have independent state | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 262 | BatchKVCacheSimple extend merges prefill batch with existing decode batch | XMLParsing | unit smoke standard full | PASS | 0.1s | pass |
| 263 | Left-padding stacks variable-length token arrays correctly | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 264 | didSample updates only the called processor's state | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 265 | BatchKVCacheSimple extract returns correct per-sequence KV | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 266 | RequestSlot elapsedTime increases over time | UnitTest | unit smoke standard full | PASS | 0.1s | pass |
| 267 | BatchAPIController response includes tool_calls with correct finish_reason | UnitTest | unit smoke standard full | PASS | 0.6s | pass |
| 268 | BatchAPIController response includes reasoningContent when model supports thinking | UnitTest | unit smoke standard full | PASS | 0.6s | pass |
| 269 | BatchAPIController response includes logprobs when provided in stream | UnitTest | unit smoke standard full | PASS | 0.6s | pass |
| 270 | Binary exists at .build/release/afm | Preflight | unit smoke standard full | PASS | 0.0s | file exists |
| 271 | Server reachable at http://127.0.0.1:9998 | Preflight | unit smoke standard full | PASS | 0.0s | 200 OK |
| 272 | /v1/models contains model ID | Lifecycle | unit smoke standard full | PASS | 0.0s | model in response |
| 273 | Basic completion returns content | Lifecycle | unit smoke standard full | PASS | 0.3s | non-empty content or reasoning |
| 274 | Stop string '5' absent from output | Stop | unit smoke standard full | PASS | 0.2s | no '5' in content |
| 275 | finish_reason is 'stop' with stop sequence | Stop | unit smoke standard full | PASS | 0.0s | stop |
| 276 | Multi-word stop 'and' truncates correctly | Stop | unit smoke standard full | PASS | 2.0s | no 'and' in output |
| 277 | Stop on newline produces single line | Stop | unit smoke standard full | PASS | 0.1s | single line |
| 278 | Multiple stop sequences [7, 12] | Stop | unit smoke standard full | PASS | 0.3s | neither found |
| 279 | Empty stop array is no-op | Stop | unit smoke standard full | PASS | 0.2s | valid output |
| 280 | Streaming: stop string '5' absent | Stop | unit smoke standard full | FAIL | 0.2s | Expected: no '5' in stream\nActual: FAIL: found '5' |
| 281 | Stop sequence '3.' truncates list | Stop | standard full | PASS | 0.2s | no '3.' in output |
| 282 | Stop 'stopped' doesn't fire on 'stopping' | Stop | standard full | PASS | 1.0s | output produced |
| 283 | Stop 'llo' fires mid-word in 'hello' | Stop | standard full | PASS | 0.1s | no 'llo' |
| 284 | ChoiceLogprobs JSON schema valid | Logprobs | smoke standard full | PASS | 0.2s | valid schema |
| 285 | top_logprobs count <= requested (5) | Logprobs | smoke standard full | PASS | 0.1s | count valid |
| 286 | logprobs=false returns null | Logprobs | smoke standard full | PASS | 0.1s | null logprobs |
| 287 | top_logprobs=99 returns 400 | Logprobs | smoke standard full | PASS | 0.0s | 400 |
| 288 | Streaming logprobs present and valid | Logprobs | standard full | PASS | 0.1s | valid |
| 289 | top_logprobs=0 returns empty arrays | Logprobs | standard full | PASS | 0.1s | empty top_logprobs |
| 290 | reasoning_content present in response | Think | smoke standard full | PASS | 0.0s | present |
| 291 | No <think> tags in content field | Think | smoke standard full | PASS | 0.0s | clean content |
| 292 | Streaming: reasoning_content in deltas | Think | smoke standard full | PASS | 1.0s | present |
| 293 | Stop sequence doesn't break think extraction | Think | smoke standard full | PASS | 2.0s | response ok |
| 294 | reasoning_content has meaningful length (>5 chars) | Think | smoke standard full | PASS | 0.0s | >5 chars |
| 295 | Basic tool call: finish_reason=tool_calls, valid args | Tools | smoke standard full | FAIL | 3.0s | Expected: valid\nActual: FAIL: finish_reason=stop, tool_calls=0 |
| 296 | tool_choice=none suppresses tool calls | Tools | smoke standard full | PASS | 1.0s | no tool calls |
| 297 | Tool arguments are valid JSON dict | Tools | standard full | FAIL | 2.9s | Expected: valid JSON\nActual: FAIL: no tool calls |
| 298 | Streaming: tool calls with finish_reason | Tools | standard full | FAIL | 2.5s | Expected: valid\nActual: FAIL: no tool_calls in stream |
| 299 | Multi-tool: at least 1 tool call with 2 tools | Tools | standard full | FAIL | 3.0s | Expected: >=1 calls\nActual: FAIL: 0 tool calls |
| 300 | Array param: todos is JSON array (not string) | Tools | standard full | FAIL | 0.9s | Expected: array\nActual: FAIL: no tool calls |
| 301 | Nullable param: anyOf [string, null] does not crash | Tools | standard full | FAIL | 0.0s | Expected: no crash\nActual: FAIL: 'str' object has no attribute 'get' |
| 302 | Multi-turn: tool round-trip (model did not call tool) | Tools | standard full | SKIP | 3.0s | skip |
| 303 | Multi-turn nullable: Jinja crash on anyOf [type, null] | Tools | standard full | FAIL | 0.0s | Expected: no crash\nActual: FAIL: ERROR:'str' object has no attribute 'get' |
| 304 | No tools: normal text response | Tools | standard full | PASS | 0.3s | text response |
| 305 | First unique request: uncached suffix remains | Cache | standard full | PASS | 0.3s | cached_tokens < prompt_tokens |
| 306 | Shared-prefix request: cached_tokens>0 | Cache | standard full | FAIL | 0.5s | Expected: >0\nActual: FAIL: cached_tokens=0 |
| 307 | Exact replay: cached response matches cold response | Cache | standard full | PASS | 0.5s | same content |
| 308 | Different unique prompt: uncached suffix remains | Cache | standard full | PASS | 0.3s | cached_tokens < prompt_tokens |
| 309 | Streaming shared-prefix: cached_tokens>0 in usage chunk | Cache | standard full | FAIL | 0.9s | Expected: >0\nActual: FAIL: no cached_tokens>0 in stream usage |
| 310 | Streaming replay: content matches non-streaming warmup | Cache | standard full | PASS | 1.1s | same content |
| 311 | Concurrent x8 shared-prefix: uncached suffix remains on every branch | Cache | standard full | FAIL | 2.9s | Expected: 0 < cached_tokens < prompt_tokens for all 8\nActual: FAIL: 1:0/61, 2:0/61, 3:0/61, 4:0/61, 5:0/61, 6:0/61, 7:0/61, 8:0/61 |
| 312 | Concurrent x8 shared-prefix: divergent suffix responses stay isolated | Cache | standard full | FAIL | 2.9s | Expected: each of 8 responses keeps only its own marker\nActual: FAIL: 1:error=Expecting value: line 1 column 1 (char 0); 2:error=Expecting value: line 1 column 1 (char 0); 3:error=Expecting value: line 1 column 1 (char 0); 4:error=Expecting value: line 1 column 1 (char 0); 5:error=Expecting value: line 1 column 1 (char 0); 6:error=Expecting value: line 1 column 1 (char 0); 7:error=Expecting value: line 1 column 1 (char 0); 8:error=Expecting value: line 1 column 1 (char 0) |
| 313 | Two simultaneous requests: both 200 | Concurrent | standard full | PASS | 0.2s | 200+200 |
| 314 | Three simultaneous requests: all 200 | Concurrent | standard full | PASS | 0.2s | all 200 |
| 315 | Empty messages → 400 | Error | smoke standard full | PASS | 0.0s | 400 |
| 316 | Malformed JSON → 400 | Error | smoke standard full | PASS | 0.0s | 400 |
| 317 | Missing messages field → 400 | Error | smoke standard full | PASS | 0.0s | 400 |
| 0 | smoke standard full | FAIL | Expected: \nActual: |
|||
| 319 | max_tokens=5 is respected | Error | smoke standard full | PASS | 0.1s | <=10 tokens |
| 320 | OPTIONS /v1/chat/completions → 200 (CORS) | Error | smoke standard full | PASS | 0.0s | 200 |
| 321 | developer role accepted (mapped to system) | Error | smoke standard full | PASS | 4.0s | valid response |
| 322 | enable_thinking=false disables thinking | Kwargs | standard full | PASS | 0.4s | no_think |
| 323 | Streaming: enable_thinking=false disables thinking | Kwargs | standard full | PASS | 0.4s | no_think |
| 324 | Default (no kwargs) retains thinking | Kwargs | standard full | PASS | 1.0s | thinking |
| 325 | enable_thinking=false (2K tokens) returns content | Kwargs | standard full | PASS | 0.4s | content present, no reasoning |
| 326 | enable_thinking=true explicitly keeps thinking | Kwargs | standard full | PASS | 1.0s | thinking |
| 327 | Prefix cache reuse across requests | Cache | standard full | PASS | 0.3s | valid response on cache hit |
| 328 | Issue #32: sequential guided-json → nullable tool (no crash) | Cache | standard full | FAIL | 2.0s | Expected: both succeed\nActual: FAIL: req1=PASS req2=FAIL: 'str' object has no attribute 'get' |
| 329 | Sequential: 3 nullable tool calls (cache reuse) | Cache | standard full | FAIL | 0.1s | Expected: 3/3 succeed\nActual: FAIL: 0/3 req3=FAIL:'str' object has no attribute 'get' |
| 330 | Multi-turn tool conversation with prefix (model skipped tool) | Cache | standard full | SKIP | 2.3s | skip |
| 331 | Long system prompt: 3 sequential requests (prefix reuse) | Cache | standard full | PASS | 5.6s | 3/3 succeed |
| 332 | Sequential stress: 5 mixed requests (text/tool/json/stream/nullable) | Cache | standard full | FAIL | 3.2s | Expected: 5/5 succeed\nActual: FAIL: 4/5 req5=nullable_fail |
| 333 | Multi-turn: 5-turn growing conversation (prefix reuse) | Cache | standard full | PASS | 4.7s | 5/5 turns |
| 334 | Streaming + non-streaming sequential (shared prefix) | Cache | standard full | PASS | 0.3s | 3/3 succeed |
| 335 | Issue #32 comprehensive: nullable tool + multi-turn + prefix | Cache | standard full | FAIL | 0.0s | Expected: no crash\nActual: FAIL: ERROR:'str' object has no attribute 'get' |
| 336 | json_schema produces valid schema-matching JSON | Structured | standard full | FAIL | 1.1s | Expected: valid JSON\nActual: FAIL: invalid JSON: Expecting value: line 1 column 1 (char 0) |
| 337 | TTFT < 5s | Perf | full | PASS | 0.6s | <5000ms |
| 338 | tok/s > 1 (91.7 tok/s) | Perf | full | PASS | 0.5s | >1 |
| 339 | Long context (~2K tokens) no crash | Perf | full | PASS | 4.0s | valid response |
| 340 | Very long context (~4K tokens) no NaN/garbage | Perf | full | PASS | 4.7s | clean output |
| 341 | XML tool call tests (model lacks tool calling) | XMLTools | standard full | SKIP | skip |
|
| 342 | afm_adaptive_xml tests (model lacks tool calling) | AdaptiveXML | standard full | SKIP | skip |
|
| 343 | Calculator tool call (non-streaming) | Grammar | standard full | FAIL | 3.2s | Expected: calculate(expression)\nActual: FAIL: no tool_calls |
| 344 | Calculator tool call (streaming) | Grammar | standard full | FAIL | 1.0s | Expected: calculate(expression) via SSE\nActual: FAIL: no tool call in stream |
| 345 | Two tools: grammar allows correct selection | Grammar | standard full | FAIL | 2.4s | Expected: get_weather selected\nActual: FAIL: no tool_calls |
| 346 | Two tools: grammar selects calculate | Grammar | standard full | FAIL | 2.8s | Expected: calculate selected\nActual: FAIL: no tool_calls |
| 347 | Grammar enforces 3 required params (send_email) | Grammar | standard full | FAIL | 3.0s | Expected: to + subject + body\nActual: FAIL: no tool_calls |
| 348 | Grammar constrains array param at generation time | Grammar | standard full | FAIL | 3.8s | Expected: tags is array\nActual: FAIL: no tool_calls |
| 349 | Grammar array param via streaming | Grammar | standard full | FAIL | 3.6s | Expected: tags is array (SSE)\nActual: FAIL: no tool call in stream |
| 350 | Complex schema: string + int + array + object | Grammar | standard full | FAIL | 5.5s | Expected: correct types\nActual: FAIL: no tool_calls |
| 351 | Header absent when grammar enabled (tool strict:true) | StrictWiring | smoke standard full | PASS | 2.1s | no header |
| 352 | Header absent when grammar enabled (schema strict:true) | StrictWiring | smoke standard full | PASS | 0.4s | no header |
| 353 | No header when strict absent | StrictWiring | smoke standard full | PASS | 2.0s | no header |
| 354 | Streaming json_schema strict:true returns valid JSON | StrictWiring | smoke standard full | PASS | 0.4s | valid JSON with name+age |
| 355 | Streaming tool strict:true returns valid tool call | StrictWiring | smoke standard full | FAIL | 3.0s | Expected: get_weather(location)\nActual: FAIL: no tool call in stream |
| 356 | strict:false does not error (best-effort) | StrictWiring | smoke standard full | FAIL | 2.0s | Expected: valid response\nActual: FAIL: no tool call and no content |
| 357 | POST /v1/files upload returns file ID | Batch | standard full | PASS | 0.0s | file-* |
| 358 | GET /v1/files/:id returns file metadata | Batch | standard full | PASS | 0.0s | file object |
| 359 | POST /v1/batches creates batch | Batch | standard full | PASS | 0.0s | in_progress/completed |
| 360 | GET /v1/batches/:id polls to completed | Batch | standard full | PASS | 2.1s | completed |
| 361 | Output JSONL contains both results (2/2) | Batch | standard full | PASS | 0.0s | assert-1 + assert-2 |
| 362 | GET /v1/batches lists completed batch | Batch | standard full | PASS | 0.0s | list with ≥1 batch |
| 363 | DELETE /v1/files/:id removes uploaded file | Batch | standard full | PASS | 0.0s | deleted=true |
| 364 | SSE multiplex: 2 non-streaming requests tagged with custom_id | Batch | standard full | PASS | 0.2s | sse-a + sse-b + [DONE] |
| 365 | SSE multiplex: streaming interleaved with finish_reason | Batch | standard full | PASS | 0.2s | str-x + str-y chunks + stop + [DONE] |
| 366 | SSE multiplex rejects duplicate custom_ids | Batch | standard full | PASS | 0.0s | 400 |
| 367 | SSE multiplex rejects empty requests | Batch | standard full | PASS | 0.0s | 400 |
| 368 | batch + top_k + streaming | PairwiseSmoke | standard full | PASS | 0.1s | content present |
| 369 | batch + presence_penalty + non-streaming | PairwiseSmoke | standard full | PASS | 0.1s | finish_reason present |
| 370 | batch + repetition_penalty + logprobs | PairwiseSmoke | standard full | PASS | 0.1s | response valid |
| 371 | batch + all sampling params combined | PairwiseSmoke | standard full | PASS | 0.2s | response valid |
| 372 | streaming parity (same seed → same output) | PairwiseSmoke | standard full | PASS | 0.2s | match |
| 373 | cache idempotency (same seed → same output) | PairwiseSmoke | standard full | FAIL | 0.2s | Expected: r1='' vs r2=''\nActual: FAIL: mismatch |