AFM Regression Test Report

Generated 2026-02-19 07:56 · v0.9.4 · Apple M-series · macOS 26

1 TEST FAILED

Total Tests

51

Passed

50

Failed

1

Skipped

0

Sections

12

Test Results by Section

▶ CLI Basics 4/4 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	--help shows usage	0s
✓ PASS	--version shows version	0s
✓ PASS	mlx --help shows usage	0s
✓ PASS	invalid option rejected	0s

▶ AFM Single Prompt 12/12 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	basic single prompt	1s
✓ PASS	single prompt with instructions	1s
✓ PASS	pipe mode	0s
✓ PASS	temperature 0.0	1s
✓ PASS	temperature 1.0	0s
✓ PASS	randomness greedy	0s
✓ PASS	randomness random	1s
✓ PASS	randomness top-p=0.9	0s
✓ PASS	randomness top-k=50	1s
✓ PASS	randomness top-p + seed	0s
✓ PASS	permissive guardrails -P	0s
✓ PASS	combined: -t -r -i -P	1s

▶ AFM Validation 8/8 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	temperature > 1.0 rejected	0s
✓ PASS	temperature < 0.0 rejected	0s
✓ PASS	temperature non-numeric rejected	0s
✓ PASS	randomness invalid rejected	0s
✓ PASS	top-p > 1.0 rejected	0s
✓ PASS	top-k=0 rejected	0s
✓ PASS	conflicting top-p + top-k rejected	0s
✓ PASS	unknown randomness param rejected	0s

▶ AFM Server 8/8 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	GET /health	0s
✓ PASS	GET /v1/models	0s
✓ PASS	POST chat completion	0s
✓ PASS	POST chat with temperature	0s
✓ PASS	POST chat streaming	1s
✓ PASS	POST multi-turn conversation	1s
✓ PASS	POST empty messages returns error	0s
✓ PASS	POST max_tokens respected	0s

▶ AFM Server Options 2/2 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	permissive guardrails server	0s
✓ PASS	server with -t 0.5 -r greedy	0s

▶ MLX Single Prompt 4/4 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	mlx single prompt	1s
✓ PASS	mlx single prompt with instructions	0s
✓ PASS	mlx single prompt with temperature	1s
✓ PASS	mlx pipe mode	0s

▶ MLX Server 6/6 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	GET /health	0s
✓ PASS	GET /v1/models	0s
✓ PASS	POST chat completion	0s
✓ PASS	POST chat with temperature	0s
✓ PASS	POST chat streaming	1s
✓ PASS	POST empty messages returns error	0s

▶ MLX Download 1/2 passed 1 FAIL

Status	Test	Time	Detail
✗ FAIL	download mlx-community/granite-4.0-350m-bf16	51s	files missing after download
✓ PASS	serve downloaded model	0s

▶ MLX VLM 1/1 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	VLM text-only chat	0s

▶ MLX MoE 1/1 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	MoE chat completion	0s

▶ MLX Validation 2/2 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	mlx without -m fails	0s
✓ PASS	mlx nonexistent model fails	0s

▶ Port Handling 1/1 passed ALL PASS

Status	Test	Time	Detail
✓ PASS	custom port 9873	0s