Tested on oMLX version 0.3.9 dev2. It is not an official release yet.
oMLX shows slightly better results compared to llama.
However, activating MTP in oMLX disables the TurboQuant option, increasing memory usage.
There were doubts about whether llama has issues with kv cache usage, but it operates without errors.
If you have enough memory, MLX shows good performance on macOS environments.
backend | MTP use | model | tokens per second | change |
|---|
oMLX | X | unsloth/Qwen3.6-27B-UD-MLX-4bit | 8 | 0% |
llama.cpp | O | unsloth/Qwen3.6-27B-GGUF-MTP:UD-Q4_K_XL | 14 | +75% |
oMLX | O | Jundot/Qwen3.6-27B-oQ4-fp16-mtp | 24 | +200% |
oMLX | O | Jundot/Qwen3.6-27B-oQ6-fp16-mtp | 17 | +112% |
llama settings
llama-server \
-hf "unsloth/Qwen3.6-27B-GGUF-MTP:UD-Q4_K_XL" \
--host 0.0.0.0 \
--port 8000 \
--temp 0.6 \
--top-p 0.95 \
--top-k 20 \
--min-p 0.05 \
--kv-unified \
--cache-type-k q4_0 \
--cache-type-v q4_0 \
--flash-attn on \
--ctx-size 102400 \
--n-gpu-layers 99 \
--batch-size 2048 \
--ubatch-size 512 \
--parallel 1 \
--spec-type draft-mtp \
--spec-draft-n-max 6
oMLX settings
ctx_window: 102400
max_token:8192
temp: 0.6
top_p: 0.95
top_k: 20
min_p: 0.05
rep_penalty: 1
presence_penalty: 0
