Environment: MacBook Pro M5 Pro RAM 64GB
LLM Interface | Model | tok/s |
|---|
omlx | unsloth/Qwen3.6-27B-UD-MLX-6bit | 8.3 |
MTPLX | Youssofal/Qwen3.6-27B-MTPLX-Optimized-speed | 28.7 |
MTPLX | Youssofal/Qwen3.6-27B-MTPLX-Optimized | 26.4 |
I also considered checking out llama.cpp, but I found MTPLX because it seemed less troublesome.
MTPLX was created by Youssofal, and I tried using the models they provide.
It's a 4-bit model, but the speed is satisfactory.
I need to use it more to determine the output quality.
When running on omlx, even 4 bits struggled to exceed 10, but now it's much more comfortable.
Usage
brew install youssofal/mtplx/mtplx
mtplx start
