In the latest version of oMLX, a function was added to offload n-grams to SSD, but the Q2 model also did not work properly.
After looking for a way, I succeeded in running it with a custom repository of llama.cpp.
The repository itself didn't run, so I asked claude to make some modifications.
https://github.com/mihailescu2m/llama.cpp
model : unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (93.7GB)
With MTP activated, the average decoding speed is about 13t/s (minimum 9, maximum 19).
I have attached the file created by making a standalone HTML for solar system simulation.
There are some bugs, but the quality is outstanding.
In my environment, Qwen3.8-27B MTP is about 18t/s and Dflash2 is about 25t/s.
I'll consider switching if the results are good.
