I was lured by the exaggerated claim that YouTube is twice as fast and tried something called Exllamav3.
https://github.com/turboderp-org/exllamav3
I found that the model I should mainly use is Qwen 3.6-35B-A3B.
https://huggingface.co/isogen/Qwen3.6-35B-A3B-StyleTune-exl3-4bpw/
So I tried giving 3 prompts, and here are the approximate statistics.
Total prompt tokens 74,745
Total generated tokens 20,058
Total processed tokens 94,803
Average prompt processing speed 2,529.9 T/s
Average generation speed 139.8 T/s
Average TTFT (first token latency) 2.29 seconds
Total execution time 185.91 seconds
Previously, it was difficult to even see 90 T/s. It's not twice as fast, but it seems quite a bit faster.
The model size is also smaller, so it's an advantage that you can increase the context size.
(It's a 5060ti dual with national overclock +300, +3000.)
