For a Beelink product with AMD AIMAX+ 395 (Strix Halo) 128GB computer,
I tried setting up Antirez's DrawfStar4, which is trending these days.
https://github.com/antirez/ds4
In short, I squeezed the Deepseek 4 Flash open-weight model to the maximum
and quantized it to 2-bit, creating a dedicated GGUF model file of about 80GB in size.
Since this can't just run on llama.cpp, I've created and released even a DF4 server dedicated solely to this one model.
Basically, it targets Apple Silicon, but it also supports NVIDIA and AMD ROCm.
The developer Antirez is a very famous and highly skilled master, so it seems possible. (I heard he's the one who made Redis)
In conclusion, if I successfully get this running....
The speed may be disappointing, but in terms of intelligence itself, it becomes the current world's best among local models that can run on 128GB-class computers. In fact, I was also very satisfied with the results of using Deepseek V4 Flash with an Opencode Go subscription.
So I struggled for the past two days and finally succeeded.
Getting the repository, compiling, and things like that went smoothly just by following Gemini's instructions...
In my case, since I restricted the maximum VRAM to 96GB on the system, the context size ended up being capped at 12kB. It's a bit insufficient. When I try to increase it, I run out of memory...
If I increase the VRAM shared value more than 96GB, I should be able to get more context.
The token output speed is under 20t/s. It's roughly 15~19t/s.
It feels like I'm watching someone typing on a computer across from me.
If you use an Apple computer, it will probably be much faster.
And it takes about 18 seconds to load the initial model into memory.
Interestingly, there's only one model file provided, but actually it provides both Deepseek V4 Flash and Deepseek V4 Pro. The Pro is the same model, but it seems to have been tuned to do more "Thinking". The answers come out richer too.
In terms of intelligence....
It is absolutely superior compared to any model I've used locally so far.
When coding in Python, when the paid model tokens are all consumed and I have to just sit idle, it seems to be at a level where I can use it in urgent situations.
Even with extreme 2-bit quantization, I barely see any Korean text corruption or issues.
I heard that Antirez is currently developing to add the GLM 5.2 model here.
Of course, no matter how much I diet GLM5.2, it won't be possible to run on a 128GB system, and it's probably for those who have systems with 512GB-class shared VRAM.
And I'm currently attaching this model to Open-webui and Hermes-agent running together on the same computer,
I succeeded in uploading it to Open-webui.
I even added simple Python proxy code to the Systemctl service so that after about 1 minute, the model is automatically unloaded when I'm using the DF4 model and want to load ComfyUI or Ollama models on the same computer, to prevent memory shortage.
I still haven't succeeded in attaching it to Hermes. ㅠㅠ
I stayed true to the motto "Forget about speed, just make it work".
Anyway, SOTA-level implementation on a personal computer like this... it makes my heart soar.
No matter how much I downgrade Deepseek, it honestly seems considerably better than the current Gemini 3.5 Flash.
