https://forums.developer.nvidia.com/t/dflash-for-qwen3-5-122b-a10b-80-tok-s-on-1x-spark/374328
I told the agent to do this.
It said it would take 30~1 hours to download and modify something, so I waited,
and it made a Docker server with vllm.
Now I have to connect this to Hermes,
but no matter what I do, it won't connect. So I tried to connect to ClaudeCode or Codex, but my agent said that if I did that, the token generation speed would be about the same as before the modification. So I asked if there was anything else,
and it said that connecting to OpenCode would do the trick.
So I asked it to connect, and it did. According to the forum post above, optimization will result in about 59 tokens being generated...
First, I measured the token generation speed by connecting to OpenCode.
About 36 tokens came out, which is quite different from the forum post, so it told me to measure several times.
The average was close to 40.
The agent interpreted that the forum post showed a smaller number of tokens for short tasks or texts, and that longer tasks would result in the maximum token generation.
Or maybe my agent just messed up, but for now, 36~40 tokens seems usable enough.
On Reddit, there's a post saying that someone optimized DeepSpeech4 by doubling Spark and succeeded in generating over 70 tokens. It seems that popular models are becoming more and more optimized and developed, so it's likely that the speed will be good enough for local operation.
I was aiming for an m5max Mac Studio with 128GB, but it looks like it's going to cost over 10 million won, so I might have to look for a used Spark... But when I looked for it, all the used Sparks were gone. ㄷㄷㄷㄷ
