I have a Steam Deck with one oled, and it turns out to be a 16gb integrated memory system.
After baking and boiling it well, I tried running the Ornith-1.0-9B model, and it worked well.
Using model 4bit quantization, kvcache turboquant 4bit quantization, and applying 64k ctx, I got about 12~14 tps.
However, the car wash benchmark failed!! haha
Currently, I am using the Hermes agent with my family as an everyday secretary, so I am connecting deepseek v4 flash using a subscription API.
Using the Steam Deck as a 24-hour server, it is stable because it doesn't generate heat due to API calls.
