I started collecting Dragon Balls on a whim when I thought I should properly run a local LLM once.
First, it was the RTX 3090.
"With 24GB VRAM, there shouldn't be anything I can't run, right?"
When I actually ran it, it was quite good. Models loaded well and the speed was satisfactory.
But as I used it, heat management was too bothersome.
In summer, the room temperature would rise, the fan noise was distracting, and it was burdensome to keep it on all the time.
Eventually, I cleaned it up.
Then suddenly I had a thought.
"Oh right, I have a new laptop."
Ryzen AI 240 with RTX 5050, 32GB RAM.
On paper, the specs looked pretty decent.
"With this, attaching Continue to VS Code and doing some vibe coding should be enough, right?"
As always, I was the one who lost.
I attached Continue and tried connecting various local models, but the results were pitiful.
The model size was ambiguous, the context was lacking, and the coding accuracy fell short of expectations.
Eventually, I wondered if I'd fallen victim to equipment fever again and let it go.
Then finally, I tried the M1 Max again.
M1 Max... the price was fair even last year.. but now the base model is 160-190.... oh the price...
Anyway, I somehow got one and tested it.
Honestly, I didn't have high expectations.
"No matter what, wouldn't it be worse than a PC with a GPU?"
But...
Oh wow.
This actually works? haha
Installed Ollama on Mac Studio M1 Max 32GB and connected Roo Code.
Loaded Qwen 3.6 27B and set it on a Flutter project.
At first, I struggled a bit with ripgrep issues and tweaked the settings, but at some point it actually started reading the project.
Looking at the logs, it was putting in over 20,000 tokens and analyzing.
"Whoa...?"
Then it modified the test file.
"No way."
Ran the test right away.
flutter test test/planet_model_test.dart
Result:
00:00 +20: All tests passed!
For a moment, I really laughed.
For a few months
3090 → 5050 → M1 Max
After going around and around, the conclusion I reached was,
What's important is not the GPU name more than I thought,
"How well you can run a decent model stably"
that seems to be the case.
Of course, in terms of absolute performance, the 3090 is still stronger.
But by the standards of a local AI coding environment that developers use daily, the M1 Max is actually really decent.
I never imagined a day would come when I could run a 27B model on something like a Mac mini and modify a Flutter project and pass tests. 😅
If I have the time, I'd like to try running an m4 max/ultra with 128GB haha
