Legacy developers who weren't interested in AI are working like crazy.
I playfully bought a 5080 in 24 installments and
before I knew it, I was able to handle 10 v100s…
Even if the performance is slow, 300GB VRAM is a beast.
I've been diligently studying and tinkering with the open webui source code since this spring.
I'm learning by debugging one by one, teaching Gemini and GPT Linux, and studying servlets.
My ultimate goal is to create a service using llama.cpp.
So far, I've benchmarked models locally, created an MCP server, and been able to generate and edit images with comfyUI.
Looking back, it feels like I've climbed 10 out of 100 steps. If I were younger, I could have jumped two steps at a time, but my stamina is lacking..!
The speed is too slow when running locally. Even with gemma4 32b, it takes 30 seconds for a tool call and the satisfaction level is 50%. The average token speed is around 24-28.
Image generation takes 3 minutes to create a 1024*1024 size image with a prompt within 100 characters. Currently, I'm struggling to fix errors due to the CUDA version.
I want to go all-in on AI development, but work has been piling up, so I'm planning my next steps.
Fellow AI enthusiasts, be careful not to overwork and let's all dive into the world of tinkering together.?!
