I wanted to try using an LLM on my phone with an on-device app, so I used Edge Gallery. It only allowed for chatting and limited models.
I looked for other apps and found one called Pockecpal. You can even download models from Hugging Face.
My phone is an S26U with 12GB of RAM, so the 4B model was too small. I got the E2B model instead.
The app has a built-in benchmark, so I tried it out and got about 20 tokens.
I set it up to prioritize the NPU.
Chatting is fast if you turn off inference.
Now that I've benchmarked it, I should delete it.
