This article is the second installment in a series about the feature development of seCall — a CLI tool for collecting and searching AI agent conversation sessions. Since it was made as part of the Vibe Coding test project, I don't know the detailed code and don't want to know either. I don't need to know, I wouldn't understand it anyway I'm writing this to discuss the technologies used in features applied after trial and error and the reasons for their adoption. (Previous installment: Searching previous agent conversations in the terminal with just SQLite - Why do we need to do Korean morphological analysis?)

Problem: You can only find words if they match exactly

In the previous installment, I created BM25 search with FTS5 + Korean morphological analysis. Now I can find "모델" in "모델을". But as I used it, something didn't feel right.

When I search for "아키텍처" (architecture), "설계" (design) doesn't come up. When I search for "에러" (error), "버그" (bug) doesn't appear. Humans know they mean the same thing, but to BM25, if the characters are different, they're strangers. Especially in AI conversations where Korean and English are mixed, a conversation where you asked "embedding" and got answered "임베딩" only half-matches regardless of which way you search.

Ultimately, I needed a search on a different axis than text matching.

What is embedding — converting sentences into numbers

The core of vector search is embedding. It converts sentences into arrays of hundreds to thousands of numbers (vectors), where sentences with similar meanings produce similar number arrays.

"아키텍처를 설계했다" (designed the architecture) and "시스템 구조를 잡았다" (established the system structure) have completely different characters, but when passed through an embedding model, they produce vectors pointing in almost the same direction. If you measure the distance (cosine similarity) between these two vectors, you can numerically tell that "these two sentences are similar."

In simple terms — if BM25 is a dictionary search that looks at "do the same words exist?", vector search is a semantic search that looks at "is the context similar?"

Embedding model: BGE-M3

Which model you use to create vectors determines the search quality. In seCall, I chose BGE-M3.

• Multilingual support: Strong with mixed Korean + English text. AI conversations routinely mix Korean and English, so this was important.

• 1024 dimensions: A size that captures meaning sufficiently while not being excessive in storage and search costs.

• Open source: Anyone can download it from HuggingFace.

Actually, there was trial and error here too. At first, I also tried OpenAI's text-embedding-3 series. The quality was good, but requiring API calls meant money would be spent embedding thousands of conversations. And it didn't match the original intention of "solving everything locally." It also didn't match my philosophy of "I'll pay for Claude MAX200, but I won't charge the API!"

Where to run embeddings — three backends

Once the embedding model is decided, you need to decide where to run it. This turns out to be more of a dilemma than expected.

1. Ollama (default)

The local LLM runtime famous for Ollama also supports embedding models. One line of ollama pull bge-m3 and you're done. If you're already using Ollama, it's the most convenient. This is also seCall's default.

2. ONNX Runtime (local inference)

I wanted it to run without Ollama. With ONNX Runtime, you directly load the BGE-M3 model file (.onnx) and perform inference inside Rust. On first run, it auto-downloads the model from HuggingFace (~600MB), and afterwards runs offline. The advantage is you don't need to run a separate service like Ollama.

3. OpenAI API (cloud)

An option I kept for cases without local GPU or when speed is needed. Just put in the API key. However, it costs money and requires sending conversations to external servers, which I personally disfavor. Eliminated

Not knowing what users would prefer, I just prepared everything. It can be switched with one line in the config file. It's the same pattern as the Lindera/Kiwi choice from the previous installment. In a personal tool, this kind of swappable backend structure is quite useful — when your environment changes, you just change the settings.

sqlite-vec struggles

This is the story I previewed at the end of the previous installment. The original plan was:

FTS5 (text search) + sqlite-vec (vector search) = done with just one SQLite

sqlite-vec is a SQLite extension project that supports vector search. Everyone is using this combination, and conceptually it's clean. But when I actually ran the build, I got C compilation errors on my small and precious MacBook Air. Turns out the sqlite-vec I trusted was still in alpha (0.1), with unstable platform compatibility.

Here I had two choices:

• Parse the C build issue and fix it directly → this is a bit much (if I fix it myself, later when the main project updates, the complexity can explode)

• Move the project to Windows or Linux and work there (?)

• Work around it another way

• Assassinate Trump 🚀

Obviously I chose the latter. 😇 (The Trump assassination also failed)

Workaround: BLOB storage + usearch

In the end, I just stored vectors by serializing number arrays as bytes into a regular BLOB column in SQLite. I gave up on sqlite-vec's virtual table feature, but since the data itself is still in a single SQLite file, I kept the "single DB" principle.

I solved the search speed with a library called usearch. usearch is a Rust native library that implements HNSW (Hierarchical Navigable Small World).

What is HNSW (not the Hacker News software) — when finding "the closest one" among thousands of vectors, comparing everything is slow. HNSW connects vectors as a graph and searches by following close neighbors. It's an approximate nearest neighbor (ANN) algorithm that sacrifices a bit of accuracy for dramatically increased speed. With 1000 vectors, you don't need to compare them all; you find the most similar one with just tens of jumps.

So the final structure is:

• Vector storage: SQLite BLOB column (persistent)

• Vector search index: usearch HNSW file (performance)

• Fallback to full comparison from BLOB if HNSW index breaks or becomes stale

I'm sorry I couldn't use sqlite-vec, but as a result, the BLOB + HNSW combination is more flexible and faster. Trial and error isn't always bad(please). It doesn't feel like I sacrificed anything major(please).

Chunking — cutting conversations into appropriate sizes

Embedding models have input length limits. It's common for AI conversations to exceed thousands of characters per turn, and putting them in whole dilutes the meaning of later parts.

So I cut conversations into appropriately sized chunks. seCall cuts at roughly 3600 character units, with about 15% overlap between chunks. This overlap mitigates context loss at cutting boundaries.

Each chunk also has metadata attached: "which session's which turn is this fragment?" So you can return to the original conversation from search results later.

How long does it actually take?

To be realistic — embedding is slow. Running BGE-M3 on ONNX Runtime on a CPU takes hundreds of milliseconds per chunk. Processing 600 conversations took about 40 minutes. (A user who ran 3GB of conversations in an early version said it took 3 days... I apologize again for that)

However, this only happens on initial indexing, and afterwards you only process newly added conversations. And the search itself finishes in milliseconds thanks to HNSW. It's a typical write-once, read-many pattern: slow indexing, fast search.

Looking at actual results, it's worth spending a few hours on initial indexing. If money were abundant, you could ask Opus or Sonnet to write a wiki too, and you'd get a series of autobiographies of evil deeds.

Summary

The key was running semantic search locally, without external services. I couldn't use sqlite-vec and had to work around it, but the BLOB + HNSW combination ended up being more flexible and faster. Trial and error sometimes gives good results (please).

In the next installment, I'll discuss how to combine these two search results — BM25 (text matching) and vectors (semantic matching) — hybrid search and the RRF algorithm. If there are interested readers, I'll continue the story.

This post was written by An(no)Vibe. You can read it with confidence 😁 Getting bloodied on bare ground isn't agent development. It's a two-person, three-legged race between human intelligence based on abundant experience and domain knowledge, and artificial intelligence with a solid bridle that never tires of precise work — a great carnival collaboration.