This article is the third installment in a series about the feature development history of seCall — a CLI tool for collecting and searching AI agent conversation sessions. Since it was created as part of a Vibe Coding test project, I don't know the detailed code and don't want to know it either. You don't need to know it. If you know it, you'll get bitten. If you want to know, you can ask the person who made it(?). Rather, this is written because I want to explain the technologies used in features applied after trial and error and the reasons for their adoption.

• Part 1: SQLite FTS5 + Korean morphological analysis

• Part 2: Local vector search

The situation so far

I created two types of searches.

• BM25 (text search): Type "model" and it finds conversations containing "model". It's accurate, but can't find "design" through "architecture".

• Vector (semantic search): Search "architecture" and it also finds conversations about "design". It's smart, but sometimes it's worse than BM25 at exact keyword matching.

As you use it, things like this happen. Search "BM25 indexing" — BM25 pinpoints exactly the conversations where that word appears, while the vector search broadly pulls up similar context conversations like "search optimization" and "index structure". Conversely, for vague searches like "why is the search slow", BM25 barely catches anything, but vectors find related conversations well.

Since both have pros and cons, the natural conclusion is one: Why not use both?

What is hybrid search

Literally, it's running text search and vector search simultaneously and combining the results. The concept is simple, but the problem is how to combine them.

BM25 scores come out in ranges like 0-25, while vectors come out as cosine similarity between 0-1. The scales are completely different. If 15 points in BM25 is high, is 0.7 in vectors high? Simple comparison doesn't work.

You could normalize the scores and combine them, but then how you set the normalization baseline becomes another problem. Since the result distribution changes each time.

RRF — combining ranks instead of scores

This is where **RRF (Reciprocal Rank Fusion)** came from. The name is grand, but the idea is surprisingly simple. (And I didn't make it. Believe me) The interesting part is that while the algorithm itself is unexpectedly simple, it became something like an industry standard as ML/AI boomed — a sort of ill-fated algorithm.

Ignore the scores, and only look at what rank each search gets.

If a document ranks 3rd in BM25 and 7th in vectors — we give it a combined score based only on the fact that it's 3rd and the fact that it's 7th. The original scores don't matter.

The formula is also simple:

RRF score = 1/(k + BM25 rank) + 1/(k + vector rank)

k is a constant, and seCall uses 60. What k does is — if the value is large, you become insensitive to rank differences (reflects evenly), if it's small, you heavily favor top ranks. 60 is said to be the default value commonly used in practice, so I just used it (this is Vibe!). Honestly, I don't have enough data to tune this number either. (Please)

For example:

Conversation A ranks high because it's 1st in BM25, and Conversation C ranks high because it's 3rd in both. Even if it does well in only one search, or adequately in both, it can rise to the top. Even results that are completely missing from one search survive if they're at the top in the other.

That's the charm of RRF — it cleanly combines the results using only ranks, without worrying about matching score scales. (Once again, I didn't make this. Please believe me)

Performance optimization: don't search everything

There's another thing I paid attention to in hybrid search. Vector search fundamentally runs across all vectors. If there are thousands of conversations and tens of thousands of chunks, scanning everything each time is wasteful.

seCall runs in this order:

1. Run BM25 first — because it's fast. Extract candidates generously (3x the number requested! Vibe!)

2. Run vector search only within the session ID range found by BM25

3. Combine the two results with RRF

BM25 acts as a kind of filter. Since we only run vectors within a "range that's at least somewhat textually relevant", unnecessary comparisons are drastically reduced. If BM25 finds no results at all, then we fall back to full vector search.

Thanks to this structure, by the standard of 600 conversations, hybrid search actually feels faster than vector-only search. (Really)

Does it actually make a difference?

Honestly, at the level of hundreds of conversations, BM25 alone is often sufficient. Hybrid search shines in cases like this:

• "What was that again" kind of vague search — BM25 struggles without keywords, but vectors catch it through context

• Exact error message search — vectors bring all similar errors, but BM25 pinpoints that exact message

• Mixed Korean-English search — searching "design" also brings up "architecture" conversations

Each approach alone misses results that the other catches, and combining them catches those. It might not be a dramatic difference, but once you taste it, you don't want to go back.

So does combining actually help?

Honestly, I haven't run academic benchmarks. (I don't have the intelligence or ability to do so 😇) To measure metrics like precision and recall, you need a ground truth dataset, but there's no "correct answer" in personal conversations. Still, some validation is necessary, so I roughly ran the same search terms through three modes and compared.

For debugging purposes, I created three search modes: --lex-only (BM25 only), --vec-only (vector only), and default (hybrid). Since the search results show BM25 and vector scores separately, you can visually see which one contributed. (I didn't originally make this for this purpose...)

There are several patterns:

• Exact keyword search like "BM25 indexing" BM25: Pinpoints exactly 5 conversations where that word appears X Vectors: Broadly brings 15 conversations with surrounding contexts like "search optimization" and "index structure". Related but some are not what you want Hybrid: The exact 5 found by BM25 appear at the top, related conversations found by vectors appear below. Exact matches come first while range expands

• Vague search like "why didn't that work" BM25: Tokens like "why" and "work" are too common, so unrelated conversations rank high X Vectors: Catches error/failure context conversations well OK Hybrid: Vector results dominate, but if BM25 caught something exactly, it rises

• When searching "design", does "architecture" appear? BM25: Obviously not X Vectors: Yes OK Hybrid: Yes. Conversations with "design" used exactly appear first, "architecture" conversations appear after

Conclusion — there was no case where hybrid performed worse than either one alone. At worst, the result from whichever side did well appears. RRF's "rank summation" seems to do well in this safety net role. It's a structure where if one misfires, the other compensates. Of course, this is comparison based on intuition with about 600 conversations, so it's far from academically rigorous. However, for personal tools, "does what I want come up well when I use it" is practically the only metric, and by that standard, the hybrid approach is currently the best option I can think of (please).

Summary

Looking back, creating FTS5 + morphological analysis in #1, vector search in #2, and combining them with RRF in #3 — wasn't so much a plan as it naturally became this structure while pondering how to fill each inadequacy. (Talking only to AI without friends, my speaking style naturally became like this too.🫢) That's how Vibe Coding works. Things don't go according to plan, but a plausible structure emerges as you struggle through. However, validation is always necessary. (In this process, the workflow is idea - implementation - validation - enhancement - validation - enhancement - eating - validation - enhancement - argh!... it's an infinite loop of hell)

In the next installment, I'll conclude this series(?) by talking about organizing the conversations collected and searched this way into an Obsidian Vault as markdown wiki. Search is great, but browsing through conversations or organized documents in wiki form during free time and reviewing them has its own charm.

This post is written no-vibe. You can read it with confidence 😁 Floundering on bare ground isn't agentic development. It's a three-legged race between human intelligence accumulated through diverse experiences with solid domain knowledge and artificial intelligence with tireless precise work with a firm bridle — a grand collaboration chaos.