tunaFlow is a desktop client that orchestrates CLI agents like Claude, Codex, Gemini, and OpenCode in a single app. I haven't released it yet due to various large and small issues. It's part of a fun test project for me. However, I wanted to write this post to share the concerns I had during the process. I hope it helps, even by 0.1%, those working on AI development with similar goals elsewhere. If you have counterarguments about the content, you're all right. 😁 😁 😁

What Should We Tell the Agent

When using just one agent, it's relatively simple. You can get decent results by just passing along the previous conversation appropriately. However, things change when multiple agents start collaborating within a single project. This is because each agent can only see conversations within their own session.

> What, how much, and when should we tell each agent?

In a typical chat, you just pass the previous messages. But in multi-agent orchestration, that's not enough. There's actually quite a lot of information that needs to be passed.

- Project information — name, path, tech stack

- Current work plan — Plan, subtask list, current phase

- Statements from other agents — Roundtable (RT) discussion content, previous review results

- Codebase search results — related code found by rawq (code search engine)

- Long-term memory — compressed conversation summaries from previous sessions

- Skill documents — framework/library usage rules

- Identity — role assignment like "You are an Architect. Just make plans"

You need to assemble this appropriately for each request and put it in the prompt. In tunaFlow, we call this assembly result ContextPack. (This isn't a name I made up! Really!)

The Structure of ContextPack

Every agent request in tunaFlow passes through a single function.

build_normalized_prompt_with_budget(
    conn,              // DB connection
    conversation_id,   // current conversation
    prompt,            // user input
    project_path,      // project path
    active_skills,     // list of activated skills
    cross_session_ids, // related session IDs
    persona_fragment,  // role assignment ("Architect", "Developer", etc.)
    context_mode,      // Lite / Standard / Full
    budget_cap,        // total context budget (character count)
)

Whether it's Claude, Codex, Gemini, or anyone else, everyone gets their prompt assembled through the same function. It's common across all 4 engines.

Sections Being Assembled

Roughly, the layers are divided like this.

┌─────────────────────────────────────────────────┐
│ Project path                                    │
│ Platform rules (PLATFORM_TIER0)                 │ ← always included
│ Agent role document (Architect/Developer/...)   │
│ Identity + Persona                              │
│ Conversation participants meta                  │
├─────────────────────────────────────────────────┤
│ Recent conversation (budget-based window)       │ ← maximum within budget
│   + per-agent last-message guarantee            │
│   + tool-result pruning (older messages)        │
├─────────────────────────────────────────────────┤
│ Plan / Findings / Artifacts                     │ ← Standard+ mode
│ Retrieval (past conversation chunks, FTS5+vector search) │
│ Compressed memory (topic-wise summaries)        │
├─────────────────────────────────────────────────┤
│ Skills / rawq / code-review-graph               │ ← Full mode or conditional
│ Cross-session context                           │
│ Thread inheritance (Branch inheritance)         │
└─────────────────────────────────────────────────┘

The key point is not "there's a lot we can put in", but rather "we should put in only the information needed for this request, at the appropriate density".

Core Design: 3-Mode Variable Assembly

If we always put in all sections, tokens get exhausted quickly. So I adjusted it to 3 levels depending on the situation.

// Mode is determined automatically.
fn determine_context_mode(data: &ContextData) -> (ContextMode, &str) {
    // If user explicitly specifies
    if let Some(override_mode) = &data.context_mode_override {
        return (parse_mode(override_mode), "user-override");
    }
    // Standard if conversation is 12+ turns
    if data.current_messages.len() >= 12 {
        return (ContextMode::Standard, "long-conversation");
    }
    // Standard if there's a Plan
    if data.plan_section.is_some() {
        return (ContextMode::Standard, "has-plan");
    }
    // Default: Lite
    (ContextMode::Lite, "default")
}

Putting Full into a short question is wasteful, and conversely, putting only Lite into a long task leaves context lacking. Ultimately, what matters is not giving as much information as possible, but rather matching the information density that's needed right now.

Dynamic Budget Allocation

Within the total budget (default 60,000 characters), I didn't fix the proportion each section occupies. I calculate it dynamically by looking at both content size and weight.

let budget_alloc = allocate_budgets(total_budget, &[
    SectionBudget { name: "plan",       weight: 1.0, min: 500,  max: 4000 },
    SectionBudget { name: "plan-doc",   weight: 2.0, min: 1000, max: 6000 },
    SectionBudget { name: "findings",   weight: 1.0, min: 500,  max: 3000 },
    SectionBudget { name: "skills",     weight: 1.0, min: 500,  max: 3000 },
    SectionBudget { name: "rawq",       weight: 0.8, min: 500,  max: 3000 },
    SectionBudget { name: "retrieval",  weight: 1.2, min: 500,  max: 5000 },
    SectionBudget { name: "compressed", weight: 1.0, min: 500,  max: 4000 },
    SectionBudget { name: "cross",      weight: 0.6, min: 300,  max: 3000 },
]);

For example, if the Plan document is long, it takes more budget from the Plan side, and in turn, the budget for skills or rawq decreases. Rather than dividing in a fixed ratio, the budget moves according to the volume and importance of the content that will actually go in.

This is more important than it seems. Because in actual work, the same information isn't always what matters. Some requests are all about the plan, some are about code search results, and some depend more on past conversation searches.

Conversation History: Can't Put It All In, But Can't Cut It Either

When conversations get long, it's impossible to put in all previous messages. But if you just cut them out, the flow breaks. So tunaFlow uses three strategies.

1. Budget-based dynamic window

Fill from the latest message in reverse order, and stop when the budget runs out.

// Fill budget in reverse order from latest message

for (i, msg) in messages.iter().enumerate().rev() {
    let msg_cost = role.len() + content.len().min(max_per_msg) + 40;
    if msg_cost <= char_budget {
        trimmed.push(msg);
        char_budget -= msg_cost;
    } else if must_include.contains(&i) {
        // This message must be included even if budget is exceeded
        trimmed.push(msg);
    }
}

It looks simple, but this is the right basic operation. Because the most recent context matters most.

2. Per-agent last-message guarantee

In multi-agent systems, the more critical issue is that the last statement of each agent must not be cut off.

For example, if Alice made a crucial counter-argument 3 turns ago but that message gets cut due to budget, the next agent will discuss without knowing Alice's stance. Then it's multi-agent in form, but actually just parallel monologues.

So we guarantee that each agent's last message is always included.

// Collect last message index of each agent
let mut agent_last_idx: HashMap = HashMap::new();
for (i, msg) in messages.iter().enumerate() {
    if msg.role == "assistant" {
        agent_last_idx.insert(msg.persona.clone(), i);
    }
}
// These indices must be included even if budget is exceeded
let must_include: HashSet = agent_last_idx.values().collect();

Without this guarantee, multi-agent contexts break down more easily than you'd think.

3. Compressed memory (topic-wise compression)

When 12+ turns have passed, older messages don't go with us as-is. Instead, we create topic-wise summaries using an LLM.

## Compressed conversation memory

### Topic: API design decisions
Alice(claude): REST vs GraphQL comparison, REST choice rationale
Bob(codex): Agreement, but WebSocket recommended for subscriptions

### Topic: Authentication method
Alice(claude): JWT + refresh token proposal
Charlie(gemini): OAuth2 PKCE additionally recommended

The original is preserved as-is in the DB. You can pull it back anytime via search if needed. The prompt only contains the summary.

In other words, we don't completely discard old conversations; instead, we save the original and transmit a compressed version.

4. Engine Parity: All Engines Should Receive the Same Context

One of the important principles in tunaFlow is 4-engine parity. Whether it's Claude, Codex, Gemini, or OpenCode, for the same question they should receive the same context quality as much as possible.

If differences arise by engine, it becomes not "this model is better" but actually "this model received more information". That's not a comparison.

There's only one difference.

- Claude: separate system prompt (--append-system-prompt-file)

- Non-Claude: all context inlined into a single prompt

// Claude: system prompt separate
let system_prompt = format!("{}\n\n{}", context_sections, platform_rules);
let user_prompt = user_input;

// Non-Claude: all combined
let prompt = format!("{}\n\n---\n\n{}", context_sections, user_input);

So while the delivery format differs, the information itself stays as identical as possible. That way, when you switch agents, you can interpret result differences.

Problems from RT (Roundtable): Tokens Multiply by N

Roundtable inherently consumes a lot of tokens.

When 3 people discuss for 2 rounds, a total of 6 agent calls occur. If ContextPack is attached to each call, the calculation roughly looks like this.

6 requests × ~15k(Standard mode) = ~90k characters ≈ ~30k tokens

Even in paid plans like Claude Pro, a few runs like this quickly reduce the daily limit. It's an irony: as a multi-agent orchestrator, the more actively you use multiple agents, the less room you have for a single conversation.

Current Response: RtContextCache

So in RT, instead of rebuilding the ContextPack from scratch for each call, we build it once at the start of a round and cache it.

struct RtContextCache {
    auto_context: Option,  // for commercial engines (Claude, Codex, Gemini)
    lite_context: Option,  // for local engines (Ollama, OpenCode)
}

Even if 3 people run sequentially within the same round, ContextPack-related DB queries happen only 1-2 times. Not N times, but 1 time.

However, this is only about reducing query costs. The actual amount of text entering the prompt is still N times. In other words, caching saves the DB but doesn't eliminate token costs itself.

Future Direction: From Push to Pull

Currently, ContextPack is fundamentally a Push model.

"The agent might need this, so let's include it just in case."

Current:
[identity + project + plan + skills + rawq + memory + cross-session]
→ send everything every request
→ 7 out of 10 times don't use skills/rawq/memory but they're included anyway

The problem isn't just token cost. The bigger problem is noise.

When 10, 15 sections come in at once, from the agent's perspective it's unclear: "So what should I prioritize looking at right now?" As a result, signal-to-noise ratio drops. Giving more isn't always better.

tunaFlow already has a tool-request marker system. When an agent puts this kind of marker in its response,

tunaFlow detects it and automatically sends search results as a follow-up.

In particular, context-hub (library documentation search) has already switched to this Pull approach. The remaining sections (skills, memory, cross-session) can move to the same pattern.

The goal is ultimately small Push + selective Pull.

Tier 0 (always):  identity + basic project                     ~1.5k
Tier 1 (conditional):  plan + findings + rawq(for code questions)  ~2~4k
Tier 2 (Pull):  skills, memory, cross-session              → agent requests when needed

However, tunaFlow is not SDK-based but CLI subprocess-based. So one Pull means process restart + one new input. It's not a structure that naturally resolves tool calls within the same run.

In other words, if Pull happens once or twice, it might be beneficial, but if repeated 2+ times, it could actually be more expensive than Push. So the conclusion is neither pure Push nor pure Pull. Once again, hybrid is the answer. (Beginning-middle-end-hybrid... 😩)

Summary

ContextPack is ultimately tunaFlow's current status on "what to tell agents".

Within the constraint of CLI subprocess basis, there's ultimately one principle I want to keep.

"When agents are comfortable, result quality improves."

The intermediate conclusion is that it's not about giving lots of context, but about giving what's needed at the right density. ContextPack is currently finding that balance point!

References

Related Technical Documents

- Anthropic — Contextual Retrieval (2024.09): Adding document context prefix to chunks reduces search failure rate by 49%. Referenced in ContextPack's Tier 2 Pull design.

https://www.anthropic.com/news/contextual-retrieval

- Jina AI — Late Chunking (2024): Processing entire documents with long-context embedding models then pooling per chunk. NDCG@10 on BeIR benchmark +5~15%.

https://jina.ai/news/late-chunking-in-long-context-embedding-models

- Dense X Retrieval (Chen et al., 2024): Proposition-based chunking improves recall@5 by +12~17%. Each retrieval unit is self-contained + atomic.

https://arxiv.org/abs/2312.06648

- RAPTOR (Sarthi et al., 2024, Stanford): Recursive clustering + summarization for multi-layer indexing. NarrativeQA accuracy +20%.

https://arxiv.org/abs/2401.18059

- ColBERT v2 (Santhanam et al., 2022): Token-wise multi-vector representation achieves MRR@10 0.397 (BM25: 0.187).

https://arxiv.org/abs/2112.01488

tunaFlow Internal Documentation

- docs/ideas/contextPackTieringIdea.md — ContextPack 3-Tier hybrid design + vector context sharing + sqlite-vec + chunk quality

- docs/reference/multiAgentContextStrategy.md — Multi-agent context 3-layer strategy (participants meta + dynamic window + per-agent guarantee)

- docs/ideas/insightWorkflowIdea.md — Insight report file saving + Plan promotion UX (ContextPack token 0 agent access)

Related Tools

- rawq — Rust-based code search engine. snowflake-arctic-embed-s (384 dimensions) embedding, daemon mode resident. Handles code search + vector embedding generation for tunaFlow.

- context-hub (chub) — Library/framework documentation search CLI. Already switched to tool-request Pull approach in tunaFlow.